Google Lens will search every image and video from your smartphone screen

Google Lens (Android, iOS), used more than 10 billion times a month, will soon be able to scan any image or video displayed on the screen of Android smartphones, on websites or applications. The feature will arrive in the coming months, according to an announcement made this Wednesday, 8, which also includes other features.

“For example, let’s say you get a message from a friend who sent you a video of him exploring Paris. You would want to know which landmark you see in the video,” explained Elizabeth Reid, vice president of search at Google. In this case, simply activate Google Assistant (Android, iOS) and click on “Search screen”. Assistant will connect to Lens, which can identify the location and give you more information about it.

The Multisearch feature arrives this Wednesday in all languages ​​in which Lens is available. With it, it is possible to search for image and text at the same time, finding variations of the original visual search. For example, when searching for a sofa pattern, the user can type in a color so that the app presents the same pattern in other color variations.

Another feature, Multisearch near me, which is available in the US, will roll out globally in the coming months. By doing a search on Lens, the user will be able to search for local businesses where they can find it, like a plate of food.

Google also announced that Multisearch will be integrated into the Google search engine on smartphones and can be used on any image that appears on the search results page on mobile phones. For example, a user searches for “modern living room ideas” and finds a table they like, but prefer it in a different format. Thus, you can use Multisearch to add the word “rectangle” to the search engine, so that you can find it in this format.

Artificial intelligence

Google also highlighted at the event the role of artificial intelligence (AI) behind the new features. The company has brought AI technology to its products, such as Google Maps, in addition to investing in new services, such as Bard, a chatbot that will be launched to take on ChatGPT.

“Through our continued investments in AI, we are now able to understand information in its many forms, from language to images and videos – and even the real world. With this deep understanding, we’re moving beyond the traditional notion of search, helping people understand information in new ways,” said Prabhakar Raghavan, senior vice president at Google. “We’re creating new search experiences that work better like our minds, which reflect how we as humans naturally understand the world.”