Multimodal search is a shift in changing how we find information

Search is evolving beyond text at an alarming rate, and businesses that don't adapt will struggle with digital invisibility in years to come. Multimodal searches give us the ability to query using images, voice, video, and text simultaneously. There is a long way to go, but this represents the most significant transformation in search discovery since Google's started. I don’t think this is just another tech trend; it feels like it's reshaping how people interact with search engines and how brands must position themselves for the foreseeable future.

The old model of typing keywords is rapidly becoming antiquated. So many people now point their cameras at products, ask questions out loud and upload images for reverse searches. People have learned. They now expect intelligent responses that understand the context of what they asked for regardless of the input type they used. For many businesses, this shift demands a fundamental change in their content strategy and technical optimisation.

What multimodal search actually means

Traditional search relied on text queries matched against text-based content. Multimodal search processes multiple data types like visual, audio, and textual to understand user intent more comprehensively. Google Lens is a great example of this evolution, allowing users to photograph objects, additional queries and receive relevant information instantly.

This technological leap mirrors how us mortal human beings communicate with each other. This is where we naturally combine gestures, speech, and visual cues when conveying information. It’s taken more than 3 decades, but search engines are finally catching up to this approach, creating more natural and efficient user experiences.

The tech driving the change

Artificial intelligence, computer vision (the ability to see and interpret and understand images and video) and natural language processing, power this multimodal capability. Machine learning models now analyse images, recognise objects, try to interpret spoken language, and understand contextual relationships between the different types of media. It’s not perfect; the spoken language part is still embarrassedly bad at interpreting regional and local accents.

Companies like Google, Microsoft, and OpenAI are investing billions in these technologies because they recognise the competitive advantage of providing a superior search experience. Just like the early days of search, early adopters in optimisation will capture more market share than the competition as user behaviour shifts.

How users actually search today

Voice search dominance

Voice search isn't a revolution; it’s more of an evolution of how we use technology and what we expect from it.

Research shows us that voice queries now account for over 50% of adult searches, with users speaking longer, more conversational phrases than they type. An example being “Best Italian restaurant near me with outdoor seating" replaces typed searches like "Italian restaurant."

This shift suggests you need content optimised for natural language patterns (NLP) and question-based queries. Businesses must anticipate how customers speak about their products, not just how they type about them.

The demographic shift suggests that a larger percentage of mobile and voice searches are made by younger “adult” searches and oldies are still catching up. There is nothing new about this, I’m a firm believer that young people get older and old people eventually learn. This is the future, and I am an “oldie”.

Visual search grows

Pinterest reports over 600 million visual searches monthly, while Google Lens processes billions of queries. We all love taking photos of clothing, furniture, plants and food to find similar items or gather information. It's so simple to use, even my mum uses it and she's in her eighties.

E-commerce businesses benefit from visual search optimisation. But there is nothing new about this. SEO’s have been banging on about for years, so here’s a reminder. Product images must include detailed alt text, structured data markup, and high-quality visuals that algorithms can accurately interpret. The one thing that has changed is the ability to interpret the image itself. If you put these images within the context of the content around it and optimise them properly, you will have more success than many of the competition.

Combined input methods for searching

The most sophisticated searches combine multiple types of input. Users might take a photo and ask "How old is this building?" or upload an image while typing additional context like “price” or “owner”. This behaviour creates complex optimisation challenges but enormous opportunities for building out comprehensive content strategies. But the best part is, we are seeing more relevant results being presented to us and It’s only going to improve in time.

Optimising for multimodal visibility

Image optimisation beyond alt text

Traditional image SEO focused on file names and alt attributes. Multimodal optimisation requires understanding how AI interprets visual content. I suggest you use descriptive file names, descriptive alt text, and structured data markup that provides context search engines can process. The more precise you are with these elements the better the results. Don't be frightened to try tweaking the results if you don't see an improvement.

You may find that high resolution images perform better in visual searches, but file size optimisation remains crucial for page speed. You may have to strike a balance between quality and performance through latest image formats like Googles WebP format and proper, traditional image compression techniques.

Contextual content strategies

Deliver answers across every format your audience might use. If someone searches visually for a product, ensure the page includes rich body text, supporting video, and authentic customer reviews. The more comprehensive and consistent your content, the stronger your relevance signals across search types… so give more, to get more.

FAQ sections also become particularly valuable, not only for SEO purposes, but for addressing the growing number of conversational queries used in voice search. This can provide structured content that supports both visual and text-based search visibility.

Basic SEO technical requirements for multimodal search

Most CMS platforms provide some kind of very basic schema mark up. The more you can add the better. If the CMS isn’t up for the challenge, write your own and fire the tags using Google tag manager.

It works particularly well for products, events, content, and local businesses. This structured data helps search engines understand your content's context and purpose across different search types.

Ensure your site performs well on mobile devices, where most multimodal searches occur. Page speed, responsive design, and intuitive navigation directly impact visibility and user experience. But please remember there are platforms that can provide a great page speed that struggle to pages indexed and crawled because they use too much Javascript or they cannot provide server-side rendering (SSR) by default. These platforms are great for making page layout and landing pages for paid search, but awful for providing organic visibility or many parts of the “multimodal” experience.

Voice search giving the man a suitable local result, He has found his Nun outfit .. in his size!

The local business advantage

In some ways this is just local search, not a lot has changed here except we get to use buzzwords like “Multimodal”. But there is AI element to all of this.

Local businesses gain significant advantages from search (multimodal or otherwise). They always have. Users now frequently combine location data with visual or voice queries: "What's this restaurant?" while taking a photo of the front of it, or "give me a fancy dress shop with a nun outfit for a man" via voice command.

Remember that Google My Business listings prefer high-quality images, detailed descriptions, and consistent information across all the platforms they found you on. But without the basics of getting good reviews, and a decent GMB listing etc, your multimodal experience will be nearly pointless. Get the basics of local search in place first.

On the point of "consistent information across all the platforms". LLMs are learning all the time and there are fact checking tools that identify the citations and links that they know about that are relevant to your brand. It's at this point that you; the brand owner, can use these tools to control the narrative and validate these facts giving you the advantage against local competitors. You will know which websites you need to contact and what sort of content you need to create and where it should be placed or distributed.

The future

In ten years time we will all have our own personalised multimodal experience based on what we do and how we are tracked ! The future has already started, so I suggest businesses should start addressing the opportunities, what do you think?