Google AI Mode Launches New Features: Planning Canvas and Search Live Visual Search

Google just announced new capabilities for AI Mode, an experimental feature that lets users ask complex questions and sub-questions to dig deeper into a topic directly in Search.

Canvas helps create learning plans and organize information

Google just announced an upgrade to AI Mode, an experimental feature that allows users to ask complex questions and track details right in Search.

The standout feature is Canvas, which helps create a study plan and organize information across multiple sessions in a side panel. Users can press the “Create Canvas” button to get started, and AI Mode will then build the plan in the Canvas panel, allowing for refinement through subsequent questions.

In the near future, users will be able to upload documents like class notes or syllabi to customize their study plans. Canvas will be available in the US for users participating in the AI ​​Mode Labs test in the coming weeks.

Search Live: Integrating Project Astra capabilities

Google also brings Project Astra's capabilities directly into AI Mode through Search Live, which is integrated with Google Lens, the tech giant's visual search tool.

“When you use Search Live, it’s like having an expert on hand to help, seeing what you see and working through difficult concepts with you in real time, all with easy access to helpful links across the web,” Robby Stein, VP of Product, Google Search, shared in a press release.

To use the feature, users open Lens in the Google app, tap the Live icon, and ask a question while pointing the camera at an object. With the feature, users can have a back-and-forth conversation with Search in AI Mode using visual context from their camera feed.

Search Live with video input is rolling out this week on mobile in the US for users enrolled in the AI ​​Mode Labs trial.

Additionally, Google announced that users will soon be able to use Lens in AI Mode to ask questions about what's on their desktop screen.

“Maybe you’re looking at a geometry problem and want to understand one of the graphs better,” Stein says. “Click ‘Ask Google about this page’ from the address bar and select the graph. You’ll get an AI Overview with a summary of key information directly in a sidebar. And this week, you’ll be able to ask additional questions via AI Mode, either by selecting AI Mode at the top of the Lens search results or by clicking the ‘Dive Deeper’ button at the bottom of the AI ​​Overview.”

Additionally, while users have been able to use AI Mode in the Google app to ask questions about images, they can now do so on desktop. Google is also adding support for PDF uploads on desktop, allowing users to ask detailed questions about documents.

Google AI Mode will support additional file types beyond PDFs and images later this year, including Google Drive files.