Google DeepMind has introduced a new feature that turns a standard mouse cursor into an AI assistant powered by Gemini models. The technology allows the system to recognize the semantic context of objects on the screen and perform operations without typing text queries.
Google DeepMind has introduced a new feature that turns a standard mouse cursor into an AI assistant powered by Gemini models. The technology allows the system to recognize the semantic context of objects on the screen and perform operations without typing text queries.
The development was announced on the official Google DeepMind blog. The main goal of the tool is to provide interaction with the neural network directly in the user’s working environment, avoiding switching between application windows.
«We aim to create intuitive AI that interacts with users across all the tools they use, without interrupting their workflow,» the developers explain.
The system is based on Gemini’s ability to capture the visual and semantic context around the pointer. This allows you to perform actions using short commands or gestures:
Create PDF resumes directly in your email client.
Convert numeric tables into charts with a mouse hover.
Editing code blocks and images through voice commands like «fix this.»
Automatic recognition of objects (locations on maps, products in stores) and their transformation into interactive elements.
The functionality is already starting to be implemented in the Chrome browser. The developers also announced Magic Pointer — a hardware or software feature for the new line of Googlebook laptops that activates Gemini with a single touch. Experimental cursor capabilities are available in Google AI Studio and on the Disco research platform from Google Labs.
Recall that on May 12 at The Android Show , Google announced the Googlebook — a new category of laptops built from the ground up around Gemini Intelligence. This isn’t just another Chromebook with AI features on top — Google says it’s the first laptop designed from the ground up specifically for Gemini to provide personalized and proactive assistance.
