Microsoft is testing a new update to Copilot for Windows that will allow users to show the AI assistant what’s happening on their screen or in an app. Previously, Copilot Vision was only available in the Microsoft Edge browser, but with the update, it’s expanding to any app.
Microsoft is testing a new update to Copilot for Windows that will allow users to show the AI assistant what’s happening on their screen or in an app. Previously, Copilot Vision was only available in the Microsoft Edge browser, but with the update, it’s expanding to any app.
With Copilot Vision, the user can ask the AI assistant for help with Adobe Photoshop functions or analyzing a photo or web page.
«I got to see an early version of Copilot Vision for Windows at Microsoft’s 50th anniversary party last week, where the AI assistant guided me through a game of Minecraft and helped me optimize settings in Microsoft’s Clipchamp video editor», — notes The Verge’s reviewer.
He added that Copilot Vision also works in the Copilot app on iOS and Android.
To give Copilot access to your screen, tap the glasses icon in the AI assistant menu and select the browser window or app you want to analyze by asking a question. To stop sharing, tap Stop or the X in this menu.
Microsoft has also taught Copilot to find, open, and answer questions about the contents of files on a user’s computer. The feature allows you to set permissions for accessing, receiving, or reading files. File search supports most file types and can read .docx, .xlsx, .pptx, .txt, .pdf, and .json files.
«Ask Copilot questions like, ‘Can you find my resume?’, ‘Open the travel planning document I worked on last week?’, or ‘Look at my budget and tell me how much I spent on lunch last month’», — describe the use cases for the feature at Microsoft.
An update to the Copilot app (version 1.25034.133.0 and above) with these features is starting to roll out to Insiders via the Microsoft Store. The company says that Copilot Vision for Windows will initially be available to Insiders in the US, while File Search will be available to Insiders worldwide. The update is rolling out gradually, so not all Insiders will receive the update at once.
