Microsoft’s Copilot Vision preview watches your browser to offer AI help

Microsoft is slowly launching Copilot Vision in preview, a version of Copilot that “reads over your shoulder” as you browse the web, to Copilot Pro subscribers today.

Microsoft is pitching Copilot Vision as a collaborative assistant that works along with you as you browse the Internet. Vision was part of a suite of new Copilot demonstrations that Microsoft showed off in October, and was easily the most underwhelming example of new Copilot technologies that I’d seen.

Copilot Vision isn’t a multimodal AI capability that can interpret the world around you. Instead, it’s a cross between Windows Recall and the current version of Copilot. Vision is basically a real-time version of Copilot that’s only available in Microsoft’s Edge browser. Instead of asking Copilot for insights, Vision apparently can “scan, analyze and offer insights” based on what your eyes, and its AI sensors, see on the page.

Where might Copilot Vision be useful? Dealing with an overwhelming array of information, possibly. Think of a densely packed shopping page, all with similar products; making sense of the mess is what Copilot Vision promises. In this scenario, asking Copilot Vision what items meet your preferences, or offer the most value based on the available features, might be of interest.

In the October demonstration Copilot didn’t offer anything insightful, often basically reading aloud what it — and presumably you — saw on the page. Furthermore, Microsoft is taking a very hesitant approach to Vision, possibly as a reaction to the negative response to its Recall tool. Copilot Vision is only available via Edge, and it’s only available on a limited group of websites, too. Microsoft isn’t saying how many sites Vision will work on, or what those sites they will be.

“The list of allowed websites is determined by category and on a case-by-case basis,” a Microsoft representative wrote in response to my question, asking for more clarification on what sites would be included.

Copilot Vision is opt-in, however. For one thing, you’ll need a $20 monthly Copilot Pro subscription to activate it, and participate on Copilot Labs. You’ll need to manually turn it on, too. When you’re done using Copilot Vision, all of the data that you’ve shared with Copilot Vision and how you’ve interacted with it will be deleted, Microsoft says.

After Microsoft’s Recall fiasco, a substantial number of Windows users probably want nothing to do with yet another Microsoft tool that records what you see, even temporarily. Others won’t care quite as much. It’s those adventurers that Microsoft is hoping to entice with the Copilot Vision preview.