Microsoft unveiled a swath of new AI features during its 50th birthday bash in Redmond last week, and I got to try many of them. Copilot Vision holds the most potential for consumers, but Copilot’s shopping features aren’t far behind.
If you’re concerned about Copilot’s new Memory and Personality traits, I wouldn’t be — it looks way too early to get a sense of what it will eventually be. Those were my impressions after sampling most of the new Copilot demonstrations on display at Microsoft’s 50th anniversary celebration.
Seeing these features in action left me with some different impressions. My top three? Copilot Vision (for Windows), Copilot Shopping, and Copilot Search. As for the other features, they didn’t quite land for me.
Copilot Vision: This has big potential
Microsoft offered me the choice of several different scenarios to see a demonstration of Copilot Vision, which had impressed me with the way that it navigated through Photoshop. Seeing a gaming demo on the menu, I asked for it.
Unfortunately, I ended up seeing a version of Minecraft being played with Copilot Vision that Microsoft had referenced before rather than something brand new like navigating through the Blender CAD app.
Copilot correctly identified the emeralds and the leather armor (and what they were good for)as the player played Minecraft (Some weird lighting element bisected the photo.).
Mark Hachman / Foundry
As of now, Copilot Vision is launched via the Copilot application. Copilot “knows” what applications you’re looking at but won’t actively interpret the results until you toggle on a switch allowing it access. In this demo, Copilot Vision has the option of looking at either Minecraft or Clipchamp. We chose Minecraft.
The demo was pretty short. Copilot Vision looked at the contents of a chest, determined that there was some armor and emeralds inside it, and explained what they were for. Vision then looked at some vegetables and determined what they were and that they could be harvested by clicking on them.
The Microsoft employee opened Clipchamp and asked a question about how to facilitate transitions in a video, although she used the phrase “blend more seamlessly between all these clips.” Copilot Vision interpreted this as “seeking smoother transitions.” The employee then asked Copilot to “highlight” what to do. After a substantial pause, Copilot Vision created a halo or highlight around the proper control.
Here, Copilot identified the “vegetables’ it saw on the screen.
Mark Hachman / Foundry
Microsoft has said that Copilot Vision will be rolling out as part of the Windows Insider preview program soon, in about a week or so.
Copilot Shopping: Excellent potential
I was more impressed than I thought I would be with Copilot’s ability to go shopping, only because Microsoft’s original documentation indicated that it would be limited to a set of specific partners. ChatGPT had announced this capability two years ago with vendors like OpenTable, but it seems like it was held up. OpenAI sort of re-announced this agentic capability in January as Operator, currently a research preview.
Copilot’s implementation is straightforward. You just type in what you’d like and Copilot goes hunting. For now, the main screen of Copilot shows you how it’s working, with snapshots of various home pages and product pages, and a sidebar summarizing where it’s going and what it’s doing.
Mark Hachman / Foundry
The Shopping feature was pretty interesting to watch. The demonstrator asked for two dozen red roses from a specific site and Copilot went hunting. She then asked for a simpler, more generic request and Copilot searched multiple sites. (My question, of course, is whether Copilot will favor Microsoft’s partners.)
Shopping is also designed to perform more than one task in conjunction with one another. Want to book a trip to New York on a given date? Copilot Shopping should — if it operates as expected — be able to book a flight, knowing who’s going and what their preferences are, book a hotel for those dates, and then check for tables at restaurants, knowing how many will attend and what their preferences might be.
Copilot surfed through the Web, checking for offers.
Mark Hachman / Foundry
Shopping is very much part of this concept of “memory” and personalization that Microsoft is trying to imbue into Copilot. It will be rolling out soon.
Copilot Search: A unique twist on traditional search
Copilot Search isn’t a preview, it’s live. It’s worth trying out, too. Anthropic uses AI search; Google did, too, before it backed off.
While I still think it’s worth your time to visit web pages to gather the context you’ll need, Copilot Search delivers. In part, it actually “rewrites” your search to widen the focus. For example, “What’s the best version of Windows?” could be interpreted as the “most reliable version of Windows” or the “version of Windows with the most features.” (Its answer? Windows 7, which was a bit different than my own ranking.)
It’s scary to see AI try to replace our expertise in real time. It’s also interesting that Microsoft doesn’t think Copilot Search is actually a Copilot feature, it’s part of Bing. I don’t know what kind of future AI search offers, but it’s an interesting way forward.
Some Copilot features aren’t as impressive
Mark Hachman / Foundry
I’ve been deeply impressed with Google Gemini’s Deep Research, which samples from dozens of web pages and assimilates its findings. A demo of Copilot’s Deep Research doesn’t seem as in-depth?
I didn’t get a clear sense of which sources Copilot would pull from and how many tokens and/or words would be in the final output. It’s easy for an AI to summarize its findings. It’s harder for it to present a long, well-sourced, and coherent piece of writing–that’s a sign of a superior AI output.
The whole concept of a “personalized AI” also took a weak swing and foul-tipped one into the mitt. I was asked to talk to Copilot about my interests (baseball pitching, in my case) and some other corollary pieces of information. I was told that Microsoft, ideally, would use that to build a profile of my likes and dislikes, also knowing that (for example) I have kids or that I’ve enjoyed my visits to the Pacific Northwest because I like cooler weather.
But there was no evidence of that at all, selling the whole thing on my suspension of disbelief.
Mark Hachman / Foundry
Microsoft did show off an AI-generated “video game,” spitting out a few frames per second, intermittently, as it generated a first-person shooter level via AI based on the PC classic Quake. Why not consider this to be one of the success stories that Microsoft showed off? Because it’s really awful save for the concept. You can try it yourself if you want. It’s just a mishmash of Quake scenery without consistency or cohesion. You’ll probably be bored within seconds.
AI may be headed to your PC, but not everything is going to be a winner. Developers, you’ve still got jobs.
by #AI [2.0] April 7, 2025 · Automatic / Editor's Picks [News]
