This week, Microsoft offered its first look at Windows Copilot, an AI-powered assistant that appears to be a one-stop shop for managing all sorts of tasks within Windows 11.
Microsoft showed Windows Copilot standing in for Bing Chat, the AI chatbot that’s rivaled OpenAI’s ChatGPT as the main AI assistants of the Web. But Windows Copilot appears to be much more, as a concierge of sorts for everything that your PC and its apps can do. There’s a real question, however, of what class of hardware Windows Copilot will demand — and Microsoft offered the first hints of how it may manage the diversity of PCs that wish to run it.
Windows Copilot, at least for now, appears to be a sidebar that for right now will open up as a column on the right side of your screen, where your Windows notifications typically reside. While Copilot leans heavily on text, Microsoft showed it interacting with photos and even files.
Microsoft chief executive Satya Nadella introduced the following video at his opening keynote on Tuesday, and chief product officer Panos Panay played it again during his keynote address as well. It’s a short overview of Windows Copilot’s capabilities, before Panay offered more of a hands-on demonstration later on in his speech.
Announcing Windows Copilot
Replace Bing Chat
We’ve already seen Bing Chat, Microsoft’s AI-powered chatbot, in Windows — kind of. Windows Copilot looks like the more sophisticated version that we originally anticipated. The video opens by claiming Copilot will offer “answers to complex questions,” such as “help me plan my fishing trip.” That sounds like Bing Chat to us.
Likewise, Panay copied a bunch of code and asked Windows Copilot to interpret what it is and what it does. Again, that’s a capability that Bing Chat has been able to fulfill.
In his own demonstration, Panay typed in “how can I adjust my system to get work done.” Copilot assumed he was referring to a Windows 11 system, and provided an answer. On the Web, Bing Chat doesn’t make the same assumption. That could mean that Copilot is searching local help files, rather than the Internet. Why does that matter? We’ll talk about that more, below.
An automated Settings assistant
There’s a case to be made that Windows Copilot is an admission that Windows is just too complex for ordinary users, who feel like they can’t use it to its full advantage. “Control your Windows environment,” the next suggestion, will save you from digging through menus and apps for specific tasks like turning on dark mode, for example, or something a little broader like suggestions to use Windows to ease eye strain. It’s the latter that benefits from Copilot, in that suggestions like “adjust my settings so I can focus” need some intelligent interpretation.
Interestingly, when Panay asked Copilot to adjust his system to get work done, Copilot launched Windows 11’s dark mode and its Focus mode by itself.
Windows Copilot switched to Windows to dark mode and launched a Focus Session.
Mark Hachman / IDG
Work across documents
Windows Copilot will adapt some of the features in Microsoft Edge Copilot, namely AI summarization and content creation. This is why we think Microsoft 365 Copilot (AI for Office apps) might be Microsoft’s killer AI app, in that it will save time reading and responding to information. There’s no indication that you’ll need a Microsoft 365 subscription for this, yet, but we almost certainly expect a Microsoft account will be necessary.
Launch apps
In his demonstration, Panay asked Copilot, “What’s great music to work to?” Copilot responded by opening Spotify and suggesting three playlists. The two big next steps in Bing Chat and OpenAI’s ChatGPT are both plugins and web browsing — both sources of information to supplement what both already know. On the PC, it seems that the “plugin” will simply be apps like Spotify.
Mark Hachman / IDG
Later, Panay asked Windows Copilot “how do I create a logo for my business?” and Windows Copilot not only responded, promoting Adobe Express, but also opened the app as well. It then sent the logo to a coworker, using Teams.
Of course, that suggests the question: what, if any, apps will Copilot not be able to launch or understand?
Automated transcription
Voice transcription is near and dear to our journalist hearts, as it’s always handy to have a comprehensive, textual record of what someone actually says. Microsoft has made this part of its business services (Teams, mainly) for a number of years now, but it looks like it will come to Windows, too. Panay dragged an MP3 recording into the chat box (interesting in and of itself!) and Windows Copilot began generating a transcription.
Dropping files into Windows Copilot for transcription or analysis seems pretty cool.
Mark Hachman / IDG
What will be Windows Copilot’s hardware requirements be?
All of this begs the question: How will Windows Copilot actually run?
Recall Windows 11’s strict hardware requirements, and how they evolved: first a Trusted Platform Module, then not, and finally some middle ground between the two. Likewise, AI art apps like Stable Diffusion can run on local PCs, but they’ve traditionally run best on laptops or desktops with discrete GPUs and lots of video memory to run the algorithms, plus storage to store the code itself.
CPUs with dedicated AI blocks attached to them add another wrinkle. Qualcomm’s Snapdragon 8cx Gen 3 processor has a Hexagon DSP attached to it specifically for AI processing, while the AMD Ryzen 7040U and its Ryzen AI block pack dedicated AI hardware too. Specific Intel 13th-gen Core chips have a Movidius AI card attached, and Intel’s Meteor Lake will have integrated AI, according to chief executive Pat Gelsinger. All these, however, make up a small percentage of the available PCs.
We don’t know how Microsoft plans to address this disparity, but we do have a few hints. For one, Microsoft launched machine-learning API Windows ML a few years back to address AI, but without any killer apps, it faded from public view. This week, Microsoft said that it’s working with AMD, Intel, and Nvidia to optimize their transformer and diffusion models to run AI locally on their hardware. for apps like Stable Diffusion, and AMD will release optimized drivers for AI on its Radeon RX 7900 GPU and Ryzen 7040 CPU in the next month, Microsoft said.
There’s still the question of whether local hardware can keep up with the demands of Windows Copilot and other AI. For that, Microsoft might apply something called Hybrid Loop, which would allow the cloud and the PC to work in tandem. At Build, Pavan Davuluri, a corporate vice president a Microsoft, said that Hybrid Loop makes Microsoft’s Azure cloud “show up just like a coprocessor in Windows, just like an NPU or a GPU.” That might allow a PC to use a local GPU or AI coprocessor for as much processing as it can, tapping the Azure cloud if it can’t keep up.
That still doesn’t answer the question of how Copilot will work on your PC, especially if you don’t have the absolute latest hardware. We asked Microsoft for comment, and company officials responded that they have nothing official to share. Rest assured, however, that if Microsoft is as driven as they seem to make Windows Copilot part of your PC, they’ll need to solve these problems. The answer, though, might demand you buy a more powerful PC.
by #AI May 25, 2023 · Automatic
