Microsoft’s new Copilot will change Office documents forever

Microsoft’s new AI-powered Copilot summarized my meeting instantly yesterday (the meeting was with Microsoft to discuss Copilot, of course) before listing out the questions I’d asked just seconds before. I’ve watched Microsoft demo the future of work for years with concepts about virtual assistants, but Copilot is the closest thing I’ve ever seen to them coming true.

“In our minds this is the new way of computing, the new way of working with technology, and the most adaptive technology we’ve seen,” says Jon Friedman, corporate vice president of design and research at Microsoft, in an interview with The Verge.

I was speaking to Friedman in a Teams call when he activated Copilot midway through our meeting to perform its AI-powered magic. Microsoft has a flashy marketing video that shows off Copilot’s potential, but seeing Friedman demonstrate this in real time across Office apps and in Teams left me convinced it will forever change how we interact with software, create documents, and ultimately, how we work.

Copilot appears in Office apps as a useful AI chatbot on the sidebar, but it’s much more than just that. You could be in the middle of a Word document, and it will gently appear when you highlight an entire paragraph — much like how Word has UI prompts that highlight your spelling mistakes. You can use it to rewrite your paragraphs with 10 suggestions of new text to flick through and freely edit, or you can have Copilot generate entire documents for you.

This adaptability is what sets it apart from Microsoft just shoving ChatGPT into a sidebar in Office. Copilot doesn’t just offer up a chatbot interface — you can use it to command Office apps like Excel and PowerPoint. If you’re looking at a slide deck and wish every title were orange instead of blue, just ask Copilot instead of having to dig into PowerPoint features.

In Excel, you can have Copilot generate a PivotTable, create a graph, or just help you understand the rows and columns of data in front of you. “One of the ways we’re starting with Copilot is helping analyze and understand data,” says Friedman. “You can ask Copilot what it makes of the data, you can get graphs from Copilot based on trends it sees in the data, and you can insert those trends into a spreadsheet.” Excel even has a “show me” feature for Copilot that will let this AI teach you how it just completed a command so you can improve your Office knowledge.

It feels like Microsoft is slowly building on the vision it had for its Cortana assistant or even Clippy decades before. “I love that our heritage is full of products that try to adapt to people,” says Friedman. “Copilot shares some similarities with some things we’ve done in the past, but it’s far more capable, it’s humble, and it’s there to serve things up for you that help you save time.”

Microsoft has customized this Copilot system for every Office app, so there are different ways to command it. Friedman demonstrated to me how Copilot can help you write emails in Outlook, offering up short or long message drafts with options to change the tone. It even works in the mobile version of Outlook, which got me thinking about the ways this could speed up work on the go.

“Outlook mobile is the first place where we’re doing a big push,” explains Friedman. Outlook can summarize all your emails on the go, generate drafts, and generally make it easier to triage your inbox. But imagine creating entire Word documents from your phone without having to type on a tiny on-screen keyboard. “We’ll have more to talk about mobile in the coming months,” says Friedman. But you can imagine where things will go.

As impressive as Copilot is, we’ve seen the myriad ways that large language models can fail, including inserting racial or gender bias into text and simply making things up. Those traits are alarming enough in a search engine, but when you’re talking about Excel (which arguably powers the world’s economy) or your email inbox, it’s a whole different level of ethics, privacy, and data concerns.

“It gets things right a lot of the time but not all of the time,” admits Friedman. “In the user experience we do things like put in affordances to not send something until you’ve read it, or to encourage you to try again, edit, and discard.”

Microsoft also has a number of warnings inside Copilot that appear as you use it. In PowerPoint, you’ll see a message that says, “Content is generated by AI and might contain inaccuracies for sensitive material. Be sure to verify information.” Elsewhere, there are prompts that say, “AI-generated content may be incorrect.” Microsoft is trying to design the system in a way that reminds you that you’re in charge.

“We give you tools to report it when it’s wrong. We create prompt suggestions to help you write good prompts. Everything we’re doing in the user experience is to make it conversational and give you agency,” says Friedman.

We’ve seen what happens when it goes wrong in Microsoft’s Bing search engine. The AI-powered chatbot has hallucinated on multiple occasions, and Microsoft has had to put limits in place to control its outbursts. In one conversation with The Verge, Bing even claimed it spied on Microsoft’s employees through webcams on their laptops and manipulated them.

“Everything we’re learning from Bing in preview is helping us mitigate those risks,” says Friedman. “We’re applying all that learning and thinking to Copilot as well.” Microsoft is also starting off small with its Copilot rollout. It will initially be available to just 20 businesses before Microsoft opens it up to more when it’s ready. Microsoft is also starting with enterprise customers first before rolling it out to consumers.

“We feel pretty good about what we have as a starting point, but we don’t yet know if it’s performing the way we want it to and helping really empower people to accomplish their jobs,” says Friedman. “We’ll quickly iterate, we’ve been building fast, very fast. But we pause and we learn a lot and we’ll update really fast. Our plan is to move as fast as we possibly can to scale to more of enterprise in a thoughtful and responsible way and make sure the experience is awesome.”

Is Microsoft moving too fast, though? Google announced its own AI features for Gmail and Docs earlier this week, and the AI race has many experts worried that tech giants aren’t properly considering the impact of these new tools.

“In our mind we’re being thoughtfully quick,” says Friedman. “We’re being thoughtful in that we’re rolling it out to 20 customers and working side by side with them.” Despite reports of Microsoft laying off its ethics and society team that taught employees how to make AI tools responsibly, Friedman says Microsoft is growing the people working on these concerns. “In terms of our investment in ethics and AI, we’ve grown more and more ethics and AI experts in all of the product teams working on this stuff,” says Friedman. “We have to scale way bigger, so we’ve been investing more heavily and it continues to grow year over year.”

Microsoft knows Copilot isn’t perfect and that it’s going to take some time to get there. Although it impressed me during the Teams meeting recap, it could have easily confused my voice for someone else’s if I’d been using a bad microphone, or Outlook could pull out the wrong summary in an email thread. There are big challenges ahead, but Microsoft is hoping the work it does on making it easy to edit responses, correct sources, and issue feedback will ultimately improve the system.

“We know AI gets things wrong, we know it hallucinates and we know it does it confidently,” admits Friedman. “We continue to work on making it better at doing that less, but also that the user experience really empowers people and puts them in the driver’s seat.”

For all the challenges, the future of the Copilot system won’t just be text-based generation, either. Microsoft has a clear vision to use Copilot to generate images, video, and more once large language models can handle these features well.

Microsoft has already integrated OpenAI’s DALL-E model into its Designer app, which allows people to generate images based on text. Designer will also help PowerPoint select the best imagery for AI-generated slides. “We’re going to bake Designer further into Copilot so you can change things within Designer,” says Friedman. “The Designer stuff you’ve seen today is just scratching the surface. I fully suspect we’re going to use Copilot to do amazing multimedia things.”

So where else could we see Copilot appear? I asked Friedman about Windows integration. “We’re looking at all sorts of places and ways to expand [Copilot] out. I believe this is the next major wave of computing and it will change the way we work with all devices in the coming years,” says Friedman.

The future will also include a multiplayer experience for Copilot, too. Loop components, one of the biggest changes to Office documents in decades, are available in Teams and Outlook. Loop components, the branding for Microsoft’s Fluid work, are blocks of collaborative text or content that can live independently and be copied, pasted, and shared freely.

Now, imagine copying a Loop component of text into an email and having multiple people edit it and interact with the Copilot. “In the component while we’re editing, the conversation is a clickable history of the content being generated that you can go back and forth through,” says Friedman. “What’s so cool about it is that it feels like a totally new mental model with how to work with a Copilot with a group of people.”

All of these Copilot features for Office and Microsoft 365 feel like they will forever change how we work and communicate, especially as these large language models evolve in the years ahead. Microsoft’s push to integrate this AI deep into its products could have a lasting effect on the job market.

“Every time there’s a new tech advancement there are both opportunities and things we have to consider,” says Friedman. “We believe infusing this AI is going to create new job opportunities long term, and increase job satisfaction short term. We expect it’s going to change the nature of a lot of jobs and create new jobs that didn’t exist before. That’s why empowering people and building this common design system is so important to us.”