Microsoft’s Ambitious Plan: Democratizing AI Search for the Web with NLWeb

Microsoft is on a mission to transform the web, not by centralizing power, but by distributing it. At this year's Build developer conference, Microsoft unveiled NLWeb, a new open protocol that aims to empower every website to run its own AI-powered search, bypassing the dominance of centralized chatbots like ChatGPT and Bing. Is this the key to a more open and dynamic web?

Ramanathan V. Guha, a technical fellow at Microsoft and a key figure behind RSS and Schema.org, envisions NLWeb as the fourth revolution in personal computing, following graphical user interfaces, the internet, and mobile. This revolution centers on communicating with applications using free-form language. However, Guha fears the current trend of AI communication being mediated by large language models (LLMs) will create a web dominated by chatbots that extract knowledge without returning value.

NLWeb offers a solution by enabling website owners to easily integrate ChatGPT-style interaction features directly into their platforms. By adding a few lines of NLWeb code, choosing an AI model, and supplying the model with their own data, website owners can create custom chatbots in minutes. According to Guha, "It's a protocol, and the protocol is a way of asking a natural-language question, and the answer comes back in structured form." Essentially, NLWeb handles the complex logistics, allowing developers to focus on their data and expertise.

Guha demonstrated NLWeb's capabilities with examples from Serious Eats and Backcountry. On Serious Eats, NLWeb quickly provided a list of spicy and crunchy appetizer recipes suitable for a vegetarian Diwali celebration. On Backcountry, it recommended jackets warm enough for Quebec, showcasing its ability to understand context and remember preferences. Crucially, these examples highlight the potential for businesses to provide highly relevant and personalized search experiences without relying on external AI services.

According to TechCrunch, Microsoft is also launching newEdge APIs that would tap existing models built into the browser itself further empowering developers.

One of the critical benefits of NLWeb is its affordability. Traditional web search indexes require extensive crawling and indexing, which are expensive for both search engines and websites. With NLWeb, Guha claims, websites can simply use an RSS feed and a vector database, powered by an inexpensive model like GPT-4o Mini. This approach offers a far cheaper and easier way to enable intelligent search and interaction.

Microsoft's CTO, Kevin Scott, envisions NLWeb as a key component of the "agentic web," where AI agents can seamlessly interact with websites and services on behalf of users. These agents will require open protocols like the Model Context Protocol (MCP), initially developed by Anthropic, to navigate and access information in a structured and controlled way. While the agentic web is still in its early stages, Scott believes that it represents a fundamental shift in how users interact with the internet.

"If the way that people want to do research or they want to transact business is via their agents, and that's where the intent lives and where the user desire originates, then you're going to want to have some mechanism where you can connect to that," Scott said in a recent interview with The Verge.

However, the biggest challenge facing NLWeb is centralization. The web has a history of consolidating around aggregators like Yahoo and search engines like Google. Despite this trend, Guha hopes to empower individual websites with the best technology, allowing them to build a more decentralized and dynamic internet.

NLWeb faces challenges, including securing widespread support and overcoming the centralization forces that have historically shaped the web. However, if successful, it could revolutionize online interaction, empowering websites to deliver personalized and intelligent search experiences while fostering a more open and decentralized internet.

What impact do you think NLWeb could have on the future of the web? Share your thoughts in the comments below.

• AI
• Microsoft
• NLWeb