Many organizations can be hesitant to overtake their tech stack and begin from scratch.
Not Notion.
For the three.0 model of its productiveness software program (launched in September), the corporate didn’t hesitate to rebuild from the bottom up; they acknowledged that it was obligatory, actually, to help agentic AI at enterprise scale.
Whereas conventional AI-powered workflows contain specific, step-by-step directions based mostly on few-shot studying, AI brokers powered by superior reasoning fashions are considerate about software definition, can determine and comprehend what instruments they’ve at their disposal and plan subsequent steps.
“Quite than making an attempt to retrofit into what we had been constructing, we needed to play to the strengths of reasoning fashions,” Sarah Sachs, Notion’s head of AI modeling, informed VentureBeat. “We've rebuilt a brand new structure as a result of workflows are completely different from brokers.”
Re-orchestrating so fashions can work autonomously
Notion has been adopted by 94% of Forbes AI 50 corporations, has 100 million whole customers and counts amongst its clients OpenAI, Cursor, Figma, Ramp and Vercel.
In a quickly evolving AI panorama, the corporate recognized the necessity to transfer past easier, task-based workflows to goal-oriented reasoning programs that enable brokers to autonomously choose, orchestrate, and execute instruments throughout linked environments.
In a short time, reasoning models have change into “much better” at studying to make use of instruments and comply with chain-of-thought (CoT) directions, Sachs famous. This enables them to be “much more unbiased” and make a number of choices inside one agentic workflow. “We rebuilt our AI system to play to that," she stated.
From an engineering perspective, this meant changing inflexible prompt-based flows with a unified orchestration mannequin, Sachs defined. This core mannequin is supported by modular sub-agents that search Notion and the net, question and add to databases and edit content material.
Every agent makes use of instruments contextually; for example, they will determine whether or not to look Notion itself, or one other platform like Slack. The mannequin will carry out successive searches till the related data is discovered. It could actually then, for example, convert notes into proposals, create follow-up messages, monitor duties, and spot and make updates in data bases.
In Notion 2.0, the crew centered on having AI carry out particular duties, which required them to “suppose exhaustively” about easy methods to immediate the mannequin, Sachs famous. Nonetheless, with model 3.0, customers can assign duties to brokers, and brokers can really take motion and carry out a number of duties concurrently.
“We reorchestrated it to be self-selecting on the instruments, relatively than few-shotting, which is explicitly prompting easy methods to undergo all these completely different eventualities,” Sachs defined. The goal is to make sure every little thing interfaces with AI and that “something you are able to do, your Notion agent can do.”
Bifurcating to isolate hallucinations
Notion’s philosophy of “higher, quicker, cheaper,” drives a steady iteration cycle that balances latency and accuracy via fine-tuned vector embeddings and elastic search optimization. Sachs’ crew employs a rigorous analysis framework that mixes deterministic assessments, vernacular optimization, human-annotated information and LLMs-as-a-judge, with model-based scoring figuring out discrepancies and inaccuracies.
“By bifurcating the analysis, we're in a position to determine the place the issues come from, and that helps us isolate pointless hallucinations,” Sachs defined. Additional, making the structure itself easier means it’s simpler to make modifications as fashions and strategies evolve.
“We optimize latency and parallel pondering as a lot as doable,” which results in “manner higher accuracy,” Sachs famous. Fashions are grounded in information from the net and the Notion linked workspace.
In the end, Sachs reported, the funding in rebuilding its structure has already offered Notion returns when it comes to functionality and quicker charge of change.
She added, “We’re totally open to rebuilding it once more, when the subsequent breakthrough occurs, if we’ve to.”
Understanding contextual latency
When constructing and fine-tuning fashions, it’s vital to know that latency is subjective: AI should present probably the most related data, not essentially probably the most, at the price of velocity.
“You'd be shocked on the alternative ways clients are keen to attend for issues and never watch for issues,” Sachs stated. It makes for an fascinating experiment: How gradual are you able to go earlier than folks abandon the mannequin?
With pure navigational search, for example, customers might not be as affected person; they need solutions near-immediately. “Should you ask, ‘What's two plus two,’ you don't wish to wait in your agent to be looking out in all places in Slack and JIRA,” Sachs identified.
However the longer the time it's given, the extra exhaustive a reasoning agent might be. As an illustration, Notion can carry out 20 minutes of autonomous work throughout a whole bunch of internet sites, recordsdata and different supplies. In these cases, customers are extra keen to attend, Sachs defined; they permit the mannequin to execute within the background whereas they attend to different duties.
“It's a product query,” stated Sachs. “How will we set consumer expectations from the UI? How will we confirm consumer expectations on latency?”
Notion is its greatest consumer
Notion understands the significance of utilizing its personal product — actually, its workers are amongst its greatest energy customers.
Sachs defined that groups have lively sandboxes that generate coaching and analysis information, in addition to a “actually lively” thumbs-up-thumbs-down consumer suggestions loop. Customers aren’t shy about saying what they suppose needs to be improved or options they’d prefer to see.
Sachs emphasised that when a consumer thumbs down an interplay, they’re explicitly giving permission to a human annotator to research that interplay in a manner that de-anonymizes them as a lot as doable.
“We’re utilizing our personal software as an organization all day, day by day, and so we get actually quick suggestions loops,” stated Sachs. “We’re actually dogfooding our personal product.”
That stated, it’s their very own product they’re constructing, Sachs famous, in order that they perceive that they could have goggles on in the case of high quality and performance. To steadiness this out, Notion has trusted "very AI-savvy" design companions who’re granted early entry to new capabilities and supply vital suggestions.
Sachs emphasised that that is simply as vital as inner prototyping.
“We're all about experimenting within the open, I feel you get a lot richer suggestions,” stated Sachs. “As a result of on the finish of the day, if we simply have a look at how Notion makes use of Notion, we're not likely giving the perfect expertise to our clients.”
Simply as importantly, steady inner testing permits groups to judge progressions and ensure fashions aren't regressing (when accuracy and efficiency degrades over time). "All the pieces you're doing stays trustworthy," Sachs defined. " that your latency is inside bounds."
Many corporations make the error of focusing too intensely on retroactively-focused evans; this makes it tough for them to know how or the place they're enhancing, Sachs identified. Notion considers evals as a "litmus check" of growth and forward-looking development and evals of observability and regression proofing.
“I feel an enormous mistake a number of corporations make is conflating the 2,” stated Sachs. “We use them for each functions; we take into consideration them actually in another way.”
Takeaways from Notion's journey
For enterprises, Notion can function a blueprint for easy methods to responsibly and dynamically operationalize agentic AI in a linked, permissioned enterprise workspace.
Sach’s takeaways for different tech leaders:
-
Don’t be afraid to rebuild when foundational capabilities change; Notion totally re-engineered its structure to align with reasoning-based fashions.
-
Deal with latency as contextual: Optimize per use case, relatively than universally.
-
Floor all outputs in reliable, curated enterprise information to make sure accuracy and belief.
She suggested: “Be keen to make the laborious choices. Be keen to take a seat on the prime of the frontier, so to talk, on what you're growing to construct the perfect product you’ll be able to in your clients.”
