Nolita is being reconstructed with a new Python codebase, built around preliminary navigation planning and hierarchical agent delegation. For more information on what this means, read below.
Wormholes form lateral transport: within the same universe; across different universes; and through gravitational sleights of hand you could theoretically travel through time, though for the moment it remains science fiction. Still, nothing stops us from lateral transport within the realms of imagination. We entered our exploratory mode willing to reinvent anything so long as it perfected our mission to the broadest possible audience: to connect and extend new intelligence.
In our first missive today we describe a new architecture for our autonomous navigation stack, Nolita, and its Memory Index, currently in preview.
Northern Little Italy
While redeveloping the Memory Index, it was evident that a lot of the work involved in more intelligent objective assessment and optimisation was tied into Nolita proper.
Whether we rely on the Index, or on Nolita, for different functionalities depended on where we delivered the most extensible and robust experience for developers. High Dimensional Research crafts tools and provides supplemental, data-oriented extensions for those tools as our core service; our tools therefore need to be as easy to integrate within your own stack as they possibly can be. For this reason, we've historically avoided serving a black boxed model of our own.
We believe in treating models as relatively agnostic slots within a stack, where you can use weaker or stronger models for the task as necessary. We also believe in being more resourceful than bringing a bigger gun, so to speak, when the work involved requires higher precision.
State of affairs
Our pre-existing Nolita codebase takes a monolithic agent approach; it's comparatively naive in its simplicity, and yet it's gotten decent results for simpler tasks.
- We enter an objective and a starting URL and give it to Nolita.
- It enters into a loop between itself and an LLM where we go from page to page, see all our previous actions on the page (with an assessment of our current progress) and generate new steps.
- Inbetween steps, our Memory Index can substitute actions if we've seen similar objectives and actions in the past. Given the intent of the user query and the pages in question, we assess if we have similar trajectories in the Index to use to advance the state without generating steps with the LLM.
So how can we improve upon this? Either we can improve the way we fetch steps in the Index, or we can reconsider the single agent approach entirely.
Navigation hivemind
Obviously, we chose the latter. Instead of one agent, we have three high-level agents and over a dozen low-level agents handled by an orchestrator.
A planning agent, an execution agent, and an evaluation agent all cooperate to more robustly navigate through tasks. Imagine the difference between holding the high-level picture behind a task in your head as you do it -- and having a manager hold onto the bigger picture while only telling you what you need to focus on. When the plan goes awry, we refer back to our planner to amend the plan. The execution agent has a team of agents filling in the details when using any function it can perform in a web browser -- and all of these agents can integrate with and substitute steps from the Index.
Most importantly, we no longer have to pass a full DOM tree analysis into context; we can preprocess it with a dedicated agent and stream the page down to relevant details -- this, too, can use previous sessions from the Index to analyse what each DOM element "does" for a user.
The goal behind these changes is to continue to take load off of an LLM for a task, leaving its main speciality to be its continuous malleability when encountering new situations.
Why Python?
When performing this rewrite, we found that orchestration support was simply superior inside a Python codebase. Additionally, we've found in user interviews that developers are integrating Nolita inside predominantly Python codebases. It's seemed overdue that we change to match.
For current users of Nolita, we intend to retire Nolita as written, leaving it as the nolita-legacy codebase, and stripping calls to the Index, leaving it in a workable state for basic workflows in TypeScript.
Preview
We'll have more to say when we can show this version of Nolita publicly. We're currently in private testing with design partners. If you're interested in testing the next generation of Nolita, get in touch with Tynan at tynan.daly@hdr.is.