You shove a 200,000-word technical manual into the latest LLM. You ask it how to configure the parameter on page 85. The model thinks for a long time. It spits out an answer, dead wrong. It made up a parameter and burned through 10 yuan in tokens to do it.
This happens every day. Today’s models have ever-larger context windows — in theory, they can read hundreds of thousands, even millions of words. But the more text you cram in, the worse the model often answers. The Chroma team ran a test. They took 18 models, stuffed long text into their windows, and evaluated comprehension. The result: all 18 models, without exception, degraded in performance. The industry calls this context rot.
This shows that simply making windows bigger won’t solve the problem. Anthropic’s engineering team also published an article. They said the path to better long-context capability is definitely not blindly enlarging windows.
And the longer the text and the harder the task, the faster the model forgets. In a needle-in-a-haystack task (NIAH), it can barely cope. But in complex OOLONG tasks, where every line’s semantics must be compared, the model is helpless.
To tackle this headache, Alex Zhang proposed a new approach: Recursive Language Models (RLM).
This approach handles context differently. It doesn’t try to solve the computational bottleneck of infinite context. But it points to a path: we should stop expecting to endlessly stretch a model’s native window.
RLM changes the game. In the past, we stuffed context into the prompt and had the model read it all at once. RLM treats context as an external data object. The model writes code and queries it strategically from the outside.
The change is theoretically appealing. But based on current evidence, it also has clear boundaries. The approach only works under depth=1. In this configuration, the system is a root model leading a group of worker models.
Running multi-layer recursive scheduling on current models simply doesn’t work. For models not trained on it, RLM actually runs slower, costs more, and answers worse. So at this stage, RLM is more of an interface convention. It’s not yet a mature silver bullet.
To solve the problems of insufficient window size and models getting dumber, the industry has four mainstream approaches. But when high-fidelity tasks are needed, each has its own shortcomings.
The first is compaction. Some try to reduce the number of tokens. For example, the Context Folding approach, or the JetBrains team’s method. But these methods lose critical details when facing fine-grained tasks. In OOLONG-Pairs testing, compaction approaches scored only 0.1% F1. This shows that once a task demands high detail, compaction drops key information.
The second is vanilla RAG. With RAG, you have to pre-split documents (chunking). But you cannot predict every query dimension at runtime. When a task requires finding clues across paragraphs, vector search struggles to pinpoint them. It easily misses key fragments.
The third is code-writing agents, like OpenCode. When dealing with large files, they write scripts or run grep. This approach is good at finding keywords. But when it comes to stitching together and comparing semantics, it falls flat. In OOLONG-Pairs testing, the OpenCode approach scored only 4.8% F1.
The fourth is stuffing everything into a big window. This hits physical limits. Beyond the limit, the model stops working. And when text is too long — even without hitting the limit — the model misses key points buried deep inside.
There have also been attempts at virtual memory management. MemGPT is one such effort. But when processing data, it still has to constantly move context back into the model’s window. It hasn’t escaped its dependence on the native window. At its core, it’s still “load first, manage later” caching.
RLM takes a different path. It fundamentally reverses the relationship between model and context: instead of stuffing text into the window, it leaves the text outside, treating it as external data.
When the root model (Root LM) starts working, its window contains only metadata — the structure, outline, and line numbers of the long text, not the actual text itself. The real long text exists as variables in an external REPL.
The root model doesn’t need to read all tokens at once. It acts like a commander, actively querying data by writing code. It sends instructions to the REPL, like peek or grep. It can also split long text into smaller chunks.
When it finds useful fragments, it dispatches parallel worker models (sub-LLM). These worker models receive filtered text slices and analyze or compute in the background.
Finally, the root model assembles the results from each worker model into the final answer.
This approach is fundamentally different from CodeAct. Traditional code-writing agents do run code, but they still need to read data back into the window to make decisions.
RLM has a hard constraint: the original long text must never enter the root model’s window. This is a “never-load” design. It breaks the old cycle of “load first, manage later.”
Let’s look at the numbers. In the paper, researchers had RLM running with GPT-5-mini on OOLONG tests. At 132K context, it scored 34 points higher than native GPT-5.
BrowseComp-Plus tested 1K docs (6-11M tokens). Native GPT-5 scored 0%, while RLM reached 91.3% accuracy. On the OOLONG-Pairs task, native GPT-5 scored only 0.04% F1. With RLM, that number rose to 58.0%.
The open-source community has also shown enthusiasm for this model. The alexzhang13/rlm project has 5230 stars on GitHub. It currently has 850 forks and 81 open issues.
Someone did an independent replication. It works well at depth=1. DeepSeek v3.2 on OOLONG went from 0% to 42.1%.
Researchers fine-tuned an 8B model with 1000 samples. After fine-tuning, the model’s median performance improved by 28.3%.
But behind these shiny numbers lurk plenty of holes and limitations.
The 91.3% accuracy on BrowseComp-Plus may be inflated. Anthropic’s team found 20+ data leakage sources in this test set during evaluation. This means the model may have already seen the answers.
OOLONG-Pairs is a test set designed by the author himself. Native models got 0.04%, their own approach got 58.0%. This smells like self-dealing. Moreover, all baseline comparisons were implemented by the author. They could hardly tune their opponents’ parameters to the best.
Set the call depth to depth=2 and the system goes completely off the rails. Replication data shows models get confused handling more than two layers of scheduling, and logic starts breaking down. This results in 28x latency and 100x token consumption. In practice, depth=2 is nearly unusable.
Some tasks actually get worse with RLM. Prime Intellect ran ablation studies and delivered bad news: in math and scientific research (DeepDive) tasks, RLM’s performance degrades. These tasks require rigorous logical deduction and aren’t suited for RLM.
Models dynamically query data at runtime, which makes the number of queries wildly unstable. Under complex tasks, the high-percentile (Q95) bill can spike to over 10x the median, making costs completely unpredictable.
The open-source community is skeptical too. AI developer Teknium voiced doubt on social media: Can someone explain to me how RLM is not just grep that all coding agents already use but in a subagent.
We’ve previously talked about the concept of “Context Governance.” Back then, the idea was for a system outside the model to manage context. The system helps the model slice and filter, feeds specific data into the window, and limits what the model can read.
RLM hands this power to the model itself. It lets the model decide: when to read, how much to read, how to aggregate data.
This doesn’t mean RLM can replace the external governance layer. In real engineering, the governance layer does the heavy lifting and sets the big-picture direction. Things like isolating security permissions, auditing API calls, capping budgets, and guaranteeing reliability.
RLM does the fine-grained work. It helps the model pick and address local data at runtime.
Without a governance layer capping the budget, RLM turns into an infinite money-burning loop. But without RLM’s interface convention, the governance layer is just a rigid pipeline of hardcoded rules.
So the two can work together. RLM is the next step in context governance: the system sets the rules, the model queries data itself.
Current RLM can’t run depth=2, and it faces issues like data leakage and sky-high costs. But it offers a provocative new angle.
The path forward for long-context processing is probably not endlessly stretching attention windows. We should explore how to equip data with a scheduling interface inside an external REPL.
RLM’s true value isn’t the scores. It turns context into an external data space. In this space, the model finds, previews, and processes data by executing code instructions.
Turning this interface convention into a standard product may well be the real direction for managing long context well.