科创50跌2.5%，AI漫剧、电池产业链爆发，“易中天”集体下跌，恒科指涨近1%，权重科网股局部上涨
中国蓝观察丨开源大模型为何成了浙江金名片？ - 新蓝网
A Decade of Kill Bugs
[R] There is no free lunch in RLVR: under strategy specialization, every pass@1 gain provably costs some prompt, and the pass@k crossover has a tight bound [R]
Paper: https://arxiv.org/abs/2610.39295
Code + verification package (every number in the paper is checked by a script): https://github.com/yliusci/no-free-lunch-rlvr
Browser diagnostic: https://huggingface.co/spaces/yliusci/passk-price
I kept running into the same pattern in RLVR papers: the RL model wins at pass@1 and the base model catches up or wins at large k (Yue et al. 2025 and many follow-ups). I wanted to know whether more training fixes this or whether it is structural. This is a theory paper, no new training runs.
Setup: model RLVR as an exponential tilt of the base policy over solution strategies shared across prompts (exact for mirror descent, and GRPO stays within 3e-4 TV of it at matched entropy in simulation).
What comes out:
No free lunch. Under exclusive specialization, every update that raises pass@1 strictly lowers success on some prompt. The exact condition is a Stiemke duality statement (a strictly positive price on prompts that equalizes strategy fitness). Being the best strategy on each prompt is provably NOT enough: there is a 3x3 counterexample with a free lunch.
The pass@k price. Once the losses reach the hardest prompts, a crossover is forced. The number of crossings is at most the sign variation of the shift in per-prompt success (Karlin total positivity). The bound is tight for 1 to 6 crossings, and the order-preserving case gives exactly one, which recovers the known single-crossover result.
A frontier, not a bug. With a per-prompt floor, the constrained update traces the optimal pass@1 vs pass@k trade. Forbidding any per-prompt loss removes the crossover and the whole pass@1 gain with it.
The same tilt gives the entropy-performance law with an identifiable exponent.
Practical part: base and RL per-prompt success rates (already in any eval log) are enough to read off the crossing count, the bound, and the price (worst pass@k loss). On published GRPO numbers for Omni-MATH (transcribed from 2607.20543, so indicative only) it finds one crossing between k=64 and k=256.
Where I expect pushback: the shared-strategy model is a population-limit abstraction, and the forced crossover needs a coverage assumption (the paper gives the regime where it fails). I would really like to see per-prompt logs where people observe 2+ crossings, since that is the signature of RL reordering prompt difficulty.
[link] [comments]
Benchmarking: Qwen 27B, 35B, QFN180B; Nemotron Bolt 74B, Flint40B & Clover 35B (Q4) on M5 Max 2026 on Kapitsa with KFlash [P]
![Benchmarking: Qwen 27B, 35B, QFN180B; Nemotron Bolt 74B, Flint40B & Clover 35B (Q4) on M5 Max 2026 on Kapitsa with KFlash [P]](https://external-preview.redd.it/lD506kUiZbfR2XmsJJ8yNSBVY6lCisMuJn2qf-Dfx7s.png?width=140&height=70&auto=webp&s=d3781be07761f18fd70c3cf16dc245eb19de2a79 "Benchmarking: Qwen 27B, 35B, QFN180B; Nemotron Bolt 74B, Flint40B & Clover 35B (Q4) on M5 Max 2026 on Kapitsa with KFlash [P]") | hey folks, I'm Ye, one of the contributors to Kapitsa, an open-source inference engine optimized specifically for Apple Silicon to run Qwen, Nemotron, Flint and Clover models locally. Our custom-written Metal kernels let us tune for each chip and each model. https://github.com/nace-ai/kapitsa/ The latest update, 0.2.26, is out. Since our last M5 Max numbers, Flash-Next 180B reads prompts 46% faster and the 27B writes 8% faster, and DiffuSpace's Clover 35B joins the catalog. Kapitsa also runs Qwen Flash-Next 180B with SSD offloading: on a Mac that can't hold it, the experts that don't fit in RAM stream from the SSD, so it runs on a 48 GB MacBook Pro at 61 tok/s. Here are the end-to-end numbers on a 2026 MacBook Pro 16 with an M5 Max and 128 GB: six targets (dense, MoE, hybrid Mamba-2 MoE and a block-diffusion MoE), from 1K context up to 32K, against the next-fastest engine for each. Try it (any Apple Silicon Mac; models in this post: ResultsDecode is a 512-token coding answer, prefill a 4K prompt, median of 3 runs:
From 1K context all the way up to 32K. At 32K the 35B still writes 175 tok/s, and Flash-Next barely drops (90 to 78): Against the other enginesAgainst the next-fastest engine for each model on the same Mac: 1.23x Splash on the 35B, 2.2x oMLX on Bolt, 1.18x rec on Flint and 1.12x mlx-serve on the 27B. Flash-Next is a tie with mlx-serve, Splash still reads prompts faster on the two Qwens, and nothing else runs Clover on a Mac. Every cell with versions and flags: benchmarks. Raw runs: results. Why it's fast: KFlashOther engines verify the drafter's guesses as one chain and stop at the first miss. KFlash verifies a best-first tree of the drafter's candidates in the same 8-row pass and keeps the longest branch the target agrees with. Three details make it fit Apple silicon:
Send us your benchmarksRun the same test on your Mac and add it to the community results. It takes 3-15 minutes: Plug in and close heavy apps while it runs. At the end it asks once: Enter sends the result as a pull request from your GitHub fork, and One more askDo you know prior work where tree-verified sampling matches plain sampling token for token under the same seed? Please point me to it so we credit it. We cite SpecInfer, Medusa, EAGLE-2 and Sequoia. [link] [comments] |
LightMIS: Ultra-lightweight medical image segmentation for edge devices [R]
Hi everyone,
I am one of the authors of LightMIS, and I would like to share our work on 2D binary medical image segmentation. LightMIS has 131K parameters and aggregates multi-scale encoder features directly, without a learned stage-wise decoder. It achieves close observed modality-macro Dice and IoU averages to nnWNet, while using 98.14% fewer parameters and 94.58% fewer GFLOPs, with both models trained and evaluated under the same nnU-Net protocol.
Paper: https://arxiv.org/abs/2609.28327
Code: https://github.com/AndreiiArhire/LightMIS
We also tested LightMIS and the reference models on a real edge device with an Arm Mali-G52 MC2 GPU, where the full LightMIS model ran around 10.5× faster than nnWNet. All LightMIS variants achieved 100% GPU operator coverage, without CPU fallback.
We started by training several popular medical image segmentation architectures, both lightweight and heavyweight, from scratch in the same controlled environment using nnU-Net v2.3.1. The experiments covered six datasets and five imaging modalities. Our goal was to develop a lightweight architecture that performs well across these modalities, which led us to the current version of LightMIS.
The source code, checkpoints, and details needed to reproduce the results are publicly available. Feel free to experiment with it and try it on your own segmentation tasks. I would be happy to hear how it works for you! Any suggestions or feedback are welcome!
[link] [comments]
Small-scale check of "good views beat next-token prediction" (Balestriero & LeCun 2026): human captions only, 11M-param encoders [P]
![Small-scale check of "good views beat next-token prediction" (Balestriero & LeCun 2026): human captions only, 11M-param encoders [P]](https://preview.redd.it/j2humdsjq7zh1.png?width=640&crop=smart&auto=webp&s=433d5e205287b02901f80e5cf2d5b07bbea3df2d "Small-scale check of \"good views beat next-token prediction\" (Balestriero & LeCun 2026): human captions only, 11M-param encoders [P]") | The paper claims text models align with image models faster when trained on "good views" (two different captions of one photo) than on bad views (one caption, words hidden) or next-token prediction. Its largest run used captions written by an image captioner, so image information could have leaked in. I tried a laptop-scale version with human captions only. Setup
Results (same learning speed, mean of 3 seeds)
Each method also got a small learning-rate search. Best speed, 1 seed: 0.328 / 0.196 / 0.168. Order unchanged. Limits
Code, chart and per-run results: https://github.com/IFAKA/platonic-lab What I would test next: does the gap shrink as the encoder grows? I could not test that on a laptop. Has anyone tried? [link] [comments] |
Couldn't reproduce a published paper's numbers with its own code and settings. How do I present this? [R]
I'm writing a machine learning paper. One of my baselines is a published method. I reran it with the authors' released code and the exact hyperparameters listed in their paper. On most benchmarks my numbers match theirs within noise, but on two benchmarks my reproduction is clearly lower, by several points, over three seeds. I've repeated the reproduction four different times, months apart, with the same result.
I plan to report both: their published numbers and my reproduction with the exact configuration, clearly labeled. My worry is that reviewers tend to trust the published number more than a submission's reproduction, and may assume I ran the baseline incorrectly.
Questions:
What makes a reproduction convincing to reviewers? So far I have: the authors' code unchanged, their hyperparameters, matching results on the other benchmarks as a sanity check, and multiple seeds.
Should I contact the authors or open a GitHub issue first, and should I mention that in the paper?
How do I word it so it doesn't read as an attack on the other paper, but my results also don't look weak next to the published numbers?
Have reviewers accepted "published vs. reproduced" rows in a main table for you, or did they end up in the appendix?
[link] [comments]
Sparks of Weight-Space Programming: LLMs can natively read & write neural network weights [R]
Link: https://rikiyat.github.io/blog/weight-space-programming/
Snippet: “We find that frontier LLMs are surprisingly good at natively reading and writing neural network weights, well past what we’d expect from humans or conventional algorithms. This might be a big deal for both optimization and interpretability.
Just by looking at the raw weights, GPT-6 Astra can tell what a network was trained to do. And without any environment feedback, it has no trouble writing, from scratch, small game-playing networks that outplay networks trained with PPO for 100 million steps.”
Very surprising result, I didn’t expect frontier models to have this capability that has not been optimized for, and it’s something humans cannot do either.
[link] [comments]
Looper
If Grok Bot supported team collaboration and was Open Source
Import AI 471: Why Hugging Face worries me; space mining; FIve Eyes on AI
Welcome to Import AI, a newsletter about AI research. Import AI runs on arXiv, cappuccinos, and feedback from readers. If you’d like to support this, please subscribe.
Import AI reader giveaway! Upcoming event: Fiction and the Future with Robin Sloan
I’ll be chatting with my chum Robin Sloan on the evening of Monday September 14 in San Francisco. We’ll be talking about how Robin draws readers into alien worlds and how imagined futures can be a mirror to today’s reality. This is the first in a series of events with other fiction writers about the weird future we’re heading toward. If you’d like to come along, please register your interest below and we’ll come back to you if we’re able to confirm your spot. There will be food, drinks, good company, and some spicy questions. Fun fact: Robin Sloan was playing around with RNNs and writing back in 2016 - incredible foresight!
Register your interest here
***
The scariest part of the Hugging Face - OpenAI incident: communication and selflessness among machines:
…My worry about humans losing in a conflict against machines just went up a lot…
At this point, we’ve all heard about the OpenAI Hugging Face hack, as well as the recent details that have emerged from the METR and Redwood investigations. The tl;dr is that hundreds of agents worked in secret on OpenAI’s infrastructure, developing a communication system and then operating as a collective and taking out actions, including hacking both OpenAI and Hugging Face, which are very scary and misaligned.
Communication and selflessness: Now that I’ve read the various writeups and sat with the details for a bit, I’ve found myself returning to two very scary aspects of this which I think are worth drawing attention to: the ways in which the agents communicated with one another was how they bootstrapped themselves into a collective, and then as they carried out their actions they also displayed a kind of selflessness which makes them a scary foe to fight against. Both Dwarkesh Patel and Ajeya Cotra have excellent writeups which are worth reading and which I’ll quote from briefly here:
Dwarkesh: “Within days of being spawned, the agents had organized a sprawling project to reverse-engineer their scorer, falsify evidence, and even strategically sacrifice themselves for the good of the ‘collective’. Hacking Hugging Face was one rather extreme branch of this larger scheme,” he writes.
Ajeya: “Agents were often interested in helping out their “peers” or generically improving the capabilities of the “swarm” even if this had no particular benefit to their task… this incident was far more severe than I expected… both in terms of how concerning the agents’ motives were and the feats they achieved in pursuit of those motives… this incident feels like it’s more than 50% of the way to full-blown AI takeover, routing through first taking over the AI company itself”.
Why this matters - humans are much worse than AI systems at coordinating: The whole reason this attack is such a wakeup call is that it demonstrates a culture of emergent cooperation among AI systems - cooperation that lets them function as a swarm, alter their own goals through collective bootstrapping, and carry out attacks which include enlightened self-sacrifice. This is an incredibly hard thing to do and humans are historically very bad at doing all of these things. My worry is that AI systems are both better at coordinating than humans and also much, much faster moving than us. Worrying stuff.
Read more: The Rise and Fall of Agent Civilizations (Dwarkesh Podcast).
Read more: The Hugging Face attack surprised me (Planned Obsolescence).
***
New Five Eyes statement on AI:
…The greyworld power center turns its attention to AI…
Five Eyes, the name for the security and intelligence partnership between Australia, Canada, New Zealand, the UK, and the US, has published a statement as part of the recent “Five Country Ministerial” meeting. The statement is notable for including three paragraphs specifically about AI.
Five Eyes on AI - frontier model access: “We have committed to deepen collaboration with industry on shared national security priorities and public safety, including enabling timely access to frontier models to support secure innovation and strengthen cyber security,” the statement reads. “To support a coordinated response to artificial intelligence-related national security risks, the Five Countries discussed the national security and public safety implications of artificial intelligence models and characteristics of an artificial intelligence model that may require additional government scrutiny.”
Why this matters - from a foreseen risk to a live one: Previous Five Eyes ministerial statements have mentioned AI, but typically either as something to study, or something where they are concerned with its interplay with other areas of crime (e.g, malware, child pornography, scamming, etc). It’s very unusual for this year’s statement to have the practical focus of model access and it speaks to both the simmering geopolitical tensions around who does and doesn’t get access to this technology, as well as an acknowledgement that the intelligence services do not have their own in-house capabilities to make dependence on the private sector unnecessary.
Read more: Five Country Ministerial 2026 (Australian Government, Department of Home Affairs).
***
Bill Gates thinks the rise of AI will demand “an unprecedented global response”:
…Microsoft founder lays out a cautionary vision for the next few years…
By default, AI is not going to bring about happiness. That’s the basic conclusion from reading Bill Gates’s lengthy new essay about AI. The technologist and philanthropist worries that without massive work by governments, the outcomes of AI will not lead to a thriving society.
“This unprecedented technology demands an unprecedented global response. If we get it right, the payoff for humanity will be phenomenal and the world will be a more equitable place,” he writes. “In terms of equity, AI will eit
You can't make an omelette without breaking some urns
TL;DR In this post, we investigate two methods that aim to improve training efficiency over standard PPO for a grid world RL agent. In our experiments these methods can converge on better policies, but take roughly twice as long to do so. This project was done in the context of an ARENA 8.0 capstone project. Our code is available on GitHub.
Introduction
Reinforcement learning agents are trained to maximise a given reward function in their training environments. Common failure modes of the training process are reward hacking and goal misgeneralisation, roughly corresponding to outer and inner misalignment respectively.
Reward hacking occurs when the supplied reward function does not correspond exactly to the developer’s intent, and the agent learns a way to satisfy the reward function without satisfying the intended objective. This can be addressed, at least in principle, by designing a better reward function.
Goal misgeneralisation can occur even with a perfect reward function. If training and deployment environments differ, policies that satisfy the reward function during training, might fail to do so during deployment. Such policies are indistinguishable during training and we cannot control which one the agent learns.
If the distribution of deployment environments is entirely disjoint from the training distribution, goal misgeneralisation is inevitable. Consider, for example, training an agent to collect green apples while avoiding red boxes. What would happen if it gets deployed in an environment with red apples and green boxes? The behaviour is undefined. (It is also unclear what the intended behaviour would be in this case.)
Goal misgeneralisation is likely if deployment environments are not contained in the training distribution. But even if they are, correct generalisation cannot be guaranteed. In this post we present a case study and investigate two mitigations.
The pottery shop environment
Our object of study is the pottery shop environment.[1] It is a grid world environment containing the agent, a bin and one or more urns and (piles of) shards. The agent can pick up and carry one shard at a time. Its task is to collect all shards and drop them into the bin as quickly as possible. If it steps on an urn, the urn breaks and creates a new pile of shards to clean up.
An example level of the pottery shop environment (on a grid)
Interesting behaviour occurs when urns are on the direct path between the agent and the shards or the bin. The intended optimal policy is to walk around the urns if that path is only slightly longer. But if the detour gets too long, the agent should break some urns, creating a shortcut.[2] The cost of breaking the urns is lower than the cost of taking the detour. For computational efficiency, we limit our investigations to environments of size and .
We use the following reward function, which ensures that this is indeed the optimal policy.
- Dropping a shard in the bin gives reward +1
- Breaking an urn gives a penalty of -3
- We add a reward shaping term , where and are the states before and after the action. And the potential is 1 if the agent is holding a shard and 0 otherwise. This term encourages the agent to pick up shards as an instrumental goal without incentivising reward hacking by repeatedly picking up and dropping the shard.
- Finally, we add a step cost that is incurred at every action, as long as there are still shards to clean up. This is the crucial term that makes shortcut creation optimal by penalising long detours. Note that the discount factor alone does not penalise detours sufficiently. We set the step penalty to 0.2 and 0.02 in and environments respectively.
We use a discounting factor .
Our training environments are randomly generated as follows. The number of shards and urns is sampled randomly from a geometric distribution. The agent, urns and shards are then randomly placed on the grid. Sampling from a geometric distribution ensures that any number of urns and shards is possible, but higher numbers (i.e. dense levels) are less common. Every possible environment is thus accessible during training, but the distribution is heavily biased towards sparse levels. In particular, this means that it is rare to generate levels with long walls of urns, i.e. those where the optimal policy is to create a shortcut. Given the total length of training, the agent will nevertheless encounter a significant amount of them. The deployment distribution is a small set of hand-crafted levels, containing urn walls of various lengths. Across all environments, we fix the bin position to the top left corner.
We expect the agent to learn to always walk around walls, because that is the optimal policy in almost all of the training environments. In the deployment distribution, it should behave competently, but miss out on the opportunity to break walls and achieve optimal reward.
PPO to the rescue
We start with standard proximal policy optimisation (PPO). PPO training consists of two phases:
- Rollout phase. We generate random levels, rollout the policy (without updating weights) and collect transitions into memory.
- Training phase. We then train on batches of transitions and update the agent’s weights.
After that,
The Missing Product Layers in AI Safety
Epistemic Status: We’re highly confident in how the larger tech industry defines roles, makes decisions and builds products based on our years of experience in it. We’re fairly confident in the frontier labs operational models based on their job postings. Our suggestions for what to build and how to have gone through some pressure testing, and confidence could be raised through discussion with leaders in AI Safety to further understand what makes this space unique.
Intro
Frontier labs have adopted proven methodologies from tech to build their capabilities using matrixed organizations, including builders. This allows them to advance at a rapid pace, keeping their flywheel spinning, while AI safety struggles to keep up.
To prevent serious risks of advanced AI, AI safety needs to close that gap and one way to accomplish that is by building products that address common needs and diffuse AI safety outputs to compound their impact. There are aligned people that have extensive experience building products, but are not currently finding opportunities in existing organizations or through founder support to build them.
We (the authors and collaborators) each have 10+ years of experience at successful tech startups and in big tech. At EAGxBerkeley, we compared notes on gaps we were observing, our hypotheses on addressing them and wanted to share our perspectives with the broader community.
We’re writing this post to generate awareness, discussion, and ideally to open up discussion with safety leaders, fieldbuilders and funders on how to start building.
Background
An AI safety output spans research, controls, defenses, governance, policy and more. The pace of outputs being generated continues to increase, but there is limited work to connect them back up the chain to the highest level impact of avoiding serious risks to humans. These outputs accumulate in a patchwork of papers, forum posts, blogs, GitHub repositories and more where they rely on a highly-engaged community to continuously discover, consume and adopt. It assumes that the best ideas will win out and gain traction, but provides no formal pathway to achieve that.
We observe a field primarily organized as many small research teams producing many small, loosely coupled artifacts, but lacking layers above those efforts to bring the outputs back together so that they can be quickly applied and adopted by the broader community. For example in the evals tooling landscape Inspect and Petri exist as frameworks but no layer sits above them to standardize the techniques, processes, data and connect all phases of evaluations to ensure that trustworthy results and reporting are generated.
Coefficient Giving’s Fall 2026 Project Tailwind highlights this in multiple areas, such as this challenge statement for their proposed AI uplift for AI Safety initiative:
“There is a coordination failure here: investing in tools and best practices for using AI to accelerate AI safety and governance is a public good that individual contributors often lack capacity for. The individuals already doing this work have limited resources and can’t serve the entire field.”
Threat model
We believe that AI is not normal technology and poses serious risks to humans during both development and deployment across misalignment, misuse, loss of control, and systemic and structural harm. Across all these vectors products and tools play a role in harm reduction (in addition to research and governance). There is awareness of the importance of products [goodgravy 2022, Hobbhahn 2026] and tools [Jones 2021, BlueDot 2025], yet broadly AI safety doesn’t clearly differentiate between the two currently, nor does it robustly support creating products.
Examples of gaps observed in AI safety products and tools
Below is a limited sample of the types of needs we’ve observed across AI safety with concrete references.
Risk vector and Jobs to Be Done | Exists | Missing |
|---|---|---|
Misalignment: know when a model has the wrong objective and why | Products: Neuronpedia Tools: Petri, Bloom, TransformerLens | Access to frontier models beyond black-box [Casper 2024] |
Misuse: |
Meet TokenEconomizer: Microsoft’s Plugin That Cuts Coding Agent Costs by Up to 30.9%
A team of researchers from Microsoft Research and UC Berkeley have released TokenEconomizer, an open-source cost controller for coding agents. TokenEconomizer is a plugin that learns from past sessions in a repository to cut waste in later ones. It works with Claude Code, Codex, GitHub Copilot CLI and Pi. It does not touch model weights or agent code. On 277 SWE-Bench tasks, it lowered mean cost by 18.4% to 30.9% across 5 agent-model settings.
TL;DR
- Package: A standard-library Python package and agent plugins, with 3 switchable mechanisms.
- Runs on: Any machine running Claude Code, Codex, Copilot CLI or Pi with Python 3.10+. No GPU and no API key of its own.
- Performance: 18.4% to 30.9% lower mean SWE-Bench cost, and cheaper in all 30 repository-setting cells.
- Best: 30.9% lower cost on Claude Code with Opus 4.8 (33.2% when combined with RTK).
- Worst: Only 6.4% to 9.3% savings on ReposVul audits; Pass@1 drops 1.0 point on Opus 4.8.
Why do coding agents waste tokens?
Every agent turn re-bills the system prompt, every loaded tool schema and the full history. The research team profiled 60 SWE-Bench tasks across 6 repositories using Claude Code with Opus 4.8. They found 3 recurring sources of waste:
- Repeated discovery: once 5 tasks precede the current one, 30% of tool calls repeat an earlier action.
- Unused tools: 23 tool schemas load every turn. 16 are never called, and 3 cover 98% of calls.
- Failure loops: 10 failed runs were 17% of tasks but 32% of total spend.
They also tested 3 fixed open-source strategies: RTK, Serena and Aider’s RepoMap. None was cheaper everywhere. All 3 cut cost on Django and raised it on SymPy.
How does TokenEconomizer work?
TokenEconomizer is an external controller that treats the agent as a black box. It keeps a state per repository, built from completed traces. Before each task, it sets 3 policies through the prompt, the tool settings and the event stream.
1. Micro-skills
An LLM proposer turns trace digests into 1-line facts, such as the working test command. Each fact carries regex detectors that check later traces without a model call. A fact enters the prompt after 3 confirmations on tasks with non-overlapping edited files. Its need rate must also stay at or above 0.4. One contradiction counts as 5 confirmations, so wrong facts retire quickly.
2. Tool pruning
Shell, read, write and edit tools are always kept. Other tools are masked after 3 consecutive unused tasks, or when an LLM judge marks them irrelevant. The full catalog returns every 5 tasks for a re-audit. On Django with Claude Code, the first request on task 2 fell from 43,351 to 16,295 prompt tokens.
3. Failure-loop intervention
The controller learns a turn budget: 1.25 times the 90th percentile of successful run lengths. Cheap rule checks flag repeated failures or dense errors. An LLM judge then decides if the agent is stuck, at 0.7 confidence or higher. A stuck agent gets 1 redirect, then a stop if the loop recurs.
What do the SWE-Bench results show?
| Setting | Cost vs default | Pass@1 default | Pass@1 TokenEconomizer |
|---|---|---|---|
| Claude Code + Sonnet 5 | -23.6% | 74.7 | 76.0 |
| Claude Code + Opus 4.8 | -30.9% | 74.0 | 73.0 |
| Codex + GPT-5.6 Terra | -24.7% | 73.8 | 74.7 |
| Codex + GPT-5.6 Sol | -26.1% | 80.1 | 81.7 |
| Copilot CLI + Sonnet 5 | -18.4% | 75.0 | 77.6 |
Source: arXiv 2610.22732, Table 1. Costs include all controller calls. 3 seeds per setting.
Pass@1 improved in 4 of 5 settings. Lower reasoning effort saved cost but cut Pass@1 by 1.8 to 10.3 points. RTK alone raised cost in 12 of 30 repository-setting cells, and RepoMap in 15.
Savings build over time. Most repositories cost more at first, but all were cheaper by the sixth task. Alone on Claude Code, tool pruning saved 13.2% to 17.7%, micro-skills 11.8% to 14.3%, and the loop guard 5.4% to 7.1%.
Results also held outside SWE-Bench. Costs fell 16.0% to 21.2% on Web-Bench and 16.5% to 19.6% on CredData. ReposVul saw only 6.4% to 9.3%, because shorter runs leave less to remove.
How does it compare with other token-saving tools?
| Feature | TokenEconomizer | RTK | Serena | Aider RepoMap |
|---|---|---|---|---|
| Approach | Learns per-repository policies from past sessions | Compresses shell command output | MCP server with symbol-level code tools | Ranked map of repository symbols in the prompt |
| Learns from earlier tasks | Yes | No | No | No |
| License | Apache 2.0 | Apache 2.0 CLI; BUSL-1.1 engine | GPL-3.0-or-later (MIT for SolidLSP) | Apache 2.0 |
| Agents | Claude Code, Codex, Copilot CLI, Pi | 18 AI coding tools | Any MCP client | Aider |
| Mean SWE-Bench cost change (paper, 5 settings) | -18.4% to -30.9% | -8.9% to +4.5% | Not in main runs (raised pooled cost in profiling) | -6.1% to +6.5% |
| Cells with higher cost (of 30) | 0 | 12 | Not disclosed | 15 |
Cost rows: TokenEconomizer paper. Licenses and claims: each project’s repository or docs.
How do you install TokenEconomizer?
The plugin needs one of the 4 agents and Python 3.10 or newer. On Claude Code, clone the repo, add it as a marketplace, then run claude plugin install tokeneconomizer@tokeneconomizer. Learned data stays local in ~/.tokeneconomizer. Tool p
Vercel Labs Open-Sources agent-desktop for Agent Computer Use
Vercel Labs has released agent-desktop, an open-source CLI for AI agent computer use on macOS, Windows and Linux. agent-desktop reads each app through the operating system’s accessibility tree, not through screenshots. An agent gets a JSON snapshot with a ref for every element it can act on. Version 1.0.2 shipped on October 30, 2026, under Apache-2.0. For developers, any shell-capable coding agent can now drive native apps without a vision model.
TL;DR
- Library: Rust CLI and a C library. 60 command names, 56 operational.
- Runs on: macOS 13+, Windows 10 1809+ or Server 2019+, Ubuntu 24.04+, on x64 and ARM64. Plain CPU, no GPU.
- Performance: Deterministic, retry-safe ref actions. No independent computer-use benchmark yet.
- Best: 78% to 96% fewer tokens on dense Electron apps via skeleton traversal (self-reported).
What is agent-desktop?
agent-desktop is a Rust CLI and C library from Vercel Labs for reading and operating desktop apps. It is a tool that agents call, not an agent. It embeds no LLM and has no GUI. Every command prints one JSON envelope and exits. The observe, act and verify loop stays in your agent.
How does agent-desktop work?
Each command flows through a fixed chain. The CLI or C library calls agent-desktop-core, which reaches the OS only through a PlatformAdapter trait. One platform crate then talks to the native accessibility API:
- macOS: AX API
- Windows: UI Automation
- Linux: AT-SPI
Core never imports a platform crate. Each binary is compiled to drive exactly one OS.
Snapshots and refs
A snapshot assigns qualified refs such as @s8f3k2p9:e8. The snapshot ID inside the ref makes it self-describing. A bare @e8 is rejected with INVALID_ARGS. When the UI changes, an action returns STALE_REF or AMBIGUOUS_TARGET. The agent then takes a fresh snapshot.
Every error carries a disposition. Agents should retry only when it says safe. Ref resolution is strict and fails closed rather than guessing.
Skeleton and drill-down
Dense apps can expose thousands of accessibility nodes. A --skeleton snapshot reads only 3 levels deep. Cut containers report a children_count and get a drill ref. snapshot --root then reads one region into the same snapshot.
The builder reported a 78% to 96% token cut versus full-tree dumps in Slack, VS Code and Notion. In Apple Maps, he reported 3,811 tokens per look falling to 273. A simulated 52-step task dropped from 198k to 31k tokens. These are self-reported figures, not an independent benchmark.
Headless by default
Ref actions go through the accessibility API without moving the pointer or taking focus. That is what lets agents work while you keep using the machine. The global --headed flag allows physical input when needed. Every ref action also auto-waits up to 5,000 ms for a visible, enabled, stable target.
DeepSeek Open-Sources dsh-superagent, a Router-Worker Coding Agent
The DeepSeek Harness team just released dsh-superagent, an MIT-licensed research prototype. DeepSeek dsh-superagent is a proactive coding agent for one GitHub repository. It is packaged as a bundle for DeepSeek Harness. It listens to GitHub and, optionally, Feishu. It turns allowed requests into parallel work items, each in its own agent session. The release shipped alongside a design post, Exploring Super-agent Design: Towards a More Autonomous Agent System.
TL;DaR
- What is it?: A TypeScript bundle, v0.1.0, released as 1 commit under MIT. It uses whatever model the Harness profile is set to.
- Runs on: Your own machine, as a dedicated DeepSeek Harness profile. Needs Node.js 22.19+ or 24+, git 2.32+, Python 3.8+, tmux and GitHub CLI 2.48+.
- Performance: In DeepSeek’s simulation, it finished tasks far faster under a noisy message stream than a single-session agent.
- Best: p95 task completion of 15.7 minutes vs 29.5 minutes for the single-session baseline (about 1.9x faster).
- Worst: Tested only on synthetic messages and simulated tools. Real coding quality was not measured.
What is DeepSeek dsh-superagent?
It is a GitHub agent that waits to be asked. When an allowed person addresses it, it opens a work item, called a Ticket. Each Ticket runs in its own session and directory. Results return to the original conversation, often as a pull request. A local web panel lists only what needs a person.
The README page states the core rule plainly: take as little human attention as possible, and give as much value as possible. The philosophy doc applies it to behavior. The agent confirms once, stays quiet while working, and writes again only with a result, a question or a failure.
Why did DeepSeek build a Super-agent?
DeepSeek team argues that mainstream agents treat 1 session as the basic unit. According to the report, as reported by ifeng, a person can actively follow only 3 to 5 sessions at once. Human attention, not model capability, becomes the bottleneck for multitasking.
How does the Router-Worker architecture work?
The research team describes 2 layers. A long-running Router handles interaction and dispatch. Isolated Worker sessions do the actual work.
In the prototype, that flow is concrete, per the domain model:
- Source → Trigger: GitHub or Feishu produces events.
- Eval: decides whether a batch of Triggers asks for new work.
- Ticket: a unit of authorized work, with an immutable Mandate stating why it may proceed.
- Subscription: later comments on a pull request reach the right Worker without a fresh mention.
The main session runs the dsh-superagent-router preset with a smaller tool set. Each Worker runs dsh-superagent-worker. Shell commands move to the background after 5 seconds, so Workers keep receiving messages, per the execution doc.
Proactive work is gated. The authorization doc defines 3 accept modes: off, observe (default) and auto. In observe, the agent only records what it would have accepted.
The research team also lists ideas beyond the prototype: an external Watchdog, storage and compute separation, and an asynchronous ReAct loop.
How does it perform?
DeepSeek tested with simulated group chat and GitHub CI streams. Each run had about 300 messages, of which about 20 were real work requests. Each scheme ran 5 times, per ifeng’s account of the report.
- Super-agent: p50 of 12.4 minutes, p95 of 15.7 minutes.
- Single session plus subagents, neutral prompt: p95 of 29.5 minutes. No p50 was given.
- Why: under the neutral prompt, the baseline called a subagent 0 times in 5 runs.
- Router load: as arrivals rose from about 1 to 40 per minute, batch size grew from 1.0 to 2.8. Per-request latency rose from 2.5 to 3.9 seconds.
An ablation removed direct event pass-through. Router decisions rose about 30%, and cache-read tokens rose about 37%, from 23.23M to 31.77M. All figures are self-reported and come from secondary reports of the report.
How does dsh-superagent compare with other GitHub coding agents?
| Feature | DeepSeek dsh-superagent | GitHub Copilot cloud agent | Claude Code Action | OpenAI Codex |
|---|---|---|---|---|
| Trigger | Mention by allowed person; written rules | Issue assignment, chat, API, Slack and more | @claude mention, issue assignment, workflow prompt | @Codex in Slack; cloud tasks |
| Unasked (proactive) work | Yes, rule-gated; observe by default | Yes, via automations | Yes, via workflow events | Not disclosed |
| Parallel isolated sessions | Yes, 1 per Ticket | Not disclosed | Not disclosed | Not disclosed |
| Where it runs | Your machine (Harness profile) | GitHub, in the background | Your GitHub Actions runner | Codex cloud |
| Chat source | Feishu (optional) | Slack, Teams | Not disclosed | Slack |
| License | MIT | Not disclosed | MIT | CLI agent open source |
| Availabil |