德国学者点赞DeepSeek：德国近50%中小企业都用中国开源大模型 - 驱动之家
Google News 中国大模型德国学者点赞DeepSeek：德国近50%中小企业都用中国开源大模型 驱动之家
I tested whether changing who an argument is attributed to changes how LLMs evaluate the exact same argument [R]
Reddit r/MachineLearning submitted by /u/chronosAndCode
[link] [comments]
I’m the author of a new preprint looking at a fairly simple question: if the argument itself stays the same, does changing the claimed source change how an LLM evaluates it?
I tested 20 arguments across 10 topics under four source conditions:
- relevant expert
- irrelevant expert
- non-expert
- no source
The arguments were evaluated by GPT-5.6 Sol, Claude Opus 5.5, Gemini 4 Argon, and DeepSeek Flash across four dimensions: logic, credibility, persuasiveness, and agreement.
A few findings stood out:
Credibility was by far the most source-sensitive dimension. Source attribution affected credibility much more consistently than logic or agreement.
More interestingly, the effect did not usually look like a simple “expert bonus.” In several models, the stronger pattern was an irrelevant-expertise penalty: attributing an argument to someone presented as an expert in an unrelated domain could reduce its credibility relative to giving no source at all.
Agreement was comparatively resistant to source attribution.
I also tested whether source effects depended on argument strength. Those interaction results were not statistically robust after correction, and the human validation of strong vs. weak arguments was imperfect, so I treat that part of the study as exploratory rather than a strong conclusion.
The preprint is not yet peer reviewed, so I’m especially interested in criticism of the design and interpretation.
Preprint:
https://www.researchsquare.com/article/rs-11544999/v1
One question I’d be interested in hearing opinions on:
Should source sensitivity in an LLM evaluator be considered a bias when source expertise is genuinely relevant, or is the real problem specifically when the model overweights irrelevant authority?
[link] [comments]
AgentHop: A Diagnostic Benchmark for Agentic Multi-Hop Question Answering over Scientific Citation Graphs
arXiv cs.AIarXiv:2609.34428v2 Announce Type: replace-cross
Abstract: Agentic tasks require a large language model to interact with the world, navigating information and gathering evidence across multiple steps with restricted resources. Due to this complexity, agentic task failures arise from various sources, and pinpointing these failure causes is essential to diagnose and improve agentic systems. Existing benchmarks, however, tend to focus on a single leaderboard score, leaving the underlying failure modes opaque. To fill this gap, we introduce AgentHop, a diagnostic benchmark of 1,011 multiple-choice questions paired with a controlled seven-tool sandbox under fixed token, turn, and tool-call constraints. AgentHop reveals model vulnerabilities by dissecting a single accuracy score along four axes of agent operation: retrieval, synthesis, tool-call, and resource management. Across 19 models, we find that behavior clusters by model family, with tool-call signatures revealing distinct family fingerprints: GPT models commit early, Anthropic and GLM checkpoints verify before committing, DeepSeek and Kimi over-search, and Gemini-3 Pro stays balanced. Decomposed axes further expose within-family structure: Claude Opus 4.6 and Sonnet 4.6 land within one accuracy point yet diverge on retrieval-versus-synthesis emphasis, with Opus retrieving more and Sonnet synthesizing better. We release the full benchmark set and the harness to support diagnostic agent benchmarking.
LEAD: Length-Efficient Adaptive and Dynamic Reasoning for Large Language Models
arXiv cs.AIarXiv:2605.09806v2 Announce Type: replace-cross
Abstract: Large reasoning models, such as OpenAI o1 and DeepSeek-R1, tend to become increasingly verbose as their reasoning capabilities improve. These inflated Chain-of-Thought (CoT) trajectories often exceed what the underlying problems require, wasting compute, latency, and context budgets. While introducing length-based efficiency rewards during reinforcement learning offers a natural remedy, existing methods struggle with two fundamental challenges: the optimal balance between correctness and efficiency is non-stationary throughout training, and intrinsic reasoning budgets vary drastically across problems. Relying on static reward weights and global length constraints inevitably forces a compromise between degraded accuracy and unrealized compression. To overcome these limitations, we propose LEAD (Length-Efficient Adaptive and Dynamic reasoning), a method that replaces static heuristics with online, self-adaptive mechanisms. LEAD dynamically calibrates the correctness--efficiency trade-off at each step using a Potential-Scaled Instability, directing optimization capacity to the most informative learning signal. Furthermore, it estimates an adaptive per-problem target length online based on the model's own correct rollouts, applying a symmetric efficiency reward that penalizes both overthinking and over-compression. Evaluated on five mathematical reasoning benchmarks, LEAD achieves the highest accuracy and Accuracy-Efficiency Score among RL-trained efficient-reasoning methods in our matched comparisons while producing substantially shorter outputs than the base model.
AI-Enabled Question-Answering for Chartered Accountancy: A Retrieval-Augmented Reasoning Framework
arXiv cs.AIarXiv:2605.00257v2 Announce Type: replace-cross
Abstract: The inception of Large Language Models (LLMs) has catalyzed AI adoption in the finance sector, yet their reliability in complex, jurisdiction-specific tasks like Indian Chartered Accountancy (CA) remains limited. The models display difficulty in executing numerical tasks which require multiple steps while also needing advanced knowledge about legal regulations and the method of scaling their operations is not feasible in settings which have limited access to resources. We present CA-ThinkFlow as a parameter-efficient Retrieval-Augmented Generation (RAG) framework which operates with a 14B, 4-bit-quantized reasoning model, 14B-DeepSeek-R1, and a layout-aware Docling extraction system which maintains document structure during extraction. CA-ThinkFlow uses a basic RAG method which automatically adds retrieved information into the prompt, while it depends on the model's built-in Chain-of-Thought (CoT) functions to create context and produce correct answers. The system we developed system operates at performance levels which match large proprietary models when we tested it on the multi-level CA-Ben benchmark, achieving Scholastic Reliability Coefficient (SRC) results which equal 68.75\% of GPT-4o and Claude 3.5 Sonnet. The framework shows high efficiency and strength in handling parameters, but essential reasoning abilities fail to process complex regulatory texts which exist in fields such as Taxation.
IndicDBBench - Benchmarking Multilingual Text-to-SQL Capabilities in Indian Languages
arXiv cs.AIarXiv:2604.13686v2 Announce Type: replace-cross
Abstract: While Large Language Models (LLMs) have significantly advanced Text-to-SQL performance, existing benchmarks largely focus on Western contexts leaving a gap in real-world, non-Western applications. We introduce IndicDBBench, a comprehensive multilingual Text-to-SQL benchmark designed to evaluate cross-lingual semantic parsing across diverse Indic language families. The underlying schemas are derived from real-world open-data platforms (NDAP and IDP), ensuring realistic structural complexity. IndicDBBench comprises 20 databases spanning 237 tables. To construct complex relational structures from denormalized government data, we employ a three-agent pipeline (Architect, Auditor, Refiner), achieving high relational density (11.85 tables per database) and join depths of up to six. We generate 15,612 tasks spanning seven languages: English and six Indic languages, ensuring that queries are grounded in real database values and require multi-table reasoning. We evaluate state-of-the-art models, including DeepSeek V3.2, MiniMax M2.7, Llama 3.3, and Qwen3, across seven linguistic variants. Our results reveal a 9.0-point drop in execution accuracy from English to Indic languages, exposing an "Indic Gap" that, for the two models tested, persists even when gold tables and literal values are supplied, since this grounding raises accuracy in every language by similar amounts. The dataset and code are publicly available at https://anonymous.4open.science/r/multilingualText2Sql-Indic--DDCC/
Can Coding Agents Sustain Continual Evolution without Forgetting?
arXiv cs.AIarXiv:2611.03219v1 Announce Type: cross
Abstract: Recently, harness self-evolution has emerged as an important paradigm for improving agent performance. Prior studies show that evolving the agent harness can effectively enhance agent capabilities without modifying the foundation model's parameters. We find that existing approaches largely study agent evolution on a single task. In practice, coding agents may need to handle multiple task types over time. Whether an agent harness can continuously evolve across multiple tasks without suffering from catastrophic forgetting remains an important open question.
In this paper, we investigate this open question through a comprehensive empirical study of continual harness self-evolution. We organize three software engineering task datasets into six different evolution directions and apply two representative harness evolution methods to perform continual evolution along these directions. Our experimental results show that evolution on a subsequent dataset can substantially undermine the performance on previously seen datasets. For example, with DeepSeek-V4-Flash-0731, Meta-Harness improves its performance on SWE-Bench to 94.0% after evolution. However, after further evolution on FeatureBench, its performance on SWE-Bench drops to 32.0% by the fifth round, corresponding to a 66% relative decrease. Finally, based on our experimental findings, we identify several insights, providing directions for the continual self-evolution of agent harnesses. Code is available at https://github.com/THU-Agent/continual-harness-evolution.
Reimagining computational quantum research with AI scientists
arXiv cs.AIarXiv:2611.02833v1 Announce Type: cross
Abstract: AI scientists are expanding automated research across scientific domains, yet their ability to turn the scientific record into a foundation for autonomous discovery remains elusive. Here we uncover a fundamental barrier to this transition and demonstrate a route to overcome it in computational quantum science, where advances shape both what we can learn from complex quantum systems and what quantum computing can ultimately deliver. To identify this barrier, we introduce OmniQ, an expert-audited virtual research environment that converts published studies into executable reproduction tasks. Across 59 papers spanning classical simulation, quantum algorithms and quantum-property characterization, we systematically evaluate five agentic systems powered by large language models and six third-party agent harnesses. Our findings reveal that current AI scientists often fail to reproduce published results, exposing a fundamental mismatch between what scientific papers communicate and what autonomous agents require. By contrast, human expert intervention can turn these failures into successful reproductions, showing that the missing information can be identified and converted into reusable research experience. Guided by this observation, we develop Residual Agents (RA), a quantum-tailored self-improving AI scientist that distils externally verified workflow repairs into such experience. RA substantially improves research performance under fixed base models and limited adaptation budgets. For example, RA raises DeepSeek V4.1 Flash performance from 68% to 85%, matching Claude Code with Opus 5 at a third of its cost despite using a weaker base model. We integrate all reproduced results into an extensible platform built on the model context protocol, making verified research workflows directly accessible to AI scientists and allowing new scientific papers to be continuously incorporated.
Precedent Labels and Facts Shift LLM Approval Decisions in Opposite Directions
arXiv cs.AIarXiv:2611.00672v1 Announce Type: cross
Abstract: When a language model decides refund or claim requests under written policies, the system can show it similar past cases. Each past case has two parts: its facts and the decision recorded for it. We separate the two. A model sees a policy, a request and three past cases, and we change either the recorded decisions or the facts, keeping the request fixed. We select requests that are close calls: deciding without past cases, DeepSeek V4.1 Flash approved each in some of four tries and denied it in others. When the past cases are near-copies of the request (its facts with a few details changed), switching their recorded decisions from DENY to APPROVE raises approval by about 30 to 55 percentage points in DeepSeek V4.1 Flash, GPT-5.6 Luna and Claude Haiku 5.5 on policies written for the study, and DeepSeek and Luna show the same on nine public policies paraphrased from real ones. The facts push the other way: shown without their recorded decisions, past cases whose facts support approval make these two models approve less often than past cases whose facts support denial. One reading is that the request looks weaker next to them. This push is not just a correction toward the right answer: in an additional DeepSeek analysis, it was of similar estimated size on requests that human annotators judged approvable and on those they judged deniable. In a separate DeepSeek run on the public policies, when the past cases are near-copies of a different request, switching their decisions moves approval by 9 points instead of 43, while facts that support approval still lower it by 22 points against facts that support denial. Systems that retrieve past decisions should test the two parts separately.
CoworkOS:Scaling Agent Coworkers for Real-world Automation
arXiv cs.AIarXiv:2611.03178v1 Announce Type: new
Abstract: Rapid advances in cowork agents are shifting human-AI collaboration toward real-world automation, delivering software, designs, and other artifacts. However, such productivity work demands both high-quality delivery on complex productivity tasks and continued monitoring and decision-making over long horizons, a combination that remains an open challenge. We introduce CoworkOS, a general-purpose harness for long-horizon work that separates task strategy from execution management: the model decides actions, delegation, and acceptance, while the runtime maintains state and feedback. Its three-layer architecture enables direct execution or delegation to Subagents, synchronously or asynchronously, under stable responsibilities. Through a shared application layer, humans and agents inspect and revise the same evolving documents, presentations, CAD models, and Blender scenes, so human-in-the-loop participation can enter both through conversation and through direct modification of the work itself. To sustain continuity across agents and sessions, CoworkOS further introduces hierarchical workspaces and persistent semantic memory, so working files, key decisions, and long-running Job logs remain shareable, reusable, and resumable after interruption. Across 12 benchmarks and 6 categories of real-world applications, CoworkOS proves highly competitive with frontier agent applications such as Codex and DeepSeek Harness. As a sustained-work case study, CoworkOS uses GLM-5.3 as its backbone to conduct RSI-driven automated post-training on Qwen3.6-35B-A3B. Over five rounds, the model's average score across eight benchmarks rose from 33.19 to 42.95 (+9.76). Looking ahead, we believe CoworkOS opens a meaningful path toward general cowork-agent intelligence.
AttentionIndex: Accelerating Learned Sparse Attention with a Causal Inverted-File Index
arXiv cs.AIarXiv:2611.02508v1 Announce Type: new
Abstract: Learned sparse attention, used in production models such as DeepSeek-V4 and GLM-5, lets each query attend to only k keys chosen by a lightweight indexer, but the indexer still scores every key in the prefix. At one million tokens, this selection takes more than half of prefill time. Two-stage selection over contiguous blocks, as in HiSA, reduces this cost, but its recall declines as context grows at a fixed candidate budget. We present AttentionIndex, which routes each query over an inverted-file index that groups indexer keys by similarity and refines the candidates with the native indexer score. AttentionIndex builds the index during prefill, making new keys searchable immediately, reusing previous clustering to keep maintenance cheap, and exposing each query only to clusters built from its own prefix. On GLM-5.3 and DeepSeek-V4-Flash at one million tokens, AttentionIndex delivers 1.3-1.8 times the prefill throughput of exhaustive selection, and at throughput within 5% of HiSA, it recovers 17.0 and 10.0 percentage points more of the exhaustive top-k, respectively. AttentionIndex also improves accuracy on long-context benchmarks by up to 18.4 percentage points over HiSA at the same candidate budget.
\textsc{ChatDify}: Automating End-to-End Dify Workflow Synthesis from Natural Language via Multi-Agent Collaboration
arXiv cs.AIarXiv:2611.01563v1 Announce Type: new
Abstract: The proliferation of Large Language Models has catalyzed powerful application development platforms like Dify, which enable users to build sophisticated LLM-powered workflows through modular node orchestration. However, manually configuring these workflows requires deep platform expertise and remains time-consuming and error-prone, creating a significant barrier for non-expert users. To address this challenge, we propose \textsc{ChatDify}, a novel multi-agent framework that automates end-to-end Dify workflow synthesis from natural language descriptions. \textsc{ChatDify} employs an Actor-Critic architecture comprising three specialized agents: the \textit{Planner Agent} that decomposes user requests into structured execution plans, the \textit{Executor Agent} that performs context-aware parameter synthesis and schema-augmented DSL compilation, and the \textit{Reviewer Agent} that ensures reliability through hybrid verification with closed-loop refinement. We evaluate \textsc{ChatDify} on a curated benchmark of 100 production-inspired tasks across three state-of-the-art LLM backends, achieving 0.96, 0.91, and 0.75 success rates with Qwen-3.7, GLM-5.1, and DeepSeek-v4, respectively. These results demonstrate its robustness and practical applicability for lowering the technical barrier to creating sophisticated LLM applications.
Decision-Centric Research: A Calibrated Decision Model and Benchmark for Scientific Agents
arXiv cs.AIarXiv:2611.00774v1 Announce Type: new
Abstract: Scientific agents make consequential choices about evidence, next actions and research strategy, but choices embedded in generated text are difficult to evaluate independently. Decision-Centric Research (DCR) is introduced to make individual choices the common unit of supervision, prediction and evaluation through typed predictions over an explicit Research State. DCR defines 24 decision types, ten of which are instantiated in the Research Decision Benchmark (RD-Bench) of 2,000 rule-generated cases. The Decision-Centric Research Model (DCRM) shares a candidate-token readout across types to produce answers and option probabilities in one forward pass. Under a held-out research-domain split, a 0.6B DCRM (state-aware) achieves 89.4% Top-1 accuracy, compared with 27.8% for DCRM (question-only) trained with the same recipe and 36.4% for DeepSeek-V4.1-Flash. All 30 type-seed comparisons against DCRM (question-only) are significant after Holm correction. After held-out calibration, the resulting prediction sets contain a mean of 1.515 candidates at 97.1% empirical label coverage, reducing mean set size by 57.4% relative to DCRM (question-only) at similar coverage. These results establish a compact state-aware decision layer with accurate typed prediction and an inspectable uncertainty interface.
The Agent Said It Was Done. The Database Disagreed.
HuggingFace Blog
Welcome Le Chonk: Mistral Large 4 on Hugging Face
HuggingFace Blog
CWE-bench v2: More coverage, quality, and transparency
HuggingFace Blog
[问与答] 自己给电瓶车接线双电池，是否简单？
V2EX京东自营买了黄金锂电 7240
本来是包安装的，结果听说我原电池保留，安装师傅就不干了。
去了我买车的店里，老板找了线来准备弄，我一看线怎么这么细，一问，说是 1.5 平方的，闹麻了！
我说太细了，老板说给我用两根弄！！！
就停止安装了。
----
目前自己把材料买的差不多了，DeepSeek 和 Gemini 也都问的差不多了，主要如下：
最终线路配置与改线清单（照此施工即可）
线路位置 线规规格 改动操作
铅酸正极 ➔ 空开 63a 接口 1 原车 4 平方 （保持不动（拧入新空开左上角即可） ）
锂电正极 ➔ 空开 63a 接口 2 新换 6 平方硅胶软线 （压接冷压端子，接入新空开右上角 ）
空开下端 ➔ 控制器正极 新换 6 平方硅胶软线 （下端左右并接短线与输出线统一用 6 平方 ）
锂电负极 ➔ 控制器负极 新换 6 平方硅胶软线 （接入控制器负极（与原车铅酸负极共用接点））
铅酸负极 ➔ 控制器负极 原车 4 平方 （保持不动（在控制器负极处与锂电负极并接））
这个方案下来，看着自己动手很简单的
PS：车是苍穹电自，网上听说控制器最大 40a 。1200w 电机（峰值 2500w ），原车上了 7222 铅电池
本来是包安装的，结果听说我原电池保留，安装师傅就不干了。
去了我买车的店里，老板找了线来准备弄，我一看线怎么这么细，一问，说是 1.5 平方的，闹麻了！
我说太细了，老板说给我用两根弄！！！
就停止安装了。
----
目前自己把材料买的差不多了，DeepSeek 和 Gemini 也都问的差不多了，主要如下：
最终线路配置与改线清单（照此施工即可）
线路位置 线规规格 改动操作
铅酸正极 ➔ 空开 63a 接口 1 原车 4 平方 （保持不动（拧入新空开左上角即可） ）
锂电正极 ➔ 空开 63a 接口 2 新换 6 平方硅胶软线 （压接冷压端子，接入新空开右上角 ）
空开下端 ➔ 控制器正极 新换 6 平方硅胶软线 （下端左右并接短线与输出线统一用 6 平方 ）
锂电负极 ➔ 控制器负极 新换 6 平方硅胶软线 （接入控制器负极（与原车铅酸负极共用接点））
铅酸负极 ➔ 控制器负极 原车 4 平方 （保持不动（在控制器负极处与锂电负极并接））
这个方案下来，看着自己动手很简单的
PS：车是苍穹电自，网上听说控制器最大 40a 。1200w 电机（峰值 2500w ），原车上了 7222 铅电池
[分享创造] ✌️ 我让 AI 给我做了个 APP，电饭锅做早饭计算预约时间
V2EX
就是这阵子开始自己用电饭锅做早饭了，晚上会提前把玉米、鸡蛋、红薯、山药、芋头、土豆给电饭锅里，设置蒸制时间和预约时间（目标要多久之后做好）
因为我这个电饭锅不是智能的，烹饪时间和预约时间都得手动设置（我这个电饭锅设置的预约时间是包含了烹饪时间的，也还算清晰
但是呢，我的计算能力很差了，也想过打印一张表贴厨房，让 AI 设计了一下，感觉挺复杂的
前阵子是每次问小爱同学：“距离明天早上 7:40 有多久”，反应时间很慢
一开始尝试让 AI 写 APP 了，一开始的尝试是让 workbuddy 写了个 DroidScript 的脚本，能在桌面创建快捷方式形成一个应用，点进去能显示距离下一个 7:40 （能改）还有多久（精确到 10 分钟，下取整）
但是这个就没有小组件，每次要点进去，不够炫酷和方便
后来就考虑，不如直接做个 apk 得了，
就让 workbuddy 帮我下了 android 和 flutter 的环境，让 workbuddy 写了个小工具脚手架，添加了这个功能，同时也让它根据小米的规范，做了 deeplink （跟小爱说一些特别的说法，能直达到功能里，还不够顺溜），和桌面小工具
现在在晚上准备第二天的早饭就很优雅，只要把所有东西都洗了，放锅里，顺手掏出手机，滑到放了这个小工具的这一屏，喽一眼就可以很方便设置了，完全不用动脑口算，也不用等慢吞吞的超级小爱反应
然后 workbuddy 用 deepseek v4 flash 的审美是真的丑，我自己重新纸上画了一版，有空再骑着它重新做一下
继续滚动加载更多…