We're training a Figma-fluent editing model

2026-10-29T21:00:00.000Z · 大模型,算力芯片,AI应用,Google,Meta,阿里巴巴,快手,文生视频,Agent智能体,搜索RAG,扩散模型,MoE架构,强化学习,微调蒸馏,提示工程,模型安全对齐,端侧AI,招聘HR,榜单评测,开发者生态

NeMo-DCR: Bit-Exact Delta-Compressed Refit for Scalable Agentic RL at Trillion-Parameter Scale

arXiv cs.AIarXiv:2610.08430v2 Announce Type: replace-cross Abstract: Agentic reinforcement learning (RL) disaggregates training from rollout into different clusters, so each policy update must reach rollout clusters before the next batch: a full 1T checkpoint takes 87.5 min between two AWS regions, yet only 1-2% of BF16 elements change their stored values per step. Concurrent systems use this sparsity but reimplement placement rules, assemble full tensors, rebuild values arithmetically, or rely on a cross-cluster collective, and none fully recovers from mid-refit failures. We present NeMo-DCR (Delta-Compressed Refit), which sends only changes yet gives receivers the bits of a dense refit. NeMo-DCR describes changes in the Hugging Face canonical coordinates that both sides already share and leaves their placement to the serving runtime's native loader. It projects over 96% of MoE weight bytes directly and converts the rest, under 4%. It encodes changes as compressible XOR masks for over 96% of these bytes and as overwrites where XOR masks cannot apply, streams them over object storage or a relay tree without a cross-cluster collective, and updates receivers in place, recovering from failures with overwrite retries and a joint commit. With real updates, NeMo-DCR is $7\times$ faster than AReaL-DTE. In 3% and 5% stress cases, 30B-1T refits are 12-40$\times$ faster than a transport-only full-checkpoint reference over the same object storage, and would make a synchronous Kimi K2 RL step 1.4-1.5$\times$ faster at 1T.

2026-11-04 05:00:00 · 大模型,AI应用,开源,月之暗面,Agent智能体,搜索RAG,MoE架构,强化学习,招聘HR,论文

RaMP: Runtime-Aware Megakernel Polymorphism for Mixture-of-Experts

arXiv cs.AIarXiv:2604.26039v2 Announce Type: replace-cross Abstract: LLM serving systems autotune their GPU kernels offline by timing candidate kernel configurations, like tile sizes, at a range of batch sizes. This works when the number of tokens determines a kernel's work, as in dense layers. However, we show that it does not hold for Mixture-of-Experts (MoE) layers where the router decides how the tokens are split across experts and GPUs. We find that the best kernel configuration depends on this split, and not only on the number of tokens--under expert parallelism, choosing the configuration for each GPU's actual token distribution is up to 1.45x faster than the existing choice. The split is not known until the router runs, so it cannot be looked up in advance. Hence, we introduce RaMP, which chooses the kernel configuration at run time from the routing split itself. RaMP compiles top configurations into one polymorphic kernel, and a small cost model on the GPU picks among them in a few microseconds. On serving traffic its cost model was never fit on, the configurations RaMP's cost model chooses from the kernel's full configuration space are up to 1.45x faster than those vLLM's offline autotuner picks for the same kernel on batches of 128 tokens or more (1.27x over all batches), and within 3.5% of an exhaustive search that times every configuration. RaMP's kernel and choice together make the MoE layer up to 1.84x faster than vLLM's Triton MoE on batches of 128 tokens or more. In vLLM, under both tensor and expert parallelism, RaMP lowers time per output token by up to 1.11x over the fastest stock MoE backend.

2026-11-04 05:00:00 · 大模型,算力芯片,Google,MoE架构,榜单评测,论文

Mixture-of-Models: Distributed Inference over Heterogeneous LLMs as Model-Level Mixture-of-Experts

arXiv cs.AIarXiv:2601.16863v2 Announce Type: replace Abstract: Switch Transformers scale MoE model capacity by routing tokens in memory for one jointly trained model, requiring high-bandwidth interconnects, large memory, centralised ownership, and lack scalable composability. We propose Mixture-of-Models (MoM) architecture as Switch Transformer scaling to semantic level which composes inference across independently trained distributed LLMs, each run on its own GPU. The models reach joint consensus using N-Way Self-Evaluating Deliberation protocol: a combination of social-choice methods such as Delphi method and Quadratic Voting as well as Inference-Time scaling paradigm. We start with an empirical model that characterizes consensus as an inference-time trade-off between signal extraction and contextual noise accumulation, establishing a mathematical basis for convergence driven inference stopping strategies. We show experimental results where ensemble of smaller (<21B) models matches closely 120B MoE (GPT-OSS-120B) on AIME 2025 while larger ensemble of open-weight models reaches 90.0%, close to proprietary models at time of writing.

2026-11-04 05:00:00 · 大模型,算力芯片,OpenAI,Transformer,MoE架构,网络安全,榜单评测,论文

Sonar-MoE: In-Backward Expert Updates for Memory-Efficient Mixture-of-Experts Training

arXiv cs.AIarXiv:2611.03577v1 Announce Type: cross Abstract: Most parameters of a Mixture-of-Experts (MoE) model belong to its experts, and standard training keeps every expert gradient in GPU memory until the optimizer step. This memory grows with the number and size of the experts, so it rises as MoE models scale. We introduce Sonar-MoE, which updates each expert tensor during the backward pass and frees its gradient right away. It brings optimizer-in-backward from dense models to batched MoE expert tensors, so the GPU never holds more than one layer of expert gradients. When we fine-tune OLMoE-1B-7B and Granite-3.1-3B-A800M in bf16 with two expert-kernel backends, Sonar-MoE lowers peak memory by 19% to 22%. Its updates are bitwise identical to standard training without expert gradient clipping, and held-out loss stays within 0.003 of all baselines. Each step also takes 17% to 23% less time than standard training with global-norm clipping. Updating whole batched tensors rather than individual experts is what keeps Sonar-MoE fast, as it avoids a 20% to 72% slowdown while using the same memory. Sonar-MoE matches optimizer-in-backward over all parameters in memory and speed while still clipping the dense parameters.

2026-11-04 05:00:00 · 算力芯片,MoE架构,微调蒸馏,论文

Library of Layers (LoL): Adaptively Ordering Transformer Layers at Runtime

arXiv cs.AIarXiv:2611.03526v1 Announce Type: cross Abstract: Computer programmers chain together the same reusable computational primitives, for example, conditional statements or loops, but in different orders to implement different algorithms. In contrast, existing language model architectures always execute their layers in the same order. We introduce Library of Layers (LoL), an architecture that dynamically orders its layers during inference to solve the problem at hand. LoL has a single hyperparameter, the size of its global library of transformer blocks, that allows it to transition seamlessly between parameter reuse and sparsity, generalizing the parameter-efficiency and FLOPs-efficiency tradeoffs of dense, depth-recurrent, and mixture of experts architectures in one unified family. We show that LoL achieves competitive performance with all three above baselines when pretrained on text data. On synthetic sudoku and maze data, we observe that the model employs distinct layer orderings to implement different solution strategies, and we can intervene on routing to force a model to apply a specific solution strategy to a problem instance. In a multilingual natural language setting, we find that LoL chooses different layer order patterns based on language and script. Similarly, we train LoL on safety data and fit probes to its router outputs, finding that these probes distinguish safe from unsafe text with high AUROC. LoL provides a path toward a future in which models dynamically compose their own computational graph, enabling more efficient and controllable reasoning.

2026-11-04 05:00:00 · 算力芯片,Google,推理思考,Transformer,扩散模型,MoE架构,预训练,招聘HR,论文

VideoEditor-MoE: Scaling Complex Video Editing with Mixture of Experts

arXiv cs.AIarXiv:2611.03043v1 Announce Type: cross Abstract: Complex video editing requires understanding complex instructions while jointly fulfilling multiple editing requests. Meeting these demands through dense model scaling substantially increases computational cost. We introduce VideoEditor-MoE, the first open-source mixture-of-experts framework for video editing. Beyond sparse MLP-MoE layers, MoE-Attention expands attention capacity through routed key--value experts while preserving global interactions among video tokens. Initialized from a 5B video editing model, VideoEditor-MoE scales to 14B and 56B parameters, with approximately 8B parameters activated per token in both variants. Comparisons with existing methods and two complementary baselines, Explicit-MoE and MLP-MoE, assess its benefits over explicit expert guidance and MLP-only expert architectures. Further analyses demonstrate that all experts remain active and reveal that learned routing responds more strongly to sample content and diffusion noise levels than to human-defined editing categories. Spatial analysis further indicates sensitivity to object-level semantic features. We also identify decoupled weight decay as a strategy for stabilizing expert scaling.

2026-11-04 05:00:00 · Transformer,扩散模型,MoE架构,招聘HR,论文

VioTech: Frame-Level Violin Playing Technique Detection with Multi-Scale Mixture-of-Experts

arXiv cs.AIarXiv:2611.02680v1 Announce Type: cross Abstract: Playing technique is crucial for expressive violin performance, yet few violin technique detection models exist. Current systems mainly rely on note-level pipelines that first estimate note boundaries and then classify techniques. This is especially problematic for bowed string instruments such as the violin, where soft onsets and continuous bow-string interaction render note segmentation unreliable. Here, we propose VioTech, a frame-level violin technique detection framework that predicts techniques directly at 10 ms resolution without explicit note segmentation. To handle the large temporal-scale variation among techniques, VioTech combines a frozen MERT encoder with a multi-scale Mixture-of-Experts (MoE) architecture and task-specific routing for tonal, articulation, and legato-related predictions. We also introduce VioTech120, the first manually annotated dataset of violin playing techniques constructed from real solo performance recordings, as well as an accompanying note-labeling user interface. Results demonstrate that VioTech achieved consistent improvements over prior methods and significantly enhanced cross-dataset generalization, establishing a new state-of-the-art in violin technique-aware transcription.

2026-11-04 05:00:00 · MoE架构,论文

Scene-Adaptive Readout and Decoding in Diffusion Driving VLAs

arXiv cs.AIarXiv:2611.02076v1 Announce Type: cross Abstract: Diffusion driving vision-language-action models repeatedly execute the backbone at every denoising stage, incurring substantial inference cost, while it remains unclear whether all scenes require the same decoding computation. To distinguish when a trajectory becomes readable from how much further refinement it requires, we analyze trajectory readout across network depth and decoding progress. We identify a depth-decoding dissociation in which late-layer representations already support useful trajectory readout after the first forward sweep, whereas early-layer representations benefit more from continued decoding. We further find that both the preferred readout layer and decoding depth vary across scenes. Based on these observations, we propose a scene-adaptive readout and decoding framework. A scene-conditioned readout MoE adaptively weights and combines trajectory predictions from layer-specific late-layer representations after a single backbone sweep, while an adaptive decoding policy applies additional native refinement only when required by the scene. We evaluate our framework on two diffusion driving VLAs across the Waymo E2E and nuScenes benchmarks. Our method improves validation RFS by up to 0.56 while also improving test performance, and achieves an end-to-end speedup of up to $6.0\times$ on Waymo E2E, while consistently reducing trajectory error on nuScenes. These results demonstrate that scene-level readout and decoding allocation improves planning quality while avoiding unnecessary decoding. Our code is available at https://github.com/sjyu001/SARD

2026-11-04 05:00:00 · 自动驾驶,开源,Google,扩散模型,MoE架构,强化学习,模型评测,网络安全,论文

MatMoE: Matryoshka Mixture-of-Experts with Dynamic Mixed-Precision Quantization for Efficient Inference

arXiv cs.AIarXiv:2611.01704v1 Announce Type: cross Abstract: Mixture-of-Experts (MoE) architecture has been widely adopted in large language models (LLMs) since it reduces the computational cost through sparse expert activation. However, the massive parameter scale of MoE models hinders their deployment on resource-constrained edge devices. Although existing mixed-precision quantization reduces the model size, they are inherently static, fixing the bit-width of each expert across decoding steps. Such static schemes yield suboptimal performance as they overlook the dynamic variation in expert importance during decoding. Furthermore, they fail to adapt to the dynamic latency constraint driven by the fluctuating system utilization. To address these challenges, we propose MatMoE, a dynamic mixed-precision quantization framework for efficient MoE inference. MatMoE features: (1) a Matryoshka MoE PTQ algorithm that enables experts to operate flexibly at multiple precisions within a stringent memory budget; (2) a floor-and-compensation quantization method that eliminates redundant memory accesses in the Matryoshka representation; and (3) a dynamic bit-width allocation strategy that determines the precision for each expert based on their routing scores to satisfy the dynamic latency requirement. We implement our MatMoE system atop llama.cpp for both CPU and GPU platforms. Extensive evaluations across three modern MoE models and challenging benchmarks demonstrate that MatMoE achieves $4.0\% \sim 18.3\%$ higher accuracy than state-of-the-art static mixed-precision method. Compared to llama.cpp, MatMoE delivers $1.3\times \sim 3.7\times$ speedup on Raspberry Pi 5, and $1.2\times \sim 1.3\times$ speedup on NVIDIA RTX 4090 and Jetson AGX Orin.

2026-11-04 05:00:00 · 大模型,算力芯片,Meta,NVIDIA,MoE架构,微调蒸馏,模型评测,招聘HR,榜单评测,论文

SIDRec: End-to-End Semantic ID Representation and Modeling for Recommendation

arXiv cs.AIarXiv:2611.01136v1 Announce Type: cross Abstract: Semantic IDs (SIDs) have emerged as a promising approach for discriminative recommendation by encoding continuous item rep- resentations into compact discrete identifiers. However, existing SID-based methods face two fundamental limitations: semantic mis- match between content semantics and collaborative preferences, and objective mismatch between SID construction and downstream recommendation optimization. To address these challenges, we propose SIDRec, an end-to-end recommendation-aware SID learn- ing framework that jointly models content and collaborative in- formation while aligning SID construction with recommendation objectives. Specifically, SIDRec introduces a self-updating fusion mechanism that explicitly integrates pretrained content represen- tations with collaborative embeddings initialized from a trained recommendation model, enabling preference-aware semantic mod- eling. Furthermore, SIDRec employs an MMoE-inspired joint opti- mization mechanism with shared experts and task-specific gates, allowing recommendation supervision to directly guide differen- tiable quantization and produce task-adaptive discrete representa- tions. Extensive experiments on three public benchmark datasets demonstrate that SIDRec consistently outperforms all competing methods across both AUC and GAUC, achieving up to 5.15% relative improvements over the strongest baselines. Further ablation stud- ies validate the effectiveness and complementarity of collaborative semantic modeling and end-to-end SID optimization.

2026-11-04 05:00:00 · MoE架构,微调蒸馏,预训练,模型评测,向量数据库,招聘HR,论文

Decomposing Irregular Time Series into Interpretable Temporal Patterns with Mixture-of-Experts

arXiv cs.AIarXiv:2611.00337v1 Announce Type: cross Abstract: Irregularly-spaced sequential data, such as transaction logs, encode multi-scale temporal structure that is critical for prediction but difficult to recover explicitly with standard sequence models, which often trade interpretability for performance. We introduce MoTIF, an interpretable framework using mixture-of-experts (MoE) attention to decompose irregular time series into temporal patterns spanning minute-to-year scales. Each expert is trained via temporal alignment that induces attention specialization to a designated cyclical or within-range pattern, with label-free routing supervision that learns to activate the correct expert from patterns present in the data. On synthetic data with known temporal structure, MoTIF recovers correct routing well above chance (0.389/0.829 for range/cyclical patterns vs. 0.091 random) and outperforms all baselines, including a standard transformer and an unspecialized MoE, on next-value prediction. On a public credit card transaction dataset, MoTIF achieves higher macro-F1 than all baselines for next-amount prediction. Its learned routing reveals structure a black-box model cannot: transactions are governed primarily by intraday and short-interval rhythms, while fraudulent transactions show bursty, calendar-agnostic timing versus the steadier periodicity of legitimate ones. On downstream fraud detection, MoTIF matches standard transformer and MoE baselines and clearly outperforms recent sequential recommendation models used as embedding baselines. These results show that decomposing temporal structure into interpretable, expert-specific patterns need not sacrifice predictive accuracy, offering a practical route to sequence models that are both accurate and auditable in high-stakes settings. Code is available at https://anonymous.4open.science/r/MoTIF-82E4.

2026-11-04 05:00:00 · Transformer,扩散模型,MoE架构,向量数据库,模型安全对齐,论文

Autonomous Analysis of Thermoelectric Materials Big Data by Multi-Agentic AI and Construction of a Knowledge Base and Specialized AI

arXiv cs.AIarXiv:2611.00627v1 Announce Type: new Abstract: Extracting knowledge from large databases of experimental materials data requires repeated analyses and reusable records of their choices and results. We developed Multi-Agentic AI (MAA), a multi-agent system in which AI agents with distinct roles analyze experimental thermoelectric data under predefined instructions and quality checks, saving analysis records traceable to their sources. In four experiments varying theme and cycle counts, we measured what accumulated in these records. In the largest experiment (100 analyses across 10 themes and 10 cycles), concept co-occurrences were concentrated within themes, vocabulary was shared across themes and some relation descriptions were first detected in later cycles. A specialized AI that searches this experiment's knowledge bases and reads their analysis records answered materials questions with cited sources. Retrospective checks, including tracing an answer generated with a preliminary corpus back to the source data, exposed errors: traceability makes errors findable but not absent.

2026-11-04 05:00:00 · AI应用,Agent智能体,MoE架构,论文

NVIDIA Shows 3 Ways to Speed Up MoE Inference on DGX Spark, Lifting Ling-3.0-flash Throughput 42.5%

MarkTechPost

NVIDIA’s latest technical blog shows how to speed up mixture-of-experts (MoE) inference on one DGX Spark. The test model is Ant Group inclusionAI’s Ling-3.0-flash, a 124B-parameter MoE with 5.1B active parameters. NVIDIA stacked three optimizations: a new MoE kernel backend, an FP8 output layer and built-in speculative decoding. On all 1,319 GSM8K questions, decode throughput rose 42.5% with no statistically detectable accuracy change.

So why is it relevant for Devs? DGX Spark is a desktop system, not a data center node. NVIDIA’s results show a 124B MoE serving single-user requests faster on that desk-side box. The technical post also shows where MoE decode time goes, and how to cut it.

TL;DR

Size: Ling-3.0-flash-fp4: 124B total, 5.1B active per token; 256K native context.
Runs on: One NVIDIA DGX Spark (GB10, 128 GB unified memory, 273 GB/s), batch size one.
Performance: Three stacked optimizations lift GSM8K decode throughput from 37.81 to 53.89 tokens/s.
Best: +42.5% throughput and -24.5% mean latency, with accuracy flat (96.89% vs 96.82%).
Lagging: Synthetic gains (+185.1%) shrink on GSM8K as MTP acceptance drops from 3.90 to 2.68.

Why is MoE decoding on DGX Spark memory-bound?

DGX Spark is NVIDIA’s desktop system built on the GB10 Grace Blackwell Superchip. It offers 128 GB of unified memory and 273 GB/s of memory bandwidth. Peak compute is up to 1 petaFLOP of sparse FP4. NVIDIA rates it for inference on models up to 200B parameters.

In the NVIDIA technical post, the NVIDIA team profiled Ling-3.0-flash-fp4 at batch size 1. Decoding was memory-bound: reading model weights drove execution time. Two operations stood out, the MoE computation and the LM head that scores the vocabulary. That points to two levers: move fewer bytes per decode step, and generate more tokens per verification round.

The baseline followed the official W4A8 recipe. It used FlashInfer MXFP4 MoE kernels, CUTLASS dense GEMM and an FP32 LM-head path. On a synthetic 32K-input, 2K-output test, it ran at 27.71 ms per token (36.09 tokens per second).

What are the 3 optimizations?

1. Humming MXFP4 MoE kernels: NVIDIA swapped FlashInfer’s MoE backend for Humming, an independent kernel library. Humming uses NVRTC to compile kernels at runtime for the actual matrix shapes, data types and GPU. That includes the narrow 768-wide expert layer. Each kernel compiles in seconds. The FlashInfer CUTLASS path compiles 97 targets with ninja, on the order of tens of minutes. Throughput rose 9.3% to 39.45 tokens per second.

2. Online FP8 LM head: The LM head is a 157,184 x 2,560 matrix stored in BF16. Every generated token reads all of it. Quantizing only this layer to FP8 at load time halves those reads. Setting SGLANG_ENABLE_FP8_LM_HEAD=1 enables it. It cannot be combined with the baseline –enable-fp32-lm-head option. Combined with Humming, throughput reached 43.94 tokens per second. That is 21.7% over baseline and 11.4% over Humming alone.

3. MTP speculative decoding with NEXTN: Ling-3.0-flash ships with a built-in NEXTN prediction layer, so no extra draft model is needed. It drafts three tokens, and the model verifies four positions at once. Verification costs about 2.26x a normal decode step. The main cost is MoE, not attention. Verifying four tokens can activate up to 32 experts instead of 8. On the synthetic test, an average of 3.90 tokens were accepted per round. Throughput jumped to 102.88 tokens per second, up 185.1%.

The three techniques do not overlap. Humming and the FP8 LM head cut the cost of each step. MTP cuts the number of steps.

The integration trap: Stacking these is not automatic. With FP8 LM head and MTP both on, the draft must share the full LM-head module. Otherwise the framework silently falls back, and the draft interprets FP8 weights as BF16. Accept length drops to 1.00, and throughput falls below the unoptimized baseline. The NVIDIA team advises checking at startup that each optimization is active. NVIDIA points to the Ling cookbook notebook for enabling both together.

What happens on a real workload?

Synthetic tests flatter speculative decoding, so the NVIDIA team also ran all 1,319 GSM8K questions. It used 5-shot prompting with chain-of-thought, temperature 0.6, top_p 0.95 and top_k 29. With all three optimizations, time per output token fell from 26.45 ms to 18.56 ms. Decode throughput rose 42.5%, from 37.81 to 53.89 tokens per second. Mean latency dropped 24.5%, from 15.21 to 11.49 seconds. Accuracy moved from 96.89% to 96.82%, a change NVIDIA calls statistically undetectable.

Accept length here was 2.68, versus 3.90 on synthetic data. The NVIDIA team gives three reasons for the gap. Synthetic outputs are more repetitive, so drafts are accepted more often. The synthetic test used greedy decoding, while GSM8K used sampling. GSM8K answers are also short, with a median near 310 tokens versus a fixed 2K.

What is Ling 3.0, and why does it suit DGX Spark?

Ling-3.0-flash belongs to Ant Group inclusionAI’s Ling 3.0 family, released under the MIT license. The open-weight lineup also includes Ling-3.0-tiny, a 7.9B MoE with 1.3B active parameters. Both come with base checkpoints and FP8, INT4 and GGUF builds; flash also has FP4. OpenRouter also lists inclusionAI’s vision-language, finance and health variants built on Ling 3.0 Flash.

Ling-3.0-flash is built to keep per-token cost low:

Sparse MoE: Each MoE layer routes a token to 8 of 512 routed experts, along with 1 shared expert. About 4.1% of the weights are active per token.
Hybrid linear attention: 35 Kimi Delta Attention (KDA) layers and 7 Gated MLA layers alterna

2026-11-03 20:28:22 · 大模型,算力芯片,AI应用,开源,Google,NVIDIA,月之暗面,搜索RAG,Transformer,MoE架构,提示工程,端侧AI,招聘HR,榜单评测

Anthropic Releases FlashProve: Cryptographic Proofs That a Reply Came From Committed Weights

MarkTechPost

Anthropic’s Frontier Red Team has released FlashProve, an open-source research prototype for provable inference. FlashProve is a zkSNARK system from Anthropic that proves a reply came from a committed model. The proof reveals nothing about the model weights. On Llama 4 Maverick, a 400B mixture-of-experts model, it proves about 110 tokens/second on 1 H100. For developers, this targets a real gap: today you cannot check which model actually answered your API call.

TL;DR

Size: Proves a 400B MoE model (~17B active per token); context capped at 1,024 tokens per request.
Runs on: Prover needs 1 H100 80 GB, 24 CPU cores and ~290 GB host RAM. Verifier runs on a CPU, no GPU.
Performance: Orders of magnitude faster than prior open-source zkML systems on the same 400B model.
Best: 110 tokens/second proving on 1 H100, with 18.78 MB proofs checked in 17 seconds.
Worst: Proving costs ~70 to 95x the GPU time of plain vLLM inference.

What problem does provable inference solve?

A user querying a model cannot verify which model produced the response. A routing service could swap in a cheaper model. An attacker could tamper with the token stream. Judging from output quality alone is unreliable.

Anthropic lists 4 use cases:

Regulators could confirm hardware serves inference, supporting AI pacing rules.
Labs could detect tampered inference servers.
Users could verify text provenance.
Regulators could enforce serving of approved models only.

How does FlashProve work?

FlashProve has a prover (the model provider) and a verifier (the user). It runs in 2 phases:

Offline: The provider publishes a commitment, a short fingerprint binding one exact set of weights. This takes about 39 minutes and covers about 16 million requests.
Online: For each request, the GPU produces a proof that the committed model produced those tokens.

The proof system compiles an interactive oracle proof into a cryptographic proof using hash functions. Anthropic says this offers post-quantum security. It is the same family used by validity-proof L2 networks such as Starknet.

Three engineering choices drive the speed. Prover work scales with the ~17B active parameters, not all 400B. Requests are proved in batches after the fact, like prefill. Weights stream from host memory, so a single 80 GB GPU suffices.

How fast is it? Benchmarks

Anthropic measured the public 4-bit Maverick weights on 1 H100 with 24 CPU cores. Per the repo performance guide:

Batch of 24: 111.23 tokens/second, ~221 to 226 seconds per batch.
Batch of 1: 17.41 tokens/second. Batch of 64: 123.31 tokens/second, flagged as unreliable on 80 GB cards.
Proof size: 18.78 MB. Client downloads ~6.5 MB once per commitment.
Verification: 17 seconds on 24 threads, 21 seconds on 1 thread.

Against vLLM running FP8 Maverick on 8 H100s, proving costs about 72x the GPU time per batch. At vLLM’s highest tested load, the gap is about 94x.

How does FlashProve compare with other zkML systems?

Metrics	FlashProve	Attestable	VerInf	DeepProve	zkLLM
Model proven	Llama 4 Maverick (400B MoE)	Gemma 4 31B (dense)	Llama 4 Maverick (400B MoE)	GPT-2, Gemma 3	LLaMA-2-13B
Throughput	~110 tok/s	53 to 77 tok/s	~0.02 tok/s (1,000 tokens in 14.3 h)	~2.9 tok/s (GPT-2), ~1.4 tok/s (Gemma 3)	Full inference in under 15 min
Hardware	1 H100 + 24 CPU cores	1 H100	1 DGX Spark	1 machine; 31 tok/s distributed	GPU (CUDA)
Proof size	18.78 MB	4.35 to 7.92 MiB	93.6 GB	Not disclosed	Under 200 kB
Verify time	17 s (24 threads)	157 to 648 ms	17.7 h (20 cores)	1 to 3.7 s	Not disclosed
Context	1,024 tokens	Up to 16K tokens	1,000 tokens	Not disclosed	Not disclosed
Code / license	Open, Apache 2.0	Not open source	Open, MIT	Open source per Lagrange	Paper (CCS 2024)

How is the security checked?

Cryptographic code is brittle, so Anthropic formalized the protocol in Lean. It proves completeness, soundness and zero knowledge. The README states a cheating provider is caught except with probability about q/2^124 + 2^-101.

FlashProve ships 2 verifiers. A fast C++ verifier is for daily use but carries no formal guarantee. A slower Lean verifier is proven to accept only what the soundness theorem accepts. Anthropic team notes Claude helped optimize the system.

Key Takeaways

FlashProve proves a 400B MoE model at ~110 tokens/second on 1 H100.
Proofs are 18.78 MB and verify on a CPU in 17 seconds.
Proving costs ~70 to 95x the GPU time of vLLM inference.
Security properties are formalized in Lean; weights stay hidden.
Limited to 1,024 tokens, greedy decoding, and MoE models for now.

Check out the research post, technical report, protocol specification and GitHub repo. Als

2026-11-03 22:08:22 · 大模型,算力芯片,AI应用,具身智能,开源,OpenAI,Google,Anthropic,Meta,NVIDIA,搜索RAG,扩散模型,MoE架构,强化学习,模型评测,模型安全对齐,招聘HR,论文,开发者生态

Holo4: powering generalist computer-use agents

HuggingFace Blog

2026-09-28T09:44:05.300Z · 大模型,算力芯片,AI应用,开源,OpenAI,Google,阿里巴巴,Agent智能体,推理思考,搜索RAG,数字人,办公效率,MoE架构,强化学习,微调蒸馏,模型评测,提示工程,端侧AI,招聘HR,榜单评测,开发者生态

Welcome Le Chonk: Mistral Large 4 on Hugging Face

HuggingFace Blog

2026-10-29T00:00:00.761Z · 大模型,算力芯片,AI应用,开源,Google,Meta,NVIDIA,阿里巴巴,DeepSeek,Mistral,智谱,多模态,Agent智能体,推理思考,搜索RAG,数字人,Transformer,扩散模型,MoE架构,强化学习,微调蒸馏,预训练,模型评测,提示工程,模型安全对齐,长上下文,端侧AI,招聘HR,榜单评测,开发者生态

Transformers Model Integration Contracts

HuggingFace Blog

2026-11-03T00:00:00.763Z · 大模型,算力芯片,AI应用,开源,Google,Meta,语音音频,Agent智能体,搜索RAG,数字人,Transformer,MoE架构,强化学习,微调蒸馏,提示工程,端侧AI,招聘HR,榜单评测,开发者生态

IST undergraduate researcher presents at international AI ethics conference - The Pennsylvania State University

Google News AI (英文)IST undergraduate researcher presents at international AI ethics conference The Pennsylvania State University

2026-11-02 17:32:29 · Google,MoE架构,招聘HR

Benchmarking: Qwen 27B, 35B, QFN180B; Nemotron Bolt 74B, Flint40B & Clover 35B (Q4) on M5 Max 2026 on Kapitsa with KFlash [P]

Reddit r/MachineLearning

https://preview.redd.it/260fkihtk3zh1.png?width=2400&format=png&auto=webp&s=a80b6861abdeee3c9a4e705b6ad9b337be005722

hey folks, I'm Ye, one of the contributors to Kapitsa, an open-source inference engine optimized specifically for Apple Silicon to run Qwen, Nemotron, Flint and Clover models locally. Our custom-written Metal kernels let us tune for each chip and each model. https://github.com/nace-ai/kapitsa/

The latest update, 0.2.26, is out. Since our last M5 Max numbers, Flash-Next 180B reads prompts 46% faster and the 27B writes 8% faster, and DiffuSpace's Clover 35B joins the catalog. Kapitsa also runs Qwen Flash-Next 180B with SSD offloading: on a Mac that can't hold it, the experts that don't fit in RAM stream from the SSD, so it runs on a 48 GB MacBook Pro at 61 tok/s.

Here are the end-to-end numbers on a 2026 MacBook Pro 16 with an M5 Max and 128 GB: six targets (dense, MoE, hybrid Mamba-2 MoE and a block-diffusion MoE), from 1K context up to 32K, against the next-fastest engine for each.

Try it (any Apple Silicon Mac; models in this post: 35b, 27b, flash-next, bolt, flint, clover):

brew install nace-ai/tap/kapitsa kapitsa serve --model 35b # OpenAI and Anthropic APIs on localhost kapitsa opencode --model 35b # or claude, codex, pi kapitsa bench # benchmark your own Mac

Results

Decode is a 512-token coding answer, prefill a 4K prompt, median of 3 runs:

Model	Decode, tok/s	Prefill, tok/s
Qwen3.6 35B-A3B	346	6,036
Clover 35B-A3B	304	5,981
Flint 40B	265	5,770
Nemotron Bolt 74B	154	1,075
Qwen3.8 27B	143	904
Qwen Flash-Next 180B	108	1,701

https://preview.redd.it/r1t8d4jtk3zh1.png?width=2400&format=png&auto=webp&s=6f707ed68fe883ddf206de9e30e5f94cef4bd738

From 1K context all the way up to 32K. At 32K the 35B still writes 175 tok/s, and Flash-Next barely drops (90 to 78):

https://preview.redd.it/hxod2sktk3zh1.png?width=2400&format=png&auto=webp&s=9703ef7d4356cac43d685cf5e284a60a51764786

https://preview.redd.it/b2wqfimtk3zh1.png?width=2400&format=png&auto=webp&s=fd2da7b94e354af519fde41fbdaede61aa6df883

Against the other engines

Against the next-fastest engine for each model on the same Mac: 1.23x Splash on the 35B, 2.2x oMLX on Bolt, 1.18x rec on Flint and 1.12x mlx-serve on the 27B. Flash-Next is a tie with mlx-serve, Splash still reads prompts faster on the two Qwens, and nothing else runs Clover on a Mac.

https://preview.redd.it/zpjhe6otk3zh1.png?width=2400&format=png&auto=webp&s=2bf9bbc25c8231fcc1376e9f9efcf8d024914cea

Every cell with versions and flags: benchmarks. Raw runs: results.

Why it's fast: KFlash

Other engines verify the drafter's guesses as one chain and stop at the first miss. KFlash verifies a best-first tree of the drafter's candidates in the same 8-row pass and keeps the longest branch the target agrees with.

https://preview.redd.it/sle8vzptk3zh1.png?width=1200&format=png&auto=webp&s=c2e4afec6cad457c34108beab8824353c0123514

Three details make it fit Apple silicon:

Nearly free rows. Apple GPUs work in 8-row tiles, so on dense models an 8-row verify costs about 1.1-1.25x one decode step.
Sized per Mac. Each tree is sized cycle by cycle from a budget measured on that Mac: GPU time per row count, plus a per-position calibration of the drafter against its own acceptances.
Seed-exact. A guess is accepted only if it equals the token the target itself samples with plain decoding's random number, so a sampled reply is the one plain sampling writes with the same seed.

Write-up and measurements

Send us your benchmarks

Run the same test on your Mac and add it to the community results. It takes 3-15 minutes:

brew install nace-ai/tap/kapitsa brew install gh && gh auth login # results are sent as a GitHub pull request kapitsa bench --submit # pick a model; it asks before sending anything

Plug in and close heavy apps while it runs. At the end it asks once: Enter sends the result as a pull request from your GitHub fork, and n keeps it on your Mac. What it sends is about 2 KB: your Mac's model, chip, RAM, macOS version and the numbers, with no serial number, user name or paths (add --dry-run to see exactly what it would send). No GitHub? Paste the numbers it prints in a comment here.

One more ask

Do you know prior work where tree-verified sampling matches plain sampling token for token under the same seed? Please point me to it so we credit it. We cite SpecInfer, Medusa, EAGLE-2 and Sequoia.

submitted by /u/DotaMate
[link] [comments]

2026-11-02 01:05:33 · 大模型,算力芯片,开源,OpenAI,Anthropic,Meta,阿里巴巴,代码生成,扩散模型,MoE架构,模型评测,提示工程,招聘HR,榜单评测,开发者生态

继续滚动加载更多…