Union Types and Option Types
the lost malloc()
My favourite laptop
[R] There is no free lunch in RLVR: under strategy specialization, every pass@1 gain provably costs some prompt, and the pass@k crossover has a tight bound [R]
Paper: https://arxiv.org/abs/2610.39295
Code + verification package (every number in the paper is checked by a script): https://github.com/yliusci/no-free-lunch-rlvr
Browser diagnostic: https://huggingface.co/spaces/yliusci/passk-price
I kept running into the same pattern in RLVR papers: the RL model wins at pass@1 and the base model catches up or wins at large k (Yue et al. 2025 and many follow-ups). I wanted to know whether more training fixes this or whether it is structural. This is a theory paper, no new training runs.
Setup: model RLVR as an exponential tilt of the base policy over solution strategies shared across prompts (exact for mirror descent, and GRPO stays within 3e-4 TV of it at matched entropy in simulation).
What comes out:
No free lunch. Under exclusive specialization, every update that raises pass@1 strictly lowers success on some prompt. The exact condition is a Stiemke duality statement (a strictly positive price on prompts that equalizes strategy fitness). Being the best strategy on each prompt is provably NOT enough: there is a 3x3 counterexample with a free lunch.
The pass@k price. Once the losses reach the hardest prompts, a crossover is forced. The number of crossings is at most the sign variation of the shift in per-prompt success (Karlin total positivity). The bound is tight for 1 to 6 crossings, and the order-preserving case gives exactly one, which recovers the known single-crossover result.
A frontier, not a bug. With a per-prompt floor, the constrained update traces the optimal pass@1 vs pass@k trade. Forbidding any per-prompt loss removes the crossover and the whole pass@1 gain with it.
The same tilt gives the entropy-performance law with an identifiable exponent.
Practical part: base and RL per-prompt success rates (already in any eval log) are enough to read off the crossing count, the bound, and the price (worst pass@k loss). On published GRPO numbers for Omni-MATH (transcribed from 2607.20543, so indicative only) it finds one crossing between k=64 and k=256.
Where I expect pushback: the shared-strategy model is a population-limit abstraction, and the forced crossover needs a coverage assumption (the paper gives the regime where it fails). I would really like to see per-prompt logs where people observe 2+ crossings, since that is the signature of RL reordering prompt difficulty.
[link] [comments]
Benchmarking: Qwen 27B, 35B, QFN180B; Nemotron Bolt 74B, Flint40B & Clover 35B (Q4) on M5 Max 2026 on Kapitsa with KFlash [P]
![Benchmarking: Qwen 27B, 35B, QFN180B; Nemotron Bolt 74B, Flint40B & Clover 35B (Q4) on M5 Max 2026 on Kapitsa with KFlash [P]](https://external-preview.redd.it/lD506kUiZbfR2XmsJJ8yNSBVY6lCisMuJn2qf-Dfx7s.png?width=140&height=70&auto=webp&s=d3781be07761f18fd70c3cf16dc245eb19de2a79 "Benchmarking: Qwen 27B, 35B, QFN180B; Nemotron Bolt 74B, Flint40B & Clover 35B (Q4) on M5 Max 2026 on Kapitsa with KFlash [P]") | hey folks, I'm Ye, one of the contributors to Kapitsa, an open-source inference engine optimized specifically for Apple Silicon to run Qwen, Nemotron, Flint and Clover models locally. Our custom-written Metal kernels let us tune for each chip and each model. https://github.com/nace-ai/kapitsa/ The latest update, 0.2.26, is out. Since our last M5 Max numbers, Flash-Next 180B reads prompts 46% faster and the 27B writes 8% faster, and DiffuSpace's Clover 35B joins the catalog. Kapitsa also runs Qwen Flash-Next 180B with SSD offloading: on a Mac that can't hold it, the experts that don't fit in RAM stream from the SSD, so it runs on a 48 GB MacBook Pro at 61 tok/s. Here are the end-to-end numbers on a 2026 MacBook Pro 16 with an M5 Max and 128 GB: six targets (dense, MoE, hybrid Mamba-2 MoE and a block-diffusion MoE), from 1K context up to 32K, against the next-fastest engine for each. Try it (any Apple Silicon Mac; models in this post: ResultsDecode is a 512-token coding answer, prefill a 4K prompt, median of 3 runs:
From 1K context all the way up to 32K. At 32K the 35B still writes 175 tok/s, and Flash-Next barely drops (90 to 78): Against the other enginesAgainst the next-fastest engine for each model on the same Mac: 1.23x Splash on the 35B, 2.2x oMLX on Bolt, 1.18x rec on Flint and 1.12x mlx-serve on the 27B. Flash-Next is a tie with mlx-serve, Splash still reads prompts faster on the two Qwens, and nothing else runs Clover on a Mac. Every cell with versions and flags: benchmarks. Raw runs: results. Why it's fast: KFlashOther engines verify the drafter's guesses as one chain and stop at the first miss. KFlash verifies a best-first tree of the drafter's candidates in the same 8-row pass and keeps the longest branch the target agrees with. Three details make it fit Apple silicon:
Send us your benchmarksRun the same test on your Mac and add it to the community results. It takes 3-15 minutes: Plug in and close heavy apps while it runs. At the end it asks once: Enter sends the result as a pull request from your GitHub fork, and One more askDo you know prior work where tree-verified sampling matches plain sampling token for token under the same seed? Please point me to it so we credit it. We cite SpecInfer, Medusa, EAGLE-2 and Sequoia. [link] [comments] |
What actually broke when I tried to fine-tune on web-scraped data and what fixed it [D]
Fine tuning on self collected web data felt straightforward until it wasn’t. Here’s what actually caused problems:
Boilerplate contamination was the silent killer. Navigation menus, cookie banners, and footer text were leaking into training examples. The model started generating weirdly structured outputs that looked like half-rendered HTML. Solved it with aggressive DOM-targeting rather than raw text extraction and switching to a rotating residential proxy layer (per-request IP rotation means target sites serve you the same rendered DOM a real browser session would get, rather than a bot-filtered skeleton) made a noticeable difference in how clean the raw HTML was before I even touched the parser.
Deduplication order mattered more than I expected. Running exact dedup before fuzzy dedup produced meaningfully different training sets than the reverse. The model’s tendency to regurgitate phrasing shifted noticeably depending on the order.
The thing I still haven’t fully solved: near-duplicate detection across different languages. MinHash works fine monolingually, but multilingual dedup keeps surfacing edge cases I don’t love.
Curious whether others have hit the same issues, especially around multilingual corpora. Also genuinely wondering if anyone has a principled approach to deciding when scraped data quality is “good enough” to stop cleaning and just train.
[link] [comments]
How to handle object deduplication across multiple camera angles using YOLO (especially for tiny 20x22px crops) across isolated user sessions? [D]
I’m building a computer vision pipeline where users upload multiple photos of the same physical scene taken from different angles (front view, side view, close-up) to build a clean list of unique objects and their attributes.
I’m running YOLO for object detection to extract bounding boxes and crop each detected object into lightweight .webp images. Because the same physical object appears across 3 or 4 different photos from different angles, I need to make sure each unique object is only counted once per session, and avoid running expensive downstream AI calls to re-describe an object we already processed from an earlier photo.
[link] [comments]
Any update with the accomodation from NeurIPS [D]
Hi! For people who have recieved the financial grant, is there any update regarding the accomodation from the organizers?
Edit; Asking for Sydney
[link] [comments]
Operators Day : 10 founders hiring for founding ML roles - Nov 5 [n]
Hi all! I'm part of the EWOR team and we're organising Operators Day this Thursday, November 5 (4:30 PM CET / 9 PM IST), online on Zoom.
Ten founders will be pitching live, sharing what they're building and the founding roles they're hiring for - AI/ML engineers, robotics engineers, CTOs, CPOs, marketing and sales leads, and more.
These are real founding opportunities with meaningful equity.
If you're an ML engineer or researcher exploring your next move, or know someone who might be a great fit, come join us!
Spots are limited.
Register here: https://luma.com/OperatorsDay?utm_source=reddit&utm_content=machinelearning
Disclosure: I work with EWOR and am helping organise the event.
[link] [comments]
LightMIS: Ultra-lightweight medical image segmentation for edge devices [R]
Hi everyone,
I am one of the authors of LightMIS, and I would like to share our work on 2D binary medical image segmentation. LightMIS has 131K parameters and aggregates multi-scale encoder features directly, without a learned stage-wise decoder. It achieves close observed modality-macro Dice and IoU averages to nnWNet, while using 98.14% fewer parameters and 94.58% fewer GFLOPs, with both models trained and evaluated under the same nnU-Net protocol.
Paper: https://arxiv.org/abs/2609.28327
Code: https://github.com/AndreiiArhire/LightMIS
We also tested LightMIS and the reference models on a real edge device with an Arm Mali-G52 MC2 GPU, where the full LightMIS model ran around 10.5× faster than nnWNet. All LightMIS variants achieved 100% GPU operator coverage, without CPU fallback.
We started by training several popular medical image segmentation architectures, both lightweight and heavyweight, from scratch in the same controlled environment using nnU-Net v2.3.1. The experiments covered six datasets and five imaging modalities. Our goal was to develop a lightweight architecture that performs well across these modalities, which led us to the current version of LightMIS.
The source code, checkpoints, and details needed to reproduce the results are publicly available. Feel free to experiment with it and try it on your own segmentation tasks. I would be happy to hear how it works for you! Any suggestions or feedback are welcome!
[link] [comments]
What did Ilya Sutskever mean when he said "The models, they just want to learn"? [D]
It was just before OpenAI started that I met Ilya, who you interviewed. One of the first things he said to me was — “Look. The models, they just want to learn. You have to understand this. The models, they just want to learn.” And it was a bit like a Zen Koan. I listened to this and I became enlightened.
And over the years, I would be the one who would formalize a lot of these things and kind of put them together, but what that told me was that the phenomenon that I'd seen wasn't just some random thing. It was broad. It was more general. The models just want to learn. You get the obstacles out of their way. You give them good data, you give them enough space to operate in, you don't do something stupid like condition them badly numerically, and they want to learn. They'll do it.
I'm actually not entirely satisfied with Dario's elaboration! If Ilya meant precisely "given sufficient resources, do your job, and the models will do theirs [during training]", I think he would have phrased his advice slightly differently. What does it mean to "want" to learn? Why personify the model in that way, and not the optimizer or gradient descent itself or something? And I don't get why the koan would necessarily imply that scaling would have worked so well.
I realize that Ilya likes to cultivate an air of prophecy about himself (and I'd be lying if I said it didn't work on me). But it's just such a strange way to communicate the message, that I can't help but think the message must have been about something broader.
Possibly much more mundane in context, this must have been 10 years ago. Idk I wonder what it must have been like to be on the frontier at that time, maybe I take such things for granted now.
[link] [comments]
Small-scale check of "good views beat next-token prediction" (Balestriero & LeCun 2026): human captions only, 11M-param encoders [P]
![Small-scale check of "good views beat next-token prediction" (Balestriero & LeCun 2026): human captions only, 11M-param encoders [P]](https://preview.redd.it/j2humdsjq7zh1.png?width=640&crop=smart&auto=webp&s=433d5e205287b02901f80e5cf2d5b07bbea3df2d "Small-scale check of \"good views beat next-token prediction\" (Balestriero & LeCun 2026): human captions only, 11M-param encoders [P]") | The paper claims text models align with image models faster when trained on "good views" (two different captions of one photo) than on bad views (one caption, words hidden) or next-token prediction. Its largest run used captions written by an image captioner, so image information could have leaked in. I tried a laptop-scale version with human captions only. Setup
Results (same learning speed, mean of 3 seeds)
Each method also got a small learning-rate search. Best speed, 1 seed: 0.328 / 0.196 / 0.168. Order unchanged. Limits
Code, chart and per-run results: https://github.com/IFAKA/platonic-lab What I would test next: does the gap shrink as the encoder grows? I could not test that on a laptop. Has anyone tried? [link] [comments] |
ML PhDs: Would you rather have done your PhD before or after the rise of AI chatbots? [D]
For those doing or having completed a PhD in ML, I’m curious:
If you had the choice, would you rather have done your PhD before ChatGPT and other AI chatbots became widely available, or in today’s AI era?
[link] [comments]
Couldn't reproduce a published paper's numbers with its own code and settings. How do I present this? [R]
I'm writing a machine learning paper. One of my baselines is a published method. I reran it with the authors' released code and the exact hyperparameters listed in their paper. On most benchmarks my numbers match theirs within noise, but on two benchmarks my reproduction is clearly lower, by several points, over three seeds. I've repeated the reproduction four different times, months apart, with the same result.
I plan to report both: their published numbers and my reproduction with the exact configuration, clearly labeled. My worry is that reviewers tend to trust the published number more than a submission's reproduction, and may assume I ran the baseline incorrectly.
Questions:
What makes a reproduction convincing to reviewers? So far I have: the authors' code unchanged, their hyperparameters, matching results on the other benchmarks as a sanity check, and multiple seeds.
Should I contact the authors or open a GitHub issue first, and should I mention that in the paper?
How do I word it so it doesn't read as an attack on the other paper, but my results also don't look weak next to the published numbers?
Have reviewers accepted "published vs. reproduced" rows in a main table for you, or did they end up in the appendix?
[link] [comments]
ICLR 2027 review results [D]
The reviews will be out this week. What are the expectations with 60K+ submissions and the small scale of review rating?
[link] [comments]