Scene-Adaptive Readout and Decoding in Diffusion Driving VLAs
Progress and Cons
Epistemic status: Reflections on ProgressCon 2026, written up over the following weeks. Quotes are reconstructed from memory. Artistic license taken.
For in much wisdom is much grief: and he that increaseth knowledge increaseth sorrow.
– Ecclesiastes 1:18
“I’m sure you’ve all heard about the math stuff that OpenAI released earlier this week.”
It is Thursday, October 8, 2026, and the CEO of a bio lab automation startup is giving us a tour of their San Francisco office.
“Raise your hand if you think that’s a good thing.” A dozen hands shoot up out of thirty visitors. Along with ten more, my arm rises weakly to half-join them.
“Raise your hand if you think it’s not a good thing.” I look around. A couple of quiet chuckles. No hands.
He leads us through the kitchen and the standard startup office space and into the room with the robots. They are simple 3 foot by 2 foot by 2 foot boxes, set on four rows of desks. They have been chosen because they are cheap (only a couple thousand dollars each) and run on open source software on a Raspberry Pi, which means they can be modified arbitrarily.
Inside are six standard 96-well plates (8 by 12 rows of tiny plastic tubes). Above them hangs a simple robot arm with one available motion: pipetting. He shows us a series of janky bright green 3D-printed contraptions that they attach to the arm. The pipetting mechanism is connected to gears and levers that translate the energy into whatever they want.
It turns out, most of biology is just moving small quantities of liquid between test tubes, mixing them, heating them up, and cooling them down.
They 3D print everything. Claude is really good at 3D modeling now, so if they want a new part or to change something on an existing part, they just tell Claude, and it changes the model and prints a new one. They have 3D printers in all their labs around the world. When they design a new tool, they don’t have to rely on supply chains in China, they just send the design to all the labs and print them.
Scientists used to take weeks to run these tests. Now they drop their samples off in a dropbox, the company picks them up and gets them on the next plane. By 1AM, their biggest lab, located inside the FedEx airport in Louisville, Kentucky, has already received the sample and the order form/protocol. Their lab robots run the samples according to the protocol, and the results are ready by 6AM.
Just in the last few weeks, the models are good enough to do vibe-science, where you talk to Claude, and it comes up with the lab protocol to run your experiment.
I wander over to one of the biologists who works there. He says Claude used protein design models to come up with some candidate proteins that might bind to a particular receptor. He is running 96 candidates through a protocol it designed.
The little arm lifts the samples out of the first wash, moves them horizontally a few inches, and, slightly jiggling, lowers them into the second wash.
“Where did you make the candidates?”
“Oh, we have a machine in the other room. I can show you if you like.”
“Is that automated?”
“Not yet.”
I have come to ProgressCon 2026 at Lighthaven to find potential cofounders and investors for my startup.
I notice someone with ___ Ventures on their well-designed badge.
“Hi. Jackson.”
[Handshake]
“So what brings you here.”
“I’m trying to get my AI startup off the ground.”
“Ah, cool. [reads badge]. Minerval AI. What’s that?”
“It’s AI for epistemics.”
“AI for epidemics? Isn’t that...”
“Uh, no, it’s, like, using AI to figure out what’s true, and help people hold more correct beliefs.”
“Oh.”
I am still honing my pitch.
Someone who worked on frontier bio risk evals describes what it was like.
“You would talk to one expert and they would say ‘This makes it easy to do [X] and [Y], but you still need [Z]’. Then you’d talk to another expert and they’d say, ‘[Y] and [Z] are easy. The hard part is [X]. We don’t actually know how much uplift it would give someone. It’s not like we’re hiring a bunch of terrorists and running a randomized trial to see if they can make a bioweapon.”
“So why do you think is holding back all the bioweapons?”
At a nearby table, the author of You Can Just Do Things is giving out signed copies.
I learned about the conference too late to apply normally. It was already full by the time I started Minerval a few months ago. There were slots for sponsors, though. $5,000 and a few emails later, I had a ticket for one entry and Minerval’s logo on the conference website.
“So how does it work?”
“Well, first we ingest claims, from,
System helps humans predict when self-driving cars will make mistakes
Self-driving cars are often controlled by deep learning models that sometimes fail in unexpected situations. For instance, the car might inexplicably brake and block the path of an oncoming emergency vehicle. A human driver or passenger may need to react rapidly to prevent a collision.
To help humans better anticipate a vehicle’s mistakes, researchers from MIT and autonomous vehicle technology company Motional developed a new method that provides clear explanations of the underlying model’s decisions.
Usually, the internal reasoning process of a deep learning model is opaque and difficult to understand. But the new method, called the Concept-Wrapper Network (CW-Net), translates that reasoning process into concepts that faithfully describe the autonomous vehicle’s decisions without altering its driving performance.
CW-Net explains the decisions of machine learning-based planners using understandable concepts, like “approaching stopped vehicle” or “close to cyclist.” These explanations can correct misconceptions drivers and passengers have about vehicle behavior and improve their situational awareness.
In road tests on a private track, CW-Net explanations helped safety drivers more accurately predict vehicle behavior; a larger simulation study with nonexpert users yielded similar results. These experiments show how CW-Net can provide important feedback for engineers as they troubleshoot in-vehicle artificial intelligence systems. In the longer term, this technique could boost the safety and transparency of autonomous vehicles, while building appropriate trust in drivers and passengers.
“This work shows how explanations are supportive to the human’s mental model and understanding of the behavior of a system, and how it could be used in engineering and development to improve the technology,” says Julie Shah, an MIT professor of aeronautics and astronautics, director of the Interactive Robotics Group in the Computer Science and Artificial Intelligence Laboratory (CSAIL), and co-senior author of the paper on CW-Net. “Unless we are building these technologies in a way that we can rely on and predict their behavior, then it is a shaky and unsafe foundation for their use.”
She is joined on the paper by lead author Eoin Kenny, a former MIT postdoc who is now a senior AI researcher at J.P. Morgan Chase; co-senior author Momchil Tomov, a staff research scientist at Motional; as well as Motional team members Akshay Dharmavaram, Sang Uk Lee, Tung Phan-Minh, Shreyas Rajesh, Yunqing Hu, and Laura Major, president and CEO of Motional. The research appears today in Nature.
Faithful explanations
Machine-learning-based planners act as the “brain” of a self-driving car. These powerful deep-learning architectures process data from the vehicle’s cameras and lidar sensors, generate a high-level summary of the vehicle’s environment, decide what the car should do next, and output a trajectory for it to follow.
The planners are usually black-box models, which means their internal decision-making process is so complex it is difficult to understand. This can leave scientists and safety drivers in the dark about why an autonomous vehicle made an unexpected decision, like phantom braking.
The researchers designed CW-Net to explain a vehicle’s decisions using understandable concepts, while ensuring those explanations accurately reflect the true reasons behind its behavior.
“Especially in high-stakes settings like self-driving cars, it’s important that the explanations are not potentially misleading. Because CW-Net is causally faithful in how it makes decisions, that provides certain guarantees around the explanations,” Kenny says.
CW-Net is a “concept classifier,” an AI algorithm that has been trained to predict the high-level concepts that exist within input data. The researchers plug the CW-Net module into the middle of an autonomous vehicle’s existing machine-learning planner architecture.
It translates the model’s internal reasoning process into understandable concepts, like “approaching stopped vehicle” or “close to cyclist.” Then it forces the final piece of the planning model architecture to use those concepts when it decides what the vehicle should do next. In this way, CW-Net ensures the concepts faithfully explain the vehicle’s actions.
At the same time, CW-Net uses the concepts it classified to generate clear explanations that are output along with the vehicle trajectory, in real-time.
“Instead of just wondering why the car stopped, having real-time data provides feedback that lets you test the system during deployment. You could also give that data to an engineer to potentially improve the system,” Kenny says.
The researchers trained CW-Net to predict concepts using a dataset of 130 million examples of scenes from self-driving cars, with multiple labeled concepts in each scene. Using such a large, labeled dataset enables it to identify concepts accurately in a wide range of settings.
They also designed CW-Net to mimic the driving decisions of machine-learning-based planners, so the module would not negatively impact vehicle performance.
In the end, CW-Net generates accurate, understandable explanations without altering the original deep learning model.
Improving situational awareness
To test CW-Net, the researchers deployed the module on a real autonomous driving test vehicle (a Motional robotaxi) on a private track with a safety driver. They found that CW-Net helped the safety driver better predict how the vehicle would behave in surprising situations.
For instance, the vehicle consistently stopped when it approached a cyclist, and the safety driver assumed it did so because it detected that cyclist. But CW-Net explanations revealed that the model wasn’t properly configured to detect the cyclist and chose a trajectory that would have caused a collision. Instead, it stopped because its emergency braking procedure kicked in when it got too close.
Armed with this information about the model’s mistake, the safety driver could reduce speed or engage manual driving mode sooner in similar situations. This could also help engineers fix the model to avoid this failure in the future.
In larger online simulation studies using real driving situations captured on the roads of Las Vegas, the researchers saw similar results. CW-Net explanations significantly improved participants’ abilities to predict how an autonomous vehicle will behave.
In the future, the researchers could extend CW-Net so the module can cover more concepts and explore different training and design techniques that could boost performance and improve interpretability.
“Our study shows how crucial interpretability can be to these high-stakes environments, and how it should be on the mind of people as they are making AI in the future, for self-driving cars or other safety-critical environments,” Kenny says.
Computational tools for society’s most complex challenges
As far back as she can remember, Cathy Wu ’12, MNG ’13 wanted to find ways to solve problems to improve people’s lives. Her parents were Taiwanese immigrants, and her father had a long commute to his job, which took him away from the family. On a tight budget, the rest of the family often stayed home on a street that was too busy for playing outdoors. Wu and her siblings ended up playing a lot of computer games.
Wu says her desire to make the world a better place, her dad’s daily battle against traffic, and the games she played, like “SimCity,” were the seeds of her motivation to design safe, efficient transportation systems.
Wu is an associate professor in the MIT Department of Civil and Environmental Engineering (CEE) and the Institute for Data, Systems, and Society (IDSS), and a principal investigator in the Laboratory for Information and Decision Systems. Her research focuses on using machine learning and reinforcement learning (RL) to advance reliable strategies for improving a range of complex systems, including transportation.
“Designing transportation systems consists of modeling and analyzing dozens, if not hundreds or thousands, of variants, which means that an evidence-driven approach to designing those systems is simply not within reach of today’s tools,” Wu says. “This is the role that RL plays. If successful, it would free transportation researchers and enable their practitioner partners to design the systems they want.”
Wu credits her older sister with instilling in her the desire to improve people’s lives, and Wu’s interest in transportation fits neatly into that ideal.
“I like transportation because it connects everyone. We all use it, we all experience it, we all have issues with it. So, at some level, we’re all interested in the system being better,” she says.
Wu got interested in applying artificial intelligence to transportation while earning her undergraduate degree at MIT, after attending a lecture on autonomous vehicles by the late professor Seth Teller. The lecture, which Teller gave during an Independent Activities Period robotics competition (that Wu actually won), was the event that honed her particular approach to transportation research, Wu says. She began working with Teller, and when he stopped concentrating on autonomous vehicles, he encouraged Wu to transfer to Professor Daniela Rus, who had done research on robotaxis.
“I’m very grateful to the people who helped me explore those interests and helped me become the person I am now,” she says, specifically naming Teller, Rus, and “my friends at Dropbox,” who invited her to do a second internship focused on transportation issues.
After her master’s degree at MIT, Wu went on to earn her PhD at the University of California at Berkeley. During that time, she observed that transportation researchers were spending years developing optimization methods to model and analyze a single new variant of a system. Her approach as a computer scientist working to develop RL and optimization methodologies to address transportation challenges held the promise of exponentially improved efficiency.
In 2018, Wu’s last year of her PhD at UC Berkeley, she successfully applied RL to a traffic problem: automatically analyzing the potential traffic flow impact of autonomous vehicles in a range of different traffic networks. The research went viral.
While this could have been a “the rest is history” moment for Wu, RL turned out to be a flighty friend. Wu worked on RL theory in a postdoc at Microsoft and came back to MIT as faculty drawn, she says, by the sustainability focus of CEE, and IDSS’s emphasis on infusing data science into other disciplines.
Yet over the next two years, Wu’s further attempts to apply RL to traffic problems failed.
“That was stressful,” Wu says, “it was unclear whether the problem was me (the advisor), my students, the traffic domain, or RL itself.”
Still, the earlier research was a proof-of-concept demonstration that RL could be applied to transportation systems.
And in 2022, she and her students identified that RL algorithms are so sensitive that an algorithm that works on one problem may not on even a closely related one. A key result, which Wu says she is proudest of “because it was like the light at the end of a long tunnel of negative results,” came in 2023. She and her team of researchers devised a way to work around the sensitivity of RL. The team found that while RL may not train well on 90 percent of a group of problems, it can train quite well on 10 percent. And by training RL models on those problems that solve and generalize well, the resultant models collectively perform well on a set of related problems, even those that would not have been solved through direct training. The researchers designed an algorithm to determine which problems to use RL to train, and that algorithm improved training efficiency by up to 30 times, meaning that what would normally have required 100 training models may only require three models.
“This work gave me back the confidence that reinforcement learning can play an important role in solving hard optimization problems, including in transportation,” Wu says. “Now, a good chunk of my group works on the topic of contextual RL, which is the setting where RL seeks to solve a space of related problems.”
Wu’s more recent research applies RL to solve a hard transportation optimization problem with important policy implications: the work shows that eco-driving measures in which vehicle speeds are intelligently controlled to reduce excessive stopping and starting could reduce vehicle emissions by between 11 and 22 percent. The system provides evidence that policies instituting such measures could significantly improve system efficiency, and is “a demonstration that RL can be used to inform transportation policy on problems of practical importance,” Wu says.
“I am a big fan of evidence-based policy and believe it’s the basis for a thriving democratic society, yet our societal systems are so complex,” Wu says. “People can bicker forever about what’s better or worse, but I do believe that there are questions we bicker about that can be analyzed systematically using data and have objective answers. A large part of the reason I am in academia is to better understand how technology can support democratic societal decision-making.”
Wu says that much of the work she and her team have done over the last several years has produced algorithms “to streamline the development of solvers for hard optimization problems, whether they are related to transportation or to other systems, such as logistics, supply chains, manufacturing, and resource allocation.
“This alludes to my preferred style of work,” Wu says, “which is called use-inspired basic research,” explaining that such research addresses a practical problem, developing fundamental knowledge that often translates to other practical problems. Her students start by probing consequential problems ranging from safety to congestion to accessibility, identifying where existing methods fall short, and allowing the problems themselves to shape the direction of the research.
At the same time, Wu’s desire to help others on a more personal level plays out in her teaching.
“I love working with students, both in the classroom and research mentoring,” she says. “It makes my day when I am able to teach someone something — when I see that light bulb go on in a student.”
In addition to earning academic honors, including a 2023 National Science Foundation Faculty Early Career Development Award, Wu has also been formally celebrated for her teaching and mentoring, including with the Ole Madsen Mentoring Award in 2025.
What does she tell students confronting extremely complicated problems?
“Be patient. Start small. Societal impact is a lifelong endeavor, not som
China Publicly Discloses Plan For a Lunar Landing Within Three Years
Read more of this story at Slashdot.
Cops Bust Botnet of 1.3 Million Android TV Boxes
Read more of this story at Slashdot.
MIT Researchers Used GPT-6 Astra To Edit Qwen's Open Weights Directly, Boosting Math Scores
Read more of this story at Slashdot.
SUSE To Cut One In Ten Employees Because of AI
Read more of this story at Slashdot.
Google Will Turn Off Search For Schools That Reject AI
Read more of this story at Slashdot.
Fujifilm To Boost Tape Production 5x As AI Fuels Archiving Demand
Read more of this story at Slashdot.
Binance Says Hackers Used Fake Job Interviews, AI Tool to Steal $6.3 Billion In Crypto
Read more of this story at Slashdot.
Firefox's New Mini Window Is Every Multitasker's Dream
Read more of this story at Slashdot.
Bethesda Confirms The Elder Scrolls 6 Is Called 'Sentinel'
Read more of this story at Slashdot.
EU Court Advisor Warns Google AI Chatbot May Breach Copyright Rules
Read more of this story at Slashdot.
iOS 27.3 Adds Vertical Tabs, Adaptive Dark Mode to Safari
Read more of this story at Slashdot.
Ukraine's Drone Strikes Spur Hasty Russian Plans For Underground Data Centers
Read more of this story at Slashdot.
[酷工作] OmnixDrive 人才招募
OmnixDrive 人才招募
高级人工智能工程师|智能驾驶数据与模型研发
公司简介
OmnixDrive 是一家成立于 2026 年、总部位于新加坡的人工智能初创科技公司,专注于智能驾驶底层人工智能技术与基础设施研发。技术方向涵盖大规模驾驶数据处理、深度学习模型训练与优化、视觉感知、多传感器融合及算法工程化,为智能驾驶系统提供底层技术支持。
招聘信息
- 招聘人数: 3–5 名
- 工作模式: 混合办公,可远程协作或根据项目需求现场工作
- 协作时区: UTC+8(新加坡时间)
- 合作形式: 项目制合作
岗位职责
- 设计和开发智能驾驶数据处理体系,包括数据清洗、筛选、标注、增强及训练数据集构建。
- 研发视觉感知算法,涉及目标检测、图像分割、目标跟踪及三维环境感知。
- 负责深度学习模型的设计、训练、微调与优化,提升模型精度、鲁棒性及泛化能力。
- 参与摄像头、激光雷达、毫米波雷达等多源传感器数据处理与融合。
- 优化模型训练流程、计算资源利用率、推理效率及部署性能。
- 独立负责核心技术模块,解决复杂算法与工程问题,按项目要求完成交付。
任职要求
- 10 年以上软件开发经验,其中至少 5 年机器学习、深度学习或智能驾驶人工智能研发经验。
- 具备扎实的计算机科学、数学及机器学习基础,深入理解神经网络和模型训练机制。
- 精通 Python,熟练使用 PyTorch 或 TensorFlow;具备 C++ 开发能力者优先。
- 具备大规模训练数据处理经验,熟悉数据清洗、标注、增强、样本筛选及质量评估。
- 在目标检测、语义分割、目标跟踪或三维感知等计算机视觉方向具备深入研发经验。
- 具备完整的模型训练与优化经验,能够独立完成模型设计、训练、参数调优及性能评估。
- 能够解决模型过拟合、训练不稳定、长尾场景识别及模型泛化能力不足等问题。
- 熟悉模型部署、推理优化及 GPU 计算资源管理,具备独立技术攻关和项目交付能力。
优先条件
- 具备自动驾驶或高级辅助驾驶系统研发经验。
- 熟悉鸟瞰视角感知、占用网络、三维目标检测及多目标跟踪技术。
- 具备摄像头、激光雷达及毫米波雷达多传感器融合经验。
- 熟悉 Transformer、卷积神经网络及视觉基础模型。
- 具备大规模驾驶数据管理、自动化标注及数据闭环建设经验。
- 熟悉 CUDA、TensorRT、ONNX、分布式训练及模型量化、推理加速技术。
- 具备量产级智能驾驶系统或实际道路场景研发经验。
合作与报酬
采用项目制合作,报酬根据项目规模、技术难度、工作量及交付要求协商,可按项目阶段或交付成果结算。优秀人才可获得长期合作机会。
申请方式
请将个人简历发送至:
careers@omnixdrive.com
申请材料请注明工作年限、技术专长、相关项目经历、个人技术贡献、模型研发经验、期望项目报酬及可投入时间。
OmnixDrive — 以人工智能驱动下一代智能出行。
OmnixDrive Careers
Senior AI Engineer – Autonomous Driving Data & Model Engineering
About OmnixDrive
OmnixDrive is a Singapore-based AI technology startup established in 2026, specializing in foundational AI technologies and infrastructure for intelligent driving. Its technical focus includes large-scale driving data processing, deep learning model training and optimization, visual perception, multi-sensor fusion, and algorithm engineering.
Position Overview
- Openings: 3–5
- Work Arrangement: Hybrid, with remote collaboration and on-site work depending on project requirements.
- Collaboration Time Zone: UTC+8 (Singapore Time)
- Engagement: Project-based
Key Responsibilities
- Design autonomous driving data pipelines, including data cleaning, filtering, annotation, augmentation, and dataset construction.
- Develop perception algorithms for object detection, image segmentation, tracking, and 3D environment understanding.
- Train, fine-tune, and optimize deep learning models to improve accuracy, robustness, and generalization.
- Develop data processing and fusion solutions for cameras, LiDAR, and radar.
- Optimize training workflows, computing resources, inference efficiency, and deployment performance.
- Lead core technical modules and deliver solutions to complex algorithmic and engineering challenges.
Requirements
- 10+ years of software engineering experience, including at least 5 years in machine learning, deep learning, or autonomous driving AI development.
- Strong foundations in computer science, mathematics, and machine learning, with an in-depth understanding of neural networks and model training.
- Expert proficiency in Python and hands-on experience with PyTorch or TensorFlow. C++ experience is preferred.
- Proven experience in large-scale training data processing, including cleaning, annotation, augmentation, sample selection, and quality assessment.
- Substantial experience in at least one computer vision area, such as object detection, semantic segmentation, tracking, or 3D perception.
- End-to-end model development experience, covering architecture design, training, hyperparameter tuning, and performance evaluation.
- Ability to diagnose and resolve overfitting, training instability, long-tail recognition, and generalization issues.
- Experience in model deployment, inference optimization, and GPU resource management, with strong independent problem-solving and delivery capabilities.
Preferred Qualifications
- Experience in autonomous driving or advanced driver-assistance systems.
- Knowledge of BEV perception, occupancy networks, 3D object detection, and multi-object tracking.
- Experience in camera, LiDAR, and radar sensor fusion.
- Familiarity with Transformer architectures, CNNs, and vision foundation models.
- Experience in large-scale driving data management, automated annotation, and data-loop infrastructure.
- Knowledge of CUDA, TensorRT, ONNX, distributed training, model quantization, and inference acceleration.
- Experience with production-grade autonomous driving systems or real-world road applications.
Engagement & Compensation
This is a project-based engagement. Compensation will be negotiated according to project scope, technical complexity, workload, and delivery requirements. Payments may be arranged by project milestones or deliverables. Long-term collaboration opportunities are available.
How to Apply
Please send your CV/Resume to:
careers@omnixdrive.com
Please include your professional experience, technical expertise, relevant projects, individual contributions, model development experience, expected project compensation, and availability.
Website: https://OmnixDrive.com
OmnixDrive — Driving the Future of Intelligent Mobility Through AI.
奔驰新款长轴E级车上市:首次搭载中国团队开发的AI座舱
11月2日,2026年款梅赛德斯-奔驰长轴距E级车今日上市,起售价为42.38万元。新车率先搭载由奔驰中国研发团队主导、独立开发的全新AI智能座舱,车载虚拟助手融合豆包AI大模型,新增声纹识别等功能。
阅读全文