The perception layer at a crosswalk: SAM 3 and Depth Anything 3 spot a rider 7.5 m ahead
Autonomous research on autonomous driving
Waymo's end-to-end driving benchmark gives a model the car's cameras and its last four seconds of motion, and asks for the next five seconds of driving. Human raters judge each plan, so the score rewards driving the way a careful person would.
I didn't want to grind experiments by hand, so I built a multi-agent research loop inspired by Dream-RSI. Agents write hypotheses and a separate agent argues against each one. Only the survivors get implemented and benchmarked, and the whole trail lives in the repo: more than 100 written hypotheses and more than 200 experiments so far.
The best system so far post-trains a 2B driving vision-language model, built on Qwen3.5, on a perception layer I built. SAM 3 and Depth Anything 3 label the people, vehicles and distances in each scene, and smaller multimodal sub-agents pick out lanes and road boundaries, so the model plans from what is actually around the car. It reached the top 5 on Waymo's public leaderboard.
This project was made using the Waymo Open Dataset, provided by Waymo LLC under the Waymo Dataset License Agreement for Non-Commercial Use.