DrivingBench was about answering this question: can frontier general-purpose LLMs drive a car in the real world?

Uses an empty parking lot and cones to mark a very short and low speed (<3m/s) path to follow.

Astra observed about every 5–6 seconds

Imagine if I opened my eyes once very 5 seconds while driving…

GPT-6 Astra was the only model to fully complete the course (on attempt 2, in about 5 minutes).

They only tried twice, giving it a 50% success rate. They wouldn’t even give me a learner’s permit with that score.

Frontier models have improved to the point where they can drive real vehicles in real life at sufficiently low speeds

A single frontier model succeeded once and failed once. The other models all failed. The conclusion is not supported by their own evidence.

This doesn’t really answer its own question and also is based on a very flawed premise: get a text model to look at a series of images. It’s so reliant on other tools that the LLM isn’t really relevant other than to coordinate, which they all also basically fail at.