The Wrong KPIs Lead to the Wrong Pilot Projects

An autonomous vehicle operating on public roads is not, by itself, proof of public benefit. At first, it is simply a vehicle on the road.

This is where one of the biggest misunderstandings in the debate over autonomous mobility in public transportation begins. Too often, success is measured by whether a shuttle works technically, completes a test route, or a demonstration vehicle is visibly on the road. But if the only thing you measure is whether a vehicle is moving, you are missing the real objective. After all, public mobility is not introduced so that vehicles can move. It is introduced so that people can be more mobile.

The Handbook on Autonomous Driving in Public Transport explicitly addresses monitoring, evaluation, and optimization as part of implementation. This makes it clear: The question of the right evaluation criteria is not a secondary concern, but is at the core of system development.

Source: More information

This raises a crucial question: What are the right KPIs for autonomous mobility in public transportation? Certainly not just mileage, test kilometers, or technical availability in the narrow sense. Other metrics are more relevant: accessibility, service quality, utilization, integration with existing services, operational stability, acceptance, cost-effectiveness, and actual contribution to public mobility.

The German federal government explicitly describes autonomous driving as a tool for new mobility services, improved accessibility, and greater participation.

Source: More information

But if that is the political and public vision, then the KPIs must align with it. It must become clear whether people can travel more easily, whether service gaps are being closed, whether regions are becoming better connected, whether the system operates reliably, whether it is actually being adopted, and whether this leads to an operation that can become organizationally and economically viable.

This is precisely where the wrong pilot projects often arise. When funding logic, reporting, and public perception respond primarily to technical visibility, projects are designed specifically for that purpose. Then the priority becomes having something operational—not necessarily solving a meaningful problem. This results in a demonstration rather than real impact. And then systems are tested without their public value being truly demonstrated.

Bitkom also argues for larger model regions, larger service areas, and higher vehicle counts, because reliable insights into scalability, economic viability, and impact cannot be gained from small-scale individual cases.

Source: More information

This makes it clear: The choice of KPIs helps determine what kind of projects are developed in the first place. Those who focus on technical metrics end up with technical pilot projects. Those who measure public impact end up building more meaningful systems. KPIs are therefore not a neutral management issue. They guide attention, funding, procurement, and strategic decisions.

This is where the technical side of the discussion becomes critical. After all, public impact involves more than just a vehicle being able to drive autonomously under ideal conditions. It is equally important that vehicle behavior remains predictable and controllable during operation.

That is why the evaluation of autonomous mobility in public transit requires a more nuanced approach: not just drivability, but controllability. Not just test performance, but operational stability. Not just visible movement, but reliable behavior under real-world conditions. Only then does it become clear whether technology can actually become a resilient system.

An international perspective: In its transportation policy, Japan focuses not only on technology but also on the reorganization of regional public transit networks and new mobility services. This is precisely what is crucial for autonomous mobility: When the policy framework prioritizes accessibility, network quality, and new services, the resulting projects differ from those in a system that primarily rewards test mileage and visibility.

Source: More information

After all, this is crucial in public transportation, where the focus is on public services. What matters is whether a service reliably reaches people, improves mobility chains, and effectively complements public transportation.

If you measure the wrong things here, you’ll end up learning the wrong lessons. Then you’ll optimize projects that don’t hold up in practice, that don’t scale real benefits. And then the transition from pilot project to public system gets stuck exactly where it should actually begin: in an honest assessment of what really matters.

What matters is whether this results in a better public system—and whether that is precisely what is being measured.