A Little Help From My Friends It’s worth defining some terms. What self-driving-car developers usually call “remote operations” refer to a few different sorts of human jobs. There are, first, the operators who deal with other humans. These are people trained to interact with autonomous taxi riders when they have questions or need assistance in an emergency. Alphabet’s Waymo, the undisputed leader in self-driving, has a big Support button on its in-car passenger screens, which can connect riders with these folks. These operators can also be people trained to interact with law enforcement or emergency responders when they need assistance. Then there are operators who deal with the autonomous system. Some of these people may work in “remote assistance.” More confusion: Self-driving car developers give these tasks different names and titles. Amazon subsidiary Zoox uses “teleguidance”; the self-driving-truck developers at Aurora like “teleassistance”; Tesla appears to stick to “teleoperation”; Waymo calls these workers “fleet response agents.” Whatever they’re called, these humans are meant to guide the autonomous system when it needs help. A car might alert people, for example, when it has encountered a roadblock, like construction equipment; the remote assistant might suggest a lane change or a turn or even a quick dip onto a road’s shoulder to get around it. These human assistants can also sometimes reroute an autonomous vehicle if its planned itinerary no longer makes sense—maybe a road is unexpectedly closed off for a street fair. They can also sometimes help the car identify objects that have, for whatever reason, confused its sensors: a plastic bag flapping in the wind or a traffic light (red or green?). Got a Tip? Are you a former or current Tesla employee or know more about the company’s Robotaxi operations? WIRED would like to speak to you. Using a non-work phone or computer, please reach out to reporter Aarian Marshall on Signal at aarianm.30 Remote assistance should be a part of every safe self-driving-vehicle program, says Philip Koopman, a professor at Carnegie Mellon University who studies autonomous vehicle safety. “The technology is not there for them to be able to handle everything, and that’s OK,” he says. Having humans operate in the background of autonomous systems, then, isn’t “cheating” at self-driving. It’s understanding the limits of today’s technology—and what it takes to run a profitable business based on self-driving cars. Still, some remote assistance programs are safer than others, Koopman says. Some of that comes down to “triggers,” or how the system knows it needs help and turns to its human overlords. Is it the human assistant’s job to notice that the vehicle is stuck—or about to smash into something? Or is the onus on the vehicle to ask for aid? The safer bet, he says, is to train the technology to know when it needs intervention rather than relying on the vigilance of the human auxiliaries. Asking for help can be very hard for people—but it might be harder for robots. That’s because autonomous systems must automatically provide the humans with everything they need to know to assist: what happened, where the vehicle is in space, and if there’s anything around it. Only then can the humans guide the robots. A Tesla Cybercab prototype at a Tesla store in San Jose, California, in November 2024. Photograph: David Paul Morris/Getty Images Humans in the Loop If that sounds tough, even tougher—and more controversial—is “remote driving.” This is what most people probably think of when they hear “teleoperations”: Someone far away from the supposed self-driving car, behind their own steering wheel or joystick, piloting it like a long-distance RC car. Remote driving has even more technical challenges. Ben Shukman should know—as an engineer at a startup called Phantom Auto that focused on remote driving, he believes he was the first to do it on public roads. The first issue is connectivity. “Your ability to drive a car without being in the car is only as stable as the internet connection that connects you to it,” he says. But anyone who has called a friend on a long drive knows that networks drop in and out as you move in space. There are technical ways to knit together networks, but those aren’t foolproof. This leads to big issues with latency. So imagine the worst-case scenario: A robot car needs help navigating around an accident on a highway, a remote driver gets it moving, and then … the connection dies. Another challenge in remote driving: Helping drivers understand the experience of driving without actually being in the car. It’s hard to understand how quickly the momentum is shifting, or how hard you’re braking if you’re not inside the car. Shukman says it’s possible to build a user interface that gives remote drivers a sense of what it’s like on the road, but this takes thought and time. For these reasons, Shukman says, remote driving is less than safe in environments where vehicles are moving quickly in unpredictable environments, even above a handful of miles per hour. Today, the technique is mostly used in public to get delivery robots out of jams. Those move so slowly that a few milliseconds of dropped connection likely won’t spell disaster. The startup where he worked, Phantom Auto, eventually pivoted to operating forklifts remotely in warehouse settings. (It shut down last year, but its founders are sticking with its thesis: Their new startup keeps humans “in the loop” by building a platform that allows people to easily intervene to help in AI search.) The Tesla Question If Tesla does make good on its promise to start its Robotaxi service in Austin this month, how will it handle teleoperations? A Morgan Stanley research note from its head of global autos and shared mobility research Adam Jonas claims the service will be heavily teleoperated, though Tesla has not confirmed any of this. The job posting for software engineers working in teleoperation on “Optimus & Robotaxi” explicitly says the company’s remote operators will be “transported into the device’s world using a state-of-the-art VR rig” that will supposedly let them remotely execute intricate tasks requiring some form of human involvement. This sounds more like remote driving—direct intervention with the driving task—than remote assistance. If Tesla’s remote operators are meant to closely monitor its Robotaxis’ systems, “I think it’s going to be very difficult to expand beyond a few vehicles and a small area,” says Koopman, the professor. Or at least, to do it safely. In fact, that’s what CEO Elon Musk has said—that Tesla’s Robotaxi launch will start with just 10 to 20 cars and will expand from there. Maybe there will be some robots. But the better questions are what the humans supporting them will be doing—and whether they’ll be doing it safely.