The 2004 DARPA Grand Challenge was a spectacular failure. The Defense Advanced Research Projects Agency had offered a US $1 million prize for the team that could design an autonomous ground vehicle capable of completing an off-road course through sometimes flat, sometimes winding and mountainous desert terrain. As IEEE Spectrum reported at the time, it was “the motleyest assortment of vehicles assembled in one place since the filming of Mad Max 2: The Road Warrior.” Not a single entrant made it across the finish line. Some didn’t make it out of the parking lot.

Videos of the attempts are comical, although any laughter comes at the expense of the many engineers who spent countless hours and millions of dollars to get to that point.

So it’s all the more remarkable that in the second DARPA Grand Challenge, just a year and a half later, five vehicles crossed the finish line. Stanley, developed by the Stanford Racing Team, eked out a first-place win to claim the $2 million purse. This modified Volkswagen Touareg [shown at top] completed the 212-kilometer course in 6 hours, 54 minutes. Carnegie Mellon’s Sandstorm and H1ghlander took second and third place, respectively, with times of 7:05 and 7:14.

Kat-5, sponsored by the Gray Insurance Co. of Metairie, La., came in fourth with a respectable 7:30. The vehicle was named after Hurricane Katrina, which had just pummeled the Gulf Coast a month and a half earlier. Oshkosh Truck’s TerraMax also finished the circuit, although its time of 12:51 exceeded the 10-hour time limit set by DARPA.

So how did the Grand Challenge go from a total bust to having five robust finishers in such a short period of time? It’s definitely a testament to what can be accomplished when engineers rise to a challenge. But the outcome of this one race was preceded by a much longer path of research, and that plus a little bit of luck are what ultimately led to victory.

Let’s back up to 1998, when computer scientist Sebastian Thrun was working at Carnegie Mellon and experimenting with a very different robot: a museum tour guide. For two weeks in the summer, Minerva, which looked a bit like a Dalek from “Doctor Who,” navigated an exhibit at the Smithsonian National Museum of American History. Its main task was to roll around and dispense nuggets of information about the displays.

Minerva was a museum tour-guide robot developed by Sebastian Thrun.

In an interview at the time, Thrun acknowledged that Minerva was there to entertain. But Minerva wasn’t just a people pleaser ; it was also a machine learning experiment. It had to learn where it could safely maneuver without taking out a visitor or a priceless artifact. Visitor, nonvisitor; display case, not-display case; open floor, not-open floor. It had to react to humans crossing in front of it in unpredictable ways. It had to learn to “see.”

Fast-forward five years: Thrun transferred to Stanford in July 2003. Inspired by the first Grand Challenge, he organized the Stanford Racing Team with the aim of fielding a robotic car in the second competition.

In a vast oversimplification of Stanley’s main task, the autonomous robot had to differentiate between road and not-road in order to navigate the route successfully. The Stanford team decided to focus its efforts on developing software and used as much off-the-shelf hardware as they could, including a laser to scan the immediate terrain and a simple video camera to scan the horizon. Software overlapped the two inputs, adapted to the changing road conditions on the fly, and determined a safe driving speed. (For more technical details on Stanley, check out the team’s paper.) A remote-control kill switch, which DARPA required on all vehicles, would deactivate the car before it could become a danger. About 100,000 lines of code did that and much more.

The Stanford team hadn’t entered the 2004 Grand Challenge and wasn’t expected to win the 2005 race. Carnegie Mellon, meanwhile, had two entries—a modified 1986 Humvee and a modified 1999 Hummer—and was the clear favorite. In the 2004 race, CMU’s Sandstorm had gone furthest, completing 12 km. For the second race, CMU brought an improved Sandstorm as well as a new vehicle, H1ghlander.

Many of the other 2004 competitors regrouped to try again, and new ones entered the fray. In all, 195 teams applied to compete in the 2005 event. Teams included students, academics, industry experts, and hobbyists.

After site visits in the spring, 43 teams made it to the qualifying event, held 27 September through 5 October at the California Speedway, in Fontana. Each vehicle took four runs through the course, navigating through checkpoints and avoiding obstacles. A total of 23 teams were selected to attempt the main course across the Mojave Desert. Competing was a costly endeavor—CMU’s Red Team spent more than $3 million in its first year—and the names of sponsors were splashed across the vehicles like the logos on race cars.

In the early hours of 8 October, the finalists gathered for the big race. Each team had a staggered start time to help avoid congestion along the route. About two hours before a team’s start, DARPA gave them a CD containing approximately 3,000 GPS coordinates representing the course. Once the team hit go, it was hands off: The car had to drive itself without any human intervention. PBS’s NOVA produced an excellent episode on the 2004 and 2005 Grand Challenges that I highly recommend if you want to get a feel for the excitement, anticipation, disappointment, and triumph.

In the 2005 Grand Challenge, Carnegie Mellon University’s H1ghlander was one of five autonomous cars to finish the race.Damian Dovarganes/AP

H1ghlander held the pole position, having placed first in the qualifying rounds, followed by Stanley and Sandstorm. H1ghlander pulled ahead early and soon had a substantial lead. That’s where luck, or rather the lack of it, came in.

About two hours into the race, H1ghlander slowed down and started rolling backward down a hill. Although it eventually resumed moving forward, it never regained its top speed, even on long, straight, level sections of the course. The slower but steadier Stanley caught up to H1ghlander at the 163-km (101.5-mile) marker, passed it, and never let go of the lead.

What went wrong with H1ghlander remained a mystery, even after extensive postrace analysis. It wasn’t until 12 years after the race—and once again with a bit of luck—that CMU discovered the problem: Pressing on a small electronic filter between the engine control module and the fuel injector caused the engine to lose power and even turn off. Team members speculated that an accident a few weeks before the competition had damaged the filter. (To learn more about how CMU finally figured this out, see Spectrum Senior Editor Evan Ackerman’s 2017 story.)

Regardless of who won the Grand Challenge, many success stories came out of the contest. A year and a half after the race, Thrun had already made great progress on adaptive cruise control and lane-keeping assistance, which is now readily available on many commercial vehicles. He then worked on Google’s Street View and its initial self-driving cars. CMU’s Red Team worked with NASA to develop rovers for potentially exploring the moon or distant planets. Closer to home, they helped develop self-propelled harvesters for the agricultural sector.

Stanford team leader Sebastian Thrun holds a $2 million check, the prize for winning the 2005 Grand Challenge.Damian Dovarganes/AP

Of course, there was also a lot of hype, which tended to overshadow the race’s militaristic origins—remember, the “D” in DARPA stands for “defense.” Back in 2000, a defense authorization bill had stipulated that one-third of the U.S. ground combat vehicles be “unmanned” by 2015, and DARPA conceived of the Grand Challenge to spur development of these autonomous vehicles. The U.S. military was still fighting in the Middle East, and DARPA promoters believed self-driving vehicles would help minimize casualties, particularly those caused by improvised explosive devices.

DARPA sponsored more contests, such as the 2007 Urban Challenge, in which vehicles navigated a simulated city and suburban environment; the 2012 Robotics Challenge for disaster-response robots; and the 2022 Subterranean Challenge for—you guessed it—robots that could get around underground. Despite the competitions, continued military conflicts, and hefty government contracts, actual advances in autonomous military vehicles and robots did not take off to the extent desired. As of 2023, robotic ground vehicles made up only 3 percent of the global armored-vehicle market.

Today, there are very few fully autonomous ground vehicles in the U.S. military; instead, the services have forged ahead with semiautonomous, operator-assisted systems, such as remote-controlled drones and ship autopilots. The one Grand Challenge finisher that continued to work for the U.S. military was Oshkosh Truck, the Wisconsin-based sponsor of the TerraMax. The company demonstrated a palletized loading system to transport cargo in unmanned vehicles for the U.S. Army.

Much of the contemporary reporting on the Grand Challenge predicted that self-driving cars would take us closer to a “Jetsons” future, with a self-driving vehicle to ferry you around. But two decades after Stanley, the rollout of civilian autonomous cars has been confined to specific applications, such as Waymo robotaxis transporting people around San Francisco or the GrubHub Starships struggling to deliver food across my campus at the University of South Carolina.

I’ll be watching to see how the technology evolves outside of big cities. Self-driving vehicles would be great for long distances on empty country roads, but parts of rural America still struggle to get adequate cellphone coverage. Will small towns and the spaces that surround them have the bandwidth to accommodate autonomous vehicles? As much as I’d like to think self-driving autos are nearly here, I don’t expect to find one under my carport anytime soon.

Not long after the 2005 race, Stanley was ready to retire. Recalling his experience testing Minerva at the National Museum of American History, Thrun thought the museum would make a nice home. He loaned it to the museum in 2006, and since 2008 it has resided permanently in the museum’s collections, alongside other remarkable specimens in robotics and automobiles. In fact, it isn’t even the first Stanley in the collection.

Stanley now resides in the collections of the Smithsonian Institution’s National Museum of American History, which also houses another Stanley—this 1910 Stanley Runabout. Behring Center/National Museum of American History/Smithsonian Institution

That distinction belongs to a 1910 Stanley Runabout, an early steam-powered car introduced at a time when it wasn’t yet clear that the internal-combustion engine was the way to go. Despite clear drawbacks—steam engines had a nasty tendency to explode—“Stanley steamers” were known for their fine craftsmanship. Fred Marriott set the land speed record while driving a Stanley in 1906. It clocked in at 205.5 kilometers per hour, which was significantly faster than the 21st-century Stanley’s average speed of 30.7 km/hr. To be fair, Marriott’s Stanley was racing over a flat, straight course rather than the off-road terrain navigated by Thrun’s Stanley.

Across the century that separates the two Stanleys, it’s easy to trace a narrative of progress. Both are clearly recognizable as four-wheeled land vehicles, but I suspect the science-fiction dreamers of the early 20th century would have been hard-pressed to imagine the suite of technologies that would propel a 21st-century self-driving car. What will the vehicles of the early 22nd century be like? Will they even have four tires, or will they run on something entirely new?

Part of a continuing series looking at historical artifacts that embrace the boundless potential of technology.

An abridged version of this article appears in the February 2025 print issue as “Slow and Steady Wins the Race.”

Sebastian Thrun and his colleagues at the Stanford Artificial Intelligence Laboratory, along with members of the other groups that sponsored Stanley, published “Stanley: The Robot That Won the DARPA Grand Challenge.” This paper, from the Journal of Field Robotics, explains the vehicle’s development.

The NOVA PBS episode “The Great Robot Race” provides interviews and video footage from both the failed first Grand Challenge and the successful second one. I personally liked the side story of GhostRider, an autonomous motorcycle that competed in both competitions but didn’t quite cut it. (GhostRider also now resides in the Smithsonian’s collection.)

Smithsonian curator Carlene Stephens kindly talked with me about how she collected Stanley for the National Museum of American History and where she sees artifacts like this fitting into the stream of history.

The 2004 DARPA Grand Challenge was a spectacular failure. The Defense Advanced Research Projects Agency had offered a US $1 million prize for the team that could design an autonomous ground vehicle capable of completing an off-road course through sometimes flat, sometimes winding and mountainous desert terrain. As IEEE Spectrum reported at the time, it was “the motleyest assortment of vehicles assembled in one place since the filming of Mad Max 2: The Road Warrior.” Not a single entrant made it across the finish line. Some didn’t make it out of the parking lot.

Videos of the attempts are comical, although any laughter comes at the expense of the many engineers who spent countless hours and millions of dollars to get to that point.

So it’s all the more remarkable that in the second DARPA Grand Challenge, just a year and a half later, five vehicles crossed the finish line. Stanley, developed by the Stanford Racing Team, eked out a first-place win to claim the $2 million purse. This modified Volkswagen Touareg [shown at top] completed the 212-kilometer course in 6 hours, 54 minutes. Carnegie Mellon’s Sandstorm and H1ghlander took second and third place, respectively, with times of 7:05 and 7:14.

Kat-5, sponsored by the Gray Insurance Co. of Metairie, La., came in fourth with a respectable 7:30. The vehicle was named after Hurricane Katrina, which had just pummeled the Gulf Coast a month and a half earlier. Oshkosh Truck’s TerraMax also finished the circuit, although its time of 12:51 exceeded the 10-hour time limit set by DARPA.

So how did the Grand Challenge go from a total bust to having five robust finishers in such a short period of time? It’s definitely a testament to what can be accomplished when engineers rise to a challenge. But the outcome of this one race was preceded by a much longer path of research, and that plus a little bit of luck are what ultimately led to victory.

Let’s back up to 1998, when computer scientist Sebastian Thrun was working at Carnegie Mellon and experimenting with a very different robot: a museum tour guide. For two weeks in the summer, Minerva, which looked a bit like a Dalek from “Doctor Who,” navigated an exhibit at the Smithsonian National Museum of American History. Its main task was to roll around and dispense nuggets of information about the displays.

Minerva was a museum tour-guide robot developed by Sebastian Thrun.

In an interview at the time, Thrun acknowledged that Minerva was there to entertain. But Minerva wasn’t just a people pleaser ; it was also a machine learning experiment. It had to learn where it could safely maneuver without taking out a visitor or a priceless artifact. Visitor, nonvisitor; display case, not-display case; open floor, not-open floor. It had to react to humans crossing in front of it in unpredictable ways. It had to learn to “see.”

Fast-forward five years: Thrun transferred to Stanford in July 2003. Inspired by the first Grand Challenge, he organized the Stanford Racing Team with the aim of fielding a robotic car in the second competition.

In a vast oversimplification of Stanley’s main task, the autonomous robot had to differentiate between road and not-road in order to navigate the route successfully. The Stanford team decided to focus its efforts on developing software and used as much off-the-shelf hardware as they could, including a laser to scan the immediate terrain and a simple video camera to scan the horizon. Software overlapped the two inputs, adapted to the changing road conditions on the fly, and determined a safe driving speed. (For more technical details on Stanley, check out the team’s paper.) A remote-control kill switch, which DARPA required on all vehicles, would deactivate the car before it could become a danger. About 100,000 lines of code did that and much more.

The Stanford team hadn’t entered the 2004 Grand Challenge and wasn’t expected to win the 2005 race. Carnegie Mellon, meanwhile, had two entries—a modified 1986 Humvee and a modified 1999 Hummer—and was the clear favorite. In the 2004 race, CMU’s Sandstorm had gone furthest, completing 12 km. For the second race, CMU brought an improved Sandstorm as well as a new vehicle, H1ghlander.

Many of the other 2004 competitors regrouped to try again, and new ones entered the fray. In all, 195 teams applied to compete in the 2005 event. Teams included students, academics, industry experts, and hobbyists.

After site visits in the spring, 43 teams made it to the qualifying event, held 27 September through 5 October at the California Speedway, in Fontana. Each vehicle took four runs through the course, navigating through checkpoints and avoiding obstacles. A total of 23 teams were selected to attempt the main course across the Mojave Desert. Competing was a costly endeavor—CMU’s Red Team spent more than $3 million in its first year—and the names of sponsors were splashed across the vehicles like the logos on race cars.

In the early hours of 8 October, the finalists gathered for the big race. Each team had a staggered start time to help avoid congestion along the route. About two hours before a team’s start, DARPA gave them a CD containing approximately 3,000 GPS coordinates representing the course. Once the team hit go, it was hands off: The car had to drive itself without any human intervention. PBS’s NOVA produced an excellent episode on the 2004 and 2005 Grand Challenges that I highly recommend if you want to get a feel for the excitement, anticipation, disappointment, and triumph.

In the 2005 Grand Challenge, Carnegie Mellon University’s H1ghlander was one of five autonomous cars to finish the race.Damian Dovarganes/AP

H1ghlander held the pole position, having placed first in the qualifying rounds, followed by Stanley and Sandstorm. H1ghlander pulled ahead early and soon had a substantial lead. That’s where luck, or rather the lack of it, came in.

About two hours into the race, H1ghlander slowed down and started rolling backward down a hill. Although it eventually resumed moving forward, it never regained its top speed, even on long, straight, level sections of the course. The slower but steadier Stanley caught up to H1ghlander at the 163-km (101.5-mile) marker, passed it, and never let go of the lead.

What went wrong with H1ghlander remained a mystery, even after extensive postrace analysis. It wasn’t until 12 years after the race—and once again with a bit of luck—that CMU discovered the problem: Pressing on a small electronic filter between the engine control module and the fuel injector caused the engine to lose power and even turn off. Team members speculated that an accident a few weeks before the competition had damaged the filter. (To learn more about how CMU finally figured this out, see Spectrum Senior Editor Evan Ackerman’s 2017 story.)

Regardless of who won the Grand Challenge, many success stories came out of the contest. A year and a half after the race, Thrun had already made great progress on adaptive cruise control and lane-keeping assistance, which is now readily available on many commercial vehicles. He then worked on Google’s Street View and its initial self-driving cars. CMU’s Red Team worked with NASA to develop rovers for potentially exploring the moon or distant planets. Closer to home, they helped develop self-propelled harvesters for the agricultural sector.

Stanford team leader Sebastian Thrun holds a $2 million check, the prize for winning the 2005 Grand Challenge.Damian Dovarganes/AP

Of course, there was also a lot of hype, which tended to overshadow the race’s militaristic origins—remember, the “D” in DARPA stands for “defense.” Back in 2000, a defense authorization bill had stipulated that one-third of the U.S. ground combat vehicles be “unmanned” by 2015, and DARPA conceived of the Grand Challenge to spur development of these autonomous vehicles. The U.S. military was still fighting in the Middle East, and DARPA promoters believed self-driving vehicles would help minimize casualties, particularly those caused by improvised explosive devices.

DARPA sponsored more contests, such as the 2007 Urban Challenge, in which vehicles navigated a simulated city and suburban environment; the 2012 Robotics Challenge for disaster-response robots; and the 2022 Subterranean Challenge for—you guessed it—robots that could get around underground. Despite the competitions, continued military conflicts, and hefty government contracts, actual advances in autonomous military vehicles and robots did not take off to the extent desired. As of 2023, robotic ground vehicles made up only 3 percent of the global armored-vehicle market.

Today, there are very few fully autonomous ground vehicles in the U.S. military; instead, the services have forged ahead with semiautonomous, operator-assisted systems, such as remote-controlled drones and ship autopilots. The one Grand Challenge finisher that continued to work for the U.S. military was Oshkosh Truck, the Wisconsin-based sponsor of the TerraMax. The company demonstrated a palletized loading system to transport cargo in unmanned vehicles for the U.S. Army.

Much of the contemporary reporting on the Grand Challenge predicted that self-driving cars would take us closer to a “Jetsons” future, with a self-driving vehicle to ferry you around. But two decades after Stanley, the rollout of civilian autonomous cars has been confined to specific applications, such as Waymo robotaxis transporting people around San Francisco or the GrubHub Starships struggling to deliver food across my campus at the University of South Carolina.

I’ll be watching to see how the technology evolves outside of big cities. Self-driving vehicles would be great for long distances on empty country roads, but parts of rural America still struggle to get adequate cellphone coverage. Will small towns and the spaces that surround them have the bandwidth to accommodate autonomous vehicles? As much as I’d like to think self-driving autos are nearly here, I don’t expect to find one under my carport anytime soon.

Not long after the 2005 race, Stanley was ready to retire. Recalling his experience testing Minerva at the National Museum of American History, Thrun thought the museum would make a nice home. He loaned it to the museum in 2006, and since 2008 it has resided permanently in the museum’s collections, alongside other remarkable specimens in robotics and automobiles. In fact, it isn’t even the first Stanley in the collection.

Stanley now resides in the collections of the Smithsonian Institution’s National Museum of American History, which also houses another Stanley—this 1910 Stanley Runabout. Behring Center/National Museum of American History/Smithsonian Institution

That distinction belongs to a 1910 Stanley Runabout, an early steam-powered car introduced at a time when it wasn’t yet clear that the internal-combustion engine was the way to go. Despite clear drawbacks—steam engines had a nasty tendency to explode—“Stanley steamers” were known for their fine craftsmanship. Fred Marriott set the land speed record while driving a Stanley in 1906. It clocked in at 205.5 kilometers per hour, which was significantly faster than the 21st-century Stanley’s average speed of 30.7 km/hr. To be fair, Marriott’s Stanley was racing over a flat, straight course rather than the off-road terrain navigated by Thrun’s Stanley.

Across the century that separates the two Stanleys, it’s easy to trace a narrative of progress. Both are clearly recognizable as four-wheeled land vehicles, but I suspect the science-fiction dreamers of the early 20th century would have been hard-pressed to imagine the suite of technologies that would propel a 21st-century self-driving car. What will the vehicles of the early 22nd century be like? Will they even have four tires, or will they run on something entirely new?

Part of a continuing series looking at historical artifacts that embrace the boundless potential of technology.

An abridged version of this article appears in the February 2025 print issue as “Slow and Steady Wins the Race.”

Sebastian Thrun and his colleagues at the Stanford Artificial Intelligence Laboratory, along with members of the other groups that sponsored Stanley, published “Stanley: The Robot That Won the DARPA Grand Challenge.” This paper, from the Journal of Field Robotics, explains the vehicle’s development.

The NOVA PBS episode “The Great Robot Race” provides interviews and video footage from both the failed first Grand Challenge and the successful second one. I personally liked the side story of GhostRider, an autonomous motorcycle that competed in both competitions but didn’t quite cut it. (GhostRider also now resides in the Smithsonian’s collection.)

Smithsonian curator Carlene Stephens kindly talked with me about how she collected Stanley for the National Museum of American History and where she sees artifacts like this fitting into the stream of history.

Video Friday is your weekly selection of awesome robotics videos, collected by your friends at IEEE Spectrum robotics. We also post a weekly calendar of upcoming robotics events for the next few months. Please send us your events for inclusion.

Enjoy today’s videos!

This video about ‘foster’ Aibos helping kids at a children’s hospital is well worth turning on auto-translated subtitles for.

[ Aibo Foster Program ]

[ Unitree ]

Happy Chinese New Year from PNDbotics!

[ PNDbotics ]

[ LimX Dynamics ]

Mr. Bucket is my favorite.

[ Mitsubishi Electric Research Laboratories ]

Thanks, Yuki!

[ Zoox ]

The Humanoids Summit at the Computer History Museum in December was successful enough (either because of or in spite of my active participation) that it’s not only happening again in 2025: There’s also going to be a spring version of the conference in London in May!

[ Humanoids Summit ]

I’m not sure it’ll ever be practical at scale, but I do really like JSK’s musculoskeletal humanoid work.

[ Paper ]

[ Canadian Space Agency ]

[ Corvus Robotics ]

Thanks, Joan!

I would have said that this controller was too small to be manipulated with a pinch grasp. I would be wrong.

[ Pollen ]

[ NASA ]

[ Team BlackSheep ]

[ Zoox ]

Video Friday is your weekly selection of awesome robotics videos, collected by your friends at IEEE Spectrum robotics. We also post a weekly calendar of upcoming robotics events for the next few months. Please send us your events for inclusion.

Enjoy today’s videos!

This video about ‘foster’ Aibos helping kids at a children’s hospital is well worth turning on auto-translated subtitles for.

[ Aibo Foster Program ]

[ Unitree ]

Happy Chinese New Year from PNDbotics!

[ PNDbotics ]

[ LimX Dynamics ]

Mr. Bucket is my favorite.

[ Mitsubishi Electric Research Laboratories ]

Thanks, Yuki!

[ Zoox ]

The Humanoids Summit at the Computer History Museum in December was successful enough (either because of or in spite of my active participation) that it’s not only happening again in 2025: There’s also going to be a spring version of the conference in London in May!

[ Humanoids Summit ]

I’m not sure it’ll ever be practical at scale, but I do really like JSK’s musculoskeletal humanoid work.

[ Paper ]

[ Canadian Space Agency ]

[ Corvus Robotics ]

Thanks, Joan!

I would have said that this controller was too small to be manipulated with a pinch grasp. I would be wrong.

[ Pollen ]

[ NASA ]

[ Team BlackSheep ]

[ Zoox ]

Most people know that robots no longer sound like tinny trash cans. They sound like Siri, Alexa, and Gemini. They sound like the voices in labyrinthine customer support phone trees. And even those robot voices are being made obsolete by new AI-generated voices that can mimic every vocal nuance and tic of human speech, down to specific regional accents. And with just a few seconds of audio, AI can now clone someone’s specific voice.

This technology will replace humans in many areas. Automated customer support will save money by cutting staffing at call centers. AI agents will make calls on our behalf, conversing with others in natural language. All of that is happening, and will be commonplace soon.

But there is something fundamentally different about talking with a bot as opposed to a person. A person can be a friend. An AI cannot be a friend, despite how people might treat it or react to it. AI is at best a tool, and at worst a means of manipulation. Humans need to know whether we’re talking with a living, breathing person or a robot with an agenda set by the person who controls it. That’s why robots should sound like robots.

You can’t just label AI-generated speech. It will come in many different forms. So we need a way to recognize AI that works no matter the modality. It needs to work for long or short snippets of audio, even just a second long. It needs to work for any language, and in any cultural context. At the same time, we shouldn’t constrain the underlying system’s sophistication or language complexity.

We have a simple proposal: all talking AIs and robots should use a ring modulator. In the mid-twentieth century, before it was easy to create actual robotic-sounding speech synthetically, ring modulators were used to make actors’ voices sound robotic. Over the last few decades, we have become accustomed to robotic voices, simply because text-to-speech systems were good enough to produce intelligible speech that was not human-like in its sound. Now we can use that same technology to make robotic speech that is indistinguishable from human sound robotic again.

A ring modulator has several advantages: It is computationally simple, can be applied in real-time, does not affect the intelligibility of the voice, and--most importantly--is universally “robotic sounding” because of its historical usage for depicting robots.

Responsible AI companies that provide voice synthesis or AI voice assistants in any form should add a ring modulator of some standard frequency (say, between 30-80 Hz) and of a minimum amplitude (say, 20 percent). That’s it. People will catch on quickly.

Here are a couple of examples you can listen to for examples of what we’re suggesting. The first clip is an AI-generated “podcast” of this article made by Google’s NotebookLM featuring two AI “hosts.” Google’s NotebookLM created the podcast script and audio given only the text of this article. The next two clips feature that same podcast with the AIs’ voices modulated more and less subtly by a ring modulator:

Raw audio sample generated by Google’s NotebookLM Your browser does not support the audio element.

Audio sample with added ring modulator (30 Hz-25%) Your browser does not support the audio element.

Audio sample with added ring modulator (30 Hz-40%) Your browser does not support the audio element.

We were able to generate the audio effect with a 50-line Python script generated by Anthropic’s Claude. One of the most well-known robot voices were those of the Daleks from Doctor Who in the 1960s. Back then robot voices were difficult to synthesize, so the audio was actually an actor’s voice run through a ring modulator. It was set to around 30 Hz, as we did in our example, with different modulation depth (amplitude) depending on how strong the robotic effect is meant to be. Our expectation is that the AI industry will test and converge on a good balance of such parameters and settings, and will use better tools than a 50-line Python script, but this highlights how simple it is to achieve.

Of course there will also be nefarious uses of AI voices. Scams that use voice cloning have been getting easier every year, but they’ve been possible for many years with the right know-how. Just like we’re learning that we can no longer trust images and videos we see because they could easily have been AI-generated, we will all soon learn that someone who sounds like a family member urgently requesting money may just be a scammer using a voice-cloning tool.

We don’t expect scammers to follow our proposal: They’ll find a way no matter what. But that’s always true of security standards, and a rising tide lifts all boats. We think the bulk of the uses will be with popular voice APIs from major companies--and everyone should know that they’re talking with a robot.

Most people know that robots no longer sound like tinny trash cans. They sound like Siri, Alexa, and Gemini. They sound like the voices in labyrinthine customer support phone trees. And even those robot voices are being made obsolete by new AI-generated voices that can mimic every vocal nuance and tic of human speech, down to specific regional accents. And with just a few seconds of audio, AI can now clone someone’s specific voice.

This technology will replace humans in many areas. Automated customer support will save money by cutting staffing at call centers. AI agents will make calls on our behalf, conversing with others in natural language. All of that is happening, and will be commonplace soon.

But there is something fundamentally different about talking with a bot as opposed to a person. A person can be a friend. An AI cannot be a friend, despite how people might treat it or react to it. AI is at best a tool, and at worst a means of manipulation. Humans need to know whether we’re talking with a living, breathing person or a robot with an agenda set by the person who controls it. That’s why robots should sound like robots.

You can’t just label AI-generated speech. It will come in many different forms. So we need a way to recognize AI that works no matter the modality. It needs to work for long or short snippets of audio, even just a second long. It needs to work for any language, and in any cultural context. At the same time, we shouldn’t constrain the underlying system’s sophistication or language complexity.

We have a simple proposal: all talking AIs and robots should use a ring modulator. In the mid-twentieth century, before it was easy to create actual robotic-sounding speech synthetically, ring modulators were used to make actors’ voices sound robotic. Over the last few decades, we have become accustomed to robotic voices, simply because text-to-speech systems were good enough to produce intelligible speech that was not human-like in its sound. Now we can use that same technology to make robotic speech that is indistinguishable from human sound robotic again.

A ring modulator has several advantages: It is computationally simple, can be applied in real-time, does not affect the intelligibility of the voice, and--most importantly--is universally “robotic sounding” because of its historical usage for depicting robots.

Responsible AI companies that provide voice synthesis or AI voice assistants in any form should add a ring modulator of some standard frequency (say, between 30-80 Hz) and of a minimum amplitude (say, 20 percent). That’s it. People will catch on quickly.

Here are a couple of examples you can listen to for examples of what we’re suggesting. The first clip is an AI-generated “podcast” of this article made by Google’s NotebookLM featuring two AI “hosts.” Google’s NotebookLM created the podcast script and audio given only the text of this article. The next two clips feature that same podcast with the AIs’ voices modulated more and less subtly by a ring modulator:

Raw audio sample generated by Google’s NotebookLM Your browser does not support the audio element.

Audio sample with added ring modulator (30 Hz-25%) Your browser does not support the audio element.

Audio sample with added ring modulator (30 Hz-40%) Your browser does not support the audio element.

We were able to generate the audio effect with a 50-line Python script generated by Anthropic’s Claude. One of the most well-known robot voices were those of the Daleks from Doctor Who in the 1960s. Back then robot voices were difficult to synthesize, so the audio was actually an actor’s voice run through a ring modulator. It was set to around 30 Hz, as we did in our example, with different modulation depth (amplitude) depending on how strong the robotic effect is meant to be. Our expectation is that the AI industry will test and converge on a good balance of such parameters and settings, and will use better tools than a 50-line Python script, but this highlights how simple it is to achieve.

Of course there will also be nefarious uses of AI voices. Scams that use voice cloning have been getting easier every year, but they’ve been possible for many years with the right know-how. Just like we’re learning that we can no longer trust images and videos we see because they could easily have been AI-generated, we will all soon learn that someone who sounds like a family member urgently requesting money may just be a scammer using a voice-cloning tool.

We don’t expect scammers to follow our proposal: They’ll find a way no matter what. But that’s always true of security standards, and a rising tide lifts all boats. We think the bulk of the uses will be with popular voice APIs from major companies--and everyone should know that they’re talking with a robot.

This article is part of our exclusive IEEE Journal Watch series in partnership with IEEE Xplore.

Swarms of autonomous robots are increasingly being tested and deployed in complex missions, yet a certain level of human oversight during these missions is still required. Which means a major question remains: How many robots—and how complex a mission—can a single human manage before becoming overwhelmed?

In a study funded by the U.S. Defense Advanced Research Projects Agency (DARPA), experts show that humans can single-handedly and effectively manage a heterogenous swarm of more than 100 autonomous ground and aerial vehicles, while feeling overwhelmed only for brief periods of time during an overall small portion of the mission. For instance, in a particularly challenging, multi-day experiment in an urban setting, human controllers were overloaded with stress and workload only three percent of the time. The results were published 19 November in IEEE Transactions on Field Robotics.

Julie A. Adams, the associate director of research at Oregon State University’s Collaborative Robotics and Intelligent Systems Institute, has been studying human interactions with robots and other complex systems, such as aircraft cockpits and nuclear power plant control rooms, for 35 years. She notes that robot swarms can be used to support missions where work may be particularly dangerous and hazardous for humans, such as monitoring wildfires.

“Swarms can be used to provide persistent coverage of an area, such as monitoring for new fires or looters in the recently burned areas of Los Angeles,” Adams says. “The information can be used to direct limited assets, such as firefighting units or water tankers to new fires and hotspots, or to locations at which fires were thought to have been extinguished.”

These kinds of missions can involve a mix of many different kinds of unmanned ground vehicles (such as the Aion Robotics R1 wheeled robot) and aerial autonomous vehicles (like the Modal AI VOXL M500 quadcopter), and a human controller may need to reassign individual robots to different tasks as the mission unfolds. Notably, some theories over the past few decades—and even Adams’ early thesis work—suggest that a single human has limited capacity to deploy very large numbers of robots.

“These historical theories and the associated empirical results showed that as the number of ground robots increased, so did the human’s workload, which often resulted in reduced overall performance,” says Adams, noting that, although earlier research focused on unmanned ground vehicles (UGVs), which must deal with curbs and other physical barriers, unmanned aerial vehicles (UAVs) often encounter fewer physical barriers.

Human controllers managed their swarms of autonomous vehicles with a virtual display. The fuschia ring represents the area the person could see within their head-mounted display.DARPA

As part of DARPA’s OFFensive Swarm-Enabled Tactics (OFFSET) program, Adams and her colleagues sought to explore whether these theories applied to very complex missions involving a mix of unmanned ground and air vehicles. In November 2021, at Fort Campbell in Kentucky, two human controllers took turns engaging in a series of missions over the course of three weeks with the objective of neutralizing an adversarial target. Both human controllers had significant experience controlling swarms, and participated in alternating shifts that ranged from 1.5 to 3 hours per day.

During the tests, the human controllers were positioned in a designated area on the edge of the testing site, and used a virtual reconstruction of the environment to keep tabs on where vehicles were and what tasks they were assigned to.

The largest mission shift involved 110 drones, 30 ground vehicles, and up to 50 virtual vehicles representing additional real-world vehicles. The robots had to navigate through the physical urban environment, as well as a series of virtual hazards represented using AprilTags—simplified QR codes that could represent imaginary hazards—that were scattered throughout the mission site.

DARPA made the final field exercise exceptionally challenging by providing thousands of hazards and pieces of information to inform the search. “The complexity of the hazards was significant,” Adams says, noting that some hazards required multiple robots to interact with them simultaneously, and some hazards moved around the environment.

Throughout each mission shift, the human controller’s physiological responses to the tasks at hand were monitored. For example, sensors collected data on their heart-rate variability, posture, and even their speech rate. The data were input into an established algorithm that estimates workload levels and was used to determine when the controller was reaching a workload level that exceeded a normal range, called an “overload state.”

Adams notes that, despite the complexity and large volume of robots to manage in this field exercise, the number and duration of overload state instances were relatively short—a handful of minutes during a mission shift. “The total percentage of estimated overload states was 3 percent of all workload estimates across all shifts for which we collected data,” she says.

www.youtube.com

The most common reason for a human commander to reach an overload state is when they had to generate multiple new tactics or inspect which vehicles in the launch zone were available for deployment.

Adams notes that these finding suggest that—counter to past theories—the number of robots may be less influential on human swarm control performance than previously thought. Her team is exploring the other factors that may impact swarm control missions, such as other human limitations, system designs and UAS designs, the results of which will potentially inform US Federal Aviation Administration drone regulations, she says.

This article is part of our exclusive IEEE Journal Watch series in partnership with IEEE Xplore.

Swarms of autonomous robots are increasingly being tested and deployed in complex missions, yet a certain level of human oversight during these missions is still required. Which means a major question remains: How many robots—and how complex a mission—can a single human manage before becoming overwhelmed?

In a study funded by the U.S. Defense Advanced Research Projects Agency (DARPA), experts show that humans can single-handedly and effectively manage a heterogenous swarm of more than 100 autonomous ground and aerial vehicles, while feeling overwhelmed only for brief periods of time during an overall small portion of the mission. For instance, in a particularly challenging, multi-day experiment in an urban setting, human controllers were overloaded with stress and workload only three percent of the time. The results were published 19 November in IEEE Transactions on Field Robotics.

Julie A. Adams, the associate director of research at Oregon State University’s Collaborative Robotics and Intelligent Systems Institute, has been studying human interactions with robots and other complex systems, such as aircraft cockpits and nuclear power plant control rooms, for 35 years. She notes that robot swarms can be used to support missions where work may be particularly dangerous and hazardous for humans, such as monitoring wildfires.

“Swarms can be used to provide persistent coverage of an area, such as monitoring for new fires or looters in the recently burned areas of Los Angeles,” Adams says. “The information can be used to direct limited assets, such as firefighting units or water tankers to new fires and hotspots, or to locations at which fires were thought to have been extinguished.”

These kinds of missions can involve a mix of many different kinds of unmanned ground vehicles (such as the Aion Robotics R1 wheeled robot) and aerial autonomous vehicles (like the Modal AI VOXL M500 quadcopter), and a human controller may need to reassign individual robots to different tasks as the mission unfolds. Notably, some theories over the past few decades—and even Adams’ early thesis work—suggest that a single human has limited capacity to deploy very large numbers of robots.

“These historical theories and the associated empirical results showed that as the number of ground robots increased, so did the human’s workload, which often resulted in reduced overall performance,” says Adams, noting that, although earlier research focused on unmanned ground vehicles (UGVs), which must deal with curbs and other physical barriers, unmanned aerial vehicles (UAVs) often encounter fewer physical barriers.

Human controllers managed their swarms of autonomous vehicles with a virtual display. The fuschia ring represents the area the person could see within their head-mounted display.DARPA

As part of DARPA’s OFFensive Swarm-Enabled Tactics (OFFSET) program, Adams and her colleagues sought to explore whether these theories applied to very complex missions involving a mix of unmanned ground and air vehicles. In November 2021, at Fort Campbell in Kentucky, two human controllers took turns engaging in a series of missions over the course of three weeks with the objective of neutralizing an adversarial target. Both human controllers had significant experience controlling swarms, and participated in alternating shifts that ranged from 1.5 to 3 hours per day.

During the tests, the human controllers were positioned in a designated area on the edge of the testing site, and used a virtual reconstruction of the environment to keep tabs on where vehicles were and what tasks they were assigned to.

The largest mission shift involved 110 drones, 30 ground vehicles, and up to 50 virtual vehicles representing additional real-world vehicles. The robots had to navigate through the physical urban environment, as well as a series of virtual hazards represented using AprilTags—simplified QR codes that could represent imaginary hazards—that were scattered throughout the mission site.

DARPA made the final field exercise exceptionally challenging by providing thousands of hazards and pieces of information to inform the search. “The complexity of the hazards was significant,” Adams says, noting that some hazards required multiple robots to interact with them simultaneously, and some hazards moved around the environment.

Throughout each mission shift, the human controller’s physiological responses to the tasks at hand were monitored. For example, sensors collected data on their heart-rate variability, posture, and even their speech rate. The data were input into an established algorithm that estimates workload levels and was used to determine when the controller was reaching a workload level that exceeded a normal range, called an “overload state.”

Adams notes that, despite the complexity and large volume of robots to manage in this field exercise, the number and duration of overload state instances were relatively short—a handful of minutes during a mission shift. “The total percentage of estimated overload states was 3 percent of all workload estimates across all shifts for which we collected data,” she says.

www.youtube.com

The most common reason for a human commander to reach an overload state is when they had to generate multiple new tactics or inspect which vehicles in the launch zone were available for deployment.

Adams notes that these finding suggest that—counter to past theories—the number of robots may be less influential on human swarm control performance than previously thought. Her team is exploring the other factors that may impact swarm control missions, such as other human limitations, system designs and UAS designs, the results of which will potentially inform US Federal Aviation Administration drone regulations, she says.

Video Friday is your weekly selection of awesome robotics videos, collected by your friends at IEEE Spectrum robotics. We also post a weekly calendar of upcoming robotics events for the next few months. Please send us your events for inclusion.

Enjoy today’s videos!

Are wheeled quadrupeds going to run out of crazy new ways to move anytime soon? Looks like maybe not.

[ Deep Robotics ]

A giant eye and tiny feet make this pipe inspection robot exceptionally cute, I think.

[ tmsuk ] via [ Robotstart ]

Agility seems to be one of the few humanoid companies talking seriously about safety.

[ Agility Robotics ]

[ University of Michigan ]

[ Paper ]

Thanks, Kento!

If I’m understanding this correctly, if you’re careful, it’s possible to introduce chaos into a blind juggling robot to switch synced juggling to alternate juggling.

[ ETH Zurich ]

Drones with beaks? Sure, why not.

[ GRVC ]

[ OLogic ]

[ University of Texas at Austin ]

Video Friday is your weekly selection of awesome robotics videos, collected by your friends at IEEE Spectrum robotics. We also post a weekly calendar of upcoming robotics events for the next few months. Please send us your events for inclusion.

Enjoy today’s videos!

Are wheeled quadrupeds going to run out of crazy new ways to move anytime soon? Looks like maybe not.

[ Deep Robotics ]

A giant eye and tiny feet make this pipe inspection robot exceptionally cute, I think.

[ tmsuk ] via [ Robotstart ]

Agility seems to be one of the few humanoid companies talking seriously about safety.

[ Agility Robotics ]

[ University of Michigan ]

[ Paper ]

Thanks, Kento!

If I’m understanding this correctly, if you’re careful, it’s possible to introduce chaos into a blind juggling robot to switch synced juggling to alternate juggling.

[ ETH Zurich ]

Drones with beaks? Sure, why not.

[ GRVC ]

[ OLogic ]

[ University of Texas at Austin ]

Seabed observation plays a major role in safeguarding marine systems by keeping tabs on the species and habitats on the ocean floor at different depths. This is primarily done by underwater robots that use optical imaging to collect high quality data that can be fed into environmental models, and compliment the data obtained through sonar in large-scale ocean observations.

Different underwater robots have been trialed over the years, but many have struggled with performing near-seabed observations because they disturb the local seabed by destroying coral and disrupting the sediment. Gang Wang, from Harbin Engineering University in China, and his research team have recently developed a maneuverable underwater vehicle that is better suited to seabed operations because it doesn’t disturb the local environment by floating above the seabed and possessing a specially engineering propeller system to manuever. These robots could be used to better protect the seabed while studying it, and improve efforts to preserve marine biodiversity and explore for underwater resources such as minerals for EV batteries.

Many underwater robots are wheeled or legged, but “these robots face substantial challenges in rugged terrains where obstacles and slopes can impede their functionality,” says Wang. They can also damage coral reefs.

Floating robots don’t have this issue, but existing options disturb the sediment on the seabed because their thrusters create a downward current during ascension. The waves generated as the propeller’s wake directly hit the seafloor in most floating robots, which causes sediment to move in the immediate vicinity. In a similar way to dust blowing in front of your digital or smartphone camera, the particles moving through the water can obscure the view of the cameras on the robot and reduce the quality of the images it captures. “Addressing this issue was crucial for the functional success of our prototype and for increasing its acceptance among engineers,” says Wang.

After further investigation, Wang and the rest of the team found that the robot’s shape influences the local water resistance, or drag, even at low speeds. “During the design process, we configured the robot with two planes exhibiting significant differences in water resistance,” says Wang. This led to the researchers developing a robot with a flattened body and angling the thruster relative to the central axis. “We found that the robot’s shape and the thruster layout significantly influence its ascent speed,” says Wang.

Clockwise from left: relationship between rotational speed of the thruster and the resultant force and torque in the airframe coordinate system, overall structure of the robot, side view of the thruster arrangement and main electronics components.Gang Wang, Kaixin Liu et al.

The researchers created a navigational system where the thrusters generate a combined force that slants downwards but still allows the robot to ascend, changing the wake distribution during ascent so that it doesn’t disturb the sediment on the seafloor. “Flattening the robot’s body and angling the thruster relative to the central axis is a straightforward approach for most engineers, enhancing the potential for broader application of this design” in seabed monitoring, says Wang.

“By addressing the navigational concerns of floating robots, we aim to enhance the observational capabilities of underwater robots in near-seafloor environments,” says Wang. The vehicle was tested in a range of marine environments, including sandy areas, coral reefs, and sheer rock, to show its ability to minimally disturb sediments in multiple potential environments.

Alongside the structural design advancements, the team incorporated an angular acceleration feedback control to keep the robot as close to the seafloor as possible without actually hitting it—called bottoming out. They also developed external disturbance observation algorithms and designed a sensor layout structure that enables the robot to quickly recognize and resist external disturbances, as well as plot a path in real time. This approach allowed the new vehicle to travel along at only 20 centimeters above the seafloor without bottoming out.

By implanting this control, the robot was able to get close to the sea floor and improve the quality of the images it took by reducing light refraction and scattering caused by the water column. “Given the robot’s proximity to the seafloor, even brief periods of instability can lead to collisions with the bottom, and we have verified that the robot shows excellent resistance to strong disturbances,” says Wang.

With the success of this new robot achieving a closer approach to the seafloor without disturbing the seabed or crashing, Wang has stated that they plan to use the robot to closely observe coral reefs. Coral reef monitoring currently relies on inefficient manual methods, so the robots could widen the areas that are observed, and do so more quickly.

Wang adds that “effective detection methods are lacking in deeper waters, particularly in the mid-light layer. We plan to improve the autonomy of the detection process to substitute divers in image collection, and facilitate the automatic identification and classification of coral reef species density to provide a more accurate and timely feedback on the health status of coral reefs.”

Seabed observation plays a major role in safeguarding marine systems by keeping tabs on the species and habitats on the ocean floor at different depths. This is primarily done by underwater robots that use optical imaging to collect high quality data that can be fed into environmental models, and compliment the data obtained through sonar in large-scale ocean observations.

Different underwater robots have been trialed over the years, but many have struggled with performing near-seabed observations because they disturb the local seabed by destroying coral and disrupting the sediment. Gang Wang, from Harbin Engineering University in China, and his research team have recently developed a maneuverable underwater vehicle that is better suited to seabed operations because it doesn’t disturb the local environment by floating above the seabed and possessing a specially engineering propeller system to manuever. These robots could be used to better protect the seabed while studying it, and improve efforts to preserve marine biodiversity and explore for underwater resources such as minerals for EV batteries.

Many underwater robots are wheeled or legged, but “these robots face substantial challenges in rugged terrains where obstacles and slopes can impede their functionality,” says Wang. They can also damage coral reefs.

Floating robots don’t have this issue, but existing options disturb the sediment on the seabed because their thrusters create a downward current during ascension. The waves generated as the propeller’s wake directly hit the seafloor in most floating robots, which causes sediment to move in the immediate vicinity. In a similar way to dust blowing in front of your digital or smartphone camera, the particles moving through the water can obscure the view of the cameras on the robot and reduce the quality of the images it captures. “Addressing this issue was crucial for the functional success of our prototype and for increasing its acceptance among engineers,” says Wang.

After further investigation, Wang and the rest of the team found that the robot’s shape influences the local water resistance, or drag, even at low speeds. “During the design process, we configured the robot with two planes exhibiting significant differences in water resistance,” says Wang. This led to the researchers developing a robot with a flattened body and angling the thruster relative to the central axis. “We found that the robot’s shape and the thruster layout significantly influence its ascent speed,” says Wang.

Clockwise from left: relationship between rotational speed of the thruster and the resultant force and torque in the airframe coordinate system, overall structure of the robot, side view of the thruster arrangement and main electronics components.Gang Wang, Kaixin Liu et al.

The researchers created a navigational system where the thrusters generate a combined force that slants downwards but still allows the robot to ascend, changing the wake distribution during ascent so that it doesn’t disturb the sediment on the seafloor. “Flattening the robot’s body and angling the thruster relative to the central axis is a straightforward approach for most engineers, enhancing the potential for broader application of this design” in seabed monitoring, says Wang.

“By addressing the navigational concerns of floating robots, we aim to enhance the observational capabilities of underwater robots in near-seafloor environments,” says Wang. The vehicle was tested in a range of marine environments, including sandy areas, coral reefs, and sheer rock, to show its ability to minimally disturb sediments in multiple potential environments.

Alongside the structural design advancements, the team incorporated an angular acceleration feedback control to keep the robot as close to the seafloor as possible without actually hitting it—called bottoming out. They also developed external disturbance observation algorithms and designed a sensor layout structure that enables the robot to quickly recognize and resist external disturbances, as well as plot a path in real time. This approach allowed the new vehicle to travel along at only 20 centimeters above the seafloor without bottoming out.

By implanting this control, the robot was able to get close to the sea floor and improve the quality of the images it took by reducing light refraction and scattering caused by the water column. “Given the robot’s proximity to the seafloor, even brief periods of instability can lead to collisions with the bottom, and we have verified that the robot shows excellent resistance to strong disturbances,” says Wang.

With the success of this new robot achieving a closer approach to the seafloor without disturbing the seabed or crashing, Wang has stated that they plan to use the robot to closely observe coral reefs. Coral reef monitoring currently relies on inefficient manual methods, so the robots could widen the areas that are observed, and do so more quickly.

Wang adds that “effective detection methods are lacking in deeper waters, particularly in the mid-light layer. We plan to improve the autonomy of the detection process to substitute divers in image collection, and facilitate the automatic identification and classification of coral reef species density to provide a more accurate and timely feedback on the health status of coral reefs.”

Video Friday is your weekly selection of awesome robotics videos, collected by your friends at IEEE Spectrum robotics. We also post a weekly calendar of upcoming robotics events for the next few months. Please send us your events for inclusion.

Enjoy today's videos!

[ Unitree ]

This is just lovely.

[ Mimus CNK ]

There’s a lot to like about Grain Weevil as an effective unitasking robot, but what I really appreciate here is that the control system is just a remote and a camera slapped onto the top of the bin.

[ Grain Weevil ]

This video, “Robot arm picking your groceries like a real person,” has taught me that I am not a real person.

[ Extend Robotics ]

A robot walking like a human walking like what humans think a robot walking like a robot walks like.

And that was my favorite sentence of the week.

[ Engineai ]

[ Pollen Robotics ]

So a thing that I have come to understand about ships with sails (thanks, Jack Aubrey!) is that sailing in the direction that the wind is coming from can be tricky. Turns out that having a boat with two fronts and no back makes this a lot easier.

[ Paper ] from [ 2023 IEEE/ASME International Conference on Advanced Intelligent Mechatronics ] via [ IEEE Xplore ]

[ Paper ]

Thanks, Kento!

[ Cybathlon ]

Thanks, Robert!

If you’re going to work on robot dogs, I’m honestly not sure whether Purina would be the most or least appropriate place to do that.

[ Michigan Robotics ]

Video Friday is your weekly selection of awesome robotics videos, collected by your friends at IEEE Spectrum robotics. We also post a weekly calendar of upcoming robotics events for the next few months. Please send us your events for inclusion.

Enjoy today's videos!

[ Unitree ]

This is just lovely.

[ Mimus CNK ]

There’s a lot to like about Grain Weevil as an effective unitasking robot, but what I really appreciate here is that the control system is just a remote and a camera slapped onto the top of the bin.

[ Grain Weevil ]

This video, “Robot arm picking your groceries like a real person,” has taught me that I am not a real person.

[ Extend Robotics ]

A robot walking like a human walking like what humans think a robot walking like a robot walks like.

And that was my favorite sentence of the week.

[ Engineai ]

[ Pollen Robotics ]

So a thing that I have come to understand about ships with sails (thanks, Jack Aubrey!) is that sailing in the direction that the wind is coming from can be tricky. Turns out that having a boat with two fronts and no back makes this a lot easier.

[ Paper ] from [ 2023 IEEE/ASME International Conference on Advanced Intelligent Mechatronics ] via [ IEEE Xplore ]

[ Paper ]

Thanks, Kento!

[ Cybathlon ]

Thanks, Robert!

If you’re going to work on robot dogs, I’m honestly not sure whether Purina would be the most or least appropriate place to do that.

[ Michigan Robotics ]

In 1942, the legendary science fiction author Isaac Asimov introduced his Three Laws of Robotics in his short story “Runaround.” The laws were later popularized in his seminal story collection I, Robot.

• First Law: A robot may not injure a human being or, through inaction, allow a human being to come to harm.
• Second Law: A robot must obey orders given it by human beings except where such orders would conflict with the First Law.
• Third Law: A robot must protect its own existence as long as such protection does not conflict with the First or Second Law.

While drawn from works of fiction, these laws have shaped discussions of robot ethics for decades. And as AI systems—which can be considered virtual robots—have become more sophisticated and pervasive, some technologists have found Asimov’s framework useful for considering the potential safeguards needed for AI that interacts with humans.

But the existing three laws are not enough. Today, we are entering an era of unprecedented human-AI collaboration that Asimov could hardly have envisioned. The rapid advancement of generative AI capabilities, particularly in language and image generation, has created challenges beyond Asimov’s original concerns about physical harm and obedience.

The proliferation of AI-enabled deception is particularly concerning. According to the FBI’s 2024 Internet Crime Report, cybercrime involving digital manipulation and social engineering resulted in losses exceeding US $10.3 billion. The European Union Agency for Cybersecurity’s 2023 Threat Landscape specifically highlighted deepfakes—synthetic media that appears genuine—as an emerging threat to digital identity and trust.

Social media misinformation is spreading like wildfire. I studied it during the pandemic extensively and can only say that the proliferation of generative AI tools has made its detection increasingly difficult. To make matters worse, AI-generated articles are just as persuasive or even more persuasive than traditional propaganda, and using AI to create convincing content requires very little effort.

Deepfakes are on the rise throughout society. Botnets can use AI-generated text, speech, and video to create false perceptions of widespread support for any political issue. Bots are now capable of making and receiving phone calls while impersonating people. AI scam calls imitating familiar voices are increasingly common, and any day now, we can expect a boom in video call scams based on AI-rendered overlay avatars, allowing scammers to impersonate loved ones and target the most vulnerable populations. Anecdotally, my very own father was surprised when he saw a video of me speaking fluent Spanish, as he knew that I’m a proud beginner in this language (400 days strong on Duolingo!). Suffice it to say that the video was AI-edited.

Even more alarmingly, children and teenagers are forming emotional attachments to AI agents, and are sometimes unable to distinguish between interactions with real friends and bots online. Already, there have been suicides attributed to interactions with AI chatbots.

In his 2019 book Human Compatible, the eminent computer scientist Stuart Russell argues that AI systems’ ability to deceive humans represents a fundamental challenge to social trust. This concern is reflected in recent policy initiatives, most notably the European Union’s AI Act, which includes provisions requiring transparency in AI interactions and transparent disclosure of AI-generated content. In Asimov’s time, people couldn’t have imagined how artificial agents could use online communication tools and avatars to deceive humans.

Therefore, we must make an addition to Asimov’s laws.

• Fourth Law: A robot or AI must not deceive a human by impersonating a human being.

We need clear boundaries. While human-AI collaboration can be constructive, AI deception undermines trust and leads to wasted time, emotional distress, and misuse of resources. Artificial agents must identify themselves to ensure our interactions with them are transparent and productive. AI-generated content should be clearly marked unless it has been significantly edited and adapted by a human.

Implementation of this Fourth Law would require:

• Mandatory AI disclosure in direct interactions,
• Clear labeling of AI-generated content,
• Technical standards for AI identification,
• Legal frameworks for enforcement,
• Educational initiatives to improve AI literacy.

Of course, all this is easier said than done. Enormous research efforts are already underway to find reliable ways to watermark or detect AI-generated text, audio, images, and videos. Creating the transparency I’m calling for is far from a solved problem.

But the future of human-AI collaboration depends on maintaining clear distinctions between human and artificial agents. As noted in the IEEE’s 2022 “Ethically Aligned Design“ framework, transparency in AI systems is fundamental to building public trust and ensuring the responsible development of artificial intelligence.

Asimov’s complex stories showed that even robots that tried to follow the rules often discovered the unintended consequences of their actions. Still, having AI systems that are trying to follow Asimov’s ethical guidelines would be a very good start.

In 1942, the legendary science fiction author Isaac Asimov introduced his Three Laws of Robotics in his short story “Runaround.” The laws were later popularized in his seminal story collection I, Robot.

• First Law: A robot may not injure a human being or, through inaction, allow a human being to come to harm.
• Second Law: A robot must obey orders given it by human beings except where such orders would conflict with the First Law.
• Third Law: A robot must protect its own existence as long as such protection does not conflict with the First or Second Law.

While drawn from works of fiction, these laws have shaped discussions of robot ethics for decades. And as AI systems—which can be considered virtual robots—have become more sophisticated and pervasive, some technologists have found Asimov’s framework useful for considering the potential safeguards needed for AI that interacts with humans.

But the existing three laws are not enough. Today, we are entering an era of unprecedented human-AI collaboration that Asimov could hardly have envisioned. The rapid advancement of generative AI capabilities, particularly in language and image generation, has created challenges beyond Asimov’s original concerns about physical harm and obedience.

The proliferation of AI-enabled deception is particularly concerning. According to the FBI’s 2024 Internet Crime Report, cybercrime involving digital manipulation and social engineering resulted in losses exceeding US $10.3 billion. The European Union Agency for Cybersecurity’s 2023 Threat Landscape specifically highlighted deepfakes—synthetic media that appears genuine—as an emerging threat to digital identity and trust.

Social media misinformation is spreading like wildfire. I studied it during the pandemic extensively and can only say that the proliferation of generative AI tools has made its detection increasingly difficult. To make matters worse, AI-generated articles are just as persuasive or even more persuasive than traditional propaganda, and using AI to create convincing content requires very little effort.

Deepfakes are on the rise throughout society. Botnets can use AI-generated text, speech, and video to create false perceptions of widespread support for any political issue. Bots are now capable of making and receiving phone calls while impersonating people. AI scam calls imitating familiar voices are increasingly common, and any day now, we can expect a boom in video call scams based on AI-rendered overlay avatars, allowing scammers to impersonate loved ones and target the most vulnerable populations. Anecdotally, my very own father was surprised when he saw a video of me speaking fluent Spanish, as he knew that I’m a proud beginner in this language (400 days strong on Duolingo!). Suffice it to say that the video was AI-edited.

Even more alarmingly, children and teenagers are forming emotional attachments to AI agents, and are sometimes unable to distinguish between interactions with real friends and bots online. Already, there have been suicides attributed to interactions with AI chatbots.

In his 2019 book Human Compatible, the eminent computer scientist Stuart Russell argues that AI systems’ ability to deceive humans represents a fundamental challenge to social trust. This concern is reflected in recent policy initiatives, most notably the European Union’s AI Act, which includes provisions requiring transparency in AI interactions and transparent disclosure of AI-generated content. In Asimov’s time, people couldn’t have imagined how artificial agents could use online communication tools and avatars to deceive humans.

Therefore, we must make an addition to Asimov’s laws.

• Fourth Law: A robot or AI must not deceive a human by impersonating a human being.

We need clear boundaries. While human-AI collaboration can be constructive, AI deception undermines trust and leads to wasted time, emotional distress, and misuse of resources. Artificial agents must identify themselves to ensure our interactions with them are transparent and productive. AI-generated content should be clearly marked unless it has been significantly edited and adapted by a human.

Implementation of this Fourth Law would require:

• Mandatory AI disclosure in direct interactions,
• Clear labeling of AI-generated content,
• Technical standards for AI identification,
• Legal frameworks for enforcement,
• Educational initiatives to improve AI literacy.

Of course, all this is easier said than done. Enormous research efforts are already underway to find reliable ways to watermark or detect AI-generated text, audio, images, and videos. Creating the transparency I’m calling for is far from a solved problem.

But the future of human-AI collaboration depends on maintaining clear distinctions between human and artificial agents. As noted in the IEEE’s 2022 “Ethically Aligned Design“ framework, transparency in AI systems is fundamental to building public trust and ensuring the responsible development of artificial intelligence.

Asimov’s complex stories showed that even robots that tried to follow the rules often discovered the unintended consequences of their actions. Still, having AI systems that are trying to follow Asimov’s ethical guidelines would be a very good start.

Video Friday is your weekly selection of awesome robotics videos, collected by your friends at IEEE Spectrum robotics. We also post a weekly calendar of upcoming robotics events for the next few months. Please send us your events for inclusion.

Enjoy today’s videos!

I’m not totally sure yet about the utility of having a small arm on a robot vacuum, but I love that this is a real thing. At least, it is at CES this year.

[ Roborock ]

We posted about SwitchBot’s new modular home robot system earlier this week, but here’s a new video showing some potentially useful hardware combinations.

[ SwitchBot ]

Yes, it’s in sim, but (and this is a relatively new thing) I will not be shocked to see this happen on Unitree’s hardware in the near future.

[ Unitree ]

[ LimX Dynamics ]

We’ve seen hybrid quadrotor bipeds before, but this one , which is imitating the hopping behavior of Jacana birds, is pretty cute.

What’s a Jacana bird, you ask? It’s these things, which surely must have the most extreme foot to body ratio of any bird:

Also, much respect to the researchers for confidently titling this supplementary video “An Extremely Elegant Jump.”

[ SSRN Paper preprint ]

Twelve minutes flat from suitcase to mobile manipulator. Not bad!

[ Pollen Robotics ]

Happy New Year from Dusty Robotics!

[ Dusty Robotics ]

Video Friday is your weekly selection of awesome robotics videos, collected by your friends at IEEE Spectrum robotics. We also post a weekly calendar of upcoming robotics events for the next few months. Please send us your events for inclusion.

Enjoy today’s videos!

I’m not totally sure yet about the utility of having a small arm on a robot vacuum, but I love that this is a real thing. At least, it is at CES this year.

[ Roborock ]

We posted about SwitchBot’s new modular home robot system earlier this week, but here’s a new video showing some potentially useful hardware combinations.

[ SwitchBot ]

Yes, it’s in sim, but (and this is a relatively new thing) I will not be shocked to see this happen on Unitree’s hardware in the near future.

[ Unitree ]

[ LimX Dynamics ]

We’ve seen hybrid quadrotor bipeds before, but this one , which is imitating the hopping behavior of Jacana birds, is pretty cute.

What’s a Jacana bird, you ask? It’s these things, which surely must have the most extreme foot to body ratio of any bird:

Also, much respect to the researchers for confidently titling this supplementary video “An Extremely Elegant Jump.”

[ SSRN Paper preprint ]

Twelve minutes flat from suitcase to mobile manipulator. Not bad!

[ Pollen Robotics ]

Happy New Year from Dusty Robotics!

[ Dusty Robotics ]

Back in the day, the defining characteristic of home-cleaning robots was that they’d randomly bounce around your floor as part of their cleaning process, because the technology required to localize and map an area hadn’t yet trickled down into the consumer space. That all changed in 2010, when home robots started using lidar (and other things) to track their location and optimize how they cleaned.

Consumer pool-cleaning robots are lagging about 15 years behind indoor robots on this, for a couple of reasons. First, most pool robots—different from automatic pool cleaners, which are purely mechanical systems that are driven by water pressure—have been tethered to an outlet for power, meaning that maximizing efficiency is less of a concern. And second, 3D underwater localization is a much different (and arguably more difficult) problem to solve than 2D indoor localization was. But pool robots are catching up, and at CES this week, Wybot introduced an untethered robot that uses ultrasound to generate a 3D map for fast, efficient pool cleaning. And it’s solar powered and self-emptying, too.

Underwater localization and navigation is not an easy problem for any robot. Private pools are certainly privileged to be operating environments with a reasonable amount of structure and predictability, at least if everything is working the way it should. But the lighting is always going to be a challenge, between bright sunlight, deep shadow, wave reflections, and occasionally murky water if the pool chemicals aren’t balanced very well. That makes relying on any light-based localization system iffy at best, and so Wybot has gone old school, with ultrasound.

Ultrasound used to be a very common way for mobile robots to navigate. You may (or may not) remember venerable robots like the Pioneer 3, with those big ultrasonic sensors across its front. As cameras and lidar got cheap and reliable, the messiness of ultrasonic sensors fell out of favor, but sound is still ideal for underwater applications where anything that relies on light may struggle.

The Wybot S3 uses 12 ultrasonic sensors, plus motor encoders and an inertial measurement unit to map residential pools in three dimensions. “We had to choose the ultrasonic sensors very carefully,” explains Felix (Huo) Feng, the CTO of Wybot. “Actually, we use multiple different sensors, and we compute time of flight [of the sonar pulses] to calculate distance.” The positional accuracy of the resulting map is about 10 centimeters, which is totally fine for the robot to get its job done, although Feng says that they’re actively working to improve the map’s resolution. For path planning purposes, the 3D map gets deconstructed into a series of 2D maps, since the robot needs to clean the bottom of the pool, stairs and ledges, and also the sides of the pool.

Efficiency is particularly important for the S3 because its charging dock has enough solar panels on the top of it to provide about 90 minutes of runtime for the robot over the course of an optimally sunny day. If your pool isn’t too big, that means the robot can clean it daily without requiring a power connection to the dock. The dock also sucks debris out of the collection bin on the robot itself, and Wybot suggests that the S3 can go for up to a month of cleaning without the dock overflowing.

The S3 has a camera on the front, which is used primarily to identify and prioritize dirtier areas (through AI, of course) that need focused cleaning. At some point in the future, Wybot may be able to use vision for navigation too, but my guess is that for reliable 24/7 navigation, ultrasound will still be necessary.

One other interesting little tidbit is the communication system. The dock can talk to your Wi-Fi, of course, and then talk to the robot while it’s charging. Once the robot goes off for a swim, however, traditional wireless signals won’t work, but the dock has its own sonar that can talk to the robot at several bytes per second. This isn’t going to get you streaming video from the robot’s camera, but it’s enough to let you steer the robot if you want, or ask it to come back to the dock, get battery status updates, and similar sorts of things.

The Wybot S3 will go on sale in Q2 of this year for a staggering US $2,999, but that’s how it always works: The first time a new technology shows up in the consumer space, it’s inevitably at a premium. Give it time, though, and my guess is that the ability to navigate and self-empty will become standard features in pool robots. But as far as I know, Wybot got there first.

Back in the day, the defining characteristic of home-cleaning robots was that they’d randomly bounce around your floor as part of their cleaning process, because the technology required to localize and map an area hadn’t yet trickled down into the consumer space. That all changed in 2010, when home robots started using lidar (and other things) to track their location and optimize how they cleaned.

Consumer pool-cleaning robots are lagging about 15 years behind indoor robots on this, for a couple of reasons. First, most pool robots—different from automatic pool cleaners, which are purely mechanical systems that are driven by water pressure—have been tethered to an outlet for power, meaning that maximizing efficiency is less of a concern. And second, 3D underwater localization is a much different (and arguably more difficult) problem to solve than 2D indoor localization was. But pool robots are catching up, and at CES this week, Wybot introduced an untethered robot that uses ultrasound to generate a 3D map for fast, efficient pool cleaning. And it’s solar powered and self-emptying, too.

Underwater localization and navigation is not an easy problem for any robot. Private pools are certainly privileged to be operating environments with a reasonable amount of structure and predictability, at least if everything is working the way it should. But the lighting is always going to be a challenge, between bright sunlight, deep shadow, wave reflections, and occasionally murky water if the pool chemicals aren’t balanced very well. That makes relying on any light-based localization system iffy at best, and so Wybot has gone old school, with ultrasound.

Ultrasound used to be a very common way for mobile robots to navigate. You may (or may not) remember venerable robots like the Pioneer 3, with those big ultrasonic sensors across its front. As cameras and lidar got cheap and reliable, the messiness of ultrasonic sensors fell out of favor, but sound is still ideal for underwater applications where anything that relies on light may struggle.

The Wybot S3 uses 12 ultrasonic sensors, plus motor encoders and an inertial measurement unit to map residential pools in three dimensions. “We had to choose the ultrasonic sensors very carefully,” explains Felix (Huo) Feng, the CTO of Wybot. “Actually, we use multiple different sensors, and we compute time of flight [of the sonar pulses] to calculate distance.” The positional accuracy of the resulting map is about 10 centimeters, which is totally fine for the robot to get its job done, although Feng says that they’re actively working to improve the map’s resolution. For path planning purposes, the 3D map gets deconstructed into a series of 2D maps, since the robot needs to clean the bottom of the pool, stairs and ledges, and also the sides of the pool.

Efficiency is particularly important for the S3 because its charging dock has enough solar panels on the top of it to provide about 90 minutes of runtime for the robot over the course of an optimally sunny day. If your pool isn’t too big, that means the robot can clean it daily without requiring a power connection to the dock. The dock also sucks debris out of the collection bin on the robot itself, and Wybot suggests that the S3 can go for up to a month of cleaning without the dock overflowing.

The S3 has a camera on the front, which is used primarily to identify and prioritize dirtier areas (through AI, of course) that need focused cleaning. At some point in the future, Wybot may be able to use vision for navigation too, but my guess is that for reliable 24/7 navigation, ultrasound will still be necessary.

One other interesting little tidbit is the communication system. The dock can talk to your Wi-Fi, of course, and then talk to the robot while it’s charging. Once the robot goes off for a swim, however, traditional wireless signals won’t work, but the dock has its own sonar that can talk to the robot at several bytes per second. This isn’t going to get you streaming video from the robot’s camera, but it’s enough to let you steer the robot if you want, or ask it to come back to the dock, get battery status updates, and similar sorts of things.

The Wybot S3 will go on sale in Q2 of this year for a staggering US $2,999, but that’s how it always works: The first time a new technology shows up in the consumer space, it’s inevitably at a premium. Give it time, though, and my guess is that the ability to navigate and self-empty will become standard features in pool robots. But as far as I know, Wybot got there first.