Video Friday is your weekly selection of awesome robotics videos, collected by your friends at IEEE Spectrum robotics. We also post a weekly calendar of upcoming robotics events for the next few months. Please send us your events for inclusion.

Enjoy today’s videos!

This is moderately impressive; my favorite part is probably the handoffs and that extra little bit of HRI with what we’d call eye contact if these robots had faces. But keep in mind that you’re looking at close to best case for robotic manipulation, and that if the robots had been given the bag instead of well-spaced objects on a single color background, or if the fridge had a normal human amount of stuff in it, they might be having a much different time of it. Also, is it just me, or is the sound on this video very weird? Like, some things make noise, some things don’t, and the robots themselves occasionally sound more like someone just added in some “soft actuator sound” or something. Also also, I’m of a suspicious nature, and when there is an abrupt cut between “robot grasps door” and “robot opens door,” I assume the worst.

[ Figure ]

[ Paper ] via [ Science Robotics ]

I don’t know about you, but I always dance better when getting beaten with a stick.

[ Unitree Robotics ]

This is big news, people: Sweet Bite Ham Ham, one of the greatest and most useless robots of all time, has a new treat.

All yours for about US $100, overseas shipping included.

[ Ham Ham ] via [ Robotstart ]

[ MagicLab ]

Thanks, Ni Tao!

No, I’m not creeped out at all, why?

[ Clone Robotics ]

Happy 40th Birthday to the MIT Media Lab!

[ MIT Media Lab ]

[ Paper ] via [ BruBotics ]

Thanks, Bram!

[ Paper ] via [ IEEE Transactions on Robots ]

[ Robotics and Intelligent Systems Laboratory ]

I’m not sure I’ve ever seen this trick performed by a robot with soft fingers before.

[ Paper ]

There are a lot of robots involved in car manufacturing. Like, a lot.

[ Kawasaki Robotics ]

[ Carnegie Mellon University Robotics Institute ]

Somebody’s got to test all those luxury handbags and purses. And by somebody, I mean somerobot.

[ Qb Robotics ]

Do not trust people named Evan.

[ Tufts University Human-Robot Interaction Lab ]

[ MIT ]

About a year ago, Boston Dynamics released a research version of its Spot quadruped robot, which comes with a low-level application programming interface (API) that allows direct control of Spot’s joints. Even back then, the rumor was that this API unlocked some significant performance improvements on Spot, including a much faster running speed. That rumor came from the Robotics and AI (RAI) Institute, formerly The AI Institute, formerly the Boston Dynamics AI Institute, and if you were at Marc Raibert’s talk at the ICRA@40 conference in Rotterdam last fall, you already know that it turned out not to be a rumor at all.

Today, we’re able to share some of the work that the RAI Institute has been doing to apply reality-grounded reinforcement learning techniques to enable much higher performance from Spot. The same techniques can also help highly dynamic robots operate robustly, and there’s a brand new hardware platform that shows this off: an autonomous bicycle that can jump.

This video is showing Spot running at a sustained speed of 5.2 meters per second (11.6 miles per hour). Out of the box, Spot’s top speed is 1.6 m/s, meaning that RAI’s spot has more than tripled (!) the quadruped’s factory speed.

If Spot running this quickly looks a little strange, that’s probably because it is strange, in the sense that the way this robot dog’s legs and body move as it runs is not very much like how a real dog runs at all. “The gait is not biological, but the robot isn’t biological,” explains Farbod Farshidian, roboticist at the RAI Institute. “Spot’s actuators are different from muscles, and its kinematics are different, so a gait that’s suitable for a dog to run fast isn’t necessarily best for this robot.”

The best Farshidian can categorize how Spot is moving is that it’s somewhat similar to a trotting gait, except with an added flight phase (with all four feet off the ground at once) that technically turns it into a run. This flight phase is necessary, Farshidian says, because the robot needs that time to successively pull its feet forward fast enough to maintain its speed. This is a “discovered behavior,” in that the robot was not explicitly programmed to “run,” but rather was just required to find the best way of moving as fast as possible.

The Spot controller that ships with the robot when you buy it from Boston Dynamics is based on model predictive control (MPC), which involves creating a software model that approximates the dynamics of the robot as best you can, and then solving an optimization problem for the tasks that you want the robot to do in real time. It’s a very predictable and reliable method for controlling a robot, but it’s also somewhat rigid, because that original software model won’t be close enough to reality to let you really push the limits of the robot. And if you try to say, “Okay, I’m just going to make a superdetailed software model of my robot and push the limits that way,” you get stuck because the optimization problem has to be solved for whatever you want the robot to do, in real time, and the more complex the model is, the harder it is to do that quickly enough to be useful. Reinforcement learning (RL), on the other hand, learns offline. You can use as complex of a model as you want, and then take all the time you need in simulation to train a control policy that can then be run very efficiently on the robot.

Your browser does not support the video tag. In simulation, a couple of Spots (or hundreds of Spots) can be trained in parallel for robust real-world performance.Robotics and AI Institute

In the example of Spot’s top speed, it’s simply not possible to model every last detail for all of the robot’s actuators within a model-based control system that would run in real time on the robot. So instead, simplified (and typically very conservative) assumptions are made about what the actuators are actually doing so that you can expect safe and reliable performance.

Farshidian explains that these assumptions make it difficult to develop a useful understanding of what performance limitations actually are. “Many people in robotics know that one of the limitations of running fast is that you’re going to hit the torque and velocity maximum of your actuation system. So, people try to model that using the data sheets of the actuators. For us, the question that we wanted to answer was whether there might exist some other phenomena that was actually limiting performance.”

Searching for these other phenomena involved bringing new data into the reinforcement learning pipeline, like detailed actuator models learned from the real-world performance of the robot. In Spot’s case, that provided the answer to high-speed running. It turned out that what was limiting Spot’s speed was not the actuators themselves, nor any of the robot’s kinematics: It was simply the batteries not being able to supply enough power. “This was a surprise for me,” Farshidian says, “because I thought we were going to hit the actuator limits first.”

Spot’s power system is complex enough that there’s likely some additional wiggle room, and Farshidian says the only thing that prevented them from pushing Spot’s top speed past 5.2 m/s is that they didn’t have access to the battery voltages so they weren’t able to incorporate that real-world data into their RL model. “If we had beefier batteries on there, we could have run faster. And if you model that phenomena as well in our simulator, I’m sure that we can push this farther.”

Farshidian emphasizes that RAI’s technique is about much more than just getting Spot to run fast—it could also be applied to making Spot move more efficiently to maximize battery life, or more quietly to work better in an office or home environment. Essentially, this is a generalizable tool that can find new ways of expanding the capabilities of any robotic system. And when real-world data is used to make a simulated robot better, you can ask the simulation to do more, with confidence that those simulated skills will successfully transfer back onto the real robot.

Reinforcement learning isn’t just good for maximizing the performance of a robot—it can also make that performance more reliable. The RAI Institute has been experimenting with a completely new kind of robot that it invented in-house: a little jumping bicycle called the Ultra Mobility Vehicle, or UMV, which was trained to do parkour using essentially the same RL pipeline for balancing and driving as was used for Spot’s high-speed running.

There’s no independent physical stabilization system (like a gyroscope) keeping the UMV from falling over; it’s just a normal bike that can move forward and backward and turn its front wheel. As much mass as possible is then packed into the top bit, which actuators can rapidly accelerate up and down. “We’re demonstrating two things in this video,” says Marco Hutter, director of the RAI Institute’s Zurich office. “One is how reinforcement learning helps make the UMV very robust in its driving capabilities in diverse situations. And second, how understanding the robots’ dynamic capabilities allows us to do new things, like jumping on a table which is higher than the robot itself.”

“The key of RL in all of this is to discover new behavior and make this robust and reliable under conditions that are very hard to model. That’s where RL really, really shines.” —Marco Hutter, The RAI Institute

As impressive as the jumping is, for Hutter, it’s just as difficult (if not more difficult) to do maneuvers that may seem fairly simple, like riding backwards. “Going backwards is highly unstable,” Hutter explains. “At least for us, it was not really possible to do that with a classical [MPC] controller, particularly over rough terrain or with disturbances.”

Getting this robot out of the lab and onto terrain to do proper bike parkour is a work in progress that the RAI Institute says it will be able to demonstrate in the near future, but it’s really not about what this particular hardware platform can do—it’s about what any robot can do through RL and other learning-based methods, says Hutter. “The bigger picture here is that the hardware of such robotic systems can in theory do a lot more than we were able to achieve with our classic control algorithms. Understanding these hidden limits in hardware systems lets us improve performance and keep pushing the boundaries on control.”

Your browser does not support the video tag. Teaching the UMV to drive itself down stairs in sim results in a real robot that can handle stairs at any angle.Robotics and AI Institute

Just a few weeks ago, the RAI Institute announced a new partnership with Boston Dynamics “to advance humanoid robots through reinforcement learning.” Humanoids are just another kind of robotic platform, albeit a significantly more complicated one with many more degrees of freedom and things to model and simulate. But when considering the limitations of model predictive control for this level of complexity, a reinforcement learning approach seems almost inevitable, especially when such an approach is already streamlined due to its ability to generalize.

“One of the ambitions that we have as an institute is to have solutions which span across all kinds of different platforms,” says Hutter. “It’s about building tools, about building infrastructure, building the basis for this to be done in a broader context. So not only humanoids, but driving vehicles, quadrupeds, you name it. But doing RL research and showcasing some nice first proof of concept is one thing—pushing it to work in the real world under all conditions, while pushing the boundaries in performance, is something else.”

Transferring skills into the real world has always been a challenge for robots trained in simulation, precisely because simulation is so friendly to robots. “If you spend enough time,” Farshidian explains, “you can come up with a reward function where eventually the robot will do what you want. What often fails is when you want to transfer that sim behavior to the hardware, because reinforcement learning is very good at finding glitches in your simulator and leveraging them to do the task.”

Simulation has been getting much, much better, with new tools, more accurate dynamics, and lots of computing power to throw at the problem. “It’s a hugely powerful ability that we can simulate so many things, and generate so much data almost for free,” Hutter says. But the usefulness of that data is in its connection to reality, making sure that what you’re simulating is accurate enough that a reinforcement learning approach will in fact solve for reality. Bringing physical data collected on real hardware back into the simulation, Hutter believes, is a very promising approach, whether it’s applied to running quadrupeds or jumping bicycles or humanoids. “The combination of the two—of simulation and reality—that’s what I would hypothesize is the right direction.”

Video Friday is your weekly selection of awesome robotics videos, collected by your friends at IEEE Spectrum robotics. We also post a weekly calendar of upcoming robotics events for the next few months. Please send us your events for inclusion.

Enjoy today’s videos!

[ Meta PARTNR ]

[ Humanoid ]

[ Faboratory Research ] at [ Yale University ]

[ NASA ]

[ Mentee Robotics ]

[ Cybathlon ]

[ Paper ] via [ Mitsubishi Electric Research Laboratories ]

Thanks, Yuki!

[ Waymo ]

One of a humanoid robot’s unique advantages lies in its bipedal mobility, allowing it to navigate diverse terrains with efficiency and agility. This capability enables Moby to move freely through various environments and assist with high-risk tasks in critical industries like construction, mining, and energy.

[ UCR ]

[ Pollen Robotics ]

[ OTTO ] by [ Rockwell Automation ]

This Carnegie Mellon University Robotics Institute Seminar is from CMU’s Aaron Johnson, on “Uncertainty and Contact with the World.”

[ CMU RI ]

In theory, one of the main applications for robots should be operating in environments that (for whatever reason) are too dangerous for humans. I say “in theory” because in practice it’s difficult to get robots to do useful stuff in semi-structured or unstructured environments without direct human supervision. This is why there’s been some emphasis recently on teleoperation: Human software teaming up with robot hardware can be a very effective combination.

For this combination to work, you need two things. First, an intuitive control system that lets the user embody themselves in the robot to pilot it effectively. And second, a robot that can deliver on the kind of embodiment that the human pilot needs. The second bit is the more challenging, because humans have very high standards for mobility, strength, and dexterity. But researchers at the Italian Institute of Technology (IIT) have a system that manages to check both boxes, thanks to its enormously powerful quadruped, which now sports a pair of massive arms on its head.

“The primary goal of this project, conducted in collaboration with INAIL, is to extend human capabilities to the robot, allowing operators to perform complex tasks remotely in hazardous and unstructured environments to mitigate risks to their safety by exploiting the robot’s capabilities,” explains Claudio Semini, who leads the Robot Teleoperativo project at IIT. The project is based around the HyQReal hydraulic quadruped, the most recent addition to IIT’s quadruped family.

Hydraulics have been very visibly falling out of favor in robotics, because they’re complicated and messy, and in general robots don’t need the absurd power density that comes with hydraulics. But there are still a few robots in active development that use hydraulics specifically because of all that power. If your robot needs to be highly dynamic or lift really heavy things, hydraulics are, at least for now, where it’s at.

IIT’s HyQReal quadruped is one of those robots. If you need something that can carry a big payload, like a pair of massive arms, this is your robot. Back in 2019, we saw HyQReal pulling a three-tonne airplane. HyQReal itself weighs 140 kilograms, and its knee joints can output up to 300 newton-meters of torque. The hydraulic system is powered by onboard batteries and can provide up to 4 kilowatts of power. It also uses some of Moog’s lovely integrated smart actuators, which sadly don’t seem to be in development anymore. Beyond just lifting heavy things, HyQReal’s mass and power make it a very stable platform, and its aluminum roll cage and Kevlar skin ensure robustness.

The HyQReal hydraulic quadruped is tethered for power during experiments at IIT, but it can also run on battery power.IIT

The arms that HyQReal is carrying are IIT-INAIL arms, which weigh 10 kg each and have a payload of 5 kg per arm. To put that in perspective, the maximum payload of a Boston Dynamics Spot robot is only 14 kg. The head-mounted configuration of the arms means they can reach the ground, and they also have an overlapping workspace to enable bimanual manipulation, which is enhanced by HyQReal’s ability to move its body to assist the arms with their reach. “The development of core actuation technologies with high power, low weight, and advanced control has been a key enabler in our efforts,” says Nikos Tsagarakis, head of the HHCM Lab at IIT. “These technologies have allowed us to realize a low-weight bimanual manipulation system with high payload capacity and large workspace, suitable for integration with HyQReal.”

Maximizing reachable space is important, because the robot will be under the remote control of a human, who probably has no particular interest in or care for mechanical or power constraints—they just want to get the job done.

To get the job done, IIT has developed a teleoperation system, which is weird-looking because it features a very large workspace that allows the user to leverage more of the robot’s full range of motion. Having tried a bunch of different robotic telepresence systems, I can vouch for how important this is: It’s super annoying to be doing some task through telepresence, and then hit a joint limit on the robot and have to pause to reset your arm position. “That is why it is important to study operators’ quality of experience. It allows us to design the haptic and teleoperation systems appropriately because it provides insights into the levels of delight or frustration associated with immersive visualization, haptic feedback, robot control, and task performance,” confirms Ioannis Sarakoglou, who is responsible for the development of the haptic teleoperation technologies in the HHCM Lab. The whole thing takes place in a fully immersive VR environment, of course, with some clever bandwidth optimization inspired by the way humans see that transmits higher-resolution images only where the user is looking.

HyQReal’s telepresence control system offers haptic feedback and a large workspace.IIT

The system is designed to be used in hazardous environments where you wouldn’t want to send a human. That’s why IIT’s partner on this project is INAIL, Italy’s National Institute for Insurance Against Accidents at Work, which is understandably quite interested in finding ways in which robots can be used to keep humans out of harm’s way.

In tests with Italian firefighters in 2022 (using an earlier version of the robot with a single arm), operators were able to use the system to extinguish a simulated tunnel fire. It’s a good first step, but Semini has plans to push the system to handle “more complex, heavy, and demanding tasks, which will better show its potential for real-world applications.”

As always with robots and real-world applications, there’s still a lot of work to be done, Semini says. “The reliability and durability of the systems in extreme environments have to be improved,” he says. “For instance, the robot must endure intense heat and prolonged flame exposure in firefighting without compromising its operational performance or structural integrity.” There’s also managing the robot’s energy consumption (which is not small) to give it a useful operating time, and managing the amount of information presented to the operator to boost situational awareness while still keeping things streamlined and efficient. “Overloading operators with too much information increases cognitive burden, while too little can lead to errors and reduce situational awareness,” says Yonas Tefera, who lead the development of the immersive interface. “Advances in immersive mixed-reality interfaces and multimodal controls, such as voice commands and eye tracking, are expected to improve efficiency and reduce fatigue in the future.”

There’s a lot of crossover here with the goals of the DARPA Robotics Challenge for humanoid robots, except IIT’s system is arguably much more realistically deployable than any of those humanoids are, at least in the near term. While we’re just starting to see the potential of humanoids in carefully controlled environment, quadrupeds are already out there in the world, proving how reliable their four-legged mobility is. Manipulation is the obvious next step, but it has to be more than just opening doors. We need it to use tools, lift debris, and all that other DARPA Robotics Challenge stuff that will keep humans safe. That’s what Robot Teleoperativo is trying to make real.

You can find more detail about the Robot Teleoperativo project in this paper, presented in November at the 2024 IEEE Conference on Telepresence, in Pasadena, Calif.

Video Friday is your weekly selection of awesome robotics videos, collected by your friends at IEEE Spectrum robotics. We also post a weekly calendar of upcoming robotics events for the next few months. Please send us your events for inclusion.

Enjoy today’s videos!

[ ASAP ] from [ Carnegie Mellon University ] and [ Nvidia ]

[ RIVR ]

[ UCR ]

[ Autonomous Robots Lab ]

Thanks, Kostas!

[ Giraffe ] via [ TechCrunch ]

IROS 2025, coming this fall in Hangzhou, China.

[ IROS 2025 ]

This cute lil guy is from a “Weak Robots Exhibition” in Japan.

[ RobotStart ]

I see no problem with cheating via infrastructure to make autonomous vehicles more reliable.

[ Oak Ridge National Laboratory ]

I am not okay with how this coffee cup is handled. Neither is my editor.

[ Qb Robotics ]

[ Paper ] from [ ETH Zurich and Instituto Italiano de Technologia ]

[ Verity ]

For some reason, KUKA is reaffirming its commitment to environmental responsibility and diversity.

[ KUKA ]

Here’s a panel from the recent Humanoids Summit on generative AI for robotics, which includes panelists from OpenAI and Agility Robotics. Just don’t mind the moderator, he’s a bit of a dork.

[ Humanoids Summit ]

The 2004 DARPA Grand Challenge was a spectacular failure. The Defense Advanced Research Projects Agency had offered a US $1 million prize for the team that could design an autonomous ground vehicle capable of completing an off-road course through sometimes flat, sometimes winding and mountainous desert terrain. As IEEE Spectrum reported at the time, it was “the motleyest assortment of vehicles assembled in one place since the filming of Mad Max 2: The Road Warrior.” Not a single entrant made it across the finish line. Some didn’t make it out of the parking lot.

Videos of the attempts are comical, although any laughter comes at the expense of the many engineers who spent countless hours and millions of dollars to get to that point.

So it’s all the more remarkable that in the second DARPA Grand Challenge, just a year and a half later, five vehicles crossed the finish line. Stanley, developed by the Stanford Racing Team, eked out a first-place win to claim the $2 million purse. This modified Volkswagen Touareg [shown at top] completed the 212-kilometer course in 6 hours, 54 minutes. Carnegie Mellon’s Sandstorm and H1ghlander took second and third place, respectively, with times of 7:05 and 7:14.

Kat-5, sponsored by the Gray Insurance Co. of Metairie, La., came in fourth with a respectable 7:30. The vehicle was named after Hurricane Katrina, which had just pummeled the Gulf Coast a month and a half earlier. Oshkosh Truck’s TerraMax also finished the circuit, although its time of 12:51 exceeded the 10-hour time limit set by DARPA.

So how did the Grand Challenge go from a total bust to having five robust finishers in such a short period of time? It’s definitely a testament to what can be accomplished when engineers rise to a challenge. But the outcome of this one race was preceded by a much longer path of research, and that plus a little bit of luck are what ultimately led to victory.

Let’s back up to 1998, when computer scientist Sebastian Thrun was working at Carnegie Mellon and experimenting with a very different robot: a museum tour guide. For two weeks in the summer, Minerva, which looked a bit like a Dalek from “Doctor Who,” navigated an exhibit at the Smithsonian National Museum of American History. Its main task was to roll around and dispense nuggets of information about the displays.

Minerva was a museum tour-guide robot developed by Sebastian Thrun.

In an interview at the time, Thrun acknowledged that Minerva was there to entertain. But Minerva wasn’t just a people pleaser ; it was also a machine learning experiment. It had to learn where it could safely maneuver without taking out a visitor or a priceless artifact. Visitor, nonvisitor; display case, not-display case; open floor, not-open floor. It had to react to humans crossing in front of it in unpredictable ways. It had to learn to “see.”

Fast-forward five years: Thrun transferred to Stanford in July 2003. Inspired by the first Grand Challenge, he organized the Stanford Racing Team with the aim of fielding a robotic car in the second competition.

In a vast oversimplification of Stanley’s main task, the autonomous robot had to differentiate between road and not-road in order to navigate the route successfully. The Stanford team decided to focus its efforts on developing software and used as much off-the-shelf hardware as they could, including a laser to scan the immediate terrain and a simple video camera to scan the horizon. Software overlapped the two inputs, adapted to the changing road conditions on the fly, and determined a safe driving speed. (For more technical details on Stanley, check out the team’s paper.) A remote-control kill switch, which DARPA required on all vehicles, would deactivate the car before it could become a danger. About 100,000 lines of code did that and much more.

The Stanford team hadn’t entered the 2004 Grand Challenge and wasn’t expected to win the 2005 race. Carnegie Mellon, meanwhile, had two entries—a modified 1986 Humvee and a modified 1999 Hummer—and was the clear favorite. In the 2004 race, CMU’s Sandstorm had gone furthest, completing 12 km. For the second race, CMU brought an improved Sandstorm as well as a new vehicle, H1ghlander.

Many of the other 2004 competitors regrouped to try again, and new ones entered the fray. In all, 195 teams applied to compete in the 2005 event. Teams included students, academics, industry experts, and hobbyists.

After site visits in the spring, 43 teams made it to the qualifying event, held 27 September through 5 October at the California Speedway, in Fontana. Each vehicle took four runs through the course, navigating through checkpoints and avoiding obstacles. A total of 23 teams were selected to attempt the main course across the Mojave Desert. Competing was a costly endeavor—CMU’s Red Team spent more than $3 million in its first year—and the names of sponsors were splashed across the vehicles like the logos on race cars.

In the early hours of 8 October, the finalists gathered for the big race. Each team had a staggered start time to help avoid congestion along the route. About two hours before a team’s start, DARPA gave them a CD containing approximately 3,000 GPS coordinates representing the course. Once the team hit go, it was hands off: The car had to drive itself without any human intervention. PBS’s NOVA produced an excellent episode on the 2004 and 2005 Grand Challenges that I highly recommend if you want to get a feel for the excitement, anticipation, disappointment, and triumph.

In the 2005 Grand Challenge, Carnegie Mellon University’s H1ghlander was one of five autonomous cars to finish the race.Damian Dovarganes/AP

H1ghlander held the pole position, having placed first in the qualifying rounds, followed by Stanley and Sandstorm. H1ghlander pulled ahead early and soon had a substantial lead. That’s where luck, or rather the lack of it, came in.

About two hours into the race, H1ghlander slowed down and started rolling backward down a hill. Although it eventually resumed moving forward, it never regained its top speed, even on long, straight, level sections of the course. The slower but steadier Stanley caught up to H1ghlander at the 163-km (101.5-mile) marker, passed it, and never let go of the lead.

What went wrong with H1ghlander remained a mystery, even after extensive postrace analysis. It wasn’t until 12 years after the race—and once again with a bit of luck—that CMU discovered the problem: Pressing on a small electronic filter between the engine control module and the fuel injector caused the engine to lose power and even turn off. Team members speculated that an accident a few weeks before the competition had damaged the filter. (To learn more about how CMU finally figured this out, see Spectrum Senior Editor Evan Ackerman’s 2017 story.)

Regardless of who won the Grand Challenge, many success stories came out of the contest. A year and a half after the race, Thrun had already made great progress on adaptive cruise control and lane-keeping assistance, which is now readily available on many commercial vehicles. He then worked on Google’s Street View and its initial self-driving cars. CMU’s Red Team worked with NASA to develop rovers for potentially exploring the moon or distant planets. Closer to home, they helped develop self-propelled harvesters for the agricultural sector.

Stanford team leader Sebastian Thrun holds a $2 million check, the prize for winning the 2005 Grand Challenge.Damian Dovarganes/AP

Of course, there was also a lot of hype, which tended to overshadow the race’s militaristic origins—remember, the “D” in DARPA stands for “defense.” Back in 2000, a defense authorization bill had stipulated that one-third of the U.S. ground combat vehicles be “unmanned” by 2015, and DARPA conceived of the Grand Challenge to spur development of these autonomous vehicles. The U.S. military was still fighting in the Middle East, and DARPA promoters believed self-driving vehicles would help minimize casualties, particularly those caused by improvised explosive devices.

DARPA sponsored more contests, such as the 2007 Urban Challenge, in which vehicles navigated a simulated city and suburban environment; the 2012 Robotics Challenge for disaster-response robots; and the 2022 Subterranean Challenge for—you guessed it—robots that could get around underground. Despite the competitions, continued military conflicts, and hefty government contracts, actual advances in autonomous military vehicles and robots did not take off to the extent desired. As of 2023, robotic ground vehicles made up only 3 percent of the global armored-vehicle market.

Today, there are very few fully autonomous ground vehicles in the U.S. military; instead, the services have forged ahead with semiautonomous, operator-assisted systems, such as remote-controlled drones and ship autopilots. The one Grand Challenge finisher that continued to work for the U.S. military was Oshkosh Truck, the Wisconsin-based sponsor of the TerraMax. The company demonstrated a palletized loading system to transport cargo in unmanned vehicles for the U.S. Army.

Much of the contemporary reporting on the Grand Challenge predicted that self-driving cars would take us closer to a “Jetsons” future, with a self-driving vehicle to ferry you around. But two decades after Stanley, the rollout of civilian autonomous cars has been confined to specific applications, such as Waymo robotaxis transporting people around San Francisco or the GrubHub Starships struggling to deliver food across my campus at the University of South Carolina.

I’ll be watching to see how the technology evolves outside of big cities. Self-driving vehicles would be great for long distances on empty country roads, but parts of rural America still struggle to get adequate cellphone coverage. Will small towns and the spaces that surround them have the bandwidth to accommodate autonomous vehicles? As much as I’d like to think self-driving autos are nearly here, I don’t expect to find one under my carport anytime soon.

Not long after the 2005 race, Stanley was ready to retire. Recalling his experience testing Minerva at the National Museum of American History, Thrun thought the museum would make a nice home. He loaned it to the museum in 2006, and since 2008 it has resided permanently in the museum’s collections, alongside other remarkable specimens in robotics and automobiles. In fact, it isn’t even the first Stanley in the collection.

Stanley now resides in the collections of the Smithsonian Institution’s National Museum of American History, which also houses another Stanley—this 1910 Stanley Runabout. Behring Center/National Museum of American History/Smithsonian Institution

That distinction belongs to a 1910 Stanley Runabout, an early steam-powered car introduced at a time when it wasn’t yet clear that the internal-combustion engine was the way to go. Despite clear drawbacks—steam engines had a nasty tendency to explode—“Stanley steamers” were known for their fine craftsmanship. Fred Marriott set the land speed record while driving a Stanley in 1906. It clocked in at 205.5 kilometers per hour, which was significantly faster than the 21st-century Stanley’s average speed of 30.7 km/hr. To be fair, Marriott’s Stanley was racing over a flat, straight course rather than the off-road terrain navigated by Thrun’s Stanley.

Across the century that separates the two Stanleys, it’s easy to trace a narrative of progress. Both are clearly recognizable as four-wheeled land vehicles, but I suspect the science-fiction dreamers of the early 20th century would have been hard-pressed to imagine the suite of technologies that would propel a 21st-century self-driving car. What will the vehicles of the early 22nd century be like? Will they even have four tires, or will they run on something entirely new?

Part of a continuing series looking at historical artifacts that embrace the boundless potential of technology.

An abridged version of this article appears in the February 2025 print issue as “Slow and Steady Wins the Race.”

Sebastian Thrun and his colleagues at the Stanford Artificial Intelligence Laboratory, along with members of the other groups that sponsored Stanley, published “Stanley: The Robot That Won the DARPA Grand Challenge.” This paper, from the Journal of Field Robotics, explains the vehicle’s development.

The NOVA PBS episode “The Great Robot Race” provides interviews and video footage from both the failed first Grand Challenge and the successful second one. I personally liked the side story of GhostRider, an autonomous motorcycle that competed in both competitions but didn’t quite cut it. (GhostRider also now resides in the Smithsonian’s collection.)

Smithsonian curator Carlene Stephens kindly talked with me about how she collected Stanley for the National Museum of American History and where she sees artifacts like this fitting into the stream of history.

Video Friday is your weekly selection of awesome robotics videos, collected by your friends at IEEE Spectrum robotics. We also post a weekly calendar of upcoming robotics events for the next few months. Please send us your events for inclusion.

Enjoy today’s videos!

This video about ‘foster’ Aibos helping kids at a children’s hospital is well worth turning on auto-translated subtitles for.

[ Aibo Foster Program ]

[ Unitree ]

Happy Chinese New Year from PNDbotics!

[ PNDbotics ]

[ LimX Dynamics ]

Mr. Bucket is my favorite.

[ Mitsubishi Electric Research Laboratories ]

Thanks, Yuki!

[ Zoox ]

The Humanoids Summit at the Computer History Museum in December was successful enough (either because of or in spite of my active participation) that it’s not only happening again in 2025: There’s also going to be a spring version of the conference in London in May!

[ Humanoids Summit ]

I’m not sure it’ll ever be practical at scale, but I do really like JSK’s musculoskeletal humanoid work.

[ Paper ]

[ Canadian Space Agency ]

[ Corvus Robotics ]

Thanks, Joan!

I would have said that this controller was too small to be manipulated with a pinch grasp. I would be wrong.

[ Pollen ]

[ NASA ]

[ Team BlackSheep ]

[ Zoox ]

Most people know that robots no longer sound like tinny trash cans. They sound like Siri, Alexa, and Gemini. They sound like the voices in labyrinthine customer support phone trees. And even those robot voices are being made obsolete by new AI-generated voices that can mimic every vocal nuance and tic of human speech, down to specific regional accents. And with just a few seconds of audio, AI can now clone someone’s specific voice.

This technology will replace humans in many areas. Automated customer support will save money by cutting staffing at call centers. AI agents will make calls on our behalf, conversing with others in natural language. All of that is happening, and will be commonplace soon.

But there is something fundamentally different about talking with a bot as opposed to a person. A person can be a friend. An AI cannot be a friend, despite how people might treat it or react to it. AI is at best a tool, and at worst a means of manipulation. Humans need to know whether we’re talking with a living, breathing person or a robot with an agenda set by the person who controls it. That’s why robots should sound like robots.

You can’t just label AI-generated speech. It will come in many different forms. So we need a way to recognize AI that works no matter the modality. It needs to work for long or short snippets of audio, even just a second long. It needs to work for any language, and in any cultural context. At the same time, we shouldn’t constrain the underlying system’s sophistication or language complexity.

We have a simple proposal: all talking AIs and robots should use a ring modulator. In the mid-twentieth century, before it was easy to create actual robotic-sounding speech synthetically, ring modulators were used to make actors’ voices sound robotic. Over the last few decades, we have become accustomed to robotic voices, simply because text-to-speech systems were good enough to produce intelligible speech that was not human-like in its sound. Now we can use that same technology to make robotic speech that is indistinguishable from human sound robotic again.

A ring modulator has several advantages: It is computationally simple, can be applied in real-time, does not affect the intelligibility of the voice, and--most importantly--is universally “robotic sounding” because of its historical usage for depicting robots.

Responsible AI companies that provide voice synthesis or AI voice assistants in any form should add a ring modulator of some standard frequency (say, between 30-80 Hz) and of a minimum amplitude (say, 20 percent). That’s it. People will catch on quickly.

Here are a couple of examples you can listen to for examples of what we’re suggesting. The first clip is an AI-generated “podcast” of this article made by Google’s NotebookLM featuring two AI “hosts.” Google’s NotebookLM created the podcast script and audio given only the text of this article. The next two clips feature that same podcast with the AIs’ voices modulated more and less subtly by a ring modulator:

Raw audio sample generated by Google’s NotebookLM Your browser does not support the audio element.

Audio sample with added ring modulator (30 Hz-25%) Your browser does not support the audio element.

Audio sample with added ring modulator (30 Hz-40%) Your browser does not support the audio element.

We were able to generate the audio effect with a 50-line Python script generated by Anthropic’s Claude. One of the most well-known robot voices were those of the Daleks from Doctor Who in the 1960s. Back then robot voices were difficult to synthesize, so the audio was actually an actor’s voice run through a ring modulator. It was set to around 30 Hz, as we did in our example, with different modulation depth (amplitude) depending on how strong the robotic effect is meant to be. Our expectation is that the AI industry will test and converge on a good balance of such parameters and settings, and will use better tools than a 50-line Python script, but this highlights how simple it is to achieve.

Of course there will also be nefarious uses of AI voices. Scams that use voice cloning have been getting easier every year, but they’ve been possible for many years with the right know-how. Just like we’re learning that we can no longer trust images and videos we see because they could easily have been AI-generated, we will all soon learn that someone who sounds like a family member urgently requesting money may just be a scammer using a voice-cloning tool.

We don’t expect scammers to follow our proposal: They’ll find a way no matter what. But that’s always true of security standards, and a rising tide lifts all boats. We think the bulk of the uses will be with popular voice APIs from major companies--and everyone should know that they’re talking with a robot.

This article is part of our exclusive IEEE Journal Watch series in partnership with IEEE Xplore.

Swarms of autonomous robots are increasingly being tested and deployed in complex missions, yet a certain level of human oversight during these missions is still required. Which means a major question remains: How many robots—and how complex a mission—can a single human manage before becoming overwhelmed?

In a study funded by the U.S. Defense Advanced Research Projects Agency (DARPA), experts show that humans can single-handedly and effectively manage a heterogenous swarm of more than 100 autonomous ground and aerial vehicles, while feeling overwhelmed only for brief periods of time during an overall small portion of the mission. For instance, in a particularly challenging, multi-day experiment in an urban setting, human controllers were overloaded with stress and workload only three percent of the time. The results were published 19 November in IEEE Transactions on Field Robotics.

Julie A. Adams, the associate director of research at Oregon State University’s Collaborative Robotics and Intelligent Systems Institute, has been studying human interactions with robots and other complex systems, such as aircraft cockpits and nuclear power plant control rooms, for 35 years. She notes that robot swarms can be used to support missions where work may be particularly dangerous and hazardous for humans, such as monitoring wildfires.

“Swarms can be used to provide persistent coverage of an area, such as monitoring for new fires or looters in the recently burned areas of Los Angeles,” Adams says. “The information can be used to direct limited assets, such as firefighting units or water tankers to new fires and hotspots, or to locations at which fires were thought to have been extinguished.”

These kinds of missions can involve a mix of many different kinds of unmanned ground vehicles (such as the Aion Robotics R1 wheeled robot) and aerial autonomous vehicles (like the Modal AI VOXL M500 quadcopter), and a human controller may need to reassign individual robots to different tasks as the mission unfolds. Notably, some theories over the past few decades—and even Adams’ early thesis work—suggest that a single human has limited capacity to deploy very large numbers of robots.

“These historical theories and the associated empirical results showed that as the number of ground robots increased, so did the human’s workload, which often resulted in reduced overall performance,” says Adams, noting that, although earlier research focused on unmanned ground vehicles (UGVs), which must deal with curbs and other physical barriers, unmanned aerial vehicles (UAVs) often encounter fewer physical barriers.

Human controllers managed their swarms of autonomous vehicles with a virtual display. The fuschia ring represents the area the person could see within their head-mounted display.DARPA

As part of DARPA’s OFFensive Swarm-Enabled Tactics (OFFSET) program, Adams and her colleagues sought to explore whether these theories applied to very complex missions involving a mix of unmanned ground and air vehicles. In November 2021, at Fort Campbell in Kentucky, two human controllers took turns engaging in a series of missions over the course of three weeks with the objective of neutralizing an adversarial target. Both human controllers had significant experience controlling swarms, and participated in alternating shifts that ranged from 1.5 to 3 hours per day.

During the tests, the human controllers were positioned in a designated area on the edge of the testing site, and used a virtual reconstruction of the environment to keep tabs on where vehicles were and what tasks they were assigned to.

The largest mission shift involved 110 drones, 30 ground vehicles, and up to 50 virtual vehicles representing additional real-world vehicles. The robots had to navigate through the physical urban environment, as well as a series of virtual hazards represented using AprilTags—simplified QR codes that could represent imaginary hazards—that were scattered throughout the mission site.

DARPA made the final field exercise exceptionally challenging by providing thousands of hazards and pieces of information to inform the search. “The complexity of the hazards was significant,” Adams says, noting that some hazards required multiple robots to interact with them simultaneously, and some hazards moved around the environment.

Throughout each mission shift, the human controller’s physiological responses to the tasks at hand were monitored. For example, sensors collected data on their heart-rate variability, posture, and even their speech rate. The data were input into an established algorithm that estimates workload levels and was used to determine when the controller was reaching a workload level that exceeded a normal range, called an “overload state.”

Adams notes that, despite the complexity and large volume of robots to manage in this field exercise, the number and duration of overload state instances were relatively short—a handful of minutes during a mission shift. “The total percentage of estimated overload states was 3 percent of all workload estimates across all shifts for which we collected data,” she says.

www.youtube.com

The most common reason for a human commander to reach an overload state is when they had to generate multiple new tactics or inspect which vehicles in the launch zone were available for deployment.

Adams notes that these finding suggest that—counter to past theories—the number of robots may be less influential on human swarm control performance than previously thought. Her team is exploring the other factors that may impact swarm control missions, such as other human limitations, system designs and UAS designs, the results of which will potentially inform US Federal Aviation Administration drone regulations, she says.

Video Friday is your weekly selection of awesome robotics videos, collected by your friends at IEEE Spectrum robotics. We also post a weekly calendar of upcoming robotics events for the next few months. Please send us your events for inclusion.

Enjoy today’s videos!

Are wheeled quadrupeds going to run out of crazy new ways to move anytime soon? Looks like maybe not.

[ Deep Robotics ]

A giant eye and tiny feet make this pipe inspection robot exceptionally cute, I think.

[ tmsuk ] via [ Robotstart ]

Agility seems to be one of the few humanoid companies talking seriously about safety.

[ Agility Robotics ]

[ University of Michigan ]

[ Paper ]

Thanks, Kento!

If I’m understanding this correctly, if you’re careful, it’s possible to introduce chaos into a blind juggling robot to switch synced juggling to alternate juggling.

[ ETH Zurich ]

Drones with beaks? Sure, why not.

[ GRVC ]

[ OLogic ]

[ University of Texas at Austin ]

Seabed observation plays a major role in safeguarding marine systems by keeping tabs on the species and habitats on the ocean floor at different depths. This is primarily done by underwater robots that use optical imaging to collect high quality data that can be fed into environmental models, and compliment the data obtained through sonar in large-scale ocean observations.

Different underwater robots have been trialed over the years, but many have struggled with performing near-seabed observations because they disturb the local seabed by destroying coral and disrupting the sediment. Gang Wang, from Harbin Engineering University in China, and his research team have recently developed a maneuverable underwater vehicle that is better suited to seabed operations because it doesn’t disturb the local environment by floating above the seabed and possessing a specially engineering propeller system to manuever. These robots could be used to better protect the seabed while studying it, and improve efforts to preserve marine biodiversity and explore for underwater resources such as minerals for EV batteries.

Many underwater robots are wheeled or legged, but “these robots face substantial challenges in rugged terrains where obstacles and slopes can impede their functionality,” says Wang. They can also damage coral reefs.

Floating robots don’t have this issue, but existing options disturb the sediment on the seabed because their thrusters create a downward current during ascension. The waves generated as the propeller’s wake directly hit the seafloor in most floating robots, which causes sediment to move in the immediate vicinity. In a similar way to dust blowing in front of your digital or smartphone camera, the particles moving through the water can obscure the view of the cameras on the robot and reduce the quality of the images it captures. “Addressing this issue was crucial for the functional success of our prototype and for increasing its acceptance among engineers,” says Wang.

After further investigation, Wang and the rest of the team found that the robot’s shape influences the local water resistance, or drag, even at low speeds. “During the design process, we configured the robot with two planes exhibiting significant differences in water resistance,” says Wang. This led to the researchers developing a robot with a flattened body and angling the thruster relative to the central axis. “We found that the robot’s shape and the thruster layout significantly influence its ascent speed,” says Wang.

Clockwise from left: relationship between rotational speed of the thruster and the resultant force and torque in the airframe coordinate system, overall structure of the robot, side view of the thruster arrangement and main electronics components.Gang Wang, Kaixin Liu et al.

The researchers created a navigational system where the thrusters generate a combined force that slants downwards but still allows the robot to ascend, changing the wake distribution during ascent so that it doesn’t disturb the sediment on the seafloor. “Flattening the robot’s body and angling the thruster relative to the central axis is a straightforward approach for most engineers, enhancing the potential for broader application of this design” in seabed monitoring, says Wang.

“By addressing the navigational concerns of floating robots, we aim to enhance the observational capabilities of underwater robots in near-seafloor environments,” says Wang. The vehicle was tested in a range of marine environments, including sandy areas, coral reefs, and sheer rock, to show its ability to minimally disturb sediments in multiple potential environments.

Alongside the structural design advancements, the team incorporated an angular acceleration feedback control to keep the robot as close to the seafloor as possible without actually hitting it—called bottoming out. They also developed external disturbance observation algorithms and designed a sensor layout structure that enables the robot to quickly recognize and resist external disturbances, as well as plot a path in real time. This approach allowed the new vehicle to travel along at only 20 centimeters above the seafloor without bottoming out.

By implanting this control, the robot was able to get close to the sea floor and improve the quality of the images it took by reducing light refraction and scattering caused by the water column. “Given the robot’s proximity to the seafloor, even brief periods of instability can lead to collisions with the bottom, and we have verified that the robot shows excellent resistance to strong disturbances,” says Wang.

With the success of this new robot achieving a closer approach to the seafloor without disturbing the seabed or crashing, Wang has stated that they plan to use the robot to closely observe coral reefs. Coral reef monitoring currently relies on inefficient manual methods, so the robots could widen the areas that are observed, and do so more quickly.

Wang adds that “effective detection methods are lacking in deeper waters, particularly in the mid-light layer. We plan to improve the autonomy of the detection process to substitute divers in image collection, and facilitate the automatic identification and classification of coral reef species density to provide a more accurate and timely feedback on the health status of coral reefs.”

Video Friday is your weekly selection of awesome robotics videos, collected by your friends at IEEE Spectrum robotics. We also post a weekly calendar of upcoming robotics events for the next few months. Please send us your events for inclusion.

Enjoy today's videos!

[ Unitree ]

This is just lovely.

[ Mimus CNK ]

There’s a lot to like about Grain Weevil as an effective unitasking robot, but what I really appreciate here is that the control system is just a remote and a camera slapped onto the top of the bin.

[ Grain Weevil ]

This video, “Robot arm picking your groceries like a real person,” has taught me that I am not a real person.

[ Extend Robotics ]

A robot walking like a human walking like what humans think a robot walking like a robot walks like.

And that was my favorite sentence of the week.

[ Engineai ]

[ Pollen Robotics ]

So a thing that I have come to understand about ships with sails (thanks, Jack Aubrey!) is that sailing in the direction that the wind is coming from can be tricky. Turns out that having a boat with two fronts and no back makes this a lot easier.

[ Paper ] from [ 2023 IEEE/ASME International Conference on Advanced Intelligent Mechatronics ] via [ IEEE Xplore ]

[ Paper ]

Thanks, Kento!

[ Cybathlon ]

Thanks, Robert!

If you’re going to work on robot dogs, I’m honestly not sure whether Purina would be the most or least appropriate place to do that.

[ Michigan Robotics ]

In 1942, the legendary science fiction author Isaac Asimov introduced his Three Laws of Robotics in his short story “Runaround.” The laws were later popularized in his seminal story collection I, Robot.

• First Law: A robot may not injure a human being or, through inaction, allow a human being to come to harm.
• Second Law: A robot must obey orders given it by human beings except where such orders would conflict with the First Law.
• Third Law: A robot must protect its own existence as long as such protection does not conflict with the First or Second Law.

While drawn from works of fiction, these laws have shaped discussions of robot ethics for decades. And as AI systems—which can be considered virtual robots—have become more sophisticated and pervasive, some technologists have found Asimov’s framework useful for considering the potential safeguards needed for AI that interacts with humans.

But the existing three laws are not enough. Today, we are entering an era of unprecedented human-AI collaboration that Asimov could hardly have envisioned. The rapid advancement of generative AI capabilities, particularly in language and image generation, has created challenges beyond Asimov’s original concerns about physical harm and obedience.

The proliferation of AI-enabled deception is particularly concerning. According to the FBI’s 2024 Internet Crime Report, cybercrime involving digital manipulation and social engineering resulted in losses exceeding US $10.3 billion. The European Union Agency for Cybersecurity’s 2023 Threat Landscape specifically highlighted deepfakes—synthetic media that appears genuine—as an emerging threat to digital identity and trust.

Social media misinformation is spreading like wildfire. I studied it during the pandemic extensively and can only say that the proliferation of generative AI tools has made its detection increasingly difficult. To make matters worse, AI-generated articles are just as persuasive or even more persuasive than traditional propaganda, and using AI to create convincing content requires very little effort.

Deepfakes are on the rise throughout society. Botnets can use AI-generated text, speech, and video to create false perceptions of widespread support for any political issue. Bots are now capable of making and receiving phone calls while impersonating people. AI scam calls imitating familiar voices are increasingly common, and any day now, we can expect a boom in video call scams based on AI-rendered overlay avatars, allowing scammers to impersonate loved ones and target the most vulnerable populations. Anecdotally, my very own father was surprised when he saw a video of me speaking fluent Spanish, as he knew that I’m a proud beginner in this language (400 days strong on Duolingo!). Suffice it to say that the video was AI-edited.

Even more alarmingly, children and teenagers are forming emotional attachments to AI agents, and are sometimes unable to distinguish between interactions with real friends and bots online. Already, there have been suicides attributed to interactions with AI chatbots.

In his 2019 book Human Compatible, the eminent computer scientist Stuart Russell argues that AI systems’ ability to deceive humans represents a fundamental challenge to social trust. This concern is reflected in recent policy initiatives, most notably the European Union’s AI Act, which includes provisions requiring transparency in AI interactions and transparent disclosure of AI-generated content. In Asimov’s time, people couldn’t have imagined how artificial agents could use online communication tools and avatars to deceive humans.

Therefore, we must make an addition to Asimov’s laws.

• Fourth Law: A robot or AI must not deceive a human by impersonating a human being.

We need clear boundaries. While human-AI collaboration can be constructive, AI deception undermines trust and leads to wasted time, emotional distress, and misuse of resources. Artificial agents must identify themselves to ensure our interactions with them are transparent and productive. AI-generated content should be clearly marked unless it has been significantly edited and adapted by a human.

Implementation of this Fourth Law would require:

• Mandatory AI disclosure in direct interactions,
• Clear labeling of AI-generated content,
• Technical standards for AI identification,
• Legal frameworks for enforcement,
• Educational initiatives to improve AI literacy.

Of course, all this is easier said than done. Enormous research efforts are already underway to find reliable ways to watermark or detect AI-generated text, audio, images, and videos. Creating the transparency I’m calling for is far from a solved problem.

But the future of human-AI collaboration depends on maintaining clear distinctions between human and artificial agents. As noted in the IEEE’s 2022 “Ethically Aligned Design“ framework, transparency in AI systems is fundamental to building public trust and ensuring the responsible development of artificial intelligence.

Asimov’s complex stories showed that even robots that tried to follow the rules often discovered the unintended consequences of their actions. Still, having AI systems that are trying to follow Asimov’s ethical guidelines would be a very good start.

Video Friday is your weekly selection of awesome robotics videos, collected by your friends at IEEE Spectrum robotics. We also post a weekly calendar of upcoming robotics events for the next few months. Please send us your events for inclusion.

Enjoy today’s videos!

I’m not totally sure yet about the utility of having a small arm on a robot vacuum, but I love that this is a real thing. At least, it is at CES this year.

[ Roborock ]

We posted about SwitchBot’s new modular home robot system earlier this week, but here’s a new video showing some potentially useful hardware combinations.

[ SwitchBot ]

Yes, it’s in sim, but (and this is a relatively new thing) I will not be shocked to see this happen on Unitree’s hardware in the near future.

[ Unitree ]

[ LimX Dynamics ]

We’ve seen hybrid quadrotor bipeds before, but this one , which is imitating the hopping behavior of Jacana birds, is pretty cute.

What’s a Jacana bird, you ask? It’s these things, which surely must have the most extreme foot to body ratio of any bird:

Also, much respect to the researchers for confidently titling this supplementary video “An Extremely Elegant Jump.”

[ SSRN Paper preprint ]

Twelve minutes flat from suitcase to mobile manipulator. Not bad!

[ Pollen Robotics ]

Happy New Year from Dusty Robotics!

[ Dusty Robotics ]

Back in the day, the defining characteristic of home-cleaning robots was that they’d randomly bounce around your floor as part of their cleaning process, because the technology required to localize and map an area hadn’t yet trickled down into the consumer space. That all changed in 2010, when home robots started using lidar (and other things) to track their location and optimize how they cleaned.

Consumer pool-cleaning robots are lagging about 15 years behind indoor robots on this, for a couple of reasons. First, most pool robots—different from automatic pool cleaners, which are purely mechanical systems that are driven by water pressure—have been tethered to an outlet for power, meaning that maximizing efficiency is less of a concern. And second, 3D underwater localization is a much different (and arguably more difficult) problem to solve than 2D indoor localization was. But pool robots are catching up, and at CES this week, Wybot introduced an untethered robot that uses ultrasound to generate a 3D map for fast, efficient pool cleaning. And it’s solar powered and self-emptying, too.

Underwater localization and navigation is not an easy problem for any robot. Private pools are certainly privileged to be operating environments with a reasonable amount of structure and predictability, at least if everything is working the way it should. But the lighting is always going to be a challenge, between bright sunlight, deep shadow, wave reflections, and occasionally murky water if the pool chemicals aren’t balanced very well. That makes relying on any light-based localization system iffy at best, and so Wybot has gone old school, with ultrasound.

Ultrasound used to be a very common way for mobile robots to navigate. You may (or may not) remember venerable robots like the Pioneer 3, with those big ultrasonic sensors across its front. As cameras and lidar got cheap and reliable, the messiness of ultrasonic sensors fell out of favor, but sound is still ideal for underwater applications where anything that relies on light may struggle.

The Wybot S3 uses 12 ultrasonic sensors, plus motor encoders and an inertial measurement unit to map residential pools in three dimensions. “We had to choose the ultrasonic sensors very carefully,” explains Felix (Huo) Feng, the CTO of Wybot. “Actually, we use multiple different sensors, and we compute time of flight [of the sonar pulses] to calculate distance.” The positional accuracy of the resulting map is about 10 centimeters, which is totally fine for the robot to get its job done, although Feng says that they’re actively working to improve the map’s resolution. For path planning purposes, the 3D map gets deconstructed into a series of 2D maps, since the robot needs to clean the bottom of the pool, stairs and ledges, and also the sides of the pool.

Efficiency is particularly important for the S3 because its charging dock has enough solar panels on the top of it to provide about 90 minutes of runtime for the robot over the course of an optimally sunny day. If your pool isn’t too big, that means the robot can clean it daily without requiring a power connection to the dock. The dock also sucks debris out of the collection bin on the robot itself, and Wybot suggests that the S3 can go for up to a month of cleaning without the dock overflowing.

The S3 has a camera on the front, which is used primarily to identify and prioritize dirtier areas (through AI, of course) that need focused cleaning. At some point in the future, Wybot may be able to use vision for navigation too, but my guess is that for reliable 24/7 navigation, ultrasound will still be necessary.

One other interesting little tidbit is the communication system. The dock can talk to your Wi-Fi, of course, and then talk to the robot while it’s charging. Once the robot goes off for a swim, however, traditional wireless signals won’t work, but the dock has its own sonar that can talk to the robot at several bytes per second. This isn’t going to get you streaming video from the robot’s camera, but it’s enough to let you steer the robot if you want, or ask it to come back to the dock, get battery status updates, and similar sorts of things.

The Wybot S3 will go on sale in Q2 of this year for a staggering US $2,999, but that’s how it always works: The first time a new technology shows up in the consumer space, it’s inevitably at a premium. Give it time, though, and my guess is that the ability to navigate and self-empty will become standard features in pool robots. But as far as I know, Wybot got there first.

Autonomous systems, particularly fleets of drones and other unmanned vehicles, face increasing risks as their complexity grows. Despite advancements, existing testing frameworks fall short in addressing end-to-end security, resilience, and safety in zero-trust environments. The Secure Systems Research Center (SSRC) at TII has developed a rigorous, holistic testing framework to systematically evaluate the performance and security of these systems at each stage of development. This approach ensures secure, resilient, and safe operations for autonomous systems, from individual components to fleet-wide interactions.

Earlier this year, we reviewed the SwitchBot S10, a vacuuming and wet mopping robot that uses a water-integrated docking system to autonomously manage both clean and dirty water for you. It’s a pretty clever solution, and we appreciated that SwitchBot was willing to try something a little different.

At CES this week, SwitchBot introduced the K20+ Pro, a little autonomous vacuum that can integrate with a bunch of different accessories by pulling them around on a backpack cart of sorts. The K20+ Pro is SwitchBot’s latest effort to explore what’s possible with mobile home robots.

SwitchBot’s small vacuum can transport different payloads on top.SwitchBot

What we’re looking at here is a “mini” robotic vacuum (it’s about 25 centimeters in diameter) that does everything a robotic vacuum does nowadays: It uses lidar to make a map of your house so that you can direct it where to go, it’s got a dock to empty itself and recharge, and so on. The mini robotic vacuum is attached to a wheeled platform that SwitchBot is calling a “FusionPlatform” that sits on top of the robot like a hat. The vacuum docks to this platform, and then the platform will go wherever the robot goes. This entire system (robot, dock, and platform) is the “K20+ Pro multitasking household robot.”

SwitchBot refers to the K20+ Pro as a “smart delivery assistant,” because you can put stuff on the FusionPlatform and the K20+ Pro will move that stuff around your house for you. This really doesn’t do it justice, though, because the platform is much more than just a passive mobile cart. It also can provide power to a bunch of different accessories, all of which benefit from autonomous mobility:

The SwitchBot can carry a variety of payloads, including custom payloads.SwitchBot

From left to right, you’re looking at an air circulation fan, a tablet stand, a vacuum and charging dock and an air purifier and security camera (and a stick vacuum for some reason), and lastly just the air purifier and security setup. You can also add and remove different bits, like if you want the fan along with the security camera, just plop the security camera down on the platform base in front of the fan and you’re good to go.

This basic concept is somewhat similar to Amazon’s Proteus robot, in the sense that you can have one smart powered base that moves around a bunch of less smart and unpowered payloads by driving underneath them and then carrying them around. But SwitchBot’s payloads aren’t just passive cargo, and the base can provide them with a useful amount of power.

A power port allows you to develop your own payloads for the robot.SwitchBot

SwitchBot is actively encouraging users to “to create, adapt, and personalize the robot for a wide variety of innovative applications,” which may include “3D-printed components [or] third-party devices with multiple power ports for speakers, car fridges, or even UV sterilization lamps,” according to the press release. The maximum payload is only 8 kilograms, though, so don’t get too crazy.

Several SwitchBots can make bath time much more enjoyable.SwitchBot

What we all want to know is when someone will put an arm on this thing, and SwitchBot is of course already working on this:

SwitchBot’s mobile manipulator is still in the lab stage.SwitchBot

The arm is still “in the lab stage,” SwichBot says, which I’m guessing means that the hardware is functional but that getting it to reliably do useful stuff with the arm is still a work in progress. But that’s okay—getting an arm to reliably do useful stuff is a work in progress for all of robotics, pretty much. And if SwitchBot can manage to produce an affordable mobile manipulation platform for consumers that even sort of works, that’ll be very impressive.

Video Friday is your weekly selection of awesome robotics videos, collected by your friends at IEEE Spectrum robotics. We also post a weekly calendar of upcoming robotics events for the next few months. Please send us your events for inclusion.

Enjoy today’s videos!

It’s me. But we can all relate to this child android robot struggling to stay awake.

[ Osaka University ]

[ Team B-Human ]

[ Imperial College London ]

Thanks, Digby!

[ Zarrouk Lab ]

[ Franka Robotics ]

Aldebaran has sold an astonishing number of robots this year.

[ Aldebaran ]

[ Ailos Robotics ]

[ MAVRL via Github ]

[ Personal Robotics Lab ]

A drone’s eye-view reminder that fireworks explode in 3D.

[ Team BlackSheep ]

The future of human habitation in the sea is taking shape in an abandoned quarry on the border of Wales and England. There, the ocean-exploration organization Deep has embarked on a multiyear quest to enable scientists to live on the seafloor at depths up to 200 meters for weeks, months, and possibly even years.

“Aquarius Reef Base in St. Croix was the last installed habitat back in 1987, and there hasn’t been much ground broken in about 40 years,” says Kirk Krack, human diver performance lead at Deep. “We’re trying to bring ocean science and engineering into the 21st century.”

This article is part of our special report Top Tech 2025.

Deep’s agenda has a major milestone this year—the development and testing of a small, modular habitat called Vanguard. This transportable, pressurized underwater shelter, capable of housing up to three divers for periods ranging up to a week or so, will be a stepping stone to a more permanent modular habitat system—known as Sentinel—that is set to launch in 2027. “By 2030, we hope to see a permanent human presence in the ocean,” says Krack. All of this is now possible thanks to an advanced 3D printing-welding approach that can print these large habitation structures.

How would such a presence benefit marine science? Krack runs the numbers for me: “With current diving at 150 to 200 meters, you can only get 10 minutes of work completed, followed by 6 hours of decompression. With our underwater habitats we’ll be able to do seven years’ worth of work in 30 days with shorter decompression time. More than 90 percent of the ocean’s biodiversity lives within 200 meters’ depth and at the shorelines, and we only know about 20 percent of it.” Understanding these undersea ecosystems and environments is a crucial piece of the climate puzzle, he adds: The oceans absorb nearly a quarter of human-caused carbon dioxide and roughly 90 percent of the excess heat generated by human activity.

Deep is looking to build an underwater life-support infrastructure that features not just modular habitats but also training programs for the scientists who will use them. Long-term habitation underwater involves a specialized type of activity called saturation diving, so named because the diver’s tissues become saturated with gases, such as nitrogen or helium. It has been used for decades in the offshore oil and gas sectors but is uncommon in scientific diving, outside of the relatively small number of researchers fortunate enough to have spent time in Aquarius. Deep wants to make it a standard practice for undersea researchers.

The first rung in that ladder is Vanguard, a rapidly deployable, expedition-style underwater habitat the size of a shipping container that can be transported and supplied by a ship and house three people down to depths of about 100 meters. It is set to be tested in a quarry outside of Chepstow, Wales, in the first quarter of 2025.

The Vanguard habitat, seen here in an illustrator’s rendering, will be small enough to be transportable and yet capable of supporting three people at a maximum depth of 100 meters.Deep

The plan is to be able to deploy Vanguard wherever it’s needed for a week or so. Divers will be able to work for hours on the seabed before retiring to the module for meals and rest.

One of the novel features of Vanguard is its extraordinary flexibility when it comes to power. There are currently three options: When deployed close to shore, it can connect by cable to an onshore distribution center using local renewables. Farther out at sea, it could use supply from floating renewable-energy farms and fuel cells that would feed Vanguard via an umbilical link, or it could be supplied by an underwater energy-storage system that contains multiple batteries that can be charged, retrieved, and redeployed via subsea cables.

The breathing gases will be housed in external tanks on the seabed and contain a mix of oxygen and helium that will depend on the depth. In the event of an emergency, saturated divers won’t be able to swim to the surface without suffering a life-threatening case of decompression illness. So, Vanguard, as well as the future Sentinel, will also have backup power sufficient to provide 96 hours of life support, in an external, adjacent pod on the seafloor.

Data gathered from Vanguard this year will help pave the way for Sentinel, which will be made up of pods of different sizes and capabilities. These pods will even be capable of being set to different internal pressures, so that different sections can perform different functions. For example, the labs could be at the local bathymetric pressure for analyzing samples in their natural environment, but alongside those a 1-atmosphere chamber could be set up where submersibles could dock and visitors could observe the habitat without needing to equalize with the local pressure.

As Deep sees it, a typical configuration would house six people—each with their own bedroom and bathroom. It would also have a suite of scientific equipment including full wet labs to perform genetic analyses, saving days by not having to transport samples to a topside lab for analysis.

“By 2030, we hope to see a permanent human presence in the ocean,” says one of the project’s principals

A Sentinel configuration is designed to go for a month before needing a resupply. Gases will be topped off via an umbilical link from a surface buoy, and food, water, and other supplies would be brought down during planned crew changes every 28 days.

But people will be able to live in Sentinel for months, if not years. “Once you’re saturated, it doesn’t matter if you’re there for six days or six years, but most people will be there for 28 days due to crew changes,” says Krack.

It’s a very ambitious vision, and Deep has concluded that it can be achieved only with advanced manufacturing techniques. Deep’s manufacturing arm, Deep Manufacturing Labs (DML), has come up with an innovative approach for building the pressure hulls of the habitat modules. It’s using robots to combine metal additive manufacturing with welding in a process known as wire-arc additive manufacturing. With these robots, metal layers are built up as they would be in 3D printing, but the layers are fused together via welding using a metal-inert-gas torch.

At Deep’s base of operations at a former quarry in Tidenham, England, resources include two Triton 3300/3 MK II submarines. One of them is seen here at Deep’s floating “island” dock in the quarry. Deep

During a tour of the DML, Harry Thompson, advanced manufacturing engineering lead, says, “We sit in a gray area between welding and additive process, so we’re following welding rules, but for pressure vessels we [also] follow a stress-relieving process that is applicable for an additive component. We’re also testing all the parts with nondestructive testing.”

Each of the robot arms has an operating range of 2.8 by 3.2 meters, but DML has boosted this area by means of a concept it calls Hexbot. It’s based on six robotic arms programmed to work in unison to create habitat hulls with a diameter of up to 6.1 meters. The biggest challenge with creating the hulls is managing the heat during the additive process to keep the parts from deforming as they are created. For this, DML is relying on the use of heat-tolerant steels and on very precisely optimized process parameters.

Besides manufacturing, there are other challenges that are unique to the tricky business of keeping people happy and alive 200 meters underwater. One of the most fascinating of these revolves around helium. Because of its narcotic effect at high pressure, nitrogen shouldn’t be breathed by humans at depths below about 60 meters. So, at 200 meters, the breathing mix in the habitat will be 2 percent oxygen and 98 percent helium. But because of its very high thermal conductivity, “we need to heat helium to 31–32 °C to get a normal 21–22 °C internal temperature environment,” says Rick Goddard, director of engineering at Deep. “This creates a humid atmosphere, so porous materials become a breeding ground for mold”.

There are a host of other materials-related challenges, too. The materials can’t emit gases, and they must be acoustically insulating, lightweight, and structurally sound at high pressures.

Deep’s proving grounds are a former quarry in Tidenham, England, that has a maximum depth of 80 meters. Deep

There are also many electrical challenges. “Helium breaks certain electrical components with a high degree of certainty,” says Goddard. “We’ve had to pull devices to pieces, change chips, change [printed circuit boards], and even design our own PCBs that don’t off-gas.”

The electrical system will also have to accommodate an energy mix with such varied sources as floating solar farms and fuel cells on a surface buoy. Energy-storage devices present major electrical engineering challenges: Helium seeps into capacitors and can destroy them when it tries to escape during decompression. Batteries, too, develop problems at high pressure, so they will have to be housed outside the habitat in 1-atmosphere pressure vessels or in oil-filled blocks that prevent a differential pressure inside.

When you’re trying to be the SpaceX of the ocean, questions are naturally going to fly about the feasibility of such an ambition. How likely is it that Deep can follow through? At least one top authority, John Clarke, is a believer. “I’ve been astounded by the quality of the engineering methods and expertise applied to the problems at hand and I am enthusiastic about how DEEP is applying new technology,” says Clarke, who was lead scientist of the U.S. Navy Experimental Diving Unit. “They are advancing well beyond expectations…. I gladly endorse Deep in their quest to expand humankind’s embrace of the sea.”

2024 was the best year ever for robotics, which I’m pretty sure is not something that I’ve ever said before. But that’s the great thing about robotics—it’s always new, and it’s always exciting. What may be different about this year is the real sense that not only is AI going to change everything about robots, but that it will somehow make robots useful and practical and commercially viable. Is that true? Nobody knows yet! But it means that 2025 might actually be the best year ever for robotics, if you’ve ever wanted a robot to help you out at home or at work.

So as we look forward to 2025, here are some of our most interesting and impactful stories of the past year. And as always, thanks for reading!

Figure

This announcement from back in February is pretty much what set the tone for robotics in 2024. Figure’s Series B raise valued the company at a bonkers US $2.6 billion, and all of a sudden, humanoids were where it’s at. And by “it,” I mean everything, from funding to talent to breathless media coverage. The big question of 2024 was whether or not humanoids would be able to deliver on their promises, and that’s shaping up to be the big question of 2025, too.

Boston Dynamics

It didn’t take long for legendary robotics company Boston Dynamics to make it clear that they’re not going to be left behind when it comes to commercial humanoids. For a company that has been leading humanoid research longer than just about anyone but has bounced around from owner to owner over the last 10 years, we were a little unsure whether Atlas would ever be more than a research platform. But the new all-electric Atlas is designed for work, and we saw it get busy in 2024.

Boston Dynamics

As much as we’re excited for the new Atlas, the old hydraulic Atlas will always have a special place in our hearts. We’ve been through so much together, from the DRC to parkour to dancing. Electric robots are great and all, and I understand why they’re necessary for commercial applications, but all of that hydraulic power meant that hydraulic Atlas was able to move in dynamic ways that we may not see again for a very long time.

Nvidia

So we’ve got all these humanoid robots now with all this impressive hardware, but the really hard part (or one of them, anyway) is getting those robots to actually do something commercially useful in a safe and reliable way. Is training in simulation the answer? I don’t know, but NVIDIA sure thinks so, and they’ve made a huge commitment by investing in GR00T, a “general-purpose foundation model for humanoid robots.” And what does that mean, exactly? Nobody’s quite sure yet, but with NVIDIA behind it that’s enough to make the entire industry pay attention.

Evan Ackerman

With all the attention on humanoid robots right now, it’s critical to be able to separate real progress from hype. Unfortunately, there are all kinds of ways of cheating with robots. And there’s really nothing wrong with cheating with robots, as long as you tell people that the cheating is happening, and then (hopefully) cheat less and less as your robot gets better and better. In particular, we’re likely to see more and more teleoperation of humanoid robots (obviously or otherwise) because that’s one of the best ways of collecting training data: by having a human do it. And being able to tell that a human is doing it is an important skill to have.

Machina Labs

Some of my favorite robots are robots that are able to leverage their robotic-ness to not just do things that humans do, but also do things that humans cannot do. Robots have the patience and precision to work metal in ways that a very highly skilled human might be able to do once, but the robots (being robots) can do it over and over again. NASA is leveraging this capability to build complex toroidal tanks for spacecraft, but it has the potential to change anything that’s made out of sheet metal.

JPL-Caltech/ASU/NASA

One of the greatest robotics stories of the last several years has been Ingenuity, the little Mars helicopter. We’ve written extensively about how Ingenuity was designed, how it can fly on Mars, and how it just kept on flying, more than 50 times. But it couldn’t fly forever, and as Ingenuity was pushed to fly farther and farther over more challenging terrain, flight 72 was to be its last. After losing its ability to localize over some particularly featureless terrain, the little robot had a very rough landing. It lived to tell the tale, but not to fly again.