We tend to think about hopping robots from the ground up. That is, they start on the ground, and then, by hopping, incorporate a aerial phase into their locomotion. But there’s no reason why aerial robots can’t approach hopping from the other direction, by adding a hopping ground phase to flight. Hopcopter is the first robot that I’ve ever seen give this a try, and it’s remarkably effective, combining a tiny quadrotor with a springy leg to hop hop hop all over the place.

Songnan Bai, Runze Ding, Song Li, and Bingxuan Pu

So why in the air is it worth adding a pogo stick to an otherwise perfectly functional quadrotor? Well, flying is certainly a valuable ability to have, but does take a lot of energy. If you pay close attention to birds (acknowledged experts in the space), they tend to spend a substantial amount of time doing their level best not to fly, often by walking on the ground or jumping around in trees. Not flying most of the time is arguably one of the things that makes birds so successful—it’s that multimodal locomotion capability that has helped them to adapt to so many different environments and situations.

Hopcopter is multimodal as well, although in a slightly more restrictive sense: Its two modes are flying and intermittent flying. But the intermittent flying is very important, because cutting down on that flight phase gives Hopcopter some of the same efficiency benefits that birds experience. By itself, a quadrotor of hopcopter’s size can stay airborne for about 400 seconds, while Hopcopter can hop continuously for more than 20 minutes. If your objective is to cover as much distance as possible, Hopcopter might not be as effective as a legless quadrotor. But if your objective is instead something like inspection or search and rescue, where you need to spend a fair amount of time not moving very much, hopping could be significantly more effective.

Hopcopter is a small quadcopter (specifically a Crazyflie) attached to a springy pogo-stick leg.Songnan Bai, Runze Ding, Song Li, and Bingxuan Pu

Hopcopter can reposition itself on the fly to hop off of different surfaces.Songnan Bai, Runze Ding, Song Li, and Bingxuan Pu

The actual hopping is mostly passive. Hopcopter’s leg is two rigid pieces connected by rubber bands, with a Crazyflie microcopter stapled to the top. During a hop, the Crazyflie can add directional thrust to keep the hops hopping and alter its direction as well as its height, from 0.6 meters to 1.6 meters. There isn’t a lot of room for extra sensors on Hopcopter, but the addition of some stabilizing fins allow for continuous hopping without any positional feedback.

Besides vertical hopping, Hopcopter can also position itself in midair to hop off of surfaces at other orientations, allowing it to almost instantaneously change direction, which is a neat trick.

And it can even do mid air somersaults, because why not?

Hopcopter’s repertoire of tricks includes somersaults.Songnan Bai, Runze Ding, Song Li, and Bingxuan Pu

The researchers, based at the City University of Hong Kong, say that the Hopcopter technology (namely, the elastic leg) could be easily applied to most other quadcopter platforms, turning them into Hopcopters as well. And if you’re more interested in extra payload rather than extra endurance, it’s possible to use hopping in situations where a payload would be too heavy for continuous flight.

The researchers published their work 10 April in Science Robotics.

We tend to think about hopping robots from the ground up. That is, they start on the ground, and then, by hopping, incorporate a aerial phase into their locomotion. But there’s no reason why aerial robots can’t approach hopping from the other direction, by adding a hopping ground phase to flight. Hopcopter is the first robot that I’ve ever seen give this a try, and it’s remarkably effective, combining a tiny quadrotor with a springy leg to hop hop hop all over the place.

Songnan Bai, Runze Ding, Song Li, and Bingxuan Pu

So why in the air is it worth adding a pogo stick to an otherwise perfectly functional quadrotor? Well, flying is certainly a valuable ability to have, but does take a lot of energy. If you pay close attention to birds (acknowledged experts in the space), they tend to spend a substantial amount of time doing their level best not to fly, often by walking on the ground or jumping around in trees. Not flying most of the time is arguably one of the things that makes birds so successful—it’s that multimodal locomotion capability that has helped them to adapt to so many different environments and situations.

Hopcopter is multimodal as well, although in a slightly more restrictive sense: Its two modes are flying and intermittent flying. But the intermittent flying is very important, because cutting down on that flight phase gives Hopcopter some of the same efficiency benefits that birds experience. By itself, a quadrotor of hopcopter’s size can stay airborne for about 400 seconds, while Hopcopter can hop continuously for more than 20 minutes. If your objective is to cover as much distance as possible, Hopcopter might not be as effective as a legless quadrotor. But if your objective is instead something like inspection or search and rescue, where you need to spend a fair amount of time not moving very much, hopping could be significantly more effective.

Hopcopter is a small quadcopter (specifically a Crazyflie) attached to a springy pogo-stick leg.Songnan Bai, Runze Ding, Song Li, and Bingxuan Pu

Hopcopter can reposition itself on the fly to hop off of different surfaces.Songnan Bai, Runze Ding, Song Li, and Bingxuan Pu

The actual hopping is mostly passive. Hopcopter’s leg is two rigid pieces connected by rubber bands, with a Crazyflie microcopter stapled to the top. During a hop, the Crazyflie can add directional thrust to keep the hops hopping and alter its direction as well as its height, from 0.6 meters to 1.6 meters. There isn’t a lot of room for extra sensors on Hopcopter, but the addition of some stabilizing fins allow for continuous hopping without any positional feedback.

Besides vertical hopping, Hopcopter can also position itself in midair to hop off of surfaces at other orientations, allowing it to almost instantaneously change direction, which is a neat trick.

And it can even do mid air somersaults, because why not?

Hopcopter’s repertoire of tricks includes somersaults.Songnan Bai, Runze Ding, Song Li, and Bingxuan Pu

The researchers, based at the City University of Hong Kong, say that the Hopcopter technology (namely, the elastic leg) could be easily applied to most other quadcopter platforms, turning them into Hopcopters as well. And if you’re more interested in extra payload rather than extra endurance, it’s possible to use hopping in situations where a payload would be too heavy for continuous flight.

The researchers published their work 10 April in Science Robotics.

Last December, the AI Institute announced that it was opening an office in Zurich as a European counterpart to its Boston headquarters and recruited Marco Hutter to helm the office. Hutter also runs the Robotic Systems Lab at ETH Zurich, arguably best known as the origin of the ANYmal quadruped robot (but it also does tons of other cool stuff).

We’re doing our best to keep close tabs on the institute, because it’s one of a vanishingly small number of places that currently exist where roboticists have the kind of long-term resources and vision necessary to make substantial progress on really hard problems that aren’t quite right for either industry or academia. The institute is still scaling up (and the branch in Zurich has only just kicked things off), but we did spot some projects that the Boston folks have been working on, and as you can see from the clips at the top of this page, they’re looking pretty cool.

Meanwhile, we had a chance to check in with Marco Hutter to get a sense of what the Zurich office will be working on and how he’s going to be solving all of the hard problems in robotics. All of them!

How much can you tell us about what you’ll be working on at the AI Institute?

Marco Hutter: If you know the research that I’ve been doing in the past at ETH and with our startups, there’s an overlap on making systems more mobile, making systems more able to interact with the world, making systems in general more capable on the hardware and software side. And that’s what the institute strives for.

The institute describes itself as a research organization that aims to solve the most important and fundamental problems in robotics and AI. What do you think those problems are?

Marco Hutter is the head of the AI Institute’s new Zurich branch.Swiss Robotics Day

Hutter: There are lots of problems. If you’re looking at robots today, we have to admit that they’re still pretty stupid. The way they move, their capability of understanding their environment, the way they’re able to interact with unstructured environments—I think we’re still lacking a lot of skills on the robotic side to make robots useful in all of the tasks we wish them to do. So we have the ambition of having these robots taking over all these dull, dirty, and dangerous jobs. But if we’re honest, today the biggest impact is really only for the dull part. And I think these dirty and dangerous jobs, where we really need support from robots, that’s still going to take a lot of fundamental work on the robotics and AI side to make enough progress for robots to become useful tools.

What is it about the institute that you think will help robotics make more progress in these areas?

Hutter: I think the institute is one of these unique places where we are trying to bring the benefits of the academic world and the benefits from this corporate world together. In academia, we have all kinds of crazy ideas and we try to develop them in all different directions, but at the same time, we have limited engineering support, and we can only go so far. Making robust and reliable hardware systems is a massive effort, and that kind of engineering is much better done in a corporate lab.

You’ve seen this a little bit with the type of work my lab has been doing in the past. We built simple quadrupeds with a little bit of mobility, but in order to make them robust, we eventually had to spin it out. We had to bring it to the corporate world, because for a research group, a pure academic group, it would have been impossible. But at the same time, you’re losing something, right? Once you go into your corporate world and you’re running a business, you have to be very focused; you can’t be that explorative and free anymore.

So if you bring these two things together through the institute, with long-term planning, enough financial support, and brilliant people both in the U.S. and Europe working together, I think that’s what will hopefully help us make significant progress in the next couple of years.

“We’re very different from a traditional company, where at some point you need to have a product that makes money. Here, it’s really about solving problems and taking the next step.” —Marco Hutter, AI Institute

And what will that actually mean in the context of dynamically mobile robots?

Hutter: If you look at Boston Dynamics’ Atlas doing parkour, or ANYmal doing parkour, these are still demonstrations. You don’t see robots running around in the forests or robots working in mines and doing all kinds of crazy maintenance operations, or in industrial facilities, or construction sites, you name it. We need to not only be able to do this once as a prototype demonstration, but to have all the capabilities that bring that together with environmental perception and understanding to make this athletic intelligence more capable and more adaptable to all kinds of different environments. This is not something that from today to tomorrow we’re going to see it being revolutionized—it will be gradual, steady progress because I think there’s still a lot of fundamental work that needs to be done.

I feel like the mobility of legged robots has improved a lot over the last five years or so, and a lot of that progress has come from Boston Dynamics and also from your lab. Do you feel the same?

Hutter: There has always been progress; the question is how much you can zoom in or zoom out. I think one thing has changed quite a bit, and that’s the availability of robotic systems to all kinds of different research groups. If you look back a decade, people had to build their own robots, they had to do the control for the robots, they had to work on the perception for the robots, and putting everything together like that makes it extremely fragile and very challenging to make something that works more than once. That has changed, which allows us to make faster progress.

Marc Raibert (founder of the AI Institute) likes to show videos of mountain goats to illustrate what robots should be (or will be?) capable of. Does that kind of thing inspire you as well?

Hutter: If you look at the animal kingdom, there’s so many things you can draw inspiration from. And a lot of this stuff is not only the cognitive side; it’s really about pairing the cognitive side with the mechanical intelligence of things like the simple-seeming hooves of mountain goats. But they’re really not that simple, they’re pretty complex in how they interact with the environment. Having one of these things and not the other won’t allow the animal to move across its challenging environment. It’s the same thing with the robots.

It’s always been like this in robotics, where you push on the hardware side, and your controls become better, so you hit a hardware limitation. So both things have to evolve hand in hand. Otherwise, you have an over-dimensioned hardware system that you can’t use because you don’t have the right controls, or you have very sophisticated controls and your hardware system can’t keep up.

How do you feel about all of the investment into humanoids right now, when quadrupedal robots with arms have been around for quite a while?

Hutter: There’s a lot of ongoing research on quadrupeds with arms, and the nice thing is that these technologies that are developed for mobile systems with arms are the same technologies that are used in humanoids. It’s not different from a research point of view, it’s just a different form factor for the system. I think from an application point of view, the story from all of these companies making humanoids is that our environment has been adapted to humans quite a bit. A lot of tasks are at the height of a human standing, right? A quadruped doesn’t have the height to see things or to manipulate things on a table. It’s really application dependent, and I wouldn’t say that one system is better than the other.

Last December, the AI Institute announced that it was opening an office in Zurich as a European counterpart to its Boston headquarters and recruited Marco Hutter to helm the office. Hutter also runs the Robotic Systems Lab at ETH Zurich, arguably best known as the origin of the ANYmal quadruped robot (but it also does tons of other cool stuff).

We’re doing our best to keep close tabs on the institute, because it’s one of a vanishingly small number of places that currently exist where roboticists have the kind of long-term resources and vision necessary to make substantial progress on really hard problems that aren’t quite right for either industry or academia. The institute is still scaling up (and the branch in Zurich has only just kicked things off), but we did spot some projects that the Boston folks have been working on, and as you can see from the clips at the top of this page, they’re looking pretty cool.

Meanwhile, we had a chance to check in with Marco Hutter to get a sense of what the Zurich office will be working on and how he’s going to be solving all of the hard problems in robotics. All of them!

How much can you tell us about what you’ll be working on at the AI Institute?

Marco Hutter: If you know the research that I’ve been doing in the past at ETH and with our startups, there’s an overlap on making systems more mobile, making systems more able to interact with the world, making systems in general more capable on the hardware and software side. And that’s what the institute strives for.

The institute describes itself as a research organization that aims to solve the most important and fundamental problems in robotics and AI. What do you think those problems are?

Marco Hutter is the head of the AI Institute’s new Zurich branch.Swiss Robotics Day

Hutter: There are lots of problems. If you’re looking at robots today, we have to admit that they’re still pretty stupid. The way they move, their capability of understanding their environment, the way they’re able to interact with unstructured environments—I think we’re still lacking a lot of skills on the robotic side to make robots useful in all of the tasks we wish them to do. So we have the ambition of having these robots taking over all these dull, dirty, and dangerous jobs. But if we’re honest, today the biggest impact is really only for the dull part. And I think these dirty and dangerous jobs, where we really need support from robots, that’s still going to take a lot of fundamental work on the robotics and AI side to make enough progress for robots to become useful tools.

What is it about the institute that you think will help robotics make more progress in these areas?

Hutter: I think the institute is one of these unique places where we are trying to bring the benefits of the academic world and the benefits from this corporate world together. In academia, we have all kinds of crazy ideas and we try to develop them in all different directions, but at the same time, we have limited engineering support, and we can only go so far. Making robust and reliable hardware systems is a massive effort, and that kind of engineering is much better done in a corporate lab.

You’ve seen this a little bit with the type of work my lab has been doing in the past. We built simple quadrupeds with a little bit of mobility, but in order to make them robust, we eventually had to spin it out. We had to bring it to the corporate world, because for a research group, a pure academic group, it would have been impossible. But at the same time, you’re losing something, right? Once you go into your corporate world and you’re running a business, you have to be very focused; you can’t be that explorative and free anymore.

So if you bring these two things together through the institute, with long-term planning, enough financial support, and brilliant people both in the U.S. and Europe working together, I think that’s what will hopefully help us make significant progress in the next couple of years.

“We’re very different from a traditional company, where at some point you need to have a product that makes money. Here, it’s really about solving problems and taking the next step.” —Marco Hutter, AI Institute

And what will that actually mean in the context of dynamically mobile robots?

Hutter: If you look at Boston Dynamics’ Atlas doing parkour, or ANYmal doing parkour, these are still demonstrations. You don’t see robots running around in the forests or robots working in mines and doing all kinds of crazy maintenance operations, or in industrial facilities, or construction sites, you name it. We need to not only be able to do this once as a prototype demonstration, but to have all the capabilities that bring that together with environmental perception and understanding to make this athletic intelligence more capable and more adaptable to all kinds of different environments. This is not something that from today to tomorrow we’re going to see it being revolutionized—it will be gradual, steady progress because I think there’s still a lot of fundamental work that needs to be done.

I feel like the mobility of legged robots has improved a lot over the last five years or so, and a lot of that progress has come from Boston Dynamics and also from your lab. Do you feel the same?

Hutter: There has always been progress; the question is how much you can zoom in or zoom out. I think one thing has changed quite a bit, and that’s the availability of robotic systems to all kinds of different research groups. If you look back a decade, people had to build their own robots, they had to do the control for the robots, they had to work on the perception for the robots, and putting everything together like that makes it extremely fragile and very challenging to make something that works more than once. That has changed, which allows us to make faster progress.

Marc Raibert (founder of the AI Institute) likes to show videos of mountain goats to illustrate what robots should be (or will be?) capable of. Does that kind of thing inspire you as well?

Hutter: If you look at the animal kingdom, there’s so many things you can draw inspiration from. And a lot of this stuff is not only the cognitive side; it’s really about pairing the cognitive side with the mechanical intelligence of things like the simple-seeming hooves of mountain goats. But they’re really not that simple, they’re pretty complex in how they interact with the environment. Having one of these things and not the other won’t allow the animal to move across its challenging environment. It’s the same thing with the robots.

It’s always been like this in robotics, where you push on the hardware side, and your controls become better, so you hit a hardware limitation. So both things have to evolve hand in hand. Otherwise, you have an over-dimensioned hardware system that you can’t use because you don’t have the right controls, or you have very sophisticated controls and your hardware system can’t keep up.

How do you feel about all of the investment into humanoids right now, when quadrupedal robots with arms have been around for quite a while?

Hutter: There’s a lot of ongoing research on quadrupeds with arms, and the nice thing is that these technologies that are developed for mobile systems with arms are the same technologies that are used in humanoids. It’s not different from a research point of view, it’s just a different form factor for the system. I think from an application point of view, the story from all of these companies making humanoids is that our environment has been adapted to humans quite a bit. A lot of tasks are at the height of a human standing, right? A quadruped doesn’t have the height to see things or to manipulate things on a table. It’s really application dependent, and I wouldn’t say that one system is better than the other.

Rapid and resourceful technological improvisation has long been a mainstay of warfare, but the war in Ukraine is taking it to a new level. This improvisation is most conspicuous in the ceaselessly evolving struggle between weaponized drones and electronic warfare, a cornerstone of this war.

Weaponized civilian first-person-view (FPV) drones began dramatically reshaping the landscape of the war in the summer of 2023. Prior to this revolution, various commercial drones played critical roles, primarily for intelligence, surveillance, and reconnaissance. Since 2014, the main means of defending against these drones has been electronic warfare (EW), in its many forms. The iterative, lethal dance between drones and EW has unfolded a rich technological tapestry, revealing insights into a likely future of warfare where EW and drones intertwine.

After the invasion of Crimea, in 2014, Ukrainian forces depended heavily on commercial off-the-shelf drones, such as models from DJI, for reconnaissance and surveillance. These were not FPV drones, for the most part. Russia’s response involved deploying military-grade EW systems alongside law-enforcement tools like Aeroscope, a product from DJI that allows instant identification and tracking of drones from their radio emissions. Aeroscope, while originally a standard tool used by law enforcement to detect and track illegal drone flights, soon revealed its military potential by pinpointing both the drone and its operator.

On both sides of the line you’ll find much the same kind of people doing much the same thing: hacking.

This application turned a security feature into a significant tactical asset, providing Russian artillery units with precise coordinates for their targets—namely, Ukrainian drone operators. To circumvent this vulnerability, groups of Ukrainian volunteers innovated. By updating the firmware of the DJI drones, they closed the backdoors that allowed the drones to be tracked by Aeroscope. Nevertheless, after the start of the conflict in Crimea, commercial, off-the-shelf drones were considered a last-resort asset used by volunteers to compensate for the lack of proper military systems. To be sure, the impact of civilian drones during this period was not comparable to what occurred after the February 2022 invasion.

As Russia’s “thunder-run” strategy became bogged down shortly after the invasion, Russian forces found themselves unexpectedly vulnerable to civilian drones, in part because most of their full-scale military EW systems were not very mobile.

During a training exercise in southern Ukraine in May 2023, a drone pilot maneuvered a flier to a height of 100 meters before dropping a dummy anti-tank grenade on to a pile of tires. The test, pictured here, worked—that night the pilot’s team repeated the exercise over occupied territory, blowing up a Russian armored vehicle. Emre Caylak/Guardian/eyevine/Redux

The Russians could have compensated by deploying many Aeroscope terminals then, but they didn’t, because most Russian officers at the time had a dismissive view of the capabilities of civilian drones in a high-intensity conflict. That failure opened a window of opportunity that Ukrainian armed-forces units exploited aggressively. Military personnel, assisted by many volunteer technical specialists, gained a decisive intelligence advantage for their forces by quickly fielding fleets of hundreds of camera drones connected to simple yet effective battlefield-management systems. They soon began modifying commercial drones to attack, with grenade tosses and, ultimately, “kamikaze” operations. Besides the DJI models, one of the key drones was the R18, an octocopter developed by the Ukrainian company Aerorozvidka, capable of carrying three grenades or small bombs. As casualties mounted, Russian officers soon realized the extent of the threat posed by these drones.

By spring 2023, as the front lines stabilized following strategic withdrawals and counteroffensives, it was clear that the nature of drone warfare had evolved. Russian defenses had adapted, deploying more sophisticated counter-drone systems. Russian forces were also beginning to use drones, setting the stage for the nuanced cat-and-mouse game that has been going on ever since.

The modular construction of first-person-view drones allowed for rapid evolution to enhance their resilience against electronic warfare.

For example, early on, most Russian EW efforts primarily focused on jamming the drones’ radio links for control and video. This wasn’t too hard, given that DJI’s OcuSync protocol was not designed to withstand dense jamming environments. So by April 2023, Ukrainian drone units had begun pivoting toward first-person-view (FPV) drones with modular construction, enabling rapid adaptation to, and evasion of, EW countermeasures.

The Russian awakening to the importance of drones coincided with the stabilization of the front lines, around August 2022. Sluggish Russian offensives came at a high cost, with an increasing proportion of casualties caused directly or indirectly by drone operators. By this time, the Ukrainians were hacking commercial drones, such as DJI Mavics, to “anonymize” them, rendering Aeroscope useless. It was also at this time that the Russians began to adopt commercial drones and develop their own tactics, techniques, and procedures, leveraging their EW and artillery advantages while attempting to compensate for their delay in combat-drone usage.

On 4 March, a Ukrainian soldier flew a drone at a testing site near the town of Kreminna in eastern Ukraine. The drone was powered by a blue battery pack and carried a dummy bomb.David Guttenfelder/The New York Times/Redux

Throughout 2023, when the primary EW tactic employed was jamming, the DJI drones began to fall out of favor for attack roles. When the density of Russian jammer usage surpassed a certain threshold, DJI’s OcuSync radio protocol, which controls a drone’s flight direction and video, could not cope with it. Being proprietary, OcuSync’s frequency band and power are not modifiable. A jammer can attack both the control and video signals, and the drone becomes unrecoverable most of the time. As a result, DJI drones have lately been used farther from the front lines and relegated mainly to roles in intelligence, surveillance, and reconnaissance. Meanwhile, the modular construction of FPVs allowed for rapid evolution to enhance their resilience against EW. The Ukraine war greatly boosted the world’s production of FPV drones; at this point there are thousands of FPV models and modifications.

A “kamikaze” first-person-view drone with an attached PG-7L round, intended for use with an RPG-7 grenade launcher, is readied for a mission near the town of Horlivka, in the Donetsk region, on 17 January 2024. The drone was prepared by a Ukrainian serviceman of the Rarog UAV squadron of the 24th Separate Mechanized Brigade.Inna Varenytsia/Reuters/Redux

As of early 2024, analog video signals are the most popular option by far. This technology offers drone operators a brief window of several seconds to correct the drone’s path upon detecting interference, for example as a result of jamming, before signal loss. Additionally, drone manufacturers have access to more powerful video transmitters, up to 5 watts, which are more resistant to jamming. Furthermore, the 1.2-gigahertz frequency band is gaining popularity over the previously dominant 5.8-GHz band due to its superior obstacle penetration and because fewer jammers are targeting that band.

However, the lack of encryption in analog video transmitter systems means that a drone’s visual feed can be intercepted by any receiver. So various mitigation strategies have been explored. These include adding encryption layers and using digital-control and video protocols such as HDZero, Walksnail, or, especially, any of several new open-source alternatives.

In the war zone, the most popular of these open-source control radio protocols is ExpressLRS, or ELRS. Being open-source, ELRS not only offers more affordable hardware than its main rival, TBS Crossfire, it is also modifiable via its software. It has been hacked in order to use frequency bands other than its original 868 to 915 megahertz. This adaptation produces serious headaches for EW operators, because they have to cover a much wider band. As of March 2024, Ukrainian drone operators are performing final tests on 433-MHz ELRS transmitter-receiver pairs, further challenging prevailing EW methods.

Nevertheless, the most important recent disruption of all in the drone-versus-EW struggle is distributed mass. Instead of an envisioned blitzkrieg-style swarm with big clouds of drones hitting many closely spaced targets during very short bursts, an ever-growing number of drones are covering more widely dispersed targets over a much longer time period, whenever the weather is conducive. Distributed mass is a cornerstone of the emerging transparent battlefield, in which many different sensors and platforms transmit huge amounts of data that is integrated in real time to provide a comprehensive view of the battlefield. One offshoot of this strategy is that more and more kamikaze drones are directed toward a constantly expanding range of targets. Electronic warfare is adapting to this new reality, confronting mass with mass: massive numbers of drones against massive numbers of RF sensors and jammers.

Ukraine is the first true war of the hackers.

Attacks now often consist of far more commercial drones than a suite of RF detectors or jammers could handle even six months ago. With brute-force jamming, even if defenders are willing to accept high rates of damage inflicted on their own offensive drones, these previous EW systems are just not up to the task. So for now, at least, the drone hackers are in the lead in this deadly game of “hacksymmetrical” warfare. Their development cycle is far too rapid for conventional electronic warfare to keep pace.

But the EW forces are not standing still. Both sides are either developing or acquiring civilian RF-detecting equipment, while military-tech startups and even small volunteer groups are developing new, simple, and good-enough jammers in essentially the same improvised ways that hackers would.

Ukrainian soldiers familiarized themselves with a portable drone jammer during a training session in Kharkiv, Ukraine, on 11 March 2024.Diego Herrera Carcedo/Anadolu/Getty Images

Two examples illustrate this trend. Increasingly affordable, short-range jammers are being installed on tanks, armored personnel carriers, trucks, pickups, and even 4x4s. Although limited and unsophisticated, these systems contribute to drone-threat mitigation. In addition, a growing number of soldiers on the front line carry simple, commercial radio-frequency (RF) scanners with them. Configured to detect drones across various frequency bands, these devices, though far from perfect, have begun to save lives by providing precious additional seconds of warning before an imminent drone attack.

The electronic battlefield has now become a massive game of cat and mouse. Because commercial drones have proven so lethal and disruptive, drone operators have become high-priority targets. As a result, operators have had to reinvent camouflage techniques, while the hackers who drive the evolution of their drones are working on every modification of RF equipment that offers an advantage. Besides the frequency-band modification described above, hackers have developed and refined two-way, two-signal repeaters for drones. Such systems are attached to another drone that hovers close to the operator and well above the ground, relaying signals to and from the attacking drone. Such repeaters more than double the practical range of drone communications, and thus the EW “cats” in this game have to search a much wider area than before.

Hackers and an emerging cottage industry of war startups are raising the stakes. Their primary goal is to erode the effectiveness of jammers by attacking them autonomously. In this countermeasure, offensive drones are equipped with home-on-jam systems. Over the next several months, increasingly sophisticated versions of these systems will be fielded. These home-on-jam capabilities will autonomously target any jamming emission within range; this range, which is classified, depends on emission power at a rate that is believed to be 0.3 kilometers per watt. In other words, if a jammer has 100 W of signal power, it can be detected up to 30 km away, and then attacked. After these advances allow the drone “mice” to hunt the EW cat, what will happen to the cat?

The challenge is unprecedented and the outcome uncertain. But on both sides of the line you’ll find much the same kind of people doing much the same thing: hacking. Civilian hackers have for years lent their skills to such shady enterprises as narco-trafficking and organized crime. Now hacking is a major, indispensable component of a full-fledged war, and its practitioners have emerged from a gray zone of plausible deniability into the limelight of military prominence. Ukraine is the first true war of the hackers.

The implications for Western militaries are ominous. We have neither masses of drones nor masses of EW tech. What is worse, the world’s best hackers are completely disconnected from the development of defense systems. The Ukrainian experience, where a vibrant war startup scene is emerging, suggests a model for integrating maverick hackers into our defense strategies. As the first hacker war continues to unfold, it serves as a reminder that in the era of electronic and drone warfare, the most critical assets are not just the technologies we deploy but also the scale and the depth of the human ingenuity behind them.

Rapid and resourceful technological improvisation has long been a mainstay of warfare, but the war in Ukraine is taking it to a new level. This improvisation is most conspicuous in the ceaselessly evolving struggle between weaponized drones and electronic warfare, a cornerstone of this war.

Weaponized civilian first-person-view (FPV) drones began dramatically reshaping the landscape of the war in the summer of 2023. Prior to this revolution, various commercial drones played critical roles, primarily for intelligence, surveillance, and reconnaissance. Since 2014, the main means of defending against these drones has been electronic warfare (EW), in its many forms. The iterative, lethal dance between drones and EW has unfolded a rich technological tapestry, revealing insights into a likely future of warfare where EW and drones intertwine.

After the invasion of Crimea, in 2014, Ukrainian forces depended heavily on commercial off-the-shelf drones, such as models from DJI, for reconnaissance and surveillance. These were not FPV drones, for the most part. Russia’s response involved deploying military-grade EW systems alongside law-enforcement tools like Aeroscope, a product from DJI that allows instant identification and tracking of drones from their radio emissions. Aeroscope, while originally a standard tool used by law enforcement to detect and track illegal drone flights, soon revealed its military potential by pinpointing both the drone and its operator.

On both sides of the line you’ll find much the same kind of people doing much the same thing: hacking.

This application turned a security feature into a significant tactical asset, providing Russian artillery units with precise coordinates for their targets—namely, Ukrainian drone operators. To circumvent this vulnerability, groups of Ukrainian volunteers innovated. By updating the firmware of the DJI drones, they closed the backdoors that allowed the drones to be tracked by Aeroscope. Nevertheless, after the start of the conflict in Crimea, commercial, off-the-shelf drones were considered a last-resort asset used by volunteers to compensate for the lack of proper military systems. To be sure, the impact of civilian drones during this period was not comparable to what occurred after the February 2022 invasion.

As Russia’s “thunder-run” strategy became bogged down shortly after the invasion, Russian forces found themselves unexpectedly vulnerable to civilian drones, in part because most of their full-scale military EW systems were not very mobile.

During a training exercise in southern Ukraine in May 2023, a drone pilot maneuvered a flier to a height of 100 meters before dropping a dummy anti-tank grenade on to a pile of tires. The test, pictured here, worked—that night the pilot’s team repeated the exercise over occupied territory, blowing up a Russian armored vehicle. Emre Caylak/Guardian/eyevine/Redux

The Russians could have compensated by deploying many Aeroscope terminals then, but they didn’t, because most Russian officers at the time had a dismissive view of the capabilities of civilian drones in a high-intensity conflict. That failure opened a window of opportunity that Ukrainian armed-forces units exploited aggressively. Military personnel, assisted by many volunteer technical specialists, gained a decisive intelligence advantage for their forces by quickly fielding fleets of hundreds of camera drones connected to simple yet effective battlefield-management systems. They soon began modifying commercial drones to attack, with grenade tosses and, ultimately, “kamikaze” operations. Besides the DJI models, one of the key drones was the R18, an octocopter developed by the Ukrainian company Aerorozvidka, capable of carrying three grenades or small bombs. As casualties mounted, Russian officers soon realized the extent of the threat posed by these drones.

By spring 2023, as the front lines stabilized following strategic withdrawals and counteroffensives, it was clear that the nature of drone warfare had evolved. Russian defenses had adapted, deploying more sophisticated counter-drone systems. Russian forces were also beginning to use drones, setting the stage for the nuanced cat-and-mouse game that has been going on ever since.

The modular construction of first-person-view drones allowed for rapid evolution to enhance their resilience against electronic warfare.

For example, early on, most Russian EW efforts primarily focused on jamming the drones’ radio links for control and video. This wasn’t too hard, given that DJI’s OcuSync protocol was not designed to withstand dense jamming environments. So by April 2023, Ukrainian drone units had begun pivoting toward first-person-view (FPV) drones with modular construction, enabling rapid adaptation to, and evasion of, EW countermeasures.

The Russian awakening to the importance of drones coincided with the stabilization of the front lines, around August 2022. Sluggish Russian offensives came at a high cost, with an increasing proportion of casualties caused directly or indirectly by drone operators. By this time, the Ukrainians were hacking commercial drones, such as DJI Mavics, to “anonymize” them, rendering Aeroscope useless. It was also at this time that the Russians began to adopt commercial drones and develop their own tactics, techniques, and procedures, leveraging their EW and artillery advantages while attempting to compensate for their delay in combat-drone usage.

On 4 March, a Ukrainian soldier flew a drone at a testing site near the town of Kreminna in eastern Ukraine. The drone was powered by a blue battery pack and carried a dummy bomb.David Guttenfelder/The New York Times/Redux

Throughout 2023, when the primary EW tactic employed was jamming, the DJI drones began to fall out of favor for attack roles. When the density of Russian jammer usage surpassed a certain threshold, DJI’s OcuSync radio protocol, which controls a drone’s flight direction and video, could not cope with it. Being proprietary, OcuSync’s frequency band and power are not modifiable. A jammer can attack both the control and video signals, and the drone becomes unrecoverable most of the time. As a result, DJI drones have lately been used farther from the front lines and relegated mainly to roles in intelligence, surveillance, and reconnaissance. Meanwhile, the modular construction of FPVs allowed for rapid evolution to enhance their resilience against EW. The Ukraine war greatly boosted the world’s production of FPV drones; at this point there are thousands of FPV models and modifications.

A “kamikaze” first-person-view drone with an attached PG-7L round, intended for use with an RPG-7 grenade launcher, is readied for a mission near the town of Horlivka, in the Donetsk region, on 17 January 2024. The drone was prepared by a Ukrainian serviceman of the Rarog UAV squadron of the 24th Separate Mechanized Brigade.Inna Varenytsia/Reuters/Redux

As of early 2024, analog video signals are the most popular option by far. This technology offers drone operators a brief window of several seconds to correct the drone’s path upon detecting interference, for example as a result of jamming, before signal loss. Additionally, drone manufacturers have access to more powerful video transmitters, up to 5 watts, which are more resistant to jamming. Furthermore, the 1.2-gigahertz frequency band is gaining popularity over the previously dominant 5.8-GHz band due to its superior obstacle penetration and because fewer jammers are targeting that band.

However, the lack of encryption in analog video transmitter systems means that a drone’s visual feed can be intercepted by any receiver. So various mitigation strategies have been explored. These include adding encryption layers and using digital-control and video protocols such as HDZero, Walksnail, or, especially, any of several new open-source alternatives.

In the war zone, the most popular of these open-source control radio protocols is ExpressLRS, or ELRS. Being open-source, ELRS not only offers more affordable hardware than its main rival, TBS Crossfire, it is also modifiable via its software. It has been hacked in order to use frequency bands other than its original 868 to 915 megahertz. This adaptation produces serious headaches for EW operators, because they have to cover a much wider band. As of March 2024, Ukrainian drone operators are performing final tests on 433-MHz ELRS transmitter-receiver pairs, further challenging prevailing EW methods.

Nevertheless, the most important recent disruption of all in the drone-versus-EW struggle is distributed mass. Instead of an envisioned blitzkrieg-style swarm with big clouds of drones hitting many closely spaced targets during very short bursts, an ever-growing number of drones are covering more widely dispersed targets over a much longer time period, whenever the weather is conducive. Distributed mass is a cornerstone of the emerging transparent battlefield, in which many different sensors and platforms transmit huge amounts of data that is integrated in real time to provide a comprehensive view of the battlefield. One offshoot of this strategy is that more and more kamikaze drones are directed toward a constantly expanding range of targets. Electronic warfare is adapting to this new reality, confronting mass with mass: massive numbers of drones against massive numbers of RF sensors and jammers.

Ukraine is the first true war of the hackers.

Attacks now often consist of far more commercial drones than a suite of RF detectors or jammers could handle even six months ago. With brute-force jamming, even if defenders are willing to accept high rates of damage inflicted on their own offensive drones, these previous EW systems are just not up to the task. So for now, at least, the drone hackers are in the lead in this deadly game of “hacksymmetrical” warfare. Their development cycle is far too rapid for conventional electronic warfare to keep pace.

But the EW forces are not standing still. Both sides are either developing or acquiring civilian RF-detecting equipment, while military-tech startups and even small volunteer groups are developing new, simple, and good-enough jammers in essentially the same improvised ways that hackers would.

Ukrainian soldiers familiarized themselves with a portable drone jammer during a training session in Kharkiv, Ukraine, on 11 March 2024.Diego Herrera Carcedo/Anadolu/Getty Images

Two examples illustrate this trend. Increasingly affordable, short-range jammers are being installed on tanks, armored personnel carriers, trucks, pickups, and even 4x4s. Although limited and unsophisticated, these systems contribute to drone-threat mitigation. In addition, a growing number of soldiers on the front line carry simple, commercial radio-frequency (RF) scanners with them. Configured to detect drones across various frequency bands, these devices, though far from perfect, have begun to save lives by providing precious additional seconds of warning before an imminent drone attack.

The electronic battlefield has now become a massive game of cat and mouse. Because commercial drones have proven so lethal and disruptive, drone operators have become high-priority targets. As a result, operators have had to reinvent camouflage techniques, while the hackers who drive the evolution of their drones are working on every modification of RF equipment that offers an advantage. Besides the frequency-band modification described above, hackers have developed and refined two-way, two-signal repeaters for drones. Such systems are attached to another drone that hovers close to the operator and well above the ground, relaying signals to and from the attacking drone. Such repeaters more than double the practical range of drone communications, and thus the EW “cats” in this game have to search a much wider area than before.

Hackers and an emerging cottage industry of war startups are raising the stakes. Their primary goal is to erode the effectiveness of jammers by attacking them autonomously. In this countermeasure, offensive drones are equipped with home-on-jam systems. Over the next several months, increasingly sophisticated versions of these systems will be fielded. These home-on-jam capabilities will autonomously target any jamming emission within range; this range, which is classified, depends on emission power at a rate that is believed to be 0.3 kilometers per watt. In other words, if a jammer has 100 W of signal power, it can be detected up to 30 km away, and then attacked. After these advances allow the drone “mice” to hunt the EW cat, what will happen to the cat?

The challenge is unprecedented and the outcome uncertain. But on both sides of the line you’ll find much the same kind of people doing much the same thing: hacking. Civilian hackers have for years lent their skills to such shady enterprises as narco-trafficking and organized crime. Now hacking is a major, indispensable component of a full-fledged war, and its practitioners have emerged from a gray zone of plausible deniability into the limelight of military prominence. Ukraine is the first true war of the hackers.

The implications for Western militaries are ominous. We have neither masses of drones nor masses of EW tech. What is worse, the world’s best hackers are completely disconnected from the development of defense systems. The Ukrainian experience, where a vibrant war startup scene is emerging, suggests a model for integrating maverick hackers into our defense strategies. As the first hacker war continues to unfold, it serves as a reminder that in the era of electronic and drone warfare, the most critical assets are not just the technologies we deploy but also the scale and the depth of the human ingenuity behind them.

Video Friday is your weekly selection of awesome robotics videos, collected by your friends at IEEE Spectrum robotics. We also post a weekly calendar of upcoming robotics events for the next few months. Please send us your events for inclusion.

Enjoy today’s videos!

[ USC ]

[ Naver Labs ]

I think “needs a lot of work” really means “needs a couple more arms.”

[ Paper ]

[ UBTECH ]

I know the fins on this robot are for walking underwater rather than on land, but watching it move, I feel like it’s destined to evolve into something a little more terrestrial.

[ Paper ] via [ HERO Lab ]

iRobot has a new Roomba that vacuums and mops—and at $275, it’s a pretty good deal.

Also, if you are a robot vacuum owner, please, please remember to clean the poor thing out from time to time. Here’s how to do it with a Roomba:

[ iRobot ]

[ AVFL ]

Sony is still upgrading Aibo with new features, like the ability to listen to your terrible music and dance along.

[ Aibo ]

[ Max Planck Institute ]

[ ESA ]

Aww, they stopped before they got to the fun rocks.

[ Paper ] via [ Michigan Robotics ]

Not bad for 2016, right?

[ Namiki Lab ]

[ RILAB ] via [ KIMLAB ]

This UPenn GRASP SFI Seminar is from Eric Jang at 1X Technologies, on “Data Engines for Humanoid Robots.”

[ UPenn ]

This Microsoft Future Leaders in Robotics and AI Seminar is from Chahat Deep Singh at the University of Maryland, on “Minimal Perception: Enabling Autonomy in Palm-Sized Robots.”

[ UMD ]

Video Friday is your weekly selection of awesome robotics videos, collected by your friends at IEEE Spectrum robotics. We also post a weekly calendar of upcoming robotics events for the next few months. Please send us your events for inclusion.

Enjoy today’s videos!

[ USC ]

[ Naver Labs ]

I think “needs a lot of work” really means “needs a couple more arms.”

[ Paper ]

[ UBTECH ]

I know the fins on this robot are for walking underwater rather than on land, but watching it move, I feel like it’s destined to evolve into something a little more terrestrial.

[ Paper ] via [ HERO Lab ]

iRobot has a new Roomba that vacuums and mops—and at $275, it’s a pretty good deal.

Also, if you are a robot vacuum owner, please, please remember to clean the poor thing out from time to time. Here’s how to do it with a Roomba:

[ iRobot ]

[ AVFL ]

Sony is still upgrading Aibo with new features, like the ability to listen to your terrible music and dance along.

[ Aibo ]

[ Max Planck Institute ]

[ ESA ]

Aww, they stopped before they got to the fun rocks.

[ Paper ] via [ Michigan Robotics ]

Not bad for 2016, right?

[ Namiki Lab ]

[ RILAB ] via [ KIMLAB ]

This UPenn GRASP SFI Seminar is from Eric Jang at 1X Technologies, on “Data Engines for Humanoid Robots.”

[ UPenn ]

This Microsoft Future Leaders in Robotics and AI Seminar is from Chahat Deep Singh at the University of Maryland, on “Minimal Perception: Enabling Autonomy in Palm-Sized Robots.”

[ UMD ]

When we think about robotic manipulation, the default is usually to think about grippers—about robots using manipulators (like fingers or other end effectors) to interact with objects. For most humans, though, interacting with objects can be a lot more complicated, and we use whatever body parts are convenient to help us deal with objects that are large or heavy or awkward.

This somewhat constrained definition of robotic manipulation isn’t robotics’ fault, really. The word “manipulation” itself comes from the Latin for getting handsy with stuff, so there’s a millennium or two’s-worth of hand-related inertia behind the term. The Los Altos, Calif.-based Toyota Research Institute (TRI) is taking a more expansive view with their new humanoid, Punyo, which uses its soft body to help it manipulate objects that would otherwise be pretty much impossible to manage with grippers alone.

“An anthropomorphic embodiment allows us to explore the complexities of social interactions like physical assistance, non-verbal communication, intent, predictability, and trust, to name just a few.” —Alex Alspach, Toyota Research Institute (TRI)

Punyo started off as just a squishy gripper at TRI, but the idea was always to scale up to a big squishy humanoid, hence this concept art of a squishified T-HR3:

This concept image shows what Toyota’s T-HR3 humanoid might look like when bubble-ized.TRI

“We use the term ‘bubble-ized,’ says Alex Alspach, Tech Lead for Punyo at TRI. Alspach tells us that the concept art above doesn’t necessarily reflect what the Punyo humanoid will eventually look like, but “it gave us some physical constraints and a design language. It also reinforced the idea that we are after general hardware and software solutions that can augment and enable both future and existing robots to take full advantage of their whole bodies for manipulation.”

This version of Punyo isn’t quite at “whole” body manipulation, but it can get a lot done using its arms and chest, which are covered with air bladders that provide both sensing and compliance:

Many of those motions look very human-like, because this is how humans manipulate things. Not to throw too much shade at all those humanoid warehouse robots, but as is pointed out in the video above, using just our hands outstretched in front of us to lift things is not how humans do it, because using other parts of our bodies to provide extra support makes lifting easier. This is not a trivial problem for robots, though, because interactions between point contacts that are rigid (like how most robotics manipulators handle the world) are fairly well understood. Once you throw big squishy surfaces into the mix, along with big squishy objects, it’s just not something that most robots are ready for.

“A soft robot does not interact with the world at a single point.” —Russ Tedrake, TRI

“Current robot manipulation evolved from big, strong industrial robots moving car parts and big tools with their end effectors,” Alspach says. “I think it’s wise to take inspiration from the human form—we are strong enough to perform most everyday tasks with our hands, but when a big, heavy object comes around, we need to get creative with how we wrap our arms around it and position our body to lift it.”

Robots are notorious for lifting big and heavy objects, primarily by manipulating them with robot-y form factors in robot-y ways. So what’s so great about the human form factor, anyway? This question goes way beyond Punyo, of course, but we wanted to get the Punyo team’s take on humanoids, and we tossed a couple more questions at them just for fun.

IEEE Spectrum: So why humanoids?

Alspach: The humanoid robot checks a few important boxes. First of all, the environments we intend to work in were built for humans, so the humanoid form helps a robot make use of the spaces and tools around it. Independently, multiple teams at TRI have converged on bi-manual systems for tasks like grocery shopping and food preparation. A chest between these arms is a simple addition that gives us useful contact surfaces for manipulating big objects, too. Furthermore, our Human-Robot Interaction (HRI) team has done, and continues to do, extensive research with older adults, the people we look forward to helping the most. An anthropomorphic embodiment allows us to explore the complexities of social interactions like physical assistance, non-verbal communication, intent, predictability, and trust, to name just a few.

“We focus not on highly precise tasks but on gross, whole-body manipulation, where robust strategies help stabilize and control objects, and a bit of sloppiness can be an asset.” —Alex Alspach, TRI

Does having a bubble-ized robot make anything more difficult for you?

Russ Tedrake, VP of Robotics Research: If you think of your robot as interacting with the world at a point—the standard view from e.g. impedance control—then putting a soft, passive spring in series between your robot and the world does limit performance. It reduces your control bandwidth. But that view misses the more important point. A soft robot does not interact with the world at a single point. Soft materials fundamentally change the dynamics of contact by deforming around the material—generating patch contacts that allow contact forces and moments not achievable by a rigid interaction.

Alspach: Punyo’s softness is extreme compared to other manipulation platforms that may, say, just have rubber pads on their arms or fingers. This compliance means that when we grab an object, it may not settle exactly where we planned for it to, or, for example, if we bump that object up against the edge of a table, it may move within our grasp. For these reasons, tactile sensing is an important part of our solution as we dig into how to measure and control the state of the objects we manipulate. We focus not on highly precise tasks but on gross, whole-body manipulation, where robust strategies help stabilize and control objects, and a bit of sloppiness can be an asset.

Compliance can be accomplished in different ways, including just in software. What’s the importance of having a robot that’s physically squishy rather than just one that acts squishily?

Andrew Beaulieu, Punyo Tech Lead: We do not believe that passive and active compliance should be considered mutually exclusive, and there are several advantages to having a physically squishy robot, especially when we consider having a robot operate near people and in their spaces. Having a robot that can safely make contact with the world opens up avenues of interaction and exploration. Using compliant materials on the robot also allows it to conform to complicated shapes passively in a way that would otherwise involve more complicated articulated or actuated mechanisms. Conforming to the objects allows us to increase the contact patch with the object and distribute the forces, usually creating a more robust grasp. These compliant surfaces allow us to research planning and control methods that might be less precise, rely less on accurate object localization, or use hardware with less precise control or sensing.

What’s it like to be hugged by Punyo?

Kate Tsui, Punyo HRI Tech Lead: Although Punyo isn’t a social robot, a surprising amount of emotion comes through its hug, and it feels quite comforting. A hug from Punyo feels like a long, sustained, snug squeeze from a close friend you haven’t seen for a long time and don’t want to let go.

A series of concept images shows situations in which whole body manipulation might be useful in the home.TRI

(Interview transcript ends.)

Softness seems like it could be a necessary condition for bipedal humanoids working in close proximity to humans, especially in commercial or home environments where interactions are less structured and predictable. “I think more robots using their whole body to manipulate is coming soon, especially with the recent explosion of humanoids outside of academic labs,” Alspach says. “Capable, general-purpose robotic manipulation is a competitive field, and using the whole body unlocks the ability to efficiently manipulate large, heavy, and unwieldy objects.”

When we think about robotic manipulation, the default is usually to think about grippers—about robots using manipulators (like fingers or other end effectors) to interact with objects. For most humans, though, interacting with objects can be a lot more complicated, and we use whatever body parts are convenient to help us deal with objects that are large or heavy or awkward.

This somewhat constrained definition of robotic manipulation isn’t robotics’ fault, really. The word “manipulation” itself comes from the Latin for getting handsy with stuff, so there’s a millennium or two’s-worth of hand-related inertia behind the term. The Los Altos, Calif.-based Toyota Research Institute (TRI) is taking a more expansive view with their new humanoid, Punyo, which uses its soft body to help it manipulate objects that would otherwise be pretty much impossible to manage with grippers alone.

“An anthropomorphic embodiment allows us to explore the complexities of social interactions like physical assistance, non-verbal communication, intent, predictability, and trust, to name just a few.” —Alex Alspach, Toyota Research Institute (TRI)

Punyo started off as just a squishy gripper at TRI, but the idea was always to scale up to a big squishy humanoid, hence this concept art of a squishified T-HR3:

This concept image shows what Toyota’s T-HR3 humanoid might look like when bubble-ized.TRI

“We use the term ‘bubble-ized,’ says Alex Alspach, Tech Lead for Punyo at TRI. Alspach tells us that the concept art above doesn’t necessarily reflect what the Punyo humanoid will eventually look like, but “it gave us some physical constraints and a design language. It also reinforced the idea that we are after general hardware and software solutions that can augment and enable both future and existing robots to take full advantage of their whole bodies for manipulation.”

This version of Punyo isn’t quite at “whole” body manipulation, but it can get a lot done using its arms and chest, which are covered with air bladders that provide both sensing and compliance:

Many of those motions look very human-like, because this is how humans manipulate things. Not to throw too much shade at all those humanoid warehouse robots, but as is pointed out in the video above, using just our hands outstretched in front of us to lift things is not how humans do it, because using other parts of our bodies to provide extra support makes lifting easier. This is not a trivial problem for robots, though, because interactions between point contacts that are rigid (like how most robotics manipulators handle the world) are fairly well understood. Once you throw big squishy surfaces into the mix, along with big squishy objects, it’s just not something that most robots are ready for.

“A soft robot does not interact with the world at a single point.” —Russ Tedrake, TRI

“Current robot manipulation evolved from big, strong industrial robots moving car parts and big tools with their end effectors,” Alspach says. “I think it’s wise to take inspiration from the human form—we are strong enough to perform most everyday tasks with our hands, but when a big, heavy object comes around, we need to get creative with how we wrap our arms around it and position our body to lift it.”

Robots are notorious for lifting big and heavy objects, primarily by manipulating them with robot-y form factors in robot-y ways. So what’s so great about the human form factor, anyway? This question goes way beyond Punyo, of course, but we wanted to get the Punyo team’s take on humanoids, and we tossed a couple more questions at them just for fun.

IEEE Spectrum: So why humanoids?

Alspach: The humanoid robot checks a few important boxes. First of all, the environments we intend to work in were built for humans, so the humanoid form helps a robot make use of the spaces and tools around it. Independently, multiple teams at TRI have converged on bi-manual systems for tasks like grocery shopping and food preparation. A chest between these arms is a simple addition that gives us useful contact surfaces for manipulating big objects, too. Furthermore, our Human-Robot Interaction (HRI) team has done, and continues to do, extensive research with older adults, the people we look forward to helping the most. An anthropomorphic embodiment allows us to explore the complexities of social interactions like physical assistance, non-verbal communication, intent, predictability, and trust, to name just a few.

“We focus not on highly precise tasks but on gross, whole-body manipulation, where robust strategies help stabilize and control objects, and a bit of sloppiness can be an asset.” —Alex Alspach, TRI

Does having a bubble-ized robot make anything more difficult for you?

Russ Tedrake, VP of Robotics Research: If you think of your robot as interacting with the world at a point—the standard view from e.g. impedance control—then putting a soft, passive spring in series between your robot and the world does limit performance. It reduces your control bandwidth. But that view misses the more important point. A soft robot does not interact with the world at a single point. Soft materials fundamentally change the dynamics of contact by deforming around the material—generating patch contacts that allow contact forces and moments not achievable by a rigid interaction.

Alspach: Punyo’s softness is extreme compared to other manipulation platforms that may, say, just have rubber pads on their arms or fingers. This compliance means that when we grab an object, it may not settle exactly where we planned for it to, or, for example, if we bump that object up against the edge of a table, it may move within our grasp. For these reasons, tactile sensing is an important part of our solution as we dig into how to measure and control the state of the objects we manipulate. We focus not on highly precise tasks but on gross, whole-body manipulation, where robust strategies help stabilize and control objects, and a bit of sloppiness can be an asset.

Compliance can be accomplished in different ways, including just in software. What’s the importance of having a robot that’s physically squishy rather than just one that acts squishily?

Andrew Beaulieu, Punyo Tech Lead: We do not believe that passive and active compliance should be considered mutually exclusive, and there are several advantages to having a physically squishy robot, especially when we consider having a robot operate near people and in their spaces. Having a robot that can safely make contact with the world opens up avenues of interaction and exploration. Using compliant materials on the robot also allows it to conform to complicated shapes passively in a way that would otherwise involve more complicated articulated or actuated mechanisms. Conforming to the objects allows us to increase the contact patch with the object and distribute the forces, usually creating a more robust grasp. These compliant surfaces allow us to research planning and control methods that might be less precise, rely less on accurate object localization, or use hardware with less precise control or sensing.

What’s it like to be hugged by Punyo?

Kate Tsui, Punyo HRI Tech Lead: Although Punyo isn’t a social robot, a surprising amount of emotion comes through its hug, and it feels quite comforting. A hug from Punyo feels like a long, sustained, snug squeeze from a close friend you haven’t seen for a long time and don’t want to let go.

A series of concept images shows situations in which whole body manipulation might be useful in the home.TRI

(Interview transcript ends.)

Softness seems like it could be a necessary condition for bipedal humanoids working in close proximity to humans, especially in commercial or home environments where interactions are less structured and predictable. “I think more robots using their whole body to manipulate is coming soon, especially with the recent explosion of humanoids outside of academic labs,” Alspach says. “Capable, general-purpose robotic manipulation is a competitive field, and using the whole body unlocks the ability to efficiently manipulate large, heavy, and unwieldy objects.”

This paper presents a novel webcam-based approach for gaze estimation on computer screens. Utilizing appearance based gaze estimation models, the system provides a method for mapping the gaze vector from the user’s perspective onto the computer screen. Notably, it determines the user’s 3D position in front of the screen, using only a 2D webcam without the need for additional markers or equipment. The study presents a comprehensive comparative analysis, assessing the performance of the proposed method against established eye tracking solutions. This includes a direct comparison with the purpose-built Tobii Eye Tracker 5, a high-end hardware solution, and the webcam-based GazeRecorder software. In experiments replicating head movements, especially those imitating yaw rotations, the study brings to light the inherent difficulties associated with tracking such motions using 2D webcams. This research introduces a solution by integrating Structure from Motion (SfM) into the Convolutional Neural Network (CNN) model. The study’s accomplishments include showcasing the potential for accurate screen gaze tracking with a simple webcam, presenting a novel approach for physical distance computation, and proposing compensation for head movements, laying the groundwork for advancements in real-world gaze estimation scenarios.

Video Friday is your weekly selection of awesome robotics videos, collected by your friends at IEEE Spectrum robotics. We also post a weekly calendar of upcoming robotics events for the next few months. Please send us your events for inclusion.

Enjoy today’s videos!

[ Columbia ]

[ Stanford ]

I’m sorry, but the world does not need more drummers.

[ Fourier Intelligence ]

Always good to see NASA’s Valakyrie doing research.

[ NASA ]

[ Paper ]

[ Flyability ]

[ Paper ]

Thanks, Kento!

[ Paper ]

Thanks, Kelly!

[ Agility ]

[ Boston Dynamics ]

[ Paper ]

[ Paper ]

Catch me if you can!

[ CVUT ]

[ CaT ]

[ TED ]

[ JPL ]

[ Penn Engineering ]

Video Friday is your weekly selection of awesome robotics videos, collected by your friends at IEEE Spectrum robotics. We also post a weekly calendar of upcoming robotics events for the next few months. Please send us your events for inclusion.

Enjoy today’s videos!

[ Columbia ]

[ Stanford ]

I’m sorry, but the world does not need more drummers.

[ Fourier Intelligence ]

Always good to see NASA’s Valakyrie doing research.

[ NASA ]

[ Paper ]

[ Flyability ]

[ Paper ]

Thanks, Kento!

[ Paper ]

Thanks, Kelly!

[ Agility ]

[ Boston Dynamics ]

[ Paper ]

[ Paper ]

Catch me if you can!

[ CVUT ]

[ CaT ]

[ TED ]

[ JPL ]

[ Penn Engineering ]

At NVIDIA GTC last week, Boston Dynamics CTO Aaron Saunders gave a talk about deploying AI in real world robots—namely, how Spot is leveraging reinforcement learning to get better at locomotion (We spoke with Saunders last year about robots falling over). And Spot has gotten a lot better—a Spot robot takes a tumble on average once every 50 kilometers, even as the Spot fleet collectively walks enough to circle the Earth every three months.

That fleet consists of a lot of commercial deployments, which is impressive for any mobile robot, but part of the reason for that is because the current version of Spot is really not intended for robotics research, even though over 100 universities are home to at least one Spot. Boston Dynamics has not provided developer access to Spot’s joints, meaning that anyone who has wanted to explore quadrupedal mobility has had to find some other platform that’s a bit more open and allows for some experimentation.

Boston Dynamics is now announcing a new variant of Spot that includes a low-level application programming interface (API) that gives joint-level control of the robot. This will give (nearly) full control over how Spot moves its legs, which is a huge opportunity for the robotics community, since we’ll now be able to find out exactly what Spot is capable of. For example, we’ve already heard from a credible source that Spot is capable of running much, much faster than Boston Dynamics has publicly shown, and it’s safe to assume that a speedier Spot is just the start.

An example of a new Spot capability when a custom locomotion controller can be used on the robot.Boston Dynamics

When you buy a Spot robot from Boston Dynamics, it arrives already knowing how to walk. It’s very, very good at walking. Boston Dynamics is so confident in Spot’s walking ability that you’re only allowed high-level control of the robot: You tell it where to go, it decides how to get there. If you want to do robotics research using Spot as a mobility platform, that’s totally fine, but if you want to do research on quadrupedal locomotion, it hasn’t been possible with Spot. But that’s changing.

The Spot RL Researcher Kit is a collaboration between Boston Dynamics, Nvidia, and the AI Institute. It includes a joint-level control API, an Nvidia Jetson AGX Orin payload, and a simulation environment for Spot based on Nvidia Isaac Lab. The kit will be officially released later this year, but Boston Dynamics is starting a slow rollout through an early adopter beta program.

From a certain perspective, Boston Dynamics did this whole thing with Spot backwards by first creating a commercial product and only then making it into a research platform. “At the beginning, we felt like it would be great to include that research capability, but that it wasn’t going to drive the adoption of this technology,” Saunders told us after his GTC session. Instead, Boston Dynamics first focused on getting lots of Spots out into the world in a useful way, and only now, when the company feels like they’ve gotten there, is the time right to unleash a fully-featured research version of Spot. “It was really just getting comfortable with our current product that enabled us to go back and say, ‘how can we now provide people with the kind of access that they’re itching for?’”

Getting to this point has taken a huge amount of work for Boston Dynamics. Predictably, Spot started out as a novelty for most early adopters, becoming a project for different flavors of innovation groups within businesses rather than an industrial asset. “I think there’s been a change there,” Saunders says. “We’re working with operational customers a lot more, and the composure of our sales is shifting away from being dominated by early adopters and we’re starting to see repeat sales and interest in larger fleets of robots.”

Deploying and supporting a large fleet of Spots is one of the things that allowed Boston Dynamics to feel comfortable offering a research version. Researchers are not particularly friendly to their robots, because the goal of research is often to push the envelope of what’s possible. And part of that process includes getting very well acquainted with what turns out to be not possible, resulting in robots that end up on the floor, sometimes in pieces. The research version of Spot will include a mandatory Spot Care Service Plan, which exists to serve commercial customers but will almost certainly provide more value to the research community who want to see what kinds of crazy things they can get Spot to do.

Exactly how crazy those crazy things will be remains to be seen. Boston Dynamics is starting out with a beta program for the research Spots partially because they’re not quite sure yet how many safeguards to put in place within the API. “We need to see where the problems are,” Saunders says. “We still have a little work to do to really hone in how our customers are going to use it.” Deciding how much Spot should be able to put itself at risk in the name of research may be a difficult question to answer, but I’m pretty sure that the beta program participants are going to do their best to find out how much tolerance Boston Dynamics has for Spot shenanigans. I just hope that whatever happens, they share as much video of it as possible.

The Spot Early Adopter Program for the new RL Researcher Kit is open for applications here.

At NVIDIA GTC last week, Boston Dynamics CTO Aaron Saunders gave a talk about deploying AI in real world robots—namely, how Spot is leveraging reinforcement learning to get better at locomotion (We spoke with Saunders last year about robots falling over). And Spot has gotten a lot better—a Spot robot takes a tumble on average once every 50 kilometers, even as the Spot fleet collectively walks enough to circle the Earth every three months.

That fleet consists of a lot of commercial deployments, which is impressive for any mobile robot, but part of the reason for that is because the current version of Spot is really not intended for robotics research, even though over 100 universities are home to at least one Spot. Boston Dynamics has not provided developer access to Spot’s joints, meaning that anyone who has wanted to explore quadrupedal mobility has had to find some other platform that’s a bit more open and allows for some experimentation.

Boston Dynamics is now announcing a new variant of Spot that includes a low-level application programming interface (API) that gives joint-level control of the robot. This will give (nearly) full control over how Spot moves its legs, which is a huge opportunity for the robotics community, since we’ll now be able to find out exactly what Spot is capable of. For example, we’ve already heard from a credible source that Spot is capable of running much, much faster than Boston Dynamics has publicly shown, and it’s safe to assume that a speedier Spot is just the start.

An example of a new Spot capability when a custom locomotion controller can be used on the robot.Boston Dynamics

When you buy a Spot robot from Boston Dynamics, it arrives already knowing how to walk. It’s very, very good at walking. Boston Dynamics is so confident in Spot’s walking ability that you’re only allowed high-level control of the robot: You tell it where to go, it decides how to get there. If you want to do robotics research using Spot as a mobility platform, that’s totally fine, but if you want to do research on quadrupedal locomotion, it hasn’t been possible with Spot. But that’s changing.

The Spot RL Researcher Kit is a collaboration between Boston Dynamics, Nvidia, and the AI Institute. It includes a joint-level control API, an Nvidia Jetson AGX Orin payload, and a simulation environment for Spot based on Nvidia Isaac Lab. The kit will be officially released later this year, but Boston Dynamics is starting a slow rollout through an early adopter beta program.

From a certain perspective, Boston Dynamics did this whole thing with Spot backwards by first creating a commercial product and only then making it into a research platform. “At the beginning, we felt like it would be great to include that research capability, but that it wasn’t going to drive the adoption of this technology,” Saunders told us after his GTC session. Instead, Boston Dynamics first focused on getting lots of Spots out into the world in a useful way, and only now, when the company feels like they’ve gotten there, is the time right to unleash a fully-featured research version of Spot. “It was really just getting comfortable with our current product that enabled us to go back and say, ‘how can we now provide people with the kind of access that they’re itching for?’”

Getting to this point has taken a huge amount of work for Boston Dynamics. Predictably, Spot started out as a novelty for most early adopters, becoming a project for different flavors of innovation groups within businesses rather than an industrial asset. “I think there’s been a change there,” Saunders says. “We’re working with operational customers a lot more, and the composure of our sales is shifting away from being dominated by early adopters and we’re starting to see repeat sales and interest in larger fleets of robots.”

Deploying and supporting a large fleet of Spots is one of the things that allowed Boston Dynamics to feel comfortable offering a research version. Researchers are not particularly friendly to their robots, because the goal of research is often to push the envelope of what’s possible. And part of that process includes getting very well acquainted with what turns out to be not possible, resulting in robots that end up on the floor, sometimes in pieces. The research version of Spot will include a mandatory Spot Care Service Plan, which exists to serve commercial customers but will almost certainly provide more value to the research community who want to see what kinds of crazy things they can get Spot to do.

Exactly how crazy those crazy things will be remains to be seen. Boston Dynamics is starting out with a beta program for the research Spots partially because they’re not quite sure yet how many safeguards to put in place within the API. “We need to see where the problems are,” Saunders says. “We still have a little work to do to really hone in how our customers are going to use it.” Deciding how much Spot should be able to put itself at risk in the name of research may be a difficult question to answer, but I’m pretty sure that the beta program participants are going to do their best to find out how much tolerance Boston Dynamics has for Spot shenanigans. I just hope that whatever happens, they share as much video of it as possible.

The Spot Early Adopter Program for the new RL Researcher Kit is open for applications here.

In recent years, virtual idols have garnered considerable attention because they can perform activities similar to real idols. However, as they are fictitious idols with nonphysical presence, they cannot perform physical interactions such as handshake. Combining a robotic hand with a display showing virtual idols is the one of the methods to solve this problem. Nonetheless a physical handshake is possible, the form of handshake that can effectively induce the desirable behavior is unclear. In this study, we adopted a robotic hand as an interface and aimed to imitate the behavior of real idols. To test the effects of this behavior, we conducted step-wise experiments. The series of experiments revealed that the handshake by the robotic hand increased the feeling of intimacy toward the virtual idol, and it became more enjoyable to respond to a request from the virtual idol. In addition, viewing the virtual idols during the handshake increased the feeling of intimacy with the virtual idol. Moreover, the method of the hand-shake peculiar to idols, which tried to keep holding the user’s hand after the conversation, increased the feeling of intimacy to the virtual idol.

Video Friday is your weekly selection of awesome robotics videos, collected by your friends at IEEE Spectrum robotics. We also post a weekly calendar of upcoming robotics events for the next few months. Please send us your events for inclusion.

Enjoy today’s videos!

[ NVIDIA ]

[ Boston Dynamics ]

[ 1X ]

What I find interesting about the Unitree H1 doing a standing flip is where it decides to put its legs.

[ Unitree ]

[ Pickle ]

[ ICD-LAB ]

[ Electric Sheep ]

[ DEEP Robotics ]

If you want to collect data for robot grasping, using Stretch and a pair of tongs is about as affordable as it gets.

[ Hello Robot ]

The real reason why Digit’s legs look backwards is so that it doesn’t bang its shins taking GPUs out of the oven.

Meanwhile, some of us can bake our GPUs without even needing an oven.

[ Agility ]

[ LimX ]

And now, this.

[ Suzumori Endo Lab ]

[ Cornell ]

[ DLR ]

[ Clearpath ]

[ KIMLAB ]

[ Paper ] via [ HKUST ]

[ CYBATHLON ]

[ DJI ]

Video Friday is your weekly selection of awesome robotics videos, collected by your friends at IEEE Spectrum robotics. We also post a weekly calendar of upcoming robotics events for the next few months. Please send us your events for inclusion.

Enjoy today’s videos!

[ NVIDIA ]

[ Boston Dynamics ]

[ 1X ]

What I find interesting about the Unitree H1 doing a standing flip is where it decides to put its legs.

[ Unitree ]

[ Pickle ]

[ ICD-LAB ]

[ Electric Sheep ]

[ DEEP Robotics ]

If you want to collect data for robot grasping, using Stretch and a pair of tongs is about as affordable as it gets.

[ Hello Robot ]

The real reason why Digit’s legs look backwards is so that it doesn’t bang its shins taking GPUs out of the oven.

Meanwhile, some of us can bake our GPUs without even needing an oven.

[ Agility ]

[ LimX ]

And now, this.

[ Suzumori Endo Lab ]

[ Cornell ]

[ DLR ]

[ Clearpath ]

[ KIMLAB ]

[ Paper ] via [ HKUST ]

[ CYBATHLON ]

[ DJI ]

Automated disassembly is increasingly in focus for Recycling, Re-use, and Remanufacturing (Re-X) activities. Trends in digitalization, in particular digital twin (DT) technologies and the digital product passport, as well as recently proposed European legislation such as the Net Zero and the Critical materials Acts will accelerate digitalization of product documentation and factory processes. In this contribution we look beyond these activities by discussing digital information for stakeholders at the Re-X segment of the value-chain. Furthermore, we present an approach to automated product disassembly based on different levels of available product information. The challenges for automated disassembly and the subsequent requirements on modeling of disassembly processes and product states for electronic waste are examined. The authors use a top-down (e.g., review of existing standards and process definitions) methodology to define an initial data model for disassembly processes. An additional bottom-up approach, whereby 5 exemplary electronics products were manually disassembled, was employed to analyze the efficacy of the initial data model and to offer improvements. This paper reports on our suggested informal data models for automatic electronics disassembly and the associated robotic skills.

The targeted use of social robots for the family demands a better understanding of multiple stakeholders’ privacy concerns, including those of parents and children. Through a co-learning workshop which introduced families to the functions and hypothetical use of social robots in the home, we present preliminary evidence from 6 families that exhibits how parents and children have different comfort levels with robots collecting and sharing information across different use contexts. Conversations and booklet answers reveal that parents adopted their child’s decision in scenarios where they expect children to have more agency, such as in cases of homework completion or cleaning up toys, and when children proposed what their parents found to be acceptable reasoning for their decisions. Families expressed relief when they shared the same reasoning when coming to conclusive decisions, signifying an agreement of boundary management between the robot and the family. In cases where parents and children did not agree, they rejected a binary, either-or decision and opted for a third type of response, reflecting skepticism, uncertainty and/or compromise. Our work highlights the benefits of involving parents and children in child- and family-centered research, including parental abilities to provide cognitive scaffolding and personalize hypothetical scenarios for their children.