Feed aggregator

Model-Based Reinforcement Learning (MBRL) algorithms have been shown to have an advantage on data-efficiency, but often overshadowed by state-of-the-art model-free methods in performance, especially when facing high-dimensional and complex problems. In this work, a novel MBRL method is proposed, called Risk-Aware Model-Based Control (RAMCO). It combines uncertainty-aware deep dynamics models and the risk assessment technique Conditional Value at Risk (CVaR). This mechanism is appropriate for real-world application since it takes epistemic risk into consideration. In addition, we use a model-free solver to produce warm-up training data, and this setting improves the performance in low-dimensional environments and covers the shortage of MBRL’s nature in the high-dimensional scenarios. In comparison with other state-of-the-art reinforcement learning algorithms, we show that it produces superior results on a walking robot model. We also evaluate the method with an Eidos environment, which is a novel experimental method with multi-dimensional randomly initialized deep neural networks to measure the performance of any reinforcement learning algorithm, and the advantages of RAMCO are highlighted.

Over the past two decades, scholars developed various unmanned sailboat platforms, but most of them have specialized designs and controllers. Whereas these robotic sailboats have good performance with open-source designs, it is actually hard for interested researchers or fans to follow and make their own sailboats with these open-source designs. Thus, in this paper, a generic and flexible unmanned sailboat platform with easy access to the hardware and software architectures is designed and tested. The commonly used 1-m class RC racing sailboat was employed to install Pixhawk V2.4.8, Arduino Mega 2,560, GPS module M8N, custom-designed wind direction sensor, and wireless 433 Mhz telegram. The widely used open-source hardware modules were selected to keep reliable and low-cost hardware setup to emphasize the generality and feasibility of the unmanned sailboat platform. In software architecture, the Pixhawk V2.4.8 provided reliable states’ feedback. The Arduino Mega 2,560 received estimated states from Pixhawk V2.4.8 and the wind vane sensor, and then controlled servo actuators of rudder and sail using simplified algorithms. Due to the complexity of introducing robot operating system and its packages, we designed a generic but real-time software architecture just using Arduino Mega 2,560. A suitable line-of-sight guidance strategy and PID-based controllers were used to let the autonomous sailboat sail at user-defined waypoints. Field tests validated the sailing performance in facing WRSC challenges. Results of fleet race, station keeping, and area scanning proved that our design and algorithms could control the 1-m class RC sailboat with acceptable accuracy. The proposed design and algorithms contributed to developing educational, low-cost, micro class autonomous sailboats with accessible, generic, and flexible hardware and software. Besides, our sailboat platform also facilitates readers to develop similar sailboats with more focus on their missions.

Soft robots are ideal for underwater manipulation in sampling and other servicing applications. Their unique features of compliance, adaptability, and being naturally waterproof enable robotic designs to be compact and lightweight, while achieving uncompromized dexterity and flexibility. However, the inherent flexibility and high nonlinearity of soft materials also results in combined complex motions, which creates both soft actuator and sensor challenges for force output, modeling, and sensory feedback, especially under highly dynamic underwater environments. To tackle these limitations, a novel Soft Origami Optical-Sensing Actuator (SOSA) with actuation and sensing integration is proposed in this paper. Inspired by origami art, the proposed sensorized actuator enables a large force output, contraction/elongation/passive bending actuation by fluid, and hybrid motion sensing with optical waveguides. The SOSA design brings two major novelties over current designs. First, it involves a new actuation-sensing mode which enables a superior large payload output and a robust and accurate sensing performance by introducing the origami design, significantly facilitating the integration of sensing and actuating technology for wider applications. Secondly, it simplifies the fabrication process for harsh environment application by investigating the boundary features between optical waveguides and ambient water, meaning the external cladding layer of traditional sensors is unnecessary. With these merits, the proposed actuator could be applied to harsh environments for complex interaction/operation tasks. To showcase the performance of the proposed SOSA actuator, a hybrid underwater 3-DOFs manipulator has been developed. The entire workflow on concept design, fabrication, modeling, experimental validation, and application are presented in detail as reference for wider effective robot-environment applications.

When NASA first sent humans to the moon, astronauts often made risky blind landings on the lunar surface because of billowing dust clouds churned up during their descent. Astronauts could avoid repeating those harrowing experiences during future missions to the moon with the help of a 3D-printed lunar landing pad designed by a NASA-backed student team.

The landing pad developed by students from 10 U.S. universities and colleges is shaped to minimize the lunar dust clouds stirred up by rocket landing burns and could eventually be made from lunar regolith material found on the moon. A prototype of the pad is scheduled to undergo a rocket hot fire test under the watchful eye of both students and NASA engineers at Camp Swift, Texas in early March.

“We showed that you can 3D print the structure with our existing prototype,” says Helen Carson, a material science and engineering student at the University of Washington in Seattle and a principal investigator for the Lunar PAD team. “For now, we have a lot of flexibility with different directions we can take depending on how the materials develop.”

Such a Lunar PAD concept could prove especially helpful with NASA’s current roadmap aimed at returning humans to the moon through the Artemis Program; the U.S. space agency has already issued contracts to companies such as SpaceX, Blue Origin, and Dynetics to start developing ideas for a human lunar lander. Any future moon landings could benefit from reducing the risk of possible catastrophe that comes from flying blind in a dust cloud. Furthermore, dust and rocks accelerated to high speeds by engine exhaust could pose a serious danger to astronauts, robots, or other equipment already on the surface of the moon.

The Lunar PAD team first came together during NASA’s L’SPACE (Lucy Student Pipeline Accelerator and Competency Enabler) Virtual Academy held in the summer of 2019. Carson and her colleagues won funding from the NASA Proposal Writing and Evaluation process to move forward on the project and to make a presentation at NASA Marshall Space Flight Center in June 2020. At that event, additional funding was awarded so that the team could print and test their pad prototype. The students also presented a paper on Lunar PAD at the AIAA SciTech Forum and Exposition that was held 19-21 January 2021.

Image: Lunar PAD Team The multidisciplinary, multiuniversity team has come up with a solution to a problem that astronauts would most certainly face when humans return to the moon.

The team’s early idea included creating an inflatable deflector that would be inflated by the rocket engine exhaust and block any debris blasted outward from the landing (or launch) zone of the pad. But that would have required transporting flexible yet durable materials manufactured on Earth to the moon.

“That got pretty complicated with material choice and design, and the actual transportation of it,” says Luke Martin, a mechanical engineering student at Arizona State University. “So we tried coming up with other more in-situ resource ideas.”

Lunar PAD currently has a top surface layer where rockets and lunar landers could both land and launch. But the key to mitigating the worst of any dust clouds or small particles accelerated to high velocities is the open interior space of the pad that sits below the top layer. Slanted grates in the top layer would channel the rocket exhaust into the interior space.

The pad’s interior includes vent dividers—some shaped like teardrops or leaflets—that help channel the rocket exhaust and any accompanying dust or rock particles outward from the center of the pad. The cosmetically appealing layout of the vent dividers—which some liken to flower petals—proved to be the most efficient pattern that came out of numerous iterations tested through flow simulations.

“It's very practical, very efficient, and just so happens to also be very beautiful,” says Vincent Murai, a mechanical engineering student at Kapiolani Community College in Honolulu.

The exhaust and any accompanying particles leave the pad’s interior space through specific exits, called Kinetic Energy Diffusers, embedded in the outside walls of the circular pad. Such diffusers consist of hollow rectangular blocks that could also include fans to convert some of the rocket exhaust’s excess energy into the circular fan motion and block some particles with the turning fan blades. 

Any high-velocity particles that get through the fans would also encounter deflectors placed right outside the exits in the full-scale version of the pad. And an “apron” surrounding the landing pad would also include a perimeter deflector wall to direct any remaining exhaust-propelled particles up and away from any nearby spacecraft, people, or structures.

The subscale prototype of the pad was manufactured by a gantry-style 3D printer developed by the Austin-based company ICON. The company is already working with NASA to adapt its 3D printing technology for space-based construction on the moon and Mars.

3D printing the main layers of the subscale pad prototype took up just one day. The team also spent three additional days on tasks such as using the printer to fill various components with concrete and patching or smoothing certain parts of the pad. People also had to manually install fiber optic sensors to detect changes in strain and temperature.

But the most labor-intensive and hands-on part of the construction involved trimming and placing pre-cut blocks of water-soluble foam to provide temporary structural support for overhanging areas of the pad. Full-scale construction of such a pad on the moon or Mars would require a different and ideally more efficient solution for providing such removable supports.

“It became especially apparent after a few days of of cutting and wrapping and inserting foam that it's probably not the best use of an astronaut’s time,” says Andres Campbell, an integrated engineering student with an emphasis on aerospace engineering at Minnesota State University in Mankato and a principal investigator for the team. “This would also be something that would be robotically complex to do.”

In any case, a full-scale and operational Lunar PAD would not have to handle the dust mitigation work on its own. For example, Carson originally proposed an electrodynamic dust shielding technology that would passively push dust off the landing pad by taking advantage of the charged nature of lunar dust. Automated cleaning tools such what Campbell described as a “space Roomba” robot could also help keep the launch and landing zone dust free.

“The idea that you can combine the pad with not just electrodynamic dust shielding but any sort of passive dust mitigation system is still worth consideration,” Carson says. “Because in addition to that pad, you would still have dust that could be kicked up from other activities on the surface.”

The 3D-printed pad concept could eventually prove useful for future missions to Mars and other destinations. Such pad designs would have to account for some differences in atmosphere and gravity on rocket plumes and dust clouds, not to mention factors such as the moon’s electrostatically charged dust particles and Martian dust storms. Still, the team designed the pad to potentially work beyond lunar landing scenarios.

“Our goal was to build a reusable pad for all extraterrestrial environments,” Murai says.

Video Friday is your weekly selection of awesome robotics videos, collected by your Automaton bloggers. We’ll also be posting a weekly calendar of upcoming robotics events for the next few months; here's what we have so far (send us your events!):

HRI 2021 – March 8-11, 2021 – [Online Conference] RoboSoft 2021 – April 12-16, 2021 – [Online Conference] ICRA 2021 – May 30-5, 2021 – Xi'an, China

Let us know if you have suggestions for next week, and enjoy today's videos.

If you’ve ever swatted a mosquito away from your face, only to have it return again (and again and again), you know that insects can be remarkably acrobatic and resilient in flight. Those traits help them navigate the aerial world, with all of its wind gusts, obstacles, and general uncertainty. Such traits are also hard to build into flying robots, but MIT Assistant Professor Kevin Yufeng Chen has built a system that approaches insects’ agility.

Chen’s actuators can flap nearly 500 times per second, giving the drone insect-like resilience. “You can hit it when it’s flying, and it can recover,” says Chen. “It can also do aggressive maneuvers like somersaults in the air.” And it weighs in at just 0.6 grams, approximately the mass of a large bumble bee. The drone looks a bit like a tiny cassette tape with wings, though Chen is working on a new prototype shaped like a dragonfly.

[ MIT ]

National Robotics Week is April 3-11, 2021!

[ NRW ]

This is in a motion capture environment, but still, super impressive!

[ Paper ]

Thanks Fan!

Why wait for Boston Dynamics to add an arm to your Spot if you can just do it yourself?

[ ETHZ ]

This video shows the deep-sea free swimming of soft robot in the South China Sea. The soft robot was grasped by a robotic arm on ‘HAIMA’ ROV and reached the bottom of the South China Sea (depth of 3,224 m). After the releasing, the soft robot was actuated with an on-board AC voltage of 8 kV at 1 Hz and demonstrated free swimming locomotion with its flapping fins.

Um, did they bring it back?

[ Nature ]

Quadruped Yuki Mini is 12 DOF robot equipped with a Raspberry Pi that runs ROS. Also, BUNNIES!

[ Lingkang Zhang ]

Thanks Lingkang!

Deployment of drone swarms usually relies on inter-agent communication or visual markers that are mounted on the vehicles to simplify their mutual detection. The vswarm package enables decentralized vision-based control of drone swarms without relying on inter-agent communication or visual fiducial markers. The results show that the drones can safely navigate in an outdoor environment despite substantial background clutter and difficult lighting conditions.

[ Vswarm ]

A conventional adopted method for operating a waiter robot is based on the static position control, where pre-defined goal positions are marked on a map. However, this solution is not optimal in a dynamic setting, such as in a coffee shop or an outdoor catering event, because the customers often change their positions. We explore an alternative human-robot interface design where a human operator communicates the identity of the customer to the robot instead. Inspired by how [a] human communicates, we propose a framework for communicating a visual goal to the robot, through interactive two-way communications.

[ Paper ]

Thanks Poramate!

In this video, LOLA reacts to undetected ground height changes, including a drop and leg-in-hole experiment. Further tests show the robustness to vertical disturbances using a seesaw. The robot is technically blind, not using any camera-based or prior information on the terrain.

[ TUM ]

RaiSim is a cross-platform multi-body physics engine for robotics and AI. It fully supports Linux, Mac OS, and Windows.

[ RaiSim ]

Thanks Fan!

The next generation of LoCoBot is here. The LoCoBot is an ROS research rover for mapping, navigation and manipulation (optional) that enables researchers, educators and students alike to focus on high level code development instead of hardware and building out lower level code. Development on the LoCoBot is simplified with open source software, full ROS-mapping and navigation packages and modular opensource Python API that allows users to move the platform as well as (optional) manipulator in as few as 10 lines of code.

[ Trossen ]

MIT Media Lab Research Specialist Dr. Kate Darling looks at how robots are portrayed in popular film and TV shows.

Kate's book, The New Breed: What Our History with Animals Reveals about Our Future with Robots can be pre-ordered now and comes out next month.

[ Kate Darling ]

The current autonomous mobility systems for planetary exploration are wheeled rovers, limited to flat, gently-sloping terrains and agglomerate regolith. These vehicles cannot tolerate instability and operate within a low-risk envelope (i.e., low-incline driving to avoid toppling). Here, we present ‘Mars Dogs’ (MD), four-legged robotic dogs, the next evolution of extreme planetary exploration.

[ Team CoSTAR ]

In 2020, first-year PhD students at the MIT Media Lab were tasked with a special project—to reimagine the Lab and write sci-fi stories about the MIT Media Lab in the year 2050. “But, we are researchers. We don't only write fiction, we also do science! So, we did what scientists do! We used a secret time machine under the MIT dome to go to the year 2050 and see what’s going on there! Luckily, the Media Lab still exists and we met someone…really cool!” Enjoy this interview of Cyber Joe, AI Mentor for MIT Media Lab Students of 2050.

[ MIT ]

In this talk, we will give an overview of the diverse research we do at CSIRO’s Robotics and Autonomous Systems Group and delve into some specific technologies we have developed including SLAM and Legged robotics. We will also give insights into CSIRO’s participation in the current DARPA Subterranean Challenge where we are deploying a fleet of heterogeneous robots into GPS-denied unknown underground environments.

[ GRASP Seminar ]

Marco Hutter (ETH) and Hae-Won Park (KAIST) talk about “Robotics Inspired by Nature.”

[ Swiss-Korean Science Club ]

Thanks Fan!

In this keynote, Guy Hoffman Assistant Professor and the Mills Family Faculty Fellow in the Sibley School of Mechanical and Aerospace Engineering at Cornell University, discusses “The Social Uncanny of Robotic Companions.”

[ Designerly HRI ]

Video Friday is your weekly selection of awesome robotics videos, collected by your Automaton bloggers. We’ll also be posting a weekly calendar of upcoming robotics events for the next few months; here's what we have so far (send us your events!):

HRI 2021 – March 8-11, 2021 – [Online Conference] RoboSoft 2021 – April 12-16, 2021 – [Online Conference] ICRA 2021 – May 30-5, 2021 – Xi'an, China

Let us know if you have suggestions for next week, and enjoy today's videos.

If you’ve ever swatted a mosquito away from your face, only to have it return again (and again and again), you know that insects can be remarkably acrobatic and resilient in flight. Those traits help them navigate the aerial world, with all of its wind gusts, obstacles, and general uncertainty. Such traits are also hard to build into flying robots, but MIT Assistant Professor Kevin Yufeng Chen has built a system that approaches insects’ agility.

Chen’s actuators can flap nearly 500 times per second, giving the drone insect-like resilience. “You can hit it when it’s flying, and it can recover,” says Chen. “It can also do aggressive maneuvers like somersaults in the air.” And it weighs in at just 0.6 grams, approximately the mass of a large bumble bee. The drone looks a bit like a tiny cassette tape with wings, though Chen is working on a new prototype shaped like a dragonfly.

[ MIT ]

National Robotics Week is April 3-11, 2021!

[ NRW ]

This is in a motion capture environment, but still, super impressive!

[ Paper ]

Thanks Fan!

Why wait for Boston Dynamics to add an arm to your Spot if you can just do it yourself?

[ ETHZ ]

This video shows the deep-sea free swimming of soft robot in the South China Sea. The soft robot was grasped by a robotic arm on ‘HAIMA’ ROV and reached the bottom of the South China Sea (depth of 3,224 m). After the releasing, the soft robot was actuated with an on-board AC voltage of 8 kV at 1 Hz and demonstrated free swimming locomotion with its flapping fins.

Um, did they bring it back?

[ Nature ]

Quadruped Yuki Mini is 12 DOF robot equipped with a Raspberry Pi that runs ROS. Also, BUNNIES!

[ Lingkang Zhang ]

Thanks Lingkang!

Deployment of drone swarms usually relies on inter-agent communication or visual markers that are mounted on the vehicles to simplify their mutual detection. The vswarm package enables decentralized vision-based control of drone swarms without relying on inter-agent communication or visual fiducial markers. The results show that the drones can safely navigate in an outdoor environment despite substantial background clutter and difficult lighting conditions.

[ Vswarm ]

A conventional adopted method for operating a waiter robot is based on the static position control, where pre-defined goal positions are marked on a map. However, this solution is not optimal in a dynamic setting, such as in a coffee shop or an outdoor catering event, because the customers often change their positions. We explore an alternative human-robot interface design where a human operator communicates the identity of the customer to the robot instead. Inspired by how [a] human communicates, we propose a framework for communicating a visual goal to the robot, through interactive two-way communications.

[ Paper ]

Thanks Poramate!

In this video, LOLA reacts to undetected ground height changes, including a drop and leg-in-hole experiment. Further tests show the robustness to vertical disturbances using a seesaw. The robot is technically blind, not using any camera-based or prior information on the terrain.

[ TUM ]

RaiSim is a cross-platform multi-body physics engine for robotics and AI. It fully supports Linux, Mac OS, and Windows.

[ RaiSim ]

Thanks Fan!

The next generation of LoCoBot is here. The LoCoBot is an ROS research rover for mapping, navigation and manipulation (optional) that enables researchers, educators and students alike to focus on high level code development instead of hardware and building out lower level code. Development on the LoCoBot is simplified with open source software, full ROS-mapping and navigation packages and modular opensource Python API that allows users to move the platform as well as (optional) manipulator in as few as 10 lines of code.

[ Trossen ]

MIT Media Lab Research Specialist Dr. Kate Darling looks at how robots are portrayed in popular film and TV shows.

Kate's book, The New Breed: What Our History with Animals Reveals about Our Future with Robots can be pre-ordered now and comes out next month.

[ Kate Darling ]

The current autonomous mobility systems for planetary exploration are wheeled rovers, limited to flat, gently-sloping terrains and agglomerate regolith. These vehicles cannot tolerate instability and operate within a low-risk envelope (i.e., low-incline driving to avoid toppling). Here, we present ‘Mars Dogs’ (MD), four-legged robotic dogs, the next evolution of extreme planetary exploration.

[ Team CoSTAR ]

In 2020, first-year PhD students at the MIT Media Lab were tasked with a special project—to reimagine the Lab and write sci-fi stories about the MIT Media Lab in the year 2050. “But, we are researchers. We don't only write fiction, we also do science! So, we did what scientists do! We used a secret time machine under the MIT dome to go to the year 2050 and see what’s going on there! Luckily, the Media Lab still exists and we met someone…really cool!” Enjoy this interview of Cyber Joe, AI Mentor for MIT Media Lab Students of 2050.

[ MIT ]

In this talk, we will give an overview of the diverse research we do at CSIRO’s Robotics and Autonomous Systems Group and delve into some specific technologies we have developed including SLAM and Legged robotics. We will also give insights into CSIRO’s participation in the current DARPA Subterranean Challenge where we are deploying a fleet of heterogeneous robots into GPS-denied unknown underground environments.

[ GRASP Seminar ]

Marco Hutter (ETH) and Hae-Won Park (KAIST) talk about “Robotics Inspired by Nature.”

[ Swiss-Korean Science Club ]

Thanks Fan!

In this keynote, Guy Hoffman Assistant Professor and the Mills Family Faculty Fellow in the Sibley School of Mechanical and Aerospace Engineering at Cornell University, discusses “The Social Uncanny of Robotic Companions.”

[ Designerly HRI ]

Swabbing tests have proved to be an effective method of diagnosis for a wide range of diseases. Potential occupational health hazards and reliance on healthcare workers during traditional swabbing procedures can be mitigated by self-administered swabs. Hence, we report possible methods to apply closed kinematic chain theory to develop a self-administered viral swab to collect respiratory specimens. The proposed sensorized swab models utilizing hollow polypropylene tubes possess mechanical compliance, simple construction, and inexpensive components. In detail, the adaptation of the slider-crank mechanism combined with concepts of a deployable telescopic tubular mechanical system is explored through four different oral swab designs. A closed kinematic chain on suitable material to create a developable surface allows the translation of simple two-dimensional motion into more complex multi-dimensional motion. These foldable telescopic straws with multiple kirigami cuts minimize components involved in the system as the characteristics are built directly into the material. Further, it offers a possibility to include soft stretchable sensors for realtime performance monitoring. A variety of features were constructed and tested using the concepts above, including 1) tongue depressor and cough/gag reflex deflector; 2) changing the position and orientation of the oral swab when sample collection is in the process; 3) protective cover for the swabbing bud; 4) a combination of the features mentioned above.

The dawn of the robot revolution is already here, and it is not the dystopian nightmare we imagined. Instead, it comes in the form of social robots: Autonomous robots in homes and schools, offices and public spaces, able to interact with humans and other robots in a socially acceptable, human-perceptible way to resolve tasks related to core human needs. 

To design social robots that “understand” humans, robotics scientists are delving into the psychology of human communication. Researchers from Cornell University posit that embedding the sense of touch in social robots could teach them to detect physical interactions and gestures. They describe a way of doing so by relying not on touch but on vision.

A USB camera inside the robot captures shadows of hand gestures on the robot’s surface and classifies them with machine-learning software. They call this method ShadowSense, which they define as a modality between vision and touch, bringing “the high resolution and low cost of vision-sensing to the close-up sensory experience of touch.” 

Touch-sensing in social or interactive robots is usually achieved with force sensors or capacitive sensors, says study co-author Guy Hoffman of the Sibley School of Mechanical and Aerospace Engineering at Cornell University. The drawback to his group’s approach has been that, even to achieve coarse spatial resolution, many sensors are needed in a small area.

However, working with non-rigid, inflatable robots, Hoffman and his co-researchers installed a consumer-grade USB camera to which they attached a fisheye lens for a wider field of vision. 

“Given that the robot is already hollow, and has a soft and translucent skin, we could do touch interaction by looking at the shadows created by people touching the robot,” says Hoffman. They used deep neural networks to interpret the shadows. “And we were able to do it with very high accuracy,” he says. The robot was able to interpret six different gestures, including one- or two-handed touch, pointing, hugging and punching, with an accuracy of 87.5 to 96 percent, depending on the lighting.

This is not the first time that computer vision has been used for tactile sensing, though the scale and application of ShadowSense is unique. “Photography has been used for touch mainly in robotic grasping,” says Hoffman. By contrast, Hoffman and collaborators wanted to develop a sense that could be “felt” across the whole of the device. 

The potential applications for ShadowSense include mobile robot guidance using touch, and interactive screens on soft robots. A third concerns privacy, especially in home-based social robots. “We have another paper currently under review that looks specifically at the ability to detect gestures that are further away [from the robot’s skin],” says Hoffman. This way, users would be able to cover their robot’s camera with a translucent material and still allow it to interpret actions and gestures from shadows. Thus, even though it’s prevented from capturing a high-resolution image of the user or their surrounding environment, using the right kind of training datasets, the robot can continue to monitor some kinds of non-tactile activities. 

In its current iteration, Hoffman says, ShadowSense doesn’t do well in low-light conditions. Environmental noise, or shadows from surrounding objects, also interfere with image classification. Relying on one camera also means a single point of failure. “I think if this were to become a commercial product, we would probably [have to] work a little bit better on image detection,” says Hoffman.

As it was, the researchers used transfer learning—reusing a pre-trained deep-learning model in a new problem—for image analysis. “One of the problems with multi-layered neural networks is that you need a lot of training data to make accurate predictions,” says Hoffman. “Obviously, we don’t have millions of examples of people touching a hollow, inflatable robot. But we can use pre-trained networks trained on general images, which we have billions of, and we only retrain the last layers of the network using our own dataset.”

Demand for battery-making metals is projected to soar as more of the world’s cars, buses, and ships run on electricity. The coming mining boom is raising concerns of environmental damage and labor abuses—and it’s driving a search for more sustainable ways of making batteries and cutting-edge electronics.

Artificial intelligence could help improve the way battery metals are mined, or replace them altogether. KoBold Metals is developing an AI agent to find the most desirable ore deposits in the least problematic locations. IBM Research, meanwhile, is harnessing AI techniques to identify alternative materials that already exist and also develop new chemistries.

KoBold, a mining exploration startup, says its technology could reduce the need for costly and invasive exploration missions, which often involve scouring the Earth many times over to find rare, high-quality reserves. 

“All the stuff poking out of the ground has already been found,” said Kurt House, co-founder and CEO of the San Francisco Bay area company. “At the same time, we’ve realized we need to massively change the energy system, which requires all these new minerals.”

KoBold is partnering with Stanford University’s Center for Earth Resource Forecasting to develop an AI agent that can make decisions about how and where explorers should focus their work. The startup is mainly looking for copper, cobalt, nickel, and lithium—metals key to making electric vehicle batteries as well as solar panels, smartphones, and many other devices.

Jef Caers, a professor of geological sciences at Stanford, said the idea is to accelerate the decision-making process and enable explorers to evaluate multiple sites at once. He likened the AI agent to a self-driving car: The vehicle not only gathers and processes data about its surrounding environment but also acts upon that information to, say, navigate traffic or change speeds.

“We can’t wait another 10 or 20 years to make more discoveries,” Caers said. “We need to make them in the next few years if we want to have an impact on [climate change] and go away from fossil fuels.”

Light-duty cars alone will have a significant need for metals. The global fleet of battery-powered cars could expand from 7.5 million in 2019 to potentially 2 billion cars by 2050 as countries work to reduce greenhouse gas emissions, according to a December paper in the journal Nature. Powering those vehicles would require 12 terawatt-hours of annual battery capacity—roughly 10 times the current U.S. electricity generating capacity—and mean a “drastic expansion” of metal supply chains, the paper’s authors said. 

Photo: KoBold Metals Patrick Redmond of KoBold Metals evaluates a prospective cobalt-copper mining site in Zambia.

Almost all lithium-ion batteries use cobalt, a material that is primarily supplied by the Democratic Republic of Congo, where young children and adults often work in dangerous conditions. Copper, another important EV material, requires huge volumes of water to mine, yet much of the global supply comes from water-scarce regions near Chile’s Atacama desert. 

For mining companies, the challenge is to expand operations without wreaking havoc in the name of sustainable transportation. 

KoBold’s AI-driven approach begins with its data platform, which stores all available forms of information about a particular area, including soil samples, satellite-based hyperspectral imaging, and century-old handwritten drilling reports. The company then applies machine learning methods to make predictions about the location of compositional anomalies—that is, unusually high concentrations of ore bodies in the Earth’s subsurface.

Working with Stanford, KoBold is refining sequential decision-making algorithms to determine how explorers should next proceed to gather data. Perhaps they should fly a plane over a site or collect drilling samples; maybe companies should walk away from what is likely to be a dud. Such steps are currently risky and expensive, and companies move slowly to avoid wasting resources. 

The AI agent could make such decisions roughly 20 times faster than humans can, while also reducing the rate of false positives in mining exploration, Caers said. “This is completely new within the Earth sciences,” he added.

Image: KoBold Metals An AI visualization by KoBold Metals depicts a plot of predictions from the borehole electromagnetic model, with true values on the left and predictions on the right.

KoBold, which is backed by the Bill Gates-led Breakthrough Energy Ventures, is already exploring three sites in Australia, North America, and Sub-Saharan Africa. Field data collected this year will provide the first validations of the company’s predictions, House said.

As the startup searches for metals, IBM researchers are searching for solvents and other materials to reduce the use of battery ingredients such as cobalt and lithium.

Research teams are using AI techniques to identify and test solvents that offer higher safety and performance potential than current lithium-ion battery options. The project focuses on existing and commercially available materials that can be tested immediately. A related research effort, however, aims to create brand-new molecules entirely.

Using “generative models,” experts train AI to learn the molecular structure of known materials, as well as characteristics such as viscosity, melting point, or electronic conductivity.  

“For example, if we want a generative model to design new electrolyte materials for batteries— such as electrolyte solvents or appropriate monomers to form ion-conducting polymers—we should train the AI with known electrolyte material data,” Seiji Takeda and Young-hye Na of IBM Research said in an email. 

Once the AI training is completed, researchers can input a query such as “design a new molecular electrolyte material that meets the characteristics of X, Y, and Z,” they said. “And then the model designs a material candidate by referring to the structure-characteristics relation.”

IBM has already used this AI-boosted approach to create new molecules named called photoacid generators that could eventually help produce more environmentally friendly computing devices. Researchers also designed polymer membranes that apparently absorbs carbon dioxide better than membranes currently used in carbon capture technologies.

Designing a more sustainable battery “is our next challenge,” Takeda and Na said.

The development of biodegradable soft robotics requires an appropriate eco-friendly source of energy. The use of Microbial Fuel Cells (MFCs) is suggested as they can be designed completely from soft materials with little or no negative effects to the environment. Nonetheless, their responsiveness and functionality is not strictly defined as in other conventional technologies, i.e. lithium batteries. Consequently, the use of artificial intelligence methods in their control techniques is highly recommended. The use of neural networks, namely a nonlinear autoregressive network with exogenous inputs was employed to predict the electrical output of an MFC, given its previous outputs and feeding volumes. Thus, predicting MFC outputs as a time series, enables accurate determination of feeding intervals and quantities required for sustenance that can be incorporated in the behavioural repertoire of a soft robot.

In this paper, we present a generalized modeling tool for predicting the output force profile of vacuum-powered soft actuators using a simplified geometrical approach and the principle of virtual work. Previous work has derived analytical formulas to model the force-contraction profile of specific actuators. To enhance the versatility and the efficiency of the modelling process we propose a generalized numerical algorithm based purely on geometrical inputs, which can be tailored to the desired actuator, to estimate its force-contraction profile quickly and for any combination of varying geometrical parameters. We identify a class of linearly contracting vacuum actuators that consists of a polymeric skin guided by a rigid skeleton and apply our model to two such actuators-vacuum bellows and Fluid-driven Origami-inspired Artificial Muscles-to demonstrate the versatility of our model. We perform experiments to validate that our model can predict the force profile of the actuators using its geometric principles, modularly combined with design-specific external adjustment factors. Our framework can be used as a versatile design tool that allows users to perform parametric studies and rapidly and efficiently tune actuator dimensions to produce a force-contraction profile to meet their needs, and as a pre-screening tool to obviate the need for multiple rounds of time-intensive actuator fabrication and testing.

In my experience, there are three different types of consumer drone pilots. You’ve got people for whom drones are a tool for taking pictures and video, where flying the drone is more or less just a necessary component of that. You’ve also got people who want a drone that can be used to take pictures or video of themselves, where they don’t want to be bothered flying the drone at all. Then you have people for whom flying the drone itself is the appealing part; people who like flying fast and creatively because it’s challenging, exciting, and fun. And that typically means flying in First Person View, or FPV, where it feels like you’re a tiny little human sitting inside of a virtual cockpit in your drone.

For that last group of folks, the barrier to entry is high. Or rather, the barriers are high, because there are several. Not only is the equipment expensive, you often have to build your own system comprising the drone, FPV goggles, and accompanying transmitter and receiver. And on top of that, it takes a lot of skill to fly an FPV drone well; all of the inevitable crashes just add to the expense.

Today, DJI is announcing a new consumer first-person view drone system that includes everything you need to get started. You get an expertly designed and fully integrated high-speed FPV drone, a pair of FPV goggles with exceptional image quality and latency that’s some of the best we’ve ever seen, plus a physical controller to make it all work. Most importantly, though, there’s on-board obstacle avoidance plus piloting assistance that means even a complete novice can be zipping around with safety and confidence on day one.

Because the point of an FPV drone is to let you fly from a first-person viewpoint. The drone has a forward-facing camera that streams video to a pair of goggles in real time. This experience is a unique one, and there’s only so much that I can do to describe it, but it turns flying a drone into a much more personal, visceral, immersive thing. With an FPV drone, it feels much more like you are the drone. 

The Drone Photos: DJI DJI’s FPV drone is basically a battery with a camera and some motors attached.

DJI’s FPV drone itself is a bit of a chonker, as far as drones go. It’s optimized for going very fast while giving you a good first-person view, and no concessions are given to portability. It weighs 800g (of which 300 is the battery), and doesn’t fold up even a little bit, although the props are easy to remove.

Photo: Evan Ackerman/IEEE Spectrum Efficient design, but not small or portable.

Top speed is a terrifying 140 km/h, albeit in a mode  that you have to unlock (more on that later), and it’ll accelerate from a hover to 100 km/h in two seconds. Battery life maxes out at 20 minutes, but in practice you’ll get more like 10-15 minutes depending on how you fly. The camera on the front records in 4K at 60 FPS on an electronically stabilized tilt-only gimbal, and there’s a microSD card slot for local recording.

We’re delighted to report that the DJI FPV drone also includes some useful sensors that will make you significantly less likely to embed it in the nearest tree. These sensors include ground detection to keep the drone at a safe altitude, as well as forward-looking stereo-based obstacle detection that works well enough for the kinds of obstacles that cameras are able to see. 

The Goggles Photos: DJI You’ll look weird with these on, but they work very, very well.

What really makes this drone work are the FPV goggles along with the radio system that connects to the drone. The goggles have two tiny screens in them, right in front of your eyeballs. Each screen can display 1440 x 810p at up to 120 fps (which looks glorious), and it’ll do so while the drone is moving at 140 km/h hundreds of meters away. It’s extremely impressive, with quality that’s easily good enough to let you spot (and avoid) skinny little tree branches. But even more important than quality is latency— the amount of time it takes for the video to be captured by the drone, compressed, sent to the goggles, decompressed, and displayed. The longer this takes, the less you’re able to trust what you’re seeing, because you know that you’re looking at where the drone used to be rather than where it actually is. DJI’s FPV system has a latency that’s 28ms or better, which is near enough to real-time that it feels just like real-time, and you can fly with absolute confidence that the control inputs you’re giving and what you’re seeing through the goggles are matched up. 

Photo: Evan Ackerman/IEEE Spectrum The goggles are adjustable to fit most head sizes.

The goggles are also how you control all of the drone options, no phone necessary. You can attach your phone to the goggles with a USB-C cable for firmware updates, but otherwise, a little joystick and some buttons on the top of the goggles lead you to an intuitive interface for things like camera options, drone control options, and so on. A microSD card slot on the goggles lets you record the downlinked video, although it’s not going to be the same quality as what’s recorded on-board the drone. 

Don’t get your hopes up on comfort where the goggles are concerned. They’re fine, but that’s it. My favorite thing about them is that they don’t weigh much, because the battery that powers them has a cable so that you can keep it in your pocket rather than hanging it off the goggles themselves. Adjustable straps mean the goggles will fit most people, and there’s inter-eye distance adjustments for differently shaped faces. My partner, who statistically is smaller than 90% of adult women, found that the google inter-eye distance adjustment was almost, but not quite, adequate for her. Fortunately, the goggles not being super comfortable isn’t a huge deal because you won’t be wearing them for that long, unless you invest in more (quite expensive) battery packs.

The Controller Photos: DJI All the controls you want, none that you don’t. Except one.

The last piece of the kit is the controller, which is fairly standard as far as drone controllers go. The sticks unscrew and stow in the handles, and you can also open up the back panels to adjust the stick tension, which experienced FPV pilots will probably want to do.

Before we get to how the drone flies, a quick word on safe operation— according to the FAA, if you’re flying an FPV drone, you’ll need a spotter who keeps the drone in view at all times. And while the range of the DJI FPV drone is (DJI claims) up to 10km, here in the United States you’re not allowed to fly it higher than 400ft AGL, or farther away than your spotter can see, without an FAA exemption. Also, as with all drones, you’ll need to find places that are both safe and legal to fly. The drone will prevent you from taking off in restricted areas that DJI knows about, but it’s on you to keep the drone from flying over people, or otherwise being dangerous and/or annoying.

In Flight— Normal Mode Photos: Evan Ackerman/IEEE Spectrum It's not the most graceful looking drone, but the way it flies makes you not care.

DJI has helpfully equipped the FPV drone with three flight modes: Normal, Sport, and Manual. These three modes are the primary reason why this drone is not a terrible idea for almost everyone— Normal mode is a lot of fun, and both safe and accessible for FPV novices. Specifically, Normal mode brings the top speed of the drone down to a still rather quick 50 km/h, and will significantly slow the drone if the front sensors think you’re likely to hit something. As with DJI’s other drones, if you start getting into trouble you can simply let go of the control sticks, and the drone will bring itself to a halt and hover. This makes it very beginner friendly, and (as an FPV beginner myself) I didn’t find it at all stressful to fly. With most drones (and especially drones that cost as much as this one does) fear of crashing is a tangible thing always sitting at the back of your mind. That feeling is not gone with the DJI FPV drone, but it’s reduced so much that the experience can really just be fun.

To be clear, not crashing into stuff is not enough to make FPV flying an enjoyable experience. Arguably the best feature of DJI’s FPV drone is how much help it gives you behind the scenes to make flying effortless.

When flying a conventional drone, you’ve got four axes of control to work with, allowing the drone to pitch, roll, yaw, move vertically up and down, or do any combination of those things. Generally, a drone won’t couple these axes for you in an intelligent way. For example, if you want to go left with a drone, you can either roll left, which will move the drone left without looking left, or yaw left, which will cause the drone to look left without actually moving. To gracefully fly a drone around obstacles at speed, you need to fuse both of these inputs together, which is a skill that can take a long time to master. I have certainly not mastered it.

The drone does exactly what you think it should do, and it works beautifully.

For most people, especially beginners, it’s much more intuitive for the drone to behave more like an airplane when you want it to turn. That is, when you push the left stick (traditionally the yaw control), you want the drone to begin to roll while also yawing in the same direction and increasing throttle to execute a lovely, sweeping turn. And this is exactly what DJI FPV does—  thanks to a software option called coordinated turns that’s on by default, the drone does exactly what you think it should do, and it works beautifully. 

I could tell you how well this works for me, someone who has flown non-FPV drones for years, but my partner has flown a drone only one single time before, when she spent five minutes with the controller of my Parrot Anafi a few years ago and got it to go up and down and occasionally sideways a little bit. But within literally two minutes, she was doing graceful figure 8s with the DJI FPV drone. The combination of the FPV view and the built-in coordinated turns makes it just that intuitive.

In Flight— Sport Mode

Once you’re comfortable with Normal mode, Sport mode (which you can select at any time with a toggle switch on the controller) bumps up the speed of the drone from 50 km/h to 97 km/h. More importantly, the obstacle avoidance no longer slows the drone down for you, although it does give you escalating warnings when it thinks you’re going to hit something. As you get more comfortable with the drone, you’ll find that the obstacle avoidance tends to be on the paranoid side, which is as it should be. Once you’ve practiced a bit and you want to (say) fly between two trees that are close together, Sport mode will let you do that without slowing down.

Along with a higher top speed, Sport mode will also make the drone literally scream. When in flight it makes a loud screaming noise, especially when you ask a lot from the motors, like with rapid direction changes or gaining altitude at speed. This happens in Normal mode, but gets markedly louder in Sport mode. This doesn’t matter, really, except that if you’re flying anywhere near other people, they’re likely to find it obnoxious. 

I was surprised by how not-puking I was during high-speed FPV flight in Sport mode. I tend to suffer from motion sickness, but I had no trouble with the drone, as long as I kept my head still. Even a small head movement while the drone was in flight could lead to an immediate (although minor) wave of nausea, which passed as soon as I stopped moving. My head sometimes subconsciously moved along with the motion of the drone, to the point where after a few minutes of flying I’d realize that I’d ended up staring sideways at the sky like an idiot, so if you can just manage to keep your head mostly still and relaxed in a comfortable position, you’ll be fine. 

A word on speed— even though Sport mode has a max speed of only (only?) 97 km/h, coming from flying a Mavic Pro, it feels very, very fast. In tight turns, the video coming through the goggles sometimes looked like it was playing back at double speed. You could always ask for more speed, I suppose, and Manual mode gives it to you. But I could see myself being perfectly happy to fly in Sport mode for a long, long time, since it offers both speed and some additional safety.

The following video includes clips of Normal and Sport mode, to give you an idea of how smooth the drone moves and how fast it can go, along with a comparison between the standard 1080p recorded by the drone and what gets shown in the goggles. As you watch the video, remember that I’m not an FPV pilot. I’ve never flown an FPV drone before, and what you’re seeing is the result of less than an hour of total flight time.

In Flight— Manual Mode

There is one final mode that the DJI FPV drone comes with: Manual mode. Manual mode is for pilots who don’t need or want any hand-holding. You get full control over all axes as well as an unrestricted top speed of 140 km/h. Manual mode must be deliberately enabled in menus in the goggles (it’s not an available option by default), and DJI suggests spending some time in their included simulator before doing so. I want to stress that Manual mode doesn’t just disable the coordinated turns function, making control of the drone more like a traditional camera drone— if that’s something you want, there’s an option to do that in Normal and Sport mode. Manual mode is designed for people with drone racing experience, and enabling it turns the FPV drone into a much different thing, as I found out.

My test of Manual mode ended approximately 15 seconds after it began due to high speed contact with the ground. I wouldn’t call what happened a “crash,” in the sense that I didn’t fly the drone into an obstacle— there was a misunderstanding or a lack of information or an accidental input or some combination of those things that led to the drone shutting all of its motors of at about 150 feet up and then falling to the ground. 

Photo: Evan Ackerman/IEEE Spectrum The crash broke the drone’s two rear arms, but the expensive parts all seem perfectly fine.

I’d planned to just fly the drone in Manual mode a little bit, with plenty of altitude and over a big open space, primarily to get a sense of how much faster it is in Manual mode than in normal mode. After taking off in Normal mode and giving the drone a lot of room, I switched over to Manual mode. Immediately, the drone began to move in a much less predictable way, and after about five seconds of some tentative control inputs to see if I could get a handle on it, I felt uncomfortable enough to want the drone to stop itself.

The first thing I did was stop giving any control inputs. In Normal and Sport mode, the drone will respond to no input (centered sticks) by bringing itself to a hover. This doesn’t happen in Manual mode, and the drone kept moving. The second thing I did was push what I thought was the emergency stop button, which would switch the drone back to Normal mode and engage a thoughtfully included emergency stop mode to bring it to a stable hover as quickly as possible. I hadn’t yet needed an emergency stop in Normal or Sport mode, since just taking my fingers off of the sticks worked just fine before. What I learned post-crash was that in Manual mode, the button that says “Stop” on it (which is one of the easiest to press buttons on the controller since your right index finger naturally rests there) gains a new emergency shut-off functionality that causes the drone to disable all of its motors, whereupon it will then follow a ballistic trajectory until the inevitable happens no matter how much additional button pushing or frantic pleading you do. 

I certainly take some responsibility for this. When the DJI FPV drone showed up, it included a quick start guide and a more detailed reviewer’s guide, but neither of those documents had detailed information about what all the buttons on the controller and headset did. This sometimes happens with review units— they can be missing manuals and stuff if they get sent to us before the consumer packaging is complete. Anyway, having previous experience with DJI’s drones, I just assumed that which buttons did what would just be obvious, which was 100% my bad, and it’s what led to the crash.

Also, I recognize why it’s important to have an emergency shut-off, and as far as I know, most (if not all) of DJI’s other consumer drones include some way of remotely disabling the motors. It should only be possible to do this deliberately, though, which is why their other drones require you to use a combination of inputs that you’re very unlikely to do by accident. Having what is basically a self-destruct button on the controller where you naturally rest a finger just seems like a bad idea— I pushed it on purpose thinking it did something different, but there are all kinds of reasons why a pilot might push it accidentally. And if you do, that’s it, your drone is going down. 

DJI, to their credit, was very understanding about the whole thing, but more importantly, they pointed out that accidental damage like this would be covered under DJI Care Refresh, which will completely replace the drone if necessary. This should give new pilots some piece of mind, if you’re willing to pay for the premium. Even if you’re not, the drone is designed to be at least somewhat end-user repairable.

Fundamentally, I’m glad Manual mode is there. DJI made the right choice by including it so that your skill won’t outgrow the capabilities of the drone. I just wish that the transition to Manual mode was more gradual, like if there was a Sport Plus mode that unlocked top speed while maintaining other flight assistance features. Even without that, FPV beginners really shouldn’t feel like Manual mode needs to be a goal— Normal mode is fun, and Sport mode is even more fun, with the added piece of mind that you’ve got options if things start to get out of your control. And if things do get out of your control in Manual mode, for heaven’s sake, push the right button.

I mean, the left button.

Is This The Right Drone for You? Photo: Evan Ackerman/IEEE Spectrum My partner, who is totally not interested in drones, actually had fun flying this one.

DJI’s FPV drone kit costs $1,299, which includes the drone, goggles, one battery, and all necessary chargers and cabling. Two more batteries and a charging hub, which you’ll almost certainly want, adds $299. This is a lot of money, even for a drone, so the thing to ask yourself is whether an FPV drone is really what you’re looking for. Yes, it’ll take good quality pictures and video, but if that’s what you’re after, DJI has lots of other drones that are cheaper and more portable and have some smarts to make them better camera platforms. And of course there’s the Skydio 2, which has some crazy obstacle avoidance and autonomy if you don’t want to have to worry about flying at all. I have a fantasy that one day, all of this will be combined into one single drone, but we’re not there yet. 

If you’re sure you want to get into FPV flying, DJI’s kit seems like a great option, with the recognition that this is an expensive, equipment-intensive sport. There are definitely ways of doing it for cheaper, but you’ll need to more or less build up the system yourself, and it seems unlikely that you’d end up with the same kind of reliable performance and software features that DJI’s system comes with. The big advantage of DJI’s FPV kit is that you can immediately get started with a system that works brilliantly out of the box, in a way that’s highly integrated, highly functional, high performing while being fantastic for beginners and leaving plenty of room to grow.

In my experience, there are three different types of consumer drone pilots. You’ve got people for whom drones are a tool for taking pictures and video, where flying the drone is more or less just a necessary component of that. You’ve also got people who want a drone that can be used to take pictures or video of themselves, where they don’t want to be bothered flying the drone at all. Then you have people for whom flying the drone itself is the appealing part— people who like flying fast and creatively because it’s challenging and exciting and fun. And that typically means flying in First Person View, or FPV, where it feels like you’re a tiny little human sitting inside of a virtual cockpit in your drone.

For that last group of folks, the barrier to entry is high. Or rather, the barriers are high, because there are several. Not only is the equipment expensive, you often have to build your own system of drone, FPV goggles, and transmitter and receiver. And on top of that, it takes a lot of skill to fly an FPV drone well, and all of the inevitable crashes just add to the expense.

Today, DJI is announcing a new consumer first-person view drone system that includes everything you need to get started. You get an expertly designed and fully integrated high-speed FPV drone, a pair of FPV goggles with exceptional image quality and latency that’s some of the best we’ve ever seen, plus a physical controller to make it all work. Most importantly, though, there’s on-board obstacle avoidance plus piloting assistance that means even a complete novice can be zipping around with safety and confidence on day one.

An FPV drone is a drone that you fly from a first-person viewpoint. The drone has a forward-facing camera that streams video to a pair of goggles in real time. This experience is a unique one, and there’s only so much that I can do to describe it, but it turns flying a drone into a much more personal, visceral, immersive thing. With an FPV drone, it feels much more like you are the drone. 

The Drone Photos: DJI DJI’s FPV drone is basically a battery with a camera and some motors attached.

DJI’s FPV drone itself is a bit of a chonker, as far as drones go. It’s optimized for going very fast while giving you a good first-person view, and no concessions are given to portability. It weighs 800g (of which 300 is the battery), and doesn’t fold up even a little bit, although the props are easy to remove.

Photo: Evan Ackerman/IEEE Spectrum Efficient design, but not small or portable.

Top speed is a terrifying 140 km/h, albeit in a mode  that you have to unlock (more on that later), and it’ll accelerate from a hover to 100 km/h in two seconds. Battery life maxes out at 20 minutes, but in practice you’ll get more like 10-15 minutes depending on how you fly. The camera on the front records in 4K at 60 FPS on an electronically stabilized tilt-only gimbal, and there’s a microSD card slot for local recording.

We’re delighted to report that the DJI FPV drone also includes some useful sensors that will make you significantly less likely to embed it in the nearest tree. These sensors include ground detection to keep the drone at a safe altitude, as well as forward-looking stereo-based obstacle detection that works well enough for the kinds of obstacles that cameras are able to see. 

The Goggles Photos: DJI You’ll look weird with these on, but they work very, very well.

What really makes this drone work are the FPV goggles along with the radio system that connects to the drone. The goggles have two tiny screens in them, right in front of your eyeballs. Each screen can display 1440 x 810p at up to 120 fps (which looks glorious), and it’ll do so while the drone is moving at 140 km/h hundreds of meters away. It’s extremely impressive, with quality that’s easily good enough to let you spot (and avoid) skinny little tree branches. But even more important than quality is latency— the amount of time it takes for the video to be captured by the drone, compressed, sent to the goggles, decompressed, and displayed. The longer this takes, the less you’re able to trust what you’re seeing, because you know that you’re looking at where the drone used to be rather than where it actually is. DJI’s FPV system has a latency that’s 28ms or better, which is near enough to real-time that it feels just like real-time, and you can fly with absolute confidence that the control inputs you’re giving and what you’re seeing through the goggles are matched up. 

Photo: Evan Ackerman/IEEE Spectrum The goggles are adjustable to fit most head sizes.

The goggles are also how you control all of the drone options, no phone necessary. You can attach your phone to the goggles with a USB-C cable for firmware updates, but otherwise, a little joystick and some buttons on the top of the goggles lead you to an intuitive interface for things like camera options, drone control options, and so on. A microSD card slot on the goggles lets you record the downlinked video, although it’s not going to be the same quality as what’s recorded on-board the drone. 

Don’t get your hopes up on comfort where the goggles are concerned. They’re fine, but that’s it. My favorite thing about them is that they don’t weigh much, because the battery that powers them has a cable so that you can keep it in your pocket rather than hanging it off the goggles themselves. Adjustable straps mean the goggles will fit most people, and there’s inter-eye distance adjustments for differently shaped faces. My partner, who statistically is smaller than 90% of adult women, found that the google inter-eye distance adjustment was almost, but not quite, adequate for her. Fortunately, the goggles not being super comfortable isn’t a huge deal because you won’t be wearing them for that long, unless you invest in more (quite expensive) battery packs.

The Controller Photos: DJI All the controls you want, none that you don’t. Except one.

The last piece of the kit is the controller, which is fairly standard as far as drone controllers go. The sticks unscrew and stow in the handles, and you can also open up the back panels to adjust the stick tension, which experienced FPV pilots will probably want to do.

Before we get to how the drone flies, a quick word on safe operation— according to the FAA, if you’re flying an FPV drone, you’ll need a spotter who keeps the drone in view at all times. And while the range of the DJI FPV drone is (DJI claims) up to 10km, here in the United States you’re not allowed to fly it higher than 400ft AGL, or farther away than your spotter can see, without an FAA exemption. Also, as with all drones, you’ll need to find places that are both safe and legal to fly. The drone will prevent you from taking off in restricted areas that DJI knows about, but it’s on you to keep the drone from flying over people, or otherwise being dangerous and/or annoying.

In Flight— Normal Mode Photos: Evan Ackerman/IEEE Spectrum It's not the most graceful looking drone, but the way it flies makes you not care.

DJI has helpfully equipped the FPV drone with three flight modes: Normal, Sport, and Manual. These three modes are the primary reason why this drone is not a terrible idea for almost everyone— Normal mode is a lot of fun, and both safe and accessible for FPV novices. Specifically, Normal mode brings the top speed of the drone down to a still rather quick 50 km/h, and will significantly slow the drone if the front sensors think you’re likely to hit something. As with DJI’s other drones, if you start getting into trouble you can simply let go of the control sticks, and the drone will bring itself to a halt and hover. This makes it very beginner friendly, and (as an FPV beginner myself) I didn’t find it at all stressful to fly. With most drones (and especially drones that cost as much as this one does) fear of crashing is a tangible thing always sitting at the back of your mind. That feeling is not gone with the DJI FPV drone, but it’s reduced so much that the experience can really just be fun.

To be clear, not crashing into stuff is not enough to make FPV flying an enjoyable experience. Arguably the best feature of DJI’s FPV drone is how much help it gives you behind the scenes to make flying effortless.

When flying a conventional drone, you’ve got four axes of control to work with, allowing the drone to pitch, roll, yaw, move vertically up and down, or do any combination of those things. Generally, a drone won’t couple these axes for you in an intelligent way. For example, if you want to go left with a drone, you can either roll left, which will move the drone left without looking left, or yaw left, which will cause the drone to look left without actually moving. To gracefully fly a drone around obstacles at speed, you need to fuse both of these inputs together, which is a skill that can take a long time to master. I have certainly not mastered it.

The drone does exactly what you think it should do, and it works beautifully.

For most people, especially beginners, it’s much more intuitive for the drone to behave more like an airplane when you want it to turn. That is, when you push the left stick (traditionally the yaw control), you want the drone to begin to roll while also yawing in the same direction and increasing throttle to execute a lovely, sweeping turn. And this is exactly what DJI FPV does—  thanks to a software option called coordinated turns that’s on by default, the drone does exactly what you think it should do, and it works beautifully. 

I could tell you how well this works for me, someone who has flown non-FPV drones for years, but my partner has flown a drone only one single time before, when she spent five minutes with the controller of my Parrot Anafi a few years ago and got it to go up and down and occasionally sideways a little bit. But within literally two minutes, she was doing graceful figure 8s with the DJI FPV drone. The combination of the FPV view and the built-in coordinated turns makes it just that intuitive.

In Flight— Sport Mode

Once you’re comfortable with Normal mode, Sport mode (which you can select at any time with a toggle switch on the controller) bumps up the speed of the drone from 50 km/h to 97 km/h. More importantly, the obstacle avoidance no longer slows the drone down for you, although it does give you escalating warnings when it thinks you’re going to hit something. As you get more comfortable with the drone, you’ll find that the obstacle avoidance tends to be on the paranoid side, which is as it should be. Once you’ve practiced a bit and you want to (say) fly between two trees that are close together, Sport mode will let you do that without slowing down.

Along with a higher top speed, Sport mode will also make the drone literally scream. When in flight it makes a loud screaming noise, especially when you ask a lot from the motors, like with rapid direction changes or gaining altitude at speed. This happens in Normal mode, but gets markedly louder in Sport mode. This doesn’t matter, really, except that if you’re flying anywhere near other people, they’re likely to find it obnoxious. 

I was surprised by how not-puking I was during high-speed FPV flight in Sport mode. I tend to suffer from motion sickness, but I had no trouble with the drone, as long as I kept my head still. Even a small head movement while the drone was in flight could lead to an immediate (although minor) wave of nausea, which passed as soon as I stopped moving. My head sometimes subconsciously moved along with the motion of the drone, to the point where after a few minutes of flying I’d realize that I’d ended up staring sideways at the sky like an idiot, so if you can just manage to keep your head mostly still and relaxed in a comfortable position, you’ll be fine. 

A word on speed— even though Sport mode has a max speed of only (only?) 97 km/h, coming from flying a Mavic Pro, it feels very, very fast. In tight turns, the video coming through the goggles sometimes looked like it was playing back at double speed. You could always ask for more speed, I suppose, and Manual mode gives it to you. But I could see myself being perfectly happy to fly in Sport mode for a long, long time, since it offers both speed and some additional safety.

The following video includes clips of Normal and Sport mode, to give you an idea of how smooth the drone moves and how fast it can go, along with a comparison between the standard 1080p recorded by the drone and what gets shown in the goggles. As you watch the video, remember that I’m not an FPV pilot. I’ve never flown an FPV drone before, and what you’re seeing is the result of less than an hour of total flight time.

In Flight— Manual Mode

There is one final mode that the DJI FPV drone comes with: Manual mode. Manual mode is for pilots who don’t need or want any hand-holding. You get full control over all axes as well as an unrestricted top speed of 140 km/h. Manual mode must be deliberately enabled in menus in the goggles (it’s not an available option by default), and DJI suggests spending some time in their included simulator before doing so. I want to stress that Manual mode doesn’t just disable the coordinated turns function, making control of the drone more like a traditional camera drone— if that’s something you want, there’s an option to do that in Normal and Sport mode. Manual mode is designed for people with drone racing experience, and enabling it turns the FPV drone into a much different thing, as I found out.

My test of Manual mode ended approximately 15 seconds after it began due to high speed contact with the ground. I wouldn’t call what happened a “crash,” in the sense that I didn’t fly the drone into an obstacle— there was a misunderstanding or a lack of information or an accidental input or some combination of those things that led to the drone shutting all of its motors of at about 150 feet up and then falling to the ground. 

Photo: Evan Ackerman/IEEE Spectrum The crash broke the drone’s two rear arms, but the expensive parts all seem perfectly fine.

I’d planned to just fly the drone in Manual mode a little bit, with plenty of altitude and over a big open space, primarily to get a sense of how much faster it is in Manual mode than in normal mode. After taking off in Normal mode and giving the drone a lot of room, I switched over to Manual mode. Immediately, the drone began to move in a much less predictable way, and after about five seconds of some tentative control inputs to see if I could get a handle on it, I felt uncomfortable enough to want the drone to stop itself.

The first thing I did was stop giving any control inputs. In Normal and Sport mode, the drone will respond to no input (centered sticks) by bringing itself to a hover. This doesn’t happen in Manual mode, and the drone kept moving. The second thing I did was push what I thought was the emergency stop button, which would switch the drone back to Normal mode and engage a thoughtfully included emergency stop mode to bring it to a stable hover as quickly as possible. I hadn’t yet needed an emergency stop in Normal or Sport mode, since just taking my fingers off of the sticks worked just fine before. What I learned post-crash was that in Manual mode, the button that says “Stop” on it (which is one of the easiest to press buttons on the controller since your right index finger naturally rests there) gains a new emergency shut-off functionality that causes the drone to disable all of its motors, whereupon it will then follow a ballistic trajectory until the inevitable happens no matter how much additional button pushing or frantic pleading you do. 

I certainly take some responsibility for this. When the DJI FPV drone showed up, it included a quick start guide and a more detailed reviewer’s guide, but neither of those documents had detailed information about what all the buttons on the controller and headset did. This sometimes happens with review units— they can be missing manuals and stuff if they get sent to us before the consumer packaging is complete. Anyway, having previous experience with DJI’s drones, I just assumed that which buttons did what would just be obvious, which was 100% my bad, and it’s what led to the crash.

Also, I recognize why it’s important to have an emergency shut-off, and as far as I know, most (if not all) of DJI’s other consumer drones include some way of remotely disabling the motors. It should only be possible to do this deliberately, though, which is why their other drones require you to use a combination of inputs that you’re very unlikely to do by accident. Having what is basically a self-destruct button on the controller where you naturally rest a finger just seems like a bad idea— I pushed it on purpose thinking it did something different, but there are all kinds of reasons why a pilot might push it accidentally. And if you do, that’s it, your drone is going down. 

DJI, to their credit, was very understanding about the whole thing, but more importantly, they pointed out that accidental damage like this would be covered under DJI Care Refresh, which will completely replace the drone if necessary. This should give new pilots some piece of mind, if you’re willing to pay for the premium. Even if you’re not, the drone is designed to be at least somewhat end-user repairable.

Fundamentally, I’m glad Manual mode is there. DJI made the right choice by including it so that your skill won’t outgrow the capabilities of the drone. I just wish that the transition to Manual mode was more gradual, like if there was a Sport Plus mode that unlocked top speed while maintaining other flight assistance features. Even without that, FPV beginners really shouldn’t feel like Manual mode needs to be a goal— Normal mode is fun, and Sport mode is even more fun, with the added piece of mind that you’ve got options if things start to get out of your control. And if things do get out of your control in Manual mode, for heaven’s sake, push the right button.

I mean, the left button.

Is This The Right Drone for You? Photo: Evan Ackerman/IEEE Spectrum My partner, who is totally not interested in drones, actually had fun flying this one.

DJI’s FPV drone kit costs $1,299, which includes the drone, goggles, one battery, and all necessary chargers and cabling. Two more batteries and a charging hub, which you’ll almost certainly want, adds $299. This is a lot of money, even for a drone, so the thing to ask yourself is whether an FPV drone is really what you’re looking for. Yes, it’ll take good quality pictures and video, but if that’s what you’re after, DJI has lots of other drones that are cheaper and more portable and have some smarts to make them better camera platforms. And of course there’s the Skydio 2, which has some crazy obstacle avoidance and autonomy if you don’t want to have to worry about flying at all. I have a fantasy that one day, all of this will be combined into one single drone, but we’re not there yet. 

If you’re sure you want to get into FPV flying, DJI’s kit seems like a great option, with the recognition that this is an expensive, equipment-intensive sport. There are definitely ways of doing it for cheaper, but you’ll need to more or less build up the system yourself, and it seems unlikely that you’d end up with the same kind of reliable performance and software features that DJI’s system comes with. The big advantage of DJI’s FPV kit is that you can immediately get started with a system that works brilliantly out of the box, in a way that’s highly integrated, highly functional, high performing while being fantastic for beginners and leaving plenty of room to grow.

Haru is a social, affective robot designed to support a wide range of research into human–robot communication. This article analyses the design process for Haru beta, identifying how both visual and performing arts were an essential part of that process, contributing to ideas of Haru’s communication as a science and as an art. Initially, the article examines how a modified form of Design Thinking shaped the work of the interdisciplinary development team—including animators, performers and sketch artists working alongside roboticists—to frame Haru’s interaction style in line with sociopsychological and cybernetic–semiotic communication theory. From these perspectives on communication, the focus is on creating a robot that is persuasive and able to transmit precise information clearly. The article moves on to highlight two alternative perspectives on communication, based on phenomenological and sociocultural theories, from which such a robot can be further developed as a more flexible and dynamic communicative agent. The various theoretical perspectives introduced are brought together by considering communication across three elements: encounter, story and dance. Finally, the article explores the potential of Haru as a research platform for human–robot communication across various scenarios designed to investigate how to support long-term interactions between humans and robots in different contexts. In particular, it gives an overview of plans for humanities-based, qualitative research with Haru.

Photo: CIA Museum

CIA roboticists designed Catfish Charlie to take water samples undetected. Why they wanted a spy fish for such a purpose remains classified.

In 1961, Tom Rogers of the Leo Burnett Agency created Charlie the Tuna, a jive-talking cartoon mascot and spokesfish for the StarKist brand. The popular ad campaign ran for several decades, and its catchphrase “Sorry, Charlie” quickly hooked itself in the American lexicon.

When the CIA’s Office of Advanced Technologies and Programs started conducting some fish-focused research in the 1990s, Charlie must have seemed like the perfect code name. Except that the CIA’s Charlie was a catfish. And it was a robot.

More precisely, Charlie was an unmanned underwater vehicle (UUV) designed to surreptitiously collect water samples. Its handler controlled the fish via a line-of-sight radio handset. Not much has been revealed about the fish’s construction except that its body contained a pressure hull, ballast system, and communications system, while its tail housed the propulsion. At 61 centimeters long, Charlie wouldn’t set any biggest-fish records. (Some species of catfish can grow to 2 meters.) Whether Charlie reeled in any useful intel is unknown, as details of its missions are still classified.

For exploring watery environments, nothing beats a robot

The CIA was far from alone in its pursuit of UUVs nor was it the first agency to do so. In the United States, such research began in earnest in the 1950s, with the U.S. Navy’s funding of technology for deep-sea rescue and salvage operations. Other projects looked at sea drones for surveillance and scientific data collection.

Aaron Marburg, a principal electrical and computer engineer who works on UUVs at the University of Washington’s Applied Physics Laboratory, notes that the world’s oceans are largely off-limits to crewed vessels. “The nature of the oceans is that we can only go there with robots,” he told me in a recent Zoom call. To explore those uncharted regions, he said, “we are forced to solve the technical problems and make the robots work.”

Image: Thomas Wells/Applied Physics Laboratory/University of Washington An oil painting commemorates SPURV, a series of underwater research robots built by the University of Washington’s Applied Physics Lab. In nearly 400 deployments, no SPURVs were lost.

One of the earliest UUVs happens to sit in the hall outside Marburg’s office: the Self-Propelled Underwater Research Vehicle, or SPURV, developed at the applied physics lab beginning in the late ’50s. SPURV’s original purpose was to gather data on the physical properties of the sea, in particular temperature and sound velocity. Unlike Charlie, with its fishy exterior, SPURV had a utilitarian torpedo shape that was more in line with its mission. Just over 3 meters long, it could dive to 3,600 meters, had a top speed of 2.5 m/s, and operated for 5.5 hours on a battery pack. Data was recorded to magnetic tape and later transferred to a photosensitive paper strip recorder or other computer-compatible media and then plotted using an IBM 1130.

Over time, SPURV’s instrumentation grew more capable, and the scope of the project expanded. In one study, for example, SPURV carried a fluorometer to measure the dispersion of dye in the water, to support wake studies. The project was so successful that additional SPURVs were developed, eventually completing nearly 400 missions by the time it ended in 1979.

Working on underwater robots, Marburg says, means balancing technical risks and mission objectives against constraints on funding and other resources. Support for purely speculative research in this area is rare. The goal, then, is to build UUVs that are simple, effective, and reliable. “No one wants to write a report to their funders saying, ‘Sorry, the batteries died, and we lost our million-dollar robot fish in a current,’ ” Marburg says.

A robot fish called SoFi

Since SPURV, there have been many other unmanned underwater vehicles, of various shapes and sizes and for various missions, developed in the United States and elsewhere. UUVs and their autonomous cousins, AUVs, are now routinely used for scientific research, education, and surveillance.

At least a few of these robots have been fish-inspired. In the mid-1990s, for instance, engineers at MIT worked on a RoboTuna, also nicknamed Charlie. Modeled loosely on a blue-fin tuna, it had a propulsion system that mimicked the tail fin of a real fish. This was a big departure from the screws or propellers used on UUVs like SPURV. But this Charlie never swam on its own; it was always tethered to a bank of instruments. The MIT group’s next effort, a RoboPike called Wanda, overcame this limitation and swam freely, but never learned to avoid running into the sides of its tank.

Fast-forward 25 years, and a team from MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL) unveiled SoFi, a decidedly more fishy robot designed to swim next to real fish without disturbing them. Controlled by a retrofitted Super Nintendo handset, SoFi could dive more than 15 meters, control its own buoyancy, and swim around for up to 40 minutes between battery charges. Noting that SoFi’s creators tested their robot fish in the gorgeous waters off Fiji, IEEE Spectrum’s Evan Ackerman noted, “Part of me is convinced that roboticists take on projects like these...because it’s a great way to justify a trip somewhere exotic.”

SoFi, Wanda, and both Charlies are all examples of biomimetics, a term coined in 1974 to describe the study of biological mechanisms, processes, structures, and substances. Biomimetics looks to nature to inspire design.

Sometimes, the resulting technology proves to be more efficient than its natural counterpart, as Richard James Clapham discovered while researching robotic fish for his Ph.D. at the University of Essex, in England. Under the supervision of robotics expert Huosheng Hu, Clapham studied the swimming motion of Cyprinus carpio, the common carp. He then developed four robots that incorporated carplike swimming, the most capable of which was iSplash-II. When tested under ideal conditions—that is, a tank 5 meters long, 2 meters wide, and 1.5 meters deep—iSpash-II obtained a maximum velocity of 11.6 body lengths per second (or about 3.7 m/s). That’s faster than a real carp, which averages a top velocity of 10 body lengths per second. But iSplash-II fell short of the peak performance of a fish darting quickly to avoid a predator.

Of course, swimming in a test pool or placid lake is one thing; surviving the rough and tumble of a breaking wave is another matter. The latter is something that roboticist Kathryn Daltorio has explored in depth.

Daltorio, an assistant professor at Case Western Reserve University and codirector of the Center for Biologically Inspired Robotics Research there, has studied the movements of cockroaches, earthworms, and crabs for clues on how to build better robots. After watching a crab navigate from the sandy beach to shallow water without being thrown off course by a wave, she was inspired to create an amphibious robot with tapered, curved feet that could dig into the sand. This design allowed her robot to withstand forces up to 138 percent of its body weight.

Photo: Nicole Graf

This robotic crab created by Case Western’s Kathryn Daltorio imitates how real crabs grab the sand to avoid being toppled by waves.

In her designs, Daltorio is following architect Louis Sullivan’s famous maxim: Form follows function. She isn’t trying to imitate the aesthetics of nature—her robot bears only a passing resemblance to a crab—but rather the best functionality. She looks at how animals interact with their environments and steals evolution’s best ideas.

And yet, Daltorio admits, there is also a place for realistic-looking robotic fish, because they can capture the imagination and spark interest in robotics as well as nature. And unlike a hyperrealistic humanoid, a robotic fish is unlikely to fall into the creepiness of the uncanny valley.

In writing this column, I was delighted to come across plenty of recent examples of such robotic fish. Ryomei Engineering, a subsidiary of Mitsubishi Heavy Industries, has developed several: a robo-coelacanth, a robotic gold koi, and a robotic carp. The coelacanth was designed as an educational tool for aquariums, to present a lifelike specimen of a rarely seen fish that is often only known by its fossil record. Meanwhile, engineers at the University of Kitakyushu in Japan created Tai-robot-kun, a credible-looking sea bream. And a team at Evologics, based in Berlin, came up with the BOSS manta ray.

Whatever their official purpose, these nature-inspired robocreatures can inspire us in return. UUVs that open up new and wondrous vistas on the world’s oceans can extend humankind’s ability to explore. We create them, and they enhance us, and that strikes me as a very fair and worthy exchange.

This article appears in the March 2021 print issue as “Catfish, Robot, Swimmer, Spy.”

About the Author

Allison Marsh is an associate professor of history at the University of South Carolina and codirector of the university’s Ann Johnson Institute for Science, Technology & Society.

Video Friday is your weekly selection of awesome robotics videos, collected by your Automaton bloggers. We’ll also be posting a weekly calendar of upcoming robotics events for the next few months; here's what we have so far (send us your events!):

HRI 2021 – March 8-11, 2021 – [Online Conference] RoboSoft 2021 – April 12-16, 2021 – [Online Conference] ICRA 2021 – May 30-5, 2021 – Xi'an, China

Let us know if you have suggestions for next week, and enjoy today's videos.

Shiny robotic cat toy blimp!

I am pretty sure this is Google Translate getting things wrong, but the About page mentions that the blimp will “take you to your destination after appearing in the death of God.”

[ NTT DoCoMo ] via [ RobotStart ]

If you have yet to see this real-time video of Perseverance landing on Mars, drop everything and watch it.

During the press conference, someone commented that this is the first time anyone on the team who designed and built this system has ever seen it in operation, since it could only be tested at the component scale on Earth. This landing system has blown my mind since Curiosity.

Here's a better look at where Percy ended up:

[ NASA ]

The fact that Digit can just walk up and down wet, slippery, muddy hills without breaking a sweat is (still) astonishing.

[ Agility Robotics ]

SkyMul wants drones to take over the task of tying rebar, which looks like just the sort of thing we'd rather robots be doing so that we don't have to:

The tech certainly looks promising, and SkyMul says that they're looking for some additional support to bring things to the pilot stage.

[ SkyMul ]

Thanks Eohan!

Flatcat is a pet-like, playful robot that reacts to touch. Flatcat feels everything exactly: Cuddle with it, romp around with it, or just watch it do weird things of its own accord. We are sure that flatcat will amaze you, like us, and caress your soul.

I don't totally understand it, but I want it anyway.

[ Flatcat ]

Thanks Oswald!

This is how I would have a romantic dinner date if I couldn't get together in person. Herman the UR3 and an OptiTrack system let me remotely make a romantic meal!

[ Dave's Armoury ]

Here, we propose a novel design of deformable propellers inspired by dragonfly wings. The structure of these propellers includes a flexible segment similar to the nodus on a dragonfly wing. This flexible segment can bend, twist and even fold upon collision, absorbing force upon impact and protecting the propeller from damage.

[ Paper ]

Thanks Van!

In the 1970s, The CIA​ created the world's first miniaturized unmanned aerial vehicle, or UAV, which was intended to be a clandestine listening device. The Insectothopter was never deployed operationally, but was still revolutionary for its time.

It may never have been deployed (not that they'll admit to, anyway), but it was definitely operational and could fly controllably.

[ CIA ]

Research labs are starting to get Digits, which means we're going to get a much better idea of what its limitations are.

[ Ohio State ]

This video shows the latest achievements for LOLA walking on undetected uneven terrain. The robot is technically blind, not using any camera-based or prior information on the terrain.

[ TUM ]

We define "robotic contact juggling" to be the purposeful control of the motion of a three-dimensional smooth object as it rolls freely on a motion-controlled robot manipulator, or “hand.” While specific examples of robotic contact juggling have been studied before, in this paper we provide the first general formulation and solution method for the case of an arbitrary smooth object in single-point rolling contact on an arbitrary smooth hand.

[ Paper ]

Thanks Fan!

A couple of new cobots from ABB, designed to work safely around humans.

[ ABB ]

Thanks Fan!

It's worth watching at least a little bit of Adam Savage testing Spot's new arm, because we get to see Spot try, fail, and eventually succeed at an autonomous door-opening behavior at the 10 minute mark.

[ Tested ]

SVR discusses diversity with guest speakers Dr. Michelle Johnson from the GRASP Lab at UPenn; Dr Ariel Anders from Women in Robotics and first technical hire at Robust.ai; Alka Roy from The Responsible Innovation Project; and Kenechukwu C. Mbanesi and Kenya Andrews from Black in Robotics. The discussion here is moderated by Dr. Ken Goldberg—artist, roboticist and Director of the CITRIS People and Robots Lab—and Andra Keay from Silicon Valley Robotics.

[ SVR ]

RAS presents a Soft Robotics Debate on Bioinspired vs. Biohybrid Design.

In this debate, we will bring together experts in Bioinspiration and Biohybrid design to discuss the necessary steps to make more competent soft robots. We will try to answer whether bioinspired research should focus more on developing new bioinspired material and structures or on the integration of living and artificial structures in biohybrid designs.

[ RAS SoRo ]

IFRR presents a Colloquium on Human Robot Interaction.

Across many application domains, robots are expected to work in human environments, side by side with people. The users will vary substantially in background, training, physical and cognitive abilities, and readiness to adopt technology. Robotic products are expected to not only be intuitive, easy to use, and responsive to the needs and states of their users, but they must also be designed with these differences in mind, making human-robot interaction (HRI) a key area of research.

[ IFRR ]

Vijay Kumar, Nemirovsky Family Dean and Professor at Penn Engineering, gives an introduction to ENIAC day and David Patterson, Pardee Professor of Computer Science, Emeritus at the University of California at Berkeley, speaks about the legacy of the ENIAC and its impact on computer architecture today. This video is comprised of lectures one and two of nine total lectures in the ENIAC Day series.

There are more interesting ENIAC videos at the link below, but we'll highlight this particular one, about the women of the ENIAC, also known as the First Programmers.

[ ENIAC Day ]

Video Friday is your weekly selection of awesome robotics videos, collected by your Automaton bloggers. We’ll also be posting a weekly calendar of upcoming robotics events for the next few months; here's what we have so far (send us your events!):

HRI 2021 – March 8-11, 2021 – [Online Conference] RoboSoft 2021 – April 12-16, 2021 – [Online Conference] ICRA 2021 – May 30-5, 2021 – Xi'an, China

Let us know if you have suggestions for next week, and enjoy today's videos.

Shiny robotic cat toy blimp!

I am pretty sure this is Google Translate getting things wrong, but the About page mentions that the blimp will “take you to your destination after appearing in the death of God.”

[ NTT DoCoMo ] via [ RobotStart ]

If you have yet to see this real-time video of Perseverance landing on Mars, drop everything and watch it.

During the press conference, someone commented that this is the first time anyone on the team who designed and built this system has ever seen it in operation, since it could only be tested at the component scale on Earth. This landing system has blown my mind since Curiosity.

Here's a better look at where Percy ended up:

[ NASA ]

The fact that Digit can just walk up and down wet, slippery, muddy hills without breaking a sweat is (still) astonishing.

[ Agility Robotics ]

SkyMul wants drones to take over the task of tying rebar, which looks like just the sort of thing we'd rather robots be doing so that we don't have to:

The tech certainly looks promising, and SkyMul says that they're looking for some additional support to bring things to the pilot stage.

[ SkyMul ]

Thanks Eohan!

Flatcat is a pet-like, playful robot that reacts to touch. Flatcat feels everything exactly: Cuddle with it, romp around with it, or just watch it do weird things of its own accord. We are sure that flatcat will amaze you, like us, and caress your soul.

I don't totally understand it, but I want it anyway.

[ Flatcat ]

Thanks Oswald!

This is how I would have a romantic dinner date if I couldn't get together in person. Herman the UR3 and an OptiTrack system let me remotely make a romantic meal!

[ Dave's Armoury ]

Here, we propose a novel design of deformable propellers inspired by dragonfly wings. The structure of these propellers includes a flexible segment similar to the nodus on a dragonfly wing. This flexible segment can bend, twist and even fold upon collision, absorbing force upon impact and protecting the propeller from damage.

[ Paper ]

Thanks Van!

In the 1970s, The CIA​ created the world's first miniaturized unmanned aerial vehicle, or UAV, which was intended to be a clandestine listening device. The Insectothopter was never deployed operationally, but was still revolutionary for its time.

It may never have been deployed (not that they'll admit to, anyway), but it was definitely operational and could fly controllably.

[ CIA ]

Research labs are starting to get Digits, which means we're going to get a much better idea of what its limitations are.

[ Ohio State ]

This video shows the latest achievements for LOLA walking on undetected uneven terrain. The robot is technically blind, not using any camera-based or prior information on the terrain.

[ TUM ]

We define "robotic contact juggling" to be the purposeful control of the motion of a three-dimensional smooth object as it rolls freely on a motion-controlled robot manipulator, or “hand.” While specific examples of robotic contact juggling have been studied before, in this paper we provide the first general formulation and solution method for the case of an arbitrary smooth object in single-point rolling contact on an arbitrary smooth hand.

[ Paper ]

Thanks Fan!

A couple of new cobots from ABB, designed to work safely around humans.

[ ABB ]

Thanks Fan!

It's worth watching at least a little bit of Adam Savage testing Spot's new arm, because we get to see Spot try, fail, and eventually succeed at an autonomous door-opening behavior at the 10 minute mark.

[ Tested ]

SVR discusses diversity with guest speakers Dr. Michelle Johnson from the GRASP Lab at UPenn; Dr Ariel Anders from Women in Robotics and first technical hire at Robust.ai; Alka Roy from The Responsible Innovation Project; and Kenechukwu C. Mbanesi and Kenya Andrews from Black in Robotics. The discussion here is moderated by Dr. Ken Goldberg—artist, roboticist and Director of the CITRIS People and Robots Lab—and Andra Keay from Silicon Valley Robotics.

[ SVR ]

RAS presents a Soft Robotics Debate on Bioinspired vs. Biohybrid Design.

In this debate, we will bring together experts in Bioinspiration and Biohybrid design to discuss the necessary steps to make more competent soft robots. We will try to answer whether bioinspired research should focus more on developing new bioinspired material and structures or on the integration of living and artificial structures in biohybrid designs.

[ RAS SoRo ]

IFRR presents a Colloquium on Human Robot Interaction.

Across many application domains, robots are expected to work in human environments, side by side with people. The users will vary substantially in background, training, physical and cognitive abilities, and readiness to adopt technology. Robotic products are expected to not only be intuitive, easy to use, and responsive to the needs and states of their users, but they must also be designed with these differences in mind, making human-robot interaction (HRI) a key area of research.

[ IFRR ]

Vijay Kumar, Nemirovsky Family Dean and Professor at Penn Engineering, gives an introduction to ENIAC day and David Patterson, Pardee Professor of Computer Science, Emeritus at the University of California at Berkeley, speaks about the legacy of the ENIAC and its impact on computer architecture today. This video is comprised of lectures one and two of nine total lectures in the ENIAC Day series.

There are more interesting ENIAC videos at the link below, but we'll highlight this particular one, about the women of the ENIAC, also known as the First Programmers.

[ ENIAC Day ]

Now that DeepMind has taught AI to master the game of Go—and furthered its advantage in chess—they’ve turned their attention to another board game: Diplomacy. Unlike Go, it is seven-player, it requires a combination of competition and cooperation, and on each turn players make moves simultaneously, so they must reason about what others are reasoning about them, and so on.

“It’s a qualitatively different problem from something like Go or chess,” says Andrea Tacchetti, a computer scientist at DeepMind. In December, Tacchetti and collaborators presented a paper at the NeurIPS conference on their system, which advances the state of the art, and may point the way toward AI systems with real-world diplomatic skills—in negotiating with strategic or commercial partners or simply scheduling your next team meeting. 

Diplomacy is a strategy game played on a map of Europe divided into 75 provinces. Players build and mobilize military units to occupy provinces until someone controls a majority of supply centers. Each turn, players write down their moves, which are then executed simultaneously. They can attack or defend against opposing players’ units, or support opposing players’ attacks and defenses, building alliances. In the full version, players can negotiate. DeepMind tackled the simpler No-Press Diplomacy, devoid of explicit communication. 

Historically, AI has played Diplomacy using hand-crafted strategies. In 2019, the Montreal research institute Mila beat the field with a system using deep learning. They trained a neural network they called DipNet to imitate humans, based on a dataset of 150,000 human games. DeepMind started with a version of DipNet and refined it using reinforcement learning, a kind of trial-and-error. 

Exploring the space of possibility purely through trial-and-error would pose problems, though. They calculated that a 20-move game can be played nearly 10868 ways—yes, that’s 10 with 868 zeroes after it.

So they tweaked their reinforcement-learning algorithm. During training, on each move, they sample likely moves of opponents, calculate the move that works best on average across these scenarios, then train their net to prefer this move. After training, it skips the sampling and just works from what its learning has taught it. “The message of our paper is: we can make reinforcement learning work in such an environment,” Tacchetti says. One of their AI players versus six DipNets won 30 percent of the time (with 14 percent being chance). One DipNet against seven of theirs won only 3 percent of the time.

In April, Facebook will present a paper at the ICLR conference describing their own work on No-Press Diplomacy. They also built on a human-imitating network similar to DipNet. But instead of adding reinforcement learning, they added search—the techniques of taking extra time to plan ahead and reason about what every player is likely to do next. On each turn, SearchBot computes an equilibrium, a strategy for each player that the player can’t improve by switching only its own strategy. To do this, SearchBot evaluates each potential strategy for a player by playing the game out a few turns (assuming everyone chooses subsequent moves based on the net’s top choice). A strategy consists not of a single best move but a set of probabilities across 50 likely moves (suggested by the net), to avoid being too predictable to opponents. 

Conducting such exploration during a real game slows SearchBot down, but allows it beat DipNet by an even greater margin than DeepMind’s system does. SearchBot also played anonymously against humans on a Diplomacy website and ranked in the top 2 percent of players. “This is the first bot that’s demonstrated to be competitive with humans,” says Adam Lerer, a computer scientist at Facebook and paper co-author.

“I think the most important point is that search is often underestimated,” Lerer says. One of his Facebook collaborators, Noam Brown, implemented search in a superhuman poker bot. Brown says the most surprising finding was that their method could find equilibria, a computationally difficult task.

“I was really happy when I saw their paper,” Tacchetti says, “because of just how different their ideas were to ours, which means that there’s so much stuff that we can try still.” Lerer sees a future in combining reinforcement learning and search, which worked well for DeepMind’s AlphaGo.

Both teams found that their systems were not easily exploitable. Facebook, for example, invited two top human players to each play 35 straight games against SearchBot, probing for weaknesses. The humans won only 6 percent of the time. Both groups also found that their systems didn’t just compete, but also cooperated, sometimes supporting opponents. “They get that in order to win, they have to work with others,” says Yoram Bachrach, from the DeepMind team.

That’s important, Bachrach, Lerer, and Tacchetti say, because games that combine competition and cooperation are much more realistic than purely competitive games like Go. Mixed motives occur in all realms of life: driving in traffic, negotiating contracts, and arranging times to Zoom. 

How close are we to AI that can play Diplomacy with “press,” negotiating all the while using natural language?

“For Press Diplomacy, as well as other settings that mix cooperation and competition, you need progress,” Bachrach says, “in terms of theory of mind, how they can communicate with others about their preferences or goals or plans. And, one step further, you can look at the institutions of multiple agents that human society has. All of this work is super exciting, but these are early days.”

The human ability of keeping balance during various locomotion tasks is attributed to our capability of withstanding complex interactions with the environment and coordinating whole-body movements. Despite this, several stability analysis methods are limited by the use of overly simplified biped and foot structures and corresponding contact models. As a result, existing stability criteria tend to be overly restrictive and do not represent the full balance capabilities of complex biped systems. The proposed methodology allows for the characterization of the balance capabilities of general biped models (ranging from reduced-order to whole-body) with segmented feet. Limits of dynamic balance are evaluated by the Boundary of Balance (BoB) and the associated novel balance indicators, both formulated in the Center of Mass (COM) state space. Intermittent heel, flat, and toe contacts are enabled by a contact model that maps discrete contact modes into corresponding center of pressure constraints. For demonstration purposes, the BoB and balance indicators are evaluated for a whole-body biped model with segmented feet representative of the human-like standing posture in the sagittal plane. The BoB is numerically constructed as the set of maximum allowable COM perturbations that the biped can sustain along a prescribed direction. For each point of the BoB, a constrained trajectory optimization algorithm generates the biped’s whole-body trajectory as it recovers from extreme COM velocity perturbations in the anterior–posterior direction. Balance capabilities for the cases of flat and segmented feet are compared, demonstrating the functional role the foot model plays in the limits of postural balance. The state-space evaluation of the BoB and balance indicators allows for a direct comparison between the proposed balance benchmark and existing stability criteria based on reduced-order models [e.g., Linear Inverted Pendulum (LIP)] and their associated stability metrics [e.g., Margin of Stability (MOS)]. The proposed characterization of balance capabilities provides an important benchmarking framework for the stability of general biped/foot systems.

Pages