Feed aggregator

Colorectal cancer as a major disease that poses a serious threat to human health continues to rise in incidence. And the timely colon examinations are crucial for the prevention, diagnosis, and treatment of this disease. Clinically, gastroscopy is used as a universal means of examination, prevention and diagnosis of this disease, but this detection method is not patient-friendly and can easily cause damage to the intestinal mucosa. Soft robots as an emerging technology offer a promising approach to examining, diagnosing, and treating intestinal diseases due to their high flexibility and patient-friendly interaction. However, existing research on intestinal soft robots mainly focuses on controlled movement and observation within the colon or colon-like environments, lacking additional functionalities such as sample collection from the intestine. Here, we designed and developed an earthworm-like soft robot specifically for colon sampling. It consists of a robot body with an earthworm-like structure for movement in the narrow and soft pipe-environments, and a sampling part with a flexible arm structure resembling an elephant trunk for bidirectional bending sampling. This soft robot is capable of flexible movement and sample collection within an colon-like environment. By successfully demonstrating the feasibility of utilizing soft robots for colon sampling, this work introduces a novel method for non-destructive inspection and sampling in the colon. It represents a significant advancement in the field of medical robotics, offering a potential solution for more efficient and accurate examination and diagnosis of intestinal diseases, specifically for colorectal cancer.



About a decade ago, there was a lot of excitement in the robotics world around gecko-inspired directional adhesives, which are materials that stick without being sticky using the same van der Waals forces that allow geckos to scamper around on vertical panes of glass. They were used extensively in different sorts of climbing robots, some of them quite lovely. Gecko adhesives are uniquely able to stick to very smooth things where your only other option might be suction, which requires all kinds of extra infrastructure to work.

We haven’t seen gecko adhesives around as much of late, for a couple of reasons. First, the ability to only stick to smooth surfaces (which is what gecko adhesives are best at) is a bit of a limitation for mobile robots. And second, the gap between research and useful application is wide and deep and full of crocodiles. I’m talking about the mean kind of crocodiles, not the cuddly kind. But Flexiv Robotics has made gecko adhesives practical for robotic grasping in a commercial environment, thanks in part to a sort of robotic tongue that licks the gecko tape clean.

If you zoom way, way in on a gecko’s foot, you’ll see that each toe is covered in millions of hair-like nanostructures called setae. Each setae branches out at the end into hundreds of more hairs with flat bits at the end called spatulas. The result of this complex arrangement of setae and spatulas is that gecko toes have a ridiculous amount of surface area, meaning that they can leverage the extremely weak van der Waals forces between molecules to stick themselves to perfectly flat and smooth surfaces. This technique works exceptionally well: Geckos can hang from glass by a single toe, and a fully adhered gecko can hold something like 140 kg (which, unfortunately, seems to be an extrapolation rather than an experimental result). And luckily for the gecko, the structure of the spatulas makes the adhesion directional, so that when its toes are no longer being loaded, they can be easily peeled off of whatever they’re attached to.

Natural gecko adhesive structure, along with a synthetic adhesive (f).Gecko adhesion: evolutionary nanotechnology, by Kellar Autumn and Nick Gravish

Since geckos don’t “stick” to things in the sense that we typically use the word “sticky,” a better way of characterizing what geckos can do is as “dry adhesion,” as opposed to something that involves some sort of glue. You can also think about gecko toes as just being very, very high friction, and it’s this perspective that is particularly interesting in the context of robotic grippers.

This is Flexiv’s “Grav Enhanced” gripper, which uses a combination of pinch grasping and high friction gecko adhesive to lift heavy and delicate objects without having to squeeze them. When you think about a traditional robotic grasping system trying to lift something like a water balloon, you have to squeeze that balloon until the friction between the side of the gripper and the side of the balloon overcomes the weight of the balloon itself. The higher the friction, the lower the squeeze required, and although a water balloon might be an extreme example, maximizing gripper friction can make a huge difference when it comes to fragile or deformable objects.

There are a couple of problems with dry adhesive, however. The tiny structures that make the adhesive adhere can be prone to damage, and the fact that dry adhesive will stick to just about anything it can make good contact with means that it’ll rapidly accumulate dirt outside of a carefully controlled environment. In research contexts, these problems aren’t all that significant, but for a commercial system, you can’t have something that requires constant attention.

Flexiv says that the microstructure material that makes up their gecko adhesive was able to sustain two million gripping cycles without any visible degradation in performance, suggesting that as long as you use the stuff within the tolerances that it’s designed for, it should keep on adhering to things indefinitely—although trying to lift too much weight will tear the microstructures, ruining the adhesive properties after just a few cycles. And to keep the adhesive from getting clogged up with debris, Flexiv came up with this clever little cleaning station that acts like a little robotic tongue of sorts:

Interestingly, geckos themselves don’t seem to use their own tongues to clean their toes. They lick their eyeballs on the regular, like all normal humans do, but gecko toes appear to be self-cleaning, which is a pretty neat trick. It’s certainly possible to make self-cleaning synthetic gecko adhesive, but Flexiv tells us that “due to technical and practical limitations, replicating this process in our own gecko adhesive material is not possible. Essentially, we replicate the microstructure of a gecko’s footpad, but not its self-cleaning process.” This likely goes back to that whole thing about what works in a research context versus what works in a commercial context, and Flexiv needs their gecko adhesive to handle all those millions of cycles.

Flexiv says that they were made aware of the need for a system like this when one of their clients started using the gripper for the extra-dirty task of sorting trash from recycling, and that the solution was inspired by a lint roller. And I have to say, I appreciate the simplicity of the system that Flexiv came up with to solve the problem directly and efficiently. Maybe one day, they’ll be able to replicate a real gecko’s natural self-cleaning toes with a durable and affordable artificial dry adhesive, but until that happens, an artificial tongue does the trick.


About a decade ago, there was a lot of excitement in the robotics world around gecko-inspired directional adhesives, which are materials that stick without being sticky using the same van der Waals forces that allow geckos to scamper around on vertical panes of glass. They were used extensively in different sorts of climbing robots, some of them quite lovely. Gecko adhesives are uniquely able to stick to very smooth things where your only other option might be suction, which requires all kinds of extra infrastructure to work.

We haven’t seen gecko adhesives around as much of late, for a couple of reasons. First, the ability to only stick to smooth surfaces (which is what gecko adhesives are best at) is a bit of a limitation for mobile robots. And second, the gap between research and useful application is wide and deep and full of crocodiles. I’m talking about the mean kind of crocodiles, not the cuddly kind. But Flexiv Robotics has made gecko adhesives practical for robotic grasping in a commercial environment, thanks in part to a sort of robotic tongue that licks the gecko tape clean.

If you zoom way, way in on a gecko’s foot, you’ll see that each toe is covered in millions of hair-like nanostructures called setae. Each setae branches out at the end into hundreds of more hairs with flat bits at the end called spatulas. The result of this complex arrangement of setae and spatulas is that gecko toes have a ridiculous amount of surface area, meaning that they can leverage the extremely weak van der Waals forces between molecules to stick themselves to perfectly flat and smooth surfaces. This technique works exceptionally well: Geckos can hang from glass by a single toe, and a fully adhered gecko can hold something like 140 kg (which, unfortunately, seems to be an extrapolation rather than an experimental result). And luckily for the gecko, the structure of the spatulas makes the adhesion directional, so that when its toes are no longer being loaded, they can be easily peeled off of whatever they’re attached to.

Natural gecko adhesive structure, along with a synthetic adhesive (f).Gecko adhesion: evolutionary nanotechnology, by Kellar Autumn and Nick Gravish

Since geckos don’t “stick” to things in the sense that we typically use the word “sticky,” a better way of characterizing what geckos can do is as “dry adhesion,” as opposed to something that involves some sort of glue. You can also think about gecko toes as just being very, very high friction, and it’s this perspective that is particularly interesting in the context of robotic grippers.

This is Flexiv’s “Grav Enhanced” gripper, which uses a combination of pinch grasping and high friction gecko adhesive to lift heavy and delicate objects without having to squeeze them. When you think about a traditional robotic grasping system trying to lift something like a water balloon, you have to squeeze that balloon until the friction between the side of the gripper and the side of the balloon overcomes the weight of the balloon itself. The higher the friction, the lower the squeeze required, and although a water balloon might be an extreme example, maximizing gripper friction can make a huge difference when it comes to fragile or deformable objects.

There are a couple of problems with dry adhesive, however. The tiny structures that make the adhesive adhere can be prone to damage, and the fact that dry adhesive will stick to just about anything it can make good contact with means that it’ll rapidly accumulate dirt outside of a carefully controlled environment. In research contexts, these problems aren’t all that significant, but for a commercial system, you can’t have something that requires constant attention.

Flexiv says that the microstructure material that makes up their gecko adhesive was able to sustain two million gripping cycles without any visible degradation in performance, suggesting that as long as you use the stuff within the tolerances that it’s designed for, it should keep on adhering to things indefinitely—although trying to lift too much weight will tear the microstructures, ruining the adhesive properties after just a few cycles. And to keep the adhesive from getting clogged up with debris, Flexiv came up with this clever little cleaning station that acts like a little robotic tongue of sorts:

Interestingly, geckos themselves don’t seem to use their own tongues to clean their toes. They lick their eyeballs on the regular, like all normal humans do, but gecko toes appear to be self-cleaning, which is a pretty neat trick. It’s certainly possible to make self-cleaning synthetic gecko adhesive, but Flexiv tells us that “due to technical and practical limitations, replicating this process in our own gecko adhesive material is not possible. Essentially, we replicate the microstructure of a gecko’s footpad, but not its self-cleaning process.” This likely goes back to that whole thing about what works in a research context versus what works in a commercial context, and Flexiv needs their gecko adhesive to handle all those millions of cycles.

Flexiv says that they were made aware of the need for a system like this when one of their clients started using the gripper for the extra-dirty task of sorting trash from recycling, and that the solution was inspired by a lint roller. And I have to say, I appreciate the simplicity of the system that Flexiv came up with to solve the problem directly and efficiently. Maybe one day, they’ll be able to replicate a real gecko’s natural self-cleaning toes with a durable and affordable artificial dry adhesive, but until that happens, an artificial tongue does the trick.

Accurate texture classification empowers robots to improve their perception and comprehension of the environment, enabling informed decision-making and appropriate responses to diverse materials and surfaces. Still, there are challenges for texture classification regarding the vast amount of time series data generated from robots’ sensors. For instance, robots are anticipated to leverage human feedback during interactions with the environment, particularly in cases of misclassification or uncertainty. With the diversity of objects and textures in daily activities, Active Learning (AL) can be employed to minimize the number of samples the robot needs to request from humans, streamlining the learning process. In the present work, we use AL to select the most informative samples for annotation, thus reducing the human labeling effort required to achieve high performance for classifying textures. We also use a sliding window strategy for extracting features from the sensor’s time series used in our experiments. Our multi-class dataset (e.g., 12 textures) challenges traditional AL strategies since standard techniques cannot control the number of instances per class selected to be labeled. Therefore, we propose a novel class-balancing instance selection algorithm that we integrate with standard AL strategies. Moreover, we evaluate the effect of sliding windows of two-time intervals (3 and 6 s) on our AL Strategies. Finally, we analyze in our experiments the performance of AL strategies, with and without the balancing algorithm, regarding f1-score, and positive effects are observed in terms of performance when using our proposed data pipeline. Our results show that the training data can be reduced to 70% using an AL strategy regardless of the machine learning model and reach, and in many cases, surpass a baseline performance. Finally, exploring the textures with a 6-s window achieves the best performance, and using either Extra Trees produces an average f1-score of 90.21% in the texture classification data set.



Video Friday is your weekly selection of awesome robotics videos, collected by your friends at IEEE Spectrum robotics. We also post a weekly calendar of upcoming robotics events for the next few months. Please send us your events for inclusion.

Cybathlon Challenges: 2 February 2024, ZURICHEurobot Open 2024: 8–11 May 2024, LA ROCHE-SUR-YON, FRANCEICRA 2024: 13–17 May 2024, YOKOHAMA, JAPAN

Enjoy today’s videos!

Is “scamperiest” a word? If not, it should be, because this is the scamperiest robot I’ve ever seen.

[ ABS ]

GITAI is pleased to announce that its 1.5-meter-long autonomous dual robotic arm system (S2) has successfully arrived at the International Space Station (ISS) aboard the SpaceX Falcon 9 rocket (NG-20) to conduct an external demonstration of in-space servicing, assembly, and manufacturing (ISAM) while onboard the ISS. The success of the S2 tech demo will be a major milestone for GITAI, confirming the feasibility of this technology as a fully operational system in space.

[ GITAI ]

This work presents a comprehensive study on using deep reinforcement learning (RL) to create dynamic locomotion controllers for bipedal robots. Going beyond focusing on a single locomotion skill, we develop a general control solution that can be used for a range of dynamic bipedal skills, from periodic walking and running to aperiodic jumping and standing.

And if you want to get exhausted on behalf of a robot, the full 400-meter dash is below.

[ Hybrid Robotics ]

NASA’s Ingenuity Mars Helicopter pushed aerodynamic limits during the final months of its mission, setting new records for speed, distance, and altitude. Hear from Ingenuity chief engineer Travis Brown on how the data the team collected could eventually be used in future rotorcraft designs.

[ NASA ]

BigDog: 15 years of solving mobility problems its own way.

[ Boston Dynamics ]

[Harvard School of Engineering and Applied Sciences] researchers are helping develop resilient and autonomous deep space and extraterrestrial habitations by developing technologies to let autonomous robots repair or replace damaged components in a habitat. The research is part of the Resilient ExtraTerrestrial Habitats institute (RETHi) led by Purdue University, in partnership with [Harvard] SEAS, the University of Connecticut and the University of Texas at San Antonio. Its goal is to “design and operate resilient deep space habitats that can adapt, absorb and rapidly recover from expected and unexpected disruptions.”

[ Harvard SEAS ]

Researchers from Huazhong University of Science and Technology (HUST) in a recent T-RO paper describe and construct a novel variable stiffness spherical joint motor that enables dexterous motion and joint compliance in omni-directions.

[ Paper ]

Thanks, Ram!

We are told that this new robot from HEBI is called “Mark Suckerberg” and that they’ve got a pretty cool application in mind for it, to be revealed later this year.

[ HEBI Robotics ]

Thanks, Dave!

Dive into the first edition of our new Real-World-Robotics class at ETH Zürich! Our students embarked on an incredible journey, creating their human-like robotic hands from scratch. In just three months, the teams designed, built, and programmed their tendon-driven robotic hands, mastering dexterous manipulation with reinforcement learning! The result? A spectacular display of innovation and skill during our grand final.

[ SRL ETHZ ]

Carnegie Mellon researchers have built a system with a robotic arm atop a RangerMini 2.0 robotic cart from AgileX robotics to make what they’re calling a platform for “intelligent movement and processing.”

[ CMU ] via [ AgileX ]

Picassnake is our custom-made robot that paints pictures from music. Picassnake consists of an arm and a head, embedded in a plush snake doll. The robot is connected to a laptop for control and music processing, which can be fed through a microphone or an MP3 file. To open the media source, an operator can use the graphical user interface or place a text QR code in front of a webcam. Once the media source is opened, Picassnake generates unique strokes based on the music and translates the strokes to physical movement to paint them on canvas.

[ Picassnake ]

In April 2021, NASA’s Ingenuity Mars Helicopter became the first spacecraft to achieve powered, controlled flight on another world. With 72 successful flights, Ingenuity has far surpassed its originally planned technology demonstration of up to five flights. On Jan. 18, Ingenuity flew for the final time on the Red Planet. Join Tiffany Morgan, NASA’s Mars Exploration Program Deputy Director, and Teddy Tzanetos, Ingenuity Project Manager, as they discuss these historic flights and what they could mean for future extraterrestrial aerial exploration.

[ NASA ]



Video Friday is your weekly selection of awesome robotics videos, collected by your friends at IEEE Spectrum robotics. We also post a weekly calendar of upcoming robotics events for the next few months. Please send us your events for inclusion.

Cybathlon Challenges: 2 February 2024, ZURICHEurobot Open 2024: 8–11 May 2024, LA ROCHE-SUR-YON, FRANCEICRA 2024: 13–17 May 2024, YOKOHAMA, JAPAN

Enjoy today’s videos!

Is “scamperiest” a word? If not, it should be, because this is the scamperiest robot I’ve ever seen.

[ ABS ]

GITAI is pleased to announce that its 1.5-meter-long autonomous dual robotic arm system (S2) has successfully arrived at the International Space Station (ISS) aboard the SpaceX Falcon 9 rocket (NG-20) to conduct an external demonstration of in-space servicing, assembly, and manufacturing (ISAM) while onboard the ISS. The success of the S2 tech demo will be a major milestone for GITAI, confirming the feasibility of this technology as a fully operational system in space.

[ GITAI ]

This work presents a comprehensive study on using deep reinforcement learning (RL) to create dynamic locomotion controllers for bipedal robots. Going beyond focusing on a single locomotion skill, we develop a general control solution that can be used for a range of dynamic bipedal skills, from periodic walking and running to aperiodic jumping and standing.

And if you want to get exhausted on behalf of a robot, the full 400-meter dash is below.

[ Hybrid Robotics ]

NASA’s Ingenuity Mars Helicopter pushed aerodynamic limits during the final months of its mission, setting new records for speed, distance, and altitude. Hear from Ingenuity chief engineer Travis Brown on how the data the team collected could eventually be used in future rotorcraft designs.

[ NASA ]

BigDog: 15 years of solving mobility problems its own way.

[ Boston Dynamics ]

[Harvard School of Engineering and Applied Sciences] researchers are helping develop resilient and autonomous deep space and extraterrestrial habitations by developing technologies to let autonomous robots repair or replace damaged components in a habitat. The research is part of the Resilient ExtraTerrestrial Habitats institute (RETHi) led by Purdue University, in partnership with [Harvard] SEAS, the University of Connecticut and the University of Texas at San Antonio. Its goal is to “design and operate resilient deep space habitats that can adapt, absorb and rapidly recover from expected and unexpected disruptions.”

[ Harvard SEAS ]

Researchers from Huazhong University of Science and Technology (HUST) in a recent T-RO paper describe and construct a novel variable stiffness spherical joint motor that enables dexterous motion and joint compliance in omni-directions.

[ Paper ]

Thanks, Ram!

We are told that this new robot from HEBI is called “Mark Suckerberg” and that they’ve got a pretty cool application in mind for it, to be revealed later this year.

[ HEBI Robotics ]

Thanks, Dave!

Dive into the first edition of our new Real-World-Robotics class at ETH Zürich! Our students embarked on an incredible journey, creating their human-like robotic hands from scratch. In just three months, the teams designed, built, and programmed their tendon-driven robotic hands, mastering dexterous manipulation with reinforcement learning! The result? A spectacular display of innovation and skill during our grand final.

[ SRL ETHZ ]

Carnegie Mellon researchers have built a system with a robotic arm atop a RangerMini 2.0 robotic cart from AgileX robotics to make what they’re calling a platform for “intelligent movement and processing.”

[ CMU ] via [ AgileX ]

Picassnake is our custom-made robot that paints pictures from music. Picassnake consists of an arm and a head, embedded in a plush snake doll. The robot is connected to a laptop for control and music processing, which can be fed through a microphone or an MP3 file. To open the media source, an operator can use the graphical user interface or place a text QR code in front of a webcam. Once the media source is opened, Picassnake generates unique strokes based on the music and translates the strokes to physical movement to paint them on canvas.

[ Picassnake ]

In April 2021, NASA’s Ingenuity Mars Helicopter became the first spacecraft to achieve powered, controlled flight on another world. With 72 successful flights, Ingenuity has far surpassed its originally planned technology demonstration of up to five flights. On Jan. 18, Ingenuity flew for the final time on the Red Planet. Join Tiffany Morgan, NASA’s Mars Exploration Program Deputy Director, and Teddy Tzanetos, Ingenuity Project Manager, as they discuss these historic flights and what they could mean for future extraterrestrial aerial exploration.

[ NASA ]

Online questionnaires that use crowdsourcing platforms to recruit participants have become commonplace, due to their ease of use and low costs. Artificial intelligence (AI)-based large language models (LLMs) have made it easy for bad actors to automatically fill in online forms, including generating meaningful text for open-ended tasks. These technological advances threaten the data quality for studies that use online questionnaires. This study tested whether text generated by an AI for the purpose of an online study can be detected by both humans and automatic AI detection systems. While humans were able to correctly identify the authorship of such text above chance level (76% accuracy), their performance was still below what would be required to ensure satisfactory data quality. Researchers currently have to rely on a lack of interest among bad actors to successfully use open-ended responses as a useful tool for ensuring data quality. Automatic AI detection systems are currently completely unusable. If AI submissions of responses become too prevalent, then the costs associated with detecting fraudulent submissions will outweigh the benefits of online questionnaires. Individual attention checks will no longer be a sufficient tool to ensure good data quality. This problem can only be systematically addressed by crowdsourcing platforms. They cannot rely on automatic AI detection systems and it is unclear how they can ensure data quality for their paying clients.

Introduction: Navigation satellite systems can fail to work or work incorrectly in a number of conditions: signal shadowing, electromagnetic interference, atmospheric conditions, and technical problems. All of these factors can significantly affect the localization accuracy of autonomous driving systems. This emphasizes the need for other localization technologies, such as Lidar.

Methods: The use of the Kalman filter in combination with Lidar can be very effective in various applications due to the synergy of their capabilities. The Kalman filter can improve the accuracy of lidar measurements by taking into account the noise and inaccuracies present in the measurements.

Results: In this paper, we propose a parallel Kalman algorithm in three-dimensional space to speed up the computational speed of Lidar localization. At the same time, the initial localization accuracy of the latter is preserved. A distinctive feature of the proposed approach is that the Kalman localization algorithm itself is parallelized, rather than the process of building a map for navigation. The proposed algorithm allows us to obtain the result 3.8 times faster without compromising the localization accuracy, which was 3% for both cases, making it effective for real-time decision-making.

Discussion: The reliability of this result is confirmed by a preliminary theoretical estimate of the acceleration rate based on Ambdahl’s law. Accelerating the Kalman filter with CUDA for Lidar localization can be of significant practical value, especially in real-time and in conditions where large amounts of data from Lidar sensors need to be processed.



Just because an object is around a corner doesn’t mean it has to be hidden. Non-line-of-sight imaging can peek around corners and spot those objects, but it has so far been limited to a narrow band of frequencies. Now, a new sensor can help extend this technique from working with visible light to infrared. This advance could help make autonomous vehicles safer, among other potential applications.

Non-line-of-sight imaging relies on the faint signals of light beams that have reflected off surfaces in order to reconstruct images. The ability to see around corners may prove useful for machine vision—for instance, helping autonomous vehicles foresee hidden dangers to better predict how to respond to them, says Xiaolong Hu, the senior author of the study and a professor at Tianjin University in Tianjin, China. It may also improve endoscopes that help doctors peer inside the body.

The light that non-line-of-sight imaging depends on is typically very dim, and until now, the detectors that were efficient and sensitive enough for non-line-of-sight imaging could only detect either visible or near-infrared light. Moving to longer wavelengths might have several advantages, such as dealing with less interference from sunshine, and the possibility of using lasers that are safe around eyes, Hu says.

Now Hu and his colleagues have for the first time performed non-line-of-sight imaging using 1,560- and 1,997-nanometer infrared wavelengths. “This extension in spectrum paves the way for more practical applications,” Hu says.

The researchers imaged several objects with a non-line-of-sight infrared camera, both without [middle column] and with [right column] de-noising algorithms.Tianjin University

In the new study, the researchers experimented with superconducting nanowire single-photon detectors. In each device, a 40-nanometer-wide niobium titanium nitride wire was cooled to about 2 kelvins (about –271 °C), rendering the wire superconductive. A single photon could disrupt this fragile state, generating electrical pulses that enabled the efficient detection of individual photons.

The scientists contorted the nanowire in each device into a fractal pattern that took on similar shapes at various magnifications. This let the sensor detect photons of all polarizations, boosting its efficiency.

The new detector was up to nearly three times as efficient as other single-photon detectors at sensing near- and mid-infrared light. This let the researchers perform non-line-of-sight imaging, achieving a spatial resolution of roughly 1.3 to 1.5 centimeters.

In addition to an algorithm that reconstructed non-line-of-sight images based off multiple scattered light rays, the scientists developed a new algorithm that helped remove noise from their data. When each pixel during the scanning process was given 5 milliseconds to collect photons, the new de-noising algorithm reduced the root mean square error—a measure of its deviation from a perfect image—of reconstructed images by about eightfold.

The researchers now plan to arrange multiple sensors into larger arrays to boost efficiency, reduce scanning time, and extend the distance over which imaging can take place, Hu says. They would also like to test their device in daylight conditions, he adds.

The scientists detailed their findings 30 November in the journal Optics Express.



Just because an object is around a corner doesn’t mean it has to be hidden. Non-line-of-sight imaging can peek around corners and spot those objects, but it has so far been limited to a narrow band of frequencies. Now, a new sensor can help extend this technique from working with visible light to infrared. This advance could help make autonomous vehicles safer, among other potential applications.

Non-line-of-sight imaging relies on the faint signals of light beams that have reflected off surfaces in order to reconstruct images. The ability to see around corners may prove useful for machine vision—for instance, helping autonomous vehicles foresee hidden dangers to better predict how to respond to them, says Xiaolong Hu, the senior author of the study and a professor at Tianjin University in Tianjin, China. It may also improve endoscopes that help doctors peer inside the body.

The light that non-line-of-sight imaging depends on is typically very dim, and until now, the detectors that were efficient and sensitive enough for non-line-of-sight imaging could only detect either visible or near-infrared light. Moving to longer wavelengths might have several advantages, such as dealing with less interference from sunshine, and the possibility of using lasers that are safe around eyes, Hu says.

Now Hu and his colleagues have for the first time performed non-line-of-sight imaging using 1,560- and 1,997-nanometer infrared wavelengths. “This extension in spectrum paves the way for more practical applications,” Hu says.

The researchers imaged several objects with a non-line-of-sight infrared camera, both without [middle column] and with [right column] de-noising algorithms.Tianjin University

In the new study, the researchers experimented with superconducting nanowire single-photon detectors. In each device, a 40-nanometer-wide niobium titanium nitride wire was cooled to about 2 kelvins (about –271 °C), rendering the wire superconductive. A single photon could disrupt this fragile state, generating electrical pulses that enabled the efficient detection of individual photons.

The scientists contorted the nanowire in each device into a fractal pattern that took on similar shapes at various magnifications. This let the sensor detect photons of all polarizations, boosting its efficiency.

The new detector was up to nearly three times as efficient as other single-photon detectors at sensing near- and mid-infrared light. This let the researchers perform non-line-of-sight imaging, achieving a spatial resolution of roughly 1.3 to 1.5 centimeters.

In addition to an algorithm that reconstructed non-line-of-sight images based off multiple scattered light rays, the scientists developed a new algorithm that helped remove noise from their data. When each pixel during the scanning process was given 5 milliseconds to collect photons, the new de-noising algorithm reduced the root mean square error—a measure of its deviation from a perfect image—of reconstructed images by about eightfold.

The researchers now plan to arrange multiple sensors into larger arrays to boost efficiency, reduce scanning time, and extend the distance over which imaging can take place, Hu says. They would also like to test their device in daylight conditions, he adds.

The scientists detailed their findings 30 November in the journal Optics Express.

Introduction: Humans and robots are increasingly collaborating on complex tasks such as firefighting. As robots are becoming more autonomous, collaboration in human-robot teams should be combined with meaningful human control. Variable autonomy approaches can ensure meaningful human control over robots by satisfying accountability, responsibility, and transparency. To verify whether variable autonomy approaches truly ensure meaningful human control, the concept should be operationalized to allow its measurement. So far, designers of variable autonomy approaches lack metrics to systematically address meaningful human control.

Methods: Therefore, this qualitative focus group (n = 5 experts) explored quantitative operationalizations of meaningful human control during dynamic task allocation using variable autonomy in human-robot teams for firefighting. This variable autonomy approach requires dynamic allocation of moral decisions to humans and non-moral decisions to robots, using robot identification of moral sensitivity. We analyzed the data of the focus group using reflexive thematic analysis.

Results: Results highlight the usefulness of quantifying the traceability requirement of meaningful human control, and how situation awareness and performance can be used to objectively measure aspects of the traceability requirement. Moreover, results emphasize that team and robot outcomes can be used to verify meaningful human control but that identifying reasons underlying these outcomes determines the level of meaningful human control.

Discussion: Based on our results, we propose an evaluation method that can verify if dynamic task allocation using variable autonomy in human-robot teams for firefighting ensures meaningful human control over the robot. This method involves subjectively and objectively quantifying traceability using human responses during and after simulations of the collaboration. In addition, the method involves semi-structured interviews after the simulation to identify reasons underlying outcomes and suggestions to improve the variable autonomy approach.

To effectively control a robot’s motion, it is common to employ a simplified model that approximates the robot’s dynamics. Nevertheless, discrepancies between the actual mechanical properties of the robot and the simplified model can result in motion failures. To address this issue, this study introduces a pneumatic-driven bipedal musculoskeletal robot designed to closely match the mechanical characteristics of a simplified spring-loaded inverted pendulum (SLIP) model. The SLIP model is widely utilized in robotics due to its passive stability and dynamic properties resembling human walking patterns. A musculoskeletal bipedal robot was designed and manufactured to concentrate its center of mass within a compact body around the hip joint, featuring low leg inertia in accordance with SLIP model principles. Furthermore, we validated that the robot exhibits similar dynamic characteristics to the SLIP model through a sequential jumping experiment and by comparing its performance to SLIP model simulation.

Deep generative models (DGM) are increasingly employed in emergent communication systems. However, their application in multimodal data contexts is limited. This study proposes a novel model that combines multimodal DGM with the Metropolis-Hastings (MH) naming game, enabling two agents to focus jointly on a shared subject and develop common vocabularies. The model proves that it can handle multimodal data, even in cases of missing modalities. Integrating the MH naming game with multimodal variational autoencoders (VAE) allows agents to form perceptual categories and exchange signs within multimodal contexts. Moreover, fine-tuning the weight ratio to favor a modality that the model could learn and categorize more readily improved communication. Our evaluation of three multimodal approaches - mixture-of-experts (MoE), product-of-experts (PoE), and mixture-of-product-of-experts (MoPoE)–suggests an impact on the creation of latent spaces, the internal representations of agents. Our results from experiments with the MNIST + SVHN and Multimodal165 datasets indicate that combining the Gaussian mixture model (GMM), PoE multimodal VAE, and MH naming game substantially improved information sharing, knowledge formation, and data reconstruction.

Exoskeletons that assist in ankle plantarflexion can improve energy economy in locomotion. Characterizing the joint-level mechanisms behind these reductions in energy cost can lead to a better understanding of how people interact with these devices, as well as to improved device design and training protocols. We examined the biomechanical responses to exoskeleton assistance in exoskeleton users trained with a lengthened protocol. Kinematics at unassisted joints were generally unchanged by assistance, which has been observed in other ankle exoskeleton studies. Peak plantarflexion angle increased with plantarflexion assistance, which led to increased total and biological mechanical power despite decreases in biological joint torque and whole-body net metabolic energy cost. Ankle plantarflexor activity also decreased with assistance. Muscles that act about unassisted joints also increased activity for large levels of assistance, and this response should be investigated over long-term use to prevent overuse injuries.

Introduction: Patients who are hospitalized may be at a higher risk for falling, which can result in additional injuries, longer hospitalizations, and extra cost for healthcare organizations. A frequent context for these falls is when a hospitalized patient needs to use the bathroom. While it is possible that “high-tech” tools like robots and AI applications can help, adopting a human-centered approach and engaging users and other affected stakeholders in the design process can help to maximize benefits and avoid unintended consequences.

Methods: Here, we detail our findings from a human-centered design research effort to investigate how the process of toileting a patient can be ameliorated through the application of advanced tools like robots and AI. We engaged healthcare professionals in interviews, focus groups, and a co-creation session in order to recognize common barriers in the toileting process and find opportunities for improvement.

Results: In our conversations with participants, who were primarily nurses, we learned that toileting is more than a nuisance for technology to remove through automation. Nurses seem keenly aware and responsive to the physical and emotional pains experienced by patients during the toileting process, and did not see technology as a feasible or welcomed substitute. Instead, nurses wanted tools which supported them in providing this care to their patients. Participants envisioned tools which helped them anticipate and understand patient toileting assistance needs so they could plan to assist at convenient times during their existing workflows. Participants also expressed favorability towards mechanical assistive features which were incorporated into existing equipment to ensure ubiquitous availability when needed without adding additional mass to an already cramped and awkward environment.

Discussion: We discovered that the act of toileting served more than one function, and can be viewed as a valuable touchpoint in which nurses can assess, support, and encourage their patients to engage in their own recovery process as they perform a necessary and normal function of life. While we found opportunities for technology to make the process safer and less burdensome for patients and clinical staff alike, we believe that designers should preserve and enhance the therapeutic elements of the nurse-patient interaction rather than eliminate it through automation.

Introduction: Communication from automated vehicles (AVs) to pedestrians using augmented reality (AR) could positively contribute to traffic safety. However, previous AR research for pedestrians was mainly conducted through online questionnaires or experiments in virtual environments instead of real ones.

Methods: In this study, 28 participants conducted trials outdoors with an approaching AV and were supported by four different AR interfaces. The AR experience was created by having participants wear a Varjo XR-3 headset with see-through functionality, with the AV and AR elements virtually overlaid onto the real environment. The AR interfaces were vehicle-locked (Planes on vehicle), world-locked (Fixed pedestrian lights, Virtual fence), or head-locked (Pedestrian lights HUD). Participants had to hold down a button when they felt it was safe to cross, and their opinions were obtained through rating scales, interviews, and a questionnaire.

Results: The results showed that participants had a subjective preference for AR interfaces over no AR interface. Furthermore, the Pedestrian lights HUD was more effective than no AR interface in a statistically significant manner, as it led to participants more frequently keeping the button pressed. The Fixed pedestrian lights scored lower than the other interfaces, presumably due to low saliency and the fact that participants had to visually identify both this AR interface and the AV.

Discussion: In conclusion, while users favour AR in AV-pedestrian interactions over no AR, its effectiveness depends on design factors like location, visibility, and visual attention demands. In conclusion, this work provides important insights into the use of AR outdoors. The findings illustrate that, in these circumstances, a clear and easily interpretable AR interface is of key importance.



Citing “no path to regulatory approval in the European Union,” Amazon and iRobot have announced the termination of an acquisition deal first announced in August of 2022 that would have made iRobot a part of Amazon and valued the robotics company at US $1.4 billion.

The European Commission released a statement today that explained some of its concerns, which to be fair, seem like reasonable things to be concerned about:

Our in-depth investigation preliminarily showed that the acquisition of iRobot would have enabled Amazon to foreclose iRobot’s rivals by restricting or degrading access to the Amazon Stores.… We also preliminarily found that Amazon would have had the incentive to foreclose iRobot’s rivals because it would have been economically profitable to do so. All such foreclosure strategies could have restricted competition in the market for robot vacuum cleaners, leading to higher prices, lower quality, and less innovation for consumers.

Amazon, for its part, characterizes this as “undue and disproportionate regulatory hurdles.” Whoever you believe is correct, the protracted strangulation of this acquisition deal has not been great for iRobot, and its termination is potentially disastrous—Amazon will have to pay iRobot a $94 million termination fee, which is basically nothing for it, and meanwhile iRobot is already laying off 350 people, or 31 percent of its head count.

From one of iRobot’s press releases:

“iRobot is an innovation pioneer with a clear vision to make consumer robots a reality,” said Colin Angle, Founder of iRobot. “The termination of the agreement with Amazon is disappointing, but iRobot now turns toward the future with a focus and commitment to continue building thoughtful robots and intelligent home innovations that make life better, and that our customers around the world love.”

The reason that I don’t feel much better after reading that statement is that Colin Angle has already stepped down as chairman and CEO of iRobot. Angle was one of the founders of iRobot (along with Rodney Brooks and Helen Greiner) and has stuck with the company for its entire 30+ year existence, until just now. So, that’s not great. Also, I’m honestly not sure how iRobot is going to create much in the way of home innovations since the press release states that the company is “pausing all work related to non-floor care innovations, including air purification, robotic lawn mowing and education,” while also “reducing R&D expense by approximately $20 million year-over-year.”

iRobot’s lawn mower has been paused for a while now, so it’s not a huge surprise that nothing will move forward there, but a pause on the education robots like Create and Root is a real blow to the robotics community. And even if iRobot is focusing on floor-care innovations, I’m not sure how much innovation will be possible with a slashed R&D budget amidst huge layoffs.

Sigh.

On LinkedIn, Colin Angle wrote a little bit about what he called “the magic of iRobot”:

iRobot built the first micro rovers and changed space exploration forever. iRobot built the first practical robots that left the research lab and went on combat missions to defuse bombs, saving 1000’s of lives. iRobot’s robots crucially enabled the cold shutdown of the reactors at Fukushima, found the underwater pools of oil in the aftermath of the deep horizon oil rig disaster in the Gulf of Mexico. And pioneered an industry with Roomba, fulfilling the unfulfilled promise of over 50 years for practical robots in the home.

Why?

As I think about all the events surrounding those actions, there is a common thread. We believed we could. And we decided to try with a spirit of pragmatic optimism. Building robots means knowing failure. It does not treat blind hope kindly. Robots are too complex. Robots are too expensive. Robots are too challenging for hope alone to have the slightest chance of success. But combining the belief that a problem can be solved with a commitment to the work to solve it enabled us to change the world.

And that’s what I personally find so worrying about all of this. iRobot has a treasured history of innovation which is full of successes and failures and really weird stuff, and it’s hard to see how that will be able to effectively continue. Here are a couple of my favorite weird iRobot things, including a PackBot that flies (for a little bit) and a morphing blobular robot:

I suppose it’s worth pointing out that the weirdest stuff (like in the videos above) is all over a decade old, and you can reasonably ask whether iRobot was that kind of company anymore even before this whole Amazon thing happened. The answer is probably not, since the company has chosen to focus almost exclusively on floor-care robots. But even there we’ve seen consistent innovation in hardware and software that pretty much every floor-care robot company seems to then pick up on about a year later. This is not to say that other floor-care robots can’t innovate, but it’s undeniable that iRobot has been a driving force behind that industry. Will that continue? I really hope so.



Citing “no path to regulatory approval in the European Union,” Amazon and iRobot have announced the termination of an acquisition deal first announced in August of 2022 that would have made iRobot a part of Amazon and valued the robotics company at US $1.4 billion.

The European Commission released a statement today that explained some of its concerns, which to be fair, seem like reasonable things to be concerned about:

Our in-depth investigation preliminarily showed that the acquisition of iRobot would have enabled Amazon to foreclose iRobot’s rivals by restricting or degrading access to the Amazon Stores.… We also preliminarily found that Amazon would have had the incentive to foreclose iRobot’s rivals because it would have been economically profitable to do so. All such foreclosure strategies could have restricted competition in the market for robot vacuum cleaners, leading to higher prices, lower quality, and less innovation for consumers.

Amazon, for its part, characterizes this as “undue and disproportionate regulatory hurdles.” Whoever you believe is correct, the protracted strangulation of this acquisition deal has not been great for iRobot, and its termination is potentially disastrous—Amazon will have to pay iRobot a $94 million termination fee, which is basically nothing for it, and meanwhile iRobot is already laying off 350 people, or 31 percent of its head count.

From one of iRobot’s press releases:

“iRobot is an innovation pioneer with a clear vision to make consumer robots a reality,” said Colin Angle, Founder of iRobot. “The termination of the agreement with Amazon is disappointing, but iRobot now turns toward the future with a focus and commitment to continue building thoughtful robots and intelligent home innovations that make life better, and that our customers around the world love.”

The reason that I don’t feel much better after reading that statement is that Colin Angle has already stepped down as chairman and CEO of iRobot. Angle was one of the founders of iRobot (along with Rodney Brooks and Helen Greiner) and has stuck with the company for its entire 30+ year existence, until just now. So, that’s not great. Also, I’m honestly not sure how iRobot is going to create much in the way of home innovations since the press release states that the company is “pausing all work related to non-floor care innovations, including air purification, robotic lawn mowing and education,” while also “reducing R&D expense by approximately $20 million year-over-year.”

iRobot’s lawn mower has been paused for a while now, so it’s not a huge surprise that nothing will move forward there, but a pause on the education robots like Create and Root is a real blow to the robotics community. And even if iRobot is focusing on floor-care innovations, I’m not sure how much innovation will be possible with a slashed R&D budget amidst huge layoffs.

Sigh.

On LinkedIn, Colin Angle wrote a little bit about what he called “the magic of iRobot”:

iRobot built the first micro rovers and changed space exploration forever. iRobot built the first practical robots that left the research lab and went on combat missions to defuse bombs, saving 1000’s of lives. iRobot’s robots crucially enabled the cold shutdown of the reactors at Fukushima, found the underwater pools of oil in the aftermath of the deep horizon oil rig disaster in the Gulf of Mexico. And pioneered an industry with Roomba, fulfilling the unfulfilled promise of over 50 years for practical robots in the home.

Why?

As I think about all the events surrounding those actions, there is a common thread. We believed we could. And we decided to try with a spirit of pragmatic optimism. Building robots means knowing failure. It does not treat blind hope kindly. Robots are too complex. Robots are too expensive. Robots are too challenging for hope alone to have the slightest chance of success. But combining the belief that a problem can be solved with a commitment to the work to solve it enabled us to change the world.

And that’s what I personally find so worrying about all of this. iRobot has a treasured history of innovation which is full of successes and failures and really weird stuff, and it’s hard to see how that will be able to effectively continue. Here are a couple of my favorite weird iRobot things, including a PackBot that flies (for a little bit) and a morphing blobular robot:

I suppose it’s worth pointing out that the weirdest stuff (like in the videos above) is all over a decade old, and you can reasonably ask whether iRobot was that kind of company anymore even before this whole Amazon thing happened. The answer is probably not, since the company has chosen to focus almost exclusively on floor-care robots. But even there we’ve seen consistent innovation in hardware and software that pretty much every floor-care robot company seems to then pick up on about a year later. This is not to say that other floor-care robots can’t innovate, but it’s undeniable that iRobot has been a driving force behind that industry. Will that continue? I really hope so.

Background: Robots are increasingly used as interaction partners with humans. Social robots are designed to follow expected behavioral norms when engaging with humans and are available with different voices and even accents. Some studies suggest that people prefer robots to speak in the user’s dialect, while others indicate a preference for different dialects.

Methods: Our study examined the impact of the Berlin dialect on perceived trustworthiness and competence of a robot. One hundred and twenty German native speakers (Mage = 32 years, SD = 12 years) watched an online video featuring a NAO robot speaking either in the Berlin dialect or standard German and assessed its trustworthiness and competence.

Results: We found a positive relationship between participants’ self-reported Berlin dialect proficiency and trustworthiness in the dialect-speaking robot. Only when controlled for demographic factors, there was a positive association between participants’ dialect proficiency, dialect performance and their assessment of robot’s competence for the standard German-speaking robot. Participants’ age, gender, length of residency in Berlin, and device used to respond also influenced assessments. Finally, the robot’s competence positively predicted its trustworthiness.

Discussion: Our results inform the design of social robots and emphasize the importance of device control in online experiments.



Video Friday is your weekly selection of awesome robotics videos, collected by your friends at IEEE Spectrum robotics. We also post a weekly calendar of upcoming robotics events for the next few months. Please send us your events for inclusion.

Cybathlon Challenges: 2 February 2024, ZURICHEurobot Open 2024: 8–11 May 2024, LA ROCHE-SUR-YON, FRANCEICRA 2024: 13–17 May 2024, YOKOHAMA, JAPANRoboCup 2024: 17–22 July 2024, EINDHOVEN, NETHERLANDS

Enjoy today’s videos!

Made from beautifully fabricated steel and eight mobile arms, medusai can play percussion and strings with human musicians, dance with human dancers, and move in time to multiple human observers. It uses AI-driven computer vision to know what human observers are doing and responds accordingly through snake gestures, music, and light.

If this seems a little bit unsettling, that’s intentional! The project was designed to explore the concepts of trust and risk in the context of robots, and of using technology to influence emotion.

[ medusai ] via [ Georgia Tech ]

Thanks, Gil!

On 19 April 2021, NASA’s Ingenuity Mars Helicopter made history when it completed the first powered, controlled flight on the Red Planet. It flew for the last time on 18 January 2024.

[ NASA JPL ]

Teleoperation plays a crucial role in enabling robot operations in challenging environments, yet existing limitations in effectiveness and accuracy necessitate the development of innovative strategies for improving teleoperated tasks. The work illustrated in this video introduces a novel approach that utilizes mixed reality and assistive autonomy to enhance the efficiency and precision of humanoid robot teleoperation.

Sometimes all it takes is one good punch, and then you can just collapse.

[ Paper ] via [ IHMC ]

Thanks, Robert!

The new Dusty Robotics FieldPrinter 2 enhances on-site performance and productivity through its compact design and extended capabilities. Building upon the success of the first-generation FieldPrinter, which has printed over 91 million square feet of layout, the FieldPrint Platform incorporates lessons learned from years of experience in the field to deliver an optimized experience for all trades on site.

[ Dusty Robotics ]

Quadrupedal robots have emerged as a cutting-edge platform for assisting humans, finding applications in tasks related to inspection and exploration in remote areas. Nevertheless, their floating base structure renders them susceptible to failure in cluttered environments, where manual recovery by a human operator may not always be feasible. In this study, we propose a robust all-terrain recovery policy to facilitate rapid and secure recovery in cluttered environments.

[ DreamRiser ]

The work that Henry Evans is doing with Stretch (along with Hello Robot and Maya Cakmak’s lab at the University of Washington) will be presented at Humanoids this spring.

[ UW HCRL ]

Thanks, Stefan!

I like to imagine that these are just excerpts from one very long walk that Digit took around San Francisco.

[ Hybrid Robotics Lab ]

Boxing, drumming, stacking boxes, and various other practices...those are the daily teleoperation testing of our humanoid robot. Collaborating with engineers, our humanoid robots collect real-world data from teleoperation for learning to iterate control algorithms.

[ LimX Dynamics ]

The OpenDR project aims to develop a versatile and open tool kit for fundamental robot functions, using deep learning to enhance their understanding and decision-making abilities. The primary objective is to make robots more intelligent, particularly in critical areas like health care, agriculture, and production. In the health care setting, the TIAGo robot is deployed to offer assistance and support within a health care facility.

[ OpenDR ] via [ PAL Robotics ]

[ ARCHES ]

Christoph Bartneck gives a talk entitled “Social robots: The end of the beginning or the beginning of the end?”

[ Christoph Bartneck ]

Professor Michael Jordan offers his provocative thoughts on the blending of AI and economics and takes us on a tour of Trieste, a beautiful and grand city in northern Italy.

[ Berkeley ]

Pages