Frontiers in Robotics and AI

RSS Feed for Frontiers in Robotics and AI | New and Recent Articles
Subscribe to Frontiers in Robotics and AI feed

The Internet of Things (IoT) and Industrial IoT (IIoT) have developed rapidly in the past few years, as both the Internet and “things” have evolved significantly. “Things” now range from simple Radio Frequency Identification (RFID) devices to smart wireless sensors, intelligent wireless sensors and actuators, robotic things, and autonomous vehicles operating in consumer, business, and industrial environments. The emergence of “intelligent things” (static or mobile) in collaborative autonomous fleets requires new architectures, connectivity paradigms, trustworthiness frameworks, and platforms for the integration of applications across different business and industrial domains. These new applications accelerate the development of autonomous system design paradigms and the proliferation of the Internet of Robotic Things (IoRT). In IoRT, collaborative robotic things can communicate with other things, learn autonomously, interact safely with the environment, humans and other things, and gain qualities like self-maintenance, self-awareness, self-healing, and fail-operational behavior. IoRT applications can make use of the individual, collaborative, and collective intelligence of robotic things, as well as information from the infrastructure and operating context to plan, implement and accomplish tasks under different environmental conditions and uncertainties. The continuous, real-time interaction with the environment makes perception, location, communication, cognition, computation, connectivity, propulsion, and integration of federated IoRT and digital platforms important components of new-generation IoRT applications. This paper reviews the taxonomy of the IoRT, emphasizing the IoRT intelligent connectivity, architectures, interoperability, and trustworthiness framework, and surveys the technologies that enable the application of the IoRT across different domains to perform missions more efficiently, productively, and completely. The aim is to provide a novel perspective on the IoRT that involves communication among robotic things and humans and highlights the convergence of several technologies and interactions between different taxonomies used in the literature.

This paper describes a portable, prosthetic control system and the first at-home use of a multi-degree-of-freedom, proportionally controlled bionic arm. The system uses a modified Kalman filter to provide 6 degree-of-freedom, real-time, proportional control. We describe (a) how the system trains motor control algorithms for use with an advanced bionic arm, and (b) the system's ability to record an unprecedented and comprehensive dataset of EMG, hand positions and force sensor values. Intact participants and a transradial amputee used the system to perform activities-of-daily-living, including bi-manual tasks, in the lab and at home. This technology enables at-home dexterous bionic arm use, and provides a high-temporal resolution description of daily use—essential information to determine clinical relevance and improve future research for advanced bionic arms.

It has been 10 years since the publication of the first article looking at plants as a biomechatronic system and as model for robotics. Now, roboticists have started to look at plants differently and consider them as a model in the field of bioinspired robotics. Despite plants have been seen traditionally as passive entities, in reality they are able to grow, move, sense, and communicate. These features make plants an exceptional example of morphological computation - with probably the highest level of adaptability among all living beings. They are a unique model to design robots that can act in- and adapt to- unstructured, extreme, and dynamically changing environments exposed to sudden or long-term events. Although plant-inspired robotics is still a relatively new field, it has triggered the concept of growing robotics: an emerging area in which systems are designed to create their own body, adapt their morphology, and explore different environments. There is a reciprocal interest between biology and robotics: plants represent an excellent source of inspiration for achieving new robotic abilities, and engineering tools can be used to reveal new biological information. This way, a bidirectional biology-robotics strategy provides mutual benefits for both disciplines. This mini-review offers a brief overview of the fundamental aspects related to a bioengineering approach in plant-inspired robotics. It analyses the works in which both biological and engineering aspects have been investigated, and highlights the key elements of plants that have been milestones in the pioneering field of growing robots.

The ability to learn new tasks by sequencing already known skills is an important requirement for future robots. Reinforcement learning is a powerful tool for this as it allows for a robot to learn and improve on how to combine skills for sequential tasks. However, in real robotic applications, the cost of sample collection and exploration prevent the application of reinforcement learning for a variety of tasks. To overcome these limitations, human input during reinforcement can be beneficial to speed up learning, guide the exploration and prevent the choice of disastrous actions. Nevertheless, there is a lack of experimental evaluations of multi-channel interactive reinforcement learning systems solving robotic tasks with input from inexperienced human users, in particular for cases where human input might be partially wrong. Therefore, in this paper, we present an approach that incorporates multiple human input channels for interactive reinforcement learning in a unified framework and evaluate it on two robotic tasks with 20 inexperienced human subjects. To enable the robot to also handle potentially incorrect human input we incorporate a novel concept for self-confidence, which allows the robot to question human input after an initial learning phase. The second robotic task is specifically designed to investigate if this self-confidence can enable the robot to achieve learning progress even if the human input is partially incorrect. Further, we evaluate how humans react to suggestions of the robot, once the robot notices human input might be wrong. Our experimental evaluations show that our approach can successfully incorporate human input to accelerate the learning process in both robotic tasks even if it is partially wrong. However, not all humans were willing to accept the robot's suggestions or its questioning of their input, particularly if they do not understand the learning process and the reasons behind the robot's suggestions. We believe that the findings from this experimental evaluation can be beneficial for the future design of algorithms and interfaces of interactive reinforcement learning systems used by inexperienced users.

Soft grippers with soft and flexible materials have been widely researched to improve the functionality of grasping. Although grippers that can grasp various objects with different shapes are important, a large number of industrial applications require a gripper that is targeted for a specified object. In this paper, we propose a design methodology for soft grippers that are customized to grasp single dedicated objects. A customized soft gripper can safely and efficiently grasp a dedicated target object with lowered surface contact forces while maintaining a higher lifting force, compared to its non-customized counterpart. A simplified analytical model and a fabrication method that can rapidly customize and fabricate soft grippers are proposed. Stiffness patterns were implemented onto the constraint layers of pneumatic bending actuators to establish actuated postures with irregular bending curvatures in the longitudinal direction. Soft grippers with customized stiffness patterns yielded higher shape conformability to target objects than non-patterned regular soft grippers. The simplified analytical model represents the pneumatically actuated soft finger as a summation of interactions between its air chambers. Geometric approximations and pseudo-rigid-body modeling theory were employed to build the analytical model. The customized soft grippers were compared with non-patterned soft grippers by measuring their lifting forces and contact forces while they grasped objects. Under the identical actuating pressure, the conformable grasping postures enabled customized soft grippers to have almost three times the lifting force than that of non-patterned soft grippers, while the maximum contact force was reduced to two thirds.

Manipulation of deformable objects has given rise to an important set of open problems in the field of robotics. Application areas include robotic surgery, household robotics, manufacturing, logistics, and agriculture, to name a few. Related research problems span modeling and estimation of an object's shape, estimation of an object's material properties, such as elasticity and plasticity, object tracking and state estimation during manipulation, and manipulation planning and control. In this survey article, we start by providing a tutorial on foundational aspects of models of shape and shape dynamics. We then use this as the basis for a review of existing work on learning and estimation of these models and on motion planning and control to achieve desired deformations. We also discuss potential future lines of work.

The study of sustainability challenges requires the consideration of multiple coupled systems that are often complex and deeply uncertain. As a result, traditional analytical methods offer limited insights with respect to how to best address such challenges. By analyzing the case of global climate change mitigation, this paper shows that the combination of high-performance computing, mathematical modeling, and computational intelligence tools, such as optimization and clustering algorithms, leads to richer analytical insights. The paper concludes by proposing an analytical hierarchy of computational tools that can be applied to other sustainability challenges.

Automatic fingerprint identification systems (AFIS) make use of global fingerprint information like ridge flow, ridge frequency, and delta or core points for fingerprint alignment, before performing matching. In latent fingerprints, the ridges will be smudged and delta or core points may not be available. It becomes difficult to pre-align fingerprints with such partial fingerprint information. Further, global features are not robust against fingerprint deformations; rotation, scale, and fingerprint matching using global features pose more challenges. We have developed a local minutia-based convolution neural network (CNN) matching model called “Combination of Nearest Neighbor Arrangement Indexing (CNNAI).” This model makes use of a set of “n” local nearest minutiae neighbor features and generates rotation-scale invariant feature vectors. Our proposed system doesn't depend upon any fingerprint alignment information. In large fingerprint databases, it becomes very difficult to query every fingerprint against every other fingerprint in the database. To address this issue, we make use of hash indexing to reduce the number of retrievals. We have used a residual learning-based CNN model to enhance and extract the minutiae features. Matching was done on FVC2004 and NIST SD27 latent fingerprint databases against 640 and 3,758 gallery fingerprint images, respectively. We obtained a Rank-1 identification rate of 80% for FVC2004 fingerprints and 84.5% for NIST SD27 latent fingerprint databases. The experimental results show improvement in the Rank-1 identification rate compared to the state-of-art algorithms, and the results reveal that the system is robust against rotation and scale.

This article presents a method for grasping novel objects by learning from experience. Successful attempts are remembered and then used to guide future grasps such that more reliable grasping is achieved over time. To transfer the learned experience to unseen objects, we introduce the dense geometric correspondence matching network (DGCM-Net). This applies metric learning to encode objects with similar geometry nearby in feature space. Retrieving relevant experience for an unseen object is thus a nearest neighbor search with the encoded feature maps. DGCM-Net also reconstructs 3D-3D correspondences using the view-dependent normalized object coordinate space to transform grasp configurations from retrieved samples to unseen objects. In comparison to baseline methods, our approach achieves an equivalent grasp success rate. However, the baselines are significantly improved when fusing the knowledge from experience with their grasp proposal strategy. Offline experiments with a grasping dataset highlight the capability to transfer grasps to new instances as well as to improve success rate over time from increasing experience. Lastly, by learning task-relevant grasps, our approach can prioritize grasp configurations that enable the functional use of objects.

We describe and evaluate a neural network-based architecture aimed to imitate and improve the performance of a fully autonomous soccer team in RoboCup Soccer 2D Simulation environment. The approach utilizes deep Q-network architecture for action determination and a deep neural network for parameter learning. The proposed solution is shown to be feasible for replacing a selected behavioral module in a well-established RoboCup base team, Gliders2d, in which behavioral modules have been evolved with human experts in the loop. Furthermore, we introduce an additional performance-correlated signal (a delayed reward signal), enabling a search for local maxima during a training phase. The extension is compared against a known benchmark. Finally, we investigate the extent to which preserving the structure of expert-designed behaviors affects the performance of a neural network-based solution.

Continuously measuring the engagement of users with a robot in a Human-Robot Interaction (HRI) setting paves the way toward in-situ reinforcement learning, improve metrics of interaction quality, and can guide interaction design and behavior optimization. However, engagement is often considered very multi-faceted and difficult to capture in a workable and generic computational model that can serve as an overall measure of engagement. Building upon the intuitive ways humans successfully can assess situation for a degree of engagement when they see it, we propose a novel regression model (utilizing CNN and LSTM networks) enabling robots to compute a single scalar engagement during interactions with humans from standard video streams, obtained from the point of view of an interacting robot. The model is based on a long-term dataset from an autonomous tour guide robot deployed in a public museum, with continuous annotation of a numeric engagement assessment by three independent coders. We show that this model not only can predict engagement very well in our own application domain but show its successful transfer to an entirely different dataset (with different tasks, environment, camera, robot and people). The trained model and the software is available to the HRI community, at, as a tool to measure engagement in a variety of settings.

Pervasive sensing is increasing our ability to monitor the status of patients not only when they are hospitalized but also during home recovery. As a result, lots of data are collected and are available for multiple purposes. If operations can take advantage of timely and detailed data, the huge amount of data collected can also be useful for analytics. However, these data may be unusable for two reasons: data quality and performance problems. First, if the quality of the collected values is low, the processing activities could produce insignificant results. Second, if the system does not guarantee adequate performance, the results may not be delivered at the right time. The goal of this document is to propose a data utility model that considers the impact of the quality of the data sources (e.g., collected data, biographical data, and clinical history) on the expected results and allows for improvement of the performance through utility-driven data management in a Fog environment. Regarding data quality, our approach aims to consider it as a context-dependent problem: a given dataset can be considered useful for one application and inadequate for another application. For this reason, we suggest a context-dependent quality assessment considering dimensions such as accuracy, completeness, consistency, and timeliness, and we argue that different applications have different quality requirements to consider. The management of data in Fog computing also requires particular attention to quality of service requirements. For this reason, we include QoS aspects in the data utility model, such as availability, response time, and latency. Based on the proposed data utility model, we present an approach based on a goal model capable of identifying when one or more dimensions of quality of service or data quality are violated and of suggesting which is the best action to be taken to address this violation. The proposed approach is evaluated with a real and appropriately anonymized dataset, obtained as part of the experimental procedure of a research project in which a device with a set of sensors (inertial, temperature, humidity, and light sensors) is used to collect motion and environmental data associated with the daily physical activities of healthy young volunteers.

Human beings can achieve a high level of motor performance that is still unmatched in robotic systems. These capabilities can be ascribed to two main enabling factors: (i) the physical proprieties of human musculoskeletal system, and (ii) the effectiveness of the control operated by the central nervous system. Regarding point (i), the introduction of compliant elements in the robotic structure can be regarded as an attempt to bridge the gap between the animal body and the robot one. Soft articulated robots aim at replicating the musculoskeletal characteristics of vertebrates. Yet, substantial advancements are still needed under a control point of view, to fully exploit the new possibilities provided by soft robotic bodies. This paper introduces a control framework that ensures natural movements in articulated soft robots, implementing specific functionalities of the human central nervous system, i.e., learning by repetition, after-effect on known and unknown trajectories, anticipatory behavior, its reactive re-planning, and state covariation in precise task execution. The control architecture we propose has a hierarchical structure composed of two levels. The low level deals with dynamic inversion and focuses on trajectory tracking problems. The high level manages the degree of freedom redundancy, and it allows to control the system through a reduced set of variables. The building blocks of this novel control architecture are well-rooted in the control theory, which can furnish an established vocabulary to describe the functional mechanisms underlying the motor control system. The proposed control architecture is validated through simulations and experiments on a bio-mimetic articulated soft robot.

Lower extremity exoskeletons offer the potential to restore ambulation to individuals with paraplegia due to spinal cord injury. However, they often rely on preprogrammed gait, initiated by switches, sensors, and/or EEG triggers. Users can exercise only limited independent control over the trajectory of the feet, the speed of walking, and the placement of feet to avoid obstacles. In this paper, we introduce and evaluate a novel approach that naturally decodes a neuromuscular surrogate for a user's neutrally planned foot control, uses the exoskeleton's motors to move the user's legs in real-time, and provides sensory feedback to the user allowing real-time sensation and path correction resulting in gait similar to biological ambulation. Users express their desired gait by applying Cartesian forces via their hands to rigid trekking poles that are connected to the exoskeleton feet through multi-axis force sensors. Using admittance control, the forces applied by the hands are converted into desired foot positions, every 10 milliseconds (ms), to which the exoskeleton is moved by its motors. As the trekking poles reflect the resulting foot movement, users receive sensory feedback of foot kinematics and ground contact that allows on-the-fly force corrections to maintain the desired foot behavior. We present preliminary results showing that our novel control can allow users to produce biologically similar exoskeleton gait.

Computer Tomography (CT) is an imaging procedure that combines many X-ray measurements taken from different angles. The segmentation of areas in the CT images provides a valuable aid to physicians and radiologists in order to better provide a patient diagnose. The CT scans of a body torso usually include different neighboring internal body organs. Deep learning has become the state-of-the-art in medical image segmentation. For such techniques, in order to perform a successful segmentation, it is of great importance that the network learns to focus on the organ of interest and surrounding structures and also that the network can detect target regions of different sizes. In this paper, we propose the extension of a popular deep learning methodology, Convolutional Neural Networks (CNN), by including deep supervision and attention gates. Our experimental evaluation shows that the inclusion of attention and deep supervision results in consistent improvement of the tumor prediction accuracy across the different datasets and training sizes while adding minimal computational overhead.

Muscle models and animal observations suggest that physical damping is beneficial for stabilization. Still, only a few implementations of physical damping exist in compliant robotic legged locomotion. It remains unclear how physical damping can be exploited for locomotion tasks, while its advantages as sensor-free, adaptive force- and negative work-producing actuators are promising. In a simplified numerical leg model, we studied the energy dissipation from viscous and Coulomb damping during vertical drops with ground-level perturbations. A parallel spring- damper is engaged between touch-down and mid-stance, and its damper auto-decouples from mid-stance to takeoff. Our simulations indicate that an adjustable and viscous damper is desired. In hardware we explored effective viscous damping and adjustability, and quantified the dissipated energy. We tested two mechanical, leg-mounted damping mechanisms: a commercial hydraulic damper, and a custom-made pneumatic damper. The pneumatic damper exploits a rolling diaphragm with an adjustable orifice, minimizing Coulomb damping effects while permitting adjustable resistance. Experimental results show that the leg-mounted, hydraulic damper exhibits the most effective viscous damping. Adjusting the orifice setting did not result in substantial changes of dissipated energy per drop, unlike adjusting the damping parameters in the numerical model. Consequently, we also emphasize the importance of characterizing physical dampers during real legged impacts to evaluate their effectiveness for compliant legged locomotion.

Applications in remote inspection and medicine have motivated the recent development of innovative thin, flexible-backboned robots. However, such robots often experience difficulties in maintaining their intended posture under gravitational and other external loadings. Thin-stemmed climbing plants face many of the same problems. One highly effective solution adopted by such plants features the use of tendrils and tendril-like structures, or the intertwining of several individual stems to form braid-like structures. In this paper, we present new plant-inspired robotic tendril-bearing and intertwining stem hardware and corresponding novel attachment strategies for thin continuum robots. These contributions to robotics are motivated by new insights into plant tendril and intertwining mechanics and behavior. The practical applications of the resulting GrowBots is discussed in the context of space exploration and mining operations.

Visual reasoning is a critical stage in visual question answering (Antol et al., 2015), but most of the state-of-the-art methods categorized the VQA tasks as a classification problem without taking the reasoning process into account. Various approaches are proposed to solve this multi-modal task that requires both abilities of comprehension and reasoning. The recently proposed neural module network (Andreas et al., 2016b), which assembles the model with a few primitive modules, is capable of performing a spatial or arithmetical reasoning over the input image to answer the questions. Nevertheless, its performance is not satisfying especially in the real-world datasets (e.g., VQA 1.0& 2.0) due to its limited primitive modules and suboptimal layout. To address these issues, we propose a novel method of Dual-Path Neural Module Network which can implement complex visual reasoning by forming a more flexible layout regularized by the pairwise loss. Specifically, we first use the region proposal network to generate both visual and spatial information, which helps it perform spatial reasoning. Then, we advocate to process a pair of different images along with the same question simultaneously, named as a “complementary pair,” which encourages the model to learn a more reasonable layout by suppressing the overfitting to the language priors. The model can jointly learn the parameters in the primitive module and the layout generation policy, which is further boosted by introducing a novel pairwise reward. Extensive experiments show that our approach significantly improves the performance of neural module networks especially on the real-world datasets.

On the eve of Human-Robot-Interaction (HRI) becoming customary in our lives, the performance of HRI robotic devices remains strongly conditioned by their gearboxes. In most industrial robots, two relatively unconventional transmission technologies—Harmonic Drives© and Cycloid Drives—are usually found, which are not so broadly used in other industries. Understanding the origin of this singularity provides valuable insights in the search for suitable, future robotic transmission technologies. In this paper we propose an assessment framework strongly conditioned by HRI applications, and we use it to review the performance of conventional and emerging robotic gearbox technologies, for which the design criterion is strongly shifted toward aspects like weight and efficiency. The framework proposes to use virtual power as a suitable way to assess the inherent limitations of a gearbox technologies to achieve high efficiencies. This paper complements the existing research dealing with the complex interaction between gearbox technologies and the actuators, with a new gearbox-centered perspective particularly focused on HRI applications.