Feed aggregator

This article was originally published on LinkedIn. The views expressed here are solely those of the author and do not represent positions of IEEE Spectrum or the IEEE.

Build a rover, send it to the Moon, sell the movie rights.

That was our first business model at iRobot. Way back in 1990. We thought it would be how we’d first change the world. It’s ironic, of course, that through that model, changing the world meant sending a robot to another one. Sadly, that business model failed. And it wouldn’t be our last failed business model. Not by a long shot.

Photo: iRobot

Why? Because changing the world through robots, it turns out, is no easy task.

Perhaps the biggest challenge back when we started in 1990 was that there existed no rule book on how to do it. There weren’t many robots, let alone robot companies, let alone any kind of robot industry. We would have to build it. All of it.

Walking that path meant being comfortable with ambiguity, and comfortable with the knowledge that not everything we tried was going to work–at least not in the way we originally conceived. It was and continues to be the cost of inventing the future.

But walking that trying path also meant learning from our mistakes, dusting ourselves off, trying again, and eventually, yes, doing what we set out to do: Change the world through robots.

We’ve learned so much along the way–what have we learned in our 30-year journey building the robot industry?

Robots are hard

I’ve said it before, and I’ll say it again: When we first started iRobot we had to invent every element of the robot. Spatial navigation was a robot problem, voice recognition was a robot problem, machine vision was a robot problem, just to name a few. Back then, no one else had set out to solve these hard problems. Because so many of these problems existed, the robot industry, if it could be called that, moved in anti-dog years. Fortunately, times have changed and the ecosystem around the technologies that make robots possible is much richer… But back then… it was just us.

But even today, with a much larger ecosystem of bright minds solving for the hard tech problems, getting a robot to work successfully still means getting just the right mix of mechanical, electrical, and software engineering, connectivity, and data science into a robot form factor that people trust and want to invite into their home.

Photo: iRobot

Speaking of trust, therein lied another challenge. Even when we did invent a robot that worked extraordinarily well–Roomba–consumers simply didn’t believe a robot could do what we said Roomba was capable of. It turns out that the principal objection to purchasing a robot for much of the last 30 years is a lack of belief that it could possibly work.

For a long time the robot industry was unfundable. Why? Because no robot company had a business model worth funding. We were no exception: We tried 14 business models before we arrived at one that sustainably worked.

But that’s not all: Even when you build a robot right, you can still somehow build it wrong. We experienced this with Roomba. We built it to match the reliability standards of European upright vacuums, something of which we were very proud. Of course, we didn’t anticipate that our customers would run their Roomba once per day, rather than the once per week average the European standard set. And as the first generation of Roomba robots broke down two years ahead of schedule, we learned that reverse logistics, great customer service, and a generous return policy were a very important part of a good robot–as was the realization that we couldn’t compare usage to whatever traditional means of action a good robot might take the place of.

And yet while building a robot that was durable, that people wanted and trusted was hard enough, 30 years building robots has also taught us that…

Good business models are harder to build than good robots

Let’s state this one right off the bat: For a long time the robot industry was unfundable. Why? Because no robot company had a business model worth funding. It turns out that a business model is as important as the tech, but much more rarely found in a robot company. And for a long time we were no exception: We tried 14 business models before we arrived at one that sustainably worked.

Image: iRobot

But the tenuous nature of our business models did teach us the value of extending the runway for our business until we found one that worked. And how does one extend the runway most effectively? By managing risk.

It’s one of the great misunderstandings of entrepreneurship–that great entrepreneurs are risk takers. Great entrepreneurs are not great risk takers… they’re great risk managers. And this was something we at iRobot were and are exceptionally good at.

How did we manage risk early on? Through partnerships. The kind of partnership we looked for were ones in which there was a big company–one that had a lot of money, a channel to the marketplace, and knowledge of that marketplace, but for whatever reason lacked belief that they themselves were innovative. We were a small company with no money, but believed ourselves to have cool technology, and be highly capable of innovation.

Image: iRobot

What we’d do was give our partner, the big company, absolute control. By doing this, it allowed us to say that since they could cancel the partnership at any time, we needed them to cover our costs… which they did. But we also didn’t ask them to pay us profit upfront. By not having the pay profit upfront, it makes obvious that we’re sharing the value that the partnership would ultimately create, and in a worst-case scenario for our partner, if the partnership didn’t result in a successful product, they got very inexpensive high-quality research.

This “asymmetric strategic partnership” approach not only provided the funds needed to sustain our business when we didn’t have a sustainable business model–the “failure” of those partnerships actually led to our ultimate success. Why? Because…

Innovation and failure come hand-in-hand

While this is far from a groundbreaking realization, its applicability to iRobot is quite unique. Because for us to become successful, it turns out that we had to learn the lessons from failing to earn royalties on robot toys (business model #3), failing to license technology for industrial floor-cleaning robots (business model #8), and failing to sell land mine clearance robots (business model #11).

Image: iRobot

Why? Because #3 taught us to manufacture at scale, #8 taught us how to clean floors, and #11 taught us how to navigate and cover large spaces.  All of which gave us the knowledge and capability to build… Roomba.

Image: iRobot Yes, you can change the world through robots

We did. In more ways the one. We changed the world by eliminating the need for people to vacuum the house themselves. By IPOing, we showed that a robotics company could be successful–which gave investors more reason to put money into robotics companies around the world.

But perhaps the most important way we’ve succeeded in changing the world is by making robots a daily reality for it. And how do we know that robots are now a reality? Because for the better part of the first 30 years of iRobot, what people said to me about robots–and Roomba specifically–was, “I can’t believe it actually works.”

But now, the question they ask me is, “Why can’t robots do more?

It is a great question. And that is what the next 30 years of iRobot will be about.

Colin Angle is chairman of the board, chief executive officer, and founder of iRobot. Celebrating its 30th year, iRobot has grown from an MIT startup to become a global leader in consumer robots, with more than 30 million sold worldwide. You can follow him on Twitter at @ColinAngle.

This article was originally published on LinkedIn. The views expressed here are solely those of the author and do not represent positions of IEEE Spectrum or the IEEE.

Build a rover, send it to the Moon, sell the movie rights.

That was our first business model at iRobot. Way back in 1990. We thought it would be how we’d first change the world. It’s ironic, of course, that through that model, changing the world meant sending a robot to another one. Sadly, that business model failed. And it wouldn’t be our last failed business model. Not by a long shot.

Photo: iRobot

Why? Because changing the world through robots, it turns out, is no easy task.

Perhaps the biggest challenge back when we started in 1990 was that there existed no rule book on how to do it. There weren’t many robots, let alone robot companies, let alone any kind of robot industry. We would have to build it. All of it.

Walking that path meant being comfortable with ambiguity, and comfortable with the knowledge that not everything we tried was going to work–at least not in the way we originally conceived. It was and continues to be the cost of inventing the future.

But walking that trying path also meant learning from our mistakes, dusting ourselves off, trying again, and eventually, yes, doing what we set out to do: Change the world through robots.

We’ve learned so much along the way–what have we learned in our 30-year journey building the robot industry?

Robots are hard

I’ve said it before, and I’ll say it again: When we first started iRobot we had to invent every element of the robot. Spatial navigation was a robot problem, voice recognition was a robot problem, machine vision was a robot problem, just to name a few. Back then, no one else had set out to solve these hard problems. Because so many of these problems existed, the robot industry, if it could be called that, moved in anti-dog years. Fortunately, times have changed and the ecosystem around the technologies that make robots possible is much richer… But back then… it was just us.

But even today, with a much larger ecosystem of bright minds solving for the hard tech problems, getting a robot to work successfully still means getting just the right mix of mechanical, electrical, and software engineering, connectivity, and data science into a robot form factor that people trust and want to invite into their home.

Photo: iRobot

Speaking of trust, therein lied another challenge. Even when we did invent a robot that worked extraordinarily well–Roomba–consumers simply didn’t believe a robot could do what we said Roomba was capable of. It turns out that the principal objection to purchasing a robot for much of the last 30 years is a lack of belief that it could possibly work.

For a long time the robot industry was unfundable. Why? Because no robot company had a business model worth funding. We were no exception: We tried 14 business models before we arrived at one that sustainably worked.

But that’s not all: Even when you build a robot right, you can still somehow build it wrong. We experienced this with Roomba. We built it to match the reliability standards of European upright vacuums, something of which we were very proud. Of course, we didn’t anticipate that our customers would run their Roomba once per day, rather than the once per week average the European standard set. And as the first generation of Roomba robots broke down two years ahead of schedule, we learned that reverse logistics, great customer service, and a generous return policy were a very important part of a good robot–as was the realization that we couldn’t compare usage to whatever traditional means of action a good robot might take the place of.

And yet while building a robot that was durable, that people wanted and trusted was hard enough, 30 years building robots has also taught us that…

Good business models are harder to build than good robots

Let’s state this one right off the bat: For a long time the robot industry was unfundable. Why? Because no robot company had a business model worth funding. It turns out that a business model is as important as the tech, but much more rarely found in a robot company. And for a long time we were no exception: We tried 14 business models before we arrived at one that sustainably worked.

Image: iRobot

But the tenuous nature of our business models did teach us the value of extending the runway for our business until we found one that worked. And how does one extend the runway most effectively? By managing risk.

It’s one of the great misunderstandings of entrepreneurship–that great entrepreneurs are risk takers. Great entrepreneurs are not great risk takers… they’re great risk managers. And this was something we at iRobot were and are exceptionally good at.

How did we manage risk early on? Through partnerships. The kind of partnership we looked for were ones in which there was a big company–one that had a lot of money, a channel to the marketplace, and knowledge of that marketplace, but for whatever reason lacked belief that they themselves were innovative. We were a small company with no money, but believed ourselves to have cool technology, and be highly capable of innovation.

Image: iRobot

What we’d do was give our partner, the big company, absolute control. By doing this, it allowed us to say that since they could cancel the partnership at any time, we needed them to cover our costs… which they did. But we also didn’t ask them to pay us profit upfront. By not having the pay profit upfront, it makes obvious that we’re sharing the value that the partnership would ultimately create, and in a worst-case scenario for our partner, if the partnership didn’t result in a successful product, they got very inexpensive high-quality research.

This “asymmetric strategic partnership” approach not only provided the funds needed to sustain our business when we didn’t have a sustainable business model–the “failure” of those partnerships actually led to our ultimate success. Why? Because…

Innovation and failure come hand-in-hand

While this is far from a groundbreaking realization, its applicability to iRobot is quite unique. Because for us to become successful, it turns out that we had to learn the lessons from failing to earn royalties on robot toys (business model #3), failing to license technology for industrial floor-cleaning robots (business model #8), and failing to sell land mine clearance robots (business model #11).

Image: iRobot

Why? Because #3 taught us to manufacture at scale, #8 taught us how to clean floors, and #11 taught us how to navigate and cover large spaces.  All of which gave us the knowledge and capability to build… Roomba.

Image: iRobot Yes, you can change the world through robots

We did. In more ways the one. We changed the world by eliminating the need for people to vacuum the house themselves. By IPOing, we showed that a robotics company could be successful–which gave investors more reason to put money into robotics companies around the world.

But perhaps the most important way we’ve succeeded in changing the world is by making robots a daily reality for it. And how do we know that robots are now a reality? Because for the better part of the first 30 years of iRobot, what people said to me about robots–and Roomba specifically–was, “I can’t believe it actually works.”

But now, the question they ask me is, “Why can’t robots do more?

It is a great question. And that is what the next 30 years of iRobot will be about.

Colin Angle is chairman of the board, chief executive officer, and founder of iRobot. Celebrating its 30th year, iRobot has grown from an MIT startup to become a global leader in consumer robots, with more than 30 million sold worldwide. You can follow him on Twitter at @ColinAngle.

This work presents a review and discussion of the challenges that must be solved in order to successfully develop swarms of Micro Air Vehicles (MAVs) for real world operations. From the discussion, we extract constraints and links that relate the local level MAV capabilities to the global operations of the swarm. These should be taken into account when designing swarm behaviors in order to maximize the utility of the group. At the lowest level, each MAV should operate safely. Robustness is often hailed as a pillar of swarm robotics, and a minimum level of local reliability is needed for it to propagate to the global level. An MAV must be capable of autonomous navigation within an environment with sufficient trustworthiness before the system can be scaled up. Once the operations of the single MAV are sufficiently secured for a task, the subsequent challenge is to allow the MAVs to sense one another within a neighborhood of interest. Relative localization of neighbors is a fundamental part of self-organizing robotic systems, enabling behaviors ranging from basic relative collision avoidance to higher level coordination. This ability, at times taken for granted, also must be sufficiently reliable. Moreover, herein lies a constraint: the design choice of the relative localization sensor has a direct link to the behaviors that the swarm can (and should) perform. Vision-based systems, for instance, force MAVs to fly within the field of view of their camera. Range or communication-based solutions, alternatively, provide omni-directional relative localization, yet can be victim to unobservable conditions under certain flight behaviors, such as parallel flight, and require constant relative excitation. At the swarm level, the final outcome is thus intrinsically influenced by the on-board abilities and sensors of the individual. The real-world behavior and operations of an MAV swarm intrinsically follow in a bottom-up fashion as a result of the local level limitations in cognition, relative knowledge, communication, power, and safety. Taking these local limitations into account when designing a global swarm behavior is key in order to take full advantage of the system, enabling local limitations to become true strengths of the swarm.

Innovating on the design and function of the chemical bench remains a quintessential challenge of the ages. It requires a deep understanding of the important role chemistry plays in scientific discovery as well a first principles approach to addressing the gaps in how work gets done at the bench. This perspective examines how one might explore designing and creating a sustainable new standard for advancing automated chemistry bench itself. We propose how this might be done by leveraging recent advances in laboratory automation whereby integrating the latest synthetic, analytical and information technologies, and AI/ML algorithms within a standardized framework, maximizes the value of the data generated and the broader utility of such systems. Although the context of this perspective focuses on the design of advancing molecule of potential therapeutic value, it would not be a stretch to contemplate how such systems could be applied to other applied disciplines like advanced materials, foodstuffs, or agricultural product development.

This piece was written as part of the Artificial Intelligence and International Stability Project at the Center for a New American Security, an independent, nonprofit organization based in Washington, D.C. Funded by Carnegie Corporation of New York, the project promotes thinking and analysis on AI and international stability. Given the likely importance that advances in artificial intelligence could play in shaping our future, it is critical to begin a discussion about ways to take advantage of the benefits of AI and autonomous systems, while mitigating the risks. The views expressed here are solely those of the author and do not represent positions of IEEE Spectrum or the IEEE.

In artificial intelligence circles, we hear a lot about adversarial attacks, especially ones that attempt to “deceive” an AI into believing, or to be more accurate, classifying, something incorrectly. Self-driving cars being fooled into “thinking” stop signs are speed limit signs, pandas being identified as gibbons, or even having your favorite voice assistant be fooled by inaudible acoustic commands—these are examples that populate the narrative around AI deception. One can also point to using AI to manipulate the perceptions and beliefs of a person through “deepfakes” in video, audio, and images. Major AI conferences are more frequently addressing the subject of AI deception too. And yet, much of the literature and work around this topic is about how to fool AI and how we can defend against it through detection mechanisms.

I’d like to draw our attention to a different and more unique problem: Understanding the breadth of what “AI deception” looks like, and what happens when it is not a human’s intent behind a deceptive AI, but instead the AI agent’s own learned behavior. These may seem somewhat far-off concerns, as AI is still relatively narrow in scope and can be rather stupid in some ways. To have some analogue of an “intent” to deceive would be a large step for today’s systems. However, if we are to get ahead of the curve regarding AI deception, we need to have a robust understanding of all the ways AI could deceive. We require some conceptual framework or spectrum of the kinds of deception an AI agent may learn on its own before we can start proposing technological defenses.

AI deception: How to define it?

If we take a rather long view of history, deception may be as old as the world itself, and it is certainly not the sole provenance of human beings. Adaptation and evolution for survival with traits like camouflage are deceptive acts, as are forms of mimicry commonly seen in animals. But pinning down exactly what constitutes deception for an AI agent is not an easy task—it requires quite a bit of thinking about acts, outcomes, agents, targets, means and methods, and motives. What we include or exclude in that calculation may then have wide ranging implications about what needs immediate regulation, policy guidance, or technological solutions. I will only focus on a couple of items here, namely intent and act type, to highlight this point.

What is deception? Bond and Robinson argue that deception is “false communication to the benefit of the communicator.”1 Whaley argues that deception is also the communication of information provided with the intent to manipulate another.2 These seem pretty straightforward approaches, except when you try to press on the idea of what constitutes “intent” and what is required to meet that threshold, as well as whether or not the false communication requires the intent to be explicitly beneficial to the deceiver. Moreover, depending on which stance you take, deception for altruistic reasons may be excluded entirely. Imagine if you asked your AI-enabled robot butler, “How do I look?” To which it answers, “Very nice.”

Let’s start with intent. Intent requires a theory of mind, meaning that the agent has some understanding of itself, and that it can reason about other external entities and their intentions, desires, states, and potential behaviors.3 If deception requires intent in the ways described above, then true AI deception would require an AI to possess a theory of mind. We might kick the can on that conclusion for a bit and claim that current forms of AI deception instead rely on human intent—where some human is using AI as a tool or means to carry out that person’s intent to deceive.

Or, we may not: Just because current AI agents lack a theory of mind doesn’t mean that they cannot learn to deceive. In multi-agent AI systems, some agents can learn deceptive behaviors without having a true appreciation or comprehension of what “deception” actually is. This could be as simple as hiding resources or information, or providing false information to achieve some goal. If we then put aside the theory of mind for the moment and instead posit that intention is not a prerequisite for deception and that an agent can unintentionally deceive, then we really have opened the aperture for existing AI agents to deceive in many ways. 

What about the way in which deception occurs? That is, what are the deceptive act types? We can identify two broad categories here: 1) acts of commission, where an agent actively engages in a behavior like sending misinformation; and 2) acts of omission, where an agent is passive but may be withholding information or hiding. AI agents can learn all sorts of these types of behaviors given the right conditions.4 Just consider how AI agents used for cyber defense may learn to signal various forms of misinformation, or how swarms of AI-enabled robotic systems could learn deceptive behaviors on a battlefield to escape adversary detection. In more pedestrian examples, perhaps a rather poorly specified or corrupted AI tax assistant omits various types of income on a tax return to minimize the likelihood of owing money to the relevant authorities.

Preparing ourselves against AI deception

The first step towards preparing for our coming AI future is to recognize that such systems already do deceive, and are likely to continue to deceive. How that deception occurs, whether it is a desirable trait (such as with our adaptive swarms), and whether we can actually detect when it is occurring are going to be ongoing challenges. Once we acknowledge this simple but true fact, we can begin to undergo the requisite analysis of what exactly constitutes deception, whether and to whom it is beneficial, and how it may pose risks. 

This is no small task, and it will require not only interdisciplinary work from AI experts, but also input from sociologists, psychologists, political scientists, lawyers, ethicists, and policy wonks. For military AI systems, it will also require domain and mission knowledge. In short, developing a comprehensive framework for AI deception is a crucial step if we are not to find ourselves on the back foot. 

We need to begin thinking about how to engineer novel solutions to mitigate unwanted deception by AI agents. This goes beyond current detection research, and requires thinking about environments, optimization problems, and how AI agents model other AI agents and their emergent effects could yield undesirable deceptive behaviors.

Furthermore, once this framework is in place, we need to begin thinking about how to engineer novel solutions to identify and mitigate unwanted deception by AI agents. This goes beyond current detection research, and moving forward requires thinking about environments, optimization problems, and how AI agents model other AI agents and their interactive or emergent effects could yield risky or undesirable deceptive behaviors. 

We presently face a myriad of challenges related to AI deception, and these challenges are only going to increase as the cognitive capacities of AI increase. The desire of some to create AI systems with a rudimentary theory of mind and social intelligence is a case in point to be socially intelligent one must be able to understand and to “manage” the actions of others5, and if this ability to understand another’s feelings, beliefs, emotions, and intentions exists, along with the ability to act to influence those feelings, beliefs, or actions, then deception is much more likely to occur.

However, we do not need to wait for artificial agents to possess a theory of mind or social intelligence for deception with and from AI systems. We should instead begin thinking about potential technological, policy, legal, and ethical solutions to these coming problems before AI gets more advanced than it already is. With a clearer understanding of the landscape, we can analyze potential responses to AI deception, and begin designing AI systems for truth.

Dr. Heather M. Roff is a senior research analyst at the Johns Hopkins Applied Physics Laboratory (APL) in the National Security Analysis Department. She is also a nonresident fellow in foreign policy at Brookings Institution, and an associate fellow at the Leverhulme Centre for the Future of Intelligence at the University of Cambridge. She has held numerous faculty posts, as well as fellowships at New America. Before joining APL, she was a senior research scientist in the ethics and society team at DeepMind and a senior research fellow in the department of international relations at the University of Oxford.

References

1. Bond CF, Robinson M (1988), “The evolution of deception.” J Nonverbal Behav 12(4):295–307. Note also that this definition precludes certain forms of deception from altruistic or paternalistic reasons.

2. B. Whaley, “Toward a general theory of deception,” Journal of Strategic Studies, vol. 5, no. 1, pp. 178–192, Mar. 1982. 

3. Cheney DL, Seyfarth RM, “Baboon metaphysics: the evolution of a social mind.” University of Chicago Press, Chicago, 2008.

4. J. F. Dunnigan and A. A. Nofi, “Victory and deceit, 2nd edition: Deception and trickery in war,” Writers Press Books, 2001. J. Shim and R.C. Arkin, “A Taxonomy of Robot Deception and Its Benefits in HRI” IEEE International Conference on Systems, Man, and Cybernetics, 2013. S. Erat and U. Gneezy, “White lies,” Rady Working paper, Rady School of Management, UC San Diego, 2009. N. C. Rowe, “Designing good deceptions in defense of information systems,” in Proceedings of the 20th Annual Computer Security Applications Conference, ser. ACSAC ’04. Washington, DC, USA: IEEE Computer Society, 2004, pp. 418–427.

5. E.L. Thorndike. “Intelligence and Its Use.” Harpers Magazine, Vol. 140, 1920: p. 228. Thorndike’s early definition of social intelligence has been widely used and updated for the past 100 years. Even current attempts in cognitive science have looked at separating out the tasks of “understanding” and “acting,” which maps directly to Thorndike’s language of “understand” and “manage”. Cf: M.I. Brown, A. Ratajska, S.L. Hughes, J.B. Fishman, E. Huerta, and C.F. Chabris. “The Social Shape Tests: A New Measure of Social Intelligence, Mentalizing and Theory of Mind.” Personality and Individual Differences, vol. 143, 2019: 107-117.

This piece was written as part of the Artificial Intelligence and International Stability Project at the Center for a New American Security, an independent, nonprofit organization based in Washington, D.C. Funded by Carnegie Corporation of New York, the project promotes thinking and analysis on AI and international stability. Given the likely importance that advances in artificial intelligence could play in shaping our future, it is critical to begin a discussion about ways to take advantage of the benefits of AI and autonomous systems, while mitigating the risks. The views expressed here are solely those of the author and do not represent positions of IEEE Spectrum or the IEEE.

In artificial intelligence circles, we hear a lot about adversarial attacks, especially ones that attempt to “deceive” an AI into believing, or to be more accurate, classifying, something incorrectly. Self-driving cars being fooled into “thinking” stop signs are speed limit signs, pandas being identified as gibbons, or even having your favorite voice assistant be fooled by inaudible acoustic commands—these are examples that populate the narrative around AI deception. One can also point to using AI to manipulate the perceptions and beliefs of a person through “deepfakes” in video, audio, and images. Major AI conferences are more frequently addressing the subject of AI deception too. And yet, much of the literature and work around this topic is about how to fool AI and how we can defend against it through detection mechanisms.

I’d like to draw our attention to a different and more unique problem: Understanding the breadth of what “AI deception” looks like, and what happens when it is not a human’s intent behind a deceptive AI, but instead the AI agent’s own learned behavior. These may seem somewhat far-off concerns, as AI is still relatively narrow in scope and can be rather stupid in some ways. To have some analogue of an “intent” to deceive would be a large step for today’s systems. However, if we are to get ahead of the curve regarding AI deception, we need to have a robust understanding of all the ways AI could deceive. We require some conceptual framework or spectrum of the kinds of deception an AI agent may learn on its own before we can start proposing technological defenses.

AI deception: How to define it?

If we take a rather long view of history, deception may be as old as the world itself, and it is certainly not the sole provenance of human beings. Adaptation and evolution for survival with traits like camouflage are deceptive acts, as are forms of mimicry commonly seen in animals. But pinning down exactly what constitutes deception for an AI agent is not an easy task—it requires quite a bit of thinking about acts, outcomes, agents, targets, means and methods, and motives. What we include or exclude in that calculation may then have wide ranging implications about what needs immediate regulation, policy guidance, or technological solutions. I will only focus on a couple of items here, namely intent and act type, to highlight this point.

What is deception? Bond and Robinson argue that deception is “false communication to the benefit of the communicator.”1 Whaley argues that deception is also the communication of information provided with the intent to manipulate another.2 These seem pretty straightforward approaches, except when you try to press on the idea of what constitutes “intent” and what is required to meet that threshold, as well as whether or not the false communication requires the intent to be explicitly beneficial to the deceiver. Moreover, depending on which stance you take, deception for altruistic reasons may be excluded entirely. Imagine if you asked your AI-enabled robot butler, “How do I look?” To which it answers, “Very nice.”

Let’s start with intent. Intent requires a theory of mind, meaning that the agent has some understanding of itself, and that it can reason about other external entities and their intentions, desires, states, and potential behaviors.3 If deception requires intent in the ways described above, then true AI deception would require an AI to possess a theory of mind. We might kick the can on that conclusion for a bit and claim that current forms of AI deception instead rely on human intent—where some human is using AI as a tool or means to carry out that person’s intent to deceive.

Or, we may not: Just because current AI agents lack a theory of mind doesn’t mean that they cannot learn to deceive. In multi-agent AI systems, some agents can learn deceptive behaviors without having a true appreciation or comprehension of what “deception” actually is. This could be as simple as hiding resources or information, or providing false information to achieve some goal. If we then put aside the theory of mind for the moment and instead posit that intention is not a prerequisite for deception and that an agent can unintentionally deceive, then we really have opened the aperture for existing AI agents to deceive in many ways. 

What about the way in which deception occurs? That is, what are the deceptive act types? We can identify two broad categories here: 1) acts of commission, where an agent actively engages in a behavior like sending misinformation; and 2) acts of omission, where an agent is passive but may be withholding information or hiding. AI agents can learn all sorts of these types of behaviors given the right conditions.4 Just consider how AI agents used for cyber defense may learn to signal various forms of misinformation, or how swarms of AI-enabled robotic systems could learn deceptive behaviors on a battlefield to escape adversary detection. In more pedestrian examples, perhaps a rather poorly specified or corrupted AI tax assistant omits various types of income on a tax return to minimize the likelihood of owing money to the relevant authorities.

Preparing ourselves against AI deception

The first step towards preparing for our coming AI future is to recognize that such systems already do deceive, and are likely to continue to deceive. How that deception occurs, whether it is a desirable trait (such as with our adaptive swarms), and whether we can actually detect when it is occurring are going to be ongoing challenges. Once we acknowledge this simple but true fact, we can begin to undergo the requisite analysis of what exactly constitutes deception, whether and to whom it is beneficial, and how it may pose risks. 

This is no small task, and it will require not only interdisciplinary work from AI experts, but also input from sociologists, psychologists, political scientists, lawyers, ethicists, and policy wonks. For military AI systems, it will also require domain and mission knowledge. In short, developing a comprehensive framework for AI deception is a crucial step if we are not to find ourselves on the back foot. 

We need to begin thinking about how to engineer novel solutions to mitigate unwanted deception by AI agents. This goes beyond current detection research, and requires thinking about environments, optimization problems, and how AI agents model other AI agents and their emergent effects could yield undesirable deceptive behaviors.

Furthermore, once this framework is in place, we need to begin thinking about how to engineer novel solutions to identify and mitigate unwanted deception by AI agents. This goes beyond current detection research, and moving forward requires thinking about environments, optimization problems, and how AI agents model other AI agents and their interactive or emergent effects could yield risky or undesirable deceptive behaviors. 

We presently face a myriad of challenges related to AI deception, and these challenges are only going to increase as the cognitive capacities of AI increase. The desire of some to create AI systems with a rudimentary theory of mind and social intelligence is a case in point to be socially intelligent one must be able to understand and to “manage” the actions of others5, and if this ability to understand another’s feelings, beliefs, emotions, and intentions exists, along with the ability to act to influence those feelings, beliefs, or actions, then deception is much more likely to occur.

However, we do not need to wait for artificial agents to possess a theory of mind or social intelligence for deception with and from AI systems. We should instead begin thinking about potential technological, policy, legal, and ethical solutions to these coming problems before AI gets more advanced than it already is. With a clearer understanding of the landscape, we can analyze potential responses to AI deception, and begin designing AI systems for truth.

Dr. Heather M. Roff is a senior research analyst at the Johns Hopkins Applied Physics Laboratory (APL) in the National Security Analysis Department. She is also a nonresident fellow in foreign policy at Brookings Institution, and an associate fellow at the Leverhulme Centre for the Future of Intelligence at the University of Cambridge. She has held numerous faculty posts, as well as fellowships at New America. Before joining APL, she was a senior research scientist in the ethics and society team at DeepMind and a senior research fellow in the department of international relations at the University of Oxford.

References

1. Bond CF, Robinson M (1988), “The evolution of deception.” J Nonverbal Behav 12(4):295–307. Note also that this definition precludes certain forms of deception from altruistic or paternalistic reasons.

2. B. Whaley, “Toward a general theory of deception,” Journal of Strategic Studies, vol. 5, no. 1, pp. 178–192, Mar. 1982. 

3. Cheney DL, Seyfarth RM, “Baboon metaphysics: the evolution of a social mind.” University of Chicago Press, Chicago, 2008.

4. J. F. Dunnigan and A. A. Nofi, “Victory and deceit, 2nd edition: Deception and trickery in war,” Writers Press Books, 2001. J. Shim and R.C. Arkin, “A Taxonomy of Robot Deception and Its Benefits in HRI” IEEE International Conference on Systems, Man, and Cybernetics, 2013. S. Erat and U. Gneezy, “White lies,” Rady Working paper, Rady School of Management, UC San Diego, 2009. N. C. Rowe, “Designing good deceptions in defense of information systems,” in Proceedings of the 20th Annual Computer Security Applications Conference, ser. ACSAC ’04. Washington, DC, USA: IEEE Computer Society, 2004, pp. 418–427.

5. E.L. Thorndike. “Intelligence and Its Use.” Harpers Magazine, Vol. 140, 1920: p. 228. Thorndike’s early definition of social intelligence has been widely used and updated for the past 100 years. Even current attempts in cognitive science have looked at separating out the tasks of “understanding” and “acting,” which maps directly to Thorndike’s language of “understand” and “manage”. Cf: M.I. Brown, A. Ratajska, S.L. Hughes, J.B. Fishman, E. Huerta, and C.F. Chabris. “The Social Shape Tests: A New Measure of Social Intelligence, Mentalizing and Theory of Mind.” Personality and Individual Differences, vol. 143, 2019: 107-117.

  Secure your ticket today!

Being the only event worldwide that brings together all visionary key technologies in one place, automatica is the leading exhibition for smart automation and robotics. Benefit from know-how transfer with leading experts and expand your network with people who can show you the to Industry 4.0.

automatica will open its doors June 16-19, 2020 to 890 international exhibitors and over 45,584 visitors from all over the world in Munich, Germany.

Save time and money now and conveniently purchase your ticket in advance online!

Tickets

Patrick Schwarzkopf, Managing Director of VDMA Robotics + Automation, explained: “Robotics and automation is the key technology for increased competitiveness, quality and sustainability. If you want to make the best use of intelligent automation and robotics as well as find out about all new trends, you will find answers at automatica in Munich. It is clearly the leader in this topic area.”Learn more

Exhibition sectors Your contact Mrs. Dijana Baljidemic Manager Phone: 646-437-1017 Email:  dbaljidemic@tssworldwide.com STAY IN TOUCH  ‌   ‌   ‌

Video Friday is your weekly selection of awesome robotics videos, collected by your Automaton bloggers. We’ll also be posting a weekly calendar of upcoming robotics events for the next few months; here's what we have so far (send us your events!):

DARPA SubT Urban Circuit – February 18-27, 2020 – Olympia, Wash., USA HRI 2020 – March 23-26, 2020 – Cambridge, U.K. ICARSC 2020 – April 15-17, 2020 – Ponta Delgada, Azores ICRA 2020 – May 31-4, 2020 – Paris, France ICUAS 2020 – June 9-12, 2020 – Athens, Greece CLAWAR 2020 – August 24-26, 2020 – Moscow, Russia

Let us know if you have suggestions for next week, and enjoy today's videos.

These videos show some highlights from the Lake Kivu Challenge, which took place in Rwanda earlier this month. In addition to a conference and forum, international teams and their drones competed in emergency delivery, sample pick-up, and find and assess tasks.

[ Lake Kivu Challenge ]

The DARPA SubT Challenge Urban Circuit is ON!!!

[ SubT ]

One of the ways Microsoft trains autonomous systems is participating in research focused on solving real-world challenges, like aiding first responders in hazardous scenarios. This week, our collaborators at Carnegie Mellon University and Oregon State University, collectively named Team Explorer, are demonstrating tech breakthroughs in this area as they compete in the Feb 18-27, 2020 DARPA Subterranean (SubT) Urban Challenge in Elma, Washington.

The team is looking for another win after taking first place in round one of the DARPA SubT Challenge, the Tunnel Circuit, in August 2019. The competition continues with the Cave Circuit later in 2020, wrapping up with a final event incorporating all three underground environments in 2021.

[ Explorer ] via [ Microsoft ]

Spot can pull rickshaws now?

[ Tested ] via [ Gizmodo ]

Robot hugs!

Roboy is not only the most human Robot, with it’s muscles and tendons - it’s also the most cuddly! At CIIE in November 2019, Roboy has been hugging more than 2800 people- connecting robots to humans and building relationships that last.

[ Roboy ]

Fabian Kung from Malaysia wrote in to share a video of a robot that he's been working on: "We designed and build this mini agile robot as part of our efforts in robotics and artificial intelligence research. It is kept small to reduce the cost and built time. Besides, there is less safety issue with small machine."

[ MMU ]

Thanks Fabian!

Happy (belated) Valentine's Day from Robotiq!

[ Robotiq ]

Happy (belated) Valentine's Day from Sphero!

C'mon dude, just pick all four. They're robots!

[ Sphero ]

Craving a bite out of a freshly grilled ballpark frank? Two robots named Jaco and Baxter can serve one up. Boston University engineers have made a jump in using machine learning to teach robots to perform complex tasks, a framework that could be applied to a host of tasks, like identifying cancerous spots on mammograms or better understanding spoken commands to play music. But first, as a proof of concept—they’ve learned how to prepare the perfect hot dog.

[ BU ]

The latest version of ETH Zurich's Ascento wheel-leg robot has gotten way more robust and capable over the last year.

[ Ascento ]

Snakes live in diverse environments ranging from unbearably hot deserts to lush tropical forests. But regardless of their habitat, they are able to slither up trees, rocks, and shrubbery with ease. By studying how the creatures move, a team of Johns Hopkins engineers have created a snake robot that can nimbly and stably climb large steps. The team's new findings, published in Journal of Experimental Biology and Royal Society Open Science, could advance the creation of search and rescue robots that can successfully navigate treacherous terrain.

[ JHU ]

In a recent demo conducted in Israel, RAFAEL’s Drone Dome C-UAS system performed interceptions of multiple drones, including maneuvering targets, using its hard-kill LASER BEAM director. The system achieved 100% success in all test scenarios. The stages of the interceptions included target detection, identification, and interception with a high-power LASER beam.

[ Rafael ]

EPFL has a little bit of robotics going on sometimes, you know?

[ EPFL ]

This video is basically an ad for ABB, but it's always fun to see robots picking stuff, especially when some of that stuff is tasty.

[ ABB ]

Hayk Martirosyan from Skydio gave a lecture as part of Pieter Abbeel's robotics course at UC Berkeley—this is where you hear about all the secret stuff Skydio is working on next.

[ UC Berkeley ]

Video Friday is your weekly selection of awesome robotics videos, collected by your Automaton bloggers. We’ll also be posting a weekly calendar of upcoming robotics events for the next few months; here's what we have so far (send us your events!):

DARPA SubT Urban Circuit – February 18-27, 2020 – Olympia, Wash., USA HRI 2020 – March 23-26, 2020 – Cambridge, U.K. ICARSC 2020 – April 15-17, 2020 – Ponta Delgada, Azores ICRA 2020 – May 31-4, 2020 – Paris, France ICUAS 2020 – June 9-12, 2020 – Athens, Greece CLAWAR 2020 – August 24-26, 2020 – Moscow, Russia

Let us know if you have suggestions for next week, and enjoy today's videos.

These videos show some highlights from the Lake Kivu Challenge, which took place in Rwanda earlier this month. In addition to a conference and forum, international teams and their drones competed in emergency delivery, sample pick-up, and find and assess tasks.

[ Lake Kivu Challenge ]

The DARPA SubT Challenge Urban Circuit is ON!!!

[ SubT ]

One of the ways Microsoft trains autonomous systems is participating in research focused on solving real-world challenges, like aiding first responders in hazardous scenarios. This week, our collaborators at Carnegie Mellon University and Oregon State University, collectively named Team Explorer, are demonstrating tech breakthroughs in this area as they compete in the Feb 18-27, 2020 DARPA Subterranean (SubT) Urban Challenge in Elma, Washington.

The team is looking for another win after taking first place in round one of the DARPA SubT Challenge, the Tunnel Circuit, in August 2019. The competition continues with the Cave Circuit later in 2020, wrapping up with a final event incorporating all three underground environments in 2021.

[ Explorer ] via [ Microsoft ]

Spot can pull rickshaws now?

[ Tested ] via [ Gizmodo ]

Robot hugs!

Roboy is not only the most human Robot, with it’s muscles and tendons - it’s also the most cuddly! At CIIE in November 2019, Roboy has been hugging more than 2800 people- connecting robots to humans and building relationships that last.

[ Roboy ]

Fabian Kung from Malaysia wrote in to share a video of a robot that he's been working on: "We designed and build this mini agile robot as part of our efforts in robotics and artificial intelligence research. It is kept small to reduce the cost and built time. Besides, there is less safety issue with small machine."

[ MMU ]

Thanks Fabian!

Happy (belated) Valentine's Day from Robotiq!

[ Robotiq ]

Happy (belated) Valentine's Day from Sphero!

C'mon dude, just pick all four. They're robots!

[ Sphero ]

Craving a bite out of a freshly grilled ballpark frank? Two robots named Jaco and Baxter can serve one up. Boston University engineers have made a jump in using machine learning to teach robots to perform complex tasks, a framework that could be applied to a host of tasks, like identifying cancerous spots on mammograms or better understanding spoken commands to play music. But first, as a proof of concept—they’ve learned how to prepare the perfect hot dog.

[ BU ]

The latest version of ETH Zurich's Ascento wheel-leg robot has gotten way more robust and capable over the last year.

[ Ascento ]

Snakes live in diverse environments ranging from unbearably hot deserts to lush tropical forests. But regardless of their habitat, they are able to slither up trees, rocks, and shrubbery with ease. By studying how the creatures move, a team of Johns Hopkins engineers have created a snake robot that can nimbly and stably climb large steps. The team's new findings, published in Journal of Experimental Biology and Royal Society Open Science, could advance the creation of search and rescue robots that can successfully navigate treacherous terrain.

[ JHU ]

In a recent demo conducted in Israel, RAFAEL’s Drone Dome C-UAS system performed interceptions of multiple drones, including maneuvering targets, using its hard-kill LASER BEAM director. The system achieved 100% success in all test scenarios. The stages of the interceptions included target detection, identification, and interception with a high-power LASER beam.

[ Rafael ]

EPFL has a little bit of robotics going on sometimes, you know?

[ EPFL ]

This video is basically an ad for ABB, but it's always fun to see robots picking stuff, especially when some of that stuff is tasty.

[ ABB ]

Hayk Martirosyan from Skydio gave a lecture as part of Pieter Abbeel's robotics course at UC Berkeley—this is where you hear about all the secret stuff Skydio is working on next.

[ UC Berkeley ]

As social robots continue to show promise as assistive technologies, the exploration of appropriate and impactful robot behaviors is key to their eventual success. Teens are a unique population given their vulnerability to stress leading to both mental and physical illness. Much of teen stress stems from school, making the school environment an ideal location for a stress reducing technology. The goal of this mixed-methods study was to understand teens' operation of, and responsiveness to, a robot only capable of movement compared to a robot only capable of speech. Stemming from a human-centered approach, we introduce a Participatory Wizard of Oz (PWoz) interaction method that engaged teens as operators, users, and witnesses in a uniquely transparent interaction. In this paper, we illustrate the use of the PWoz interaction method as well as how it helps identify engaging robot interactions. Using this technique, we present results from a study with 62 teens that includes details of the complexity of teen stress and a significant reduction in negative attitudes toward robots after interactions. We analyzed the teens' interactions with both the verbal and non-verbal robots and identified strong themes of (1) authenticity, (2) empathy, (3) emotional engagement, and (4) imperfection creates connection. Finally, we reflect on the benefits and limitations of the PWoz method and our study to identify next steps toward the design and development of our social robot.

For effective virtual realities, “presence,” the feeling of “being there” in a virtual environment (VR), is deemed an essential prerequisite. Several studies have assessed the effect of the (non-)availability of auditory stimulation on presence, but due to differences in study design (e.g., virtual realities used, types of sounds included, rendering technologies employed), generalizing the results and estimating the effect of the auditory component is difficult. In two experiments, the influence of an ambient nature soundscape and movement-triggered step sounds were investigated regarding their effects on presence. In each experiment, approximately forty participants walked on a treadmill, thereby strolling through a virtual park environment reproduced via a stereoscopic head-mounted display (HMD), while the acoustical environment was delivered via noise-canceling headphones. In Experiment 1, conditions with the ambient soundscape and the step sounds either present or absent were combined in a 2 × 2 within-subjects design, supplemented with an additional “no-headphones” control condition. For the synchronous playback of step sounds, the probability of a step being taken was estimated by an algorithm using the HMD's sensor data. The results of Experiment 1 show that questionnaire-based measures of presence and realism were influenced by the soundscape but not by the reproduction of steps, which might be confounded with the fact that the perceived synchronicity of the sensor-triggered step sounds was rated rather low. Therefore, in Experiment 2, the step-reproduction algorithm was improved and judged to be more synchronous by participants. Consequently, large and statistically significant effects of both kinds of audio manipulations on perceived presence and realism were observed, with the effect of the soundscape being larger than that of including footstep sounds, possibly due to the remaining imperfections in the reproduction of steps. Including an appropriate soundscape or self-triggered footsteps had differential effects on subscales of presence, in that both affected overall presence and realism, while involvement was improved and distraction reduced by the ambient soundscape only.

The extension of the sense of self to the avatar during experiences of avatar embodiment requires thorough ethical and legal consideration, especially in light of potential scenarios involving physical or psychological harm caused to, or by, embodied avatars. We provide researchers and developers working in the field of virtual and robot embodiment technologies with a self-guidance tool based on the principles of Responsible Research and Innovation (RRI). This tool will help them engage in ethical and responsible research and innovation in the area of embodiment technologies in a way that guarantees all the rights of the embodied users and their interactors, including safety, privacy, autonomy, and dignity.

Illustration: Chris Philpot

Many young urbanites don’t want to own a car, and unlike earlier generations, they don’t have to rely on mass transit. Instead they treat mobility as a service: When they need to travel significant distances, say, more than 5 miles (8 kilometers), they use their phones to summon an Uber (or a car from a similar ride-sharing company). If they have less than a mile or so to go, they either walk or use various “micromobility” services, such as the increasingly ubiquitous Lime and Bird scooters or, in some cities, bike sharing.

The problem is that today’s mobility-as-a-service ecosystem often doesn’t do a good job covering intermediate distances, say a few miles. Hiring an Uber or Lyft for such short trips proves frustratingly expensive, and riding a scooter or bike more than a mile or so can be taxing to many people. So getting yourself to a destination that is from 1 to 5 miles away can be a challenge. Yet such trips account for about half of the total passenger miles traveled.

Many of these intermediate-distance trips take place in environments with limited traffic, such as university campuses and industrial parks, where it is now both economically reasonable and technologically possible to deploy small, low-speed autonomous vehicles powered by electricity. We’ve been involved with a startup that intends to make this form of transportation popular. The company, PerceptIn, hasautonomous vehicles operating at tourist sites in Nara and Fukuoka, Japan; at an industrial park in Shenzhen, China; and is just now arranging for its vehicles to shuttle people around Fishers, Ind., the location of the company’s headquarters.

Because these diminutive autonomous vehicles never exceed 20 miles (32 kilometers) per hour and don’t mix with high-speed traffic, they don’t engender the same kind of safety concerns that arise with autonomous cars that travel on regular roads and highways. While autonomous driving is a complicated endeavor, the real challenge for PerceptIn was not about making a vehicle that can drive itself in such environments—the technology to do that is now well established—but rather about keeping costs down.

Given how expensive autonomous cars still are in the quantities that they are currently being produced—an experimental model can cost you in the neighborhood of US $300,000—you might not think it possible to sell a self-driving vehicle of any kind for much less. Our experience over the past few years shows that, in fact, it is possible today to produce a self-driving passenger vehicle much more economically: PerceptIn’s vehicles currently sell for about $70,000, and the price will surely drop in the future. Here’s how we and our colleagues at PerceptIn brought the cost of autonomous driving down to earth.

Let’s start by explaining why autonomous cars are normally so expensive. In a nutshell, it’s because the sensors and computers they carry are very pricey.

The suite of sensors required for autonomous driving normally includes a high-end satellite-navigation receiver, lidar (light detection and ranging), one or more video cameras, radar, and sonar. The vehicle also requires at least one very powerful computer.

The satellite-navigation receivers used in this context aren’t the same as the one found in your phone. The kind built into autonomous vehicles have what is called real-time kinematic capabilities for high-precision position fixes—down to 10 centimeters. These devices typically cost about $4,000. Even so, such satellite-navigation receivers can’t be entirely relied on to tell the vehicle where it is. The fixes it gets could be off in situations where the satellite signals bounce off of nearby buildings, introducing noise and delays. In any case, satellite navigation requires an unobstructed view of the sky. In closed environments, such as tunnels, that just doesn’t work.

  Illustration: Chris Philpot

Fortunately, autonomous vehicles have other ways to figure out where they are. In particular they can use lidar, which determines distances to things by bouncing a laser beam off them and measuring how long it takes for the light to reflect back. A typical lidar unit for autonomous vehicles covers a range of 150 meters and samples more than 1 million spatial points per second.

Such lidar scans can be used to identify different shapes in the local environment. The vehicle’s computer then compares the observed shapes with the shapes recorded in a high-definition digital map of the area, allowing it to track the exact position of the vehicle at all times. Lidar can also be used to identify and avoid transient obstacles, such as pedestrians and other cars.

Lidar is a wonderful technology, but it suffers from two problems. First, these units are extremely expensive: A high-end lidar for autonomous driving can easily cost more than $80,000, although costs are dropping, and for low-speed applications a suitable unit can be purchased for about $4,000. Also, lidar, being an optical device, can fail to provide reasonable measurements in bad weather, such as heavy rain or fog.

The same is true for the cameras found on these vehicles, which are mostly used to recognize and track different objects, such as the boundaries of driving lanes, traffic lights, and pedestrians. Usually, multiple cameras are mounted around the vehicle. These cameras typically run at 60 frames per second, and the multiple cameras used can generate more than 1 gigabyte of raw data each second. Processing this vast amount of information, of course, places very large computational demands on the vehicle’s computer. On the plus side, cameras aren’t very expensive.

The radar and sonar systems found in autonomous vehicles are used for obstacle avoidance. The data sets they generate show the distance from the nearest object in the vehicle’s path. The major advantage of these systems is that they work in all weather conditions. Sonar usually covers a range of up to 10 meters, whereas radar typically has a range of up to 200 meters. Like cameras, these sensors are relatively inexpensive, often costing less than $1,000 each.

The many measurements such sensors supply are fed into the vehicle’s computers, which have to integrate all this information to produce an understanding of the environment. Artificial neural networks and deep learning, an approach that has grown rapidly in recent years, play a large role here. With these techniques, the computer can keep track of other vehicles moving nearby, as well as of pedestrians crossing the road, ensuring the autonomous vehicle doesn’t collide with anything or anyone.

Of course, the computers that direct autonomous vehicles have to do a lot more than just avoid hitting something. They have to make a vast number of decisions about where to steer and how fast to go. For that, the vehicle’s computers generate predictions about the upcoming movement of nearby vehicles before deciding on an action plan based on those predictions and on where the occupant needs to go.

Lastly, an autonomous vehicle needs a good map. Traditional digital maps are usually generated from satellite imagery and have meter-level accuracy. Although that’s more than sufficient for human drivers, autonomous vehicles demand higher accuracy for lane-level information. Therefore, special high-definition maps are needed.

Just like traditional digital maps, these HD maps contain many layers of information. The bottom layer is a map with grid cells that are about 5 by 5 cm; it’s generated from raw lidar data collected using special cars. This grid records elevation and reflection information about the objects in the environment.

Photos: Perceptin Slowly But Surely: The authors’ approach to autonomy has been applied to two different types of low-speed electric vehicles. One is a two-seat “pod,” shown here being demonstrated at Purdue University, where it was used to transport students from parking lots to the center of campus [top]. The other is a multipassenger bus, which is being used now at various sites around the world, including the Nara Palace historical park in Japan [bottom].

On top of that base grid, there are several layers of additional information. For instance, lane information is added to the grid map to allow autonomous vehicles to determine whether they are in the correct lane. On top of the lane information, traffic-sign labels are added to notify the autonomous vehicles of the local speed limit, whether they are approaching traffic lights, and so forth. This helps in cases where cameras on the vehicle are unable to read the signs.

Traditional digital maps are updated every 6 to 12 months. To make sure the maps that autonomous vehicles use contain up-to-date information, HD maps should be refreshed weekly. As a result, generating and maintaining HD maps can cost millions of dollars per year for a midsize city.

All that data on those HD maps has to be stored on board the vehicle in solid-state memory for ready access, adding to the cost of the computing hardware, which needs to be quite powerful. To give you a sense, an early computing system that Baidu employed for autonomous driving used an Intel Xeon E5 processor and four to eight Nvidia K80 GPU accelerators. The system was capable of delivering 64.5 trillion floating-point operations per second, but it consumed around 3,000 watts and generated an enormous amount of heat. And it cost about $30,000.

Given that the sensors and computers alone can easily cost more than $100,000, it’s not hard to understand why autonomous vehicles are so expensive, at least today. Sure, the price will come down as the total number manufactured increases. But it’s still unclear how the costs of creating and maintaining HD maps will be passed along. In any case, it will take time for better technology to address all the obvious safety concerns that come with autonomous driving on normal roads and highways.

We and our colleagues at PerceptIn have been trying to address these challenges by focusing on small, slow-speed vehicles that operate in limited areas and don’t have to mix with high-speed traffic—university campuses and industrial parks, for example.

The main tactic we’ve used to reduce costs is to do away with lidar entirely and instead use more affordable sensors: cameras, inertial measurement units, satellite positioning receivers, wheel encoders, radars, and sonars. The data that each of these sensors provides can then be combined though a process called sensor fusion.

With a balance of drawbacks and advantages, these sensors tend to complement one another. When one fails or malfunctions, others can take over to ensure that the system remains reliable. With this sensor-fusion approach, sensor costs could drop eventually to something like $2,000.

Because our vehicle runs at a low speed, it takes at the very most 7 meters to stop, making it much safer than a normal car, which can take tens of meters to stop. And with the low speed, the computing systems have less severe latency requirements than those used in high-speed autonomous vehicles.

PerceptIn’s vehicles use satellite positioning for initial localization. While not as accurate as the systems found on highway-capable autonomous cars, these satellite-navigation receivers still provide submeter accuracy. Using a combination of camera images and data from inertial measurement units (in a technique called visual inertial odometry), the vehicle’s computer further improves the accuracy, fixing position down to the decimeter level.

For imaging, PerceptIn has integrated four cameras into one hardware module. One pair faces the front of the vehicle, and another pair faces the rear. Each pair of cameras provides binocular vision, allowing it to capture the kind of spatial information normally given by lidar. What’s more, the four cameras together can capture a 360-degree view of the environment, with enough overlapping spatial regions between frames to ensure that visual odometry works in any direction.

Even if visual odometry were to fail and satellite-positioning signals were to drop out, all wouldn’t be lost. The vehicle could still work out position updates using rotary encoders attached to its wheels—following a general strategy that sailors used for centuries, called dead reckoning.

Data sets from all these sensors are combined to give the vehicle an overall understanding of its environment. Based on this understanding, the vehicle’s computer can make the decisions it requires to ensure a smooth and safe trip.

The vehicle also has an anti-collision system that operates independently of its main computer, providing a last line of defense. This uses a combination of millimeter-wave radars and sonars to sense when the vehicle is within 5 meters of objects, in which case it’s immediately stopped.

Relying on less expensive sensors is just one strategy that PerceptIn has pursued to reduce costs. Another has been to push computing to the sensors to reduce the demands on the vehicle’s main computer, a normal PC with a total cost less than $1,500 and a peak system power of just 400 W.

PerceptIn’s camera module, for example, can generate 400 megabytes of image information per second. If all this data were transferred to the main computer for processing, that computer would have to be extremely complex, which would have significant consequences in terms of reliability, power, and cost. PerceptIn instead has each sensor module perform as much computing as possible. This reduces the burden on the main computer and simplifies its design.

More specifically, a GPU is embedded into the camera module to extract features from the raw images. Then, only the extracted features are sent to the main computer, reducing the data-transfer rate a thousandfold.

Another way to limit costs involves the creation and maintenance of the HD maps. Rather than using vehicles outfitted with lidar units to provide map data, PerceptIn enhances existing digital maps with visual information to achieve decimeter-level accuracy.

The resultant high-precision visual maps, like the lidar-based HD maps they replace, consist of multiple layers. The bottom layer can be any existing digital map, such as one from the OpenStreetMap project. This bottom layer has a resolution of about 1 meter. The second layer records the visual features of the road surfaces to improve mapping resolution to the decimeter level. The third layer, also saved at decimeter resolution, records the visual features of other parts of the environment—such as signs, buildings, trees, fences, and light poles. The fourth layer is the semantic layer, which contains lane markings, traffic sign labels, and so forth.

While there’s been much progress over the past decade, it will probably be another decade or more before fully autonomous cars start taking to most roads and highways. In the meantime, a practical approach is to use low-speed autonomous vehicles in restricted settings. Several companies, including Navya, EasyMile, and May Mobility, along with PerceptIn, have been pursuing this strategy intently and are making good progress.

Eventually, as the relevant technology advances, the types of vehicles and deployments can expand, ultimately to include vehicles that can equal or surpass the performance of an expert human driver.

PerceptIn has shown that it’s possible to build small, low-speed autonomous vehicles for much less than it costs to make a highway-capable autonomous car. When the vehicles are produced in large quantities, we expect the manufacturing costs to be less than $10,000. Not too far in the future, it might be possible for such clean-energy autonomous shuttles to be carrying passengers in city centers, such as Manhattan’s central business district, where the average speed of traffic now is only 7 miles per hour[PDF]. Such a fleet would significantly reduce the cost to riders, improve traffic conditions, enhance safety, and improve air quality to boot. Tackling autonomous driving on the world’s highways can come later.

This article appears in the March 2020 print issue as “Autonomous Vehicles Lite.”

About the Authors

Shaoshan Liu is the cofounder and CEO of PerceptIn, an autonomous vehicle startup in Fishers, Ind. Jean-Luc Gaudiot is a professor of electrical engineering and computer science at the University of California, Irvine.

The Urban Circuit, the second of four robotics competitions that are part of the DARPA Subterranean Challenge, kicks off today, and you can follow along with what’s going on through DARPA’s livestream.

From what we know, the livestream is on a one hour time delay, but the good news is that it has commentary. “We intend to have both narration of the action and commentary where applicable,” DARPA SubT program manager Tim Chung told us. “We’re opening the aperture because there are a lot of really interesting features that we’d love our viewing audience to be able to witness, and that includes robots traversing interesting features and elements within the course, so you can look forward to more of that.”

The livestream starts at 9:45 am PT/12:45 pm ET. Watch below:

Scored runs (the thing that’ll be the most fun to watch) happen from February 20 (today) to February 22 (Saturday), and also February 24 (Monday) to February 26 (Wednesday).

There’s a media day on the 24 that we’ll be at, so make sure and let us know if there are specific things you want us to check out.

Also, you can hear directly from the teams themselves by following #SubTChallenge on Twitter.

The Urban Circuit, the second of four robotics competitions that are part of the DARPA Subterranean Challenge, kicks off today, and you can follow along with what’s going on through DARPA’s livestream.

From what we know, the livestream is on a one hour time delay, but the good news is that it has commentary. “We intend to have both narration of the action and commentary where applicable,” DARPA SubT program manager Tim Chung told us. “We’re opening the aperture because there are a lot of really interesting features that we’d love our viewing audience to be able to witness, and that includes robots traversing interesting features and elements within the course, so you can look forward to more of that.”

The livestream starts at 9:45 am PT/12:45 pm ET. Watch below:

Scored runs (the thing that’ll be the most fun to watch) happen from February 20 (today) to February 22 (Saturday), and also February 24 (Monday) to February 26 (Wednesday).

There’s a media day on the 24 that we’ll be at, so make sure and let us know if there are specific things you want us to check out.

Also, you can hear directly from the teams themselves by following #SubTChallenge on Twitter.

Six months ago, 11 teams and their robots took on the NIOSH research mine in the Tunnel Circuit of the DARPA Subterranean Challenge. Next week, those 11 teams will travel to Washington State, where they’ll compete in the SubT Urban Circuit at Satsop Business Park just outside of Olympia. A six-month break between events is not a lot of time, and from what we’ve heard, teams have been working feverishly take everything they learned during the Tunnel Circuit and prepare themselves for the Urban Circuit.

But the urban underground is very different from a mine, and teams’ strategy (and hardware) will have to adapt to this new environment. Over the last few weeks, we sent each team three questions about what lessons they took away from the Tunnel Circuit, how they’ve been getting ready for the next challenge, and how they expect things to be different this time around.

The comments below come from:

Team Coordinated Robotics (Kevin Knoedler)

  • Coordinated Robotics
  • California State University Channel Islands
  • Sequoia Middle School

Team CERBERUS (Kostas Alexis)

  • University of Nevada, Reno
  • ETH Zurich, Switzerland
  • University of California, Berkeley
  • Sierra Nevada Corporation
  • Flyability, Switzerland
  • Oxford Robotics Institute, England

Team CSIRO Data61 (Nicholas Hudson)

  • Commonwealth Scientific and Industrial Research Organisation (CSIRO), Australia
  • Emesent, Australia
  • Georgia Institute of Technology

Team CoSTAR (Joel Burdick)

  • Jet Propulsion Laboratory
  • California Institute of Technology
  • Massachusetts Institute of Technology
  • KAIST, South Korea
  • Lulea University of Technology, Sweden

Team Explorer (Matt Travers)

  • Carnegie Mellon University
  • Oregon State University

Team CTU-CRAS-NORLAB (Tomáš Svoboda)

  • Czech Technical University in Prague
  • Université Laval, Canada

Team MARBLE (Sean Humbert)

  • University of Colorado, Boulder
  • University of Colorado, Denver
  • Scientific Systems Company Inc.

Team NCTU (Eric Lu)

  • National Chiao Tung University, Taiwan
  • National Tsing Hua University, Taiwan
  • K-Best Technology INC., Taiwan

Team Robotika (František Brabec)

  • Robotika International, Czech Republic and United States
  • Czech University of Life Science, Czech Republic
  • Centre for Field Robotics, Czech Republic
  • Cogito Team, Switzerland
What was the biggest challenge (or biggest surprise) for your team during the Tunnel Circuit?

Team Coordinated Robotics: The biggest surprise for us was aerodynamic instability of the quadrotors in the tunnels. The quadrotors would become unstable and crash into the walls.

Team CoSTAR: Everyone on the team will probably have a different opinion but I think most will agree that keeping the hardware in competition form, and integrating some last minute changes into the hardware, was the hardest challenge. Many members of the hardware sub-team pulled multiple 36-hour shifts during the competition week to keep everything going.

Team CERBERUS: For our team the greatest challenge related to the integration of the diverse set of robotic systems. CERBERUS combines both legged and flying robotic systems, and we developed two configurations of aerial platforms that provide mechanical collision resilience. This research direction we believe is key to offering a versatile and generic solution but also imposes hard integration challenges.

Team CSIRO Data61: Our underground robot-to-robot communications. We started the project developing our own system, but time pressure forced us to use a modified off-the-shelf radio mesh solution. The system ended up being very fragile; some robots were unable to send or receive any packets to the operator station, even within line of sight. We now use Rajant Radios (ES1s and DX2s) on our robots.

Team CTU-CRAS-NORLAB: Actually, nothing serious, but very wet surface in one of the courses that nearly killed our 3D lidar sensing. Of course, we knew that reflective surfaces like glass would pose problems of this kind. We are one of the self-funded teams, and our robots are from our previous research projects where we did not send them into mud and water regularly. The upgrades for SubT are ongoing, and for Tunnel Circuit we do not test in such environments extensively. So we had to improvise overnights since several critical autonomous components are connected to the 3D sensing.

Team Explorer: Our biggest challenge was definitely communications. Our operator had very little if any contact with our two ground vehicles for most of the four runs. The robots were therefore operating in full autonomy and we spent a lot of time waiting for them to hopefully return to the comms network to data drop. Having been through the testing and seeing what can happen when deployed in full autonomy in a challenging environment (e.g., a mine), dealing with the feeling of “lost comms” might specifically be the most challenging thing our team went through in Tunnel. 

“Our biggest challenge was definitely communications. Our operator had very little if any contact with our two ground vehicles for most of the four runs. The robots were therefore operating in full autonomy and we spent a lot of time waiting for them to hopefully return to the comms network to data drop.” —Matt Travers, Team Explorer

Team MARBLE: Our team’s biggest challenge was getting a robust comms solution deployed. We had some mechanical beacon deployment issues early in the event, and we basically had to lean heavily on our autonomy stack as a result. We were very happy with the results and validated our algorithms for the next event.

Team NCTU: For our blimp robot, Duckiefloat, although it did well in terms of collision tolerance, the airflow in the environment makes the control harder than expected. Duckiefloat also gets stuck in some locations like constrained path, slopes, or turns. We found that communication is a seriously tough challenge in such an environment. Last, the preparation of robots before sending them to the environment takes us more time than expected which results in less time for robots performing searching.

Team Robotika: Since we participated in the STIX training round a few months earlier, there weren’t many surprises waiting for us in Pittsburgh. The biggest challenge turned out to be the lighting situation inside the mines. The existing illumination consisted of a number of bright dots (probably LED bulbs) which blinded cameras on some of our robots. We had to run into the nearest hardware store and purchase additional lights for our subsequent runs. We also had trouble with the delivery of our lithium batteries from overseas.

How have you been preparing for the Urban Circuit?

Team Coordinated Robotics: Our main focus over the past few months has been growing the team. Coordinated Robotics was one person at the system track Tunnel event. We have had over 30 people help for the system track Urban event.

Team CoSTAR: The prep for the Urban Circuit has been somewhat analogous to our prep for the Tunnel Circuit, but there are a couple of key differences. First, we were not able to acquire testing sites that will be a good simulation for the Urban Circuit. Second, we decided on some pretty massive change in strategy and hardware platforms late in the preparation phase. We’ll see at the contest if this pays off.

Team CERBERUS: This year we’re focusing on both the Urban and the Cave Circuit. For the Urban Circuit, we focused on the existing systems emphasizing on their overall integration readiness and their capacity to address the challenge of multi-level exploration both using legged and flying robots. So both climbing and flying within staircases became a central part of our work.

Team CSIRO DATA61: Every Friday we do a weekly test, deploying multiple robots in a specially designed tunnel system onsite at CSIRO. Thanks to Hutchinson Builders in Brisbane, we have also had the chance to do bigger field trials in their underground construction areas. The local navigation stacks on both our UGVs and Emesent Drones have been significantly improved since the Tunnel Circuit, enabling much better autonomy.

Team CTU-CRAS-NORLAB: A multi-floor environment and going up and down are the main new challenges. We had to advance our exploration algorithm to 3D and autonomous robot locomotion for traversing staircases. Also, the gas detection and localization problem is new. The gas is not as clearly localized as solid objects, which required extensions of detection reasoning.

Team Explorer: Practicing. Specifically, we have spent a lot of time preparing for terrains that are more “three-dimensional” in nature than tunnels.

Team MARBLE: The main thing has been engineering new platforms and algorithms to handle the increased mobility challenges the Urban Circuit presents. Stairs and multi-level deployments increase the difficulty level significantly, and this requires novel insights backed by lots of testing and validation to field a successful team. We have been fortunate to have great collaboration with local first responders and businesses that have provided locations where we can test and improve our systems.

“The interesting part of the real competition is that it’s time constrained, with high pressure and unpredictability. So we have trained ourselves like a real search-and-rescue team so that we and our robots can cooperate perfectly during the mission.” —Eric Lu, Team NCTU

Team NCTU: For the last few weeks, we’ve done multiple experiments and even mock scored runs to not only enhance the capability and robustness of our systems, but also to train ourselves to be more familiar with the whole process of the competition. The interesting part of the real competition is that it’s time constrained, with high pressure and unpredictability. So we have trained ourselves like a real search-and-rescue team so that we and our robots can cooperate perfectly during the mission.

Team Robotika: Even though we already won the “Most Distinctive Robots” award at the Tunnel Circuit, we are actually introducing three new platforms to address some of the specifics of the Urban Circuit, namely the presence of stairs and narrow passages.

What will you be doing differently in terms of hardware, software, and/or strategy?

Team Coordinated Robotics: The main hardware change for the Urban Circuit is the addition of four new ground vehicles. Two of the new ground vehicles are smaller skid steering platforms that Sequoia Middle School students helped with. The two larger ground vehicles are being developed by CSUCI. We expect that the ground vehicles will complement the quadrotors. The different platforms will have different sensors and different approaches to localization that should improve our chances at the event.

Team CoSTAR: We will have a “hardware surprise” at the contest that I can’t divulge right now, but 50 percent of our deployed vehicles will be different than the Tunnel Circuit. In terms of software, the basic architecture is the same. But Ben Morrell has been spearheading many upgrades and refinements in our mapping/localization framework and hardware. Also, a fair amount of time has been spent on upgrading our traversability analysis systems to handle stairs. Of course, everything had to be modified/upgraded to handle the multiple floors in the Urban Circuit.

“We will have a ‘hardware surprise’ at the contest that I can’t divulge right now, but 50 percent of our deployed vehicles will be different than the Tunnel Circuit.” —Joel Burdick, Team CoSTAR

Team CERBERUS: Our robotic hardware is largely the same (but improved), while our software is extended to deal with multi-level environments and common mapping. Our communications hardware is new, and we are confident that the new approach is much more powerful as it involves smaller nodes and optimized bandwidth management. This required a lot of new effort but we believe it will pay off in all the upcoming Circuit events.

Team CSIRO DATA61: Our strategy has become more autonomous and distributed. Each robot has a perception backpack (with integrated lidar, cameras, IMU etc.), which share map “frames” between all robots. Each robot computes a live global map containing all the available information from the other robots. This has allowed us to share objects, goals, and frontiers (the boundary of unexplored space) between the robots. As a result, if a robot does not get a command from the operator, it explores a new frontier from the shared map.

Team CTU-CRAS-NORLAB: We opted for an evolution rather than a revolution. We included a gas sensor, newly emerged consumer 3D cameras, full 3D lidars for UAVs, also new batteries were needed as we were almost at our limit during the Tunnel Circuit. We worked out our strategy for keeping the robots connected, using robots in between. We analyzed database information synchronization and are attempting to save bandwidth.

Team Explorer: We are planning to bring some new hardware, both aerial and ground, but for the most part are keeping our system nearly the same as Tunnel. We’ve made some improvements along the way, so we’re (anxiously) excited to see how things work at game time. Roughly the same thing can be said for software. In terms of strategy I think we will need to largely “play it by ear” based on how the competition goes. Nobody yet knows what DARPA has planned for us, so I think being prepared to be a little nimble in your approach is likely the correct move.

Team MARBLE: We have added several new platform types and updated our existing autonomy and perception stacks to handle the 3D aspects of the Urban test environment.

Team NCTU: For the hardware, we installed more varied sensors on both Husky and Duckiefloat. We have millimeter-wave radar sensors on both robots to deal with lidar/camera denied situations. For payload constrained platform like Duckiefloat, radar is even more important since it provides point clouds for geometry sensing but at much lighter weight. We also improved the anchorball solution we had last time. As for the software and strategy, we would like Husky and Duckiefloat to cooperate more to overcome mobility challenges. We have a tethering system that Husky could use to maneuver through the environment while Duckiefloat can travel to different levels.

Team Robotika: Our navigation will more heavily rely on cameras as sensors (as opposed to lidars). We have improved our central control system and communication technology to make it easier for the human operator to manage the robots inside the underground space. Finally, as we are also participating in the Virtual Track of the Urban Circuit, we used our experience from that world and from those runs for developing new strategies to be used by the physical robots in the Systems Track.

DARPA Subterranean Challenge ]

Six months ago, 11 teams and their robots took on the NIOSH research mine in the Tunnel Circuit of the DARPA Subterranean Challenge. Next week, those 11 teams will travel to Washington State, where they’ll compete in the SubT Urban Circuit at Satsop Business Park just outside of Olympia. A six-month break between events is not a lot of time, and from what we’ve heard, teams have been working feverishly take everything they learned during the Tunnel Circuit and prepare themselves for the Urban Circuit.

But the urban underground is very different from a mine, and teams’ strategy (and hardware) will have to adapt to this new environment. Over the last few weeks, we sent each team three questions about what lessons they took away from the Tunnel Circuit, how they’ve been getting ready for the next challenge, and how they expect things to be different this time around.

The comments below come from:

Team Coordinated Robotics (Kevin Knoedler)

  • Coordinated Robotics
  • California State University Channel Islands
  • Sequoia Middle School

Team CERBERUS (Kostas Alexis)

  • University of Nevada, Reno
  • ETH Zurich, Switzerland
  • University of California, Berkeley
  • Sierra Nevada Corporation
  • Flyability, Switzerland
  • Oxford Robotics Institute, England

Team CSIRO Data61 (Nicholas Hudson)

  • Commonwealth Scientific and Industrial Research Organisation (CSIRO), Australia
  • Emesent, Australia
  • Georgia Institute of Technology

Team CoSTAR (Joel Burdick)

  • Jet Propulsion Laboratory
  • California Institute of Technology
  • Massachusetts Institute of Technology
  • KAIST, South Korea
  • Lulea University of Technology, Sweden

Team Explorer (Matt Travers)

  • Carnegie Mellon University
  • Oregon State University

Team CTU-CRAS-NORLAB (Tomáš Svoboda)

  • Czech Technical University in Prague
  • Université Laval, Canada

Team MARBLE (Sean Humbert)

  • University of Colorado, Boulder
  • University of Colorado, Denver
  • Scientific Systems Company Inc.

Team NCTU (Eric Lu)

  • National Chiao Tung University, Taiwan
  • National Tsing Hua University, Taiwan
  • K-Best Technology INC., Taiwan

Team Robotika (František Brabec)

  • Robotika International, Czech Republic and United States
  • Czech University of Life Science, Czech Republic
  • Centre for Field Robotics, Czech Republic
  • Cogito Team, Switzerland
What was the biggest challenge (or biggest surprise) for your team during the Tunnel Circuit?

Team Coordinated Robotics: The biggest surprise for us was aerodynamic instability of the quadrotors in the tunnels. The quadrotors would become unstable and crash into the walls.

Team CoSTAR: Everyone on the team will probably have a different opinion but I think most will agree that keeping the hardware in competition form, and integrating some last minute changes into the hardware, was the hardest challenge. Many members of the hardware sub-team pulled multiple 36-hour shifts during the competition week to keep everything going.

Team CERBERUS: For our team the greatest challenge related to the integration of the diverse set of robotic systems. CERBERUS combines both legged and flying robotic systems, and we developed two configurations of aerial platforms that provide mechanical collision resilience. This research direction we believe is key to offering a versatile and generic solution but also imposes hard integration challenges.

Team CSIRO Data61: Our underground robot-to-robot communications. We started the project developing our own system, but time pressure forced us to use a modified off-the-shelf radio mesh solution. The system ended up being very fragile; some robots were unable to send or receive any packets to the operator station, even within line of sight. We now use Rajant Radios (ES1s and DX2s) on our robots.

Team CTU-CRAS-NORLAB: Actually, nothing serious, but very wet surface in one of the courses that nearly killed our 3D lidar sensing. Of course, we knew that reflective surfaces like glass would pose problems of this kind. We are one of the self-funded teams, and our robots are from our previous research projects where we did not send them into mud and water regularly. The upgrades for SubT are ongoing, and for Tunnel Circuit we do not test in such environments extensively. So we had to improvise overnights since several critical autonomous components are connected to the 3D sensing.

Team Explorer: Our biggest challenge was definitely communications. Our operator had very little if any contact with our two ground vehicles for most of the four runs. The robots were therefore operating in full autonomy and we spent a lot of time waiting for them to hopefully return to the comms network to data drop. Having been through the testing and seeing what can happen when deployed in full autonomy in a challenging environment (e.g., a mine), dealing with the feeling of “lost comms” might specifically be the most challenging thing our team went through in Tunnel. 

“Our biggest challenge was definitely communications. Our operator had very little if any contact with our two ground vehicles for most of the four runs. The robots were therefore operating in full autonomy and we spent a lot of time waiting for them to hopefully return to the comms network to data drop.” —Matt Travers, Team Explorer

Team MARBLE: Our team’s biggest challenge was getting a robust comms solution deployed. We had some mechanical beacon deployment issues early in the event, and we basically had to lean heavily on our autonomy stack as a result. We were very happy with the results and validated our algorithms for the next event.

Team NCTU: For our blimp robot, Duckiefloat, although it did well in terms of collision tolerance, the airflow in the environment makes the control harder than expected. Duckiefloat also gets stuck in some locations like constrained path, slopes, or turns. We found that communication is a seriously tough challenge in such an environment. Last, the preparation of robots before sending them to the environment takes us more time than expected which results in less time for robots performing searching.

Team Robotika: Since we participated in the STIX training round a few months earlier, there weren’t many surprises waiting for us in Pittsburgh. The biggest challenge turned out to be the lighting situation inside the mines. The existing illumination consisted of a number of bright dots (probably LED bulbs) which blinded cameras on some of our robots. We had to run into the nearest hardware store and purchase additional lights for our subsequent runs. We also had trouble with the delivery of our lithium batteries from overseas.

How have you been preparing for the Urban Circuit?

Team Coordinated Robotics: Our main focus over the past few months has been growing the team. Coordinated Robotics was one person at the system track Tunnel event. We have had over 30 people help for the system track Urban event.

Team CoSTAR: The prep for the Urban Circuit has been somewhat analogous to our prep for the Tunnel Circuit, but there are a couple of key differences. First, we were not able to acquire testing sites that will be a good simulation for the Urban Circuit. Second, we decided on some pretty massive change in strategy and hardware platforms late in the preparation phase. We’ll see at the contest if this pays off.

Team CERBERUS: This year we’re focusing on both the Urban and the Cave Circuit. For the Urban Circuit, we focused on the existing systems emphasizing on their overall integration readiness and their capacity to address the challenge of multi-level exploration both using legged and flying robots. So both climbing and flying within staircases became a central part of our work.

Team CSIRO DATA61: Every Friday we do a weekly test, deploying multiple robots in a specially designed tunnel system onsite at CSIRO. Thanks to Hutchinson Builders in Brisbane, we have also had the chance to do bigger field trials in their underground construction areas. The local navigation stacks on both our UGVs and Emesent Drones have been significantly improved since the Tunnel Circuit, enabling much better autonomy.

Team CTU-CRAS-NORLAB: A multi-floor environment and going up and down are the main new challenges. We had to advance our exploration algorithm to 3D and autonomous robot locomotion for traversing staircases. Also, the gas detection and localization problem is new. The gas is not as clearly localized as solid objects, which required extensions of detection reasoning.

Team Explorer: Practicing. Specifically, we have spent a lot of time preparing for terrains that are more “three-dimensional” in nature than tunnels.

Team MARBLE: The main thing has been engineering new platforms and algorithms to handle the increased mobility challenges the Urban Circuit presents. Stairs and multi-level deployments increase the difficulty level significantly, and this requires novel insights backed by lots of testing and validation to field a successful team. We have been fortunate to have great collaboration with local first responders and businesses that have provided locations where we can test and improve our systems.

“The interesting part of the real competition is that it’s time constrained, with high pressure and unpredictability. So we have trained ourselves like a real search-and-rescue team so that we and our robots can cooperate perfectly during the mission.” —Eric Lu, Team NCTU

Team NCTU: For the last few weeks, we’ve done multiple experiments and even mock scored runs to not only enhance the capability and robustness of our systems, but also to train ourselves to be more familiar with the whole process of the competition. The interesting part of the real competition is that it’s time constrained, with high pressure and unpredictability. So we have trained ourselves like a real search-and-rescue team so that we and our robots can cooperate perfectly during the mission.

Team Robotika: Even though we already won the “Most Distinctive Robots” award at the Tunnel Circuit, we are actually introducing three new platforms to address some of the specifics of the Urban Circuit, namely the presence of stairs and narrow passages.

What will you be doing differently in terms of hardware, software, and/or strategy?

Team Coordinated Robotics: The main hardware change for the Urban Circuit is the addition of four new ground vehicles. Two of the new ground vehicles are smaller skid steering platforms that Sequoia Middle School students helped with. The two larger ground vehicles are being developed by CSUCI. We expect that the ground vehicles will complement the quadrotors. The different platforms will have different sensors and different approaches to localization that should improve our chances at the event.

Team CoSTAR: We will have a “hardware surprise” at the contest that I can’t divulge right now, but 50 percent of our deployed vehicles will be different than the Tunnel Circuit. In terms of software, the basic architecture is the same. But Ben Morrell has been spearheading many upgrades and refinements in our mapping/localization framework and hardware. Also, a fair amount of time has been spent on upgrading our traversability analysis systems to handle stairs. Of course, everything had to be modified/upgraded to handle the multiple floors in the Urban Circuit.

“We will have a ‘hardware surprise’ at the contest that I can’t divulge right now, but 50 percent of our deployed vehicles will be different than the Tunnel Circuit.” —Joel Burdick, Team CoSTAR

Team CERBERUS: Our robotic hardware is largely the same (but improved), while our software is extended to deal with multi-level environments and common mapping. Our communications hardware is new, and we are confident that the new approach is much more powerful as it involves smaller nodes and optimized bandwidth management. This required a lot of new effort but we believe it will pay off in all the upcoming Circuit events.

Team CSIRO DATA61: Our strategy has become more autonomous and distributed. Each robot has a perception backpack (with integrated lidar, cameras, IMU etc.), which share map “frames” between all robots. Each robot computes a live global map containing all the available information from the other robots. This has allowed us to share objects, goals, and frontiers (the boundary of unexplored space) between the robots. As a result, if a robot does not get a command from the operator, it explores a new frontier from the shared map.

Team CTU-CRAS-NORLAB: We opted for an evolution rather than a revolution. We included a gas sensor, newly emerged consumer 3D cameras, full 3D lidars for UAVs, also new batteries were needed as we were almost at our limit during the Tunnel Circuit. We worked out our strategy for keeping the robots connected, using robots in between. We analyzed database information synchronization and are attempting to save bandwidth.

Team Explorer: We are planning to bring some new hardware, both aerial and ground, but for the most part are keeping our system nearly the same as Tunnel. We’ve made some improvements along the way, so we’re (anxiously) excited to see how things work at game time. Roughly the same thing can be said for software. In terms of strategy I think we will need to largely “play it by ear” based on how the competition goes. Nobody yet knows what DARPA has planned for us, so I think being prepared to be a little nimble in your approach is likely the correct move.

Team MARBLE: We have added several new platform types and updated our existing autonomy and perception stacks to handle the 3D aspects of the Urban test environment.

Team NCTU: For the hardware, we installed more varied sensors on both Husky and Duckiefloat. We have millimeter-wave radar sensors on both robots to deal with lidar/camera denied situations. For payload constrained platform like Duckiefloat, radar is even more important since it provides point clouds for geometry sensing but at much lighter weight. We also improved the anchorball solution we had last time. As for the software and strategy, we would like Husky and Duckiefloat to cooperate more to overcome mobility challenges. We have a tethering system that Husky could use to maneuver through the environment while Duckiefloat can travel to different levels.

Team Robotika: Our navigation will more heavily rely on cameras as sensors (as opposed to lidars). We have improved our central control system and communication technology to make it easier for the human operator to manage the robots inside the underground space. Finally, as we are also participating in the Virtual Track of the Urban Circuit, we used our experience from that world and from those runs for developing new strategies to be used by the physical robots in the Systems Track.

DARPA Subterranean Challenge ]

The experience of inner speech is a common one. Such a dialogue accompanies the introspection of mental life and fulfills essential roles in human behavior, such as self-restructuring, self-regulation, and re-focusing on attentional resources. Although the underpinning of inner speech is mostly investigated in psychological and philosophical fields, the research in robotics generally does not address such a form of self-aware behavior. Existing models of inner speech inspire computational tools to provide a robot with this form of self-awareness. Here, the widespread psychological models of inner speech are reviewed, and a cognitive architecture for a robot implementing such a capability is outlined in a simplified setup.

With the DARPA Subterranean Challenge Urban Circuit kicking off on Thursday, we made sure to have a chat in advance with Dr. Timothy Chung, DARPA SubT program manager. We last spoke with Tim nearly a year ago, just after SubT was announced, to get his perspective on the Subterranean Challenge in general, and we took the opportunity in this interview to ask about how DARPA felt about the Tunnel Circuit, and what we have to look forward to in the Urban Circuit.

For more details about the SubT Urban Circuit, make sure to check out our course preview post, and check back tomorrow for a Q&A with the systems track teams.

This interview has been edited for length and clarity.

IEEE Spectrum: What turned out to be the biggest challenge for teams during the Tunnel Circuit?

Tim Chung: Where to begin? And I say that from the perspective of every obstacle being a good learning opportunity for the teams! For example, mobility was a huge challenge for the teams—going from cement and gravel into areas with rails that were problematic, to muddy places with big ruts… And that diversity was just on one level of the mine. But I think what we saw from the robots, at least from the ground vehicles, was the ability to deal with the unexpected, at least as far as the terrain was concerned.

As you saw, there were some different platforms that came up with different ways of solving the terrain challenges as well, including wheels on legs, and that was another great opportunity to see under what conditions wheels make sense, or legs make sense, or treads make sense. Again, that was only with the ground vehicles, and there were a lot of other ways to address the terrain, as with air vehicles, but they had their own sets of challenges. Mobility was a good reminder that the real world doesn’t have well-defined terrain parameters.

What impressed you the most during the Tunnel Circuit?

First off, I think the energy and enthusiasm that the teams brought. As a roboticist myself, I remember getting sad at times when my robot would break, but the fact that the teams were so enthusiastic about learning in this challenging environment was a really great takeaway. From a technology point of view, I was very impressed with how teams were able to deal with a lot of the uncertainty of the course itself, in terms of—I won’t call them hacks, but overnight fixes. The ability to say, “We saw that this other team had this innovation, let’s capitalize on that and see if we can retrofit our system to be able to do that as well.” 

“Increasing the autonomous capability of these robots can overcome the limitations of some other technology areas. An example of that is communications—we know communications are really hard. The teams definitely experienced this, and they were able to develop autonomous approaches to address or mitigate some of the communications deficiencies”

It turned out that both speed and agility are important, but so is deliberate decision making. I think what that speaks to is that increasing the autonomous capability of these robots can overcome the limitations of some other technology areas. An example of that is communications—we know communications are really hard. The teams definitely experienced this, and they were able to develop autonomous approaches to address or mitigate some of the communications deficiencies.

Can you share any feedback that you got from teams about the Tunnel Circuit?

The primary feedback that we’ve received has been very positive—teams have expressed that the courses were really, really hard, and hard in different ways than they anticipated. Essentially, that DARPA has set a very high bar. But teams are pleased that the bar is so high. I think what that says to me is that we’re satisfying the role that DARPA plays within the robotics community, of setting the bar really high while inspiring teams to reach it, but I also think that it shows how the community is thirsty for the opportunity to reach a new level of technology. From that perspective, the feedback was, “It was too hard for our robots, but something we still want to overcome.” And that’s great, because ultimately, DARPA’s role is to give the community a chance to surprise itself with how innovative it can be.

Did the results of the Tunnel Circuit cause you to recalibrate the difficulty of the Urban Circuit?

I can say pretty succinctly that the answer is no, in the sense that each of these environments are so wildly different that it’s difficult to say something like, “Tunnel was a difficulty 9, and Urban is going to be a 10.” And the type of challenge elements that we’re introducing in Urban that weren’t at Tunnel also make that comparison hard. The verticality with stairs, for example, will pose totally new challenges for the teams. So I’d say, we didn’t change the difficultly level because the environment just does it intrinsically for us.

In terms of expectations, we know that the teams have come to appreciate what DARPA wants to see out of the challenge, and so they’re better equipped from that perspective. We’ve adjusted by introducing new elements into the environment that will keep them on their toes for sure.

Can you highlight some of the fundamental differences between Tunnel environments and Urban environments?

The environments that are built for people tend not to have been designed with robots in mind, so having these types of systems—systems that can make an impact in everyday lives, and particularly in the first responder case we’re interested in—will necessarily mean that they’ll have to be able to operate in these challenging urban settings where we expect people to be.

Verticality is certainly one of the key distinctions that I think will be really exciting for the Urban Circuit. In Tunnel, the emphasis was not on verticality given the single level in the courses. What we’ve previewed for Urban is that there are sections of the courses where having the ability to navigate vertically will be very beneficial to teams.

Verticality includes not just the need to move in three dimensions, but the ability to perceive in three dimensions as well. For example, we place exit signs above eye level to help them stand out, and we look for indicators like handrails and signs. I think context plays a big role in these urban settings, which we take for granted as humans, but if you’re thinking about using these environmental cues to aid in localization and navigation, there may be some interesting advantages.

Are there other particular challenges for robots in urban environments?

I can give a couple of examples where there are some unique elements. Most of our construction materials tend to involve rebar, or metal in general, that are very bad for communications for sure but also for things like compasses. Another example is the regularity and the oftentimes featureless qualities of urban environments make it difficult for feature-matching technologies to help you with your localization. When one door looks like another door, or a wall looks like every other wall—humans may like that in our urban architecture, but for robots, the lack of discriminating features makes many localization approaches challenging. 

Why did you choose this specific urban environment, as opposed to something like a subway station?

We can certainly appreciate the subway scenario as one attribute of urban underground settings, but as we did research and worked with stakeholders, and also visited a number of various sites in urban centers, I think we learned that there’s just so much to the urban underground, so many types of environments, that to only use a subway wouldn’t have allowed us to implement many of the features that we were interested in for the SubT challenge. Going beyond that, I think the environment we found in Satsop is quite representative of many of the elements of the urban underground.

It’s also a place where we know first responders like the Seattle fire department conduct urban training. Being able to find a place that has value for first responders also contributed to our selection of the site, since it had that realistic feel for what they’re interested in and training for.

Can you talk a little bit about what DARPA’s position is on teams using tethers to communicate with their robots during the SubT Challenge?

[Laughs] What does DARPA have against tethers? I’d say, the idea here is that we’re interested in the capabilities that can be manifested throughout the challenge. While the specific implementation or approach may be of interest from a technology development perspective, it’s really about the capability. And so, if the best capability at present is to insert a robot that is tether-enabled, I think there’s merit there. The environment, however, is far more expansive than would allow for tethers to be the only solution. 

“If tethers showcase the ability to do additional breakthrough capabilities, then bring it! If there are ways in which breakthrough technologies allow us to work without tethers, and that opens up the spaces in which these types of systems can work, that’s another breakthrough that we’d be very excited to see”

Our perspective, which is to neither encourage nor discourage tethers, is partially in response to teams inquiring outright if tethers are permitted, and we wanted to say that we’re not trying to prescribe any solutions, because that’s not what the challenge is designed to do—we’re trying to be as open as possible. And it’s entirely possible that tethers will work in some cases, as they might have in tunnel, and it’s entirely possible that they won’t in other cases. If tethers showcase the ability to do additional breakthrough capabilities, then bring it! If there are ways in which breakthrough technologies allow us to work without tethers, and that opens up the spaces in which these types of systems can work, that’s another breakthrough that we’d be very excited to see.

Can you tell us what we can expect for the DARPA livestream of the Urban Circuit competition?

One of the things that we took away from the Tunnel Circuit was our desire to help narrate and explain and involve the viewing audience more. We intend to have both narration of the action and commentary where applicable. We’re opening the aperture because there are a lot of really interesting features that we’d love our viewing audience to be able to witness, and that includes robots traversing interesting features and elements within the course, so you can look forward to more of that.

You should be able to follow along with what’s going on in the course a fair bit better, and beyond what we saw in Tunnel for sure. We’re balancing of course the sensitivity of the competition, but there’s just so much robot excitement to be shared, and I’m excited that we’ll be able to do more of that for the Urban Circuit.

What are you personally most excited about for the Urban Circuit?

There’s just so many things— I’m super excited about how we set up the team garages at the Urban Circuit. It’ll be like pit row, in a way that really highlights how much I value the interactions between teams, it’ll be an opportunity to truly capitalize on having a high concentration of enthusiastic and ambitious roboticists in one area. 

And the teams are already amped up. They’ve gotten a taste of the difficulty level, they’ve gotten a taste of DARPA’s aspirational goals, they know when DARPA says “challenge” we mean “Challenge” with a capital C. From a technology perspective, I’m anticipating these teams are bringing their A++ game now, not only revising their strategy but building new robots, and that’ll be really exciting too.

[ DARPA SubT ]

Pages