ROBOTS (Sabine): Hi Nils, welcome to Robots. Can you introduce yourself to our listeners?
Nils: My name is Nils Napp. I did my PhD at the University of Washington working on self-assembly and coordination algorithms for robots, and now I’m a Postdoc at Harvard working in Radhika Nagpal’s group on bio-inspired control algorithms for robots.
ROBOTS: I know you’re interested in amorphous materials. Why amorphous materials, and what are they in the first place?
Nils: I’m interested in amorphous materials because they are ubiquitous in nature. An amorphous material in our book is a material that doesn’t have a well-defined shape until it interacts with the environment. ‘Morphous’ means shape, and ‘amorphous’ means not having a shape.
When a robot is working with amorphous materials, it doesn’t always know what the final shape of the material will be, but it still [needs to be able to] plan when using these types of materials.
ROBOTS: You mentioned bio-inspiration before … I imagine there is some bio-inspiration here?
Nils: Yes, absolutely. There are lots of different animals, such as termites, that make really large structures. You have millions of little termites and they use these little balls of mud to make very complicated shapes, and they do that very reliably. We would like to do the same thing with robots.
We think that some part of this reliability comes from the fact that the termites can build with a material that conforms to irregular obstacles. By using these types of materials with robots, we would like to be able to build structures over really irregular obstacles.
ROBOTS: Can you give us a concrete example for the human world where this could be useful?
Nils: One of the target applications for us is to be able to work in unstructured environments like disaster areas, where there is a lot very irregular terrain and loose rubble. It’s pretty difficult right now for robots to operate in these environments, so we would like to be able to send robots into disaster areas and have them build structures. Some example structures might be access ramps, levies or support structures for things that are about to fall down. [We want to] have robots be able to figure out how to build these [structures] autonomously and reliably.
ROBOTS: I’ve seen some of your robots and they’re pretty awesome. Can you tell us a bit more about what they look like, what they can do so far?
Nils: The robot that I’m working on for this project is basically like a little car … with wheels or treads … and the idea is that we want it to be able to drive over rough terrain [so that it can] work in these unstructured environments. On the front of the robot there is deposition mechanism, and we’re using polyurethane foam …
ROBOTS: [interrupts] What is that?
Nils: … It’s like insulation foam: it comes out liquid and then it expands a whole bunch, and it’s very rigid once it cures … The idea is that we want this robot to be able to put down as much material as possible. We are using foam because it expands a lot, so you can have small tanks and then build large structures. There’s a foam print head on the front of this robot, so it’s able to put depositions in front of it.
ROBOTS: I’ve also seen toothpick-throwing robots and sandbag-holding robots. Can you tell us about those?
Nils: Yes, we looked at several different materials that would qualify as amorphous from our perspective… We looked at a mechanism that would shoot toothpicks that were covered with glue; they would form piles and conform to whichever shape they were falling on. We also used sandbags for the same reason: when you put it down, the sandbag moves around a little bit and conforms, and then we can stack those on top of each other to make larger structures.
ROBOTS: Sandbags are actually things that we use in reality, for example to build walls in areas that might be flooded …
Nils: Exactly. In flood zones, people build levies out of sandbags all the time. That was one of the first things we tried as an amorphous material because it’s something that people already use.
ROBOTS: You said that it’s difficult to work in these unstructured environments and it’s even more difficult to work with these amorphous materials. What are some of the challenges that you face?
Nils: There are several challenges … and maybe you were just hinting at one of them by pointing out all the different materials that we’ve tried … One challenge is mechanical: handling these materials and deposing them.
Another challenge, and the one that I’m working on, is on the planning side. [Even though] these robots don’t quite know what the final shape will be, we still want to be able to build large structures reliably. For example, if we want to make a very large levy, we will have some idea of what this levy should look like… but there’s still a lot of uncertainty of what the exact shape will be because these little components will deform once they’re put down.
I’m working on making sure that, from the robot’s perspective, it can keep this large goal in mind and manage the uncertainty of all these individual building steps.
ROBOTS: What types of structures have you built so far?
Nils: My work currently focuses on ramps, and the reason for this is that it’s an intuitive shape and [building a ramp is one of the first things] you would want to be able to accomplish in [an unstructured] environment because it allows a robot to get from one point to another. [Ramp-building] is a useful behavior, and it’s also something that is pretty easy to plan.
ROBOTS: Tell us about the planning that goes into building a ramp …
Nils: The ramp-building algorithm that we have figured out so far is pretty simple. The idea is that we have some way of measuring whether or not a robot can move past a certain obstacle. Robots then try to make progress in [a certain] direction, and as soon as they encounter an obstacle, they deposit some amorphous material on this obstacle to make it easier for the robot to drive past.
This is a very small-scale type of planning, because all [the robot] needs to know is whether or not it can make progress in its very small planning horizon, and then make a deposition to make it more navigable.
ROBOTS: Could it get trapped in there?
Nils: It can if [the robot] makes the wrong depositions. We figured out a good way of putting these small depositions together to make sure that the final structure is something that the robot can drive on.
ROBOTS: Can you give us some intuition for how your final algorithm works?
Nils: The robot tries to make progress until it finds one of these obstacles and makes a deposition. Then it backs up to make sure that the deposition hasn’t created another obstacle. By doing this repeatedly, the robot guarantees that it will eventually make a large ramp, even though in the short term it might be making more obstacles for itself.
You have to allow for that because there are some types of obstacles that robots can’t get rid of in a single operation. If I have a very large cliff, and a robot can only make small depositions, then it’s not really possible for the robot to make one deposition to get over this large cliff. It needs to create these intermediate steps of making a smaller ramp (that becomes an obstacle because it’s too steep), and then responding to that obstacle by making yet another ramp, and so on, and so on.
ROBOTS: If there’s no global plan, and the robots use local information, how easy is it to scale this to multiple robots?
Nils: This a good leading question. It’s pretty easy to scale to multiple robots, exactly because it is all based on local information. Robots don’t need to share a consistent global state the way they would if they were following some kind of global plan, because they don’t need to know whether the [obstacles] were built by other robots or if they were preexisting. [Each] robot responds to its environment in exactly the same way.
You can take multiple robots that all respond to the environment, and they communicate with each other by all interacting with the same environment. If a robot builds part of this ramp and then runs out of material and needs to go get more, a different robot can come along and seamlessly continue the work because the robot doesn’t differentiate between the structure it’s trying to build and obstacles.
ROBOTS: Have you looked at shapes other than ramps?
Nils: Yes, we’re currently working on coming up with an algorithm to take arbitrary shapes and then break them down into a series of ramps. Because we already know how to make ramps, we can use this algorithm that is very scalable and robust to make any shape that we want. For example, if you wanted to build a levy (which sort of looks like a ramp, but not quite), then you can break it down into a whole bunch of ramps, run the same algorithm and build it.
ROBOTS: You have these robots, you have algorithms … What would it take to bring these to the field? To actually have them build something in reality?
Nils: Right now the robots are testing the pieces of the algorithm. To bring these to the field we need to make a larger, more robust robot, and probably look at different materials. We chose the materials we’re working with right now because they’re easy to work with and they have the properties that we want, but if you tried to do this on a large scale you would probably make different choices. Your cost would be more of a consideration: you would probably use something that is based on sand or dirt – small granular materials that are easy to find in the field.
ROBOTS: What would you do as a next step?
Nils: For this project, it’s working more on the robot and making it work well in the lab. I think there are a lot of really interesting theoretical next questions. One that I’m particular excited about is that this type of planning (and managing the uncertainty of what the shapes are that you’re building with) might allow you build with found materials in a way that would be really powerful…
If I have a robot that doesn’t need to carry all the material that it’s trying to build with, but instead could look for rocks or rubble or [other] objects that it finds in its environment and then put these together and rearrange them so that only a small amount of the material that it’s carrying is needed to make a really large structure — that would be something that would be very useful, especially in situations like disaster areas where there might be a lot of loose material.
The really big challenge in doing that is: you don’t really know what you’re going to find, so it’s very hard to plan ahead and say, ‘I need exactly this type of rock or that type of rock, and then I can stack them together to make the structure that I want to build.’
[By studying] amorphous materials, we already know how to build things without knowing the exact shape of the [smaller] building components. [We are hoping to] use basically the same plan using rigid materials … but the uncertainty comes from the fact that we don’t know the exact [nature] of these rigid materials that we’re going to find in the environment.
ROBOTS: That’s fascinating. It’s completely opposite to the way people build things now, where you have a plan, everyone knows what they are doing, and they use very structured material. Do you see some tradeoffs there?
Nils: There are obviously tradeoffs. Modern structures and buildings are incredibly efficient in their use of material. By having an I-beam you’re optimizing the stiffness of this beam and using this expensive raw material in a very efficient way. Construction techniques for building houses are optimized for time, because they’re very labor intensive and that’s very expensive. The types of things that I’m working on are not aimed at replacing those types of construction — they’re aimed at enabling construction in areas where automation has been extremely difficult.
Robots have a hard time operating in unstructured environments like disaster areas, but those are exactly [the kinds of areas] where you want robots to do work because often it’s dangerous for people to be there, or this work has to happen very quickly. It would be nice if you could just send in a team of 100 robots instead of having to look for volunteers that can work 24 hours a day to make something very quickly. It would be much nicer if you could have your robots in a warehouse and, when you need them, you send them in to do the work [without having to] put people in harm’s way. [Modern] constructions are not really optimized to be built that way.
ROBOTS: Where do you see yourself in five years?
Nils: Hopefully I’ll have a bunch of these robots doing work for me, I’ll see some of these ideas play out, and that will spawn a whole bunch of other ideas and interesting things to research.
This project started by looking at termites, and obviously what I’m working on now looks pretty different. One of the things that is really exciting about robotics research — and research in general — is that you work on something for a while and then suddenly you make all these very interesting connections … and then you get to follow up on those. In five years I’d like to have the implementation done on this project; where the theory will lead from there is hard to tell.
ROBOTS: Do you think you could tie back to biology one day?
Nils: For this project, probably not. I think where engineering and biology have a really good synergy is that, as engineers, we train to be problem-solvers. And we have all these tools we’re trained in using because, especially as roboticists, we have to work with sensors and noisy sensor information all the time.
There’s a huge opportunity for engineers to help biologists probe their systems better. If you talk to biologists, [they will say that there is] often a huge hurdle in getting data, managing data and creating the tools [needed] to get a particular kind of data, and that they don’t get a lot of training [in this area with a] biology background, but that’s exactly the type of thing that we’re trained to do as engineers.
I think there’s a lot of technical expertise that can flow back from engineering robotics to biology. And then by working with these systems, there are lot of interesting ideas and algorithms that come back to the engineers.
I do think there is a really strong synergy and great opportunity for collaboration, but I’m not really sure the algorithms that we come up with directly feed back to biology.
ROBOTS: I know you’re also working on synthetic biology. Do you want to tell us a bit about that?
Nils: The particular project I’m working on is trying to implement some of the sensor fusion and signal processing algorithms that we typically use in robotics on a molecular scale, so that you can put them inside cells. One of the problems when you try to design these synthetic circuits and put them inside cells is that there’s actually a lot of noise in the signals that you’re getting, and there is a lot of uncertainty in what exactly these individual pieces do. It’s really frustrating when you’re trying to make these things work.
But as engineers we have a lot of experience in working with noisy sensors, and in trying to figure out how to take a whole bunch of noisy sensors and turn them into one good sensor. It’s called the ‘sensor fusion problem.’ I’m working with a collaborator in Computer Science, Ryan Adams, who is an expert in this type of modeling and processing, and we’re trying to figure out how to take these ideas and shove them inside cells, so that synthetic biologists can benefit from all the tools that we have developed in engineering (and have specifically developed in robotics) to manage noise and uncertainty.
ROBOTS: I know a couple of synthetic biologists, and I know it’s a pain to make the system work the way you want it to work, so being able to somehow fuse these sensors is really interesting. Are you thinking of fusing the sensors? Are you looking at an individual bacterium or cell? Or are you looking at the population and figuring out how to fuse the sensing at the level of the swarm?
Nils: The algorithms are the same. It really depends on how you think of computation, where computation is taking place. Right now, we’re trying to make these systems as small as possible, and implemented as chemical reactions that will go inside a single organism… inside a single E. Coli, for example, or a single cell. If you can do arbitrary computation, it is very easy to imagine how you would take this same type of algorithm and then implement computation, where one cell does a little bit of computation and another cell does another bit of computation, and then they exchange some information and as a swarm do intelligent sensing.
ROBOTS: So the first step is to look for circuits that would reduce noise, for example, at the level of the cell?
Nils: Yes, that’s what I’m working on, but there is obviously a lot of work [going on] in robotics and sensing with robot swarms or sensor networks. If you know how to do computation in cells, it’s certainly feasible to take those same types of algorithms and try to implement them in cells.
ROBOTS: Thanks, Nils, for being here with us on Robots.
Nils: Thanks for having me Sabine.