August 9th, 2012

Transcript – Field Robots

Field robots

Interview with Salah Sukkarieh

ROBOTS (Per): So welcome to the podcast. We are here at the Australian Centre for Field Robotics, and we are going to start with an overview of the centre and its activities.

Salah Sukkarieh: The Australian Centre for Field Robotics has been around for about 15 years. It is a research institute within the University of Sydney. There are just over a hundred researchers. At the moment we have about 55 PhD students, there is a full time academic staff and there are a number of research fellows, about 40 research fellows across the group, and we work in various areas. As the name says, we are a field robotics group so we like working with robotic systems that work outdoors, predominantly, in the field and that causes a lot of different problems when it comes to robotic work, and we have to deal with many different research areas that you would not normally find in an indoor application. And our work is spread out in various areas, so we work in marine, in aerospace, ground vehicles; we work for mining, stevedoring, agriculture, environment work. We are starting to dabble into social robotics, both media robotics as well as medical robotic applications. So there is a whole wide spectrum of activity that we deal with. And in all of those activities, we generally have research that focuses on probably 4 key areas, and they mix and match in different ways. One of them is perception, so how does a robot understand its world and deal with its world, especially in a dynamic world environment, unstructured world environment. Control or decision making is probably the second element and very much about how you deal with making decisions and actions in a very stochastic, uncertain world. The third area is a relatively recent area which is in machine learning, data mining, and how that impacts robotic technology, so not just machine learning algorithms but using them in real time, using them within the environment. And the final area is in what we would call systems or modeling. So how do you start to build or construct complex autonomous systems and their interaction with other humans and other systems. These are systems with systems concepts, and a lot of our industrial applications deal with that kind of an area as well.

ROBOTS: Could you tell us about any particular project you have been working with here?

Salah Sukkarieh: Okay, so we have, if I look at the aerospace world, we have been working with robotic aircraft now for about 13 years, in different ways, both what we call fixed wing and rotary wing, so just standard aircraft that you would normally expect a well as autonomous helicopters. The work started a lot in security and defense in the early years, mainly dealing with multi-robot systems, how would they share information statistically, build a common view of the world. That moved on to active decision making so once we got this common view of the world, how do we even make them informed decisions, no operator input, to do things such as search, or map, or track. The technology then was very expensive for the ordinary user, the civilian user, so that is why it was mainly driven by government and defense and so forth. But then about 7 years ago, you could really see the transformation of the technology. It got cheaper. You could start to buy autopilots off the shelf; you could start to buy robotic platforms off the shelf, so the civilian industry, or the civilian area, started to get a lot more interested in the application so we then started to get some research work from the government to look at robotic aircraft for environment mapping, weed detection, animal tracking, these are just some of the examples that we have had in the past. Completely different philosophy to how you would approach the problem, while in the first group of activity that you worked on, you had a very technical or a very educated industry partner, people were very technically orientated. They could understand the technology a lot better; they knew what the risks were; they knew what the potential was, but they also knew what the limitations were very easily. You start to talk to the civilian areas about this that they assume that these things could tackle very large problems, and it was a lot more of an education to what the limitations were and so forth, and trying to drive the cost down. And a lot of that technology now we are starting to see the transition into civilian aviation, commercial aviation realm, so even though they are not interested in automation, a lot of the technology, sensors, algorithms, and so forth that we had developed in the past are now kind of starting to feed into those areas. So you kind of see a train of different activities both from a very technical savvy audience, being defense industry, to an audience that just wants to see something work because they think it will solve all their problems, but why does not it work properly, etcetera, to another audience which is interested in not so much in the automation but in the technology spin offs that came out of the automation process back into aspects such as commercial aviation. That is one example. Another example is mining. Australia is a very large place where most of the population lives on the Eastern seaboard, but all of our mineral wealth is located in the centre and out to the west. Not many people want to work out there. With the mining boom that we have had recently over the last 5 years maybe, there has been a lot of interest from mining companies to actually look at automation to help solve both their labor but their efficiency costs as well, and so you can go out now to western Australia, you can see a large amount of activity there with autonomous haul trucks, and people starting to think about in 10 years time what is the mine of the future going to look like, and we have had again over the last 15 years a lot of involvement with a various mining companies. At the moment now we have a sub-centre which is called the ACFR Rio Tinto Centre for Mine Automation, and that has about 15 research fellows, just as many PhD students and a number software, mechanical, [and] electrical engineers, and we work with Rio Tinto looking at some of the aspects about solving some of the concepts that they are looking at for the mine of the future as well. Environmental applications is another one, so in addition to the robotic aircraft activity that I mentioned before, there is the similar type of activity in the underwater realm, so we have an underwater robotics or marine robotics group and they work a lot with scientists, and marine biologists, and government agencies to look at mapping, predominantly coral reefs and classification of corals and looking at change detection over a number of years, to see whether things such as climate change or pollution or whatever might be affecting it. And that helps in terms of being able to deliver imagery that they could never get before, as well as help with reducing the amount of imagery that they need to look at by looking at automatic classification, identification of that imagery, and helping marine biologist to be able to identify changes in the environment. So that is just some of the examples I think that look at the activities that we have here.

ROBOTS: You said that the centre is about 15 years old. How was it started?

Salah Sukkarieh: At the University of Sydney there is a group within the School of Aerospace, Mechanical and Mechatronic Engineering, so back in 1991, something like that, there was a mechatronics program started, there were 3 full time academics involved in that activity. They had been doing some robotic work in various areas. In ’95, ’96, the group appointed a professor, Hugh Durrant Whyte, who came from Oxford, and his drive I guess was to actually look at field robotics, he’d started to look at machinery outdoors, and robotic systems outdoors, so the group started with that. There were 3 or 4 PhD students that came on board and their first flagship project I guess was (so at that time I was a PhD student), the first flagship project was probably the automated straddle carrier project which was looking at port operations and automation of port operations. So we did a lot of research work in the area of navigation and control. The next flagship project was probably the multi-UAV multi robotic aircraft activity which was sponsored by BA systems. And with those 2 flagship projects, it kind of grew, you know the number of PhD students, research fellows, engineers, and slowly one thing led to another; one project kind of [?] showed your worth and what the centre was capable of, and that kind of drew in the next one and the next one, and kind of rapidly grew to what we have now, which is just over a hundred researchers in the area.

ROBOTS: So you are doing all this now, and you of course are going to continue to do that, where do you see the new exciting projects for the future? I mean, you are doing what you do now, what projects are you starting now or would you like to start now?

Salah Sukkarieh: I think the environment and agriculture are probably big areas, at least in Australia anyway, are big areas that we think robotics will have a very positive influence on. The problem that we have now is that labor is costly in the area. Australia is competing against much more labor intensive, but cheaper labor costs, within the Asia and South America areas so the agriculture industry has to kind of compete against that, and at the same time we are not getting enough labor in that area, because they are either not coming to it or they are going to mining or something like that so there is not enough labor coming to that area. So being able to do things where you can run farms a lot more efficiently and effectively, know more about your farm, collecting data, harvesting, pruning, etcetera, so that whole process I think just has a fruitful, pardon the pun, but a fruitful area to deal with in the future. The environment I think is another big thing for Australia. There is massive land mass, water mass that we need to deal with and not enough in terms of remote sensing and manipulation or sensing the environment, but also touching the environment in various ways, whether it is ground or air or underwater, and knowing what is going on both in terms of ecological models that help scientists, but also in terms of just land management uses and things like that. So that is one big area. Another big area would be intelligent transport systems, it is looking now more at robotic systems but in a context, so everything is kind of transitioning away from the robot itself, but then how is the robot used, and in what context is it used, and how does it kind of fit within that bigger picture, and that draws on other challenges, human robot interaction, human robot interfaces. There is one thing, many heterogeneous robots working together, how do you deal with that, cooperative control, coordinated control? Other aspects such as architecture, still there are robots that talk to each other. There are aspects like that that you need to deal with. So I think they are kind of like the big areas. But then also you start to look around some of the other transformational technologies like cloud, machine learning, etcetera, so how do they then also transform robotics and where the future of robotics is going to, so I think they are trying to understand those areas and how those areas fit in along the way. But Australia has a very unique area, whether it is better or not is a different question I think but a unique area in the sense of we are just as large as the US or Europe in terms of land mass. We have a large economic zone, in terms of the water mass around us, but we only have 24 million people, and they all like living near the beaches on the east coast. So there are a lot of challenges there but a lot of unique opportunity for robotics, so I think there will always be things such as in mining and in defense and border protection in environment, there will always be those activities that I think will drive it. But I think it is just the transition now of not just the robot but the robot in context.

ROBOTS: So when we talk for instance about using this in farming, are you seeing this as are we going to have a high tech farmer that uses a lot of robotics to monitor his farm and I guess again you have a very unique problem here in Australia, because a farm in Australia can be very, very large if you talking about the cattle farms, there are, I do not know?

Salah Sukkarieh: Hundreds of thousands of acres, massive, yeah.

ROBOTS: There is simply no way you can go out and have a look.

Salah Sukkarieh: That’s right and that is what I think the future will be. So, what I should say there is lots of technology pieces that you can bring together that will allow that to happen, and the know-how is there to be able to do that. In the operational context about a farm, there are many issues that we need to deal with but we know how to build the robots, we know how to deal with communication over large distances, human-robot interactions or human-robot interfaces, or one human controlling many robots. We know how to look at perception issues and that fits into control, etcetera. How it works on a farm is a different question, and that is really where the challenges are now and understanding how you might use a robot in that context, but I could envision that if you were not to worry about money and if you were not to worry about commitment, I guess that people would go in the long run, you know 5, 10 years time, you can start to see autonomous farms in that context. Harvesting is always the trickiest issue but many of the other aspects that you could probably solve quite easily with the technology that we have.

ROBOTS: Which means that you could actually remote into your farm, because, if you use robotics, you can actually live in Sydney, except for the 2 weeks when you harvest.

Salah Sukkarieh: Exactly, and we do that now in mining, we do that now in defense, there is nothing to say that we cannot do that in agriculture as well. There is obviously a mind shift, there is obviously people understanding of how robotics would be used in that context, but you can envision that that is how you would take it. But also, not to forget, there are also the concepts of the urban farm as well, which is a multistory farm, that are operated autonomously as well. It does not have to be I am going out to the middle of wherever to be able to deal with my farms, but in both those areas, what the future of agriculture might hold, robotics is going to have a key element in that.

ROBOTS: You talked about autonomous transportation. Is that like automated lorry transportation or is it cars or?

Salah Sukkarieh: In Australia, I think it is probably more got to do with transport systems that support things such as long haulage operations, mining, …

ROBOTS: The automated rail trains…

Salah Sukkarieh: Exactly or mining vehicles, trains systems, truck, freight, etcetera, so a lot of those aspects I think will be important in the future. And again to support things such as mining, agriculture.

ROBOTS: This leads to a very interesting question that you are probably perfect to answer, and this is just when we step out of the enclosed space and we do what the Centre for Field Robotics, I mean go out there in a nitty gritty world, do you think that how long time will it be, what are the hard questions around actually taking the full step of being fully out there, as you said, interacting with people and other vehicles, are we going to be able to do that soon or is it further in the future, what are the main issues around that?

Salah Sukkarieh: I think it is one of those things where it is not less about the technology it is more about the commitment of the partners that you are working with. Do they want to go down that path or not? Is it easier for them to just segregate an area and just say we are going to do it here because we do not want to deal with all the OH&S issues and aspects like that. Technologywise, yes, there are things that you could throw in now that would help do things such as dealing with human-robot interactions or manned/unmanned or many different types of systems working with each other. They are probably not perfect but if you had the commitment from an individual, or company or whatever it might be, to take it further, you would know what you would need to tackle in order to be able to be able to solve that. So I would not say that the technology is an issue in that particular case, except for little bits and pieces here and there that I think you can solve. It is more of a commitment of whether the collaborator or the partner that you are working with wants to take it down that path, but at the moment now, it is a lot easier to just segregate out an area and just say it is going to be autonomous.

ROBOTS: Certainly. And I guess we also need to work with interfacing with governments and regulations. I guess you had to do all the research around documenting that these systems are in fact safe. We can still claim them to be, and so do you think that that would be a hard part of the problem? Documenting and proving in one sense that they are safe?

Salah Sukkarieh: I think with things like that, you have got to prove both from the levels of science and of research so my algorithms, my sensors, I can prove that if a sensor fails, it can be detected, etcetera, etcetera. Mathematically I want to be able to do that, scientifically, algorithmically, but then also the engineering aspect, so the redundancy that you need to do in terms of hardware and software, to be able to make that work. And then there is proving or validating that whole concept under different scenarios, and, when I am working in the field, it could be fog, smoke and all of a sudden my laser, my vision system does not work and how am I compensating for all that with another sensor, different types of algorithms, etcetera, so it is not just here is the documentation, it is from the beginning when you start to think about the problem, the science behind it all, the algorithms that you want to implement then the development phase and what you need to do in terms of redundancy and checking and so forth. Then the proving on the ground itself under those different scenarios, so you still have to sit down and capture the requirements of the operation itself, and be able to put that in there, because we are not in a position yet where we can say it will work all the time under every situation.

ROBOTS: So time perspective on this part of the research, do you think that if we started this research now, when could we come to the authorities and say that an autonomous vehicle or a robot of some kind is safe to operate in this environment, say on the road or…?

Salah Sukkarieh: We can do that now, but it is a case-by-case basis. So you can do it whether it is in say stevedoring, you can sit down and say ‘this is what it is but how do we deal with that?’ Well the fundamental thing will be making sure that you do not accidentally injure or kill any person. So, as we said before, segregating but then proving that the segregating of that system works, and then back down to the individual vehicle, and how do you know the individual vehicle will not just run away, etcetera, and proving that part works. I think the case with Google, say for example in Nevada, it is a very unique, special case whether that catches on and follows with everybody else is a different story but at the moment now, like you said, 99.9% of the rest of the world has not taken that leap of faith yet. And then whether it works on that particular vehicle itself, in a particular situation, a scenario, whether I can transport that whole thing across and just through it in the middle of Australia and off it goes, but you need to prove that as well.

ROBOTS: We know for instance that they cannot take snow. And they have said that. We do not work well in the snow. And of course this is a thing I have talked to other roboticists about, that, as long as the robot can know its own limitation, we are kind of okay; it is probably not good to stop at the highway, but it is much better to stop than to hit something.

Salah Sukkarieh: Yes, but with all these things, you do not know everything. There are unknowns that can creep up and just pop up in front of you, so that is why I think it is safer at the moment. Without getting scared of what the future science is in that area, let us go out there and actually apply it and make it work, develop it, commercialize it or see it in operation, it is safer to constrain the environment and say this is the requirement that this robot is going to work under and you develop towards that.

ROBOTS: So that is what we are doing today…

Salah Sukkarieh: And I think it will stay like that for some time.

ROBOTS: Yes. So that is probably a challenge to the new students and PhDs to push that limit.

Salah Sukkarieh: That’s right, but then you start to open up to other areas, predictive maintenance or preventative, health monitoring aspects, self-healing or self-understanding, and then provability of your software, and there are many other issues that start creeping into it that fit outside of robotics, and so you know it is a question of how robotics itself kind of pulls that in or whether it is something that is developed outside and pushed in.

ROBOTS: Isn’t this not a good indication that robotics is kind of starting to mature, that we are having these problems?

Salah Sukkarieh: Yes, but we are not, at least when it comes to industry, we are not in a position anymore like before where we would have to say ‘believe us, we know what we are doing.’ Now you can sit back and you can say ‘these are the projects that have been done in the past, these are the capabilities, so now we are interested in how we are going to apply it to your system.’ So I think from that perspective it shows there is maturity in the overall process, we know a lot more now, we know what sensors work, what does not work, we know what algorithms will work under certain situations, we know how far to dabble in the control, in the planning to making it. And we are kind of getting to the other point, where we are becoming a lot more mature as roboticists, in terms of dealing with industry, and saying we are taking the requirements that they are telling us and then housing or developing and designing a robot that works within that particular context. And I think the other aspect of maturity is we know our limitations as well. While before ‘gung ho and off you go,’ or you might be too scared to say anything, and now you are confident in areas and you know where the limitations are and what needs to be done. So you can go to a new project and you can say this would be development, this would be research, this probably does not fit into the category at this point in time until we figure out a bit more about the underlying problem.

ROBOTS: Now we can actually invite industries in and we can work with them to develop, as you say, robots that assist them in whatever they are doing, and we can give them a predictable line, road ahead.

Salah Sukkarieh: That is right. There is a lot more confidence now, when you are talking to industries, saying what you know works, what you know does not work, and what you know you don’t know, and that is probably an open question. You probably do not want to have that in this project now, because it would just take us years and let us just see how we do with this one, or you might be interested in that, or that might be what the government is interested in, hence it is just a separate source of funding and dealing with that.

ROBOTS: Very interesting. So this is actually an invitation to industry out there that has some interest in this and they see potential that want to know whether their project or what they think could be done and also just come to the roboticist and explain your industry and they can say okay we can help you here, that is probably in the future, well this area we should not go to at all.

Salah Sukkarieh: Yes, and I think it requires industry to be confident and understand that there are capabilities and technologies that may benefit. On the other hand, there is the research end or the academic end, it is also them becoming more aware about industry requirements in the sense of it is not about ‘oh look yes, we can solve that, give us this whole bag of money and we will over-engineer a whole system to make it work.’ You have really got to work within the parameters of, not just the mission scope, but everything else, all the other costs that come into the process as well, and so it then is a focus on the research or the academic or whatever it might be to actually understand a bit more about those other aspects to the problem .

ROBOTS: Thank you very much, and I think we will be back for more about all that is going on at the Centre for Field Robotics soon.

| More