ROBOTS: So welcome to the podcast. Today we are going to talk about autonomous vehicles. It is going to be all about autonomous vehicles, but it is also going to be about the story from research to corporation which is not always an easy transition. So can you start with how you got into robotics?
Mel: I was working on some space shuttle programs and was walking down the hall in the engineering building and saw a wheel chair driving by itself; and I was in an undergrad program with research and tracked down this wheel chair and found out who was running it and who its boss was, and then I started to bug him for a job and about 3 months later got stuck in the same elevator with this boss and he finally gave me a job for soldering, and worked my way up in that lab. They paid for my graduate work, and started managing programs for the department of energy, department of defence. John Deere saw some of our publications and said ‘hey, we would like to start working with you at the University’. That went well and they asked us to start a company and so we spun out about 12 years ago and I have been doing robotics across the ground vehicle space ever since.
ROBOTS: So an autonomously, remotely or autonomously driven wheel chair or?
Mel: It was autonomous and so the idea was someone with limited spacial mobility skills could go down the middle of a hallway and it would just keep you in the middle, and so it had ultrasonic sensors bouncing off the walls; and that project did die because they were afraid of someone flying off the stairs and so liability kind of shut that down, but it fascinated me. It seemed to combine the electrical, mechanical and software interest that I had, a little more down to earth than some of the space shuttle work, where it really seemed to be helping people, so I was excited about the potential.
ROBOTS: And then you went into the research and you did that. Can you tell us a bit about what your different research topics were within this autonomous vehicles phase?
Mel: It started out kind of boring with more of just control systems closing the loop on driving actuators, driving vehicles straight, but then evolved into the obstacle detection and safety and that is what brought John Deere in. They wondered if I have a 3 year old running out in front of a tractor, can we stop this thing safely? That is a key determination before we really get serious about this market and so we actually had remote control Styrofoam cars that were the size of 3 year-olds driving at tractors and trying to get run over and that was kind of the research there. It’s just, with the current sensors and the price in points, can we keep something safe?
ROBOTS: And then I guess since this is now out in the field, we are going to talk about that later. You were successful, even if it was a human trying to get run over by the tractor, they would not be, yeah?
ROBOTS: That is amazing that even if you tried to, you could not be, because a human can be pretty tricky if he wants to.
Mel: Yeah, and I think that was 14 years ago. The sensors were definitely more limited than they are now and a child crawling through the bushes, those kind of scenarios, I think you can find a way to get run over, but the scenarios they wanted were ‘I am out doing standard tillage, standard farming operations in an open field, can I get run over?’
ROBOTS: Yeah, yeah.
Mel: We can get into orchards, vineyards and jumping out of a tree, you are going to get smacked by them.
ROBOTS: So tilling and just treating these big open spaces, that seems to be a more manageable problem. But you are also working in orchards and you are working in vineyards and other venues like that, that must be much more challenging.
Mel: Yeah, it is definitely difficult but it is where the bleeding is happening as far as the need for labour. It is very difficult to find workers in the US that can handle the work load, the difficulties in the hot weather and it is also difficult because right now tractors are pushing away the branches as they drive through and so how do you get a sensor that can detect what is really passable and what is not. You do not have radar that you can dial in the density of wood that I care to avoid so there are some challenges there but we have some solutions that we use and it is going well. We are excited.
ROBOTS: So we talked about basically lower crops. Is it also corn and higher crops you are trying to address?
Mel: Yeah, we have done crop spray for example with sprayers that are tall enough; wheels are high enough that they can straddle the corn. So yeah, really farming in general.
ROBOTS: And I guess you have built the common platform because most of the technology is the same between them and then you adapt them to the different vehicles.
Mel: Yeah, the key to us being able to do as many applications as possible is really having those common building blocks and then whatever is specific to that application trying to keep that as small and portable as possible so that we can leverage the work and minimize the effort because profitability is always a challenge if you are customising every application you do.
ROBOTS: Because you have adapted this technology to a fantastic array of vehicles. We are talking tractors, we are talking construction equipment, and we are talking special equipment for police and rescue workers, military. There is a very broad range, that is just amazing but it is possible then to do that at a reasonable cost because you use basically the same technology.
Mel: Yeah, it has really been a saviour for us because, like they say, some economies can crash and certain markets, and if we did not have that diversity we would have been in big trouble especially in the inevitable way the recession took out our mining and our ad business. So having those common building blocks enables that and it has been important for our survivability.
ROBOTS: Yeah. So you started with the agro-business and did you then go to construction or mining… agro-business and mining were the 2 first ones or?
Mel: No, we started in agro and we immediately started writing government grants for robotics development and that started to flow. The agro, we started the business in the agro doing orchard tractors. That got killed after the first year. There were liability concerns, lawsuits, those kind of things, working with a big corporate partner and luckily we had volunteered to do a free lawnmower for golf courses in our spare time in the evenings and so 80 hour weeks we had done these golf course mowers, demonstrated that and the agro dropped out and went to zero completely but we were able to start doing golf course mowing and that really was our first near death experience and it kind of took off from there, a lot more diversity, then we started to do autonomous bacos along with the golf course mowing and then agro picked up again later.
ROBOTS: And then where did mining come in?
Mel: Mining came in about 2004 and 5. That is when, again they were facing some of the same labour challenges we are seeing in orchards and vineyards right now, extreme high cost especially in the outback, Northern Canada, Alaska and so we were approached by the largest copper mining company in the world that is privately held and asked us to start automating trucks and bulldozers so we applied those same building blocks. The building blocks consist of the hardware, electronic component, typically automotive, coupled with the software embedded in those processors and we applied that to the mining and started doing autonomous trucks and dozers and through ’05, ’06, ’07 and ’08.
ROBOTS: And that was for mining or for construction?
ROBOTS: Yeah, okay.
Mel: We had done construction starting in probably ’03, ’04.
ROBOTS: Yeah, that is very impressive. This all came out of research and I think you have some interesting information to share about how you got the research out, how to bring the research with you when you exit academia and want to start a business. I know you have some challenges there. Can you tell us a bit more about what your experiences are?
Mel: Well it is definitely difficult in academics because of the different priorities; they need to publish, they need to tell the world what they are doing. There is a desire to be like Stanford & Carnegie Mellon and really have a large portfolio of products that are out there in the field feeding a lot of money back to the University and so that makes it challenging. I had an incredible Professor at the University who was very supportive and helped that transition. There were certain things we had to start from scratch and rewrite but we continue to face that as we fund students in the University, as we fund University programs. We have good relationships with Utah State University, University of Florida and you have to work hard to find someone that is flexible, willing to adapt their financial models so that you can take out the technology and still have something feasible in the market that will be affordable and successful. So it is a challenge but we continue to learn and it is working so far.
ROBOTS: Have you worked only with US Universities? Do you see a difference in other regions of the world?
Mel: I definitely see some impressive things in Europe and Australia but…
ROBOTS: With respect to this getting the research out, I meant.
Mel: Oh. I have not gotten that far into it to explore how hard or easy it is to get that out. There is definitely impressive technology but I think we have been so consumed with even just the US opportunities that we have not had an opportunity to look further.
ROBOTS: You also participated in the grand challenges, the 2 drastic ones and then the urban one I guess?
Mel: Yes, again we funded University of Florida for the first two, so we provided software and money and expertise and worked in the trenches with them on the first two, and then because the urban one was so close to the mining work that we were engaged in, we decided to go that one alone and really put our building blocks from the urban challenge vehicle onto mining trucks as we were going so every week; as we did a build on the urban challenge, we would download that code onto the mining trucks and the mines in Arizona and run those same algorithms because a lot of the same problems were being solved.
ROBOTS: My next question then was: the urban challenges, the DARPA program had been tremendously successful, and you just answered that by saying you actually downloaded them once every week.
ROBOTS: Because it has been very significant, right?
Mel: Yeah, I believe so. I am not sure about everyone’s experience with it. We got a million dollars through that program and I did not prioritise winning the race. I prioritised we will have building blocks from these capabilities and used in mining, and that is non-negotiable. Winning is secondary and we got the mining trucks and we are shipping mining trucks so it was definitely a great opportunity to solve the problems that we needed in the real world and we are using it.
ROBOTS: When we are talking of mining, is it so that this is field robotics in a sense that the mine is totally closed to humans or are they interacting with vehicles driven by humans or humans in the environment without the vehicle?
Mel: It is going to be a while before all the vehicles will be fully autonomous. So you have human operators doing the loading, for example, one of the tougher challenges. So we have human vehicles, they are all marked with GPS, and then we have sort of logic and traffic management to keep the vehicles at safe distances, automatically regulate following distances, you can do lock out, tag outs to keep certain areas safe when you are servicing a vehicle and then levels of authority and logging in so that a service man can take over a platform and it will not autonomously power up and run over him. So there is the realities you have to deal with humans in that environment, and so we have had to develop the different components, both on human driven vehicles and the unmanned ones to keep everything synchronised and safe.
ROBOTS: Yeah. And the track record so far, have you had any issues?
Mel: Issues. Not as far as safety. There are obviously technical hurdles that you work through. We found that the technical challenge is maybe 20, 30 percent of the problem, you have got especially in mines, if you have unions, just ‘this is how we have always done it’ kind of challenges, in education and changing the approach to the problem, and there is the liability hurdles as you bring in partners and yourself, the liability as we look at the threats or what could really hurt our company, and accident is the highest, and so you have got to deal with that threat. So the technical challenge is really a minority in percentage compared to all the other problems that you have to solve to really get a full autonomous system up and running and successful.
ROBOTS: So you also are working on getting even further out where there are more people and other things to take into consideration when you do automated convoys. Can you tell us a bit about that? What is the context you want to use that in?
Mel: Yeah, we have kind of got a unique approach there because of this challenge of getting it right now. How do you create a system that driving through streets in Iraq does not veer off and take out someone on the sidewalk and that can be deployed right now? And so we have done GPS following, we have done GPS navigation but you have got inertial solutions, you have got lasers and radars and GPS that all can either be jammed, that can be blocked by the trees; tons of challenges there, so we actually went with a dog leash approach and so you have to see videos to believe it, but basically there is a dog leash on the front of all these trucks. You pull the dog leash out, the vehicle starts up and starts following the exact path that you take, and a lot of people thought that it was stupid. It was almost cancelled by the Pentagon until the representative got in the truck and drove, and it drove just like a human and followed in the exact path of the vehicles in front of it and now it is shipping to Afghanistan, we have units in Singapore. So it is something that can be field, its safety is certifiable right now because it is basically just automotive sensors versus doing a mean time between failure on GPS blockage coupled with multiple sensors. It is impossible to prove that it is safe right now. So that is a system that can immediately reduce labour by 2 or 3 depending on how many trucks you put in a row and it is getting fielded, so we are pretty excited about that.
ROBOTS: Yeah. How many trucks are you looking at? Is it 2 or 3 right now and how do you see it for the future? When will we see a hundred of them in a row?
Mel: Yes, we need to do a hundred. I think right now 3, as you get the narrow roads, the cost of the sensor to get the angular resolutions to keep you within a narrow road, winding through a jungle, 3 is about the limit or cost point that we have. If you want more then you need to add more expensive sensors on the solution but it is just a string. You can repair it with your shoe lace and it has got a couple of neat applications both in the convoy and the robotics climate; another example we envisioned cutting the caps off all the tractors so how do you transport a tractor from field A to field B across a public road with this leash system, you just grab the string, you hook it up your pick-up truck or your four wheeler and pull 3 pieces of large farm equipment down a public road but that don’t have cabs on them.
ROBOTS: And portion them out as they come to the field where they are supposed to work.
Mel: Yeah. Another neat thing about that solution is its speed based following. So as you go 60, 70, 80 miles an hour, they all pull apart for a safe stopping distance. When you go into town, they all come in nice and tight for sharp turns.
ROBOTS: So this is something that could also be used for highway traffic, I mean not only in a war zone but even in like everywhere in Pittsburgh right now? Could that be used here?
Mel: Yeah, I think where we are hearing interest right now is in Australia where they can’t build a railway, so they have a bunch of sand mines that are hauling material from the mine to the port or up in Alaska where the ice fields are transferring multiple semi loads from point A to point B fairly simple long runs, that is a pretty nice and probably one of the early adopting applications.
ROBOTS: But I also know that trucking is a huge industry in America. They use that for hauling basically everything.
ROBOTS: So are we going to see these on regular public roads?
Mel: Well I called UPS. That didn’t go really well. They want the truck manufacturers to prove that out before they get to it. And for us, we have got to find the people who are bleeding, where is the pain, and that really is the out back in Australia or Northern Canada where labour is so expensive and yet hits a straight road, there is no traffic and you are just driving a long way. So that is where we have got to target and that is what we are doing.
ROBOTS: And then you will also work out any small detail that might be left and you will also prove it. I mean if it has been running up there for a number of years, it is going to be easier to come these Southern companies and say that well it works there.
ROBOTS: If it works in the freezing out back of Alaska or the scorching out back of Australia, it is probably going to work here.
Mel: Yes. Yeah, so that is sure the approach.
ROBOTS: We have talked about the really big things, the mining trucks and the bulldozers, but you also work with smaller platforms.
Mel: Yeah, automotive for example, we do just proving around automation for automotive industries, so are getting thousands of miles on autonomous vehicles working at automotive equipment manufacturers in Detroit for example, some of the largest in the world are using our kits to automate, ‘hey we have a brand new, this model of vehicle 2013, put our kit on, beat it up, put it through the rumble strips and beat the heck out of that vehicle’ and try to accelerate the testing so that you can…
ROBOTS: So they use the autonomous driving not to get from point A to point B but to use it as part of the testing?
Mel: Yeah. So we have one operator watching 7 vehicles when we want to get up to 1 to 30, so you have a human overseeing 30 different autonomous vehicles during their operations and being able to beat them up without putting the human through that kind of abuse right now. They are limited to 30 minutes in a vehicle at a time, then they have to get out, rest for 2 hours because of the abuse to the liver and back. So it is a nice application. It is nice visibility with the major OEMs in the automotive industry and it just puts thousands of hours on beating up our stuff so it is good durability.
ROBOTS: Yeah, not only the car gets beaten up but your stuff gets beaten up.
Mel: Yeah, so it might be at max a 10 million dollar business but we have got great traction and we have got some of the biggest in the world using it and many more asking for demonstrations. So that is going well, that is an example of the mid size and then down to the smaller equipment, we have done under vehicle inspection in Iraq for bomb disposal and we have some small robots in the underground mining environment and bomb disposal, so doing some 3D vision, 3D mapping, GPS and high tech navigation with the smaller robots. And the exciting thing about that small robot bomb disposal type stuff is we are leveraging it in the mining industry for the monster shovels that are a hundred ton shovel for example, leveraging the same vision and autonomy control from an arm that lifts 5 pounds. Some of the leverage there from research that started with the government.
ROBOTS: So thank you very much for being part of the podcast and we are going to link to some awesome videos and other stuff so that the listener can check out the amazing vehicles you have built.
Mel: Wonderful, it has been a pleasure.