Why assuring machine learning is crucial to self-driving
Our Zenzic CAM Creator series continues with Professor John McDermid OBE FREng, Director of the Assuring Autonomy International Programme at the University of York.
Professor John McDermid has been Director of the Assuring Autonomy International Programme, a partnership between Lloyd’s Register Foundation and the University of York, since 2018. He advises government and industry on safety and software standards, including Five and the Ministry of Defence, and was awarded an OBE in 2010. The author of 400 published papers, his 2019 article, Self-driving cars: why we can’t expect them to be ‘moral’, was highly critical of the oft-quoted trolley problem in relation to driverless vehicles.
PJM: “I’ve been at York for 30 years working on the safety of complex computer-controlled systems. What you define as complex changes all the time. In January 2018 we started a new programme, looking at the assurance of robots and autonomous systems, including automated mobility, but also robots in factories, healthcare and mining.
“It’s important to demonstrate the safety and security of novel technologies like machine learning, but there’s often a trade-off involved, because you can make things so secure they become unusable. If I open my car with the remote key I have a couple of minutes before it automatically locks again, and there’s a small possibility that someone could get their finger trapped if they try to open the door just as it automatically re-locks. We encounter these types of trade-offs all the time.”
What major shifts in UK transport do you expect over the next 10-15 years?
PJM: “Over the next decade we will get to Level4 autonomous driving, so in defined parts of the road network cars will drive themselves. We will solve the safety problems of that technology, but I’d be surprised if it is within five years. Despite the rhetoric, Tesla’s approach is not on track for safe autonomous driving within the year.
“At the same time, there will be a trend towards Mobility as a Service (MAAS). I love my car, but I’ve had it for 18 months and have only driven 7,000 miles. I sometimes ask myself why I have this expensive piece of machinery. A recent study showed that the average car in the UK is only used for 53 minutes a day. Mostly, they sit doing nothing, which, considering the huge environmental impact of manufacturing all these vehicles, is very wasteful.
“If I could call upon a reliable autonomous vehicle and be 99% certain that it would arrive in a timely manner, say within five minutes, I’d probably give up my car. It should also be noted that the two trends go hand-in-hand. Having Level4 is critical to achieving MAAS, delivering all the convenience of having your own car without any of the hassle.”
Can you address some of the data privacy concerns surrounding connected cars?
PJM: “We are back to this issue of trade-offs again. I want my MAAS so I’ve called it up and given the service provider some information about where I am. If they delete that information after I’ve paid then I’m prepared to accept that. What if the company wants to keep the information but won’t allow access except for law enforcement – would that be acceptable to the public? What can government agencies require this company to do?
“Another example: What if your 10-year-old daughter needs MAAS to take her to school? A reasonable concerned parent should be able to track that. What if the parents are divorced, can they both access that data? There’s clearly a privacy issue and there needs to be a legislative framework, but it’s a balance. For the purposes of getting from A to B, most people would accept it, so long as their data is normally kept private.”
Can you address concerns about the trolley problem in relation to self-driving cars?
PJM: “My basic feeling is that the trolley problem is a nonsense, a distraction. All these elaborate versions require self-driving vehicles to make distinctions that you or I could not.
“The big Massachusetts Institute of Technology (MIT) study sets a higher standard for autonomous vehicles than any human can manage. Who do you save, a child or an older person? The child because they can be expected to live longer and benefit more. However, this is based on false assumptions. I don’t believe in the split second of a crash you go into that sort of thought process – you focus on controlling the vehicle and in most cases the best option is to (try to) stop.
“I don’t know why people find the trolley problem so compelling, why they waste so much energy on it. I really wish it would go away. Fortunately, most people seem to be coming to that conclusion, although one of our philosophy lecturers strongly disagrees with me.”
Which sectors do you think will adopt self-driving first?
PJM: “Farming applications might come first as they are short of people in agriculture and the problems are simpler to overcome. If you geofence a field where you wish to use a combine harvester and equip it with technology so it doesn’t run over a dog lying asleep in the field – there’s already tech which is getting quite close to that – then that’s an attractive solution.
“Last mile freight via small delivery robots (like Nuro in the US and Starship here in the UK) might also come quickly, but longer distance freight will probably require a segregated lane. Even last mile robots come with risks, like people tripping over them.
“There’s a lot of commercial desire for robotaxis, and this is potentially a very big market. There are already genuine driverless taxis in the US now, but they have a much simpler road structure than here in the UK.
“The crucial technical bit is finding accepted ways of assuring the machine learning. I would say that, I work on it, but without that regulators and insurers won’t allow it.”
For further info, visit www.york.ac.uk/assuring-autonomy
