In a wide-ranging interview, our editor Neil Kennett discusses driverless cars, driver assistance systems, proposed changes to The Highway Code, robotaxis, data privacy, the trolley problem, artificial intelligence, and the Smokey and The Bandit theme song, with Dean and Sarah Gratton on the Tech Uncorked podcast.
“I’ve been a motoring journalist for 20-odd-years and I’ve become increasingly obsessed with connected and autonomous vehicles, and very dissatisfied with the majority of national media coverage,” he said.
“As I saw it, driverless cars were presented as either goodies like Kitt from Knight Rider or baddies like The Terminator, and you didn’t really get beyond that, so I launched Carsofthefuture.co.uk to explore the issues in more depth.”
Barrister Alex Glassbrook specialises in road transport and has written two books on UK autonomous vehicle (AV) law. An expert in the law of advanced, automated and electric vehicles, serious personal injury, motor insurance and high-value vehicle damages cases, he begins by highlighting three recent developments:
The Automated and Electric Vehicles Act 2018 coming into force on 21 April 2021;
The government announcement on 28 April that it isn’t yet publishing a list of AVs under Section 1 of the Act, but that it does expect to list vehicles equipped with Automated Lane Keeping Systems (ALKS) as “automated”; and
Barrister Alex Glassbrook specialises in the law of connected and autonomous vehicles.
AG: “My work overwhelmingly involves car accidents as the source of serious injury, so the AEV Act coming into force was an historic moment. Immediately though, it was clear there was something missing: the list of automated vehicles under Section 1 of the Act, which the Secretary of State is required to publish as soon as it is prepared. There was a presumption that the Act and the list would come together, but they didn’t. We have the Act but no list. In traffic light terms, we’ve gone past amber but there’s no green. What’s going on?
“That question was answered a week later with the Centre for Connected and Autonomous Vehicles’ publication of its paper for the Department for Transport on whether vehicles equipped with ALKS would be listed as automated. In summary, it said the list is not yet being published because we’re waiting to find out if these vehicles will get Whole Vehicle Type Approval from the Vehicle Certification Agency (VCA). If that happens, then the Secretary of State does expect to list them as automated under the AEV Act.
“This has huge implications for liability because it brings into effect a new line of motor insurance. Currently, under the Road Traffic Act, the motor insurer is effectively the body that will satisfy any judgment against a liable driver, or indeed can be sued directly under the direct rights against insurers regulations.
“The new AEV Act does something very different, something particular to AVs: it makes the insurer of the vehicle directly liable. This brings two important changes. One is the direct liability, which is slightly different from the direct rights regs. Second, it attaches to the vehicle rather than the driver, which is quite a radical step.
“There are obviously practical considerations behind this. Would publishing the list before the vehicles get Type Approval be putting the cart before the horse? Even so, it’s a little bit curious because the Act has already come into effect. Moreover, it’s not yet certain that ALKS-equipped vehicles will be classed as automated. The Secretary of State could change his mind.
“Running alongside this, we have the proposed amendments to the Highway Code. They’re quite eye-catching. The current Highway Code reiterates the orthodoxy, that the driver must at all times be in control of the vehicle and must understand the manufacturer’s instructions. The new proposed version is currently out for consultation, but the consultation period is very short, with a deadline of 28 May.
Proposed amendments to the Highway Code published on 28 April 2021.
“The key section reads: “On the basis of responses to the call for evidence, and the step-change that the expected introduction of the first legally recognised automated vehicles represents, we have decided to make a more ambitious amendment to The Highway Code, coinciding with the code’s 90th year anniversary.” To me, the fact it is 90 years since the Highway Code was first published in 1931 is neither here nor there. What is notable is the reference to “more ambitious”, because that implies there was an earlier draft.
“The next sentence has the wow factor. It says: “Automated vehicles no longer require the driver to pay attention to the vehicle or the road when in automated mode, except to resume control in response to a transition demand in a timely manner.” The implications of those words are immense.
“The document continues: “Automated vehicles are vehicles that are listed by the Secretary of State for Transport. While an automated vehicle is driving itself, you are not responsible for how it drives, and you do not need to pay attention to the road.”
“Well, we don’t have that list yet, and what follows is really quite striking. It proposes an instruction in the Highway Code, the official guidance to drivers, to do nothing – to pay no attention to how the vehicle is driving or what’s happening on the road. It positively advises drivers to switch off their attention.
“The next paragraph sets some parameters: “If the vehicle is designed to require you to resume driving after being prompted to, while the vehicle is driving itself, you MUST remain in a position to be able to take control. For example, you should not move out of the driving seat.”
“So, you shouldn’t get out of the driving seat – that’s quite a low standard. This appears to be saying it’s fine to watch a movie, it’s fine to go on Instagram, it’s fine to read and respond to business emails. All these tasks are entirely absorbing of concentration and require some disengaging from.
“I’ve done a lot of trials in which I’ve asked witnesses about their appreciation of time during a crash and heard expert evidence about what can happen within a short window of time. Particularly when you’ve got three or four lanes of motorway, multiple vehicles, an awful lot can happen in 10 seconds.
“There are two fairly well-known exceptions to driver control recognised in the law. One is a medical emergency, if a driver is suddenly incapacitated. The other is moments of peril, sometimes known as agony of the moment – when it is such a difficult situation that a driver causing injury by their evasive manoeuvre is not to be judged by the usual demanding standard.
“So, the common law has formed exceptions to liability, but in this case it’s more complex. First, it introduces, for want of a better phrase, artificial intelligence (AI) into the picture. Adjudicating the actions of AI is still a very undeveloped area of law. Second, it brings into the picture something that has been manufactured, namely a computer and sensor system within a moving vehicle. Again, the laws of product liability are at a very early stage of development in relation to AI and new technologies. A notorious example of that is over-the-air (OTA) software, which is not understood as goods.
“From a legal perspective, it is vitally important not to have guidance which leaves open very obvious questions. Unfortunately, these proposed changes to the Highway Code do just that. On the one hand, the Highway Code might say it’s perfectly fine to completely distract yourself from driving. But on the other hand, it’s not okay to do things like climbing out of the driving seat. That leaves open a very broad set of situations and the courts are going to find themselves dealing with some very difficult problems.
“Of course, what the court has to deal with is very much secondary, the primary question must be: what is safe? There are plenty of lessons in the history of motor vehicles when innovation has overreached. Famously, Ralph Nader’s book, Unsafe At Any Speed (published in the USA in 1965), highlighted rear suspension which lost traction when going round corners. That changed product liability law across all sectors.
“I’m not a road traffic engineer but, as an observer of many road traffic accident cases over many years, I have real doubts as to the safety of this guidance.”
With its laudable aim “to demonstrate entrepreneurship in the global public interest while upholding the highest standards of governance”, transformational technologies like autonomous vehicles are natural territory for The World Economic Forum. Here, we get the considered views of the Forum’s Automotive & Autonomous Mobility Lead, Tim Dawkins – an Englishman working for the Geneva-based organisation in sunny California.
Tim Dawkins leads a portfolio of automotive and autonomous mobility policy research activities.
Tell us about your path to autonomous vehicles and The World Economic Forum
TD: “I started out studying motorsport engineering at Brunel and my first job out of university was in vehicle security for automotive consulting firm, SBD, helping manufacturers meet Type Approval requirements with anti-theft technologies. When SBD opened an office in North America, I went there, to lead their consulting in autonomous driving. Then, in 2018, I got my MBA and wound-up joining The World Economic Forum.
“Here at the Forum, our mission is greater than to convene events for business leaders, but actually to improve the state of the world. In my domain, that means making sure that the future of transportation is as safe as possible. Broadly, we work with governments and industry leaders to help them understand each other better. In the world of autonomous vehicles that means helping governments understand how the technology is evolving and the creation of new governance structures – which can be used in regulations, standards and assessment criteria.
“A crude analogy is to think about a driving test for the self-driving cars of the future – what does that look like? It’s obviously a lot more nuanced and complex than that, but by being a neutral entity – bringing together the likes of Aurora and Cruise with leading academics and regulators to have focused discussions around autonomous vehicle operation and deployment, or what it means to define a safe autonomous vehicle – is a very effective way of achieving better outcomes for all.
“It’s not just about the advanced technologies of the future, our portfolio also includes road safety research – improving the infrastructure, reducing crashes and fatalities with today’s ADAS technologies, and looking ahead to creating a safer future of mobility with autonomous vehicles.”
With your global perspective on autonomous mobility, how is the UK doing in terms of the government’s stated aim of being “at the forefront of this change”?
TD: “The automotive industry has always been very important to the UK economy, so it is natural that that industry and the government agree on the strategic priority to make the UK an attractive place to develop and test these technologies. We have world-leading engineering talent, universities and research and test facilities within our borders, so it’s shifting the focus from sheet metal and engines over to Connected and Autonomous Vehicle (CAV) technologies. Really, it’s a great fit.
“What UK governments have done – I say governments plural, because this has been going on for over 10 years – is to create institutions which spur development. There’s been dedicated funding and research grants not only to grow the CAV ecosystem within the UK, but to encourage international organisations to come and develop in the UK as well.
“What we see now is the result of many years of building the business case, to position the UK as a competitive place to test and develop new technologies. This top-down industrial policy, combined with an open code of practice to facilitate automated vehicle trialling, make the UK a great place to test and develop AVs.
“This ecosystem view is something we study here at the Forum. We recently published a joint paper with The Autonomous – The AV Governance Ecosystem: A Guide for Decision-Makers – which looks at how the standards bodies, alliances and consortia are coming together to develop solutions which will become policy, or at least be used in future governance. You will notice that a lot of UK entities feature very prominently in this study.
“For example, BSI are one of the long-established standards institutions that have been mission-aligned to further CAV mobility, by delivering technical standards and guidance to address governance gaps in the sector, such as the new Publicly Available Specification (PAS) 1881, 1882 and 1883 documents and a vocabulary of CAV terms. Then you have entities such as Zenzic to create the business environment and inform the overall roadmap to making autonomous vehicles a reality, supported by entities such as Innovate UK, and a whole ecosystem of universities and research entities creating a thriving network for innovation.
TD: “One of the things our team like to tackle is how to incentivise these companies to go not just where they can make the most profit, but to provide services to those who most need transportation. This means providing services in areas that are underserved by public transport.
“Think about commuting into London – you drive to the train station, then get onto the TFL network. If you can make that journey more efficient, hopefully more affordable, and accessible, suddenly the economic opportunities that come with commuting into London are open to a greater swathe of people. It’s a very local issue. You have to look at each city and say: where are the areas with the least economic opportunities and how can mobility provide them with greater access to jobs, healthcare and all the things they need?
“Fundamentally, mobility should be considered a human right. It’s not codified as one, but the link between good access to mobility and access to a good future is extremely strong. When we talk to city regulators, for example, they’re very keen to view autonomous vehicles as a way of making their transportation ecosystem more efficient – using AVs to get people onto the existing network, rather than replacing buses or train services.”
That’s certainly opened our eyes to the important work of the World Economic Forum, and we’ll be hearing more from Tim’s colleague, Michelle Avary, Head of Automotive and Autonomous Mobility, at next month’s Reuters event, Car Of The Future 2021.
Law Commission proposes user-in-charge – a new legal role reflecting the responsibilities of being less than a driver but more than a passenger.
The Automated Vehicles Review at the Law Commission of England and Wales plays a pivotal role in in the UK government’s push to be at the forefront of the burgeoning global self-driving industry.
Since 2018, when the Centre for Connected and Autonomous Vehicles (CCAV) asked The Commission to undertake a far-reaching three-year review of the UK’s regulatory framework for automated vehicles, Jessica Uguccioni, the lead lawyer for the review, has been immersed in reforms to enable their safe and effective deployment.
Jessica Uguccioni (front left) and others at the Gateway project in London, 2018.
Notably, in December 2020, The Commission unveiled a consultation setting out a comprehensive regulatory scheme for automated vehicles. The consultation closed in March 2021 and the outcomes are not yet public.
Two concepts are particularly striking: 1) a start-to-finish self-driving vehicle safety assurance scheme; and 2) a user-in-charge.
Under the proposals, when the vehicle is driving in automated mode the person in the driving seat is no longer a driver, but instead a ‘user-in-charge’ with responsibilities to take over driving following a transition demand, and for driver duties that do not relate to dynamic driving (like maintenance of the vehicle, or ensuring children are wearing seatbelts).
Importantly, the user-in-charge would not be criminally liable if an accident occurred while the vehicle was in self-driving mode. Transport Minister Rachel Maclean hailed the work as “leading the way on the regulation of this technology”.
JU: “Our analysis is still evolving, not just in terms of the framework we would like to see, but suggesting changes to existing legislation and identifying gaps.
“For passenger cars, there are two main routes to market: gradually adding driving automation features to consumer vehicles, which may be capable of self-driving for part of a journey but still rely on a human driver to complete a trip; and the ride hail model, with vehicles that can carry passengers or drive empty, and can complete trips while self-driving.
“The oversight needs to be very different, although there is some common ground. The safety assurance scheme applies regardless of the use case. But for cars which cannot complete a journey in self-driving mode, it is important to have a user in charge – a new legal role reflecting the responsibilities of being less than a driver but more than a passenger. On the other hand, fleet operators play a crucial supervisory role for automated vehicles that do not need a user-in-charge.
“There is a lot of unease over the safety of the transition process: human factors input is crucial to ensure the human can be brought back into the loop and take over driving in a safe manner. Circumstances (the ‘operational design domain’ or ODD) must also be taken into account. For example, being in a dedicated lane travelling at 10mph is a very different safety case to motorway driving.
“The SAE levels are helpful, but they don’t tell the whole story. The AV must be safe within its ODD, but any public place brings an amount of randomness. The AV therefore needs to be able to cope with a wide variety of situations. For example, pedestrian safety needs to be taken into consideration for ALKS on motorways – people shouldn’t be walking along or across motorways, but sometimes they are. We need to make sure that redistribution of risk does not disadvantage vulnerable road users – that’s a priority.”
Carsofthefuture.co.uk has signed a media partnership agreement with The Small Cell Forum (SCF) for its three-day online Small Cells World Summit, The Future of Mobile Networks, on 11-13 May 2021.
Small Cells World Summit 2021 registration
As part of the deal, Carsofthefuture.co.uk editor Neil Kennett will moderate the Automotive & Transportation session from 11am on Wednesday 12 May, with high-profile speakers including: Peter Stoker, Chief Engineer for Connected and Autonomous Vehicles at Millbrook Proving Ground; Dr Maxime Flament, Chief Technology Officer at the 5G Automotive Association, one of the world’s leading authorities on Intelligent Transport Systems (ITS); Bill McKinley, Connected Car Business Lead at Keysight Technologies; and Mark Cracknell, Head of Connected and Automated Mobility at Zenzic, responsible for accelerating the self-driving revolution in the UK.
Neil Kennett, said: “We are delighted to partner with The Small Cell Forum for this exciting virtual event, which brings together mobile operators, vendors and regulators from around the globe. The Automotive & Transportation session will focus on connected and autonomous vehicle (CAV) opportunities, particularly vehicle-to-vehicle (V2V) and vehicle-to-everything (V2X) communications, in-vehicle payments, and the rival ITS-G5 and C-V2X 5G technologies.
“Small cells deliver high-quality, secure 4G and 5G coverage, so there are clearly a multitude of new use cases in the connected car world and the wider mobility ecosystem. Aside from supporting self-driving, they can facilitate everything from in-car infotainment and shopping, to fixing technical problems before they occur and pre-empting likely crash scenarios. It is no exaggeration to say they could be a lifesaver.”
Carsofthefuture.co.uk readers can benefit from a 40% discount on Small Cells World Summit 2021 tickets using the code SCWS2021. See www.smallcells.world/
“We will be running autonomous buses this year. That’s an incredible milestone.”
Our Zenzic CAM Creator series continues with Jim Hutchinson, CEO of Fusion Processing.
As a partner in the ambitious CAVForth project, predicted by the Scottish Mail on Sunday to make Edinburgh “the most ‘driverless’ city in the world”, Fusion Processing is delivering on its promise to design and build world leading systems for the automation of vehicles. Here, CEO Jim Hutchinson talks ADAS, cyclist detection and autonomous vehicle safety, explaining how CAVForth is set to make a major mark on the global self-driving map.
Jim Hutchinson, CEO of Fusion Processing
Please can you outline Fusion Processing’s work on connected and automated mobility?
JH: “We’ve been going since 2012. We set up to develop automated vehicle systems with the ultimate goal of being fully autonomous – able to do anything that a human-driven vehicle can. We knew from the start there were a lot of steps along the way and, for essentially a commercial company, we needed to have products along those steps rather than trying for a ‘Level 5 or nothing’ approach.
“We developed the CAVstar platform as a scalable solution – a drive system we could put into pretty much any vehicle, from small cars up to HGV. Along the way we’ve been involved in some great schemes like the Venturer project, one of the original three UK AV projects.
“Then, more or less in parallel with that, we were involved in the Gateway project in London. We provided the autonomous drive system for the pods that drove along the Thames path. That was a big trial with random members of the public – some who came along specifically to experience it, and many others who just wanted to get from the O2 to the other end of the route. The pods encountered various other people on the route – the vehicles had to be mindful of dog walkers and cyclists. The feedback was by and large very positive, and it was a good proof point for us of how our system can be used off-highway.
“It also led to other things, notably our partnerships with Stagecoach and Alexander Dennis. First-off we were exploring using autonomy in bus depots. Every night a lot of operations have to happen involving the movement of vehicles – they have to be fuelled, washed, made ready for the morning, so we put together a system which could automate that. The concept was based on a fleet manager directing all this from a control tower once the bus arrives back at the depot.
“The system proved very successful, demonstrating operating efficiency and improved safety for those working in the depot, so that led to CAVForth – an autonomous bus service. Again, we’re working with Stagecoach and Alexander Dennis, joined by Transport Scotland, Bristol Robotics Laboratory and Napier University.
Fusion Processing’s CAVstar
“The intent is to put into service a number of Level 4 autonomous buses between the Fife Park & Ride and the Hermiston Gait Interchange. It’s a commuter route so we’re expecting a large number of daily commuters who want to travel to the Hermiston Gait Interchange, where they can transfer on to trams for the city centre, the airport or the rail network. We expect tourists will want to use it too to reach the Forth Road Bridge, a UNESCO heritage site.
“It’s a useful service, running every day of the week, and the hope is that it will go from a pilot service to a full service. It’s being registered as a new route, providing a service that wasn’t previously there, and Stagecoach anticipate around 10,000 journeys a week.
“The route includes a mix of road environments – motorway, bus lanes, roundabouts, signalled interchanges – so from our point of view it makes for a great demonstration of capability. There’s the technology side, which Fusion is focussed on, but there’s also key research around public acceptance and uptake. That’s really exciting too.
“The launch date isn’t set in stone due to Covid uncertainties, and the point at which they start taking passengers is still to be determined, but we will be running autonomous buses this year. That’s an incredible milestone, absolutely huge. It will be a very significant achievement to demonstrate a Level 4 capability on that class of vehicle – a big thing for the UK which will be noticed around the world.
“There are one or two other groups working on similar projects, but I haven’t seen anything with this level of ambition, this level of complexity, or length of route. It’ll obviously be fantastic for us and our CAVForth partners, but also for the UK autonomous vehicle industry as a whole. It will really put us on the worldwide map.”
Please can you outline Fusion Processing’s work on driver assistance?
JH: “CycleEye is an important product for us. We identified a need for collision avoidance technology. There are lots of collisions with cyclists and quite often they occur because the bus driver doesn’t know the cyclist is there. CycleEye is like a subsystem of CAVstar in a lot of ways – one of those steps to get some proof points on bits of the technology. It recognises and classifies different types of vehicle, and the driver gets an alert when there’s a cyclist in the danger zone. It is currently being used in a few cities around the UK, including on the Bristol Metrobus. It’s a good system. Whenever it has been evaluated against other cyclist detection systems it has always come out on top.
“We’re particularly excited about the next incarnation of CycleEye, evolving it to become a camera mirror system. It’s legal now to use cameras instead of mirrors, so we can provide that functionality too – monitors in the driver’s cab instead of mirrors. That has several benefits. Mirrors, on buses particularly, can be a bit of a liability – they quite often get knocked and sometimes they knock people. They stick out and head strikes are unfortunately quite common. They also get smashed, putting the bus out of service, which is an inconvenience and an operational cost. We think that being able to offer this camera mirror with CycleEye functionality is going to prove attractive to a lot of operators.”
Van with Fusion Processing technology
Over what timescale do you expect Level 4 and 5 autonomy to be achieved in the UK and which sectors will be early adopters?
JH: “With CAVForth we’ll be running Level 4 autonomous vehicles, where you’ve got a restricted operational design domain (ODD), in the UK this year. Restricting these vehicles to particular routes or environments lends itself very well to public service, where the vehicles are maintained by an operator. That’s very achievable right now. As well as passenger service vehicles, other service vehicle fleets are easy wins, as well as off-highway stuff like industrial sites. Then you’ve got delivery vehicles.
“When it comes to true Level 5 – go anywhere, do anything vehicles – repair and maintenance is an issue. We know that with privately owned cars, some people maintain them exactly as they should, and other people don’t. There are other complications too – things that people perhaps don’t do that often but like their vehicles to be able to do, like parking in a farmer’s field at a festival – that’s a little bit further out still.
“If you just roll back slightly from true Level 5, if people want a city car or a comfortable car for a long motorway journey, nothing off-road, there’s a case for vehicles which have an autonomous mode. That certainly appeals to me.”
Can you address the concerns about ADAS, particularly handover of control, driver concentration levels and driver deskilling?
JH: “I’m not a big fan of Level 3. If you haven’t been driving for an hour to suddenly be asked to take the wheel because the car has encountered something it can’t handle, it’s just unrealistic. Whereas a Level 4 system, which can put itself into a safe state when it reaches the limits of its ODD – perhaps ready to be restarted in a manual mode when the driver wants to take control – that’s much more practical.
“If there are circumstances when the driver needs to take over then clearly the driver needs to be of a standard that they can drive safely. Once you have widespread adoption of autonomous systems, and people are not driving routinely, there is a risk of driver deskilling. If that were the case you’d really need to look at greater regulation of drivers.
“That said, you can sometime envisage problems that don’t really transpire. We’ve had cruise control and adaptive cruise control for a while now and I don’t think they’ve had the effect of particularly deskilling drivers. So, with Lane Keep, maybe it’s not such a big deal. Once you get to the point where cars are properly self-driving, there is a danger. If you haven’t got anything to do your mind will wander, that’s human nature, so it is a concern.”
The trolley problem – the question of who to save, or kill, in no-win crash situations – continues to divide opinion like no other subject in the driverless world.
That, I thought, was conclusion reached, the end of the matter. Far from it! In response to the latter article, Karim Jaser, Senior Product Manager specialising in artificial intelligence (AI) and internet of things (IoT) for blue chip companies, posted a resolute defence of the much-maligned thought experiment on our LinkedIn page.
“I do agree that humans don’t go through the trolley problem evaluation in the split decision second, however I also think not all experts agree it is a nonsense,” he said. “It is for society as a whole to discuss these ethical problems. From the point of view of self-driving technology, this can be solved in many ways, with probability theory and estimations on minimal loss, but it is not up to developers or self-driving experts alone to decide how to tackle the point. It needs the involvement of regulators, governments and the industry.”
Karim Jaser, Senior Product Manager specialising in AI and IoT.
KJ: “I was always fascinated by robot intelligence, so at university I studied telecommunication engineering. There were lots of exams on probability theory, system control, software engineering. I was also involved in coding in my spare time, and later did it as a job.
“Self-driving is a control problem first and foremost. There are elements of robotics, including perception state estimation and trajectory planning, but also software, hardware and AI working together.
“The interest grew stronger when I started studying machine learning and AI about four years ago. When I was at university in the 90s, AI was not really a popular subject. It was a topic I picked up later in my career. As a senior product manager at a high technology company, AI is everywhere now – it’s an essential part of the skills necessary to perform and innovate, from biometric scans and image recognition to automated travel.
“AI has a lot of potential to have a beneficial impact on society – fewer accidents, better mobility, less pollution, more autonomy for people with disabilities – but it doesn’t come without challenges, for example, cyber threats, and also ethical and regulatory issues, which is why I got involved on the trolley problem.”
“It’s not straightforward. If we take a step back, you we need to understand how self-driving cars take decisions. They’re using supervised learning, reinforcement learning, convolutional neural networks (CNN) and recurrent neural networks (RNN), deep learning for computer vision and prediction. Specifically, reinforcement and inverse reinforcement learning are very tightly linked to the way driverless vehicles behave through means of policies.
“Policies are related to the distribution of probabilities, but the trolley problem is an ethical choice, so I understand why a lot of people in the industry dismiss it. It’s not the way autonomous vehicles take decisions, going through philosophical considerations in a split second, so it might seem irrelevant, right? Like the Turing Test and Asimov’s Robot Rules, the trolley problem can be perceived as a distraction from more practical considerations.
“It can be distracting for two reasons: first, these considerations are corner cases – there are other priorities, more likely scenarios still to be addressed; second, autonomous vehicles will not be given ethical guidelines to link with probabilities.
“With regards to the first objection, as Patrick Lin (director of the Ethics and Emerging Sciences Group at California Polytechnic State University) has pointed out, it shouldn’t matter if these scenarios are impossible, because the job of these thought experiments is to force us to think more carefully about ethical priorities, not to simulate the realities.
“The second objection is related to self-driving cars taking decisions through distribution of probabilities. The actions of these vehicles are linked not to hard coding, but all statistical contextual information, and that makes each scenario difficult to interpret. You can potentially have millions of mini trolley problems in different contexts.
“The trolley problem is a reminder that corner cases and autonomous vehicle behaviours are not a technical irrelevance. This is an issue that belongs to society and should be discussed in the same way as other AI pitfalls like privacy and bias.
“Actually, the trolley problem is more related to the third pitfall of AI, replicability. When trying to understand why and how an autonomous vehicle takes a decision, it is important to note that most autonomous vehicle developers are taking ethical considerations into account.
“In 2017 in America, Apple commended the National Highway Traffic Safety Administration (NHTSA) for including ethical considerations in its Federal Automated Vehicles Policy. It even highlighted three particular areas: 1) the implications of algorithmic decisions for the safety, mobility and legality of automated vehicles and their occupants; 2) the challenge of ensuring privacy and security in the design of automated vehicles; and 3) the impact of automated vehicles on the public good, including their consequences for employment and public spaces.
“The automotive industry has also approached the issue of accidents caused by autonomous vehicles in relation to ethics. For example, Volvo stated in 2015 that it would take responsibility for all Volvo self-driving car accidents. This is an ethical decision, because it did so without regulation forcing it to do so.
“We will see what happens. If there are no ethical decisions by the industry, the regulators will step in. On a fun note, looking to the past, horses were not considered responsible for their actions, the rider was. Whereas in this case, responsibility for the autonomous vehicle will lie not with the owner but with the carmaker.
“So, to conclude, automakers and AV developers are taking ethical and regulatory matters into account, which underlines the importance of these discussions. We cannot just dismiss the trolley problem because it’s not the way an autonomous vehicle decides, or because it distracts from technical development.
“The way to deal with this is to discuss the implications in the right context, being aware of how autonomous vehicles are developed without scaring the public with sensationalist articles. The trolley problem might be perceived as a Terminator-style situation, and that’s where it gets on the nerves of a lot of people that are developing and testing AI. It’s not black and white, it’s a grey area, and that takes us to the path of discussions.
“The trolley problem forces us to consider ethics in vehicle development and confront the fact that ethical principles differ around the world, as documented by the Massachusetts Institute of Technology (MIT) simulation.
“Are we at the point where discussing the trolley problem should be a priority? I believe that would be beneficial to the success of the self-driving industry, guiding us in the thinking process of building the right mix of safeguards and transparency.”
CGA’s simulations train autonomous vehicles to deal with environments specific to the UK.
Our Zenzic CAM Creator series continues with Liverpool-based Jon Wetherall, Managing Director of CGA Simulation, and Max Zadow, Director of Future Coders.
By applying gaming knowledge to real-world mobility questions, CGA has created engaging simulations to study autonomous driving and smart city solutions.
JW: “My background is gaming. I used to work for the company that did Wipeout and F1 games. We made a racing game called Space Ribbon and one day, about five years ago, we got a call from The Department for Transport (DfT). They were doing a research project on virtual reality (VR) in the testing and training of drivers, specifically hazard awareness.
“We turned it into a game and it worked – people said their attitudes changed as a result of our simulations. The hardest scenario came early in the game – a parked lorry with a big blind spot – and a lot of people crashed. VR feels so visceral, the experience can be quite vivid and shocking. Of course, smarter cars will hopefully fix these types of situations.”
CGA Simulation junction and forecourt
To pursue this goal, CGA received a grant from Innovate UK to create an artificial learning environment for autonomous driving (ALEAD).
JW: “The aim was to make these cars safer and we stayed true to our computer game history. We didn’t have the resources to lidar scan the whole area, so we did our own thing using mapping data. We made a digital twin of Conwy in north Wales and unlike other simulations we kept all the ‘noise’ in – things like rain. This was important because it is now well-understood that noise is a big challenge for autonomous vehicles (AVs).
“Modern autonomous driving stacks have 20 different subsystems and we generally focus on only one or two, to do with perception. There’s been massive progress in this area over recent years, to the extent that artificial intelligence (AI) can identify an individual by their gait. What’s more, you can now do this on a computer you can put in a car – this is one of the cornerstones of driverless.
“It’s not the first time people have been excited about AI. In the 50s they were saying it was only a few years away. It has taken much longer than people thought, but major problems have now been solved.
“We are lucky to have one of the world’s leading experts in radar on our doorstep, Professor Jason Ralph of The University of Liverpool, and he helped us develop the simulation. You have to feed the car’s brain, a computer, all the information it will need – from sensors, cameras, GNSS – and you can do all that in the software.”
MZ: “In particular, The University of Liverpool were interested in how weather affects things, right down to different types of rain and mist. In California, if an AV encounters conditions it can’t handle, like heavy rain, it pulls to the side of the road. That’s ok for San Francisco but not for Manchester!
“A few years ago, everyone seemed to be using the example of an AV encountering a kangaroo. How would it cope? The point is you can use our simulations to train cars, to create algorithm antibodies for once in a lifetime events and regular things in different environments. That remains an essential part of what’s needed to make AVs a reality.
“We picked Conwy partly because it has very different patterns of land use to America. An early use case for AVs is predicted to be taxis, but in the UK these are most frequently used by people who don’t own their own car, and they often live in high density housing or narrow streets. The operational design domains (ODDs) are going to have to deal with environments specific to this country – steep hills, roads which twist and turn, and changeable weather.”
Mobility Mapper
Wetherall and Zadow’s latest collaboration is Mobility Mapper, a project to create greener and more intelligently designed transport hubs. The technology underpinning Mobility Mapper has been used previously by the team to model Covid 19 spread, autonomous vehicle technology and by the Liverpool 5G Create project (funded by DCMS as part of their 5G Testbeds and Trials Programme).
JW: “E-hubs are basically an extension of what used to be called transport hubs – train or bus stations. They’ll provide charging facilities and access to different modes of transport, for example, you can drop off an e-scooter and hop into a shared autonomous car.
“Here in Liverpool, there was a big trial of e-scooters, big in international terms not just UK. The worry was that a lot of them would end up in the canal, but that didn’t happen. The trial was incredibly successful. It’s all about linking that movement and nudging people away from car ownership.”
MZ: “We were already thinking about how Jon’s technology could be used for mobility as a service (MAAS) when we attended a virtual future transport conference in LA with the Centre for Connected and Autonomous Vehicles (CCAV).
“That was an influence, as was an Intelligent Transportation Systems (ITS) trade show in Copenhagen, where we saw an autonomous tram system designed to take bicycles. It was a small step from there to imagining autonomous trams carrying autonomous delivery pods.
“This is classic smart city stuff but you need to know how these e-hubs are likely to be used, with no track record, nothing to go on. We need simulated environments to make best guesses in. That’s Mobility Mapper.”
JW: “It is early days, still in the development phase, but the authorities in both Manchester and Liverpool have agreed there’s a need for such a predictive simulation tool.”
As we wrap-up a thoroughly enjoyable interview, Max dons his Director of Digital Creativity in Disability hat: “Autonomous delivery bots are basically electric wheelchairs without a person, so there’s clearly a potential benefit, but there needs to less wishcasting and more real work on how accessibility will be affected.”
Thanks to LinkedIn, self-driving experts from the UK and New Zealand have united to decry the trolley problem in relation to driverless cars.
Mitchell Gingrich, President of Autonomous Consulting in Christchurch, New Zealand, responded to our interview with Professor John McDermid, Director of the Assuring Autonomy International Programme at the University of York, saying: “Spot on about the trolley problem.”
Professor McDermid had asserted that: “The trolley problem is a nonsense… all these elaborate versions require self-driving vehicles to make distinctions that you or I could not.”
The trolley problem is a thought experiment which runs like this: Imagine there’s a runaway trolley and, ahead, five people are tied to the track; You are standing some distance off, next to a lever. If you pull it, the trolley will switch to a track only one person is tied to. What do you do?
Or, as Professor McDermid puts it: “Who do you save, a child or an older person? The child because they can be expected to live longer and benefit more. However, this is based on false assumptions. I don’t believe in the split second of a crash you go into that sort of thought process – you focus on controlling the vehicle and in most cases the best option is to (try to) stop.”
Gingrich opined that the March 2018 fatal accident involving an Uber Advanced Technology Group (Uber ATG) self-driving vehicle can aid in evaluating the trolley problem. The National Transportation Safety Board (NTSB) in the US recently completed an 18-month-long investigation and concluded there were 20 contributing factors. Some of those concerned the software misclassifying a pedestrian. A significant contributing factor was the safety driver’s inattentiveness.
The trolley problem assumes that a person or system is not only aware of the task of driving but also of the present and future merits of the lives of road users, he says. However, experience demonstrates that, sadly and all too frequently, road users pay the price for a lack of vigilance.
It turns out that Gingrich, a lawyer by trade, has been on quite a journey with autonomous vehicles himself. From working for Uber ATG in Phoenix, seeing first-hand the fallout from the Elaine Herzberg tragedy, to relocating to New Zealand and setting up Autonomous Consulting to push the case for driverless transport.
“I’m convinced that the future will be autonomous,” he says. “Whether it’s on public roads, in the air or on the seas, we will be utilising autonomous technology to transport our people and goods. That’s what autonomy is promising, but we’re in an interim period.
“New cars have advanced driver assistance systems (ADAS) like lane keep assist and automatic emergency braking. Some of us have been using cruise control for a long time, now it is adaptive – the car will keep its distance. These are autonomous features but not autonomy and we need to educate the public about the difference.
“Autonomy is about safety, resources and the environment. These ADAS systems expect me to pay attention to the road and the robot, and that’s not a recipe for safety. 93-94% of accidents are caused by human error, usually distraction – we think we’re paying attention, but we aren’t. There are repair and maintenance issues too, for example, around the correct calibration of sensors.
“In terms of resources, my personal car is a depreciating asset that isn’t used 90% of the time. Autonomous vehicles will also have a tremendous impact on town planning. An architect in the US imagined Manhattan pedestrianised and it freed up 60% of space.
“My freedom is not challenged by not having a personal vehicle. I’d have more money in my pocket and could use my smartphone to access different vehicles for different purposes.”
Carsofthefuture.co.uk has signed a media partnership agreement with Reuters Events for the two-day Car of the Future 2021 online event in June.
Intended to drive vehicle change to create a safer and more sustainable world, the event boasts the most senior collection of technology, autonomous vehicles (AV), electric vehicle (EV) and advanced driver-assistance system (ADAS) leaders ever seen.
High profile speakers include: Michelle Avary, Head of Automotive and Autonomous Mobility at The World Economic Forum; Carla Gohin, Research & Innovation Senior Vice President at Stellantis; Henrik Green, Chief Technology Officer at Volvo Cars; Sajjad Khan, Member of the Board of Management at Mercedes-Benz AG; José Muñoz, Global Chief Operating Officer at Hyundai Motor Company; and Dr Ken Washington, Chief Technology Officer at Ford Motor Company.
Carsofthefuture.co.uk founder, Neil Kennett, said: “We’re delighted to be a media partner for this exciting Reuters event which fits perfectly with our mission to chart the development of, and encourage sensible debate about, driverless cars in the UK. Full self-driving is a way off yet but as ever more advanced driver assistance systems become available, notably Automated Lane Keeping (ALK), it is vital that the public understands where we are with the technology and what it can and can’t do.”
Car of the Future 2021 will take place on 14-15 June. See reutersevents.com
Ahead of this, Reuters Events will host a free webinar, Connectivity: Smarter and Safer Vehicles, on 24 March. Confirmed speakers include: Michelle Avary; Szabi Patay, Head of Automotive at Commsignia; Prashant Tiwari, Director of Intelligent Connected Systems at Toyota North America; and Frank Weith, Director of Connected and Mobility Services at Volkswagen Group America. Register here.
