Of the 100+ features I’ve done for Cars of the Future, the one I find myself quoting most is last summer’s Letters from America: Partners for Automated Vehicle Education (PAVE). In particular, the evidence that live self-driving vehicle demonstration events are highly effective in boosting public trust.
The organisation with a mission to “inform the public about automated vehicles” is expanding fast – launching PAVE Canada in February and now, PAVE Europe.
Partners for Automated Vehicle Education (PAVE) in Europe
On 17 March 2022, at the Autonomy Paris sustainable mobility conference, PAVE Europe announced six “Founding Members”, and a very prestigious bunch they are too. In alphabetical order: financial services provider, Achmea; autonomous vehicle technology specialists, EasyMile and Mobileye; the world’s largest reinsurer, Swiss Re; Cologne-based safety tester TÜV Rheinland; and the company born of Google’s self-driving car project, Waymo.
From a UK perspective, we note the involvement of Lukas Nekermann, MD of London-based Neckermann Strategic Advisors and author of the influential 2015 book, The Mobility Revolution. Both he and Frederic John, who together co-authored 2020’s Being Driven, are credited as the “co-initiators” of PAVE Europe.
We caught up with Tara Andringa, Executive Director of PAVE, to find out more.
Tara Andringa, Executive Director of Partners for Automated Vehicle Education
TA: “We’ve achieved a lot since your last article – made progress on projects we spoke about and launched some brand new ones too.
“Our weekly virtual panels, which began during Covid, have continued to go extremely well. We thought there might be some Zoom fatigue once people got back to work, but in fact they’ve gotten more popular. We’ve had more than 16,000 people register for the panels live, and we have had an additional 20,000 views on YouTube.
“The virtual panels are a great way to spread the conversation about AVs and in January we won a prestigious award for them – a National Communications Award for Outstanding Public Education at the Transportation Research Board (TRB) conference in Washington.
Partners for Automated Vehicle Education at the TRB conference 2021
Automated Vehicle Survey
“Another major success last fall was our survey work. We put together a powerhouse group with the Massachusetts Institute of Technology (MIT) and JD Power to track longitudinally how views change over time. We released the first results in November and we’re going to make it an annual thing.”
The headline result in 2021 was that only 37% of respondents correctly identified the description of a fully automated self-driving vehicle (according to SAE International’s definition) from seven possibilities. 55% of respondents selected descriptions aligned with driver assistance technology.
The results led Lisa Boor, senior manager of global automotive at J.D. Power, to quote the robot from Lost In Space, describing “a ‘Danger, Will Robinson’ moment for the fully automated self-driving vehicle industry”, with “a significant gap between actual and perceived AV knowledge.”
TA: “Another thing we mentioned last time was our public sector workshop with the state of Ohio. We’ve tried to really build out that program, working a lot on state engagement here in the United States. We help to educate public sector officials, to give them the information they need to make choices for their communities and educate their constituencies.”
Which brings us to your international expansion.
TA: “Yes. We’ve realised that, while every government is taking a different regulatory approach, the public confusion and misperceptions are a global phenomenon. So, we’ve decided to expand PAVE’s mission elsewhere, so each country or continent will have its own chapter or sister organisation.
“Public sector engagement is such an important part of this. Each government is looking at different policy options, but this is a global industry. Our new Canadian and European members will be able to network with our US members, to collaborate, but also do their own thing, run their own events tailored to local needs.”
A few messages flew back and forth, and it transpired that he’s an expert in measuring driver behaviour, particularly driver-vehicle interactions in ADAS-equipped and self-driving vehicles. That was music to my ears, so we arranged a Zoom. What follows is the highly insightful result.
Lucas Noldus Ph.D., Founder of Noldus Information Technology
LN: “The future starts here. The world is changing. We see people living longer and there are more and more interactive devices – telephones, tablets, dashboards – with which we can interact, leading to greater risk of distraction while driving. I know personally how tempting it is to use these devices, always trying to keep your eyes on the road.
“We already have fascinating developments in connected driving and now, with self-driving, the role of the driver changes significantly. That has triggered research institutes, universities, OEMs and tier one suppliers to pay more attention to the user experience for both drivers and passengers.
“All these experiences are important because how people perceive the safety and comfort will influence their buying decisions, and their recommendations to other potential users.
“For autonomous driving, how far will we go towards level five? What happens at the intermediate stages? Over the coming decades, driving tasks will gradually diminish but, until full autonomy, the driver will have to remain on standby, ready to take over in certain situations. How will the vehicle know the driver is available? How quickly can he take over? These are the topics we’re involved in as a technology company.
“We make tools to allow automotive researchers to keep the human in the loop. Traditionally, automotive research focused exclusively on improving the vehicle – better engines, drivetrains etc. Until recently, nobody paid much attention to the human being (with a brain, skeletal system, muscles, motor functions), who needs to process information through his sensory organs, draw the right conclusions and take actions.
“Now, these aspects are getting more attention, especially in relation to reduced capacity, whether due to a distracting device, drugs, alcohol or neurodegeneration. As you get older your response time becomes longer, your eyesight and hearing abilities reduce, as does the speed at which you can process information.
“These are the challenges that researchers in automotive are looking at concerning the role of the driver, now and in the future. If the automated or semi-automated system wants to give control back to the driver because its AI algorithms decide a situation is too complex, can the driver safely take over while he’s been doing something like reading or taking a nap? How many milliseconds does the brain need to be alert again?
NK: “Draft legislation seems to be proceeding on a 10-second rule, but some studies say at least 30 seconds is required.”
LN: “Situational awareness – that’s a key word in this business. Not only where am I geographically, but in what situation. Oh, I’m in a situation where the road surface is very wet, there’s a vehicle just in front of me, the exit I need is near and I’m in the wrong lane. Understanding a situation like that takes time.
“If we take a helicopter view, from our perspective as a technology company, what should be measured to understand the driver behaviour? Which sensors should we use to pick up that information? If we use a microphone, a video camera, a heartbeat monitor and a link to the ECU, how do we synchronise that?
“That’s not trivial because one sensor may be sending the sampling at 300Hz and another at 25 frames per second. That’s something my company has specialised in over the years. We’re very good at merging data from different sources, whether it’s a driving simulator continuously spitting out data, a real car, or sensors mounted in the infrastructure.
“You then need to analyse that data and pull out meaningful quantitative units that give you actionable insights. Generating large matrices is no big deal, making sense of that information is the real challenge.
“For example, in dashboard design, a manufacturer might be comparing two or three displays of road quality. A driver behaviour study with our tools will give the designer a clear answer on which design leads to the least cognitive workload, the least confusion.
Noldus DriveLab
“This same technical challenge can be applied to a vast number of design objectives. The vehicle manufacturer might be looking to make incremental improvements to, say, the readability of the dashboard under certain light conditions. Or they might be working on a completely new feature, like an intelligent personal in-car assistant. A number of brands are working on that, but the concept is still relatively new.
“You cannot test every scenario on the road, it’s just too dangerous, so we work with simulator manufacturers too. On the road or in the lab, we can measure a driver’s actions with eye-tracker, audio, video, face-reader and physiology in one.”
NK: “Back to LinkedIn again, I saw a post by Perry McCarthy, the F1 driver and original Stig on Top Gear, who said something like: Simulators are getting so good these days, when you make a mistake they drop three tonnes of bricks on your legs!”
LN: “You have so-called high fidelity and low fidelity simulators – the higher the fidelity, the closer you get to the real vehicle behaviour on the road, and there are all sorts of metrics to benchmark responsiveness.
“You have simple fixed-base simulators right up to motion-based simulators which can rotate, pitch and roll, move forward, backwards, sideways and up and down. For the best ones you’re talking about 10 million euros.
“We work with OEMs, tier1 suppliers, research institutes and simulator manufacturers to build-in our DriveLab software platform. We also advise on what sensors are recommended depending on what aspects of driver behaviour they want to study.
“We try to capture all the driver-vehicle interactions, so if he pushes a pedal, changes gear or turns the steering wheel, that’s all recorded and fed into the data stream. We can also record their body motion, facial expression, what they’re saying and how they’re saying it – it all tells us something about their mental state.
Multi-camera eye tracker (Smart Eye)
“Eye tracking measures the point of gaze – what your pupils are focused on. In a vehicle, that might be the left, right and rear mirrors, through the windscreen or windows, around the interior, even looking back over your shoulders. To capture all that you need multiple eye-tracking cameras. If you just want to look at, for example, how the driver perceives distance to the car in front, you can do with just two cameras rather than six.
“Eye tracking generates all sorts of data. How long the eyes have been looking at something is called dwell time. Then there’s what direction the eyes are looking in and how fast the eyes move from one fixed position to another – that’s the saccade. People doing eye tracking research measure saccades in milliseconds.
“Another important metric is pupil diameter. If the light intensity goes up, the pupil diameter decreases. Given a stable light condition, the diameter of your pupil says something about the cognitive load to your brain – the harder you have to think, the wider your pupils will open. If you’re tired, your blink rate will go up. There’s a normal natural blink rate to refresh the fluid on your eyes with a fully awake person, but if you’re falling asleep the blink rate changes. It’s a very useful instrument.
“Then there’s body worn sensors that measure physiology. It’s harder to do in-car, but in a lab people don’t mind wearing electromyography (EMG) sensors to measure muscle tension. If you’re a designer and you want to know how easy it is for an 80-year-old lady to operate a gearshift, you need to know how much muscle power she has to exert.
“We also measure the pulse rate with a technique called photoplethysmography (PPG), like in a sports watch. From the PPG signal you can derive the heart rate (HR). However, a more accurate method is an electrocardiogram (ECG), which is based on the electrical activity of the heart.
GSR (EDA) measurement
“Further still, we measure galvanic skin response (GSR), also called electrodermal activity (EDA), the level of sweating of your skin. The more nervous you get, the more you sweat. If you’re a bit late braking approaching a traffic jam, your GSR level will jump up. A few body parts are really good for capturing GSR – the wrist, palm, fingers, and the foot.
“We also measure oxygen saturation in the blood with near infrared spectroscopy (NIRS) and brain activity with an electroencephalogram (EEG). Both EEG and NIRS show which brain region is activated.
“Another incredibly useful technique is face reading. Simply by pointing a video camera at someone’s face we can plot 500 points – the surroundings of the eyebrows, the eyelids, the nose, chin, mouth, lips. We feed this into a neural network model and classify it against a database of tens of thousands of annotated images, allowing us to identify basic emotions – happy, sad, angry, surprised, disgusted, scared or neutral. You can capture that from one photograph. For other states, like boredom or confusion, you need a series of images.
“These days we can even capture the heart rate just by looking at the face – tiny changes in colour resulting from the pulsation of the blood vessels in the skin. This field of face reading is evolving every year and I dare to claim that we are leading the pack with our tool.
“Doing this in the lab is one thing, doing it in a real car is another challenge, being able to keep your focus on the driver’s face and deal with variable backgrounds. Of course, cars also drive at night so the next question is can you do all this in darkness? We turned our company van into an instrumented vehicle and my sons agreed to be the guinea pigs.
“It took some work – overcoming the issue of light striking the face and causing sharp shadows, for instance – but we can now use infrared illuminators with our FaceReader software to make these measurements in full darkness.
“The turning of the head is also very important in studying distraction, for example, if the driver looks sideways for too long, or nods their head in sleepiness. When something shocks someone, we see the face change and the blood pressure rise, and these readings are synchronised in DriveLab.
“It is well proven that even things like changing radio station can be very distracting. Taking your eyes off the road for just a few seconds is dangerous. As we move to more and more connected devices, touchscreens and voice commands, minimising distraction is vital to ensure safety.”
NK: “I absolutely love this tech but what I actually drive is a 7-year-old Suzuki Swift Sport with a petrol engine and a manual gearbox, and I quite like it that way”
LN: “I’m doing research on cars of the future with my software but I am personally driving a 30-year old soft-top Saab 900. That’s my ultimate relaxation, getting away from high tech for a moment.
“At Noldus, we’re constantly pushing the boundaries of research, working with top level organisations in automotive – Bosch, Cat, Daimler, Fiat, Honda, Isuzu, Land Rover, Mazda, Nissan, Scania, Skoda, Toyota, Valeo and Volvo, to name just a few – and also with the Netherlands Aerospace Centre (NLR) and the Maritime Research Institute Netherlands (MARIN).
“Our aim is make it so that the client doesn’t have to worry about things like hardware to software connections – we do that for them so they can focus on their research or design challenge.”
Ahead of a flagship product launch later this week, Bill McKinley, Automotive Strategic Planner at Keysight Technologies, gives his thoughts on self-driving and the fast-changing connected and autonomous vehicle (CAV) landscape.
Avid readers may remember that Bill was on the panel I hosted at the Small Cells World Summit in May. He’s got 30+ years’ experience in wireless communications and his current focus is developing test solutions for the automotive sector.
BM: “The UK, in line with other nations around the world, is investing heavily in connectivity and electrification – both the vehicles themselves and the charging infrastructure. Connected vehicles have been demonstrated to enhance safety via cellular vehicle to everything (C-V2X) and dedicated short-range communication (DSRC).
“These technologies allow for more efficient driving, for example, by routing to avoid accidents or poor road conditions. They also enable higher levels of automation, all of which can lead to an improved overall driving experience.
“It is likely that the first fully automated vehicles will be delivery vehicles, controlled environment shuttle type services, and buses on specific routes. With the gradual introduction of robotaxis, we will no doubt start to see Mobility as a Service (MaaS) become more common over the next 10-15 years.
“Keysight was the first test and measurement company to be awarded Global Certification Forum (GCF) validation for C-V2X RF conformance. We have industry leading validated test cases for the C-V2X protocol conformance test, and we were the first to be awarded OmniAir Qualified Test Equipment (OQTE) status.
“Cybersecurity will play a critical role in connected mobility and Keysight is working with leading companies and organisations in this space to develop solutions to ensure vehicular communications remain safe and robust against attacks.
“Clearly, the main risks associated with self-driving vehicles are around the safety aspects, which in turn will heavily influence public acceptance of the technology. We are all very familiar with some of the headlines about Tesla vehicles.
“It remains incredibly challenging to overcome the complexities of urban automated driving, but things are improving all the time. Our autonomous driving emulator (ADE) system is designed with this in mind – to test many autonomous drive systems in a rich and authentic environment within the lab, before moving testing out into the field.”
More on that to follow soon. For further info see keysight.com
Inma Martinez, author of new book The Future of the Automotive Industry, on self-driving and connected cars
Described by Time magazine as “One of the best talents in human digital behaviour”, Inma Martinez advises business leaders and governments (including the UK’s Department of Culture, Media and Sport) on AI and digitisation. She’s just written a book called The Future of the Automotive Industry, so obviously we had to ask her about driverless cars.
How did you come to specialise in automotive?
IM: “I first got involved in the auto industry in the early 2000s, when BMW recognised that they had to attract female drivers and buyers. We made a series of short films with directors including Ridley Scott and John Woo, starring Clive Owen as The Driver. Guy Ritchie’s had Madonna in it. In those days, I was working as a human factors scientist, looking at how humans use technology.
“Previously, I had been a telecoms engineer specialising in internet protocols. Then, because Nokia bought two of my start-ups, I landed in their innovations department. Together with Intel, we came to the realisation that telecommunications companies had to create alliances with auto manufacturers for vehicle to everything (V2X) and vehicle to infrastructure (V2I) communications.
“I worked for Volkswagen Group designing cars with AI and met Mark Gallagher and all the Formula One crowd. I thought: I have to write about the future of this industry, because in the next five to ten years it will not look anything like today – the massive influence of the Internet of Things (IoT) and AI, sustainability and the green economy. I wrote the book during the pandemic and it came out in June.”
Setting EVs aside, how do you view the autonomous side of things?
IM: “I love the topic, firstly because it needs so much definition. People interchange ‘autonomous’ with ‘self-driving’, but they’re separate things. Unfortunately, the media is not very sophisticated in talking about that.
“For me, it’s something that’s been happening for 15 or 20 years, initially because the industry was pressed to improve safety. You got level one autonomous features, like cruise control and parking assistance, making things easier and safer. Now we’re at level three, and no one understands what on earth is going on!
“I hate it when Tesla put out press releases claiming full self-driving. The PR houses are doing a disservice to the industry because they’re confusing people. I delved into this for the book and came up to the conclusion that we’re not going to see autonomous cars until the regulation is ready for them.
“The European Union put out a good first attempt to define self-driving in 2019, and Japan has changed a lot of its traffic laws to allow Honda to start putting level three cars on the road.
“This will only happen when the legal framework is defined. Otherwise, you have the massive legal issue of who’s at fault in a crash. There’s got to be an effort in the industry to help create these legal frameworks, and I don’t think it’s too complicated.
“The way I see it, we need to differentiate an autonomous car – a level five car which can do literally everything by itself – from self-driving cars which can drive and brake and accelerate and have situational awareness, but which can’t operate constantly by themselves and still need the driver to keep their eyes on the road.”
Proposed changes to the Highway Code talk of drivers not having to pay attention anymore. Is there a danger that regulators could jump the gun?
IM: “That is frightening. You can’t put vehicles on the road driving themselves with just computer vision, you need V2X, roadside units (RSUs), Vehicular Ad Hoc Networks (VANETs) – all the beacons that make roads smart. You need 5G infrastructure, so the car is actually guided by connectedness. This has to do with urban planning and smart cities, not with the automotive industry per se.
“The point is not just whether can we make cars autonomous, it is whether we can make them street smart. The way people drive is different in every country. In Rome, people brake all the time. In Kuala Lumpur, there are mopeds everywhere. So, the car of the future is going to have to be adaptive – the AI, computer vision, all the settings will be different depending on where it is.
“There’s a wonderful thesis that asks whether people are born street smart or whether they get it when they move to a big city. I began to think about autonomous cars driving around big urban centres – they’re going to have to get the pulse of how you drive in a certain city. We need to train the system to learn how to integrate itself.
“We’ve only just begun to consider what autonomous is, and we need to have a bold vision as to what it should be. In my view, we need to make cars smart, not just autonomous.”
What are the main risks in the shift to self-driving?
IM: “We need a legal framework. We need integration into the smart city infrastructure, including telecommunications. We also need definitions.
“Cars look fabulous at the Geneva Motor Show, but nobody talks about them in contexts. Should there be designated lanes for hands-free driving? How are we going to deal with a car parc that is not all digital, that still has a lot of older vehicles?
“Automotive is one of the hardest industries to create innovation because you have the pressure of safety, safety, safety at all costs. For example, nobody’s working on voice commands anymore because it turned out they were a distraction, a nuisance.”
Can you address the challenges specific to the UK?
IM: “Yes – your road network. In the UK you have a lot of 60mph rural roads where you can barely see what’s coming. I drive in Somerset and holy cow! It’s only because humans drive in such a super intuitive way that there aren’t more crashes.
“Perhaps it’s also because your driving test is so rigorous. I did my test at school in a small town in Pennsylvania. The police would make you drive around the car park and give you your licence. That was it.
“Then you have London, which is like no other city. It is a Dickensian city with 21st century vehicles running through it. It is a costly challenge to test smart road infrastructure without creating congestion. Where are the budgets going to come from?”
Anything else you’d like to mention?
IM: “I was speaking to a board member at Volkswagen recently and he said that one of the revelations of the pandemic was that it motivated people to own a car, rather than use public transport, for health and safety reasons, and a certain level of freedom and privacy. People have conversations when driving that they wouldn’t have on a train.
“It is also worth highlighting the prospect of the automotive industry partnering with healthcare companies on predictive medicine – keeping track of your vital biometrics to help detect serious diseases. If you’re going to be sitting in this highly technical environment for two hours a day, data such as the way you check your mirrors can reveal early symptoms of things like Alzheimer’s.
“Connected cars will add another layer of personal profiling and data authentication. Digital fingerprinting companies will be able to see that it’s me on my usual route, doing what I normally do. The cybersecurity will have to be very strong though. Imagine somebody hacking into the traffic management system of a future city – that’d be the ultimate hack.”
The sheer volume of data being collected by connected cars is soaring. Forget megabytes (MB), gigabytes (GB) and even terabytes (TB), it’s time to start thinking in petabytes (PB) and exaflops (EFLOPS).
A petabyte is equal to one quadrillion (one thousand trillion) bytes. However, rather than looking at storage capacity, there’s now been a shift towards performance, measured in floating-point operations per second (FLOPS).
At the CVPR 2021 Workshop on Autonomous Driving event earlier this year, Tesla unveiled its new in-house supercomputer, boasting an eyewatering 1.8 EFLOPS.
The University Information Technology Services tells us that: “To match what a one EFLOPS computer system can do in just one second, you’d have to perform one calculation every second for 31,688,765,000 years.”
Behind this unprecedented processing power sit important questions. Back in 2019 we asked Connected cars: whose data is it anyway? with Bill Hanvey, CEO of the Auto Care Association, warning that “carmakers have no incentive to release control of the data collected from our vehicles”.
Under the headline “Customer trust is essential to large-scale adoption of connected cars”, Engineering and Technology (E&T) recently highlighted a survey, by automotive engineering company Horiba MIRA, which asked 1,038 car owners from the UK, Germany and Italy about privacy in their connected vehicles. 42% said they were not made aware that they could withdraw their consent.
Garikayi Madzudzo, advanced cybersecurity research scientist at Horiba MIRA, commented: “Industry sources estimate that on average about 480 terabytes of data was collected by every automotive manufacturer in 2013, and it is expected that this will increase to 11.1 petabytes per year during the course of 2021.
“With such large volumes of personal information being collected, it is inevitable that privacy will be a challenge.”
This dovetails with a survey by Parkers which found that 86% of people wouldn’t be happy to share driving habit data with third-party companies.
Parkers.co.uk editor, Keith Adams, told Fleet News: “We’re agreeing to all manner of terms and conditions on a daily basis – I shudder to think what Google knows about me – but it comes as a surprise to see so few drivers are aware of what their cars knows about them.”
Meanwhile, The Star Online has published some interesting thoughts on data privacy from Volkswagen Group chief executive, Herbert Diess.
“In Europe, data belongs to our customers first and foremost – they decide what happens with it,” he said.
“In China, data is considered a common good, available for the people’s good. In America, data is predominantly seen as an economic good, is not public, but remains with the companies, with Google, with Apple, in order to serve the business model there.”
Tim Dawkins explains why the UK is so well placed to develop self-driving vehicle technologies and regulations.
With its laudable aim “to demonstrate entrepreneurship in the global public interest while upholding the highest standards of governance”, transformational technologies like autonomous vehicles are natural territory for The World Economic Forum. Here, we get the considered views of the Forum’s Automotive & Autonomous Mobility Lead, Tim Dawkins – an Englishman working for the Geneva-based organisation in sunny California.
Tim Dawkins leads a portfolio of automotive and autonomous mobility policy research activities.
Tell us about your path to autonomous vehicles and The World Economic Forum
TD: “I started out studying motorsport engineering at Brunel and my first job out of university was in vehicle security for automotive consulting firm, SBD, helping manufacturers meet Type Approval requirements with anti-theft technologies. When SBD opened an office in North America, I went there, to lead their consulting in autonomous driving. Then, in 2018, I got my MBA and wound-up joining The World Economic Forum.
“Here at the Forum, our mission is greater than to convene events for business leaders, but actually to improve the state of the world. In my domain, that means making sure that the future of transportation is as safe as possible. Broadly, we work with governments and industry leaders to help them understand each other better. In the world of autonomous vehicles that means helping governments understand how the technology is evolving and the creation of new governance structures – which can be used in regulations, standards and assessment criteria.
“A crude analogy is to think about a driving test for the self-driving cars of the future – what does that look like? It’s obviously a lot more nuanced and complex than that, but by being a neutral entity – bringing together the likes of Aurora and Cruise with leading academics and regulators to have focused discussions around autonomous vehicle operation and deployment, or what it means to define a safe autonomous vehicle – is a very effective way of achieving better outcomes for all.
“It’s not just about the advanced technologies of the future, our portfolio also includes road safety research – improving the infrastructure, reducing crashes and fatalities with today’s ADAS technologies, and looking ahead to creating a safer future of mobility with autonomous vehicles.”
With your global perspective on autonomous mobility, how is the UK doing in terms of the government’s stated aim of being “at the forefront of this change”?
TD: “The automotive industry has always been very important to the UK economy, so it is natural that that industry and the government agree on the strategic priority to make the UK an attractive place to develop and test these technologies. We have world-leading engineering talent, universities and research and test facilities within our borders, so it’s shifting the focus from sheet metal and engines over to Connected and Autonomous Vehicle (CAV) technologies. Really, it’s a great fit.
“What UK governments have done – I say governments plural, because this has been going on for over 10 years – is to create institutions which spur development. There’s been dedicated funding and research grants not only to grow the CAV ecosystem within the UK, but to encourage international organisations to come and develop in the UK as well.
“What we see now is the result of many years of building the business case, to position the UK as a competitive place to test and develop new technologies. This top-down industrial policy, combined with an open code of practice to facilitate automated vehicle trialling, make the UK a great place to test and develop AVs.
“This ecosystem view is something we study here at the Forum. We recently published a joint paper with The Autonomous – The AV Governance Ecosystem: A Guide for Decision-Makers – which looks at how the standards bodies, alliances and consortia are coming together to develop solutions which will become policy, or at least be used in future governance. You will notice that a lot of UK entities feature very prominently in this study.
“For example, BSI are one of the long-established standards institutions that have been mission-aligned to further CAV mobility, by delivering technical standards and guidance to address governance gaps in the sector, such as the new Publicly Available Specification (PAS) 1881, 1882 and 1883 documents and a vocabulary of CAV terms. Then you have entities such as Zenzic to create the business environment and inform the overall roadmap to making autonomous vehicles a reality, supported by entities such as Innovate UK, and a whole ecosystem of universities and research entities creating a thriving network for innovation.
TD: “One of the things our team like to tackle is how to incentivise these companies to go not just where they can make the most profit, but to provide services to those who most need transportation. This means providing services in areas that are underserved by public transport.
“Think about commuting into London – you drive to the train station, then get onto the TFL network. If you can make that journey more efficient, hopefully more affordable, and accessible, suddenly the economic opportunities that come with commuting into London are open to a greater swathe of people. It’s a very local issue. You have to look at each city and say: where are the areas with the least economic opportunities and how can mobility provide them with greater access to jobs, healthcare and all the things they need?
“Fundamentally, mobility should be considered a human right. It’s not codified as one, but the link between good access to mobility and access to a good future is extremely strong. When we talk to city regulators, for example, they’re very keen to view autonomous vehicles as a way of making their transportation ecosystem more efficient – using AVs to get people onto the existing network, rather than replacing buses or train services.”
That’s certainly opened our eyes to the important work of the World Economic Forum, and we’ll be hearing more from Tim’s colleague, Michelle Avary, Head of Automotive and Autonomous Mobility, at next month’s Reuters event, Car Of The Future 2021.
Vehicle-to-everything (V2X) 4G and 5G connectivity via small cells can be a lifesaver.
Carsofthefuture.co.uk has signed a media partnership agreement with The Small Cell Forum (SCF) for its three-day online Small Cells World Summit, The Future of Mobile Networks, on 11-13 May 2021.
Small Cells World Summit 2021 registration
As part of the deal, Carsofthefuture.co.uk editor Neil Kennett will moderate the Automotive & Transportation session from 11am on Wednesday 12 May, with high-profile speakers including: Peter Stoker, Chief Engineer for Connected and Autonomous Vehicles at Millbrook Proving Ground; Dr Maxime Flament, Chief Technology Officer at the 5G Automotive Association, one of the world’s leading authorities on Intelligent Transport Systems (ITS); Bill McKinley, Connected Car Business Lead at Keysight Technologies; and Mark Cracknell, Head of Connected and Automated Mobility at Zenzic, responsible for accelerating the self-driving revolution in the UK.
Neil Kennett, said: “We are delighted to partner with The Small Cell Forum for this exciting virtual event, which brings together mobile operators, vendors and regulators from around the globe. The Automotive & Transportation session will focus on connected and autonomous vehicle (CAV) opportunities, particularly vehicle-to-vehicle (V2V) and vehicle-to-everything (V2X) communications, in-vehicle payments, and the rival ITS-G5 and C-V2X 5G technologies.
“Small cells deliver high-quality, secure 4G and 5G coverage, so there are clearly a multitude of new use cases in the connected car world and the wider mobility ecosystem. Aside from supporting self-driving, they can facilitate everything from in-car infotainment and shopping, to fixing technical problems before they occur and pre-empting likely crash scenarios. It is no exaggeration to say they could be a lifesaver.”
Carsofthefuture.co.uk readers can benefit from a 40% discount on Small Cells World Summit 2021 tickets using the code SCWS2021. See www.smallcells.world/
CGA’s simulations train autonomous vehicles to deal with environments specific to the UK.
Our Zenzic CAM Creator series continues with Liverpool-based Jon Wetherall, Managing Director of CGA Simulation, and Max Zadow, Director of Future Coders.
By applying gaming knowledge to real-world mobility questions, CGA has created engaging simulations to study autonomous driving and smart city solutions.
JW: “My background is gaming. I used to work for the company that did Wipeout and F1 games. We made a racing game called Space Ribbon and one day, about five years ago, we got a call from The Department for Transport (DfT). They were doing a research project on virtual reality (VR) in the testing and training of drivers, specifically hazard awareness.
“We turned it into a game and it worked – people said their attitudes changed as a result of our simulations. The hardest scenario came early in the game – a parked lorry with a big blind spot – and a lot of people crashed. VR feels so visceral, the experience can be quite vivid and shocking. Of course, smarter cars will hopefully fix these types of situations.”
CGA Simulation junction and forecourt
To pursue this goal, CGA received a grant from Innovate UK to create an artificial learning environment for autonomous driving (ALEAD).
JW: “The aim was to make these cars safer and we stayed true to our computer game history. We didn’t have the resources to lidar scan the whole area, so we did our own thing using mapping data. We made a digital twin of Conwy in north Wales and unlike other simulations we kept all the ‘noise’ in – things like rain. This was important because it is now well-understood that noise is a big challenge for autonomous vehicles (AVs).
“Modern autonomous driving stacks have 20 different subsystems and we generally focus on only one or two, to do with perception. There’s been massive progress in this area over recent years, to the extent that artificial intelligence (AI) can identify an individual by their gait. What’s more, you can now do this on a computer you can put in a car – this is one of the cornerstones of driverless.
“It’s not the first time people have been excited about AI. In the 50s they were saying it was only a few years away. It has taken much longer than people thought, but major problems have now been solved.
“We are lucky to have one of the world’s leading experts in radar on our doorstep, Professor Jason Ralph of The University of Liverpool, and he helped us develop the simulation. You have to feed the car’s brain, a computer, all the information it will need – from sensors, cameras, GNSS – and you can do all that in the software.”
MZ: “In particular, The University of Liverpool were interested in how weather affects things, right down to different types of rain and mist. In California, if an AV encounters conditions it can’t handle, like heavy rain, it pulls to the side of the road. That’s ok for San Francisco but not for Manchester!
“A few years ago, everyone seemed to be using the example of an AV encountering a kangaroo. How would it cope? The point is you can use our simulations to train cars, to create algorithm antibodies for once in a lifetime events and regular things in different environments. That remains an essential part of what’s needed to make AVs a reality.
“We picked Conwy partly because it has very different patterns of land use to America. An early use case for AVs is predicted to be taxis, but in the UK these are most frequently used by people who don’t own their own car, and they often live in high density housing or narrow streets. The operational design domains (ODDs) are going to have to deal with environments specific to this country – steep hills, roads which twist and turn, and changeable weather.”
Mobility Mapper
Wetherall and Zadow’s latest collaboration is Mobility Mapper, a project to create greener and more intelligently designed transport hubs. The technology underpinning Mobility Mapper has been used previously by the team to model Covid 19 spread, autonomous vehicle technology and by the Liverpool 5G Create project (funded by DCMS as part of their 5G Testbeds and Trials Programme).
JW: “E-hubs are basically an extension of what used to be called transport hubs – train or bus stations. They’ll provide charging facilities and access to different modes of transport, for example, you can drop off an e-scooter and hop into a shared autonomous car.
“Here in Liverpool, there was a big trial of e-scooters, big in international terms not just UK. The worry was that a lot of them would end up in the canal, but that didn’t happen. The trial was incredibly successful. It’s all about linking that movement and nudging people away from car ownership.”
MZ: “We were already thinking about how Jon’s technology could be used for mobility as a service (MAAS) when we attended a virtual future transport conference in LA with the Centre for Connected and Autonomous Vehicles (CCAV).
“That was an influence, as was an Intelligent Transportation Systems (ITS) trade show in Copenhagen, where we saw an autonomous tram system designed to take bicycles. It was a small step from there to imagining autonomous trams carrying autonomous delivery pods.
“This is classic smart city stuff but you need to know how these e-hubs are likely to be used, with no track record, nothing to go on. We need simulated environments to make best guesses in. That’s Mobility Mapper.”
JW: “It is early days, still in the development phase, but the authorities in both Manchester and Liverpool have agreed there’s a need for such a predictive simulation tool.”
As we wrap-up a thoroughly enjoyable interview, Max dons his Director of Digital Creativity in Disability hat: “Autonomous delivery bots are basically electric wheelchairs without a person, so there’s clearly a potential benefit, but there needs to less wishcasting and more real work on how accessibility will be affected.”
Our Zenzic CAM Creator series continues with Peter Stoker, Chief Engineer for Connected and Autonomous Vehicles at Millbrook.
Part of CAM Testbed UK, Millbrook Proving Ground in Bedford boasts 700 acres of private roads on which to develop and test connected and autonomous vehicle (CAV) technologies. As Chief Engineer, Peter Stoker is right at the forefront of self-driving in the UK.
Peter Stoker, Chief Engineer for Connected and Autonomous Vehicles at Millbrook
Please can you outline Millbrook’s work on connected and automated mobility?
“My primary role is to bring focus to two testbeds, our CAV testbed and our 5G testbed. We are not a purpose-built CAV testbed – we have safety, propulsion and conventional vehicle test facilities too – so CAV is something we’ve blended into the existing business.
“For the CAV testbed, we partnered with the UK Atomic Energy Authority (UKAEA), particularly the Remote Applications in Challenging Environments (RACE) division, to provide a controlled urban environment. We have three open source StreetDrone vehicles and miles of track with targets for very precise measurements, accurate to 1-2cm. We offer safety driver training and also have a simulation environment for driver-in-the-loop and hardware-in-the-loop testing. The whole idea is to fail in private, not in public, and to progress, to evolve out of the testbeds and on to open roads.
“The 5G testbed is a completely separate consortium, backed by the Department for Digital, Culture, Media and Sport (DCMS). We have 59 masts looking at all types of connectivity and I’d say the millimetre wave at 70GHz is currently the most interesting.”
Millbrook Proving Ground graphic
What major shifts in UK road transport do you expect over the next 10 years?
“Getting the crystal ball out, I see increased use of connectivity in existing vehicles and some very interesting new use cases – buses connected to city networks, video analytics from cameras, smart ambulances streaming live data, autonomous deliveries on campuses. What I don’t see within 10 years is millions of privately owned driverless cars. That will start in the luxury sector but to begin with it will be more about transporting goods.”
How do you see the testing framework for CAVs developing?
“There’s a lot of simulation in the automotive world – crash testing, fatigue testing, computational fluid dynamics. These days, manufacturers are developing whole vehicles before building a prototype. You have to have a good simulation on a good simulator and there’s an interesting shift that needs to happen on regulation. It’s early days on that, but it’s essential.
“The strength of virtual space is that you can run hundreds of scenarios in machine time – not only set up complicated scenarios that would take days with real cars, but actually speed up the process so it runs faster than real time. The national scenario database is already really good and regulation will move to being a mixture of real and virtual certification – global, European, UK and perhaps even city-specific. We are happy to advise, but don’t set policy.”
What are the biggest challenges in the shift to self-driving and how can these risks be mitigated?
“The key to the future of self-driving is education, education, education – for everyone, the public, vehicle manufacturers, the aftermarket, recovery operators. We have to work on the terminology – autonomous, driverless, CAV, CAM – it’s confusing, even to people who know what they’re talking about.
“At the moment, we’re making it harder to understand, not easier. We’re in a really grey area of transition with different trade names for systems. There’s a lot of groundwork needed to prepare people and, for example, the brilliant website mycardoeswhat.org does a great job of trying to explain it.
“If you get into a hire car, you need to have the right expectation of what it does and what it doesn’t do. If you buy a new car, you should read the manual, but how many people do? Especially with Covid, more cars are being delivered with minimal interaction – it’s a case of “there’s the key, where’s the station?”. Too often, the customer handover just isn’t there.
“How are garages, the aftermarket and the amber light sector going to deal with all this? Basic questions like how do you put it in neutral? ADAS has already led to huge changes in training and skill sets – how to calibrate and monitor them.
“We haven’t talked about over-the-air (OTA) updates, cameras embedded in the tarmac or even electrification – there’s a huge amount of things! How do you learn about them? Hopefully in testing rather than in crash situations.”
Our Zenzic CAM Creator series continues with Raunaq Bose, co-founder of Humanising Autonomy.
Before establishing predictive artificial intelligence (AI) company Humanising Autonomy in 2017, Raunaq Bose studied mechanical engineering at Imperial College London and innovation design engineering at the Royal College of Art. Focusing on the safety of vulnerable road users, Humanising Autonomy aims to redefine how machines and people interact, making cities safer for pedestrians, cyclists and drivers alike.
RB: “Our model is a novel mix of behavioural psychology, deep learning and computer algorithms. We work with OEMs and Tier 1 suppliers on the cameras on vehicles, with the aftermarket on retrofitted dashcams, and also with infrastructure. Our software works on any camera system to look for interactions between vulnerable road users, vehicles and infrastructure in order to prevent accidents and near misses. While most AI companies use black box systems where you can’t understand why decisions are made, we set out to make our models more interpretable, ethically compliant and safety friendly.
“When it comes to questions like ‘Is this pedestrian going to cross the road?’, we look at body language and factors like how close they are to the edge of the pavement. We then put a percentage on the intention. Take distraction, for example, we cannot see it but we can infer it. Are they on the phone? Are they looking at the oncoming vehicle? Is their view blocked? These are all behaviours you can see and our algorithm identifies them and puts a numerical value on them. So we can say, for example, we’re 60% sure that this pedestrian is going to cross. This less binary approach is important in building trust – you don’t want lots of false positives, for the system to be pinging all the time.
“One of the main things we’re likely to see over the next decade is increased use of micromobility, such as cycling and e-scootering. At the same time you will see more communication between these different types of transportation, and also with vehicles and infrastructure. The whole point of ADAS is to augment the driver’s vision, to reduce blind spots and, if necessary, take control of the vehicle to avoid a shunt. Then there’s the EU agreement that by 2022 all buses and trucks must have safety features to detect and warn of vulnerable road users.
“We currently only look at what’s outside the vehicle, but with self-driving there will be monitoring of the cabin. In terms of privacy, we have a lot of documentation about our GNPR processes and how we safeguard our data. Importantly, we never identify people, for example, we never watch for a particular individual between camera streams. We look to the future with autonomous cars but for now we’re focused on what’s on the road today.”
