Driverless cars need 3D maps and engineers say that's a 'very hard problem'
by Leslie Hook
When a self-driving car looks at the world, there are many things it sees. It has radars that measure distance to the next car, it has cameras that take in colour images of the street and its Lidar sensors send out laser pulses that gauge the surroundings. For any robot-driven car, one of the most important components of the journey is not just what it sees but what it knows beforehand about the area it is travelling through.
The robot needs a map, but not just any map – these cars need a three-dimensional representation of the environment around them, one that is updated continuously and is accurate down to the centimetre. As it cruises through the streets, a self-driving car collects more than a terabyte of data a day, enough to fill 1400 CDs. With that much detailed information coming from the car's many sensors, however, it is uneconomic to send it through a network like the internet.
Instead, companies have to physically move the data from one hard drive to another, a process sometimes called the "sneakernet" because engineers joke that the hard drives move at the pace of their own footwear.
The data collection is part of a great race to amass knowledge about the physical world that can be used to train the new generation of cars. Researchers hope that eventually the base layer of information will have applications not just for transport and logistics, but also for the development of augmented reality technologies – becoming like a simulation of the real world that can be used by any robot, drone or car.
The first step in realising this potential, however, is the development of effective digital mapping technologies for self-driving cars. The cumbersome storage of data is just one of the technical issues that are occupying many of the brightest engineering minds in Silicon Valley. Without better 3D maps, the much-hyped self-driving car revolution will be much slower to materialise.
'A very hard problem'
"This is a very hard problem," says Brian McClendon, a mapping expert who previously ran Google Maps and co-founded the company that became Google Earth. McClendon, who led Uber's mapping efforts after he left Google, departed from Silicon Valley last year to go into politics in Kansas. He is an adviser to DeepMap, a mapping start-up founded by former Google colleagues.
The reason these maps are so important for self-driving cars, he says, is not only for location but also "because it reduces the amount of work that the autonomous software has to do to recognise the world around it".
By comparing their actual surroundings to what was predicted in the map, he says, they can focus their attention only on things that are different, like identifying a pedestrian or a bicycle.
Investment in autonomous vehicle research has reached record levels in the past year and along with it has come a surge of funds to improve mapping. Start-ups such as Civil Maps, DeepMap and Lvl5 have attracted mapping engineers from Google, Apple and Tesla, and raised more than $US40 million in funding. Google won the first race to digitally map the world, but this array of competitors is spending big to stop them doing it again.
The biggest autonomous car companies all have their own mapping systems. Alphabet's mapping prowess is seen as a key advantage for its self-driving car unit Waymo, which has already completed more than 4 million miles of autonomous driving. (Alphabet owns Google Maps, Google Earth, Google Street View and the navigation app Waze, which tracks real-time traffic).
Tectonic plates have an impact
Within the industry there is a vibrant debate about whether to call these visual representations "maps" at all, such is their complexity. The information collected can be roughly divided into layers: the physical location of the pavements, buildings and trees; road signs and traffic lights; and how the self-driving car should behave, such as observing the speed limit. Accuracy is so important that even small changes, such as the shifting of tectonic plates that move a few inches a year, can have an impact.
"The word 'map' is an inaccurate way of describing it," says Wei Luo, chief operating officer at DeepMap. She prefers to think of it as a piece of software that feeds the car the information about its surroundings. Her colleague James Wu, DeepMap's founder, describes these maps as the "part of the brain" of the autonomous robot that allows it to understand its location.
"I tend to see the map as the 'collective memory' of all self-driving cars," says Ralf Herrtwich, head of automotive maps at Here, which is mostly owned by German carmakers after a consortium bought the map-making unit of Nokia for $US2.8 billion in 2015. "It's almost like a driving school for autonomous vehicles," he jokes.
Regardless of what they are called, making these maps is extremely difficult. The huge volume of data used in the maps is one dilemma. Another challenge is keeping them updated continuously, so that they provide the latest information to the cars.
Driverless cars are 'geofenced'
"A lot of companies have not figured out how to actually store their data," says Sravan Puttagunta, chief executive of Civil Maps, a mapping start-up. "That is why autonomous vehicles are geofenced. They physically cannot fit the data in the trunk of the car, so they are restricted to certain areas," he says.
Civil Maps is trying to deal with this problem by simplifying the map data so that it is easier to manage, but there is no single industry standard that has won out yet. Moreover, the artificial intelligence needed to generate these maps is still far from perfect. Humans are often needed to check the labelling on maps, assess the need for any updates and analyse why the cars make mistakes during test drives.
"One thing that is really not talked about with AI is how much human work is really required behind the scenes to get this technology to really work," says Alexandr Wang, an engineer whose company, Scale API, provides human trainers for AI.
"As these companies try to produce autonomous vehicles they need a bunch of humans to go through and meticulously label these maps, just like Google Maps back in the day."
Another challenge is the deep fragmentation of the sector, with no obvious common standard for these high-definition, 3D maps, nor any sharing of data because companies consider this to be important proprietary information.