Hey Folks đ
We talk about cars yet again. Coz tech is everywhere, and everything is now tech.
We humans have a simple way of driving cars. We see the vehicles around us through our eyes. Our brains help us make sense of that sight, and we use our hands and feet to take action.
This is almost second nature to many of us, but for machines, the story is quite the other way around.
If youâre in one of the few cities around the world that are experimenting with self-driving cars, you must have wondered: âHow do these cars actually see?â.
This article is exactly for that curious person wanting to know a little bit more about that driverless car going by his side without a real driver.
In my previous article, we did a high-level tour of self-driving cars. We briefly went over the big players, the key technology and the self-driving levels that companies largely ignore.
This article is the 2nd part of that same series, where weâll be going a bit deeper into the technology that powers self-driving cars and how there are two popular approaches dividing the self-driving industry.
What is LiDAR?
We went over a brief intro to LiDAR in the previous article but Iâll still go over the basics.
The term LiDAR stands for âLight Detection and Rangingâ. LiDAR sensors are like very small radars that are placed on some self-driving vehicles.
LiDAR sensors shoot a lot of invisible lasers every second that are spread in all directions, and the time it takes those lasers to bounce back is used to simulate the environment around the vehicle.
Basically LiDAR paints a real-time 3D image of the world around the vehicle.
Hereâs how LiDAR generally works:
LiDAR sensors emit millions of infrared laser pulses per second
Computer measures how long all of the pulses took to bounce back
The visualization software generates a real-time, precise 3D map of surroundings that includes objects around the cars that could be other vehicles, or even people
There is a cool analogy to explain LiDAR: Imagine going to an empty cave and shouting at the top of your lungs. Your voice would bounce echo to you but with a delay of a couple of seconds.
A similar setup is going on in LiDAR but here instead of sound we have invisible lasers bouncing back at the speed of light.
The Good and Bad of LiDAR
Some Pros of LiDAR
LiDAR sensors are highly accurate and they can sketch detailed maps of objects surrounding a self-driving car
LiDAR sensors unlike cameras also work at night
LiDAR sensors provide added redundancy. Many companies donât just use LiDAR alone, but they use LiDAR in combination to cameras, radar and AI software systems
We can say that, especially in self-driving cars, LiDAR sensors are part of the team that handles self-driving inside the cars. They are one of the players of that team along with other self-driving hardware and software that deliver self-driving capabilities.
Some Cons of LiDARs
LiDAR sensors are notorious for being expensive. A single unit costs somewhere in the thousands of dollars but recently the price has come down because of increased manufacturing
LiDAR based sensors are spinning sensors and the added hardware does add bulk and weight to the self-driving vehicle that may interfere with vehicleâs performance to some degree
Natural weather conditions like rain, fog, smog, can scatter the beams of LiDAR sensors which can directly interfere with LiDAR data
At this moment, LiDAR sensors, especially in self-driving cars appear to be harder to scale to consumer-level pricing. The only players that are using it today are heavily funded by giant companies like Alphabet (Company that owns Google)
Fun Fact: Waymoâs early prototypes had LiDAR units worth $75,000 now dropping roughly around $1,000 showing cost progress.
What is Computer Vision?
Computer Vision is a branch of Artificial Intelligence with a goal for computers to be able to see and interpret the world around us through images and videos.
We humans use our eyes to capture the imagery of our surroundings. Our eyes have a direct link to our brains. Our brain accepts the input from the eyes and makes the imagery that we process.
Computer Vision has a similar goal. It wants to teach computers how to make sense of raw image/video input from digital cameras and use AI software systems to process that input.
And thatâs why Computer Vision has a direct application in self-driving cars because out of many other applications of Computer Vision, cars make up a really good application because if done right, it can help humanityâs need to travel and even save lives and time.
So, In comparison to LiDAR systems, Computer Vision is a bunch of cameras with a relatively small computer with sophisticated AI software that is trained to take decisions just like a human driver would.
This AI software makes use of deep learning and other breakthrough technologies in Artificial Intelligence, combined with digitally controlled actuators that control your car.
LiDAR sensors can create a 3D map of a vehicleâs surroundings while Computer Vision is limited to a 2D view just like humans are through a windscreen.
A significant difference between LiDAR and Computer Vision is cost. Earlier LIDAR installations were tens of times more expensive than Computer Vision hardware but this steep difference is gradually falling.
Although part of the cost of Computer Vision is company-operated or cloud data centers that are used to train the AI models that Computer Vision vehicles use.
But still, the cost divided by a big fleet size is still less expensive than a Computer Vision system.
You may think of this as an analogy but self-driving companies literally train their AI software models to learn from dashcam videos recorded by humans. The machines literally observe humans driving and try to replicate it as best as they could.
Common examples of Computer Vision Only companies include Tesla, Comma.ai, Mobileye, etc.
The Good and Bad of Computer Vision in Self-driving Vehicles
Having gone through the pros and cons of LiDAR, letâs discuss some of the pros and cons to a Computer Vision Only approach.
Pros
Obviously the biggest pro for a Computer Vision approach is significantly cheaper hardware than LiDAR focused approach
Because Computer Vision systems are cheaper, they are much more easier to scale to big fleets of cars, Theyâre more scalable
LiDAR sensors cannot see colors like traffic signals or signs naturally, Computer Vision system do see Lane Marking, Traffic Signals, Road Banking, etc
Computer Vision systems are more close to how humans learn driving, they literally observe humans drive from millions of real-world driving miles (fleet learning)
Cons
Computer Vision systems are more prone to visual confusion. They canât see at night. Rain, glare, fog, poor lighting, etc affect these systems way more than LiDAR based systems
Computer Vision systems do have a limited depth of perception unless they are paired with sensors. This means that the actions they take are based on a limited set of data compared to LiDAR based options
Most Computer Vision Only self-driving systems are limited to only SAE Level 2â3.
If you donât know what SAE Levels are, you may check out the previous article in the Self-Driving series where I covered what these levels are and what each of them actually means for you:
The Two Approaches to Winning Self-Driving
When you study the tech-stack of most companies working on solving self-driving technology, youâd see a classic red-pill, blue-pill split among their approach to problem solving.
The two biggest players; Tesla and Waymo show this difference. The former is focusing on Computer Vision only while Waymo utilizes LiDAR sensors along with a bunch of other hardware.
Can $200,000 cost per car ever become sustainable?
The LiDAR focused approach is expensive despite LiDAR prices going down in recent years.
In the early days, automotive-grade LiDAR systems were sold even for thousands of dollars a sensor.
That is because they had moving parts and had to maintain vision while moving; that was itself an engineering challenge.
Waymo is among the most successful among companies using LiDAR focused approach for self-driving and many experts agree with their approach of using LiDAR as a redundant backup while using Computer Vision too.
Companies like Waymo that are a proponent of the LiDAR approach use Computer Vision in addition to LiDARs, radars, geofencing, additional sensors and GPS, etc.
This is how Waymo has reached close to SAE Level 4 but this is not without its own challenges.
Many of the big proponents of LiDARs are companies that are purely self-driving companies with big backers to fund them. Itâs Googleâs parent company Alphabet in Waymoâs case.
The first challenge to a LiDAR focused self-driving approach is the cost. It is rumored that a single Waymo care costs somewhere between $150,000 to $300,000.
And that is why some experts believe that these companies are just burning cash waiting for the moment to break even, that may not even happen.
The second challenge for this approach is limited geographies. Currently services like Waymo that have got to Level 3 or 4 are limited to just a few cities.
And thatâs by design because they need to rely on geofencing and other technologies that arenât as scalable across the world as Computer Vision could.
Can You Really Drive a Car with Just Cameras and AI?
The answer to this question is a lot more nuanced than it sounds and Iâm in no way qualified to present a definite answer.
But I can present opposing arguments on this debate so you can decide for yourself as there are reasonable voices from both sides.
Many companies, most prominent Tesla, believe that they could reach full self autonomy just using cameras and AI software.
There are people on both sides of this argument. Some believe that Computer Vision will get really good with time and once that happens it would scale a lot faster than self-driving cars that cost around $300,000 a unit.
There is also a large pool of people that believe that it is not a good idea to give a software this much control especially when it isnât independently verified. Some even challenge that Computer Vision would get to the point of SAE Level 3 or 4.
And that is the main factor behind Computer Vision companies. These companies are betting on software getting to a point where it can replicate human behavioral driving instead of expensive sensors, radars, LiDARs, GPS, geofencing and a few more technologies.
What the Future May Look Like
There are also two answers to this question:
Carcinization: Many experts like to believe the industry will eventually merge with both schools of thought. This means that in the future, we may see Computer Vision and LiDAR based approaches merged into self-driving solutions
Software will eat LiDAR - This is the hope for companies like Tesla. They want Computer Vision software to become so good that we donât need LiDAR or expensive sensors meaning eventually, perception algorithms may get so good that sensors become supplementary, not essential.
Still, even if we get the self-driving software to become so good that we donât need sensors, we may see regulators pushing for them just because of redundancy or increased safety,
Personally, I would love to see a world where cars drive themselves and humans focus on better things to do but I still think itâd take time to be comfortable with software making decisions for our life.
But the way AI has been advancing these last year, who knows whatâs going to happen and how far will we give up control to machines that learn from us.
Still, Iâm excited for new technologies that would come out of these very self-driving technology investments.