Self-Driving Technology: Lidar vs Camera

Nowadays, you cannot read news about investing without reading the large amount of capital that different companies in the self-driving industry attract (either through SPAC or private investments). In this week’s article we discuss two paths towards a self-driving future.

The two paths towards a self-driving future

Not all-self driving technologies are created equal. There are five levels of driving automation as defined by the Society of Automotive Engineers (SAE). The higher the level of self-driving designation, the less human intervention required.

Currently, no one has achieved the highest level, Level 5 automation (L5). Tesla’s ‘full self-driving’ (or FSD) system, despite the name, is categorized as L2 automation. Waymo has achieved some success with Level 4 on a limited basis (it runs a driverless pilot Level 4 (L4) automated robot taxi service in the Phoenix, Arizona, metro area). Currently, it is unclear when we can achieve L5. What has been true, however, is that the companies in the field have been underestimating the time it takes to achieve this milestone:

• In 2018, Waymo prepared to order 82,000 self-driving vehicles. As of this writing, it has about ~600 cars in its pilot fleet.
• Uber thought that by 2020, it would have 100,000 self-driving vehicles by 2020.It currently has none after selling its self-driving division to Aurora Technologies.
• Similarly, Lyft thought that by 2021, the “majority” of its rides would be done autonomously.

So what went wrong? Turns out, getting to L4/L5 is much harder than anyone ever thought. Roughly speaking, there are two approaches to achieving L5: (i) the evolutionary approach (ii) the straight to full autonomy approach. See Figure 2.

The evolutionary approach is a strategy where you slowly level-up towards L4 automation. This approach uses multiple off-the-shelf technologies such as radar, camera, and sonar, but not lidar (lidars are too expensive at the moment – more on this later). By using off-the-shelf technology, companies using this strategy can deploy advanced driver-assistance systems (ADAS) into existing vehicles and evolve their capabilities to achieve greater levels of automation over time. Anyone who has ever run/started a company knows that this is the preferred approach. One can continue to de-risk the technology, while getting cash flow from selling ADAS systems to existing cars. This is the strategy that Tesla, Mobileye, and several others are pursuing.

In contrast, the second approach is to develop the necessary technology (specifically lidar) to jump straight to L4 automation. This is the approach that Waymo, Cruise, and Aurora are taking. Notice that these are the companies that are amassing large amounts of funding from private investors or are going public via SPAC. These companies generally do not have products in the market yet, so they rely on investment dollars to continue to fund development – and development is taking longer than expected.

The key difference between the two camps is their answer to the question: Do you need lidar to achieve L4/L5 automation? The companies employing approach 2 believe so. They do not believe that evolving ADAS technology will ever be sufficient to achieve L4 automation.

In contrast, the camp that is taking the evolutionary approach believes that you don’t need lidar. As Elon eloquently put it: “Humans drive using 2 cameras on a slow gimbal & are often distracted. A Tesla with 8 cameras, radar, sonar & always being alert can definitely be superhuman”.

So, who’s right?

Lidar vs. vision based system

The lidar approach

What’s a lidar? Lidar is light based radar. It shoots lasers around the vehicle, and as light is reflected back to the device by the surroundings, the vehicle creates a high fidelity map of those surroundings. As you can see in Figure 3, lidar is used in conjunction with radar and cameras for an L4 vehicle. In 2015, the cost was US$75,000 per device. It has since gone down to ~US$500 per device. And despite the continued decline in costs, lidar is still too expensive for mass deployment and may not be as scalable (Tesla’s gross margin per car is around US$2,000 per car – adding lidar would destroy margins).

 

Despite lidar’s costs, proponents of this approach believe that this is the only way to achieve L4/L5 automation. Here are key characteristics of this approach: 

• Lidar captures the environment at a much higher detail than a vision based camera system alone can.
• The higher detail captured from the environment, shown in Figure 4, is then fused with other sensors (radar, vision, etc) to create an HD map and to train a machine learning model that drives the car.
• The machine learning model requires a lot of training data. This means that you either deploy a very large fleet of vehicles equipped with lidars to drive through all the different cities, under different driving conditions (which is too expensive to realistically do), or use a lot of simulated driving to train machine learning models. For example, Aurora was able to simulate 2.25 million unprotected left hand turns before testing that capability on public roads.

The vision only approach

The proponents of ADAS based systems, on the other hand, are trying to use camera (vision) based systems, sometimes augmented with other sensors (radar, sonar, etc). The most advanced on this front is likely Tesla. Unlike other proponents of the ADAS based strategy, Tesla vertically integrates its self-driving technology. It builds the car, the sensing technology, the data, and the machine learning models. Other carmakers partner with external ADAS providers (such as Mobileye – owned by Intel) that develop the sensing technology, data, and machine learning models for the car makers.

Recently, Tesla’s FSD system ditched the radar and now relies solely on cameras alone. Cameras used in this vision system are cheap and commoditized (less than US$100 for the entire car), and each Tesla is equipped with eight cameras. Here’s a video of the FSD system in action.

Not to be outdone, Mobileye showcased its vision only self-driving systems driving around New York a few days ago. Unlike Tesla, who’s all in on a vision based system, Mobileye is developing a different subsystem for vision and lidar, spreading its bet across the two approaches.

Advancement of vision only systems by these two companies was not foreseen just several years ago.

Here’s what a vision only system means:

• These are CMOS based cameras you find in your smartphones. They are cheap.
• Because they are much lower costs, you can deploy at a massive scale.
• Massive deployment means that even though the surrounding data generated is of lower resolution than that of lidar, a vision based system can overcome the resolution limitation by training its neural network with a much larger data set.
• Also, because the system cost is much lower, it can be deployed in shadow mode in production cars today. In shadow mode, the system records the driving data, lets the human drive, and compares with what the machine learning model would do. This transforms the problem into a supervised learning problem, where the user not only pays for the privilege of acquiring the car, but also helps train the system to be better! Tesla has millions of cars globally in training its machine learning model.

Sounds good so far? A few caveats! Relying on vision alone means that there’s no hardware redundancy in the self-driving system. If the camera fails, the system won’t work. Vision based systems won’t work when visibility is severely hampered (rain/snow/fog) – although lidar based systems won’t work either, since the lasers use wavelengths that are in the new visible range.

Which strategy will prevail?

Although it’s still unclear which strategy will prevail, what history tells us is that achieving L4 autonomy may take longer than expected. This is why all these lidar companies SPACed when the SPAC market was hot (Velodyne, Luminar, and soon, Aurora). As an industry, the revenue is pretty much non-existent (Figure 5). The projected industry revenue is less than US$9 billion by 2030. That’s less than 1% of Amazon’s current quarterly revenue…

Any further delays in achieving L4 automation may deal a fatal blow to these companies, as generating enough cash flow to continue development will be challenging if the macroeconomic situation changes.

 

Aurora, along with Waymo and Cruise, are the three leading companies deploying their own lidar solutions. Unlike the other two, Aurora is focusing on the commercial trucking space (this vertical has more highway miles – which is easier to tackle). Since Aurora is SPACing soon, the company provides projections about its future, and you can use this to gauge the expected time lidar based cars will be ready for prime time on public roads. 

As you can see in Figure 6, Aurora projects that it won’t make any meaningful commercial revenue from the autonomous system until 2024. If all goes well, full deployment of its technology is not going to happen until 2024/2025. So it might be that the optimistic case is ~4 years out before full deployment. 

The two technological approaches are converging towards the same L4 future. The folks at the lidar camp are racing to push the cost down as fast as possible, so that they can deploy more vehicles and get more real world coverage. Meanwhile the vision based only folks are racing to ladder up the automation level by advancing the machine learning models. 

In the past, any time there were two competing standards/approaches for the same application, the winner wiped out the losers. Do you remember Betamax vs. VHS? Or HD DVD vs. Blu-Ray? The winners of these standard wars gained mass adoption rapidly, while the losers disappeared.

For the lidar system to prevail, two things must happen: (i) It has to achieve L4/L5 automation, and (ii) it has to be at the same cost or cheaper than the camera based system.

Meanwhile, for the vision based system to prevail, only one thing has to happen: (i) Achieve L4/L5 automation level, as the cost for these cameras have declined so much thanks to the smartphone industry.

Although it is still unclear if vision based alone can achieve L4/L5 automation level – it is undeniable that the ADAS/vision based strategy has the cash flow and data advantage.

Tesla’s FSD pricing change

On a side note, previously, if you wanted to get Tesla’s ‘full self-driving’ (FSD) package, you needed to pay an extra US$10,000. Turning on FSD involves no new hardware change, however. The unlock is an over the air software update. And recently, Tesla announced that instead of paying the full amount upfront, you can pay US$200 per month, which is a 50 month payback period.

This move has two benefits for Tesla:

• Significantly lowers the barrier to adoption.
• Adds a recurring revenue stream. On a per dollar basis, recurring revenue stream is valued more highly by investors.

If the FSD is adopted widely, the add-on revenue is very profitable. In Q1 2021, Tesla’s average gross margin was US$12,900 per car sold. Adding US$10,000 of software based revenue stream (which is pure protift) can almost double its gross margin per car. More adoption also means more data for the shadow mode to train the self-driving cars.