AI in the Wild

September, 2022

How is AI shaping the future of transportation? In 2019 I tried to answer that question at Maersk Growth. My aim was to show how AI is enabling us to move robots out of the closed world environments and into the real world.

Lyft sold the Level 5 unit to Woven Planet in 2021. I left Lyft in April 2022. Yet, this talk still gives a good primer on AV development and a behind-the-scenes look at designing for applied AI.

The shift to AI is only just beginning and will push the human race forward in ways we can’t even imagine. The 20 min. talk covers our team’s work at Level 5 and why designing bits to move atoms is so important. The video includes a 30 mins. audio Q&A on the topics of feasibility, ethics, the role of design, future of cities, and the challenge of blazing a new path.

Transcript

Johan: [00:00:00] Thanks everyone for coming. My name's Johan. And let's see here. This is a talk about AI. It's not really about AI. It's about the process. We use at Lyft to make a car autonomous.

A little bit about me, my background. I am currently at Lyft where I head up the design team for the autonomous car. And before that I went and spent six years at Google, specifically Google X and Google Robotics. I worked on this bad guy up here. And if you wonder why there's like a weird Google Sign-in button right here is because the first work I got at Google was actually designing that. So, if you're ever annoyed at that you can sort come to me. Um, and before that I studied philosophy. I went, spent some time at Stanford, studied online trust and credibility, which at the time wasn't really a problem. Now it is a problem. I have a BA philosophy and MS in computer science.

I'm here because I work at Lyft and we are tackling these like big foundational changes in society. That's what we'd like to tell ourselves. And that's what I think we are doing actually. But Lyft, just for you [00:01:00] who aren't as familiar with this, Lyft is the second largest Ridesharing company in the US. We have 39% off the market. And we're, I think still, I can say this, the fastest growing, I hope. But basically this is our core service. But I think it's important to kind of set this in, in the context of where we are now. So Lyft, the mission of Lyft is to improve people's lives with the world's best transportation. And we have to do that in a number of ways. One of those is obviously our Ridesharing model. Another one is micromobility. So scooters and e-bikes. And a third is this.

The self-driving car. And the self-driving car, I think is important to frame that again. I think it's important to always explain clearly why we are doing anything these days. But the self-driving car is important because we have a lot of infrastructure and, secondly, most importantly, I think safety. So every year, 1.25 million people die in traffic related accidents. About, what is it, 94% of those are human related. So if we can cut that number, it's a massive, massive human cost difference. [00:02:00] In the US alone it's about 40,000 people every year who die in these accidents. And obviously there's a lot more we can do with the self-driving car. We can optimize our cities around people instead of cars. We can rethink what transit looks like. We can make areas that are less desirable today, more desirable if they're not on sort of a central pipeline. A bunch of stuff. But it's in the future.

Just a quick note on what I actually do and on my team, what we do. So we work on all the bits and bobs that are designed related. So, industrial design, UX design, app design, process design, experience design, partnerships, Lyft app, all that fun stuff. So it's a wide gamut. We have user researchers, 3D designers, industrial designers, all that fun stuff.

Okay. So solving the really hard problem. Getting a car to work in the real world. So imagine this. This is like worst case scenario. I think this is in New Delhi. We have cars that are not obeying the law. We have a bunch of scooters who are weaving in and out. We have scooters with two people on them. We have scooters up on the street. We have fog [00:03:00] and smog and other bad things. We have a bunch of bad scenarios. This is where we need to go. We need to solve for this in the real world. We are not starting there, just to kind of set the concepts.

So where do we start? Well, there are four processes, roughly, on how we do it and I think most of the industry does it in terms of building a self-driving car.

First, we need to collect the data. And so this is the current model we have. It's built on a Fort Fusion and we have retrofitted it with a couple sensors. So in front you see a stereoscopic camera. You see a couple of radars and we have a LIDAR on top. And LIDAR is this technology that's from the aerospace industry where it's basically a laser with a certain configuration and it spins very quickly. And what you get from that is you get two things. You get time of flight, which is basically how long it takes for this LIDAR point, this light, to hit an object and come back. So it can sort of get very, fast and very accurate estimates of where, how far it is. And you get intensity. So what kind of reflection it provides, what that tells you is something about like, is it a metal? Is it water? Is [00:04:00] it a leaf?

When we look at the car from the outside we don't see what the car sees, so the car seats, something like this, and this is like, the different sensor configurations overlaid on a simulation we did. So basically we need to kind of make sure that it has proper coverage. That it can look ahead. It can look around. We don't have blind spots. Just like a human being, right. So we need to understand what the lasers and what the cameras have seen.

The second step here is that once we've collected all this data, the computer doesn't know how to make sense of this. So our team also builds a bunch of tools around this. So this is one of the tools that's called the LIDAR annotation tool. What we need to do is we need to look at an image of what the car sees. We need to look at the LIDAR image. We need to collect those clusters that look like all those random dots. And we need to tell the computer, this is a car, this is a bike, this is a human and so on. And there's sort of a joke in the industry, which is ML doesn't stand for Machine Learning. It stands for Manual Labor. Because you need lots of manual labor for this to work. So you need to collect a lot of [00:05:00] data. You need to process that data. You need to annotate that data. And this is what this process looks like.

This is a GIF, just highlighting some of the more sophisticated use of LIDAR. Not a lot of companies do this, actually. We do. So when you look at this, this is a crop from one of the LIDAR scenes. All these points are LIDAR points. It's important, it's not pixels, it's LIDAR points. They're colored because we overlay them with the color from the RGB camera image. So I may say a bunch of gibberish to you. It's okay. Ask me questions afterwards. But basically what it does, it allows us to look through wind screens. So you can probably sort of see the outline of seats in there. It gives us a very high accuracy on what the object is. How far it is and how it's collected. But obviously still the car or, sorry, the system doesn't know what this is. We know, we can see that this is a car. We have to train the models to see what this is.

We also do a lot of mapping data, so deep mapping. And basically what that is is we, we look at the maps in the routes we're going, and then we go and we make sure that it's up to [00:06:00] snuff. And we wanna make sure that all the lanes are correct. All the rules of the road are correct. In case we have to stop at a certain stop point, we know what that is, even if we can't see it from like rain and other atmospherics. What's interesting here as well is as we sort of progress through this stack of technology here this is not something you can get. Google doesn't give you this, right. We have to go and secure it ourselves. And Lyft, I think is amongst one of the very few companies that actually can get access to this kind of data. So Lyft being a Rideshare company, we cover San Francisco about eight times an hour. All right. So that means we know a lot more about what goes on on the traffic level. Where paths go, when people stop, where people want to go, you know. So all this stuff that kind of feeds into this big model about where we need to service our cars, which routes to take, where to stop and so on and so forth.

All right. So now we've collected the data. We've annotated the data. Now we need to train the models to make sense of the data. [00:07:00] And I just put up like a random image of like, what's like a neural network and basically it's machine learning. Right? So machine learning, it's just math. And I think it's important to kind of reconcile machine learning what it does, and what it affords you to do, is to create a factory where the more data you give it, the better models you get, right? That's the ideal. Where classical programming you get fixed rules. So your products sort of is fixed in state. You can create a lot of like, if this happens, then do this. If this happens, then do this. The problem is that the, the world is complex. A lot of stuff's going on in the world. And if you don't have a good model, an interpreter, and you can't become better the more data you get, you can't really improve over time. And so machine learning is really the only way to do this. And this is important because the data that feeds the model feeds the program, right? So the better data we get, the better models you get.

All right. So now we are slowly starting to make sense of this. And most of this stuff is something my team created. Here we have a visualization and what actually tracks. Here's our car. It keeps track of the path in front of it. It keeps track of the different cars around it. It looks at the stop gate. It knows when to [00:08:00] accelerate. It knows how to steer. We do all kinds of behind the scenes optimizations to make it feel like a human's driving the car. Right. It's just visualization but basically that's what we need to.

The last step we take on this. So we've collected the data. We've edited the data. We've trained the models. Now we need to test the models. And this is super critical. So most things in Silicon Valley, most things in software; you ship it, you launch it, you get feedback. It fails. You send out a new Beta, you update it and so on. In the real world can't do that. You have to really, really, really have a strong safety stack. And you have to understand when things break and how they break and what you do when they break. So here's an example from inside of our cars. This is from last year. We have a safety driver up front. This is the person who's trained to surveil the car as it's driving to take over at a moments notice.

We have an engineer here on the left who sits behind him, who can then feed the model or he can feed different models. He can try different models on a closed circuit. On an open circus. [00:09:00] Open circuit is basically just the city streets. He can also inject different things so he can test if our safety driver is fast enough or reacting enough or doing the right things when he needs to react. Normally when we're driving, we don't have an engineer all the time. It's mostly in the early phases when you're introducing new models. And what we typically have, we have a safety driver and a copilot. And the copilot makes sure that the models are actually tracking on the objects it sees. But over time we want to reduce that, right? We want to take people out of the car. That's the whole idea, right? We wanna take people out of the car and over time there'll be no driver.

So here's an example of this GIF, this is a close circuit in the East Bay in Silicon Valley where we are injecting an error state into the car, right. So we're are injecting a state where the car thinks it needs to do hard left. And then we see how fast our safety driver can take over and get control of the car. So again, these are real atoms in the real world.

All right. I just wanna end this phase of my talk around robotics. I think self-driving cars are [00:10:00] going to be the first mass-produced, mass robots in our world. Right. It's not coming tomorrow, but it's coming soon. And I think it's important for two reasons. One, when you look at this picture, this is a, I think it's Tesla plants. We have these big KUKA 250s, these are blind, you know, senseless machines. They don't know you're there. So I don't know if you spotted the little guy in the red shirt there. Those gates that are up there those are to protect him not the robots. Because if he walks in there, he is gonna get swooshed away and maimed and badly. And this works because we have control of the environments, right? We have a closed world system basically. So when you have a closed world system, robotics is fairly trivial. You can even program this without any AI. You just make sure you control all the speed of the manufacturing line, control the temperature in there, you control the lighting, you control the paths they have to move through and so on, so forth. But when we move to the real world, right, and the wild you can't control it. I don't know when the little kid's going to run over the [00:11:00] street. You have to make sure that we'll see them. And so there's an interesting angle here which is for us to be successful and move into the real world, we need AI. We need a machine that can adapt to the real world and understand what's going on. And there's a bunch of more to the detail around here, which I kind of find fascinated from working in robotics. And I'm gonna skip over that for now, because I want to make sure that we have time for questions and all that stuff.

All right. So I've just told you that we're working on it, right. We're working on self driving cars they're coming, lots of stuff is happening. And I'm sure, you know, in the news like, "oh, you know, it's going so fast and like the future's here" and all that. But actually I'm kind of disappointed because I was promised this in my childhood. Where are all the flying cars? Where are they? And I watched science fiction. And when you watch Bladerunner in the opening sequence, they promised me in Los Angeles, November, 2019, I would see that. It's not here! Where is it? Come on. And I think's important in this constellation here, I think is that we are [00:12:00] actually not progressing as fast as we'd like to believe. The future is not here yet. Might be coming, but not here yet.

And when I reflect on this just for a second, I think it's important to recognize where we are now, where we've been. So today is December the 17th. It's the yearly date for when the Brothers Wright invented the first airplane in 1903. Two bike mechanics that got on a hill and finally figured it out. What made flight. And less than 70 years later, 70, we got a man on the moon. That's kind of insane if you think about it. But this year was also the 50th year of the moon landing. And we haven't been back since. So think about that. So in the last 50 years we've seemingly been stagnant when we look at these kind of big, bold bets about the future. Which is not fun. When I flew here with my family yesterday on our plane, it took us longer than would've in the 70s because we don't have the Concord anymore. And when you [00:13:00] look out in the city streets, you look at traffic today. It's actually moving slower. We have faster cars, but it's moving slower because we haven't innovated right? We haven't dug tunnels yet. We don't have flying cars. We haven't moved in the Z-space. What's going on? And I bet, and this is sort of my, my thesis here, that we have enormous innovation in our cell phones, in our virtual worlds, in our Google worlds, and our Wikipedia, and our gaming worlds. Incredible innovation, right. It's really, really profound change.

But when you look out in the physical world, not a lot has changed. Not a lot has actually moved in that pace. And that I think if you're moving away anything from this talk, I think it's go out and change that in the real world. That's important.

I'm gonna just end this with a couple of companies that I look to and inspire me. Because I do think that we can change the real world. I think we should change the real world. And so I'm gonna highlight a couple of companies here. I'm just gonna talk about them. There's no affiliation with Lyft or anything like that. Like these companies that inspire me.

So one company is this [00:14:00] company. It's a, it's a German company, actually. Again, just a signal that not all innovation happens in Silicon Valley. And it's called Lillium and it's based in Munich in Germany. And there are interesting things happening here. So first of all, this is probably the first flying car we're gonna see. So this is an electrical vertical takeoff and landing plane, whatever you wanna call it. It has 36 rotors that are inspired by drones. It's battery powered so it can charge. It has a bunch of redundancy systems. It's quieter, a lot quieter, than a helicopter. It's a lot cheaper to run. It's cleaner. It seats four. It can fly 300 kilometers. And it seats a pilot. And when you have electric control over your control stack, you can do a lot of other things. For instance, you can go autonomous. So you don't have to have a manual person sitting there. They wanna take the first commercial flight in 2025. I hope they get there.

Another company is one in my backyard. It's Saildrone. And maybe sort of interesting to you guys. Saildrone is autonomous boats. So these little kind of cutesy planes or cutesy boats. [00:15:00] They sail around, do missions so they can survey for pollution. They can survey for fishing populations. They can survey for weather. And what's interesting here is that once they're autonomous, you can start rethinking what that means, right? You can start thinking about what happens when we don't have to have freight with people on it? What happens when we don't have to have shipping with people on it? Right. We start thinking maybe we need to send fewer boats, more boats, different boats. I don't know. Lots of things start changing when you put autonomy into this.

Another company that's fascinating. Now a lot of companies are doing like drone and delivery right now, but this is especially interesting. So this is called Zipline and they work in rural Africa, where they send out medicine, life-saving medicine to rural villages. So normally when you send medicine out, one of the big challenges that you have a syringe and some medicine, and it's maybe 200-300 grams, right. It's nothing. But you have to send a three ton truck and a [00:16:00] driver and gas out through the jungle to get it there. Right. It's not, not a great optimization. And so these guys figured out why don't we pack it in an autonomous drone, send it out on a mission, drop it down. They open it up and they self administer. And so what's interesting here is also just like the irony of this which is drones started from a military endeavor to kill people, now they're sort of saving people. It's interesting.

RIPPA, this is a project from the University of Sydney. And so farming is also super fascinating. Lots of stuff is happening in farming. This is a farming robot. And what it does is that it patrols the field. It looks at the crop quality. It looks at whether it needs to get a little fertilizer or a little more water, or it is sick. It looks at if there are weeds growing and they can target the individual weeds. What's really fascinating about that is when you think about it, so one, we need to increase yield on our farming, right? We need to do it sustainably and healthily, right. We can't like, I think that the days of like just [00:17:00] mass dusting with pesticides are probably numbered, so we have to find other alternatives. And this is one of them. What's also interesting is that I think we're gonna go into a future in the next 50 years where human labor is gonna be more and more expensive. I think the days where we have cheap, cheap human labor that are like standing over the field and picking the individual crops are probably over. Maybe not tomorrow, but eventually. And so we need to find alternatives and this is an alternative. Where eventually we'll probably be able to pick the crops in the process.

And I can't help myself but land on this picture. So this is the final one here. Grand finale. This is SpaceX, if you haven't noticed it already. And they are also using AI in their controller basically. And so this would be impossible for humans to fly because there's so many micro orientations they have to do to actually land. There's an interesting article where they explain exactly how they did it and how they actually are doing crazy stuff on the torque, on how you actually land a rocket on its behind. And so of course, what's fascinating about this is that Elon Musk and SpaceX and the team at SpaceX [00:18:00] managed to do this while the United States government and China and other, you know, Russia and other space faring nations failed. Right. So everybody said it couldn't be done. Here we are. It's done. And I think it's just important to think. Be bold. Think how do we apply these processes in the real world? How we move atoms? It inspires me every day to kind of go off and think big. Thanks for listening. Let's open it up for Q&A.

Q&A

Host: One question, I realize you're in people mobility, but do you have any sort of idea on the use cases in transportation of cargo, ocean, land based, and what the obstacles are? For the reason why we're not seeing it today?

Johan: Yeah. Good question. First of all, Lyft isn't focused on cargo compared to you know a certain other competitor. And I think that's fine. I think there's plenty of growth in the personal transportation market. I think eventually the market will go there. [00:19:00] We will solve autonomous shipping. One challenge is effective cost per mile. Right? We talked a little bit about this earlier, but I think it's really important to think of first principles. And when you invest in these things there's always a huge upfront cost . But over time that'll go away and what you're left with is cost per mile. Okay. And cost-per-mile traveled with current systems is just too great I think. That's part of the reason why we haven't seen yet.

Second. You still can't take the driver out. So one of the big promises with self-driving, driverless, is that you take the driver out. You still need a "driver" when you are entering cities and need to dislodge your freights and get it signed and all that stuff. So I think the role of the driver for like a trucking situation is probably going to change, but they're probably going to be a person involved for the foreseeable future. That might change. I think highway autonomous is easier than inner-city driving. And then there's like other interesting challenges around regulatory, unions, you know, there's like a lot of jobs. I think it's [00:20:00] the largest or second largest job in the US is trucking. There are challenges like efficiency, do you use battery? Do you use diesel engines? There's a bunch of, sort of underlying challenges that need to be solved. And I think autonomy is one of those, but I think it's, it's gonna come in the tail end.

Guest: Next question. That kind in which self-driving cars is always the nice things around ethics. So how do you deal with let's say and we all always have a nice story that if your car has a choice of running down the old lady or the young kid which one will you take?

Johan: Yeah. So first of all when that is the problem we have to face, I am so glad. Because we have a long road ahead before we fix that. So there's like a lot of different things. The second is, I think it's important is the way I explained it with the machine learning, it's really a process you start. Right? So the machine, there's never a rule, there's never one engineer or a group of engineers that make a choice of saying, if this [00:21:00] happens, then do this, right. You have a rough, rough, rough, like here's a stop sign, you always want stop at a stop sign, stuff like that, but that's easy. Um, so in a choice like that you have a lot of complexity in the models that will try to balance different things. So when you look at a neural network, sort of visually displayed, you have a lot of different weightings. And depending on the scenario, right, that's the real crux of this, which is depending on the situation, the scenario. I mean, ideally you never want to get in that situation, right. You wanna stop beforehand. And I think that's actually what's showing up now in some of the early tests. Tesla had a couple of examples where they are going on a freeway and there's a guy in front of him and there's a bad thing happening in front of that guy. He hasn't seen it, but the Tesla has seen it. So ideally we wanna get to a situation where we never even encounter those things. I think it's important to state that safety is paramount, right? And again, I keep saying, you're moving atoms around in the real world, right? It's real people. You can't just ship a better product and hope for the best. You have to be really, you have to be very, very strict about that. There was an accident, a tragic [00:22:00] accident with one of the Uber cars, and that I think that's shook the entire industry into like, "okay, we gotta really take safety seriously." I think, A, we don't want to get in that situation first, once we get to that situation, if that's the situation we have, we're really good. I mean, again, like I said, just in US alone 40,000 people die a year. Again, it is dangerous to move people around in cars already. Then of course we have to take safety seriously, and we have to get to a good understanding of what constitutes our confidence in that we're ready. Right. That's a kind of a tricky catch-22 question. When do we know we're ready?

Guest: Okay, Thomas? Yep. All right. So do you foresee any required investments in infrastructure to really see a step change? Uh, so for instance the car have had this link keeping assistant. I can see the lines and it's all good until it starts rain, just as a suggestion. Yeah. So, I mean, for autonomous car, I would think that would be also a challenge unless you have everything really covered down into pieces.

Johan: So I think there are two [00:23:00] strategies deployed currently about that. One, I know there are different cities, there's some cities in China because they're building the cities, and they're super aggressive about this. They implement a bunch of sensors in the infrastructure itself. Philosophically, we don't do that. And I think there are a couple of reasons why you wanna be careful with that. One is that, infrastructure tends to last for a long time. Right? And so technology is a rapid clip. And so the sensor you used today may not be a great sense of use, like in five years or 10 years or something. It might give you a false sense of progress because you think now you're really safe, but you're really not safe. You're just relying on that one sensor and then that one sensor fails. So what do you do? Right. So I think philosophically speaking, we are working on the assumption that we need everything on the car. I'm not saying that you couldn't, implement it in the real world. I think it's challenging because now you have to deal with all kinds of different regulatory bodies like municipalities and cities and it becomes a whole different kind of ball game. I do think it's important to kind of make that distinction. Lastly, I think it's [00:24:00] important to recognize if a human can do it, our assumption is, the machine can do it. So even if you can't see the lanes, there are a bunch of ways we can do it. So we have different schemes for instance to detect path performance. So past paths taken, right? So when you look at a road, no one really drives in like a corner, right? Like no one drives up and then starts cornering. No one drives like that. Right. But humans sort of cut the corner and so will the AV. The AV will have to sort of do what humans do and that informs our stack and our models. And that helps drive even if you can't see the markings.

Guest: Yeah, I have a more Lyft related question. Mm-hmm. I mean, there are a lot of companies working on these developments around autonomous driving as Uber and Lyft, Google just to name a few. Yep. But, um, what makes kind of Lyft stand out in this kind of development? Why is this actually doing this?

Johan: Yeah. Great question. So I think that Lyft, and Uber for that matter, are one of the very few [00:25:00] companies that's gonna survive this race right now. We have a race. We have about 150 companies in the valley doing this. We have a bunch of companies around the world. I would start by saying, I think most people, when I say people, I mean, most entrepreneurs for a little bit of money can actually make a car kind of go, right. To prove a concept out. I don't think that's that hard. But getting it to scale. Getting it reliably, cost effective to scale is a whole different game. And like I mentioned earlier what sets Lyft apart is that we actually have a huge Rideshare network already. So we cover the most densely populated metro areas in the US and Canada already. That data is worth a lot. And so not just where to go, to send your AV to be the most lucrative and low risk pickup. But you can also learn something about the driving behaviors, the risk profiles of drivers and so on. We have different programs. We also, I think we just announced, a rental fleet program. We go do a lot systematics, data gathering on when the car brakes, where people drive. All that [00:26:00] fun stuff.

Guest: Because the cars usually don't have specific examples.

Johan: Yeah, exactly. Exactly. Exactly. So it's basically, when you look at that picture of that sensor up front, that's like the most extreme, right. It's not gonna be the car we're gonna go, I think, well, I'm sure, to scale with, right. Because it's too expensive, it's more of a science project. And so you can argue, how many cars do you need collecting data for you to be successful? You probably need more than 300, right? Do you need half a million? Maybe? Do you need 5 million? Probably not. So you need somewhere in between and whatever companies that have that. And then you can go down the Lyft, list, sorry. You can go down and you can say, well, what companies are out there collecting data at this volume at this cost. All right. So that becomes a really interesting question and I don't think you have very many actually.

Guest: Yeah. Building on that, uh, you mentioned that you sort of have this dream that we sort of move faster towards actually being autonomous or sort of slightly disappointed in technological advancement. Now you work for one specific [00:27:00] company that obviously has the aim to maximize the value capture of the technology is develops, but on the other hand technology as a whole develops faster, if different stakeholders collaborate. How do you balance that?

Johan: Yeah, good question. I think it's hard to agree on a standard when everybody wants to invent their own standard. And I think that the benefit you get from having a lot of different competitors enter the market with different solutions is that you get a lot of rapid iteration burning a lot of money to kind of accelerate the right model. Right. As soon as there's a right model with the right kind of tech, the right, you know, fit, people will align to that. I don't think we're quite there yet. Like I don't think anybody, any company has it. That said, I think it's also worth calling out that, and this is a pure postulation for my part, right. I don't have any data to back this up, but I, I speculate that there will be more billionaires created in adjacent industries than in transportation. Meaning, when you have this you will unlock other areas such as retail, such as [00:28:00] real estate. You know, you have areas that are beautiful outside of Copenhagen but it's like 45 minutes away. There's no train line. You know, you can buy a house for like 700,000 Kroners. But you can't get to the city and less than an hour, right? So no one wants to live there. But now you have an AV. You can get in an AV and now you can get there maybe in like 30 minutes, because you can drive faster. It's safer. You can sleep in it. You can take a meeting in it. Boom. You unlocked value. Right. So I think you have to think deeply about how the thing you're building into the world, how that actually cascades out into rings. And I think that's an example of that. So we create value while we are capturing value. Right. And that's ideally what you want. I mean, pick your company. But Maersk, I mean, you capture a lot of value, but you also create a lot of value. Right.

Guest: A few of the cases you mentioned about hardware and obviously sort of rocket landing, so forth, driven largely by political strategic defense initiatives, big sort of public sport, governmental, that sort of thing. And in quite a narrow practical way, you've got the regulation environment for autonomous, which, you know, perhaps it's a deceptive piece, but also very much around public. [00:29:00] Advertise attitude towards it sort of tech drive. Could you sort of talk a little bit around the point on how you sense the mergers around that the speed of regulatory reform getting there? Will it get there? The mechanisms to help?

Johan: Sure. So regulatory issues in the Valley is sort of a hot topic for sure in the US right now. I think it would benefit the entire industry. Think about it as like you have a field and you want to kind of make sure that you plow it once a while to kind of open up the, you know, all the oxygen and the good things. Right. I think it opens up opportunity. I think Lyft is uniquely positioned because we actually do a lot of work with the governments and communities. Right. We think a lot about how we responsibly integrate into cities. Everybody in the different orgs inside the company are very aware we don't wanna take away from public transit. We wanna help promote public transit. We are sort of, I hate to use the term, but I'm gonna use anyway, you're sort of at war with private car ownership. Right. And I love cars. [00:30:00] I have cars. But I think the idea that you wanna have an asset that sits still 92% of the time, and you spend, in the US the average American spends like $9,000 a year just maintaining and like value loss on their car seems preposterous. Right. When you think about it from a logical standpoint but here we are. And so I think that's where Lyft is trying to get in. And so we work with cities, we've worked with local municipalities around how we can bring bikes, bring scooters, bring transit agencies, how we can help reduce cars. And I think there's gonna be different hurdles, right? There are gonna be different kind of challenges along the way. There has been since day one. And I think that's, that's fine. It's a different kind of company you need for that really. Right. Which is like the current tech giants thrived in a different environment. And I think different companies will thrive in the new environment.

Guest: What can other companies that are focusing on AI learn from Lyft for different industries. So now we're very much talking about planes, rockets, cars. But if you look more at the [00:31:00] AI on its own. As, you know, a science or whatever you wanna call it, what industries, what is the main takeaway?

Johan: Mm-hmm. Let me frame that question a little bit. So one thing is that I think AI becomes sort of mystified and easily so. And it's not magic, right? It's math. And the algorithms that we use that are effective today are basically 20 years old. They're not that much that has changed, that are better frameworks that are a little easier to use, but what has changed is on the growth spectrum. So on the small ends, our cell phones collect a lot more data. So now we have a way you can click data, image data, you know, geo data, whatever. That's sort of interesting. So now we have this Cambrian explosion of data sets that are interesting and we can learn from. That's one thing. On the other end, we have data centers that can train on this data so we can extrapolate. We can sort of be, uh, inefficient. We don't have to build a data center of our own to train on these things. We can use Google's or use Amazon's or whatever. [00:32:00] So that's sort of what has started this. And I think that's important to understand that the algorithms and the frameworks that we use, TensorFlow et cetera are all open. And it's the data, that's the difficult part, right? It's the data. You have to understand. How you get the data? Which kind of data do you need? Do you need all of it? Do you need specific kinds of it? So be very clear in your mind, depending on what you're doing, how do you get that data? And I would even go so far saying not all problems require AI, right? Like sometimes she just needs kind of smart engineering or good product thinking or good design to solve some of these problems. Now let's assume that you wanna build something around AI and AI's a great use case for that. I think it's important to reconcile, are you sprinkling a little bit of AI on top of it, right, on top of your existing product. Is it sort of a, "Ooh, there's a nice little buy this other people bought this". And I'm not against that. I think that's fine. It's just like, fundamentally it's not like an AI product, right. This is like your sixth bullet point on your pitch slide. You added it last year because like now AI is the thing. Or is it fundamentally like the thing you are [00:33:00] rebuilding. You're fundamentally rebuilding your product based on what the machine learning framework platform, call it what you will, affords you to do, right. Allows you to do. And I think that's really interesting. So AI, ideally. Machine learning, ideally. Helps you predict certain pattern. When it works. It affords your product to become better over time. So there's an interesting point here is that it starts this flywheel effect.

So normally classical programing, you have a product, you release a new version, you add features. And you do that manually. You keep shipping. But with AI, you could sort of like have a thousand different products you're gonna have, I don't think there's like one homepage for Amazon. And I mean, that's the canonical example. Right. You can improve your product a little bit. So let's say you're building a, you know, you're building some kind of analysis tool for freight. Well, your product has AI built-in. And so every single time you add a new container ship or new sensor on that network, it gets a little better. It gets a little more data. In the beginning it's nascent. It's like [00:34:00] compounding interest. It's really slow in the beginning. In the really slow. And you don't really see the difference, but over time it's like compounding, right? So that difference starts making a difference. Versus your competitor who doesn't do that. He has the 500 ships. And when he adds like a thousand more ships or a thousand more containers, doesn't do it. The product does not become better. It doesn't become smarter. It doesn't change. So now, remember when you make changes, I mean, you might be negative change, right? So you have to clearly think about what you want your product to do. But I think generally speaking, I think that is really sort of interesting. Like you can, the product becomes better, the better data it gets. It can be highly customized. You can sort of do that. That's like the what I recommend you thinking about when you're going down that path.

Guest: Thank you. Yes. Yeah. So, I have one question. What validates the need of autonomous vehicles in the future? What kind of future proof that you have to test up right now? I know that there are a lot of companies that developing the program, but I, I haven't seen that myself have needed in the next 50 years.

Johan: [00:35:00] Right. There's like a famous kind of computer science person who said something like, anything the machine knows the answer to the machine should provide, right? So the machine should sort of solve it for you if it already knows the answer. And I think you can extrapolate that out to your car. If the car can drive itself, it should drive itself. It's gonna be safer. It's gonna be more convenient. You never have to look at parking spots. It's probably gonna be cheaper. Insurance goes down. All this stuff. And so I think it's important to realize that the context of which this will exist is gonna be very different from today. Today, it doesn't work and everybody drives everywhere. Right. And you're gonna be a time where half of the population drives and half is like autonomous. And then you're gonna have full, I think. Again, this is me speculating. Ideally we wanna get some future where we have to need a lot less cars. Ultimately. Because we have a much higher utilization rate on the existing fleet. That's sort of an interesting point which is right now, I don't know how many cars are being produced, but everybody needs a car. I'm not sure if everybody needs a car. I think we need to share them a little more, like think deeply about what that means. I think it goes [00:36:00] at the heart of whether or not a company needs to convince you that you need autonomy. I think it's a lot more around, well, what's the use case? Where do you need to go? You need to shop? You need to go for a joy ride, right? If you need to go for joy ride, it's not much joy, if it's driving you. Maybe, I don't know. It's like a rollercoaster ride at that point. But I think that's sort of the crux of it. And when you look at where we are right now, it looks feeble. It looks really silly and it's not very good. But again, so did the cell phone, right? So did everything else. It looked silly, looked like a toy. And eventually we'll get there. So I don't know if that specifically answers your question, but I think it's important saying that you might not need a car in the future. Like you might not need your own car. You might need a car, click a button and it'll be there and you'll get in. And it's like an elevator, right? No one needed elevators before we had elevator.

Guest: I have a quick question. I think you touched on that really briefly. It's about like how cars can communicate with each other in the traffic and how they exchange data, actually, especially how they do it across platforms, [00:37:00] across brands, across different kind of systems. I mean, that only makes sense when everyone can participate in the whole traffic. Right. What are your thoughts on that?

Johan: Yeah, that's great. Great. So, not that I know of, is there sort of an effort to make a university standard? I think it's the sane thing to do. But I think it's premature. You know, you don't want to do premature optimization, right? You want to make sure that you run it, you let these different projects run its course, learn, try to optimize. And then later, and this might come from like a regulatory body, right. It might be sort of, we need to share this data. Or at least we need to kind of intercommunicate. If it's beneficial to the companies they'll probably do it. Just like we have USB or whatever other standard. So I think it'll come. I think it's just premature. We're just not there yet. Like we have not that many cars. But yes, I mean, the fleet learning from the fleet is the ultimate goal. Right. But there's a lot of interesting kind of competitive advantages, just like that, in not sharing your data and your models. And it's not even like competitive advantage, it's just like, it's very, very complex. The tech stack is very, very complex. Right? You have sensors [00:38:00] and you have the vehicle and you have the models, you have the data. I mean, just like LIDAR, just agreeing on what kind of LIDAR system you're using and which hertz it's running on. You have to engineer the entire stack out of that. And then if you share that data, you might share it. But it's useless for the other vehicle because it runs on different hertz and it can't implement it. Can't react on it. Anyway, I don't have a good answer for you other than I think it's a great idea, but I think it'll take time for before we get it done.

Guest: What do you see as the biggest design / tech challenge to actually get to autonomous mobility from your perspective? And also what's the time horizon? I don't have a drivers license. Should put my faith in you or should I get one?

Johan: I just read an interesting stat on the driver's license thing, which is in the last 20 years, I think, in the US, it's been halved. 16 year olds, the 16 year old in the US, right? 16 year olds will get driver's license that fall from 46% to 24%. So you're not alone. A lot of people, young people, don't get driver's licenses. Um, [00:39:00] I think it's soon. I think it's not a question of when it's a question of where. We're gonna see it in certain cities, certain markets. It's gonna be constrained. We operate from what's called an ODD, Operational Design Domain. We down scope until we can control the, you know, enough of the stuff. And I think it'll come. I think we'll have something in the next five years. For sure. Take 10 years. For sure. Will it be everywhere? No. It takes a long time. Cars last a long time. I don't know how it'll play out. What was the other question?

Guest: What is the biggest design challenge?

Johan: Mm. Yeah, so, I mean, we work on a lot of different things. I think there are different challenges and different types of challenges. I think my personal challenge I have with the team and how we think about it is, to think big enough and to think boldly enough and be thinking outside of the box enough. It's very easy to lean into an existing pattern, an existing solution. When you're working and the same is true when I worked at [00:40:00] X, when you're working in an environment where there is no prior art in what you're doing, you just can't, you have to blaze that path. You have to set up a process where you are very conscientious about that. You don't have good models to look to. Uh, and then be inspired by other fields, right? Like maybe this is consulted in another field. But maybe this is a... maybe you reframe the problem. And so this is more like how we organize the design team, basically. Specific design challenges, I think it's about using modern tools. We use a lot more 3D than most. We use a tool called Houdini that the FX industry uses. It's very sophisticated. It affords us to do all kinds of node based, kind of, things. So it's reactive to the data we put in. So it's just, it's just re-jiggering what design means, right? Design, just as any other discipline, gets sort of boxed into sort of like, "oh, we've done websites this way and we've done apps this way for 10 years. Now, this is how we're doing an app thing." And I'm trying to like push the team to really think outside of [00:41:00] that, like, "okay, what can we do if we don't, if we scrap everything and start over?" So that's like a general challenge.

For AI or for autonomous vehicles. If we kind of look out the future, I think the main challenge will be, how do we make it safe? Um, people exit and entry. That's another thing that car manufacturers have no idea how to do. Like they haven't run a service like Lyft, right? Meaning, doing, you know, thousands upon thousands of drop offs. Making sure that you know exactly how to do that safely and reliably. What to do when people lose their stuff in the cars? This is interesting stuff. So I think those are kind of the challenges that you have to solve for in this future where it's more of a service and less of a product. And where the product is the experience you get out of it. So you get in a car and it's not your car, but you have a great experience. You know, it feels nice. It smells nice. It's clean. It's fast. It's reliable. When you buy a car today, those are things that are put on you as the buyer that you take care of. Right. And then the card needs to wow you with like its cool [00:42:00] rims or its cool sound system or whatever. Uh, which I think will be different in the future.

Guest: Maybe there's an adjunct to that question, do you have a sense of progression, so you tackle problems and progress or it's sort of iterating on the same issue again and again, in your development?

Johan: Mm. Um, so it depends. It depends on the project. So if you have a project where the outcome is known, right. So, you know what success looks like. Then you just need to kind of speed that iteration up. Right. So if you know what success looks like, you just need to iterate fast enough to get to that success point. And, um, and that's the easier of the two. It can still be painful, right? Because you can still have a lot of failure and you still like have a lot of zig-zagging around on your roadmap or code base or whatever. But, but I think it's somewhat more clear in your mind, if you have like this, this is what a good product feels like. We all know what a good cell phone experience looks like, right? Like there's no lag. It's good connection. It's like beautiful graphic design and so on. I think the harder part is when you don't know what the future will look like and you have to invent that future. Because then, [00:43:00] um, you know, you make so many choices. And mistakes. And you don't get clear signal on which are good choices and which are poor choices until much later. So I have to sort of be, you know, set up processes. I don't have a good answer other than like, be honest with yourself and your team, like, is this good? Is this something I wanna use? Is this the best in the world? I think it's also important when you talk about startups, right? You're growth and startups. It has to be world class. It has to be world class, right? It has to be. It's not enough that you compete, like you look around and you have, you sit in like an office and you have like 10 other companies, and you're like, "oh, my shit is better than theirs". Yeah. But it has to be better than like the 10 million people out there, right. It has to be really good. And that puts pressure on your own belief systems. On what's important. And it doesn't have to be better in everything mind you. Right. It has to be better in the key things that you're better than, than most people. So if it's engineering, be it engineering. If it's design, be design. Be it marketing. Be it AI. Whatever. Be it shipping, be it operations. Right. I think [00:44:00] Lyft for instance, is less of a tech company than Google was, for sure, but much more of an operations centric company than Google ever was. Right. So we are really good at getting like a thousand people on the streets and just making sure we sign up drivers, get people out, make sure that service works, all the stuff and that's important for our business. And so I think when you go through your businesses or your use cases, think about what's important? What's the real bottleneck?

And for instance with robotics. Um, what time do we we have? We have a little time? So just, I mean, raise your hand if you have more questions, but I'll tell a short story about robotics. When I joined, I joined a project and it sadly got shelved. I had, eh, many heated debates, if we should pursue this. And we worked on a project on manufacturing robotics. So the picture I showed you basically hasn't changed for like 40 years, right? These are the same robots, basically, that rolled out in the 70s. These are the same production lines, the same morphologies and all that. What's important to understand about that business is [00:45:00] that KUKA and a lot of these other big companies they make most of their money on the services when they integrate. Okay. So they actually don't have a high margin on the actual robot. So a robot is, maybe, a big one, that's like $250,000 per arm. But they make maybe the same or a little less on services. Meaning it takes about a year to have some robotic PhDs from that company to sit down and actually build the test and build a robust pipeline for them. Right. So it doesn't fail and so on. That means that their incentive structure as a company is misaligned from their customers'. Right. So it's not flexible. It's actually very inflexible. And they are making it harder to code because they don't want people to have it easy to code. They need their services. Right. And I think that creates a massive blind spot because now, and that was the project that I worked on, how do you make it easier to program a robot? Right. We actually took the code away entirely. So think about that for a second. What's the real problem you're solving, right? Is it really code? You need to do it with code? The real bottleneck [00:46:00] for robotics like this is time to value. So time to value means how long does it take you to set up, code, "code", and get value out of this plant. And today it takes an order of like weeks, if not months, if not years to build these. This is very, very complex. And if you can do that in a matter of hours, you have something. And so there's a Danish company. We talked about this many times before. We looked at it. Universal Robots got sold to an American tool manufacturer and they did something similar. They made it easier to train a small 6 DoF arm in a safe environment. So it's safer, it's easier. And that's just for robotics. So what's important here is to identify, can you identify, what is the real problem your customers have, right. And one of the reasons why Apple deploy or, or employ so many people in their manufacturing alliance is that they change their products every, I don't know, 5, 6, [00:47:00] 7 months. And it's not feasible to reprogram a robot. Right. It's just too expensive. So they have to retrain. They train the people and they can do that in a matter of weeks. To retrain on the newer line, new chip set, new thing, all right. People are, and people are smart. People and humans are super smart. Right. And so that's part of the reason why what's the real, real bottleneck. What's the real value you're unlocking? And sometimes we get sort of blinded by AI. Sometimes we get blinded by the technology because like, "oh my God, I can do all this stuff." But then like, imagine tomorrow, your robotics engineering, "oh my God. I can compile the code at three times the rate!" Yeah, that's great. But maybe that's not the problem. Maybe it doesn't matter at all. Maybe no one cares because the business is not set up to support that. So anyway, that's sort of, I think it's an important thing to think about.

Guest: Two questions. You mentioned that you guys are partnering with Ford right now. So is that something that we will see, uh, you would be owning a big Lyft fleet of Ford cars or how, where would the cars come from in the [00:48:00] future? And then is there any lobbyism going on against industry of autonomous driving vehicles in general in the US that you're aware of?

Johan: Okay, so let's start with the first one. So we use Ford Fusions on this platform. We actually bought them. And so we didn't partner with Ford. Oh, okay. No, that's okay. Uh, and so what's, and the reason why I mentioned that, like, it's kind of an interesting thing. It's a bit of a grey market because when you look at the BUS. I think it's fascinating, right. But I'll explain it quickly. When you look at the BUS in the vehicle that controls the braking and the steering and all that stuff, that's all locked down. You don't actually have access to that as normal person. But there are some company folks from Ford that went out of Ford. Bought Ford vehicles. Opened it up. And now resell it for like four times the cost of a normal Ford Fusion. But that allows a company like Lyft to buy these vehicles second hand, right. Opened up. And so now we can control these. It's kind of crazy. That's how Silicon valley works. So you kind of have to break some rules and you have to fit it. But basically we are working with some partners it's not open yet, but it's gonna be so it'll be exciting in the future. [00:49:00]

The second one, honestly, I don't know. I think Lyft is also well positioned because the future is unknown. It's very difficult to predict the future. And so I think autonomous vehicles will be some part of the solution, but just as much as micro mobility solutions will be right. Just as much as public transit agency things will be part of it. I think it's not "a winner take all" thing here. I think it's like, many different bets, with different markets and unique things. I bet that there are companies. I mean, I don't know, I don't have any data. But I... Why not right? I mean, I'm sure there are laggards in this industry, um, in auto manufacturing or elsewhere who don't wanna be disrupted. Right. They're just fine with you driving a whatever car. Right. And not being able to do it autonomously. And I think it's important to recognize that these innovation shifts typically happen when one industry, what you were good at before stops mattering. Right. So it's, it's important to notice that it's not because Ford stops building great cars or Volkswagen stops building or Mercedes. They probably built the best cars [00:50:00] still. Right. They probably build like the best engineering, the most tight seams and the most beautiful leather, whatever. But it just stops mattering. Consumers want something different or the market want something different. Either it's autonomy or it's electricity or it's something else. And I think that's important. So just, you know, there's this kind of cliche thing, the stone age didn't end because we ran out of stone.. And so I think it's like, it's the same as businesses, right? Like the auto industry is not going to fail because they stopped being good at building cars. I think it's going to. If, if it's gonna fail, if I don't know. That will happen because things that are more important starts smattering more, and they're not good at those things. Might be like deployed computes. It might be AV, stuff like that. And so in that light, why wouldn't they be lobbying to kind of put the brakes on this?

Host: I think we're on time. Thanks a lot, Johan.