I am really excited to come to this talk. I was really excited. How many of you in the room want a Tesla? Me too. People from the company is going to be there ask if they will give us one? Yes? No? Rats. Let's get started. It's my pleasure to introduce Marc and Kevin to talk about hacking a Tesla model S. Let's give them a big hand 
[ applause ]
>> Thank you all for coming. So I'm Kevin Mahaffey here with Marc Rogers.  We are here to talk about in hacking a model S.  What you are hearing now is the door chime sound. If you think it's annoying for coming in, it's even more when you are hearing this 14 hours and 3 days straight. This talk is about how to secure cars. Some cars need to be recall and that's not going to work if cars need to be patched weekly. Tesla pushed a patch yesterday. You click one button and it's applied. I'm super excited about that.  
[ applause ]
>> Try this one.
>> I'm also known as CJ you probably have seen me shouting outside. The reason we started hacking the model S because it's the most connected car in the world. If we should look at car security, we should look at arguably design every other car end up to be using. And we want to do that bearing in mind in the future we are going to see an explosion of connected things. And we already have a problem with embedded devices being vulnerable and not being updated imagine how beside that problem is going to be from few million devices to 25 billion devices.
>> And you might replace your computer or your phone every couple years. If you are not fixing vulnerability you might have a car for a couple of years and not fixed that's a giant deal. When that happens you have all these industries building new things and don't have software experience. And you might ask those companies go to the straight line of the internet and you are not aware of it bad things happen. What are the best practices in mind?  Do people know how to build secure things?  
>> Not that these companies set out to make vulnerable things. Bio companies are still following best bio med. For cars, they are not just internet security companies so they don't fully understand.
>> So if there are no best act practices you have something that proximate this. Nobody is doing anybody bad competing with competitors and they match quickly to their doom. I was happy to see that Lami's 2.
[ applause ]
Thanks. If there are best practices even companies that are not cyber expert they can still have things that go well. Right now if you are level zero they are easy way to level up 3, 4, or 5 and raise the bar.
>> Anything that man breaks, man can breaks. It's how companies deals with vulnerability.
>> Best practices they are important. Good when building a system for the first time. Trying to changing the system is hard. So why the model S? Well, the model S, we thought it would be a car that's most well designed, a lot of engineers behind it. We found actually this it's actually designed very, very well. So we hope that if anyone involved today, tomorrow in the auto industry hopefully take the learnings we have what they did right, wrong and apply to the future cars and hope the cars will get more secure.
>> Now what this talk isn't. This talk is not how everyone with a connected car is going to die.
>> Nor pull something out get hub, hack your car. Not know your next...
[ applause ]
I was wondering how that was going to go over. Good. Public service announcements.
>> You have to be really careful hacking cars. A car isn't just a computer system. A car is a computer system that's traveling 100 miles an hour, 60 miles an hour.
>> Yes, officer, I swear. And the worst thing that happens isn't the blue screen. That should be obvious. But Def Con we have a sense of mischief which I love but when you are dealing with things that fast and critical, you have to be really, really careful. So what we did we collaborated with a great group of people and set strong rules of engagement.
>> It's really, really hard to know whether you are hacking the embedded system or transversed to the company is connected to. So we made sure that we didn't accidentally hacked Tesla. The next thing we set ourselves some boundaries. We didn't do anything that would make permanent changes. We knew we were exploring black box system we couldn't foresee any changes would have to the ongoing safety of that vehicle. What happens if you overwrite a firmware that happens to the ecu that could do bad things?
>> Not good.
>> So we agree that we would do everything in memory and only probe and prove concept of things.
>> Tesla established a bug bounty program. Rule of engagement from don't touch to touch the servers they allowed us to touch.
>> And they started the bug bounty program before we started.
>> So I'm sure a lot of you are interested what is the model S what's inside, remember these words very carefully. So there are three systems. One is the IC, the instrument cluster. The 8-inch screen above your steering wheel. The number 2 is the CID, the center information display.  The 17" touchscreen.  The thing in the middle.  The really cool laptop in your car. And there's the gateway which we illustrated on the dash but attached to the PCB to CID. 
IC, CID, and gateway. Got it?  Cool. Two of which are Linux system. And there's an rtos is the gateway.
>> And both Linux systems are running on very old abonto.
>> So we will get into how we figure this all out. You have the infotainment network and the vehicle network.  The land the ethernet land on the infotainment side and you have the can which is the vehicle side.  And the only thing we can tell that bridges the two is this gateway.  That's going to be very, very important. From the system architecture standpoint, the CID and IC, two are connected from the usb ethernet adaptor to the ethernet switch. 
The CID has wifi, bluetooth and self-connectivity and exposes that to internal systems in various ways. The gateway is the only thing that bridges the ethernet and can bust, and all the of the vehicle controller are connected to that.
>> It's a phenomenon design. If you think about this, this is like the way the airplanes are designed and cars are designed.
>> This project is the hackers olympics. From physical security to browser hacking to network, binary reverse engineering. It was kind of fun because we got to exercise different parts of our brain.
>> We submitted six vulnerabilities to Tesla.  It took three of them for us to gain access to leverage one to get access, and get enough information and gain privilege access. We will go into in more detail. It took of thinking outside of the box.
>> What we were able to find. So just want a preview, what we are talking about is physical access and we implanted a remote ssh in the car and we had some fun with it.
>> So the video you will see now is model S backdoor and what we can do with that ssh backdoor. 
[ video ]
[ music playing ].
>> So what you were able to see there is we have route on to the system of the car. We were able to control the ui on the instrument cluster 17 touchscreen and also saw we were able to control some of the vehicle aspect itself.
>> What that means is we can do anything the touchscreen can do. And in the case of this car, the 17-inch touchscreen is where a lot of stuff done by the driver. We can start the car, power roof, control the head lights, horns, et cetera. 
>> So how do we actually go do that?  Let's first inventory. What's in this car?  Number one security bits. Get them from amazon.  Number two, these are really important. If you are pulling off door panels on a car. You would likely break things.
>> And when you are dismantling Tesla it makes you nervous because it's expensive, you will hear sounds like plastic breaking. It is normal.
>> These are terrible. I'm not joking. These are like my nemesis when I'm disassembling these.
>> When it sounds like plastic shattering, it makes you pause.
>> Here's the backside. This is a Tesla design board with Invidia processing module. What we are looking for is anything we can get access to. Unfortunately, not so much. Then we pulled out the CID, center information display.  Lots of cable. Opened it up and then tried to find what we can find. 
This is the backside of CID.
>> SD cards.
>> Memory cards and usb header. 
[ laughs ].
And then we have a very mysterious port. We found a number of things. That's a little scary to touch in a system. Memory card, nice. Now that we know physically on the device, we want to check out what we can do.
>> To the browser remember me saying it's an old bunch system.  The browser is very old. There are several well known vulnerabilities for it. Probably the most well known, 2014 and also used for ps peter exploitation.
>> When you are designing any of the system, qt fame work.  And if you sit downstream, Tesla not only from downstream from that webkit but also from qt webkit.  I don't know where they are pulling directly from the web kit source.  If you have multiple upstream that are required to be passed for you as end user or manufacturer to patch, it creates a logistical challenge thing for upstream.  This is the vulnerability that Marc mentioned that was used to blind the exploit x invdia.  It was through java script sort with a custom compared function and modified the array underneath that sort and write stuff to fun places.  We were not able to do that arbitrary memory rewrite.  We were doing it blind.  And got it to crash.
>> To be fair we spent 12-16 hours on it.
>> Bluetooth not special thing there.  Usb.  So this is actually a really complicated thing to do from cable hacking side of things.  You see that little header right there.  We actually have to fit this back in the case and back in the system.  We want to use the vehicle's power supply and get it to actually fit.
>> So it gave us access and we could talk to the processor. Unfortunately, the firmware locked.
>> Yep.  The tegra bootloader, we didn't have the password.  All the ways we didn't get in that's important.  All the hack you guess the password and automatically you get in.  It's like failure, failure, failure. Success.  I think it's important to remember that in a test like this.
>> Memory card was much easier.  Readable file systems.  One of them contains mapping data which is not so interesting.  The other one, much more interesting.  A file, car keys which contains keys.
>> We were very happy when we saw that.
>> An appreciative name.
>> Car keys are cryptographic nowadays.  Wifi there were no open ports when we connected cars to wifi system.  As soon as it connects to wifi, reach ability.  And then it will connect to vpn endpoint.  Of course, the question is [ inaudible ] vpn connection.  Open vpn connection that we were able to determine.  Udp they were using a tls off key.  That's not authentication.  That's we are going to h mac the packet so that you can defend the open vpn server from a ddos and no static key.  So simply presence of that doesn't give you authentication, just able to talk to the server and we looked at their pki.  So Tesla route ca is self signed, change down to a poly cca to issue nca.  So interestingly vpn server expose to nca and the car key that car key, quite literally, also change that to that same ca.  If you are familiar with x 5 extended key usage you can actually say this key can be used for x.  In this case, web server off and client was client off.  There's a setting in that whether you want to pay attention to x 593 eku.  So we ask the question which is okay.  When the car talks to our open vpn, and it sends the ssl client hello, the server sends server, what if it sends the certificate of the car?   They are chained up in the same root, not eku and treating the certificate authority, it would work.  Tesla does validate that eku.  However, if you find any certificate that chains to the issue nca and has the server off eku, you could impersonate that vpn most likely.  Unless there's crazy thing going on.  We haven't find any yet.  Word of caution open vpn make sure you are testing through that eku so you don't allow impersonation.
>> The mystery board.  It took us a while for what it was.  I would like to take credit but people on the internet found it. It's ethernet.
>> All right.  Now, we have digital connectivity inside the car.  It's going to go faster from here. As soon as we are connect, udp.  That's a lot.  500,000 per second udp broadcast.  Yikes.
>> So to answer the question of what kind of [ inaudible ] does the car collect internally, the answer is everything. Literally everything. I don't know what it does with that data but you have to pick through it to find interesting stuff.
>> First the CID we found a bunch of stuff on this thing. Lots and lots of open services. The IC fewer but yes.  Gateway, fewer.  Open stuff. We have a lot of attack surface.
>> Open x 11, really? 
[ laughs ].
>> Ssh we tested. Tried the man in the middle. They validate host keys.Typically what you would do you find two systems automatically communicate to ssh, you build honey pot hopefully they check password authentication and send that to you.  It didn't work. We did find two services.  The vulnerability in it we didn't really explore true extent of what could be done, but these are old versions with known vulnerabilities to them. So open x 11.
>> Open x 11. What's not to like?  Essentially, the most immediate thing you could do with that is little display console that sits in front of the driver implementation cluster, you can put it whenever you want up there.  My car goes a million miles an hour.  
>> Screenshot to prove it.
>> You can also run command off different services.  Having that open x 11 server is great.
>> I think that's about 3:00 a.m. at this point.  So we had a little fun with open x 11 server.  So what else?  Do the other open services do?  We found what is called the ID and CID updater, this gives a lot of juicy information.
>> These appear to be binary firm getting diagnostic information managing firmware, a lot of commands are secured.  Quite a lot of them considered to be safe including the status command which gives you tons of interesting information like the state of the system, the download for the firmware.
>> And you were very happy to see this.  This is a url that you can reach and give you the firmware.
>> If only we had the car keys.
>> Note this is not reachable from the internet only from inside the vpn.  But just that time we had a set back. Marc mentioned the internet had also found open ethernet port and Tesla issued a patch to lock it down.
>> The fix was pretty clever from what I can tell they start with you on the mtv launch and you have to go through an authentication process.  However why go through all of that reverse engineering thing when you can man in the middle from one of the legitimate systems.  
>> So that's actually, we took the instrument cluster, it had the mail connecters and we didn't have one.
>> So we made one.
>> It is scotch tape.  This is hard to find insulating tape that's that thin.  We plugged something in made an ethernet cable and another in the vehicle and hooked it up to a switch.  So now we have access restored to the system.
>> And now it's constantly authenticating that and we can plug that switch and do whatever we want.
>> The breakthrough, Marc mentioned that we have car keys.  So one day we were looking at CID updaters and it mentioned a nice url.  We have the vpn keys.  We did our man-in-the-middle so we know the configuration is.  So we can move forward right?  Actually there's a lot of vpn you have to set.  A lot of it is exposed hosts.  Ssl encrypted channel so we can't look at the cyber traffic to figure out.
>> This is when hackers really like developers put in the reverse switches.
>> If you use open vpn client and turn on verbosity up to 11, it will tell you the server side desired all the things you need to establish valid vpn connection.  Typically you don't need to blindly figure out vpn setting, but that's the case with open vpn.  So 600 megabytes later, we have the firmware.
>> Firmware is not a completely file system.  They update the firmware and they look at what bits are missing from the car from the last firmware update. 
So we end up with partial file system, which contains a lot of interesting bit.
>> Squash system, decompress it.  We see a lot of really fun things.  This is quiz time.  You have a firmware file system.  What do you look for first?  But wait.  Hold on.  We had some interesting observations we want to share first.  In firmware.  One there's a log rotate script.  This is what nailed it home.  This car is a server.  It has the same sort of management as a data center.  A car that has servers inside.  We found another fun thing.  4567.  Okay.  I'll send it over to vpn and see what happens [ laughs ].  Well-played Tesla.
>> Wtf.
>> I think I stopped about 35 thousand of these before I said got it.
>> It was so tempting to upload a shrubbery.
>> You have a file system.  What do you look for first?  Private keys.  Dash dash dash, dsa private key.  Nothing, unfortunately, that we didn't already know about.  We did find a shadow file that was for the IC.  You don't know what that is it is encrypted password file that is used to store the hashes for the account passwords to authenticate that.
>> This is where I literally cried.  After spending months of hunting big game, hack a car, trying to find a way in and it cracks under seconds.
>> So now the question is we have valid authentication to a car, ssh in, the real question is how do we get route?
>> That's when I cried again.
>> So how do we get to the CID?  We did not have the CID shadow.  What do we do?  We went back to the firmware mines and started to look for more.
>> We found the car uses security token and rotated every 24-hour.  Not bad design.  However, it also appears the security token is used in lots of places.  That's not good idea.  For example, Tesla 1 uses that security token as its password.  And it stored in plain text on the file system. 
>> So we went on the IC, found this security token and say can we authenticate to the CID.  Yes is the answer.  Okay.  So now we have route on the cluster and CID of a car.  By the way we are assuming the attacker will do this.  This should be the security model.  They will gain access.  The question is what can they do with that.  So we poked some more. 
>> I was confused by the layout.  The memory storage on the car is the petition you see are labeled.  You can see on the slash.  What it does is [ no audio ].... 
>> The CID the touch screen has dsa key into the instrument cluster.  Now we have legitimate key to it.  Our old friend the ethernet. 
>> A little bit of analysis and script and what we found is the very code that controls the gateway to the ethernet.  A little bit reverse engineering.
>> It's pretty good design.  And you need the token so the token you need to replay and inject the open ethernet to get access you have to have that security token there's no way if you are open internet you can authenticate to that car.
>> The fact that it authenticate every 30 seconds is annoying.  Penny dropped and we had to setup a script that constantly sent it.
>> The nice thing because we have those vpn keys we found the script that retrieve the keys from the mother ship the actual name of the server.  Now we have a script that can retrieve, and build the ethernet key and send every 30 seconds and now we always have access to the car's ethernet assuming we have the car key.  You have to physically disassemble the car.  Do terrible things to the dash panel and once we have the car key we can maintain access to the network.
>> It is almost like a contest in a box with cheese.
>> Where are we right now?  We can access the cid the touch screen without IC.  Security token from the mother ship and create ethernet token wherever we want, now what?  We actually found that vehicle firmware system, I think I did some poking vas, we didn't verify the firmware updates, I think that's important for people to audit to make sure that those updates can't be tamper with.
>> We didn't want to tamper with because we didn't want to risk something up to cause my break his car.  And we want to save something for phase two.
>> At Tesla center.
>> Guess what?  The key is there.  This means with the right key setup and with an a little wifi pineapple you can make wifi talk to your Tesla as long as it's on.
>> The question is is that wifi network better.
>> Open question what do you think you can do if you can get someone connected to your wifi.
>> So we have route on the system what can we do with that?  This is actually a pretty hairy challenge.  There are number of server used in the car.  They were writing in qt.  It's kind of annoying to reverse it's not all good straight c and cleanly follow it.  It's kind of difficult.  So we kind of said let's just not starting this as reverse engineering services and what's going on.  Let's step back how do we figure out what we are after.  We built a tool called test cat.  Broadcast to udp destination port.  There's 50 to 1,000 per seconds, what are the unique payloads.  It identified the payloads it doesn't seen before.  It parses the first 2 bytes of the packet seems to be highly constant.  Roughly calling it message i.d. and command i.d., roughly proximate.  We law it to pull from a p cap file you can play it offline or live listen.
>> And we released our tool as part of the talk.
>> It's already rereleased.  Now we have an anomaly system.  Now we go trigger things that we have not done before and walk the door and park and brake and we found a unique udp packet per event.
>> There are people in the region who think we are strange.  The first thing they saw is taking a brand new car where I happening it apart and then dismantled car.
>> They weren't too happy about that.  Now the question is who is sending these packets.  What CID was doing it?  Qt car vehicle and I felt stupid because I probably should have said that.  That's the nature of these things.  How can we construct the packet?  Now we can finally go to ida.  We tracked down the binary working on, I think you can see that.  Gateway message sender.  You see a bunch of interesting thing.  Gui factory mode.  Off request.  All sorts of fun things.  By the way let me take a step back to say this is awesome.  What I didn't say is can, can, can, can.
>> This is the vehicle of store procedure.  The car has a bunch of commands in the gateway that will let you send command to and relay through the can bust and anything not part of that, you can't send through.  That doesn't mean the gateway can't be compromise.  What it means that unlike with certain other vehicles when the system gets compromise you can't send thing that can't do bad things to the car.
>> That's huge.  The fact that there's isolation is extremely important.  Now we can control a number of things.  CID does touch screen we can now do. 
The question is we can't inject can frames at least so far onto the vehicle network.  We can use api.  The question is can we use that.  What do you want to try?  Ssh channel between cid and server.
>> If you want remote cyber attack to a car, what do you want to do?  We tell the car to shut off, switch off the screen, shut the computer down.  And we tried that.
>> We did this in a closed environment and at low speed.
>> We did not do this on a 70 miles an hour on a freeway. 
[ music playing ] 
[ applause ]
>> So it's important to point that this is not a remote access exploit.  Kevin was happy to sit in the car when I stopped it.  It's important to point out what you saw there was the car traveling at slow speed.  We found that if the car was going 45 miles an hour the screen will go black and music turns off and car stops.  If you are going faster than that a brief warning coming on the screen telling you you can't apply the hand brake at speed.  The screen will go black but you maintain control of the steering and the brake.  The entertainment system killed, you still get fully control of the car and stop the car.  That is phenomenal.
>> Huge 
[ applause ]
>> All right.  We found a number of things Tesla did well.  They are pushing an ota update at ludicrous speed.  In a week, which is extremely awesome.  The update yesterday two of the vulnerability.  Weak password no more and a bunch of others and also hardens the system.  You should be able to attack a car, pop the browser and that shouldn't matter.
>> Also Tesla has announced that they are hiring we all know to be their head of security 
[ applause ]
>> Huge.
>> He's sitting right there.
>> But seriously they've been good to work with.  We were worried at first.  We didn't know how a car company respond to us finding bad things [ no audio ]...
[ applause ]
>> Thank you very much.  Well when we first engineer the model S we know the software system would be the work in process.  We knew we were going to ship and car and not improve the software.  Using the over the air update system that's how we've been able to work collaboratively the research community to find new vulnerabilities and then patch them and fix them.  I really want to thank you guys for the work you've done.  It's making the car safer.  We want to award you with Tesla challenge points, all three of you, for the awesome work you've done.
>> I also want to invite another collaborator onstage as well 
[ applause ]
>> It was John's car that we so cruelly abused.
>> These are very cool.  Thank you so much.
>> One more thing. 
>> Program all the way up to $10,000 is a nominal amount and we may go higher than that in certain cases.  So we want people to participate that and send us what you find.  So thank you.
[ applause ]
>> So what better way to thank them for engaging the community so much.  First time speakers at Defcon.  Not these two who did most of the talking. 
We will give them a shot.  Good idea? 
[ applause ]
>> Thank you.  Cheers 
[ applause ]
>> These guys have 2 minutes of wrap-up time. Get back in here.  Sit down.
>> Can we get the slides back?
>> All right.  Summary of what we found.  In my opinion this is really great when you are auditing big system like this to draw this out.  Here's what you can fine and notice there's one path to gateway.  What did Tesla really do right here?  Awesome otm process.  Ludicrous speed.  Vpns were good, rotate passwords every 24 hours and strong isolation between the drive and vehicle systems.  Of course, they need to improve some things.  The wifi should using wvp enterprise and not have a static key.  There's a general parameter security model.  We definitely recommend that switch away from egg shell principle more to every individual system harden.
>> Don't store sensitive credentials on the plain text in file system.
>> There's of communications that are not mutually encrypted and authenticated on the system.  If you assume adversary in your browser, you want to make sure it's encrypted.  The question everyone is asking.
>> We came up with three recommendations.  The first one is anything man makes man can break.  You need to have a quick way to patch and update them.  You can't rely on people taking the car in.  User can't pay for.  If you are rely someone to pay for data subscription to do update, only about 20% will sign up for it.  Next one, strong isolation infotainment system and drive critical system.  That's really, really important.  Airplane manufacturers have known this.  Car manufacturers need to start doing the same thing.  None of the jeep stuff would have been possible if they had done that.  Finally, this is embedded system you don't control who and when have access to it.  That means every individual component has to have its own robust security model.  If one component is compromised, that doesn't leave to complete compromise of the vehicle.
>> This is a group project.  I want to thank all of them for helping us out.
[ applause ]
>> I want to call out this lady which is married to this gentleman here, who is extremely patient with us being there very late disassembling the car.  
>> Being a wife of a hacker is not an easy job.
>> Lots of folks at cloudfare, thanks so much.  Appreciate it.
[ applause ]