00:00:05.706,00:00:10.711 [applause] >> First thing I need to record my selfie. [pause] [off mic comments] [pause] [off 00:00:31.198,00:00:36.169 mic comments] [whistles heckling from audience] [pause] >> Thanks you guys. Whatever the outcome 00:00:36.169,00:00:41.174 is I have evidence that it was a success. [laughter] Semi success. Um [speaks in French] 00:00:54.454,00:00:58.225 [off mic comments] Excellent! [pause] [off mic comments] [breath] Oh god! Lift You guys 00:00:58.225,00:01:03.163 are working on that? Okay. [speaks in French until 1:06] Um for the yeah sorry closed 00:01:10.370,00:01:16.843 captioning. Sorry folks. A wonderful French movie um called The Dinner Game. Very dark 00:01:16.843,00:01:21.848 French humor. Um who has solar panels? Who cares about their privacy? Yeah! You didn't um 00:01:28.221,00:01:34.127 raise your hand. Get out. [laughter] There's an EFF talk I think next door. You can ask 00:01:34.127,00:01:39.132 about privacy. Still nothing? Is it working on that side? Yeah. Who's seen War Games? Excellent 00:01:42.536,00:01:47.541 movie. It hasn't aged a minute. [laughter] I did. Um but even if uh Litchfield was cool I was 00:01:53.680,00:01:58.018 much more serious about my craft. Serious enough not to have the distraction of a 00:01:58.018,00:02:04.891 girlfriend. By choice of course. Um [laughter] This quote is excellent. It is actually what I 00:02:04.891,00:02:11.264 believe I am. Trying to take things opening them up and figuring out ways to make them 00:02:11.264,00:02:16.269 better. Isn't that why you're all here? [pause, whistle, pause] It's not happy hour yet. 00:02:21.141,00:02:26.146 Hey by the way I need my speaker shot. I could use 2 actually. [off mic comment] Thank you! 00:02:32.753,00:02:37.758 [yell from audience, pause] So we're going to talk solar. Um this is a system by Tygo. 00:02:40.494,00:02:45.732 [pause] I brought the little part that is the only piece we're going to look at today. 00:02:45.732,00:02:50.737 Which is the the connection between the solar array and the internet. [pause] It's really 00:02:55.075,00:03:00.013 cool because not only does it upload configure um production data to the internet. It also 00:03:02.015,00:03:07.020 downloads configuration of the panels. Things like maximum um power voltage maximum 00:03:10.457,00:03:15.462 temperature of the panels and things like that. Of course over the internet um what it does is 00:03:21.568,00:03:28.475 gives the installer the ability to monitor remotely the production of my system. Why? 00:03:28.475,00:03:31.545 Because they have an SLA and they actually guarantee production of my array and 00:03:31.545,00:03:36.550 they'll pay me back if it doesn't produce what it's expected to. Yes indeed. Um I 00:03:39.519,00:03:44.524 could. I would not. Cause think about it. About 9000 Kilowatt hours a year of production. This 00:03:52.132,00:03:57.137 says 15 cents. Yes I could score a thousand two thousand bucks but I would get busted for it. 00:04:00.941,00:04:06.613 Because this is not the only thing that reports my production. So that angle you 00:04:06.613,00:04:11.618 can have fun um not with me. [pause] This is what's what started it all. You know how you 00:04:15.388,00:04:20.393 take your nest and or any IOT device when you initially power it it starts advertising and 00:04:22.629,00:04:28.835 access point. Uh you connect to it. Configure it. Tell it this is my home network. And then it 00:04:28.835,00:04:33.840 shuts down and becomes just a wifi client. Not this one. It connects both to my network as 00:04:36.409,00:04:41.414 well as the open access point. Um that really really bugged me. [pause] So started to need to 00:04:46.052,00:04:52.792 figure I needed to figure out how to fix that problem. And started inventorying all the 00:04:52.792,00:04:57.797 attack surfaces I had uh at my disposal. We talked about the access point. And little HDTP 00:05:00.834,00:05:05.839 server that we'll talk about later. SSH cool. Yeah except there's a built in uh defense in 00:05:10.010,00:05:16.516 depth maybe. It crashes after 15 hundred tries. I have to re-power the uh res power cycle 00:05:16.516,00:05:21.521 the device. So quickly it was no longer funny. Um serial to TCP. I never got it to work 00:05:24.124,00:05:29.129 unfortunately. But it had a nice little UI. Do you want the uh the console to be tunneled 00:05:31.264,00:05:36.770 through TCP or the display. This little guy. Or the gateway that it controls through uh through 00:05:36.770,00:05:41.775 um serial port. [pause] From a physical perspective of course I opened that box. Remember what I 00:05:45.912,00:05:50.917 told you? I take a screwdriver to anything. Um [pause] nicely labeled at the bottom left of 00:05:54.487,00:05:59.492 the screen. You see a little uh silkscreen of console. Guess what. You plug in your um serial 00:06:02.862,00:06:07.867 to USB connector and it works. So I had a nice console interface. Which unfortunately 00:06:10.804,00:06:15.809 required authentication. So back to square one. U boot. Excellent. Maybe I could boot it 00:06:18.745,00:06:25.051 in recovery mode fix the password. No unfortunately they put a password on the uh on the 00:06:25.051,00:06:30.056 boot loader. And yeah I have a confession. I live in California. This was October. 00:06:34.194,00:06:40.734 The middle of winter. This device is outdoors. It was too hard for me to take. So I had to 00:06:40.734,00:06:45.739 look at an easier path and more comfortable. Um so [pause] Behind this access point there 00:06:51.811,00:06:58.485 is a website. As I mentioned. That website has properties. If you use showdon you'll find out 00:06:58.485,00:07:03.423 that actually 12 or so uh very courageous people maybe ignorant decided to have that device also 00:07:10.397,00:07:15.402 internet accessible. [pause] Guys this is where you're supposed to laugh. Thank you. 00:07:20.674,00:07:25.679 [chuckle] Um [pause] thanks to showdon I was able to verify that my findings actually no my 00:07:28.982,00:07:33.987 lawyer's not present so do what you want with he Showdon findings. Um [pause] Remember 00:07:37.190,00:07:43.463 the open access point? It has an SS ID. So I went to those wonderful folks at wiggle dot 00:07:43.463,00:07:50.403 net. And uh looked at their database. Guess what. I'm not the only one who detected those. 00:07:50.403,00:07:55.408 Uh they're all over the world. And they're captured for posterity. You know have GPS 00:07:57.477,00:08:02.415 coordinates of all of those devices or some of those devices. Um who war drives? 00:08:05.285,00:08:10.290 Thank you. Keep doing it. Upload to wiggle um because it's a treasure trove of data about 00:08:13.827,00:08:18.832 people that like I want to say F up no mess up. Let's go back to the web server. [pause] That's 00:08:23.336,00:08:29.275 it. My talk is over. Thank you. Um there's an authentication screen we can't do much about it 00:08:29.275,00:08:34.280 can we? Of course not. It's funny how I've seen other slide decks today that also use a 00:08:38.485,00:08:43.490 password file called rock Q dot txt. Who's used it in the past? [pause] Oh come on guys. If you 00:08:48.261,00:08:53.266 didn't raise your hand that's the best password file on earth. Um so I ran my brute force. 36 00:08:57.937,00:09:02.876 hour later yeah I know I know I'm lazy but it was 36 computers at computer hours not mine. Um 00:09:05.078,00:09:10.083 turns out admin support works very well. [laughter] Okay where do we go from there? Looking 00:09:15.355,00:09:21.261 around the little website on the server. There's a nice little page that caught my attention. 00:09:21.261,00:09:27.967 No such file or directory. Ooooo, guess what happens when you put a file there? Um for 00:09:27.967,00:09:33.773 those of you who don't have their url decode option on Google glasses this is what it 00:09:33.773,00:09:38.778 looks like. [pause] Copy shadow file in to that location. What would happen? Yeah I might break 00:09:42.816,00:09:47.821 my 20 thousand dollars solar array by putting something there. Um but I didn't. [pause] 00:09:51.891,00:09:56.896 By the way this MD5 I tried to brute force it. I failed. If you ever get to it I believe it is 00:10:01.734,00:10:06.739 still on those devices. Um please send me an email. I would appreciate it. So that that 00:10:09.108,00:10:15.281 route didn't work out. Um I needed something easier. Remember I can I can essentially 00:10:15.281,00:10:20.286 run a script through that injection. Um so PS all oh guess what? The HTDP server is running 00:10:25.592,00:10:30.597 under root. Bingo. Also the manufacturer is nice enough has net cap already on the device. 00:10:33.500,00:10:38.505 Ooooo [laughter] By the way I won't admit that in public but it still took me 4 to 6 hours to 00:10:41.674,00:10:46.679 get my reverse shell working. But I didn't say that. Um I did eventually get it working. I had 00:10:49.282,00:10:55.321 root on that device. What do you do with root? I know what I didn't do. I didn't get a copy 00:10:55.321,00:11:00.260 of the file system. So once I was locked out I no longer had anything to work on. But after a 00:11:04.731,00:11:09.736 little bit of uh kung fu with the drive uh mount [pause util 11:16] come on [applause] I know 00:11:19.412,00:11:24.417 I know it feels good to pretend I'm that good. Um what I did was not rocket science. I just had 00:11:26.653,00:11:31.658 the time to do it. [pause] Clearly that manufacturer picked the wrong customer to sell a 00:11:38.531,00:11:43.536 device to. I'm sure they're still regretting that move. Um it probably cost them a lot more 00:11:45.538,00:11:50.543 in uh cleanup then it did uh in profits. So anyhow looking around the file system something 00:11:53.646,00:11:59.852 caught my attention. Actually not the file system the running processes. Open VPN. You guys 00:11:59.852,00:12:04.791 know what open VPN is for? A VPN tunnel. Guess what. That VPN tunnel was on at all times on 00:12:07.794,00:12:12.799 the device. I didn't do it. And I swear this is not a joke. I did not scan that VPN subnet. 00:12:16.369,00:12:21.374 The manufacturer confirmed that all of its little siblings are on that subnet. [pause] Of 00:12:27.513,00:12:32.518 course no where was it mentioned in any of the documentation that nobody ever reads that there was 00:12:35.321,00:12:41.961 a VPN. Remember that device that is still on my home network? I was trusting it even though it 00:12:41.961,00:12:46.966 didn't appear trustable. I was still doing that. Um so let's move on to me trying to get 00:12:50.169,00:12:55.174 something done about the device. So I tried politely in October to get their attention. Hey 00:12:57.677,00:13:03.549 guys. There might be a problem. You know it it I'd like to talk to someone who actually 00:13:03.549,00:13:08.554 understand security. Yeah by the way in the back? If the font size is too small next time 00:13:10.790,00:13:15.795 remember that DefCon is all about line con. Get early to the talk. [laugh] So a few emails 00:13:21.100,00:13:26.105 later um while still trying to reach to people that might understand me through LinkedIn 00:13:29.676,00:13:34.681 my in clueless installer and his contacts I got nowhere. Actually it got even worse. We're now in 00:13:38.818,00:13:43.823 mid December. Are you the owner of this device? Do you have the right to do what you're doing? 00:13:48.895,00:13:53.900 Yeah I've seen that play out not that well. Um [laughter] They actually already had my full 00:13:57.403,00:14:02.141 name my email address my everything. They already knew everything about me but they 00:14:02.141,00:14:07.146 couldn't find me in the database. Um this was the icing on the cake. For those in the 00:14:09.282,00:14:16.189 back I will read what is highlighted. Or I'll I'll paraphrase. We can help you get 00:14:16.189,00:14:21.194 access to the system. Do I need access to the system at that point? No. I can help myself. Um 00:14:24.230,00:14:29.235 and I I need to read that one. Quote Info of system installed on your roof is always kept as 00:14:32.171,00:14:38.811 confidential since it was installed. Apparently before it is installed not guaranteed and 00:14:38.811,00:14:43.816 you know English is my second language I don't I don't understand that sentence. So 00:14:47.086,00:14:52.592 time to stay to change strategy. Clearly I'm getting nowhere. I've been at it for 2 months 00:14:52.592,00:14:57.597 already. Um I'm talking to the wrong kind of support. So I send this email. What I'm saying 00:15:01.267,00:15:06.572 there is hey guys here's a picture you remember the root picture? Here's a picture. The 00:15:06.572,00:15:11.944 last line doesn't belong there. Forward this to whoever's in charge. I don't want to talk to 00:15:11.944,00:15:16.949 you no more. Remember the VPN tunnel? Within an hour they were logging in on that device and 00:15:22.021,00:15:28.961 they were staring cleaning up. Not not security cleaning up. Damage control cleaning up. 00:15:28.961,00:15:34.700 Disabling my account. Shutting down the web server. Uh and things like that in the process 00:15:34.700,00:15:39.705 yeah disabling my entire array went offline for 4 to 6 hours. Um I was not done helping guys. 00:15:42.608,00:15:48.648 Please. I was trying to be nice. Um thankfully I didn't tell them about one thing I had found 00:15:48.648,00:15:53.653 while browsing the file system. In that CGI bin folder there's also a file called shell. 00:15:56.856,00:16:01.794 [laughter] So I got back in and uh told them the next day about it. And repeat. So that's the 00:16:07.834,00:16:14.774 best part. Once I got to talk to someone in charge of their product development. Great guy. 00:16:14.774,00:16:19.779 Um his first response was there's a problem. This is not a production device. What? I 00:16:23.749,00:16:28.754 bought a Tesla and the Tesla price and the auto pilot crashes on me because it's a debug 00:16:31.724,00:16:37.296 version I have? No sorry Tesla guys. I'm just jealous. Everybody in my neighborhood has 00:16:37.296,00:16:42.301 one except for me. So if you guys are thankful for the talk don't hesitate. Thank You. 00:16:49.842,00:16:54.046 [laughter] [applause] Um [applause] So 6 months later I'm pretty sure they were actually 00:16:54.046,00:16:59.785 not lying. It was a very convenient excuse. But they happen to ship me a development 00:16:59.785,00:17:04.724 build. And a few thousand others uh throughout the world. [pause] god [pause] What they did well 00:17:12.732,00:17:18.070 once I had a line of communication with Tygo they were actually very welcoming of 00:17:18.070,00:17:23.075 my finding. And relatively forthcoming with sharing the insider information. Like for 00:17:25.845,00:17:31.417 example telling me oh all of those devices are on the same subnet through the VPN tunnel. 00:17:31.417,00:17:38.090 Um that would have been preferable for not them not to tell me that. Um one thing I 00:17:38.090,00:17:43.095 discovered lot shipping. Specially for the one oh this is a very important question guys. 00:17:47.833,00:17:52.838 Who in the audience is a black hat versus a white hat? Come on raise your hands. Oh my god 00:17:54.941,00:17:59.946 there's not a single hand up. [laugh] Yeah okay. [laugh] Um so next time you go into a system 00:18:02.114,00:18:07.119 you're not authorized to think about disconnecting it from the network before. Because this guy 00:18:09.488,00:18:14.493 ships its logs every half an hour. And boy was I noisy. Of course there was nobody looking. 00:18:16.696,00:18:21.701 Thank god. [laugh] But uh it's it's important to realize that even small IOT devices have that 00:18:25.071,00:18:30.076 capability. And uh you might trigger a few alerts if you're not too careful. So got root. I 00:18:36.015,00:18:42.588 made fun of the vendor. Why am I talking about this? And this is actually the most important 00:18:42.588,00:18:49.061 slide of the entire presentation. Yeah I could remotely see this little red 00:18:49.061,00:18:54.066 button? There's software behind it. I could remotely shut down any of those thousands of solar 00:18:56.969,00:19:01.907 arrays. I could be a pain to people off the grid. Maybe. I don't have there's not enough 00:19:04.877,00:19:11.217 electricity production for it to be meaningful yet. It will be in a few years. But not today. 00:19:11.217,00:19:17.056 What's more important is this is a bought. I could have a thousand of those remotely 00:19:17.056,00:19:22.061 controlled on your whole network spying on your home activity. You know oh shoot my my kid is 00:19:27.299,00:19:32.304 here so I can't say prawn but things like that. Um [pause] The biggest part the part that bugs 00:19:38.344,00:19:43.349 me the most is even though I've been a security practitioner for a long time. Only after this 00:19:47.586,00:19:52.591 device being on my network did I realize I really needed 2 networks. My home personal 00:19:55.928,00:20:00.933 network and a completely independent IOT network. On which I have of course this guy 00:20:03.302,00:20:08.307 now. He was the first candidate. But the nest um a few development boards. Who's played 00:20:11.210,00:20:16.215 with the particle photon photons? Yeah. Those are excellent devices. Um but just 00:20:19.318,00:20:25.958 like this guy don't trust them. Um my security cameras you know those cameras that I bought on 00:20:25.958,00:20:30.963 allibaba with that Chinese firmware? That is apparently very chatty. Uh I won't go 00:20:33.132,00:20:38.137 further. So yeah. Is your mom or your brother or your family expected to have 2 networks at 00:20:42.775,00:20:47.780 home and to be able to manage those? No. There is no way that that even us handle it. There is 00:20:51.283,00:20:56.288 no way that customers of IOTs can be expected to actually protect themselves from those 00:21:00.292,00:21:05.297 devices. That is a very sad state and I hope that message comes out of DefCon as much as 00:21:09.034,00:21:14.039 possible. Because it is time that we have a URL rating of devices uh that also takes into 00:21:17.276,00:21:22.047 account your privacy. Cause we all have that expectation. You don't buy a car without seat 00:21:22.047,00:21:27.052 belts. [applause] Yes. [applause] Responsible disclosure is hard. Yes. Don't 00:21:34.026,00:21:39.031 give up. Please. Follow responsible disclosure. And finally. Thank you to all IOT 00:21:42.701,00:21:47.706 devices for so much entertainment. [laugh] Thank you to quite a few people. My wife 00:21:51.010,00:21:56.015 for tolerating my late nights. Uh Rafael where are you? Stand up. [pause] Keep doing your 00:22:02.388,00:22:07.393 packet storming. [applause] And ty Tygo for not suing me. Thank you. Uh you got me scared there. 00:22:12.665,00:22:17.670 Guys thank you. [applause] [off mic comment] Yeah you screwed up [fades out]