00:00:00.567,00:00:03.971 >> So, uh, welcome. for-, thank you for joining us this 00:00:03.971,00:00:07.808 morning. It's super, uh, super not late... but early. [chuckle] 00:00:07.808,00:00:10.077 >> Someone followed the "Three, two, one rule" to the letter 00:00:10.077,00:00:15.482 last night, didn't they? >> Yea, sorry... [laughter] >> Yea... >> 00:00:15.482,00:00:19.019 My bad. [audience noise] [pause] Hey, but at least I got three... 00:00:19.019,00:00:22.322 So, uh, in, in case you don't know where you are - we're doing 00:00:22.322,00:00:25.926 Vulns 1 0 1 in here. Uh, hopefully everyone can learn a 00:00:25.926,00:00:28.161 little bit of something. We all wanted to ask a couple of 00:00:28.161,00:00:31.231 questions before we jump in further. Uh, number one, like, 00:00:31.231,00:00:35.669 how many people here is their first DefCon? [cheering] >> Wow! 00:00:35.669,00:00:39.473 >> Alright, nice... [applause] >> That's very nice. [applause] 00:00:39.473,00:00:43.610 So how many... Uh, what was the other question we wanted to ask? 00:00:43.610,00:00:46.647 >> How many of you here are, are here for this talk? You've just 00:00:46.647,00:00:49.383 started out in vulnerability research and you wanna, uh, 00:00:49.383,00:00:54.054 learn how to improve your game. [pause] [audience noise] Okay, 00:00:54.054,00:00:57.524 and how... An, and how many of you are not familiar with 00:00:57.524,00:01:00.427 research but you're curious about it and so that's why you 00:01:00.427,00:01:05.599 came here? [pause] >> It's about even. >> Okay, good. >> That 00:01:05.599,00:01:09.903 doesn't help us anyway, but thanks. [laughter] >> The, the, 00:01:09.903,00:01:12.539 yea. WHat we realized a little bit, uh, earlier this week as we 00:01:12.539,00:01:16.443 were practising is that we were trying to target two audiences - 00:01:16.443,00:01:19.046 uh, newbies and then people who were curious about vulnerability 00:01:19.046,00:01:22.449 research but didn't know much about it. And so, sometimes for 00:01:22.449,00:01:24.451 those who are curious about it we might go into some 00:01:24.451,00:01:27.888 terminology you might not be fully familiar with. Uhm, we're 00:01:27.888,00:01:29.890 just gonna move on and we'll be happy to follow up with you 00:01:29.890,00:01:32.960 afterwards as well. >> Yea, and neither of us hide on the 00:01:32.960,00:01:35.562 internet - we're pretty approachable, so don't be afraid 00:01:35.562,00:01:40.233 of us. So, introduction, I'm, uh, Josh, "jduck" I go by on the 00:01:40.233,00:01:43.070 internet, that's how you find me there. Uh, I've been doing VR 00:01:43.070,00:01:47.107 for 20 years and so this, this 20 years actually includes 00:01:47.107,00:01:50.811 several years as a hobbiest. Uh, and at one point I ran the 00:01:50.811,00:01:53.580 iDefense Vulnerability Contributor Program where we 00:01:53.580,00:01:56.049 would actually vulnerabilities from researches and get them 00:01:56.049,00:01:58.885 fixed then coordinate and all that stuff. So, that's, that' 00:01:58.885,00:02:01.221 how I met Steve and we, we did a fair amount of work together at 00:02:01.221,00:02:05.225 that point. >> Yea. An we met through the, uh, CVE programme 00:02:05.225,00:02:10.664 which I was a co-founder of and led from 1999 until, uh, the end 00:02:10.664,00:02:12.966 of last year, basically. Uhm, there's a thing called re, 00:02:12.966,00:02:16.970 responsible disclosure I am a survivor of the responsible 00:02:16.970,00:02:21.174 disclosure wars. I now call them coordinated disclosures, I 00:02:21.174,00:02:23.744 coined the responsible disclosure term for which I will 00:02:23.744,00:02:26.480 be eternally sorry! [audience noise] [laughing] But it served 00:02:26.480,00:02:30.050 its purpose. I started getting into, uhm, classification of 00:02:30.050,00:02:34.121 vulnerabilities as well which is where CWE comes from. And, uh, I 00:02:34.121,00:02:39.526 was also a participant in the development of CVSS version 2. 00:02:39.526,00:02:41.928 >> Quick question, how many people in here have a CVE to 00:02:41.928,00:02:47.668 their name? [pause] >> Alright, let's fix that. That was like a 00:02:47.668,00:02:51.605 dozen I think. >> Yea.. >> That's pretty good. So, uh, why 00:02:51.605,00:02:54.441 are we doing this? We want to fix that, right? We want more 00:02:54.441,00:02:58.011 people out there doing research into vulnerabilities, uh, 00:02:58.011,00:03:01.815 software and hardware into all kinds of systems. Uh, cause as 00:03:01.815,00:03:05.285 we've seen there's lots of crazy things possible. Uh, it was very 00:03:05.285,00:03:07.721 interesting to see the previous talk - the guy hacked into a 00:03:07.721,00:03:12.025 loyalty program that's, uh, that's good stuff. It's fun. So, 00:03:12.025,00:03:17.097 uhm, what else? >> We have this little tiny picture on the slide 00:03:17.097,00:03:18.899 on here. It's like tiny. If anyone works... >> You guys are 00:03:18.899,00:03:21.001 seeing the slides much better than we do. >> Yea we're flying 00:03:21.001,00:03:23.637 a little bit blind. >> If anybody works at Google Slides 00:03:23.637,00:03:25.872 you might need to work on this thing. [audience noise] >> We 00:03:25.872,00:03:28.642 can, uh, we can move on. >> Alright, yea, so just to get 00:03:28.642,00:03:32.012 people involved. So, disclaimer up front, right? This is our 00:03:32.012,00:03:35.782 opinion, we've did lots of stuff with vulns over the years - find 00:03:35.782,00:03:40.020 em', analyze 'em all that stuff. And, uh, you know, we just 00:03:40.020,00:03:42.222 formulated these opinions, so, they, they may not be right for 00:03:42.222,00:03:45.992 you. Uh, s... [pause] Take that with a grain of salt and, and, 00:03:45.992,00:03:49.529 you know, remember that, that you, you're your own person and 00:03:49.529,00:03:52.032 you gotta find your own path, uh, we're just trying to help 00:03:52.032,00:03:56.136 you see some of the stuff. So... [audience noise] Lastly, there's 00:03:56.136,00:03:58.438 no new exploits here so if you came here for... Who came here 00:03:58.438,00:04:05.312 to see new exploits? [pause] Okay, good, thank you. [chatter] 00:04:05.312,00:04:07.647 [pause] >> Okay, so first of all there's a question about what is 00:04:07.647,00:04:09.950 a vulnerability in the first place? And we'll start with what 00:04:09.950,00:04:13.487 a vulnerability is not. One of the most commonly confused,uh, 00:04:13.487,00:04:16.957 terms is "Exploit" versus "Vulnerability' and a lot of 00:04:16.957,00:04:19.626 people think that exploit and vulnerability actually mean the 00:04:19.626,00:04:23.630 same thing. However, they don't. An, an exploit is really a 00:04:23.630,00:04:26.199 sequence of steps that's used to take advantage of a 00:04:26.199,00:04:29.402 vulnerability. A vulnerability is a problem within the code 00:04:29.402,00:04:32.839 itself which is kinda, more or less, uh, sits there, uh, 00:04:32.839,00:04:36.710 waiting to be, uh, waiting to be exploited. These are almost 00:04:36.710,00:04:40.013 circular-sounding terms but, uhm, face a number of 00:04:40.013,00:04:43.650 difficulties in actually really defining these a little bit more 00:04:43.650,00:04:47.721 carefully. And I think that's a reflection partially of, uhm, 00:04:47.721,00:04:50.924 the relative immaturity of the vulnerability research, uh, 00:04:50.924,00:04:55.328 specialty. [pause] >> Yea, I wish I could expand this picture 00:04:55.328,00:04:59.733 for you. But i think, what does Taylor Swift say here? "To, to, 00:04:59.733,00:05:04.171 uh, to be loved is to be vulnerable"... uh and... I got.. 00:05:04.171,00:05:07.507 >> To love is to be vulnerable and to be loved is the greatest 00:05:07.507,00:05:12.312 exploit... >> Greatest exploit, right? >> Pretty good quote. So 00:05:12.312,00:05:14.881 we, we go back and forth on what this definition of what is a 00:05:14.881,00:05:18.351 vulnerability? Uh, and, there's so many different way to define 00:05:18.351,00:05:21.521 it, I think Steve did a great job already saying it. Uh, one 00:05:21.521,00:05:23.557 of the biggest things really is that you have some kind of 00:05:23.557,00:05:28.562 impact on a system. Uh, if, if, uh, you don't have some kind of 00:05:28.562,00:05:31.064 impact and you're not changing the way things working it's 00:05:31.064,00:05:34.234 pretty much not a vulnerability but when, you know, Greg McManus 00:05:34.234,00:05:38.104 at iDefence taught me, uh, you can, you know, through heavy 00:05:38.104,00:05:41.074 abuse of him asking me this question I told him I thought I 00:05:41.074,00:05:43.977 found something, so he's like "What do you have and what do 00:05:43.977,00:05:46.580 you get?" And, you know, for if what you are getting from this 00:05:46.580,00:05:50.116 vulnerability if you manage to exploit it, it's not better in 00:05:50.116,00:05:52.886 some way than what you started with. Then you do not have a 00:05:52.886,00:05:55.322 vulnerability, it's uh, it's more like a bug or something 00:05:55.322,00:05:58.325 really annoying. >> And this is one of the common problems we 00:05:58.325,00:06:01.361 run across, with, uh, with new researchers who try and report 00:06:01.361,00:06:03.363 CVEs or something like that. [audience noise] [coughing] They 00:06:03.363,00:06:06.867 find something that may be a bug or may actually be a feature but 00:06:06.867,00:06:10.103 it is something that is legitimately already allowed or 00:06:10.103,00:06:13.273 somebody... or with the privileges you already have you 00:06:13.273,00:06:17.010 can already go legitimately go through some other route to, uh, 00:06:17.010,00:06:19.846 uh, uh, obtain those, uh, additional capabilities. So 00:06:19.846,00:06:24.684 that's one particular point of confusion. [pause] >> Yea, so, 00:06:24.684,00:06:26.686 uh, I mean, let me just state, uh ano, another important point 00:06:26.686,00:06:29.422 to make about vulnerabilities is, uh, as Steve knows with, 00:06:29.422,00:06:32.459 well, his classification work in CWE that there are many many 00:06:32.459,00:06:34.995 properties of vulnerabilities, one of the, like I mentioned 00:06:34.995,00:06:37.964 before, the impact is a really important one. But, also, the 00:06:37.964,00:06:41.301 user act, the user interaction is an interesting one. [ahem] 00:06:41.301,00:06:43.737 These things are used by the, the, offensive side to 00:06:43.737,00:06:48.174 prioritise patching and strategies around defense. So, 00:06:48.174,00:06:52.312 uhm, this is just a couple of important properties. How long, 00:06:52.312,00:06:54.814 uh, another interesting one these days which is getting more 00:06:54.814,00:06:57.484 and more interesting is "How hard does this thing actually 00:06:57.484,00:06:59.753 need to exploit?". And I think, you know, and as things have 00:06:59.753,00:07:02.188 improved over the years, it's gotten [ahem] much harder to do 00:07:02.188,00:07:04.824 that. [audience noise] >> Yea it's, it's become a lot more 00:07:04.824,00:07:08.528 difficult, uh, to do that and, uh, that's, uh, thanks to the 00:07:08.528,00:07:11.598 defensive work in the, uh, build up of many different types of 00:07:11.598,00:07:15.235 protection mechanisms, and so, there's almost like a 00:07:15.235,00:07:17.737 "Heisenberg-ian", uh, approach to interpreting vulnerabilities 00:07:17.737,00:07:19.072 these days. Uhm, uh, where something was clearly 00:07:19.072,00:07:20.407 exploitable perhaps 10 years ago it may wind up taking a whole 00:07:20.407,00:07:21.775 lot of work. That's one of the great things about the defensive 00:07:21.775,00:07:23.109 side of understanding vulnerabilities which is to 00:07:23.109,00:07:28.114 really build in these system, systematic defenses. [pause] >> 00:07:38.291,00:07:40.760 I have a whole rant that we can do on this some other time. 00:07:40.760,00:07:44.397 [audience noise] >> But we'll save that for later. >> Yea. >> 00:07:44.397,00:07:47.267 Alright, and so then, finally we get to what is the vulnerability 00:07:47.267,00:07:51.604 research, and, and, in this case what we're saying is the process 00:07:51.604,00:07:57.744 of analyzing a product protocol or algorithm. [pause] You guys 00:07:57.744,00:07:59.145 already read this one... [chuckle] >> Work specification 00:07:59.145,00:08:03.650 to basically to, to try and find vulnerabilities or understand 00:08:03.650,00:08:07.120 them. Uh, one or more vulnerabilities basically. So, 00:08:07.120,00:08:10.156 there are different, uh, kind of approaches different, uh, 00:08:10.156,00:08:13.560 different kinds of products or specifications and so on that 00:08:13.560,00:08:17.497 you may decide to look at. It's all, uh, more or less falls 00:08:17.497,00:08:19.899 under the umbrella-term of "Vulnerability Research". 00:08:19.899,00:08:23.837 However, the term itself is, uhm, treated and interpreted a 00:08:23.837,00:08:26.573 little bit differently. You might sometimes hear the term 00:08:26.573,00:08:30.377 "vulnerability discovery" and that's really intended by, used 00:08:30.377,00:08:33.847 by people who want to distinguish from, let's say 00:08:33.847,00:08:38.551 academic strength research, uh, versus going and doing bug 00:08:38.551,00:08:41.688 hunting. And so, on... and some people use that term. That 00:08:41.688,00:08:45.125 distinction does wind up being important, I think, sometimes. 00:08:45.125,00:08:48.495 Uh, but again, the terminology is still kind of emerging and 00:08:48.495,00:08:51.398 you don't have a lot of agreement. >> I'll again, when I 00:08:51.398,00:08:54.501 do an exploit here and I personally think exploits and 00:08:54.501,00:08:57.170 exploit development and stuff like that falls under VR... >> 00:08:57.170,00:09:01.641 As do I. >> Yea, but it's, uh, it's not really our focus here 00:09:01.641,00:09:04.944 so let's keep going. >> It, it, it's really about uhm, uh, 00:09:04.944,00:09:07.947 solving puzzles where you don't even know what the puzzle is in 00:09:07.947,00:09:11.151 the first place. You don't even know if you'll find a puzzle. 00:09:11.151,00:09:13.953 Then maybe you find a puzzle which lead you to other puzzles 00:09:13.953,00:09:17.424 in some way. That's one of the big attractions to me, for, uhm, 00:09:17.424,00:09:21.027 vulnerability research. [pause] >> Alright, so why do it? Uh, 00:09:21.027,00:09:22.962 you know, if you, incase you're very new and you're just 00:09:22.962,00:09:25.465 curious. Here is some reasons why you might wanna get 00:09:25.465,00:09:27.967 motivated to do the hard work which is vulnerability research. 00:09:27.967,00:09:32.972 Uh, I can't really see the slide. [chuckle] [pause] Hooray 00:09:35.041,00:09:37.844 Google Slides... >> I'm serious, it's so tiny! The speaker notes 00:09:37.844,00:09:39.179 are the whole screen and the slide is like... a little tiny 00:09:39.179,00:09:40.914 square. >> Hope there aren't any Google people in the room, 00:09:40.914,00:09:42.282 but... >> I hope there are so they'll fix this stuff. 00:09:42.282,00:09:47.287 [laughter] [pause] [chuckle] So then, what are the big points to 00:09:53.493,00:09:56.729 note here as you look at this, uh, nice little word cloud is, 00:09:56.729,00:10:00.767 uh, go ahead... >> Ah! That's okay. Let's just go the next 00:10:00.767,00:10:03.369 slide. >> Yea. [chuckle] [laughter] The, the main 00:10:03.369,00:10:06.940 takeaway from the previous slide is that uh, there are many 00:10:06.940,00:10:09.742 different motivations, that many different researchers have and 00:10:09.742,00:10:12.946 your motivations may not be the same as others. And in addition 00:10:12.946,00:10:16.015 when you're dealing with vendors - vendors may only have 00:10:16.015,00:10:18.651 experienced or they may only assume certain kinds of 00:10:18.651,00:10:22.956 motivations from you. And so that potentially causes, uh, uh, 00:10:22.956,00:10:27.360 certain difficulties when interacting with vendors. 00:10:27.360,00:10:30.163 [pause] >> I personally like just about all these words one 00:10:30.163,00:10:35.869 here. Uhm... I dunno. So there's a lot of different careers. 00:10:35.869,00:10:38.171 Steve, you wanna talk to them? >> Yea, so, uhm, there's a 00:10:38.171,00:10:41.341 number of different careers but it's not like, uhm, you know, 00:10:41.341,00:10:44.811 uh, a career shop that you can go to this is still a new field, 00:10:44.811,00:10:48.481 uhm, in my way of thinking we're like, kind of entering the 00:10:48.481,00:10:51.384 second generation, we're sort of the first generation. >> We're 00:10:51.384,00:10:55.355 s... [deep breath] We're trying not to skip one. >> We're... 00:10:55.355,00:10:58.224 [chuckle] And it's really good to see a full room here actually 00:10:58.224,00:11:00.994 because we need a lot more people doing vulnerability 00:11:00.994,00:11:04.230 research cause we've only, you know, seen the tip of the 00:11:04.230,00:11:08.201 iceberg. But, what's called vulnerability research, uh, it 00:11:08.201,00:11:10.436 may vary but there's a lot of different things that one can 00:11:10.436,00:11:13.706 do. Yes you can go and you can hunt bugs and you can hunt 00:11:13.706,00:11:16.342 vulnerabilities. Other people may really like building up 00:11:16.342,00:11:19.279 exploits against those vulnerabilities. And I agree 00:11:19.279,00:11:21.981 with Josh that, uh, that's an aspect of vulnerability 00:11:21.981,00:11:24.784 research. Then there's other stuff that maybe a bit more 00:11:24.784,00:11:28.054 adjacent to vulnerabilities that still involves a lot of analysis 00:11:28.054,00:11:31.257 so... I previously mentioned things like building defensive 00:11:31.257,00:11:34.093 protection mechanisms to help prev, to help protect against 00:11:34.093,00:11:38.097 entire classes of vulnerabilities. Uhm, you know, 00:11:38.097,00:11:42.101 for CVE a lot what we did was to, uhm, catalog those 00:11:42.101,00:11:45.572 vulnerabilities and in a way try and figure out how the CVE 00:11:45.572,00:11:49.042 identifier could be used to help people to coordinate. [audience 00:11:49.042,00:11:53.179 coughing] And so on... uhm, and it, and another option is to, 00:11:53.179,00:11:57.150 uh, really work on fixing them, uh, you could be working at a, a 00:11:57.150,00:12:00.153 developer somewhere and, uh, see a vulnerability that's been 00:12:00.153,00:12:04.257 reported, and figure out how to actually fix the code. And, I 00:12:04.257,00:12:07.961 guess one, one note that just, just just popped into my mind 00:12:07.961,00:12:10.897 here is that there are a lot of vulnerability researchers who 00:12:10.897,00:12:14.200 discover vulnerabilities would have no idea actually how to fix 00:12:14.200,00:12:16.869 them. It's a completely different mindset, you have to 00:12:16.869,00:12:20.406 have, you can have a really, uhm, solid development 00:12:20.406,00:12:23.243 background - you need a solid development background - to 00:12:23.243,00:12:26.879 build a good fix. But, that's not necessarily, uhm, the kind 00:12:26.879,00:12:30.483 of skill you need to do vulnerability research. [pause] 00:12:30.483,00:12:34.153 >> Yea, that's right, uh, one thing I will add is that if you 00:12:34.153,00:12:37.457 do start doing this stuff or find a job doing this stuff you 00:12:37.457,00:12:40.126 basically have unlimited obscurity. [ahem] [audience 00:12:40.126,00:12:42.762 noise] For the, for the foreseeable future... >> So 00:12:42.762,00:12:45.098 there's a lot of different employers that, you know, you 00:12:45.098,00:12:47.867 basically could have, uhm. I mention you know, uhm, you could 00:12:47.867,00:12:50.637 work at a software vendor, you could work for a government 00:12:50.637,00:12:53.673 organisation or CIRT coordinator. You could work at 00:12:53.673,00:12:57.043 a, a commercial enterprise where they develop security products 00:12:57.043,00:13:01.114 or whether is it does consulting services. Uhm, however, these 00:13:01.114,00:13:04.817 days pretty much every business that's out there is more or less 00:13:04.817,00:13:07.987 a software developer. Think about Target, think about other 00:13:07.987,00:13:11.891 kinds of brick-and-mortar, uh, you know, companies. Those all 00:13:11.891,00:13:15.595 develop software either in-house to help them manage their 00:13:15.595,00:13:19.499 operations or externally in respect to websites and so on. 00:13:19.499,00:13:23.569 And, as you all have probably heard there is a huge demand and 00:13:23.569,00:13:27.540 so these are some of the options that you, uh, that you could 00:13:27.540,00:13:30.943 look at and you would be most likely welcomed with open arms 00:13:30.943,00:13:34.047 by someone somewhere. Because we need a lot more researches. >> 00:13:34.047,00:13:37.650 I, uh, I'm a little bit partial to the bounty programs to really 00:13:37.650,00:13:41.754 find them. And, uh, uh, if you have a good employer, like I'm 00:13:41.754,00:13:45.725 fortunate enough to do, and you can have bonus money and throw 00:13:45.725,00:13:50.296 parties at BSides and stuff like that... [laughter] Cheers! 00:13:50.296,00:13:55.768 [laughter] Yea, so let's do the next one. >> Uh, so, uh, these 00:13:55.768,00:13:58.337 next couple slides is really just kinda, uhm, a little bit 00:13:58.337,00:14:00.606 more like a disclaimer than normal in terms of our opinion 00:14:00.606,00:14:03.876 but based on our personal experiences and, I've actually 00:14:03.876,00:14:06.846 done some vulnerability research myself, it was awhile ago. But 00:14:06.846,00:14:09.749 based on our personal experiences and interacting with 00:14:09.749,00:14:13.686 other researchers, uh, there are a number of personality traits 00:14:13.686,00:14:17.256 that generally seem to be... useful for longer term success 00:14:17.256,00:14:20.493 within, uh, within vulnerability research. So, for example, a 00:14:20.493,00:14:23.362 willingness to work, uh, independently, a willingness to 00:14:23.362,00:14:26.966 learn. Uh, being very, uh, uh, critical thinking, you always 00:14:26.966,00:14:31.270 have to be more or less question your own assumptions. >> That's 00:14:31.270,00:14:35.942 a good point, I don't even think that's in the slides but that's 00:14:35.942,00:14:40.379 a really good point. >> And, uh, really it's... It's primarily a 00:14:40.379,00:14:45.251 solitary effort, you need to be, you need to diligent, uhm, and, 00:14:45.251,00:14:50.656 uh, you see some of the other features there basically. But 00:14:50.656,00:14:53.659 two of the biggest personality traits that we believe are 00:14:53.659,00:14:57.396 important are "Patience" and "Persistence". Patience is 00:14:57.396,00:15:01.367 essential not only with yourself and the process of discovering 00:15:01.367,00:15:04.437 and investigating these vulnerabilities but patience 00:15:04.437,00:15:06.739 when dealing with others. Especially when dealing with, 00:15:06.739,00:15:10.610 say, vendors that might not necessarily behaving exactly the 00:15:10.610,00:15:12.779 way you would want to when you're, uh, trying to 00:15:12.779,00:15:16.816 communicate with them. [pause] So those were some of the 00:15:16.816,00:15:18.785 "should have" personality traits, these are some of the 00:15:18.785,00:15:21.921 ones that we think are nice to have, sti, still a greater 00:15:21.921,00:15:26.125 formula for success here. To really be able to be focussed, 00:15:26.125,00:15:30.930 to, uhm, to seek to improve software, which is, uh, a common 00:15:30.930,00:15:34.133 thing. The ability to collaborate whether that's uh, 00:15:34.133,00:15:38.137 to collaborate and work with other people is something that 00:15:38.137,00:15:42.208 we believe is important. There is, there can be many rock stars 00:15:42.208,00:15:46.279 and not so rock stars that don't work well with other people. 00:15:46.279,00:15:49.649 Uhm, but that often times, especially if you're just 00:15:49.649,00:15:54.187 starting out I think is probably a career limiting, uh, uhm, kind 00:15:54.187,00:15:58.791 of attitude that one would take. And we also have here a notion 00:15:58.791,00:16:02.595 of having, kind of, an addictive personality. So for example at 00:16:02.595,00:16:08.801 CVE, you know, I stayed at CVE for 16 years through 70-thousand 00:16:08.801,00:16:11.437 vulnerabilities. Now I didn't investigate and look at all of 00:16:11.437,00:16:14.507 them but you could say that that might be kind of indicative of 00:16:14.507,00:16:17.476 an addictive sort of personality. And, Josh, you 00:16:17.476,00:16:20.012 know, how many weeks, or days, or months have you spent on, 00:16:20.012,00:16:23.683 let's say, a single bug? [pause] >> Uh, I dunno, the Stagefright 00:16:23.683,00:16:28.287 has been going for a long time. [chuckle] I think maybe one 00:16:28.287,00:16:31.691 year... [audience noise] But, uhm, yeah, it's not all at the 00:16:31.691,00:16:35.261 same time so... >> So, you know, none of the personality traits 00:16:35.261,00:16:38.397 that we're talking about is absolutely essential. Each of 00:16:38.397,00:16:41.834 you will find your own path. But if you feel that you have some 00:16:41.834,00:16:44.704 of these personality traits then you might find vulnerability 00:16:44.704,00:16:48.507 research enjoyable. >> So I think we're going to be totally 00:16:48.507,00:16:51.177 screwed on this slide, because it's small and all of them... 00:16:51.177,00:16:52.545 [laughter] >> So, we have a number of different... >> You 00:16:52.545,00:16:54.580 can probably read it fine. >> Yea. >> We can't read it at all. 00:16:54.580,00:16:56.949 >> We have a number of different skills that we sorta listed here 00:16:56.949,00:17:00.519 for long term success but I would say, probably that, uhm, 00:17:00.519,00:17:03.389 some of the, some of the biggest ones - one is about analysis 00:17:03.389,00:17:06.826 tools, and, and findings so... [audience noise] >> Not this 00:17:06.826,00:17:11.097 one, is it? >> Yip... We can skip it on the next one too. >> 00:17:11.097,00:17:13.633 Alright... I think on the big one we wanna say about 00:17:13.633,00:17:17.670 communication. I think we made that pretty clear. [laughter] >> 00:17:17.670,00:17:22.441 Uh, yea. [pause] >> Alright, so here's another awesome text 00:17:22.441,00:17:25.344 slide that we put together. And we don't wanna read it to you 00:17:25.344,00:17:27.813 but, uhm, these are some of the key terms that we feel on 00:17:27.813,00:17:30.283 vulnerability research of course the slides will be available. 00:17:30.283,00:17:33.853 Uhm, you know, if you, if you hear us, you probably already 00:17:33.853,00:17:37.189 heard us use, like, some of these terms. But when it really 00:17:37.189,00:17:39.992 comes into doing analysis and deeper research like, like some 00:17:39.992,00:17:43.596 of the stuff like root cause analysis, uh, and vulnerability 00:17:43.596,00:17:46.465 chains and classes and especially proof of concept 00:17:46.465,00:17:49.802 code, uh, become more important. >> Uh, I think, uh, that one of 00:17:49.802,00:17:53.673 the key terms here, uh, uhm, I guess we touch on it a little 00:17:53.673,00:17:57.376 bit later as well though is the notion of root cause analysis. 00:17:57.376,00:18:00.646 This is where diligence and critical thinking comes, comes 00:18:00.646,00:18:03.582 into play. You might discover something that's like symptom of 00:18:03.582,00:18:07.987 a problem. Uh, and, and it's really when you become tenacious 00:18:07.987,00:18:12.892 and dig deeper into it to find out what's really causing the 00:18:12.892,00:18:17.830 problem in the first place. Uh, where, uh, you may find, uh, 00:18:17.830,00:18:20.266 some significant success. [pause] >> Alright, so,uh, in 00:18:20.266,00:18:22.835 the industry many of you probably if you're interested in 00:18:22.835,00:18:25.237 vulnerability research already. [audience noise] Uh... know 00:18:25.237,00:18:28.541 about this thing we call a "Firehose" and that's basically 00:18:28.541,00:18:30.843 just a steady stream of information about 00:18:30.843,00:18:33.646 vulnerabilities that's coming from all angles. Uh, it includes 00:18:33.646,00:18:37.683 stuff like, some of my favourite stuff like CTFs and wargames. 00:18:37.683,00:18:40.319 [ahem] Where you can learn... [ahem] Excuse me... At your own 00:18:40.319,00:18:44.824 pace. Uh, uh, and just lots of aggregation and other places. If 00:18:44.824,00:18:47.593 you wanna learn more about vulns look at these things for sure. 00:18:47.593,00:18:50.696 >> There are a couple items that are not on that, uh, not on that 00:18:50.696,00:18:53.366 list there that I think came, came up during this week. And.. 00:18:53.366,00:18:55.868 >> Oh yea? >> Uh, one of them actually is the pwnie awards. 00:18:55.868,00:18:59.572 Because the pwnies awards often talk about, you know, individual 00:18:59.572,00:19:03.109 bugs and typically those additional bugs have additional 00:19:03.109,00:19:06.145 details. [audience noise] And then another area is uh, bug 00:19:06.145,00:19:09.682 bounty programs which, uh, can help you... >> Interact with a 00:19:09.682,00:19:12.818 lot of other stuff. >> And interact with other. >> Yea. >> 00:19:12.818,00:19:16.288 Yea. Uh, actually, by a show of hands how many people are in, or 00:19:16.288,00:19:19.025 have participated in bug-bounty programs? And have gotten some 00:19:19.025,00:19:23.095 type of reward? >> Why is... is.. >> Wow! >> Well this is 00:19:23.095,00:19:26.766 much better then how many people have seen these... [chuckle] 00:19:26.766,00:19:28.100 [laughter] [pause] >> Okay, moving on... >> But I guess 00:19:28.100,00:19:33.939 there is that rule about CB using websites and... >> So, 00:19:33.939,00:19:36.642 uhm... wow! Go over and stare really.... [laughter] So, 00:19:36.642,00:19:40.312 selecting your target, uh, there's a lot of choices if you 00:19:40.312,00:19:43.082 wanna find bugs somewhere and this is kind on the 00:19:43.082,00:19:47.420 vulnerability discovery side. Uh, you know, you can go deep or 00:19:47.420,00:19:50.356 you can go broad and what we mean by that is... You can pick 00:19:50.356,00:19:52.525 one particular type of vulnerability or something and 00:19:52.525,00:19:55.161 go look at every software you can find to see if it's 00:19:55.161,00:19:58.397 vulnerable or you could pick one particular software and just 00:19:58.397,00:20:01.434 drill down until you find something. [pause] >> There is a 00:20:01.434,00:20:04.770 lot of software that has more or less low-hanging fruit. And... 00:20:04.770,00:20:08.107 if you wanna expand on that a little? >> I will, I dunno if 00:20:08.107,00:20:12.111 it's on the slide or the next one. [chuckle] So, uh, hey, 00:20:12.111,00:20:14.780 another big point I wanted to make here is if you, if you do 00:20:14.780,00:20:16.816 do some vulnerability research and you find nothing. It's 00:20:16.816,00:20:19.385 actually quite useful for people to know that somebody actually 00:20:19.385,00:20:21.887 looked. Even if you found nothing. So, that, that's one 00:20:21.887,00:20:25.257 point. Uh, and then again, low-hanging fruit a lot of older 00:20:25.257,00:20:29.228 code is buggy, uh, complex or overly complex stuff, is, uh, 00:20:29.228,00:20:31.831 very interesting to look at. Although, you know, a lot of 00:20:31.831,00:20:34.233 times you just get lost and it just like the developer did... 00:20:36.502,00:20:38.904 [laughter] Uh, large attack surfaces like web browsers are 00:20:38.904,00:20:41.774 always fun to play with, you got a lot of possibility for things 00:20:41.774,00:20:46.712 to go wrong there. So software popularity matters, so if you're 00:20:46.712,00:20:49.648 gonna try and become super famous and you wanna go find 00:20:49.648,00:20:51.851 some vuln in something it's probably better to not pick 00:20:51.851,00:20:56.255 some, uh, random, uh, personal website project off of Source 00:20:56.255,00:21:00.726 Forge or something like that. Uh, but on the other hand if you 00:21:00.726,00:21:03.295 wanna find something in, you know, a super popular product 00:21:03.295,00:21:06.799 like Microsoft Windows server it's probably gonna not be 00:21:06.799,00:21:10.269 anywhere near as easy. [audience noise] >> Not saying that 00:21:10.269,00:21:13.873 anywhere is easy because that software, if the really popular 00:21:13.873,00:21:17.176 software is already been pounded on and pounded on by many 00:21:17.176,00:21:19.979 people, by elite researchers and so one. And so, the 00:21:19.979,00:21:22.481 lower-hanging fruit, the kind of software that doesn't 00:21:22.481,00:21:27.319 necessarily have any vulnerability history at all, 00:21:27.319,00:21:30.489 uh, or that no one's really looked at before, uh, that's 00:21:30.489,00:21:33.959 often in your area where you can find some success, uh, fairly 00:21:33.959,00:21:37.830 quickly. >> Yea, I dunno if, uh, if it, you know, the one thing 00:21:37.830,00:21:40.900 that I like to do sometimes when I get super stuck, uh, is to go 00:21:40.900,00:21:44.336 and pick on somebody lame... Uh, I think this is kind of popular 00:21:44.336,00:21:47.072 in the VR industry where we just need that redemption.. [audience 00:21:47.072,00:21:49.675 noise] Where we feel good about ourselves again... [laughter] 00:21:49.675,00:21:52.311 Uh, but the problem with that is, you know, like, uh, a good 00:21:52.311,00:21:54.680 example of that is OpenOffice or something, it's, it's pretty 00:21:54.680,00:21:57.249 easy to fuzz that and it's full of bugs and nobody really cares 00:21:57.249,00:22:00.986 about them too much. [laughter] And so, uhm, you can go find 00:22:00.986,00:22:03.556 bugs there but then you deal with the secondary problem cause 00:22:03.556,00:22:09.361 nobody's caring too much, so... [chatter] Uh... So brand new 00:22:09.361,00:22:12.798 emerging technologies is always a great place to look, uh, many 00:22:12.798,00:22:16.335 people in vuln research like to wait until a thing becomes very 00:22:16.335,00:22:19.438 popular and therefore when things are emerging nobody's 00:22:19.438,00:22:22.141 really paying attention. Uhm, I think we can say that about 00:22:22.141,00:22:25.611 IPB6, I think there's maybe a handful of IPB6 researchers 00:22:25.611,00:22:29.582 around, uh, even though that's sort of slowly becoming a norm. 00:22:29.582,00:22:33.819 Let's see, Mobile or iOT are definitely guilty of this 00:22:33.819,00:22:37.957 because, as they try to hurry up and get to market really fast 00:22:37.957,00:22:41.193 they didn't invest in the security and well, we're hoping 00:22:41.193,00:22:45.364 that we don't repeat that mistake with iOT, but we'll see. 00:22:45.364,00:22:48.567 [pause] >> One suggestion we do have which would be, uhm very 00:22:48.567,00:22:51.770 useful for the, uh, for the entire community and for 00:22:51.770,00:22:54.974 contributing to the body of knowledge is that you have 00:22:54.974,00:22:58.510 access to software or products that are, uh, very difficult for 00:22:58.510,00:23:02.181 the everyday person to get access to. Say, you know, 00:23:02.181,00:23:06.652 multi-million dollar enterprise software, or, uh, expensive 00:23:06.652,00:23:10.789 medical devices or other kind of physical devices, uhm, you know, 00:23:10.789,00:23:13.826 those aren't things that just everybody can go and grab and 00:23:13.826,00:23:15.194 look at. So, not only might you have some good chances of uh, 00:23:15.194,00:23:16.528 success in finding vulnerabilities in those kind of 00:23:16.528,00:23:17.863 products. Not a lot of people have access. [pause] >> Who 00:23:17.863,00:23:19.531 knows how to do that, like, [coughing] magnifying glass 00:23:19.531,00:23:24.536 thing on OSX? Anyone? [pause] Nobody? You want coffee or 00:23:30.109,00:23:34.546 something? [pause] Well I'll just stare at it really small 00:23:34.546,00:23:38.817 again. So we got that oooooon... [pause] [coughing] So, uh, uh, 00:23:38.817,00:23:42.922 some of it, and I've seen a lot, and, and Steve kinda coined the 00:23:42.922,00:23:46.859 term and it's "Pigpile effect". It's pretty interesting, that's 00:23:46.859,00:23:50.095 when we select a target, you see, uh people beating up on 00:23:50.095,00:23:52.197 something, three advisors getting published and you're 00:23:52.197,00:23:54.233 like "Well, hey, maybe I feel like there's, there might be 00:23:54.233,00:23:56.735 more there, I should maybe go take a look there, maybe do some 00:23:56.735,00:24:00.139 pile on work." Uh, I encourage the community to put this one on 00:24:00.139,00:24:02.408 Stagefright heavily. [coughing] I think it's good to have more 00:24:02.408,00:24:07.713 people looking. [sniff] So, tools and techniques, uhm, 00:24:07.713,00:24:10.816 there's kind of, like a, these two main one which are really 00:24:10.816,00:24:14.953 kinda high-level: design review, uh, and threat modelling. These 00:24:14.953,00:24:18.490 are, I think, really important for anyone who's developing 00:24:18.490,00:24:22.828 software, uh, to have this as part of like the cycle of 00:24:22.828,00:24:27.066 figuring out how to, how to, to stay secure, or how to basically 00:24:27.066,00:24:31.236 stop having alarm bells ringing all the time. Uh, dynamic or 00:24:31.236,00:24:34.406 static analysis is, is very important to differentiate, 00:24:34.406,00:24:37.309 depending on what kind of stuff you're gonna do. Uhm, like, on 00:24:37.309,00:24:40.346 the malware side static analysis software is a lot more popular. 00:24:40.346,00:24:42.348 With vulnerability research allows a lot more dynamic 00:24:42.348,00:24:45.284 analyses seems to be more popular. But I think the real 00:24:45.284,00:24:48.987 power here is when you have both together. One, one of my 00:24:48.987,00:24:52.891 personal bug-hunting processes is to, to start writing a fuzzer 00:24:52.891,00:24:55.561 and just let it run while I read the code and as soon as I learnt 00:24:55.561,00:24:57.529 something more about the code that will help the fuzzer be 00:24:57.529,00:24:59.798 good and I'll add it to the fuzzer. [coughing] I just keep 00:24:59.798,00:25:05.237 doing that back and forth. [pause] Uh, so code auditing and 00:25:05.237,00:25:08.907 some of these other automated tools, like, uh static code 00:25:08.907,00:25:11.877 analysis, analyzers, they're great but a lot of times they 00:25:11.877,00:25:16.482 have false positives or they other issues, and so, uh, it's 00:25:16.482,00:25:19.518 just important to be aware of the tradeoffs of kind of all of 00:25:19.518,00:25:22.588 the tools and techniques when you start getting into them. I, 00:25:22.588,00:25:26.525 I really think that a tool in this industry is the embodiment 00:25:26.525,00:25:30.195 of the technique that someone developed to a large degree. 00:25:30.195,00:25:32.898 [coughing] >> Yea, and I agree with that and... [ahem] While we 00:25:32.898,00:25:36.068 have a number of tools and techniques listed here, uhm, you 00:25:36.068,00:25:38.704 know, that doesn't mean that you have to know all of them and be 00:25:38.704,00:25:41.707 an expert in all of them in order to find any kind of 00:25:41.707,00:25:45.444 success this is part of your path, uh, but we do recommend, 00:25:45.444,00:25:48.580 as I say, to investigators to at least investigate and, uh, look 00:25:48.580,00:25:51.650 into each of these. Everyone's kinda has their own sort of 00:25:51.650,00:25:57.656 favorite techniques that they like to do. [pause] >> This 00:25:57.656,00:26:02.761 one's you man... >> So as the field becomes, uh, a little bit 00:26:02.761,00:26:07.800 more mature and, uh, ties in obviously with vulnerability 00:26:07.800,00:26:11.136 management over all. There's a number of relevant standards 00:26:11.136,00:26:13.972 that you should familiarize yourself with and uh, and 00:26:13.972,00:26:18.811 utilize wherever you can. Uh, one of the main ones is the 00:26:18.811,00:26:21.213 common ident... uh, identification scheme for 00:26:21.213,00:26:24.850 vulnerability is called CVE and, uhm, and for those of you 00:26:24.850,00:26:29.621 who've, uh, uh, who've had certain questions about CVE 00:26:29.621,00:26:33.325 especially, uh, in the last year or so with concerns about 00:26:33.325,00:26:38.130 coverage and what MITRE is doing. While I did leave CVE, 00:26:38.130,00:26:42.100 uhm, last year, I am still at MITRE and we do have one of my 00:26:42.100,00:26:46.405 colleagues here who, uh, uh, carrying the torch as it were 00:26:46.405,00:26:48.040 and wanted to talk to you. And so... [audience noise] I wanted 00:26:48.040,00:26:53.846 him to stand up here. That's Stan... [applause] He will be... 00:26:53.846,00:26:59.084 [applause] >> Hey Stan... >> Hey. >> We need to talk, buddy. 00:26:59.084,00:27:01.753 [pause] So, uh, he will be available and he would, uh, he 00:27:01.753,00:27:04.957 would love to, love to talk to you. Uhm, not all of you at 00:27:04.957,00:27:09.895 once, but, uh, you know... A few at a, a few at a time. [audience 00:27:09.895,00:27:12.731 noise] Uh... [chuckle] Another effort is the common weakness in 00:27:12.731,00:27:17.169 remuneration. And this, uh, uh, uh, when you have these 00:27:17.169,00:27:19.571 different vulnerabilities in different products. Well it 00:27:19.571,00:27:23.242 turns out that programmers make the same mistakes and many 00:27:23.242,00:27:27.079 different programmers make the same mistakes. And, so, CWE is 00:27:27.079,00:27:31.717 essentially, effectively a classification scheme for, uhm, 00:27:31.717,00:27:36.021 how programmers wind up making these kind of mistakes. Uh, 00:27:36.021,00:27:38.590 it's, uh, useful in two different ways - one as sort of 00:27:38.590,00:27:42.861 a common identifier for, uhm, characterizing what the mistake 00:27:42.861,00:27:47.466 is that you found but it also winds up being very useful as a 00:27:47.466,00:27:52.104 dictionary or as, uh, something to educate you. So for example 00:27:52.104,00:27:56.041 CWE covers 800 different, 800 different kinds of mistakes that 00:27:56.041,00:28:00.078 programmers can make and as much as you think you may know about 00:28:00.078,00:28:03.215 everything, I guarantee you that there's one or two things in 00:28:03.215,00:28:05.450 there that might surprise you or that you might not have 00:28:05.450,00:28:09.922 expected. And if you are just starting out you get good, you 00:28:09.922,00:28:14.192 get good information from things such as OWASP but CWE is as well 00:28:14.192,00:28:18.530 for stuff like SQL injection and cross-site scripting is also 00:28:18.530,00:28:25.470 pretty mature. Uh, equivalent for uhm a task is called CAPEC. 00:28:25.470,00:28:30.509 Uh, and then CVSS is a way of being able to consistently 00:28:30.509,00:28:34.580 applies a risk-related score to a particular vulnerability that 00:28:34.580,00:28:37.916 you've found. So, it may be your favorite vulnerability, you 00:28:37.916,00:28:41.153 might be in love with it, you might have worked really hard uh 00:28:41.153,00:28:45.891 but you need the cold objective, reasonably objective eye of CVSS 00:28:45.891,00:28:49.628 or something like that so you can communicate its importance, 00:28:49.628,00:28:55.100 uh, uh, effectively. >> Thank youuuu. Hey sorry about messing 00:28:55.100,00:28:56.969 with the slides, I was trying to zoom in on this little tiny 00:28:56.969,00:28:59.938 thing and it zoomed everything... So that doesn't 00:28:59.938,00:29:03.909 work. [pause] >> Uh, so, uh, disclosure models. Disclosure, 00:29:03.909,00:29:07.212 disclosure, disclosure... There, we're not gonna go into specific 00:29:07.212,00:29:09.948 details but there a number of different models, that, uh, you 00:29:09.948,00:29:13.852 can consider and more or less figure out more or less what 00:29:13.852,00:29:17.422 works for you. Uhm, I do and I think Josh agrees that we both 00:29:17.422,00:29:19.758 suggest using the coordinated disclosure model, which involves 00:29:19.758,00:29:23.762 really, uh, working with the, the, the vendor in order to try 00:29:23.762,00:29:26.898 and reach some resolution but there are other models as well 00:29:26.898,00:29:30.369 such as full disclosure. As soon as you find it you sort of put 00:29:30.369,00:29:33.171 it out independent of whether or not the vendor's been given a 00:29:33.171,00:29:36.008 chance to patch. And then there's also non-disclosures 00:29:36.008,00:29:38.143 some people may simply choose not to disclose the 00:29:38.143,00:29:40.679 vulnerabilities or, uh, to only, uh, provide them or, uh, in some 00:29:40.679,00:29:43.882 cases sell them, in uh, limited markets. But this is, this, 00:29:43.882,00:29:48.286 these are different things that you're gonna need to consider as 00:29:48.286,00:29:52.791 you move more into vulnerability research. You may have any 00:29:52.791,00:29:56.828 number of different, uhm, approaches and believes in, in 00:29:56.828,00:30:02.000 why it's important to, uh public disclosure. Uh, but I think the 00:30:02.000,00:30:07.005 more we know the better we know all of us collectively. And 00:30:07.005,00:30:10.175 finally there are a couple different standards or 00:30:10.175,00:30:13.812 standards-like documents that, uhm, will give you some guidance 00:30:13.812,00:30:16.148 in respect to coordinated disclosure or equivalent models, 00:30:16.148,00:30:19.384 uh, that you can follow or provide advice to vendors who 00:30:19.384,00:30:22.354 may not be used to handling vulnerabilities. Most 00:30:22.354,00:30:27.359 importantly is international standards organisation - ISO, 00:30:31.663,00:30:35.400 uh, document number 2 9 1 4 7 which was done by, uh, Katie 00:30:35.400,00:30:38.970 Moussouris and others. International standard - it is 00:30:38.970,00:30:42.708 something that is directed towards vendors which explains 00:30:42.708,00:30:47.779 to them how to, uh, build up a process for responding to 00:30:47.779,00:30:51.316 vulnerability reports and for interacting with, uh, the 00:30:51.316,00:30:55.987 researchers. So, uhm, as a survivor of the disclosure wars, 00:30:55.987,00:31:00.692 uh, I'm very very happy to see standards like that 2 9 1 4 7 to 00:31:00.692,00:31:03.862 come out. And yes, it did take me 6 months before I could start 00:31:03.862,00:31:10.268 rattling that number. [laughter] [pause] >> If you start to get 00:31:10.268,00:31:15.240 deeper into, uhm, uh, building your vulnerability career so to 00:31:15.240,00:31:17.476 speak then you may have different kind of, uh, 00:31:17.476,00:31:21.246 considerations for building your own kind of public, your own 00:31:21.246,00:31:26.585 disclosure policy. Based on your own experiences and your own 00:31:26.585,00:31:29.755 believes you, you, want to start evolving certain kind of 00:31:29.755,00:31:33.125 considerations for what you're gonna do in certain kinds of 00:31:33.125,00:31:36.661 circumstances. So, what you think, what would you do if you 00:31:36.661,00:31:39.664 try and contact a vendor and you can find any contact 00:31:39.664,00:31:43.535 information? Or what happens if you're working through a process 00:31:43.535,00:31:46.538 and then suddenly the, uh, vulnerability is released by 00:31:46.538,00:31:50.809 somebody else as "zero day" or something like that. [audience 00:31:50.809,00:31:53.812 noise] There's a lot of debate about, uh, how long do I 00:31:53.812,00:31:57.182 actually give the vendor before they fix the vulnerability and 00:31:57.182,00:32:00.886 push it out, uh, push out a patch? Some say 30 days, some 00:32:00.886,00:32:06.158 say 60 days, there's 90 days or however long it takes. These are 00:32:06.158,00:32:11.196 some of the questions you need to ask yourself. [pause] Yea... 00:32:11.196,00:32:15.667 [pause] What... >> That's a... >> I had a point. I forgot what 00:32:15.667,00:32:18.370 it was though. >> Let's skip this one. >> Oh, I think, I 00:32:18.370,00:32:21.640 think I was just gonna say that sometimes the disclosure process 00:32:21.640,00:32:25.577 you choose will even vary do, you know, based on the 00:32:25.577,00:32:29.447 individual vulnerabilities. Some people decide not to disclose 00:32:29.447,00:32:34.452 things that are not super awesome. [laughter] [pause] >> 00:32:34.452,00:32:37.622 Alright, so, uh, we got 10 minutes, we lost a little to the 00:32:37.622,00:32:40.792 technical... >> Yea... >> Difficulties, so, let's... We're 00:32:40.792,00:32:44.462 gonna move on, I think. >> Skip down. >> Yup.. alright so let's 00:32:44.462,00:32:46.731 talk a little bit about advisory structure and content. I'm 00:32:46.731,00:32:49.067 totally not gonna read these bullets to you. [laughter] But, 00:32:49.067,00:32:53.772 like, uhm, structured content is, is very useful. I think 00:32:53.772,00:32:56.141 Steve and I collectively, well we probably read a lot of the 00:32:56.141,00:32:59.077 same advisories but collectively probably like over thousands and 00:32:59.077,00:33:01.847 thousands and thousands... >> I agree with that. >> And, uh, and 00:33:01.847,00:33:04.216 it's like, some of them were really horrible. There's this 00:33:04.216,00:33:07.018 really offensive group, and, and when I say offensive I mean when 00:33:07.018,00:33:09.988 we read it we get offended. [laughter] I don't mean they use 00:33:09.988,00:33:12.524 bad words or anything... >> Yea, not to name names, but... >> 00:33:12.524,00:33:15.894 Yea, I'm not naming any names. You, if anybody reads advisories 00:33:15.894,00:33:18.496 they'll figure it out pretty soon. [laughter] So these are 00:33:18.496,00:33:23.235 just some fields and some guidance that we have for making 00:33:23.235,00:33:26.905 advisories and of course there's some more here... Uh, one of the 00:33:26.905,00:33:31.877 big ones is, uhm, proof concept code, uh, I think it's a really 00:33:31.877,00:33:37.349 important... uh, I, really important sort of thing to prove 00:33:37.349,00:33:40.352 your case. Uhm, you know, when you, when you disclose a 00:33:40.352,00:33:42.821 vulnerability to a vendor a lot of times you get pushed back 00:33:42.821,00:33:45.991 like there's not even a real issue here. Uh, and, uh, and of 00:33:45.991,00:33:50.595 course one the side of things it is a bit hard to argue with a 00:33:50.595,00:33:53.365 shell. But it... uh, you don't necessarily have to give anybody 00:33:53.365,00:33:56.368 a shell, uh, as your proof of concept. It could be, could be 00:33:56.368,00:33:58.870 whatever you choose. It could be a sequence of steps that they 00:33:58.870,00:34:03.608 managed to follow, to, verify it. It could be, uhm, any kind 00:34:03.608,00:34:06.878 of level of maturity, proof of concepts code, crash, crash 00:34:06.878,00:34:11.316 proof concepts. But do, do remember like the more, sort of, 00:34:11.316,00:34:14.386 detailed information you can learn and extract and provide to 00:34:14.386,00:34:17.856 them the easier it's gonna be for them to deal with that 00:34:17.856,00:34:21.493 information. So one of the reasons we have these particular 00:34:21.493,00:34:25.163 details here about advisory contents is that a lot of 00:34:25.163,00:34:27.999 researches, especially beginning researchers don't initially know 00:34:27.999,00:34:31.937 what information to provide. Or, he might submit a bug report to 00:34:31.937,00:34:34.072 a bug bounty and it comes back and says that you're not 00:34:34.072,00:34:37.075 providing enough information or you're not communicating 00:34:37.075,00:34:40.011 clearly. [laughter] So, uh, those fields that we have listed 00:34:40.011,00:34:43.048 in, you know, are on the slides and will, you know, will be an 00:34:43.048,00:34:46.851 updated version of this slide. Uh, we encourage you to really 00:34:46.851,00:34:50.722 look at those and consider seriously capturing all of that 00:34:50.722,00:34:53.325 information. >> There's some pros and cons that we came up 00:34:53.325,00:34:57.629 with. Uh, just basically, just Steve and I ranting on all the 00:34:57.629,00:35:01.933 stuff we didn't like about various advisories. Uh, you know 00:35:01.933,00:35:04.936 we want people to do simpler, you know, simple stuff, plain 00:35:04.936,00:35:08.206 text is real easy. It's very portable. It has a very low 00:35:08.206,00:35:14.012 attack surface... so... >> Ass opposed to, let's say, pdf. >> 00:35:14.012,00:35:18.984 Yea which is like a web browser basically. >> And, uh, some 00:35:18.984,00:35:22.887 people do like to do videos and, uh, that's kinda cool but I have 00:35:22.887,00:35:25.590 a couple suggestions here. One of the main ones being... is, 00:35:25.590,00:35:29.394 you know, respect the viewer of the videos, you don't wanna make 00:35:29.394,00:35:32.797 your videos too long, uhm, but you don't wanna go too quickly 00:35:32.797,00:35:37.202 either. Uhm and there's a couple other considerations up there as 00:35:37.202,00:35:40.739 well. So you wanna be mindful even of the formatting your 00:35:40.739,00:35:45.877 advisory goes out. [pause] >> This one me? >> Yea. >> So, so, 00:35:45.877,00:35:48.613 what to expect from vendors. I already mentioned some of the 00:35:48.613,00:35:53.351 stuff. You can expect total cluelessness, uh, can u, expect 00:35:53.351,00:35:56.087 in some cases, for people to threaten you with their legal 00:35:56.087,00:35:59.624 teams, uh, I don't know why they do this, I think they're 00:35:59.624,00:36:03.995 confused. But, uh, you know, these are just a bunch of, a 00:36:03.995,00:36:07.365 bunch of possibilities. >> Basically every... >> Most good 00:36:07.365,00:36:10.535 companies these days, especially the bigger ones, they're very 00:36:10.535,00:36:13.705 open to work with and actually, amazingly, even sometimes a new 00:36:13.705,00:36:16.841 vendor that has never had to deal with these problems comes 00:36:16.841,00:36:20.979 out actually understanding quite easily and being very well, good 00:36:20.979,00:36:23.748 to work with as well. >> But one thing to keep in mind is that 00:36:23.748,00:36:26.384 it's not like one size fits all all the time. And every 00:36:26.384,00:36:29.320 disclosure winds up being some kind of unique snowflake. 00:36:29.320,00:36:32.390 [audience noise] So you need to be patient as mentioned before 00:36:32.390,00:36:38.229 and also be able to be flexible. >> Keep an open mind. Uh, so... 00:36:38.229,00:36:40.532 [ahem] I'm the first, I think the first bullet, like one time 00:36:40.532,00:36:45.170 I had fence we were trying to report as "vuln" and uh, i tried 00:36:45.170,00:36:48.907 vuln calls, I tried email - well in the other order, of course, I 00:36:48.907,00:36:51.943 tried emails and then phone calls. And finally I got a 00:36:51.943,00:36:55.413 response from this security guy there when I faxed them the 00:36:55.413,00:37:00.919 advisory. [laughter] Yea,I guess... >> What year was that? 00:37:00.919,00:37:05.156 >> Uuuuuh, it was probably 2007. >> Yea? >> Yea... [laughter] So 00:37:05.156,00:37:09.594 they, they were like "Oh, this came out of the fax machine, 00:37:09.594,00:37:16.034 aaaah!". [laughter] "Let's call 'em!". Okay... [ahem] So, when 00:37:16.034,00:37:20.171 do we disclose publically? What we like to see is for people to 00:37:20.171,00:37:24.008 disclose publicly in places that are archived forever so that 00:37:24.008,00:37:26.344 becomes mailing lists, basically. [audience noise] And 00:37:26.344,00:37:29.681 these, these other things on the list. Like exploit db is in vuln 00:37:29.681,00:37:33.485 databases, those, those sites are great and uh we hope they'll 00:37:33.485,00:37:36.421 live forever but... inca, some cases in the past they have not 00:37:36.421,00:37:40.258 live forever. And ul, ultimately those sites generally are 00:37:40.258,00:37:43.027 pulling from more public sites that are archived forever 00:37:43.027,00:37:46.131 anyway. So, this , this just our preference, if you wanna, you 00:37:46.131,00:37:50.301 know, put your stuff on your blog, uh, to, for it to gain 00:37:50.301,00:37:53.438 readership, that's great. But maybe also throw a note up on 00:37:53.438,00:37:57.709 one of these lists to get traffic to your blog as well. 00:37:57.709,00:38:02.847 [pause] So, common mistakes to avoid, you know, number one is 00:38:02.847,00:38:06.718 don't test other people's stuff unless they let you. [laughter] 00:38:06.718,00:38:10.188 Uuuuh, I think there was one case with Facebook bug-running 00:38:10.188,00:38:13.858 program where a guy, like, owned the hell out of them, basically, 00:38:13.858,00:38:16.928 and then tried to do a bug bounty with them. I don't think 00:38:16.928,00:38:20.999 that worked out really well for the guy. [chuckle] >> Uh, we 00:38:20.999,00:38:23.601 have here on this slide in the next one, we aren't gonna go 00:38:23.601,00:38:25.737 into any details. Especially because we're running low on 00:38:25.737,00:38:29.574 time. But there are a lot of common mistakes, uhm, that many 00:38:29.574,00:38:33.878 researchers make, including ourselves actually. And, uh, why 00:38:33.878,00:38:35.880 don't we move ahead... >> Yes, absolutely. If you guys wanna 00:38:35.880,00:38:38.917 hear some war stories afterwards just hit us up afterwards. 00:38:38.917,00:38:42.954 [ahem] >> We have some real neat to the, uh, to the tail-end of 00:38:42.954,00:38:46.491 this presentation. >> Yup. >> Okay, so this is one of the, one 00:38:46.491,00:38:50.028 of the main ones here. This is just sort of our own model and 00:38:50.028,00:38:53.164 our first crack at this. As far as we know nobody else has sort 00:38:53.164,00:38:57.202 of started on this but, uhm, we're trying to outline 00:38:57.202,00:39:00.805 different, uh, kinds of stages of growth that you might 00:39:00.805,00:39:04.609 encounter in your career or in your technical abilities when 00:39:04.609,00:39:07.412 you're doing vulnerability research. So when you're just 00:39:07.412,00:39:09.948 starting out, you're starting at, uh, more or less a newbie 00:39:09.948,00:39:13.618 phase you might have, you might only know one crude technique 00:39:13.618,00:39:17.889 that you might apply against, uhm, you know, easy software and 00:39:17.889,00:39:22.126 you make a lot of simple mistakes. At some point once you 00:39:22.126,00:39:24.896 get, you become more familiar with things you reach what we 00:39:24.896,00:39:28.866 call the workhorse stage where you know a number of different 00:39:28.866,00:39:32.136 kinds of basic vulnerability types, you can generally find 00:39:32.136,00:39:36.307 multiple issues, uh and the you start to more or less get a hang 00:39:36.307,00:39:39.577 of, uh, certain kinds of processes. Then when you start 00:39:39.577,00:39:43.681 to move more towards the subject matter expert. These are the, 00:39:43.681,00:39:46.284 these are the times when you really like watching and looking 00:39:46.284,00:39:49.587 for the newest and latest, uh, techniques that other people 00:39:49.587,00:39:52.590 develop or you might, uh, you might go and extend those 00:39:52.590,00:39:56.194 techniques that have already been reported. And, uhm, you're 00:39:56.194,00:40:00.999 more or less pretty much, uhm, treated or assumed as reliable 00:40:00.999,00:40:07.305 by, you know, uhm people like me and someone, uh, and, uhm, at, 00:40:07.305,00:40:10.575 at some point you really start to have a clear sense on, like, 00:40:10.575,00:40:15.179 what your own disclosure policy is and you can relied on to find 00:40:15.179,00:40:18.583 a lot of things and to really write a good solid quality 00:40:18.583,00:40:23.321 report. And finally upon reaching the elite stage which, 00:40:23.321,00:40:26.791 not everyone needs to reach the elite stage and not everyone has 00:40:26.791,00:40:30.061 to and not everyone wants to and that's perfectly cool cause 00:40:30.061,00:40:32.497 there are way more vulnerabilities out there than 00:40:32.497,00:40:36.234 there are vulnerability research to handle them. But, uh, what I 00:40:36.234,00:40:41.306 think of a little bit of as an elite researcher is you discover 00:40:41.306,00:40:45.677 or invent new vulnerability classes, or you develop entirely 00:40:45.677,00:40:51.649 new techniques. Uhm, or, uh, uh, you know, you give conference 00:40:51.649,00:40:57.121 and so on. You really sort of, uh, uh, uhm push the industry 00:40:57.121,00:41:00.458 forward and that can take a number of years. You're not 00:41:00.458,00:41:03.828 gonna just read a book or look at a couple of blog posts and be 00:41:03.828,00:41:08.800 elite tomorrow... uhm, or next year. So that's something to 00:41:08.800,00:41:13.004 keep in mind as well. >> Yea it takes time for sure. I... 00:41:13.004,00:41:18.242 [pause] I can't read this again. >> So let's get out my name on 00:41:18.242,00:41:22.280 it anyways. >> Yea okay. >> You're free. [laughter] >> You 00:41:22.280,00:41:25.917 got only a minute or something like that. >> Yup, so one thing 00:41:25.917,00:41:28.619 about, about the notion of growth. There is a book by 00:41:28.619,00:41:32.223 Malcolm Gladwell, "Outliers", which basically says that it 00:41:32.223,00:41:36.127 takes about 10-thousand hours of, uh, focussed, effective 00:41:36.127,00:41:39.964 practice to reach a level of expertise and so you can do the 00:41:39.964,00:41:42.133 math and that number may be questionable but that's 00:41:42.133,00:41:45.503 something, uh, really to keep in mind. But there are a couple of 00:41:45.503,00:41:48.406 different ways you could progress a little bit further if 00:41:48.406,00:41:52.410 you want. >> I think we have like three slides left. >> Yup. 00:41:52.410,00:41:53.845 >> You wanna do this one? Or you want me to go ahead? >> Yea, 00:41:53.845,00:41:56.914 yea. >> It's got your name on it. >> But.... [laughter] Ah... 00:41:56.914,00:42:00.251 >> I can do it if you want. Getting tired? >> No, we're 00:42:00.251,00:42:03.087 getting, we're getting an "X" from the goons so... >> "Get out 00:42:03.087,00:42:08.993 of here, guys, you talk too much!" >> So, uh, we just wanted 00:42:08.993,00:42:11.529 to leave on this note on feelings and fails, right? We 00:42:11.529,00:42:14.399 mentioned that we are not perfect, uh, I believe that, 00:42:14.399,00:42:18.736 this is what I call the human condition, which basically means 00:42:18.736,00:42:22.006 you always make mistakes and have to deal with things you, 00:42:22.006,00:42:26.677 uh, your body tells you and such. But, uh, feelings are 00:42:26.677,00:42:29.847 definitely a part of that so... feelings are, remember that 00:42:29.847,00:42:32.483 feelings are okay, uh, there are a number of times when you're 00:42:32.483,00:42:35.553 doing some deep research and you're getting discouraged, you 00:42:35.553,00:42:38.489 might wanna find something easier to do. you may want to, 00:42:38.489,00:42:41.626 uh, go at it a different way and maybe just go to the beach for a 00:42:41.626,00:42:45.229 while. And, so, when other, another one of these things is 00:42:45.229,00:42:48.566 you feel like you wanna keep going and work really hard cause 00:42:48.566,00:42:51.936 you're addicted to something but it's been like 17 hours that 00:42:51.936,00:42:53.938 you've been working on it, it might be a good time to sleep. 00:42:56.741,00:42:59.143 Uhm, yea. [background noise] Feelings are okay. [applause] 00:42:59.143,00:43:03.981 Failures are okay too. [applause] >> There we go. Thank 00:43:03.981,00:43:08.419 you... >> Thank you everyone! [applause] And we are available 00:43:08.419,00:43:11.522 to talk to anybody afterwards down at the... >> Yea, we're 00:43:11.522,00:43:16.694 gonna go down, down the escalator, somewhere there... >> 00:43:16.694,00:43:17.161 If everyone could please leave the...!