00:00:00.000,00:00:06.573 >>Get started >>Everybody's like okay this is cool, but not that cool. Apart from John who's 00:00:06.573,00:00:11.578 totally cool. Okay should we start? >>Let's start! >>Alright >>you're up! >>So I'm gonna boot 00:00:13.914,00:00:18.919 this off because I I sort of started Shellphish but I'm sort of reaping the um the benefit of 00:00:24.691,00:00:29.529 it without really doing anything. These guys are actually the brains behind it 00:00:29.529,00:00:35.135 and the guys who stayed up all night doing all the work. I'm just looking at them thinking oh 00:00:35.135,00:00:42.042 my god! I remember when I did that twenty five years ago! >>Giovanni did a lot of high 00:00:42.042,00:00:49.016 level planning and sushi delivery >>Exactly, that's my role, feed them, they will poop 00:00:49.016,00:00:52.819 software [laughter] okay. Cyber grand challenge- >>If you look at our code, that's really 00:00:52.819,00:00:59.159 actually true >>That's actually true so eh eh I I'm going to be very short on this eh Shellphish 00:00:59.159,00:01:04.097 was born out of the sec lab which is the security group at UC Santa Barbara, everytime you 00:01:06.166,00:01:11.471 say UC people say Universe of California, that's not right, that's Berkeley, UC Santa 00:01:11.471,00:01:18.345 Monica, that does not exist, it's UC Santa Barbara, so get it right, Sec lab is the group, 00:01:18.345,00:01:25.118 that's where we come from. And uh the group is led currently by me and my um uh colleague 00:01:25.118,00:01:32.025 Christopher Kruegel uh we look very professional here like professors but we're actually- 00:01:32.025,00:01:37.464 >>It's basically >>Hackers behind weird handles like everybody else, I, I never got 00:01:37.464,00:01:44.371 the handle thing, but I, I needed one and so if you look about Zanardi on the on the 00:01:44.371,00:01:49.376 internet it is somebody with a gigantic nose and a ponytail which I once had >>Giovanni 00:01:49.376,00:01:54.648 would you say, Chris is your life partner? >>I think Chris ma- ke- Chris ma- [chuckle] 00:01:54.648,00:01:59.653 Christopher is my academic wife so I I have to take care of all his needs and his uh I wish he 00:02:03.390,00:02:08.862 would be here it would be very he's super proud of everybody but this is our University, not 00:02:08.862,00:02:13.900 bad and that's why Shellphish is here our lab is exactly there where the arrow points, we're 00:02:16.603,00:02:21.575 right on the beach, we have a private beach, and that's why our tagline is HEX on the beach. 00:02:21.575,00:02:28.015 Um uh we're lucky that >>Might be back here? Is it back here? >>Yes it is it is >>Hah right! 00:02:28.015,00:02:33.020 >So how this started, it started in 2004, I know it's incredibly such a long time ago uh it's me 00:02:37.357,00:02:42.362 but then I had a bunch of um grad students including Chris uh and we evolved into uh a 00:02:46.033,00:02:51.671 community and in 2005 we actually won Defcon CTF, never once since then that was the 00:02:51.671,00:02:56.943 good old days and it's all it's awesome because they say the older you get the more awesome 00:02:56.943,00:03:01.882 you were so so I'm milking it for whatever I can but we grew up you know and then suddenly 00:03:04.217,00:03:09.222 void, Chris moved to Vienna, became a professor there, recruited some more people, that 00:03:11.291,00:03:17.164 became more people, that came back to Santa Barbara because it's awesome, became more 00:03:17.164,00:03:23.370 people, more students, more students, even more students and what happened is that some 00:03:23.370,00:03:28.375 people went to Boston so we have a substantial presence in Boston and we evolved as a group more 00:03:31.011,00:03:37.851 and more in the years until at a certain point all our grad- graduate students actually 00:03:37.851,00:03:42.856 became professors as well and so a lot of U- you know UC people became professor all around the 00:03:47.160,00:03:52.165 world, in London, at Arizona State University, Eurocom in France, and right now Shellphish 00:03:54.367,00:03:59.372 is a very big group of all academic people all around the world doing interesting stuff so 00:04:02.142,00:04:07.948 right now our group is pretty much this we're very inclusive, we're you know, we foster 00:04:07.948,00:04:12.953 research and that's what we care about and with this I'll give my presentation baton to Yan 00:04:18.258,00:04:25.065 >>Thank you Giovanni. So before we go on with uh the Cyber Grand challenge itself, I'd like to 00:04:25.065,00:04:30.070 give a shout out to uh all the other Shellphish-es in the audience. So raise your hand, if 00:04:33.406,00:04:38.411 your Shellphish, oh yeah right there! [applause] nice! Nice! [whistle] yeah Shellphish is uh 00:04:41.882,00:04:48.255 bigger than just the CGC Team, the CGC Team is a straight subset but we have a lot of uh 00:04:48.255,00:04:53.460 people that were cheering us from the sidelines even on the team. So let's talk about the 00:04:53.460,00:05:00.300 Cyber Grand Challenge um DARPA has a history of Grand Challenges right? You guys are 00:05:00.300,00:05:05.071 probably familiar with the self driving car grand challenge, and the robotics grand challenge 00:05:05.071,00:05:10.410 because uh they got a lot of press similar to the cyber grand challenge just now and uh the 00:05:10.410,00:05:12.412 idea behind these is DARPA finds this fledgling technology, self driving cars, uh and they fund 00:05:12.412,00:05:17.417 it with a lot of money right? So their prizes, million dollar prizes for uh self driving cars 00:05:26.760,00:05:31.198 and this motivated a lot of people to put a lot of research into it. At the time people were 00:05:31.198,00:05:36.169 of course saying, because the time was 2006 when we didn't even have smart phones and 00:05:36.169,00:05:40.507 people were saying, 'do you really think that some day you'll be sitting inside a 00:05:40.507,00:05:47.147 computer and it'll be driving you around? That's absurd', and now we have people driving 00:05:47.147,00:05:52.319 themselves to the hospital while they're having a heart attack in their Tesla and so you know this 00:05:52.319,00:05:57.324 technology push really pays off. And it's probably gonna be the same with robotics, DARPA did 00:05:59.326,00:06:02.796 the robotics cyber grand challenge and probably in ten years we're all gonna be dead 00:06:06.867,00:06:12.005 [laughter] And it's also gonna be the same with programs so the cyber grand challenge really 00:06:12.005,00:06:17.010 pushed the frontier of automatic program analysis exploitation and defense right now it's in 00:06:20.947,00:06:27.887 it's infancy I think uh if you'll see how the CRS' did at Defcon CTF uh but maybe they 00:06:27.887,00:06:33.226 won't beat the best humans, but that's the beginning. The chess systems didn't beat the best 00:06:33.226,00:06:37.430 humans, and the self driving cars aren't going to beat the best humans in races right now, 00:06:37.430,00:06:42.836 but eventually they will and eventually mechanical phish will kill us all, or hack us all, 00:06:42.836,00:06:49.576 while the actual robots kill us. [laughter] So this is a cyber grand challenge, um let's talk 00:06:49.576,00:06:55.081 about Shellphish's involvement in the cyber grand challenge. As Giovanni said, Shellphish is a 00:06:55.081,00:07:00.020 bunch of academics and hackers right so we're kind of 'hackademics' so um at one point 00:07:03.456,00:07:08.461 we decided to shift our research uh interests in uh at UCSB closer to binary analysis right 00:07:10.997,00:07:17.904 we started looking into uh doing automated binary analysis and all of the things along with 00:07:17.904,00:07:23.510 that automatic vulnerability discovery and so forth completely independent of the 00:07:23.510,00:07:26.846 cyber grand challenge, we started doing this some time in twenty thirteen, and then late 00:07:26.846,00:07:32.752 twenty thirteen DARPA announces the cyber grand challenge right? And so I have an email somewhere 00:07:32.752,00:07:37.857 in my history saying 'hey guys! Check this out, this is this cool thing, maybe you should 00:07:37.857,00:07:41.394 participate because we were working on a lot of the same stuff' and everyone said 'yeah! 00:07:41.394,00:07:45.965 Let's do it, let's go for it!' I said 'great' and then promptly forgot about it for like a year, 00:07:45.965,00:07:50.970 right? So the deadline for registration was in late twenty fourteen, I sent in the kind of 00:07:55.141,00:08:00.347 uh application literally fifteen seconds before the deadline because that's that's how we 00:08:00.347,00:08:05.352 roll and they uh said 'great you're in congratulations uh let's you know see what you've 00:08:08.588,00:08:14.361 got the first scored event is coming up in like four months and so we were like 'okay cool' 00:08:14.361,00:08:20.567 oh no like on the graph is like in one month right? So we said cool, let's let's build a CRS 00:08:20.567,00:08:24.704 we're gonna we're gonna rock the scored event the the first kind of practice round that that was 00:08:24.704,00:08:29.642 the term DARPA used for the scored events. So the first practice round uh we're gonna 00:08:29.642,00:08:34.914 we're gonna do super awesome, we were gonna kill it and we totally forgot about it. The 00:08:34.914,00:08:39.953 morning of the practice round I wake up and I'm like shit there's a practice round for the 00:08:39.953,00:08:46.092 the CGC stuff tonight and so we start working on our CRS right so the first commit to the CRS 00:08:46.092,00:08:51.097 is two hours maybe three hours let's say before the practice round begins right? So we start 00:08:53.266,00:08:58.705 writing our CRS, practice round begins, we play the practice round with some janky ass CRS 00:08:58.705,00:09:04.544 that that kinda half works cool. So then we like alright well now that we've started we're gonna 00:09:04.544,00:09:08.882 get it all super put together before the second practice round. Second practice round 00:09:08.882,00:09:14.154 rolls around, and now we remember about it maybe three days before right? So the second 00:09:14.154,00:09:19.859 commit to the CRS happens three days before the second practice round. We uh build it up build 00:09:19.859,00:09:23.830 it up build it up play in the second round, say okay cool, now we have this uh kinda cyber 00:09:23.830,00:09:28.835 reasoning system that's uh kind of ready to play in the CGE if we keep working on it solidly 00:09:31.738,00:09:37.343 until the uh qualifiers. And then of course, we forget about it for another couple months. 00:09:37.343,00:09:42.582 And then two and a half weeks before the qualifiers we remember hey wait a second the 00:09:42.582,00:09:48.988 qualifiers are coming up so then we start working like crazy and not sleeping three weeks of 00:09:48.988,00:09:55.695 complete insanity until the cyber grand challenge qualifiers and we have a cyber reasoning 00:09:55.695,00:10:02.001 system that we can feel for the cyber uh grand challenge qualifiers and we qualify with 00:10:02.001,00:10:07.006 three weeks of absolute insanity. And so then we figured cool! Now A, we're super rich, 00:10:09.409,00:10:14.547 'cause the qualifiers came with seven hundred and fifty thousand dollars of prize money, and B we 00:10:14.547,00:10:19.552 can now spend a year working solidly, right? Solidly, with test cases, test cases, code 00:10:21.855,00:10:26.860 freezes, milestones, milestones, lots of milestones, and absolutely you know continuous 00:10:30.196,00:10:36.603 uh integration and then and you know tests uh rounds and everything for an entire 00:10:36.603,00:10:41.608 freakin' year! Agile development that's that's the key word here, none of that happened. So for uh 00:10:44.310,00:10:50.950 nine months we uh use our money to fly around the world giving conference talks and like saying 00:10:50.950,00:10:55.955 how how cool we are and how you know phish is uh is a chinese martial arts expert or wait that 00:10:58.391,00:11:03.963 was that was Kevin, Kevin's a chinese martial arts expert and you know Antonio's mysterious 00:11:03.963,00:11:07.634 and all this shit but didn't really do what we should have been doing is working on the CRS 00:11:07.634,00:11:14.541 right? And three months before the uh three months ago we realize this and we're like 00:11:14.541,00:11:20.246 crap! We should really write a CRS for real actually right? Like I mean we should take what 00:11:20.246,00:11:27.120 we had in quals and actually like you know extend it so it can win finals so three months 00:11:27.120,00:11:33.893 ago we we started working like crazy we stopped sleeping right? I have a fiance and I haven't 00:11:33.893,00:11:38.898 seen her in three months basically that that's you know the insanity. >>To the founding 00:11:41.034,00:11:46.039 agency that are listening, we're a lot more responsible than it looks. >>Yeah, yeah th- this is 00:11:48.074,00:11:53.212 our hacker persona, right? We also have an academic persona where of course we have CI, of 00:11:53.212,00:11:59.252 course, come on who doesn't have CI, when code freezes right we we'd finish all our papers two 00:11:59.252,00:12:05.525 weeks before they're due so that our professors can uh go over them and absolutely this this is 00:12:05.525,00:12:10.530 the hacker [inaudible] persona [chuckle] Alright anyways, so we went crazy for three months, we 00:12:13.132,00:12:18.137 got uh the final commit to the CRS, thirty minutes before the air gap was established, thirty 00:12:21.040,00:12:26.946 minutes alright? And it was a commit in one of the core components so shit could go 00:12:26.946,00:12:33.820 wrong, there's a slide for that. And >>Killing us >>Alright, I'm killing us, so we did it. We 00:12:33.820,00:12:39.626 played the CGC, you got third, and this is the team that we already introduced. We are from 00:12:39.626,00:12:44.631 all around the world, Italy, Germany, the U.S., India, there was a guy uh qualifying with us 00:12:48.201,00:12:54.974 who's hopefully sitting in the uh audience from Santa Golf, uh Phish is from China, we're from 00:12:54.974,00:13:01.814 all, all over the place. And we are very rich because we got two seven hundred and fifty thousand 00:13:01.814,00:13:08.021 dollar prizes now, so, that's kind of a brief intro to our involvement in the CGC I'll pass 00:13:08.021,00:13:13.026 it off to Ya Kapow to introduce the CGC as a platform and what it means. [applause] >>Right so, 00:13:15.495,00:13:20.500 thanks Yan for the very very very true and very effective introduction to the Shellphish 00:13:23.269,00:13:28.241 hacker, very distinct from academia, very distinct from the Shellphish academia, uh right 00:13:28.241,00:13:35.048 so, just very briefly, so what does it mean to actually score well in the CGC? You have to, 00:13:35.048,00:13:39.886 you're gonna go blind with binaries that you have never seen before, you have to analyze 00:13:39.886,00:13:45.692 them in whatever way you want, there's no limitation of how you do it, you have to own them, 00:13:45.692,00:13:50.797 either by a crash or by leaking a secret, and you also have to patch them so that the other 00:13:50.797,00:13:55.802 guys can not do the same to you and this is a classic um classic CTF uh structure that has some 00:14:00.440,00:14:05.678 modifications to decree in in the decree operating system to make it more moddable- more 00:14:05.678,00:14:12.552 easier to model and easier to handle for a for a program okay? So on the simplifications is 00:14:12.552,00:14:19.425 that uh so the architecture is Intel x86 all op codes are legal which can lead to interesting 00:14:19.425,00:14:26.065 situations that we will see in a- in a bit. Um syscalls are simplified, much easier to 00:14:26.065,00:14:31.671 model, pretty much read and write, select, uh allocate deallocate like modlook and 00:14:31.671,00:14:36.676 free, random and opposite exit, a lot easier to model for a program and the actual binaries 00:14:38.911,00:14:43.916 are actually a lot a lot more realistic uh are very real, they're not uh complete fake 00:14:46.319,00:14:53.292 binaries. >>So as a side note, the Defcon CTF just finished, and the Defcon CTF was also 00:14:53.292,00:14:57.930 played on the same platform so just as an example of how real and complex these binaries can 00:14:57.930,00:15:03.369 be, one of the challenges in the Defcon CTF was a powerPC interpreter and jitterer which 00:15:03.369,00:15:08.374 was awful and so there's a lot of room for complexity in these programs. >>And on the actual 00:15:12.044,00:15:17.049 pwning side um I don't know if some of you guys want to barge in but basically what it means 00:15:19.385,00:15:24.390 is that there is no there is no state every program runs once there is no state it runs you 00:15:27.827,00:15:32.331 either own it or it's gonna do its thing there's no uh there's no state there's no file system 00:15:32.331,00:15:38.271 to modify, it's a lot easier to to model for the for the qualifications and only for the 00:15:38.271,00:15:44.143 qualification it was just enough to crash the program set forward illegal instruction you will get 00:15:44.143,00:15:49.582 the point, you have owned the binaries. For the finals, things a lot more nuanced and the 00:15:49.582,00:15:55.354 actual exploitation as we will see is a lot a- is a lot more complicated and is a very 00:15:55.354,00:16:00.293 interesting application of how to use symbolic execution and static analysis uh but as as a 00:16:03.329,00:16:09.802 as a basic idea the two ways you do is either via a control crash in which you can show that you 00:16:09.802,00:16:15.775 can not only crash the program in some place but you can actually crash the program at a 00:16:15.775,00:16:21.681 place that the API, that DARPA's gonna tell you please crash the program in this place and set 00:16:21.681,00:16:27.520 this register to this value, if you can do that you verify that you have actually control of the 00:16:27.520,00:16:32.525 program or alternative that you can leak a secret flag from memory and on the patch inside 00:16:35.661,00:16:40.666 just a a brief note on how you unlo- how this API is designed so that it does not become too 00:16:42.735,00:16:47.039 easy like for instance, we can submit patches to the binary okay so what is preventing us 00:16:47.039,00:16:53.479 from just submitting a binary just exits? Kay? This programs- this program obviously never 00:16:53.479,00:16:59.585 crashes but it also does not do anything useful so the way this is prevented is b- is that there 00:16:59.585,00:17:05.291 are functionality checks if you if the program does not maintain it's benign function, if the 00:17:05.291,00:17:10.596 program is um a math calculator it needs to still be able to do all the math generation that it 00:17:10.596,00:17:15.601 can do normally and similarly there is no signal handling so no way to just hide away the 00:17:19.805,00:17:24.810 segfaults, if you segfault you are crashing, and finally how would prevent us from putting in 00:17:26.979,00:17:31.450 an interpreter that runs everything, so checks before every possible instruction, am I 00:17:31.450,00:17:35.988 gonna crash? Am I gonna crash? Obviously it would never crash, and the way this is prevented by 00:17:35.988,00:17:41.494 DARPA that you can actually do it, you can do it if you want, but you're gonna pay a 00:17:41.494,00:17:48.301 performance price you're gonna lose points for performance this is believe me not as easy as it 00:17:48.301,00:17:54.106 sound, understanding exactly how your patch is performing is definitely not an easy task many 00:17:54.106,00:17:58.945 of us looked into it, I looked into it a bit, Antonio looked into it into bit in a beet, it's 00:17:58.945,00:18:04.016 definitely pretty hard. >>And then we gave up testing performance we'd just say, this 00:18:04.016,00:18:10.523 is our patch, deal with it >>Yes, yes, that's very true and on a you know informally we know 00:18:10.523,00:18:17.029 other teams also had trouble but I think no one more than Arven, knows very well how much of a 00:18:17.029,00:18:23.002 pain, how much of a big pain it can be to actually test the performance and the 00:18:23.002,00:18:28.641 functionality of binaries so big props to Arven for actually pushing through this task 00:18:28.641,00:18:33.646 [applause] and actually making it and this actually helped us a lot doing our own internal 00:18:36.616,00:18:41.620 testing even if it did not go into the live part and I will now hand over >>Somebody just go 00:18:45.625,00:18:50.629 >>Somebody! >>Antonio! [shouting] >>Alright so the uh CQE for the qualifying event was 00:18:53.833,00:18:58.838 not the full [inaudible off mic comment] no it was not the full cyber grand challenge, it was, 00:19:03.242,00:19:07.013 you needed to patch binaries and you needed to crash binaries, you didn't need to exploit 00:19:07.013,00:19:13.653 anything, you just needed to crash it. The final event, you need to patch binaries, crash 00:19:13.653,00:19:18.958 binaries to find where vulnerabilities are and then exploit those vulnerabilities 00:19:18.958,00:19:23.963 and on top of that it wasn't just a simple game or a simple program challenge where you got 00:19:28.034,00:19:33.139 a binary and you crashed it, it was a game, so you had to have a game theoretic aspect that uh 00:19:33.139,00:19:38.144 played against other actual competitors, right? Similar to a human CTF but all with 00:19:38.144,00:19:43.149 computers. Um so the competition was actually divided into ninety six rounds uh and that wasn't 00:19:46.352,00:19:52.191 predetermined, it was, you know, however many rounds they got through in a day uh there was a 00:19:52.191,00:19:57.129 minimum time per round, and it ended up being ninety six, um there was a bunch of um 00:19:57.129,00:20:02.134 challenge binaries as they term uh as DARPA terms them uh which were provided to the teams to 00:20:04.437,00:20:09.442 hack and for each score for each round the team would have a separate round score that when 00:20:11.677,00:20:16.248 aggregated would be their total score for the game, the score was calculated based on the 00:20:16.248,00:20:22.321 multiplication of the team's availability, which means how much they fuck up the binary and 00:20:22.321,00:20:26.926 how fast the binary still was, right? How much overhead the patches had which is something 00:20:26.926,00:20:32.398 uh Yak already alluded to the security score, which is how exploitable where the binaries 00:20:32.398,00:20:37.736 still? Or were they still exploitable and the validation score which means did we find- 00:20:37.736,00:20:42.741 did the team find an exploit for this binary. So it was very easy to screw yourself in this 00:20:45.077,00:20:49.982 context because they're all multipliers, if you completely break the binary, even if you 00:20:49.982,00:20:56.222 have perfect offense, even if you find all of the exploits for this binary then you still get 00:20:56.222,00:21:02.628 zero points because you broke the binary. In developing for this competition we uh ran into 00:21:02.628,00:21:09.401 a lot of kind of uh organizational things as I alluded to earlier, we started 00:21:09.401,00:21:15.141 super late, so for example, up until depressing the short time ago this was our database, 00:21:15.141,00:21:20.146 right? >>After all this is a research group ran by an Italian [laughter] >>Yes, again this is 00:21:24.250,00:21:28.988 our hacker persona. So we actually had to do a join on this database at one point when 00:21:28.988,00:21:33.526 we uh got the real database up we were joining between the paper database and the actual 00:21:33.526,00:21:37.429 database. >>This is relevant because it's about our performance scores this is- the 00:21:37.429,00:21:40.266 graph is a database of our performance scores we're trying to analyze that's >>Yeah 00:21:40.266,00:21:43.903 >>relevant to the previous slides >>Specifically this database contains the feedback 00:21:43.903,00:21:50.809 from some uh practice sessions for the final event so this is what DARPA called sparring 00:21:50.809,00:21:56.115 partner sessions we wrote 'em down and then we had to join them with the real database to 00:21:56.115,00:22:02.655 get the actual information that we needed to tune our patches. Uh we also tried to go into code 00:22:02.655,00:22:07.660 freeze several times, so at 4:01 pm on some god forsaken day uh we froze a component of our uh 00:22:10.462,00:22:15.467 CRS called farnsworth uh and very shortly thereafter this is the commit log right so the code 00:22:18.204,00:22:23.209 freeze didn't work very well um there are commits such as this gem here [laughter] so that 00:22:26.412,00:22:32.151 that's that's Francesco here that the you know [applause] beautiful, beautiful code the- 00:22:32.151,00:22:36.722 this commit was okay actually he just has very high standards. Actually it was probably crap 00:22:36.722,00:22:41.727 but you know. Um and then of course this is uh a long time into our code freeze twelve- 00:22:45.464,00:22:52.404 fifteen hours before our nodes were shut down a couple days ago we were still changing very core 00:22:52.404,00:22:57.409 components of the system. That's me upside down, I was at this point, no longer sane. So our uh 00:23:01.113,00:23:06.118 CRS consisted of a lot of components, right? We had a um we had a central database that 00:23:10.889,00:23:15.894 we called farnsworth for some reason uh which is stored all of the data that uh we got from the 00:23:18.597,00:23:23.602 uh cyber grand challenge API through a component that uh we'll talk about later um it 00:23:25.904,00:23:30.909 stored network uh catchers it uh made uh it stored the scheduling decisions of what jobs to run 00:23:34.113,00:23:39.685 and then it stored the result of those jobs so now we're gonna go one by one into all of these 00:23:39.685,00:23:45.624 components probably pretty quickly, fifteen minutes left? And we'll start with the 00:23:45.624,00:23:50.696 organization of the core organization components and I'll hand it over to Francesco and 00:23:50.696,00:23:52.698 Kevin. >>So obviously coordination is very important if you're running a cluster of 00:23:52.698,00:23:54.700 sixty four nodes. Um and of course um since we needed to do that we essentially came up with 00:23:54.700,00:23:56.702 like using one database to essentially store all of the ground truth that we have um as 00:23:56.702,00:24:01.640 a bunch of you probably know this is from Futurama um so we just went with essentially 00:24:13.118,00:24:18.924 farnsworth because well good news everyone um and it's the only component that we actually 00:24:18.924,00:24:23.629 tested fairly well had about 69% test coverage, I think the rest probably dumps around at like 00:24:23.629,00:24:28.634 one percent um >>Zero >>Zero, oh! Perfect, even better. Um and who needs testing anyways right? 00:24:30.769,00:24:35.874 >>Angr has at least fifteen percent code coverage >>I think Francesco probably disagrees but 00:24:35.874,00:24:42.381 uh eh who cares. Um then on top of that we essentially had to meister which the Germans in 00:24:42.381,00:24:46.719 here now it's essentially just master. Um which looks at scheduling jobs and deciding 00:24:46.719,00:24:51.857 what jobs we want to run, what kind of hardware pipeline we want to run, exploits patching, 00:24:51.857,00:24:56.895 if we want to run AFL, these kind of things, and schedule them based on priority and this 00:24:56.895,00:25:01.567 obviously- sorry the last component that we actually changed with the last commit 00:25:01.567,00:25:08.574 being I guess two hours and eighteen minutes before the actual deadline so yeah this one 00:25:08.574,00:25:14.780 said twelve forty two and the same deadline to actually the node shutdown was at three pm 00:25:14.780,00:25:18.183 >>we made a commit, I think we rolled that commit thirty minutes before the deadline 00:25:18.183,00:25:21.687 >>yeah there were a bunch of commits at like two pm but we actually reverted them and 00:25:21.687,00:25:25.491 cleaned up the history just to make sure that they're actually not there, because they caused a 00:25:25.491,00:25:31.196 bunch of failures on our site. Um anyways um we would also like to give a big shout out to 00:25:31.196,00:25:35.100 essentially the open source components that we essentially rely on one of them is Python, 00:25:35.100,00:25:39.271 the Microsoft research Z3 compiler, all of our things run into- inside of docker 00:25:39.271,00:25:46.211 containers which are running ubuntu with pypy, um we're also using kubernetes, Qemu, PeeWee , 00:25:46.211,00:25:50.883 Vex, Postgres, obviously Angr which I'm sure a bunch of people are going to talk about now, and 00:25:50.883,00:25:55.888 I think that's probably Yan, possibly Sulse, Andrew, John I guess and Pizza yeah go ahead 00:25:58.924,00:26:03.862 >>I want to say something- I agree with everything he said [laughter] >>Angrs the open 00:26:08.300,00:26:12.070 source binary proj- binary analysis project that we have in the sec lab it's really really 00:26:12.070,00:26:15.574 cool it's been open sourced for like a year now we released it at Defcon last year, right? 00:26:15.574,00:26:21.380 Yeah! Um it does everything it's cool um no time it's very cool that's our logo it's creative 00:26:21.380,00:26:26.151 commons um we in order to do the actual exploitation and analysis pipeline we split it up into a 00:26:26.151,00:26:30.389 whole bunch of components and rearranged them into these weird uh things like we've used 00:26:30.389,00:26:36.028 concolic execution in order to do some basic analysis of what can go where, there's automatic 00:26:36.028,00:26:39.531 exploitation in patching which will all be talked about I think they've all go their section in 00:26:39.531,00:26:44.536 this presentation um there's crashes >>I think you can slow down a little [laughter] just a 00:26:49.041,00:26:55.013 tad >>Fine [inaudible] sorry, so who's cra- who wants to talk about crashing? >>Crashing >>Eh 00:26:55.013,00:26:59.985 guys we haven't been sleeping for three days so >>I always talk this fast I'm sorry if 00:26:59.985,00:27:04.690 you're friends with me >>To all the founding agency, we're not doing drugs, or alcohol 00:27:04.690,00:27:11.663 [laughter] looks like it but we're not >>I'm not even twenty one >>Alright crashes, Sulse, 00:27:11.663,00:27:16.668 Nick, talk about it, you see how prepared we were for this huge defcon talk >>Hello uh so uh 00:27:25.377,00:27:30.382 crashing uh so our exploitation strategy is we find crashes and we turn those into exploits uh 00:27:32.784,00:27:39.725 so >>Pretty incredible >>Uh so actually like a lot of the team's, the thing we do the most 00:27:39.725,00:27:43.128 is fuzzing and this is what generates a lot- lots of test cases, lots of crashes, the 00:27:43.128,00:27:48.133 majority are crashes but not entirely all the goodies we find. So we use AFL as our core 00:27:50.335,00:27:55.340 component uh fuzzing, we uh I'm will explain how AFL works like these slides do I suppose and uh 00:27:59.578,00:28:04.082 essentially beg- begins by generating lots of inputs which attempt to explore different 00:28:04.082,00:28:10.322 parts of the program uh the inputs are basically random uh some of them are are more or 00:28:10.322,00:28:14.626 less educated guesses and how well these inputs do when exploring the program is tracked 00:28:14.626,00:28:20.666 by instrumentation which is compound to the binary or which is provided by uh an emulator 00:28:20.666,00:28:25.671 like QMM. Um so, let's see did I go over all of these? So Avo's a great job of doing this, we've 00:28:29.541,00:28:34.346 modified it slightly to work better on CGC binaries so we have a couple of hacks which I 00:28:34.346,00:28:39.351 think we'll be open sourcing which make it perfect for CGC or at least a lot better uh okay uh 00:28:45.057,00:28:50.495 the uncrasher I don't think that's actually, that actually exists but [laughter] >>No >>I 00:28:50.495,00:28:55.367 don't think there's an uncrasher man [laughter] >>The points a flag in all this shit >>Yes so 00:28:55.367,00:29:00.305 >>it's like karaoke slides >>right uh shoot I already mentioned this right? AFL, it's 00:29:02.908,00:29:07.913 great, this is how fuzzing works uh random stuff gets put into the binary yep same input all 00:29:11.683,00:29:15.487 over again, eventually it comes up with a random thing that works, this is much harder for a 00:29:15.487,00:29:21.860 fuzzer we have to generate a very specific input fuzzing will have no luck with this of keeps 00:29:21.860,00:29:26.732 continues to lose uh makes absolutely no progress. >>If you guys can't feel like you can't 00:29:26.732,00:29:33.005 keep up with Mike Pizza, I feel like that very frequently. >>Okay so Angr on the other hand 00:29:33.005,00:29:37.342 is a symbolic execution engine, it's slower and more heavy weight, but it's great at 00:29:37.342,00:29:41.580 finding more specific cases like the one we just described. And the way this works is by 00:29:41.580,00:29:44.783 generating these states following different paths, as you can see here in the control 00:29:44.783,00:29:49.154 flow graph we have different states which are being um followed uh eventually there is 00:29:49.154,00:29:54.493 a state which will satisfy the you win expression and we talk to Z3 we ask it to generate an 00:29:54.493,00:29:59.498 input which gives us this state and boom. So what we tried to do is combine both AFL and Angr 00:30:02.300,00:30:06.138 [laugh] and we- this is called Driller, Driller begins by fuzzing it gets basic code 00:30:06.138,00:30:10.809 coverage of the program the way you would expect AFL to and ge- maybe gets a couple test cases 00:30:10.809,00:30:15.814 in this example x and y we get the cheap coverage, next slide [laughter] dyna- okay then we 00:30:18.050,00:30:22.087 take those test cases and we trace all of them with Angr so we make the input completely 00:30:22.087,00:30:26.091 concrete almost we actually make it- keep it symbolic but we can strain it to be this concrete 00:30:26.091,00:30:30.128 input that AFL generated and we see at any point in the program if we could've taken a different 00:30:30.128,00:30:35.133 path which AFL failed to take, if we could have taken that path, we talk to Z3 or Angr more 00:30:35.133,00:30:40.005 specifically and we say, give me an input which satisfies this new path. In this case we get 00:30:40.005,00:30:45.010 the CGC magic and the new test case is generated and now we continue the loop and we feed 00:30:47.179,00:30:50.549 this back in the AFL which continues to mutate that further and fuzz and it goes on and on 00:30:50.549,00:30:56.254 until we continue to get more code coverage uh >>we play video games >>Alright so this next 00:30:56.254,00:31:03.095 part is the auto exploitation, how we go from a crash, which is generated by AFL and Driller to 00:31:03.095,00:31:08.100 actually an exploit for the CGC which scores us a flag. Alright so in this example I think 00:31:10.102,00:31:16.708 there's a buf- so there's a buffer overflow inside the- he- inside this mallet object here 00:31:16.708,00:31:21.713 and when you overflow this buffer you actually control the function pointer and so we- 00:31:23.882,00:31:29.221 inputting inputting inputting symbolic bytes and eventually we c- c- control the buffer, the 00:31:29.221,00:31:34.926 symbolic address, we're gonna call in into an address we control and so to exploit this 00:31:34.926,00:31:39.931 we use Angr, we check we trace the input using Angr and check that first the IP is symbolic, 00:31:42.267,00:31:48.573 the PC here we say is a state, does a state have a symbolic PC, at that point we know it's 00:31:48.573,00:31:53.945 probably exploitable we can control where we're gonna jump to and so let's set the buffer 00:31:53.945,00:31:59.284 to contain our shellcode. We ask Z3 to give us input where the buffer point contains shellcode 00:31:59.284,00:32:05.957 and then we jump to the buffer and that'll give us an exploit. Um and to do this we synthesize 00:32:05.957,00:32:11.196 the input and in Angr that's just called state dot pausex dot dung zero. >>So in the CGC this 00:32:11.196,00:32:15.367 is discovered by taking and crashing input and tracing that with Angr, so keeping all the 00:32:15.367,00:32:20.438 input that AFL created, symbolic and then following the path that took until we have our crashing 00:32:20.438,00:32:25.443 symbolic state >>So Alright >>Keep in mind this is very simplified, we have a bunch more 00:32:27.879,00:32:33.418 techniques that handle the harder cases and that can take a not so good crash and turn it 00:32:33.418,00:32:37.589 into a better crash and you can find those all when we do our open source release and when we 00:32:37.589,00:32:43.929 release more details and papers later >>And in the open source release um this component is 00:32:43.929,00:32:49.568 called Rex if you're interested in auto exploitation, check that out. >>Alright so then the steps 00:32:49.568,00:32:55.740 again, we create a vulnerable symbolic state where we- control the PC, we add the constraints 00:32:55.740,00:33:01.913 to set the shellcode and to set the the program counter to point to the shellcode and then we 00:33:01.913,00:33:08.253 send the files to the input and that creates our exploit. >>Okay so this uh this component we'll 00:33:08.253,00:33:13.058 be talking about auto quotation of flag leaks so if you didn't know there are two types of 00:33:13.058,00:33:17.963 exploits you can generate in the CGC, the type one is sort of classic memory corruption, show 00:33:17.963,00:33:22.067 that you can control the program uh counter, show that you can also control a general purpose 00:33:22.067,00:33:27.706 register, how- however there's another type called the type two, very creative, which uh 00:33:27.706,00:33:32.143 shows that you can leak arbitrary memory from the program so in the CGC there's 00:33:32.143,00:33:38.550 actually a uh sensitive crypt- uh sensitive data that's mapped at a special address uh in every 00:33:38.550,00:33:44.756 single binary and if you leak content from this page in memory, you score points. Like 00:33:44.756,00:33:51.229 uh heartbleed for example would uh there was a heartbleed challenge in this game which uh 00:33:51.229,00:33:56.701 where the premise was leaking this data from this flag page, the- the sensitive data. So the 00:33:56.701,00:34:03.375 way we do this in a fast way, is we actually use uh the unicorn engine which Angr integrates to 00:34:03.375,00:34:09.247 make the entire input completely uh concrete the only thing which is symbolic during the flag leak 00:34:09.247,00:34:14.819 detection, is the flag page itself. So we trace the entire program and execute very fast 00:34:14.819,00:34:20.558 because everything is being concretely eh- uh emulated by QMM with Unicorn and we can 00:34:20.558,00:34:25.897 detect uh and transmit because we hook it with Angr when the flag page is actually being 00:34:25.897,00:34:29.868 emitted and then we can see exactly which transformations are done to those flag page you 00:34:29.868,00:34:33.805 m- you can tell if it's been XOR'd or if some complicated constraints been applied for 00:34:33.805,00:34:40.145 example this actually solved the Defcon CTF challenge uh which [chuckle] ye- okay, so we don't 00:34:40.145,00:34:43.581 have enough time to talk about that but we solved the Defcon CTF challenge this way, so. 00:34:43.581,00:34:49.854 >>We'll- we'll talk about it a little more later >>Patrick, go! Go fast, >>So uh >>You have 00:34:49.854,00:34:54.693 seven minutes >>So of course one of the challenge was to patch this binary so we had the 00:34:54.693,00:34:59.898 component called Patcherex that was going from patch from from a patch binary to patch binary so 00:34:59.898,00:35:04.235 they generate this via patching techniques for instance let's start- let's increase the return 00:35:04.235,00:35:08.873 address and this patching techniques generates patches such as let's start this code 00:35:08.873,00:35:13.912 here let's start this data there and this patches were injecting within the binary, we had three 00:35:13.912,00:35:20.285 different ways, the first one was slower but more reliable, and the last one was faster but 00:35:20.285,00:35:26.491 uh less a little bit less reliable and phish is probably gonna talk about the reassemble. 00:35:26.491,00:35:32.831 And so we had adversarial patches that were designed not to make uh our binary our patch 00:35:32.831,00:35:37.836 binary analyze were by others and this is one of them that is pretty cool and >>Um this is a 00:35:40.572,00:35:45.810 detect- QEMU detection, this, if you run this code in QEMU, QEMU F3D6, it'll hang forever, well 00:35:45.810,00:35:49.581 not really forever, as long as it takes to int- to increment a sixty four byte int- into the 00:35:49.581,00:35:55.286 sixty four times, that's basically forever um and we actually owned the cyber grand 00:35:55.286,00:35:59.090 challenge um visualization infrastructure with this, they're apparently using QEMU 00:35:59.090,00:36:03.028 for instruction tracing and so at one point during the CGC we noticed that their instruction 00:36:03.028,00:36:08.099 tracing had just stopped, and it stopped, right on this code which was designed to detect 00:36:08.099,00:36:12.670 QEMU and crash- well not crash but hang forever >>This was a zero day, take a picture >>Uh 00:36:12.670,00:36:17.976 there- we have a lot of open source bug fixes to contribute starting now [laugh] >>So there 00:36:17.976,00:36:23.381 were other sorts of- of sort of adversarial patches so to speak for instance our binary was 00:36:23.381,00:36:29.654 starting by transmitting the flag out but uh uh they were transmitting to stderr- so so- 00:36:29.654,00:36:34.659 to stderr so that uh this could probably confuse an analogy system that could misidentify 00:36:37.162,00:36:43.701 this as a- as a type two vulnerability. We also have a backdoor that if some team was 00:36:43.701,00:36:48.707 using our patch uh in the- in their uh submission we could actually exploit it and I'm not 00:36:52.110,00:36:57.082 sure if the backdoor worked during the CGC but for sure it worked during the CTF. >>Yeah 00:36:57.082,00:37:01.619 how many team >>During Defcon >>How many team fielded that our backdoor >>I know that other 00:37:01.619,00:37:07.225 teams use our backdoor doing uh Defcon >>Can, can, can you name names? >>I'm sure >>No no no no 00:37:07.225,00:37:12.230 names >>It was three teams that fielded our backdoor at the CTF >>During CGC? >>CTF >>Uh CTF 00:37:14.265,00:37:18.837 >>Okay cool so then we had those also sort of generic patches that are- these are more 00:37:18.837,00:37:24.109 standard academic things such as uh protecting their return pointer, protecting data codes, 00:37:24.109,00:37:29.380 and when when we are gonna release this uh code you will see all this sort of kinda more 00:37:29.380,00:37:34.986 standard techniques and then target Apache so the general idea- oh you can speak about 00:37:34.986,00:37:39.991 >>Yeah so so targeted patches right so- qu- mmm- qualification events oh we just wanted to 00:37:42.093,00:37:47.465 avoid crashes right? 'Cause anything that crashes counts as an exploit so we had some uh you 00:37:47.465,00:37:54.172 know we just checked uh using a weird quirk of one of the syscalls uh using a weird quirk 00:37:54.172,00:38:00.345 of one of the syscalls we checked to see if the um if if memory was uh readable at a 00:38:00.345,00:38:06.751 certain point if it wasn't we crashed. So I would like to take sp- specific credit for our back 00:38:06.751,00:38:12.857 one slide for our targeted patches in the final event which were exactly nil. And it worked 00:38:12.857,00:38:17.962 great so what what can I say [applause] >>And the- and one note that uh no functionality 00:38:17.962,00:38:22.233 overhead from >>I thought it was a bug in the slides >>No no that was intentional, and one one 00:38:22.233,00:38:27.338 cool thing about this that we we thought we were cooler finding this uh weird syscall tricks to 00:38:27.338,00:38:32.877 detect memory locations but actually when we analyze uh uh qualification binaries from 00:38:32.877,00:38:36.748 other teams when they were released we found that at least one other team was using exactly 00:38:36.748,00:38:41.753 the same trick. >>So you're saying you were both cool? >>Yeah we were both cool >>Okay 00:38:44.289,00:38:49.494 >>Phish! >>So uh we are running out of time so what I- the only thing I want to say is uh Angr 00:38:49.494,00:38:53.298 is awesome I spent three days in writing uh reassembler and another three days in writing 00:38:53.298,00:38:57.769 optimizer so it works out >>So so what is a reassembler? Just real quick? >>Reassembler is a 00:38:57.769,00:39:02.707 static binary uh rewriter that basically okay we'll um talk about it later >>No no no okay 00:39:07.679,00:39:13.952 >>Alright we have a- we have a- breakdown from our I think I think one of our slide guys is 00:39:13.952,00:39:20.792 uh is um >>It's fine >>okay it's fine the reassembler is awesome [laugh]. Phish wrote a binary 00:39:20.792,00:39:25.797 rewriter where you can inject code into binaries and it'll seamlessly reassemble the binary 00:39:28.466,00:39:34.572 to include that code, check it out in the open source release. You go. >>Now there's nothing 00:39:34.572,00:39:41.212 much to say, basically I tried, so we, DARPA gave us sixty four mm- powerful servers >>Wait, how 00:39:41.212,00:39:47.085 many servers? >>Sixty four?! >>No I'm not joking, sixty four! >>Holy shit! >>Not thirty, sixty 00:39:47.085,00:39:52.090 four. So we tried to maximize this usage, the usage of these nodes and yeah we kinda did it 00:39:54.993,00:39:59.998 with the CPU list [laugh] not the memory but that's it. >>That, that's it. So the sixty 00:40:02.700,00:40:07.705 four servers we had a lot of media attention over the CGC and uh what we got wh- what we got 00:40:10.174,00:40:16.614 people excited about the most strangely enough is the fact that we had sixty four servers 00:40:16.614,00:40:21.619 all to ourselves, incredible. Anyways so. We implement all these systems in uh break neck 00:40:25.723,00:40:30.728 like three months uh and we pushed as hard as could, we got it all running, we made commits 00:40:32.930,00:40:39.137 at the last second, and we played the game, or rather, our baby played the game. She walked 00:40:39.137,00:40:45.543 on her own, we walk into the room, and they told us hey! Your guys' bot started up and is 00:40:45.543,00:40:52.216 doing a lot of DISK-I-O and we fucking lost it because up until- we freaking lost it, 00:40:52.216,00:40:58.056 because up until then we thought you know it's gonna turn on and something will fail and and 00:40:58.056,00:41:04.562 it'll all crap itself so this was incredible and then we got third place top three is amazing 00:41:04.562,00:41:10.835 for us guys I can't- I can't tell you how incredible it is to have been part of this comp- and 00:41:10.835,00:41:15.840 we're going on? [applause] it was incredible. >>Since we played in the CTF we didn't 00:41:19.610,00:41:24.982 really get much of a chance to actually look at the data >>Yeah >>Um hardware we quickly briefly 00:41:24.982,00:41:28.920 looked at it so in total there were eighty two challenge sets fielded, at least our bot saw 00:41:28.920,00:41:33.725 only eighty two so if more have been fielded we might have actually missed them. In total 00:41:33.725,00:41:38.996 mechanical phish generated about two thousand four hundred fifty exploits um we generated the 00:41:38.996,00:41:44.569 total of one thousand seven exploits for fourteen out of the eighty two um challenge sets. 00:41:44.569,00:41:50.241 All of them have one hundred percent reliability and so far as score- like always leaking or 00:41:50.241,00:41:55.313 essentially um crashing at a specific address >>Did you check how many were like mostly 00:41:55.313,00:42:00.952 reliable? >>Um I did not so this essentially seems that we only got fourteen out of eighty two 00:42:00.952,00:42:06.557 challenge sets, we do not know how many essentially Gram Attack with Tech AX and Zandra got or 00:42:06.557,00:42:11.562 Mayhem with four all secure >>The rumors are that we have topic exploitation but we didn't 00:42:15.033,00:42:21.606 have the best game theory >>So like always our SLA sucks >>Our SLA is shit >>And yes so in 00:42:21.606,00:42:25.543 total-, can you back up one slide? These were essentially the exploits where we actually 00:42:25.543,00:42:30.381 generated some uh for >>Actually I- I should say, the caveats of those rumors, is Mayhem was only 00:42:30.381,00:42:35.720 up half the game, and I think they still got almost as many exploits so >>Yeah >>Yeah >>And 00:42:35.720,00:42:39.657 so we got two of the rematch challenges so so- two of the historical challenges that DARPA 00:42:39.657,00:42:45.163 introduced. One of them was SQLSlammer which I think two other teams also got but don't 00:42:45.163,00:42:51.536 quote me on that. And then there was also Crackaddr which supposedly only we got right. 00:42:51.536,00:42:55.940 And then in total if you look at essentially the different challenges that we had and the 00:42:55.940,00:42:59.544 vulnerabilities that were in there, this is the list of challenge sets that essentially 00:42:59.544,00:43:01.479 we got. And with that, from all of us >>Yeah >>Thank you for the attention [applause] >>So, real 00:43:01.479,00:43:06.484 quick, let's talk about the next steps real quick the next steps beyond automated hacking is 00:43:20.431,00:43:25.436 machines augmenting human intelligence so in defcon CTF we hooked up our CRS, Mayhem as the 00:43:28.639,00:43:35.413 winner, they played completely autonomously. We played with our CRS so I mentioned already that 00:43:35.413,00:43:42.353 the CRS actually pwned one binary without us even realizing it it actually assisted us with 00:43:42.353,00:43:47.358 five of the exploits. There were five exploits at which either after providing the crash um or 00:43:49.794,00:43:54.799 after just providing interaction it created an exploit for it. And our CRS inserts backdoors 00:43:57.335,00:44:02.006 into every binary that it patches and so you might of heard already that a lot of 00:44:02.006,00:44:07.011 teams actually use our backdoor. >>This sounds all awesome, but we didn't win even close >>yes 00:44:09.147,00:44:15.253 >>We almost got close to last so >>And- and- and- >>Let's turn down the bragging just a tiny 00:44:15.253,00:44:20.758 bit >>The CRS did amazing, but there were some issues, like for example, the Defcon organizers 00:44:20.758,00:44:26.063 had to implement a separate API for the infrastructure than DARPA did right because the 00:44:26.063,00:44:31.335 DARPA API had to be secret so that you know everyone was on an ev- even playing field. And so 00:44:31.335,00:44:36.941 there were some A- API incompatibilities. And computers are very brittle and so these 00:44:36.941,00:44:42.413 API incompatibilities screwed us until the very last day, so the last day I feel we had a good 00:44:42.413,00:44:47.785 showing up until then the CRS kept crashing, the CRS kept getting invalid data, it was 00:44:47.785,00:44:52.790 kinda touch and go. Um so as you uh might of heard we're going to open source everything. We're 00:44:56.093,00:45:01.032 gonna do [applause] Thank you. We uh we're gonna do a full open source vomit because we believe 00:45:06.504,00:45:11.509 in raising the playing field for everybody so the next time a CGC runs around, rolls around, we 00:45:13.644,00:45:18.649 expect all of you to play as well hopefully using our stuff. So [applause] we don't uh have 00:45:24.655,00:45:30.528 it all ready right now to push to Github because we're playing the CTF, we thought we'd have 00:45:30.528,00:45:34.999 time, but we don't. But Chris, do you think we could do a symbolic open sourcing of 00:45:34.999,00:45:40.004 AngrOp? >>Yeah >>Alright, let's do it. Right on stage uh I'm gonna unplug the video Kevin so 00:45:43.975,00:45:49.213 if Chris isn't logging in on this that means just don't type your password into the wrong 00:45:49.213,00:45:54.218 field [laughter] I've seen that before, at Defcon, it was incredible. It was someone 00:45:58.522,00:46:03.461 fairly famous too. Ah there we go. Better safe than sorry. I think their password was star 00:46:09.266,00:46:14.272 star star star star star star star >>I enable login before pschh >>Caio caio four is what 00:46:16.841,00:46:21.846 Giovanni says. I think that's his password though. Alright so we're gonna plug it back in 00:46:24.181,00:46:29.186 while we try to uh desperately find the settings of the open source project. So AngrOp is our 00:46:31.555,00:46:36.560 rawp compiler so if you're tired of writing return oriented programming payloads by hand you 00:46:39.497,00:46:45.636 can wait- hold on, let me explain what it is, you can uh use AngrOp which uses Angr to 00:46:45.636,00:46:51.308 compile raw payloads into whatever you want. So you say, actually just read this memory, 00:46:51.308,00:46:57.281 or execute the syscall, and it figures out the rawp payload that it needs to generate. Chris 00:46:57.281,00:47:01.652 wrote it, he's an amazing guy and it's an amazing project. And here it is being open sourced 00:47:01.652,00:47:06.657 for the world boom [applause] The rest of the code we need to scrub free of uh private keys 00:47:16.867,00:47:21.872 because there are so depressingly many um and other uh depressing uh things and then 00:47:25.142,00:47:31.549 we'll push it out this week. >>Also if you find a private key that we haven't scrubbed can you 00:47:31.549,00:47:35.886 please gently let us know instead of >>Yes please >>Destroying our infrastructure 00:47:35.886,00:47:40.458 I- we will appreciate it [laughter] >>We're hackers, hackers have some of the worst 00:47:40.458,00:47:45.463 security in the world so and then and my password is six characters long just to give you 00:47:45.463,00:47:50.468 an idea. Alright, Kevin how to I get back to our uh thing? But I think we're done, basically. 00:47:54.939,00:47:59.944 >>Thank you guys! [applause] >>Thank you guys! So stay in touch, hit us up on Twitter, by 00:48:06.016,00:48:11.756 email, jump on our C channel, you can chat with us about our CRS at SHellphish CRS and 00:48:11.756,00:48:17.561 freenode. I'm the only one there right now, it's super exclusive or on Angr at freed node on Angr 00:48:17.561,00:48:22.566 questions. Are there any actual questions? >>Yeah, hi, uh congratulations >>Thank you >>On 00:48:26.971,00:48:31.976 your uh work um so in your driller paper you said that the the fuzzing was mostly 00:48:36.180,00:48:41.185 responsible for sixty eight of the binaries whereas uh having the symbolic execution based 00:48:43.621,00:48:48.626 fuzzing only let you find uh vulnerabilities in eleven more than that so what is- that still 00:48:50.895,00:48:56.400 the case or is the symbolic execution more effective than fuzzing now? >>You want to talk 00:48:56.400,00:49:03.240 about drilling three point o? >>Uh sure, one thing we've done to actually improve [silence] 00:49:03.240,00:49:08.245 one thing- one thing we've done in [laugh] to actually improve uh Driller uh a history on CGC 00:49:11.382,00:49:16.921 binaries is to identify functions and install sym procedures uh in their place so 00:49:16.921,00:49:22.459 what this means is that a lot of basic block transitions which are hard for uh or uninteresting 00:49:22.459,00:49:26.864 for one small execution solve are more interesting a more interesting procedure- we can 00:49:26.864,00:49:31.869 talk about it more if you want to come up here. >>Mic >>Oh last question okay well congrats guys 00:49:35.072,00:49:39.777 >>Thank you! >>First uh second I wanted to know uh how compute bound you felt, like were the- 00:49:39.777,00:49:43.681 did you get enough compute power, too little, too much, would you have put something 00:49:43.681,00:49:49.920 else in there? Back plain ram? What'd you think? >>This uh this point we don't actually know 00:49:49.920,00:49:54.024 because we haven't gotten a chance to actually look through all of the logs um we had some 00:49:54.024,00:49:58.762 problems in the very beginning so actually on Wednesday still to get all of our kubernetes 00:49:58.762,00:50:03.767 parts scheduled simply because kubernetes were not catching up um we kind of solved that but we 00:50:06.103,00:50:10.741 at this point we don't really know what the status is in so far as the utilization of all 00:50:10.741,00:50:16.814 the nodes >>From watching the power consumption, it seemed that the way that it dropped off 00:50:16.814,00:50:21.652 it seemed that it had a lot of unnecessary jobs that it would deschedule later, so I think we 00:50:21.652,00:50:27.925 could have used a little less even and and it will still yeah we could've probably used thirty 00:50:27.925,00:50:33.731 two nodes and done about the same but the more the merrier especially if you can schedule 00:50:33.731,00:50:38.302 more jobs. We definitely had jobs to schedule, that we couldn't schedule, because of 00:50:38.302,00:50:45.175 delays in kubernetes. >>Cool >>Thanks >>Alright thank you >>Thank you, and thank you for 00:50:45.175,00:50:47.177 organizing this thing >>Everybody, please give Shellphish team a huge round of 00:50:47.177,00:50:51.782 applause, what they've accomplished is immense. [applause] >>Thank you guys! It 00:50:51.782,00:50:55.219 was a dream come true to be here. >>Yes