00:00:00.367,00:00:05.372 >> So I would like to introduce, uh, Dan, uh, Petro, uh, his tag is altf4, um, he is here to talk 00:00:08.509,00:00:13.513 about creating a Super Smash Bros. Melee AI that abuses frame perfect inputs, and, uh, which 00:00:16.683,00:00:22.422 makes things really really difficult for humans. Um, and, I... he's going to talk to you 00:00:22.422,00:00:27.427 about how he created it, and it's going to be pretty awesome, so enjoy. >> Cool, thanks a lot. 00:00:33.367,00:00:39.540 [applause] >> Sup defcon. We're going to talk about Melee today, we're going to have some fun. So 00:00:39.540,00:00:45.012 I am Dan, uh, I am a penetration tester at a company called Bishop Fox, um, I do things 00:00:45.012,00:00:48.849 there like hacking web applications, we do security evaluations for like, the 00:00:48.849,00:00:53.854 fortune 1000 high tech start ups, that sort of thing. Um, I also have talked at defcon a 00:00:53.854,00:00:58.292 couple times, uh, last year we, uh, I gave this great talk about, um, hacking smart safes, 00:00:58.292,00:01:01.628 before that I was known for something of a Rickroller, I came up with a little device 00:01:01.628,00:01:07.000 that, um, uh, hijacks the Google Chromecast, uh, and can play arbitrary video to that which 00:01:07.000,00:01:12.005 has to this day not been fixed, um, not because their Google is silly, it's just a low level 00:01:12.005,00:01:17.811 design problem. But that's not really why I'm here. If you're like me, and if you're in this 00:01:17.811,00:01:21.982 room I suspect you are, it's 'cause you're into video games, right, before we got into the 00:01:21.982,00:01:25.886 information security field, before I got into hacking, if you talked to middle school me, 00:01:25.886,00:01:30.991 I was super into video games, right? That was the thing that got me into technology. And so, 00:01:30.991,00:01:35.562 that's always been a side thing that, uh, I've been interested in. Particular, this game. Super 00:01:35.562,00:01:39.900 Smash Bros. Melee. Um, Smash Bros. Melee is not, like, just a video game in that sense, it is 00:01:39.900,00:01:44.171 also an e-sport. Uh, by that it means that there are competitive players, in fact there are 00:01:44.171,00:01:48.342 professional players. Um, you can see in the bottom left hand screen here, uh, those are some, 00:01:48.342,00:01:53.246 uh, competitive e-sports teams that have professional players that do nothing but play this 00:01:53.246,00:01:58.385 game, Smash Bros. Melee, for a living, right, um... uh, there's even more popular games, and 00:01:58.385,00:02:02.689 it's also known as one of the most technically demanding games, so it's very very fast. 00:02:02.689,00:02:05.625 You see in the bottom right hand corner, even though that's not Melee, that's, uh, Street 00:02:05.625,00:02:09.329 Fighter, um, it gives you good example of what they call APM, the actions per minute, just how 00:02:09.329,00:02:14.034 fast and technical the game can be, right. So in addition to like, the high-level strategy of 00:02:14.034,00:02:18.939 what it is that you're our opponent, you also have to worry about the low level intricacies 00:02:18.939,00:02:23.210 of the game in terms of like, how to actually button mash fast enough. So it's not just that 00:02:23.210,00:02:27.414 you're pressing buttons very quickly, but also with very precise timings. So, um, it's 00:02:27.414,00:02:32.819 known as a very demanding game. And has a lot of respect, it's also a very very old game, um, 00:02:32.819,00:02:37.724 Melee has been out for just short of 15 years now, um, the... I think it came out in 00:02:37.724,00:02:43.096 November, so it would be 15 years in just a couple months here. Um, and I am a player. So 00:02:43.096,00:02:47.367 this is me, um, I asked my wife to get me, uh, some of the most embarrassing and socially 00:02:47.367,00:02:52.239 compromising photos that she possibly could of me playing, uh, Smash way back in the day, 00:02:52.239,00:02:56.043 eh, I think I turned out all right. Um, so uh, yeah I've beeen playing the game basically 00:02:56.043,00:03:01.681 since it came out, uh, competitively more or less since that has been a thing, uh, and, 00:03:01.681,00:03:06.119 um, uh, those have been sort of my two loves, right, information security, this is like what I 00:03:06.119,00:03:10.424 wound up doing for career, and was also a big passion of mine as well as playing video games, 00:03:10.424,00:03:16.129 so hey, why not combine them, right? So, the story is, I was playing... I was playing, um, 00:03:16.129,00:03:21.601 some Melee, uh, something like last year with a friend of mine, uh, back in the Arizona Smash 00:03:21.601,00:03:25.105 scenes, because I'm from Phoenix, and, uh, I talked to him afterward and said 'hey, so 00:03:25.105,00:03:29.443 like, what do think a computer could be like if you could play the game frame by frame, like 00:03:29.443,00:03:34.381 perfectly. Um, how good do you think a computer could be?' and he responded to that 'uh, the 00:03:34.381,00:03:38.985 game requires too much high-level strategy, too much, like, mind games, there'd be no 00:03:38.985,00:03:44.958 way that a computer could be really good', so of course I thought, 'fucking challenge 00:03:44.958,00:03:49.663 accepted.' So I then begun on a month's long journey of binary reverse engineering and AI 00:03:49.663,00:03:54.668 research and programming, until I eventually created, um, what is now SmashBot. So, uh, before 00:03:57.437,00:04:01.374 we get into, uh, some live demo stuff, I just want to give you a really brief high-level 00:04:01.374,00:04:07.547 architecture, um, description of like what SmashBot is and how it works. Um, so, uh, there are 4 00:04:07.547,00:04:11.384 major components here that we're going to discuss. Um, first is the Dolphin Dmulator, right now 00:04:11.384,00:04:17.724 at least, um, it works on the Dolphin Emulator which is, uh, runs, uh, GameCube and, uh, Wii 00:04:17.724,00:04:23.530 games. Uh, the, uh... game then, uh, will export all of the relevant game state information, 00:04:23.530,00:04:28.902 so all the like information about how the, like, universe, like, where characters currently 00:04:28.902,00:04:33.173 are and things like that, out to a separate process, so we're not running, we're not modifying the 00:04:33.173,00:04:37.878 in-game AI in any way, SmashBot is its, um, own AI written from the ground up running a separate 00:04:37.878,00:04:44.785 machine, uh... which then does some AI magic for now, um, that we'll get into later... decides 00:04:44.785,00:04:48.388 what buttons to press and then crushes them on a virtual controller. So importantly, 00:04:48.388,00:04:51.725 SmashBot doesn't cheat, it doesn't just like make itself invincible, and it doesn't do 00:04:51.725,00:04:56.229 anything that you in principle couldn't do. So it's just pressing buttons on a virtual 00:04:56.229,00:05:01.501 controller, and looking at the screen in much the way that you would look at it. Alright, so 00:05:01.501,00:05:06.640 before we get too far, we're going to do a live demo. So this is the time for you to come 00:05:06.640,00:05:11.645 right up on stage, right here, and give SmashBot a try. So go ahead and line up, we're going 00:05:13.947,00:05:20.320 to do this, we're uh... you can take like a... we're going to get some... [audience murmuring] 00:05:20.320,00:05:25.325 There we go. [applause] [inaudible] Emulator... I'll just set up the game... 00:05:38.638,00:05:43.643 hopefully we'll get audio... Oh is it? Oh that's okay if we don't have audio for the moment, 00:05:58.258,00:06:03.196 let me just kill the emulator, I should have done this ahead of time... SmashBot... run... Um, 00:06:17.043,00:06:22.048 yep... Do do do, we're going to turn off pause... I'm not currently getting audio but I 00:06:30.090,00:06:35.095 can twiddle with that in just a moment. Okay... And we will begin, here, so, uh, just take 00:06:45.372,00:06:50.377 one stock... Um, oh here, just look right there. Yes. Uh, so, uh, yeah, it just... I just set 00:06:53.413,00:06:58.318 up so that you can, uh, take... so SmashBot is the Fox, um, as you can probably tell here. 00:06:58.318,00:07:03.256 [laughter] This is going to run... >> [inaudible] >> Okay, let me just try to switch the 00:07:05.725,00:07:09.629 audios then. >> Yeah. Sorry about that. >> That's alright. This is going to get in your 00:07:09.629,00:07:14.634 way... sorry, uh, it's on headphones... Just try put it to HDMI? [cheers] Try put it to 00:07:17.370,00:07:22.375 HDMI. Yeah, yeah. Is it testing? >> Front, left. >> It should be playing. Here we'll do it 00:07:25.879,00:07:30.884 afterward. There we go. [applause] Yeah. [laughs] When I take focus away from the window, 00:07:34.054,00:07:39.059 SmashBot stops playing, so that's what you saw. Just play. So, there's a couple things that 00:07:43.396,00:07:48.068 SmashBot is doing right here, I'll talk to it, um, as the game is going, um, so, uh, it's going 00:07:48.068,00:07:53.139 to be trying to take advantage of primarily, uh, of like, human player in two main ways. So it 00:07:53.139,00:07:57.744 does reactions and predictions, right, so reaction is the easiest way to describe how this 00:07:57.744,00:08:03.583 works, um, because, uh, the game will, uh, often require that you commit yourself to some sort of 00:08:03.583,00:08:07.954 an action before you are able to get an attack outright, so you're going to start in a 00:08:07.954,00:08:12.559 forward smash attack, and the very moment, the very exact moment that you start this 00:08:12.559,00:08:16.563 attack it knows how long it's going to take and at what point the hitbox is going to come out, 00:08:16.563,00:08:21.034 and so from there it can predict exactly where to go, um, in order to avoid it or to shield 00:08:21.034,00:08:26.239 the attack, right, and so strictly right from the, the... uh... the ability to react, 00:08:26.239,00:08:31.978 like, frame perfectly to attacks... oh there we go... [cheers] Um, it's able to get 00:08:31.978,00:08:35.682 quite an advantage on a human player, right, um, it turns out that quite a bit of the game, 00:08:35.682,00:08:40.954 um, depends on reaction, uh, and so, uh, SmashBot is able to get himself pretty far entirely on 00:08:40.954,00:08:45.291 the basis of that. However, that's not good enough. Um, the emulator sometimes has a 00:08:45.291,00:08:51.097 trouble, like lagging, but it's basically good enough. The... why does it think...? Oh, okay. 00:08:51.097,00:08:54.601 I'm amazed this is working at all, by the way, I want to give a huge shoutout to Dwango AC who 00:08:54.601,00:09:00.840 gave the amazing task bot talk, who I hope you, uh, got to talk to earlier. [applause] This... 00:09:00.840,00:09:05.712 this whole thing almost didn't happen. [audience member yells] Yeah. So, it gets you in a tech 00:09:05.712,00:09:09.249 chase combo... oh, the emulator's having a hard time. Okay. So this tech chase combo 00:09:09.249,00:09:13.420 you see is Fox will grab Marth and throw him to the ground. And then, uh, from there you do... 00:09:13.420,00:09:18.558 there's only a handful of options that the human player has, um, at that point, uh, 00:09:18.558,00:09:23.096 the... you can like fall to the left, fall the right, and no matter what you do, um, SmashBot 00:09:23.096,00:09:28.101 can in reaction, uh, figure that out, right, he can... [laughs]... he can, uh... uh, 00:09:30.403,00:09:35.675 cover all the options that the human player has. So, uh, the other thing that SmashBot does 00:09:35.675,00:09:41.581 is, uh, prediction. So he's able to look forward into the future, um, in the game state, right, 00:09:41.581,00:09:45.885 and, like, know the physics of the game, know all the attack animations of the game, and then 00:09:45.885,00:09:49.956 once you've committed to some action, um, take advantage of that into the future. Um, so, 00:09:49.956,00:09:53.993 um, depending on whether it actually comes to one neat example of this, um, one of them 00:09:53.993,00:09:57.931 is the rolling, right, the very moment, the very instant that the, uh, opponent starts to 00:09:57.931,00:10:02.969 roll, SmashBot knows precisely where he's going to end up rolling and exacly when. So he 00:10:02.969,00:10:06.339 can just throw out a grab at that exact moment, so there's no way even in principle to get out 00:10:06.339,00:10:11.144 of it. Um, the... it's a little bit more evident in edge guarding situations, uh, when, 00:10:11.144,00:10:14.981 uh, the opponent is off the stage, where the, uh, flowcharting of the, like, 00:10:14.981,00:10:19.085 options, that the opponent has, um, is a bit more apparent there. So we're just going to 00:10:19.085,00:10:22.822 let this run until the, uh, the time out basically, and I can talk a little bit more about, 00:10:22.822,00:10:28.962 um, the... uh... So, um, it should be noted that right now as of this moment the one match 00:10:28.962,00:10:34.767 up that SmashBot knows really well is this one, which is, um, Marth against Fox on FD, and I 00:10:34.767,00:10:38.738 chose that for a very specific reason, right, I wanted to tackle one problem at a time, I 00:10:38.738,00:10:42.308 believe in the engineering principle of just solving one easy problem at a time, right, 00:10:42.308,00:10:47.347 and then eventually try to add support for other characters in other stages, um, so one is that 00:10:47.347,00:10:52.152 Marth, the green, uh, player that's currently getting beat up, uh, is the, uh, is a high 00:10:52.152,00:10:56.723 tier character, so he's a very top tier character amongst, uh, human players, um, so you'll see 00:10:56.723,00:11:00.527 lots of, uh, competitive, even professional players play that character, so it's not simply 00:11:00.527,00:11:04.731 the case that SmashBot is like, beating up on some low tier character, right, um, and in 00:11:04.731,00:11:10.403 fact, this exact match up, the Marth on FD match up, um, is considered almost unwinnable for 00:11:10.403,00:11:14.674 the Fox player amongst high-level players. So this is something that, like, if two 00:11:14.674,00:11:18.611 human players were to be playing, Marth has a massive advantage. So you can take that 00:11:18.611,00:11:23.616 as an, uh, an example of, like, what, you know, is going on here. Let me get a drink. I'll 00:11:27.053,00:11:32.292 try to get some audio after this is finished I guess. Um, so, the, um, uh, some other parts to 00:11:32.292,00:11:37.830 talk about here, um, the... the tech chase combo, um, is basically unavoidable, um, it is 00:11:37.830,00:11:42.302 possible to kind of, like, slide off the stage or something like that, um, but, uh, it's, uh, 00:11:42.302,00:11:47.307 very strong, so... Ah, almost... Um, the, uh, oh he actually missed an up smash... So I'm not 00:11:51.678,00:11:55.582 going to pretend to have created the world's first bug free program, um, there's certainly, 00:11:55.582,00:12:00.186 um, some instances were SmashBot will just sort of derp off the stage or something like that, 00:12:00.186,00:12:05.191 um, it's been an iterative process... No... It can... it knows about that. Um, so, um... 00:12:07.594,00:12:11.631 [laughter] Yeah... Um, if ever you see it actually get hit, um, it's almost surely due to, um, 00:12:11.631,00:12:17.437 what's called the shield stab. So you can see when Fox puts up his shield it's this big blue 00:12:17.437,00:12:21.741 bubble, right, um, as it turns out, the way that Smash works, the... it only shields exactly 00:12:21.741,00:12:26.245 where that blue bubble is, and so if you're able to like, hit his, like, foot, that's kind of 00:12:26.245,00:12:30.984 sticking out of the shield, that will actually land. And it's hard to predict ahead of time 00:12:30.984,00:12:35.188 when that's going to happen. Also Fox will only shield after you've done your attack, so 00:12:35.188,00:12:39.525 first you have to commit to the attack, and then SmashBot will put up the shield, which means 00:12:39.525,00:12:43.162 it's not really reliable for you to try to hit it, it's basically like a random thing that'll 00:12:43.162,00:12:49.435 happen, something like 1 out of 100 times, um, where, uh, like, SmashBot will try to shield, and 00:12:49.435,00:12:54.440 then it just like won't work. [cheers] No... [cheers] Ah... yeah. You can tell when somebody 00:13:00.680,00:13:04.884 actually knows how to play Melee, like, ah, yeah, like, you kind of do a recovery there. Um, 00:13:04.884,00:13:08.554 actually it was funny, for the first maybe like 6 months or something of SmashBot's 00:13:08.554,00:13:13.226 existence, I discovered that, um, the novices, uh, did actually better than competitive 00:13:13.226,00:13:17.196 players, and partly due to the fact that, like, they were just doing weird, random things that 00:13:17.196,00:13:22.335 I hasn't considered, um, whereas like all the competitive players, um, um, like.. like, 00:13:22.335,00:13:27.240 were doing things that I had anticipated, um, or like maybe just, I would never consider 00:13:27.240,00:13:32.412 that I would... oh... okay. Emulator's having a hard time for a moment there. The, um, uh, 00:13:32.412,00:13:36.449 particularly like standing there and slashing, for like the first like 6 months of SmashBot's 00:13:36.449,00:13:38.985 existence like 'I don't know how I'm going to deal with that... I'm just going to come back to 00:13:38.985,00:13:43.923 it and hope that nobody does it.' Um, so, uh, to give you an idea of just how, um, 00:13:43.923,00:13:48.561 competitive, um, SmashBot's become, um, it has... I've been bringing it out in secret to the 00:13:48.561,00:13:53.833 local Smash tournaments in the Arizona scene, and, um, I don't want to like, name some names of 00:13:53.833,00:13:57.570 people that it has beaten because that wouldn't entirely be fair, but, um, something like 00:13:57.570,00:14:02.308 the last, like, 50% of it's matches, JV five-stocks, um, the, uh, the players there, 00:14:02.308,00:14:06.212 which is to say that it doesn't take a single hit throughout the entire match, um, and sometimes 00:14:06.212,00:14:11.317 it'll like randomly take a hit. Um, basically people realize, um, pretty quickly, that you 00:14:11.317,00:14:14.854 can't just fight it, you can't just, like, play it like a normal person as if it were a 00:14:14.854,00:14:19.025 human being, you basically have to try to pentest it right, to try and find some bug, some 00:14:19.025,00:14:22.628 corner case that it's not considering, um, and if you're able to do that, and then 00:14:22.628,00:14:27.400 execute that, like, 4 times in a row in order to like, take 4 lives, then you can beat it. But 00:14:27.400,00:14:31.404 that's just sort of been an iterative process over the last while, um, to try and find all 00:14:31.404,00:14:36.275 those little bugs, um, and then fix them before, you know, bringing out the next thing, so 00:14:36.275,00:14:40.880 um, this is really the first time that I've like shown off SmashBot in any, um, like, major 00:14:40.880,00:14:45.618 way aside from kind of like in secret showing some of my friends, um, so it's definitely 00:14:45.618,00:14:49.388 good enough to, uh, like, you know, show off at this point. Though it's worth saying that 00:14:49.388,00:14:55.495 there's still a lot of work to be done. So yeah, in exactly 20 seconds we'll get back into how 00:14:55.495,00:14:59.999 this whole thing actually works, or how it began, the AI parts of it, the reverse engineering 00:14:59.999,00:15:04.937 parts of it, um... See if we can get one hit in in 7 seconds. Oh no. It's still got you. Oh... 00:15:16.315,00:15:21.320 cool. [applause] Those... it's, uh... it's frame perfect start pressing... Really wants to 00:15:28.694,00:15:33.699 start the next match. I'll go and kill this. We, uh, I'll do, um, another, uh, round at the 00:15:35.802,00:15:41.974 end, uh, during questions, so... Okay. So, uh, now a little bit about, um, the AI. About how 00:15:41.974,00:15:45.711 does SmashBot think, right? How does it decide, like, what buttons to press? Um, it's not 00:15:45.711,00:15:50.683 simply just a series of heuristics, um, so, the... the top level has a 4 tiered 00:15:50.683,00:15:55.421 hierarchy of goals. So at the very top level are, uh, the, like, what I just called goals, 00:15:55.421,00:15:59.192 right, it's the highest level of like, what is SmashBot trying to accomplish? What is the thing 00:15:59.192,00:16:03.029 it's trying to do? So, these are things like 'kill opponent', but it's not always 'kill opponent', 00:16:03.029,00:16:06.365 right, sometimes it just like navigate the menu system, because it wants to like select 00:16:06.365,00:16:10.336 its own character, things like that, right. Um, so, and the way that this works is that the... 00:16:10.336,00:16:14.473 the little bubbles on the right hand side that you see here are actual source code files, right, 00:16:14.473,00:16:19.779 these are like the C++ files, um, then the whole point of the file is to determine the next 00:16:19.779,00:16:25.785 lowest strategy. So the next level is strategies, things like bait or sandbag, like, if our 00:16:25.785,00:16:30.423 opponent just got back, um, from the invincibility, then we mind not want to attack them, we just 00:16:30.423,00:16:36.062 sandbag, um, or bait, um, try to like bait our opponent into a wrong move. And so like, kill 00:16:36.062,00:16:40.533 opponent might choose bait as a, for example, as a strategy. And this basically keeps on going 00:16:40.533,00:16:44.971 down and down, so then, our... we're trying to like bait our opponent into a bad move, we're 00:16:44.971,00:16:49.041 going to weave in and out of their range, hope if they make an attack, and then punish them 00:16:49.041,00:16:54.247 when they do. And so, like, bait might then choose punish, we say, 'aha', like we know that 00:16:54.247,00:16:58.751 this person has exactly 17 frames of lag, say, and so we know that we can run up and give 00:16:58.751,00:17:03.689 them an up smash in that time. And then the very last level is chains, um, chains are like, uh, 00:17:05.858,00:17:09.695 button combinations that Smashers would recognize, things like wave dashing, or dash 00:17:09.695,00:17:13.933 dancing, up smash, things like that right, so these are the lowest possible level of 00:17:13.933,00:17:18.037 abstraction in terms of the actual like button press sequences, and so then the 00:17:18.037,00:17:22.708 punish would say, 'ok, I'm exactly in range, and I'm in the place where I can up smash, so 00:17:22.708,00:17:26.712 let's go ahead and do the up smash', right. And then this is going to change frequently every 00:17:26.712,00:17:31.717 single frame. Alright. So, uh, let's talk a little bit about reverse engineering, right. Um, 00:17:33.920,00:17:39.191 this is something that, uh, was a lot of fun, because, you know, being a penetration tester, um, 00:17:39.191,00:17:45.731 this is sort of up my alley, and so I, uh, there is an awesome, uh, Melee scene of hackers, um, 00:17:45.731,00:17:50.069 people like, uh... uh, Dan Salvato actually has been a huge help as well as some other guys 00:17:50.069,00:17:55.141 there, um, there's an entire, uh, Google spreadsheet that we eventually made about getting 00:17:55.141,00:17:58.511 this sort of information. But, in terms of reverse engineering what SmashBot needs to know is 00:17:58.511,00:18:01.614 it needs to be able to figure out a picture of the universe, right? It wants to be able to 00:18:01.614,00:18:06.052 see the screen in the same way that you do. Um, there's no hidden information in, uh, 00:18:06.052,00:18:10.723 Smash, not in a way like poker is, right, if I make a poker bot and said that it just 'plays by 00:18:10.723,00:18:13.926 like reading your hand, LOL', like that wouldn't be interesting at all right, so 00:18:13.926,00:18:17.897 there's no hidden information, uh, to Smash, it's all just available on the screen. Um, 00:18:17.897,00:18:22.134 that said, uh, how do we actually know where all the pieces are? So, we have to make 00:18:22.134,00:18:27.239 a couple assumptions here. One is that the game does have the game state represented in some 00:18:27.239,00:18:30.910 way. It has to, right? It's got to know where the current player positions are, it's got to know 00:18:30.910,00:18:35.147 what your damage is, and so rather than like parking a camera in front of the screen 00:18:35.147,00:18:39.218 and trying to, like, visualize it that way, I knew right off the bat that would never work, 00:18:39.218,00:18:42.989 right. So we want to be able to get information out of the game. The only trouble is, to the 00:18:42.989,00:18:47.560 game... or to the emulator, the game is a black box. So it doens't actually have any idea 00:18:47.560,00:18:50.596 of what's going on inside of it, it's just a virtual machine basically, right, the same way 00:18:50.596,00:18:56.435 that VMWare or VirtualBox has no idea what's going on inside your, like, Windows VM, it's 00:18:56.435,00:19:02.675 just running op codes and present... uh, presenting, uh, virtualized, uh, interfaces, uh, 00:19:02.675,00:19:08.180 for hardware. So un-black boxing this black box is the reverse engineering that is behind 00:19:08.180,00:19:12.752 SmashBot. In particular we don't actually care a whole lot about code, and more than that we care 00:19:12.752,00:19:17.156 about data. So inside of the game there's going to be a couple of pieces of key data, 00:19:17.156,00:19:21.761 right, things like your exact character position, XY, like, what, uh... what character is my 00:19:21.761,00:19:25.498 opponent, what stage are we on, what damage do we have, I want to be able to take that 00:19:25.498,00:19:30.136 information, figure out where it's stored in memory, and then, uh, ship that off to an external 00:19:30.136,00:19:34.206 process. So it should be noted that like when I started this almost a year ago, none of this 00:19:34.206,00:19:39.712 was like worked on at all, there was a lot of trail blazing involved. So, uh, there was also 00:19:39.712,00:19:44.083 no a great way of doing this reverse engineering, um, there are some tools like Cheat 00:19:44.083,00:19:48.220 Engine, but Cheat Engine wasn't exactly going to do what I needed it to do, um, and a lot 00:19:48.220,00:19:52.425 of, uh, built in debugging functionality to Dolphin, and there is quite a bit, um, also 00:19:52.425,00:19:56.729 wasn't quite going to do what I needed it to do. So, um, most of the debugging functionalities 00:19:56.729,00:20:00.499 about trying to like disassemble code, and again that's not exactly what I'm looking for, I 00:20:00.499,00:20:04.804 don't necessarily care about the code, and that's a good thing, because, um, the GameCube runs 00:20:04.804,00:20:10.242 on PowerPC, um, and I really didn't want to have to learn PowerPC, so, um, what we... what 00:20:10.242,00:20:14.747 I did was take memory snapshots. The GameCube is super old and only has about 24megs of usable 00:20:14.747,00:20:18.751 RAM, there are other RAM, there's like specific video memory, there's like registers, 00:20:18.751,00:20:23.823 but the main system RAM behind the console was only 24megs, um, which means I can just write it 00:20:23.823,00:20:28.294 to a file and then inspect it manually, basically. All the stuff that Cheat Engine does, 00:20:28.294,00:20:33.032 um, I'm just doing more or less by hand. So I had to make a fork of Dolphin, uh, that every time 00:20:33.032,00:20:37.937 it would take a snapshot, um, would write the entire contents of RAM out to a file, and then I 00:20:37.937,00:20:42.942 would just do vbindiffs on the, like, memory instances, right, and so I put the game into a 00:20:42.942,00:20:48.848 known state, say like, I'm going to put may damage to 47, right, so I'll have 47 damage on a 00:20:48.848,00:20:51.650 particular character, and then put the damage... uh, up to... take a snapshot, put the damage 00:20:51.650,00:20:56.655 up to 98, and then just do a search to see what regions of memory have changed from 47 to 00:20:58.824,00:21:04.563 98, right. Um, and that works really really well, um, when the memory regions are stable, um, 00:21:04.563,00:21:10.736 that tends to happen if it's like stack allocated, right. Um, however, not all the information 00:21:10.736,00:21:16.142 is stack allocated, in some case it's dynamic, uh, so in those cases it gets a little more 00:21:16.142,00:21:19.979 complicated. So that tends to be when there's like a struct, so all the player information, um, 00:21:19.979,00:21:24.550 the stuff about like actually what damage you are, like what character position, XY character 00:21:24.550,00:21:29.054 positions, um, are stored in a big struct that's allocated on the heap, and so first you have 00:21:29.054,00:21:33.058 to try to find where the struct is, right. Um, that tends to be pretty easy, um,to we could just 00:21:33.058,00:21:36.529 look for like damages or something like that. Um, and then you have to search for the 00:21:36.529,00:21:41.634 memory, so suppose, um, that was found at a particular adress, then you just scan the entire, 00:21:41.634,00:21:46.739 uh, RAM again, to find out, is there any regions in memory that contain that address? And if so, 00:21:46.739,00:21:52.945 that's probably a pointer to our struct. And now we have a stable pointer to our dynamic memory 00:21:52.945,00:21:56.282 region that otherwise would have been moving around. So this sounds sort of kind of easy, and 00:21:56.282,00:22:00.819 in concept it is, in practice this one's being a total bitch, um, the, some... uh, data 00:22:00.819,00:22:05.825 structures make no sense, it should be um, uh, going without saying that, uh, these data 00:22:05.825,00:22:09.562 structures were never meant to be read in the way that we're reading them, because of course 00:22:09.562,00:22:13.732 this is like 15 year old console game, so why would they have made these data structures to 00:22:13.732,00:22:17.837 make any sense. So sometimes there's, um, floats where there should be integers, because it's 00:22:17.837,00:22:22.107 clearly monotonically increasing, uh, like, value, but fuck it, they gave it a float, 00:22:22.107,00:22:27.613 um... uh, and that took like forever to figure it out, or sometimes there's a... uh, 00:22:27.613,00:22:32.084 there's no consistency to whether things are indexed at 0 or 1, and just sort of like 00:22:32.084,00:22:36.989 figure it out. Um, so. Before we go any further I want to talk a little bit about game 00:22:36.989,00:22:40.826 programming, um, cause if you've done some programming before, this is probably very different 00:22:40.826,00:22:44.830 than what you might have experienced. So there's the concept of a frame, and a frame 00:22:44.830,00:22:50.035 loop, which is very important, so, on the left there you can see in real time, um, Marth 00:22:50.035,00:22:54.506 doing his forward smash attack. So he's just taking his sword and throwing it down super hard, 00:22:54.506,00:22:58.944 and it looks super fast and in fact looks really really smooth, um, when in reality that's not 00:22:58.944,00:23:02.681 how it actually works. When you slow the game way down, and you can see on the right hand side 00:23:02.681,00:23:07.019 there, you can see that it's basically just an animated GIF. And not only is it, like, um, 00:23:07.019,00:23:13.092 kind of choppy, but it, um, uh... the animations are predictable. So at exactly frame 00:23:13.092,00:23:18.364 10, um, on the 10th frame of the forward smash, every single time Marth will be exactly in the 00:23:18.364,00:23:23.402 same position every single time, right, so the game is basically just a finite state machine 00:23:23.402,00:23:27.339 running very very quickly. Um, so the game runs at 60 frames per second which means a single 00:23:27.339,00:23:34.046 frame lasts approximately 16.66 milliseconds, and so the processing looks basically like 00:23:34.046,00:23:37.783 this. You start at 0, it pulls your controller input to see, like, what has the player 00:23:37.783,00:23:43.822 pressed, it runs the game engine, and produces an image on screen, and then keeps looping. 00:23:43.822,00:23:48.594 And that's more or less how the game works and also basically every 3D game works. And so 00:23:48.594,00:23:54.099 what's important here is that it's not just that the game is displaying at 60 frames per 00:23:54.099,00:23:59.104 second, it's that the game engine fundamentally runs at 60 frames per second. So you can 00:23:59.104,00:24:03.842 use this to cause all kinds of really cool bugs. So if you are running very very slowly, right, 00:24:03.842,00:24:07.179 and suppose you're someone who's totally not Mario, and you're trying to get to some 00:24:07.179,00:24:12.251 treasure... that's past a locked door right, if you walk slowly, you might just kind of run into 00:24:12.251,00:24:17.856 the door and not be able to get through it, but if you're moving super super fast, on one frame 00:24:17.856,00:24:22.695 you might be here, on the very next frame you might be here, and then on the very next frame, 00:24:22.695,00:24:26.298 you'll be there, right past the door, and the game will have no idea that you every collided 00:24:26.298,00:24:30.636 with the door, right, because it never... you never touched the door. Uh, one frame you were 00:24:30.636,00:24:35.908 before it and the next frame you're behind it. So this leads to some really cool, uh, bugs, 00:24:35.908,00:24:40.713 uh, like this... Um, so this is, uh, um, actually a task from, uh, Super Mario 64 where you're 00:24:40.713,00:24:43.882 going to see exactly that. This is the very beginning of the game, and Mario's going to go 00:24:43.882,00:24:47.820 switch... go right through, uh, what is supposed to be a locked door, uh, just by going super 00:24:47.820,00:24:52.825 super fast. Um, so uh, hopefully some audio is here, if not its not critical. [Super Mario game] 00:25:02.267,00:25:09.208 Yeah. Yeah so basically what you saw is that he just did some tricky bug thing to go super 00:25:09.208,00:25:12.811 fast and then just zips right through some doors, right. So that's important, not because 00:25:12.811,00:25:15.981 we're going to be doing some, like, zipping through doors, but just to give you an idea of, 00:25:15.981,00:25:21.086 uh... that the game is running with this internal frame loop. So, uh, the game looks a little 00:25:21.086,00:25:25.524 more like this. We're inside the game, there's this looping thing that the, uh, emulator actually 00:25:25.524,00:25:29.395 has no idea about. So, the emulator is just the hardware right, it has no idea that 00:25:29.395,00:25:33.599 there's this internal frame loop, that's the game's business. Um, and whenever it 00:25:33.599,00:25:39.004 receives frames it will go ahead and output them. Um, so, in order to get the game state 00:25:39.004,00:25:43.675 information out, now we have figured out where the, uh, like, bits are inside of the game, 00:25:43.675,00:25:48.347 right. We have to have some mechanism of exporting it out to a separate process, and so my 00:25:48.347,00:25:53.986 first forward... this was really hilarious, so first, uh, I set up a segment of shared memory 00:25:53.986,00:25:58.524 between the Dolphin Emulator, it was a modified version, another fork I made of the Dolphin 00:25:58.524,00:26:03.862 Emulator, um, to, uh, SmashBot. So this is a shared segment of memory, there's no like input 00:26:03.862,00:26:08.167 and output, it's actually just the same memory that's shared between two processes. Um, so 00:26:08.167,00:26:12.604 what I had to do is write some code that took the game memory, um, and copied out the relevant 00:26:12.604,00:26:16.975 data into a struct that's in that game. So, had to, like, move the data out into that 00:26:16.975,00:26:20.446 struct. But of course I don't have, um, any concept of when the frame is running, because 00:26:20.446,00:26:25.684 the emulator doesn't know when the frame is running, so, the natural thing to do here is just 00:26:25.684,00:26:30.222 make a spin loop. So we have one entire CPU doing nothing but spinning, doing absolutely 00:26:30.222,00:26:35.961 nothing but copying data into that, uh, shared memory region. So now SmashBot has this like 00:26:35.961,00:26:41.200 constantly updated real time view of all the game, relevant game state information. But it 00:26:41.200,00:26:45.170 doesn't know when the frame has processed. It has to like... trigger per frame and when a 00:26:45.170,00:26:52.010 frame triggers is one pieces of game, uh, data. So of course, I had to write a second spin loop 00:26:52.010,00:26:56.215 inside of Smash Bot that would regularly check that struct. Um, this is what computer scientists 00:26:56.215,00:27:03.121 would refer to as 'suboptimal'. [laughter] Um, so uh... Eventually this, um, was, uh... 00:27:03.121,00:27:07.292 integrated into the official Dolphin build as of Dolphin 5.0 there's a new feature called, 00:27:07.292,00:27:12.531 uh, Memory Watcher, which does this without the terrible spin loops. So, um, I would like... 00:27:12.531,00:27:18.570 super big thanks to the Dolphin guys for that. Um, so now we have three parts of the whole 00:27:18.570,00:27:21.974 running system. We've got the Dolphin Emulator, we have SmashBot making decisions, we're 00:27:21.974,00:27:28.981 able to pipe that data out, um, over, uh... uh, a named pipe, basically. But it's still not 00:27:28.981,00:27:33.552 playable at this point, because we still can't actually press buttons. And so that was another 00:27:33.552,00:27:38.290 kind of funny instance where my, uh, initial attempt to, uh, press on a kind of virtual 00:27:38.290,00:27:43.061 controller, um, Dolphin didn't have any mechanism for actually doing that, but what it did have 00:27:43.061,00:27:47.399 is the ability to type on a keyboard, so you can, like, map the A key to press the A button 00:27:47.399,00:27:50.536 or something like that, right. So I thought 'okay great, what I'm going to do is I'm going to 00:27:50.536,00:27:56.708 write a, like, helper that uses, um, the XOR, uh, libraries to, like, press the button, like, on 00:27:56.708,00:28:01.513 the... on the keyboard, um, and it actually, like, sort of works, it's terrible and I would 00:28:01.513,00:28:06.985 not recommend it whatsoever, um, partly because if you like move your focus away it just starts 00:28:06.985,00:28:11.323 pressing buttons like into the random window, whatever you, like, gave focus to, and just 00:28:11.323,00:28:16.528 goes haywire and it's, like, hard to cancel, um, but also because, uh, basically all these 00:28:16.528,00:28:19.565 mechanisims are going to be buffered input, and so there's going to be some indeterminate 00:28:19.565,00:28:23.769 amount of latency from when it presses the button to when it actually happens. And normally 00:28:23.769,00:28:27.906 you don't care about this, if you're just a human being like pressing buttons on a keyboard, 00:28:27.906,00:28:31.810 it doesn't matter to you if when you press the A button it doesn't happen for the next 30 00:28:31.810,00:28:36.081 milliseconds, or maybe the last couple of buttons get buffered together, like, you just don't 00:28:36.081,00:28:40.719 care, you're just incapable of physically noticing that. But SmashBot cares. It needs to be 00:28:40.719,00:28:44.222 able to have exactly frame perfect accuracy on all the button presses, it needs to get 00:28:44.222,00:28:50.762 there super fast. So, um, eventually wound up getting that integrated in with Dolphin as 00:28:50.762,00:28:56.969 well, so now we have a mechanism for pressing buttons. So, about programming. Uh, if you're 00:28:56.969,00:29:00.839 anything like me, uh, programming looks a little bit like this, where, uh, you're 00:29:00.839,00:29:05.611 more or less in a constant state of confusion, um, because if you understand the problem that 00:29:05.611,00:29:09.581 you're trying to, like, trying to, uh, program, thing is you can just solve it very quickly 00:29:09.581,00:29:14.252 and move on to the next problem, and so to be a programmer is to be in a constant state of 00:29:14.252,00:29:20.592 confusion, interrupted only shortly by tiny bursts of, like, epiphany and coding things up. 00:29:20.592,00:29:23.929 So if you were to walk up to me at any point when I'm programming SmashBot, usually 00:29:23.929,00:29:28.133 the Saturday morning eating some, like, breakfast cereal and drinking some tea, you say 'hey 00:29:28.133,00:29:32.270 Dan, how's it going?', I'm like, 'I have no idea what the fuck's going on. Nothings working...' 00:29:32.270,00:29:35.474 [laughter] 'Nothing's working and I have no idea why.' So I had to give you one cool 00:29:35.474,00:29:41.013 example, uh, of what this looked like, so for the longest time in SmashBot's history, um, up until 00:29:41.013,00:29:44.916 maybe a couple months ago, there was just this nagging bug, that I just had no idea how it 00:29:44.916,00:29:48.620 worked. It was just like the only logical explanation for it was that there was a gremlin 00:29:48.620,00:29:53.558 inside of my computer pulling on wires, and so it looked something like this where like 00:29:53.558,00:29:58.630 SmashBot would be totally cool, and then just derp right off the stage. [laughter] And like, what 00:29:58.630,00:30:02.501 is going on here? There's no reason for it to do this, I couldn't pinpoint in code why 00:30:02.501,00:30:06.271 this was happening, and it manifested itself in all kinds of ways, it wasn't just derping 00:30:06.271,00:30:10.809 off the stage. So I implemented this entire debugging mechanism, where, um, I could... you give 00:30:10.809,00:30:15.847 it a --debug flag, it will take the entire game state, per frame, and write it out to a big 00:30:15.847,00:30:20.118 CSV file, um, that winds up being like megabytes large. It's actually the best thing I ever 00:30:20.118,00:30:24.089 did in terms of debugging, because this lets you retroacively walk through what 00:30:24.089,00:30:28.026 happened throughout the entire game, and see like, oh yeah, like it pressed this button when 00:30:28.026,00:30:31.763 it should have pressed this button or whatever, right. Um, so I could see in here that 00:30:31.763,00:30:36.068 sometimes, not all the time, just randomly, seemingly, um, I would press a button and it just 00:30:36.068,00:30:40.072 wouldn't happen for a frame late. That was the source of the bug, I finally figured out, 00:30:40.072,00:30:43.675 okay, so like, there's, for some reason it's pressing a button a frame late. I don't know why, 00:30:43.675,00:30:47.846 and it was only ever one frame late, and not all the time. It was super weird, I was kind of 00:30:47.846,00:30:52.684 chalking it up to uh, a Dolphin bug maybe, there was some bug in the emulator. And so, um, 00:30:52.684,00:30:56.455 eventually I tried out this, this is, um, what you're going to see here is Fox doing frame 00:30:56.455,00:31:00.826 perfect multi shines, uh, this is, uh, it's not just doing these blindly it's actually 00:31:00.826,00:31:05.097 reacting, so on exactly the third frame of the jump animation, Fox is going to hit 00:31:05.097,00:31:09.034 down B to start the shine, the little flashy animation, and then jump out of it, and then 00:31:09.034,00:31:12.804 loop through it again. So what's important here is if he's even a single frame late on any of the 00:31:12.804,00:31:18.009 inputs, he will jump accidental. So it looks like this. So he's going along happily doing frame 00:31:18.009,00:31:23.014 perfect multi shines... and then... start jumping, he's screwing it up. And then... Go 00:31:26.017,00:31:30.322 right back to multi shining again. And then you start to notice that this is actually 00:31:30.322,00:31:35.327 cyclical, this is like not random, this is happening, um, on a, uh, exact like predictable 00:31:35.327,00:31:40.332 basis. So he'll do it again in just a moment here. That's weird. I do like it when bugs 00:31:43.201,00:31:49.274 are reproducible, so eventually, um, me, and uh... Dan Salvato, another awesome Melee hacker, 00:31:49.274,00:31:54.880 um, figured out that this picture of how the game input thing works is not entirely 00:31:54.880,00:32:00.018 accurate. And so what happens is that the game, um, input, and the game engine processing are 00:32:00.018,00:32:05.857 on separate threads. And their not perfectly synced up, and so what happens is on one frame, 00:32:05.857,00:32:10.395 uh, it'll look like this, and the very next frame the controller input will drift by a 00:32:10.395,00:32:15.767 tiny bit... and then the next frame the controller will drift a tiny bit, until eventually 00:32:15.767,00:32:20.305 they swap, the controller input is pulled afterwards. So in the very beginning you're going to 00:32:20.305,00:32:25.410 press a button, right, at 0 right, the game... the game will process without having read your 00:32:25.410,00:32:29.347 input. Then it'll read your input and not process it until the frame afterward, until 00:32:29.347,00:32:33.285 eventually it would drift backwards. Um, so then we put together a patch, well I should 00:32:33.285,00:32:38.523 say Dan put together a patch, um, for, uh, actually, like, fixing this, so you move the 00:32:38.523,00:32:42.928 controller input, um, routine onto the same thread as the game engine basically, so, um, and 00:32:42.928,00:32:48.900 that way we patched live in memory a 15 year old bug in the game that up to this point no 00:32:48.900,00:32:53.905 one has ever noticed. So that was pretty cool. Um, so, some of the bits about the future, as it 00:32:56.608,00:33:00.278 were, um, the... I wish I could have gotten this working for defcon, it's like... it's like 00:33:00.278,00:33:06.785 75% working, is, uh, running on live unmodified console, so as it turns out, um, this, uh, is 00:33:06.785,00:33:12.290 actually completely possible, um, and I was talking with, uh, uh... Dwango who did the task 00:33:12.290,00:33:15.961 bot talk right before this about how... some of the parts, about physically, like how do you send 00:33:15.961,00:33:20.265 button presses over the, uh, the console. But one of the more interesting stories is actually, 00:33:20.265,00:33:24.202 um, how to get information out of the console, so remember, I want to do this on an unmodified 00:33:24.202,00:33:28.874 console, so I don't want to just like put some like leads and open up the hardware, open up 00:33:28.874,00:33:33.278 the GameCube or something like that, right. Um, and so in order to do that we have to use a 00:33:33.278,00:33:38.750 really fun exploit, uh, through the memory card port. So it turns out that, um, uh, when in 00:33:38.750,00:33:42.287 Melee you're able to give yourself a little name tag, like uh, what my name is thats 4 00:33:42.287,00:33:46.424 characters, and of course because its 4 characters people name it lots of colorful things, 00:33:46.424,00:33:52.397 but, um, if you go into the actual save file that's on the memory card port and change your 00:33:52.397,00:33:56.868 name manually to be longer than 4 characters overflows the thing. And you can get code 00:33:56.868,00:34:00.639 execution on the game. And so there's already people that have been exploiting this and using 00:34:00.639,00:34:04.376 them to make modifications to the game, um, if you've ever seen the 20xx, uh, hack pack, or 00:34:04.376,00:34:09.781 20xx... actually no, 20xx tournament edition is the one that uses this name tag 00:34:09.781,00:34:14.319 overflow. So it's a great way of getting code execution on the game, which we can then use to, 00:34:14.319,00:34:19.724 um, grab game state information, and ship it off over the, uh, memory card port, which is then 00:34:19.724,00:34:26.231 attached over USB to a laptop. That way we can get live, um, frame data out the live, like... 00:34:26.231,00:34:29.935 running machine. So we then put SmashBot inside of the controller that would then like 00:34:29.935,00:34:33.538 be pressing buttons, um, and, uh, you'd just sort of be looking like you're playing the 00:34:33.538,00:34:37.842 game like normal, and you would never notice that it was a computer playing, unless you 00:34:37.842,00:34:41.947 like look closely and notice that SmashBot, the controller was plugged into the memory card 00:34:41.947,00:34:46.952 port instead of the controller port. Or probably both actually. Um, so before we start getting 00:34:49.421,00:34:53.792 back to the, um, the end part here, I want to, uh, empart you a little bit of Smash 00:34:53.792,00:35:00.231 philosophy, so, being a part of any, uh, like, competitive scene for sure I'd amuse you with a 00:35:00.231,00:35:04.536 certain amount of the philosphy of that game, and so I want to share this with the, uh, the 00:35:04.536,00:35:09.541 hacker world. [video sounds] >> I don't want to hit no Johns! >> John >> John >> John >> What's a 00:35:18.550,00:35:24.356 John? >> 'It was like 2am and he was tired.' >> Johns, like Johns... >> He'll get me on a 00:35:24.356,00:35:28.627 day where I'm just not playing too well. >> Johns. Just Johns. >> A lot of people don't know 00:35:28.627,00:35:32.831 where the term came from, it just started, but I believe it was a guy in Texas, his name was 00:35:32.831,00:35:37.235 John, and no matter what, every time he'd lose he'd have an excuse, he'd have a reason for 00:35:37.235,00:35:41.406 losing. >> 'My controller wasn't working, the stage, there's a little bit of lag on the TV.' >> 00:35:41.406,00:35:48.113 'I didn't sleep last night, or I don't know... >> 'It's too cold' >> We have a... like a Swedish 00:35:48.113,00:35:54.819 term, [Swedish], it's pretty much 'no Johns'. [inaudible] >> My favourite one I think was, 00:35:54.819,00:35:59.824 uh, I was playing somebody and they were like 'someone's touching my shoulder', and I was 00:36:07.766,00:36:12.771 like 'no Johns!' [music] >> So yeah, no Johns. And, thanks a lot. [applause] [inaudible] You 00:36:24.549,00:36:27.585 got something along the lines of 8 minutes, so, uh, go ahead and start the game up again, if you 00:36:27.585,00:36:30.455 want to line up here, I guess we'll have to do two lines, one for playing the game and one for 00:36:30.455,00:36:34.392 questions, and uh, we'll do that, uh, right now. Uh, if you want to take questions there's a 00:36:34.392,00:36:37.395 microphone right there, you're going to have to... otherwise I'm not going to hear anything 00:36:37.395,00:36:42.400 you're saying. Uh, here let me set up the game first actually. >> Hello. Can we see SmashBot 00:36:49.841,00:36:54.145 versus SmashBot? >> Ah yeah, so that's a question that people actually ask a lot, um, is like, 00:36:54.145,00:36:58.616 what would happen if SmashBot played itself, or like, um... the... right now, there's just a 00:36:58.616,00:37:02.821 small logisitical problem with it, which is just that, um, it only knows, uh... its... it 00:37:02.821,00:37:07.425 plays on player controller port 2, and it assumes its opponent is on player controller port 1, 00:37:07.425,00:37:11.362 and so there's just that, but, I think I can get that solved. But the more interesting question 00:37:11.362,00:37:16.668 is, like, what would happen if it played itself, or what does, um, like, truly perfect, um play 00:37:16.668,00:37:21.673 look like. Let's just give it several minutes. Here. Um, so, uh, turns out that optimal play, 00:37:24.642,00:37:27.812 um, I gave quite a bit of thought to this, um, is, uh, really really complicated. And 00:37:27.812,00:37:30.415 this is actually a good question, so in case you're sitting in the audience 00:37:30.415,00:37:34.352 thinking, like, 'hey, I bet I could make a... like a better SmashBot that would like beat 00:37:34.352,00:37:39.124 this one, right?' Well. Let me take you on a tour of what actual optimal play looks like, 00:37:39.124,00:37:42.827 right. So first off, all projectiles can be reflected, there's a 2 frame window at 00:37:42.827,00:37:47.465 which point you can reflect projectiles, and so all those are suboptimal. And so the only 00:37:47.465,00:37:51.870 way to, uh, like, attack is to just basically walk forward. And so the fastest move in the game 00:37:51.870,00:37:56.674 is shine, which is, um, Fox's down B attack, and uh, both... both, um, uh... both Foxes, both 00:37:56.674,00:37:58.676 Bots, would basically walk at each other until they're exactly within range, and then both use 00:37:58.676,00:38:00.612 their perfect one frame move at the exact same moment. They would clang off each other, not 00:38:00.612,00:38:05.617 hit, um, and then it's a deadlock from there. At each point both the, uh, optimal play 00:38:11.890,00:38:17.095 for both characters is to jump and then do frame perfect multi shines until the time limit runs 00:38:17.095,00:38:22.934 out. When the time limit runs out, the game goes into sudden death. At sudden death, like, 00:38:22.934,00:38:26.738 uh, the game goes for a little while and then bomb-ombs start falling from the stage, um, sort 00:38:26.738,00:38:30.542 of like randomly, right, and so it would be possible to put your opponent in a position where 00:38:30.542,00:38:33.878 they have to either run at you and attack or into the bomb-ombs, so it would be kind 00:38:33.878,00:38:38.016 of sort of random. But they're not actually random, right, it just uses in game's random 00:38:38.016,00:38:43.154 number generator, which is entirely predictable, so back up, the optimal strategy is not 00:38:43.154,00:38:47.392 just simply run at your opponent and shine, it's to put yourself in a position where once you 00:38:47.392,00:38:51.830 deadlock your opponent into that shining, you know that in exactly 8 minutes from now the 00:38:51.830,00:38:57.402 random number generator will be seeded such that... the bombs will fall in a way such that, 00:38:57.402,00:39:02.807 you can put them in a... disadvantageous position. So, before you go around thinking 00:39:02.807,00:39:07.812 'I'm going to make the perfect bot', know what you're getting yourself into. [laughter] Yes. 00:39:10.348,00:39:14.652 [audience member yells inaudibly] Uh, no and no. Uh I'm sorry, take the questions with 00:39:14.652,00:39:21.559 the microphone. Yeah. >> Have you taught him to do taunts at the most insulting times? >> 00:39:21.559,00:39:25.063 [laughs] Yeah, was that uh... it does do, um, uh, taunts... that was like, it didn't do taunting 00:39:25.063,00:39:29.300 for the longest time, it just sort of sat around, um, uh, but now it does frame perfect multi 00:39:29.300,00:39:32.737 shines in between stocks, as the like, [inaudible] taunts, basically. I figured that would 00:39:32.737,00:39:37.742 be a pretty cool way to do it. Yeah. >> So you mentioned that, um, the beginner players will 00:39:42.680,00:39:46.284 like, confuse it, so how do you get around it, do you use machine learning, or do just 00:39:46.284,00:39:50.355 keep on programming... >> Yeah, there is actually a separate, um, machine learning fork of 00:39:50.355,00:39:54.659 Dolphin called Phillip, I wish I had more time to talk about it here, that uses the Google's 00:39:54.659,00:39:59.230 tensorflow neural network library, um, at first it had a really hard time doing more than 00:39:59.230,00:40:03.601 just kind of moving around, um, but its actually getting pretty cool now. Um, and so one of the 00:40:03.601,00:40:08.439 neat parts about, uh, SmashBot's design, is that that was like the lowest level, like, chains, 00:40:08.439,00:40:12.076 like maybe there's no need to make an AI learn how to wave dash all on its own, right, why 00:40:12.076,00:40:16.814 don't we program that in as a primitive and then use, um, AI to kind of choose which lowest 00:40:16.814,00:40:20.985 level primitive will be best. And so that's actually like a goal of mine for the project, is 00:40:20.985,00:40:26.057 to do exactly that, um, this is about as far as I've taken like right now, but it's actually, 00:40:26.057,00:40:31.562 um, uh... Actually I should have mentioned this is an active open source project, it's available 00:40:31.562,00:40:35.400 on my GitHub, just github.com/altf4... um, or just Google for this basically and 00:40:35.400,00:40:42.073 you'll find it. Yeah. >> Um, you mentioned that with the game in the future you had plans to have 00:40:42.073,00:40:46.411 this run on an unmodified console... >> Yeah. >> Um, do you anticipate that you'll be 00:40:46.411,00:40:51.382 able to overcome, like, was it strictly on the emulator side with, uh, the drift problem, 00:40:51.382,00:40:56.454 with the controller that was causing the bug where... >> So, this is actually a bug in the 00:40:56.454,00:41:00.558 game, that isn't like... So the game is responsible for that frame loop and, uh, controller 00:41:00.558,00:41:05.596 pulling, so that is actually a bug in the game, um, that said, we haven't been able to 00:41:05.596,00:41:12.003 empirically verify that, right, so in theory that bug should be present on console, um, but, uh, 00:41:12.003,00:41:15.807 without Smash... there's really very difficult... there's basically no way to know 00:41:15.807,00:41:20.812 without, um, like, verifying that via, uh... um, like maybe some task way of doing it, but 00:41:23.214,00:41:27.185 SmashBot would actually be the best way of verifying that because it is reacting in real 00:41:27.185,00:41:30.688 time to the frames rather than just like having a script instead of button presses, so... 00:41:30.688,00:41:34.158 >> Do you anticipate that there will be some way to maybe overcome that so that you can, 00:41:34.158,00:41:38.529 uh... >> Oh, exactly, so we can code execution on the game, right, so we can just modify the 00:41:38.529,00:41:43.534 running game to fix the bug, just patch it live. >> Okay. >> Yeah. >> Thank you. >> Hey, uh, 00:41:46.137,00:41:49.640 first question, are you coming to Super Smash Con? >> I'm not, I actually um... >> Ah that's 00:41:49.640,00:41:52.977 too bad. >> ... uh, only discovered that Smash Con existed, um, after the CFP 00:41:52.977,00:41:56.381 closed... >> Ah that's too bad. >> Um, so I really would have liked to have done that >> That 00:41:56.381,00:42:00.718 would have been a great SuperSmash Con. >> Um, I'm based out in the Phoenix area, so if 00:42:00.718,00:42:03.988 ever you want to, like, play SmashBot near me... Um, if you want to run it yourself just, 00:42:03.988,00:42:08.926 you know, download the source code, run it yourself, otherwise I'll be around in the kind of 00:42:08.926,00:42:12.830 Phoenix are. I'm hoping to take this out to a larger tournament sometime in the near future, but 00:42:12.830,00:42:16.834 I have a, you know, busy travel schedule with work and stuff like that, so no promises. >> 00:42:16.834,00:42:20.104 The other thing was, I just wanted to say thank you so much for figuring out that bug, the, 00:42:20.104,00:42:24.342 uh, 3.5 to 5 frame thing, because I actually had an idea for a project a long time ago 00:42:24.342,00:42:28.880 where I was like, alright, I'm going to take a high FPS camera, sauder an LED to a controller 00:42:28.880,00:42:32.784 and figure out the amount of input lag difference... >> Yeah. There's so many problems that 00:42:32.784,00:42:36.788 happen in the analog world, that like, there's... yeah, it's really difficult. Put it this 00:42:36.788,00:42:40.158 way it doesn't matter what's going on on the screen, SmashBot's reading the live bits 00:42:40.158,00:42:44.395 out of memory. So the very frame that something happens it knows about it with taking the entire 00:42:44.395,00:42:49.700 analog universe of display, refresh rates out of the equation. >> Yeah, but um, 00:42:49.700,00:42:52.737 it's... it's so awesome that you guys figured out that bug, and I was just wondering what went 00:42:52.737,00:42:56.674 into it, 'cause like, I would have been, like, really freaking confused, and I have measured... 00:42:56.674,00:43:00.111 >> Oh I was really freaking confused. >> I have measured the FPS lag and you can see it, it's 00:43:00.111,00:43:04.582 like every, like quarter frame, like something like that, it just takes longer, and it makes 00:43:04.582,00:43:11.556 no sense. >> Yep, that is absolutely correct. >> So, thank you. >> You bet. >> I had a 00:43:11.556,00:43:17.261 couple questions, um, so can it be any... can it be anyone else other than Fox or does it have 00:43:17.261,00:43:22.166 to be Fox? >> Um, Smash ot plays Fox and probably will for the indefinite future, um, it's 00:43:22.166,00:43:26.404 clearly at this level, like, at the task level, um, the best character in the game. It's just 00:43:26.404,00:43:31.476 faster than every other character. Um, one could make an argument for Falco, but I'm not 00:43:31.476,00:43:35.413 so sure... I mean, it is kind of an open question about what is optimal play, like at the 00:43:35.413,00:43:39.250 highest levels, um, I'm sure it was... maybe if you could play this fast Donkey Kong is like 00:43:39.250,00:43:43.221 super broken, I don't know right, I doubt it, there's good reason to believe that the Fox 00:43:43.221,00:43:47.592 is the best character, um, and so this is my best stab at making that happen. >> And what 00:43:47.592,00:43:54.098 about having SmashBot play, like, 3 other characters at once? >> Simultaneously? Yeah, 00:43:54.098,00:43:59.036 so right now it only acknowledges the existence of the... of player 1, because I 00:43:59.036,00:44:03.241 just wanted to make that work first, um, I suspect that's just a losing battle, like once you 00:44:03.241,00:44:07.478 actually have 3 v 1, and the theoretical level you just lose. 'Cause even though I can frame 00:44:07.478,00:44:12.950 perfect shield stuff, um, there's lag after the shielding, and so, like, you could just hit 00:44:12.950,00:44:18.389 me after that happens. >> And one last question, is there any possible plans for other 00:44:18.389,00:44:21.893 fighting games that you would use this for? >> Um, Smash is really the only game that I 00:44:21.893,00:44:27.698 personally play competively, like at that level, um, so... not for me, but there actually 00:44:27.698,00:44:32.203 are other similar projects, other AI's for other games, there are StarCraft and 00:44:32.203,00:44:37.441 StarCraft II, um, AI tournaments that actually happen, um, so there's very similar, um, sort 00:44:37.441,00:44:42.446 of endeavors in that world. >> So you said that, um, this, uh... uh, bot is supposed to be 00:44:44.749,00:44:50.488 able to mimic human behaviour, uh, is supposed to be more... it's... you... if you look at 00:44:50.488,00:44:57.295 it... my question is, I notice whenever you die, it goes left and right really really fast... 00:44:57.295,00:45:01.866 >> Yeah >> Um, was that on purpose? Was that just... >> Yeah so it does... we just call 00:45:01.866,00:45:06.003 it dash dance, right, it just moves... moving back and forth, for the first, um, uh, it 00:45:06.003,00:45:10.141 depends on each character, I think its 7 frames, or maybe 10 or 11 or something like that for 00:45:10.141,00:45:14.812 Fox, um, the first... when you start up running you're at a dashing animation at which point 00:45:14.812,00:45:18.716 you can dash backwards very quickly, um, it's a good way of keeping mobility basically, it's 00:45:18.716,00:45:22.687 something that even like high level players do, but never with that exact amount of precision 00:45:22.687,00:45:28.125 and that amount of speed, um, I guess to... the earlier point, SmashBot is not, uh, intended 00:45:28.125,00:45:32.630 to, um, like, make you feel better, it's not meant to play like a human, it's meant to play 00:45:32.630,00:45:38.302 like a computer in the same way that like an aim bot, right, for like a shooter, does not play 00:45:38.302,00:45:41.906 like a human would. Um, and so we're trying to break fundamentally how the game is 00:45:41.906,00:45:46.410 played at that level. So, um, if you're playing like a shooter game with like team based 00:45:46.410,00:45:50.948 strategy, there's a lot of high level thoughts in terms of, like, getting your opponent to 00:45:50.948,00:45:55.219 use... or you using cover, getting your opponent to move into the center stage, but if 00:45:55.219,00:45:59.857 you're a computer, optimal strategy is to stand in the center of the stage, spin 360 as 00:45:59.857,00:46:03.394 fast as you can, and then blam people in the forehead the very moment they come out, right, and 00:46:03.394,00:46:07.031 so SmashBot's kind of taking advantage of that, right, in that it's not trying to play 00:46:07.031,00:46:10.801 like you do, it's trying to play like a computer does. >> Thank you, and thank you for letting 00:46:10.801,00:46:15.806 me try it out. >> Absolutely. [inaudible] >> I don't know, how... is that time? >> Yeah. >> 00:46:20.044,00:46:25.049 Alright thanks a lot for coming out. [applause]