So, I would like to introduce, uh, Dan, uh, Petro. Uh, his tag is Alt4. Um, he is here to
talk about creating a Super Smash Brothers Melee, uh, AI that abuses frame perfect inputs
and, uh, which makes things really, really difficult for humans. Um, and, uh, he's gonna talk
to you about how he created it and it's gonna be pretty awesome. So, enjoy. Cool. Thanks a lot.
What's up, Def Con? We're gonna talk about Melee today. We're gonna have some fun. So, I am Dan, uh, I, uh, am a penetration tester at a company called Bishop Fox. Um, I do things there like hacking web applications. We do security evaluations for, like, the Fortune 1000, high-tech startups, that sort of thing. Um, I also have, uh, talked at Def Con a couple times. Uh, last year, we, uh, gave this great talk about, um, hacking smart safes. Before that, I was known for something of a rickshaw.
Uh, I came up with a little device that, um, uh, hijacks the Google Chromecast, uh, and can play arbitrary video to that, which has, to this day, not been fixed. Um, not because the Google was silly, just a, just a low-level design problem. But that's not really why I'm here. If you're like me, and if you're, if you're in this room, I suspect you are, it's because you're into video games, right? Before we got into the information security field, before I got into hacking, if you talked to middle school me, I was super into video games, right? That was the thing that got me into technology.
And so, that's always been a side thing that, uh, I've been interested in. In particular, uh, this game, Super Smash Bros. Melee. Um, Smash Bros. Melee is not, like, just a video game in that sense. It is also an e-sport. Um, by that, it means that there are competitive players. In fact, there are professional players. Um, you can see in the bottom left-hand screen here, uh, those are some competitive e-sports teams that have professional players that do nothing but play this game, Smash Bros. Melee, for a living, right? Um, uh, there's even more popular games, and it's also known as, uh, one of the most popular games in the world.
It's one of the most technically demanding games. So, it's very, very fast. You can see in the bottom right-hand corner, even though that's not Melee, that's Street Fighter. Um, it gives you a good example of what they call APM, uh, actions per minute. Just how fast and technical the game can be, right? So, in addition to, like, the high-level strategy of what it is that you're going to do to your opponent, you also have to worry about the low-level intricacies, um, of the game in terms of, like, how to actually button mash fast enough. So, it's not just that you're pressing buttons very quickly, but also with very precise timings. So, um, it's known as a very demanding game.
Um, and, uh,
it has a lot of respect. It's also, um, a very, very old game. Um, Melee has been out for just short of 15 years now. Um, the, I think it came out in November, so it would be, uh, 15 years and just a couple months here. Um, and I am a player. So, this is me. Um, I asked my wife to get me, uh, some of the most embarrassing and socially compromising photos that she possibly could of me playing, uh, Smash way back in the day, and I think I turned out all right. Um, so, uh, yeah, I've been playing the game basically since it came out, uh, and competitively, more or less, since that
has been a thing.
Uh, and, um, uh, those have been sort of my two loves, right? Information security, this is, like, what I wound up doing for a career and is also a big passion of mine, as well as playing video games. So, hey, why not combine them, right?
So, the story is, I was playing, I was playing, um, some Melee, uh, something like last year with a friend of mine, uh, back in the, uh, Arizona, uh, Smash scene, since I'm from Phoenix. And, uh, I talked to him afterward and said, hey, so, like, what do you think a computer could be like if you could play the game frame by frame, like, perfect?
Um, how good do you think a computer could be? And he responded that, uh, the game requires too much high-level strategy, too much, like, uh, mind games, and there'd be no way that a computer could be really good. So, of course, I thought, fucking challenge accepted.
So, I then begun on a month's long journey of binary reverse engineering and AI research and programming until I eventually created, um, what is now SmashBot.
So, uh, before we get into, uh, some live demo stuff, I just want to give you a really brief high-level architecture, um, description of, like, what SmashBot is and how it works.
Um, so, uh, there are four major components here that we're going to discuss. Um, first is the Dolphin emulator, right now, at least. Um, it works on the Dolphin emulator, which is, uh, runs, uh, GameCube and, uh, Wii games.
Uh, the, uh, game then, uh, will export all the relevant game state information.
So, all the, like, information about how, uh, the, like, universe, like, where characters currently are and things like that out to a separate process.
So, we're not running, we're not modifying the in-game AI in any way.
SmashBot is its own, um, AI written from the ground up running a separate machine, uh, which then does some AI magic for now, um, that we'll, uh, get into, uh, later, decides what buttons to press, and then presses them on a virtual controller.
So, importantly, uh, SmashBot doesn't cheat.
It doesn't just, like, make itself invincible, and it doesn't do anything that you, in principle, couldn't do.
So, it's just pressing buttons on a virtual controller, and looking at the screen in much the way that you would look at it.
All right.
So, before we get too far, we're gonna do a live demo.
So, this is the time for you to come right up on stage right here and give SmashBot a try.
So, go ahead and line up.
We're gonna do this where, uh, uh, you can take, like, uh, we're gonna get some...
There we go.
One sec.
Let's do this.
Restart emulator.
Just set up the game.
Hopefully, we'll get audio.
Okay.
Okay.
Okay.
Okay.
Okay.
Okay.
Okay.
Okay.
Okay.
Okay.
Okay.
Oh, is it?
That's okay if we don't have audio for the moment.
Let me just kill the emulator.
I should have done this ahead of time.
Smashbox, run.
Yep.
We are going to turn off pause.
I'm not currently getting audio,
but I can twiddle with that in just a moment.
Okay.
And we will begin here.
So just take one stock.
Oh, dear, just look right there.
Yes.
So, yeah, I just set it up so that you can take...
So Smashbox is the fox, as you can probably tell here.
This is going to run...
Okay, let me just try to switch the audios on.
That's all right.
This is going to get in your way.
Sorry.
It's on headphones.
Should I put it to HDMI?
Should I put it to HDMI?
Yeah, yeah.
Is it testing?
Front, left.
It should be playing.
Yeah, we'll do it afterward.
There we go.
Yeah.
When I take focus away from the window,
Smashbox stops playing.
So that's what you saw.
Just play.
So there's a couple things that Smashbox is doing right here.
I'll talk to it as the game is going.
So it's going to be trying to take advantage
primarily of the human player in two main ways.
So it does reactions and predictions, right?
So reaction is the easiest way to describe how this works
because the game will often require
that you commit yourself to some sort of an action
before you are able to get an attack out, right?
So you're going to start in a forward smash attack
and the very moment, the very exact moment
that you start this attack,
it knows how long it's going to take
and at what point the hitbox is going to come out.
And so from there, it can predict exactly where to go
in order to avoid it or to shield the attack, right?
And so strictly right from the ability to react,
like frame perfectly to attacks.
Oh, there we go.
It's able to get quite an advantage on a human player, right?
It turns out that quite a bit of the game depends on reaction.
And so Smashbox is able to get itself pretty far
entirely on the basis of that.
However, that's not good enough.
The emulator sometimes has a trouble like lagging.
But it's basically good enough.
What is it thinking?
Oh, okay.
I'm amazed this is working at all, by the way.
I want to give a huge shout out to DuangoAC
who gave the amazing Taspot talk,
which I hope you get to talk to earlier.
This whole thing almost didn't happen.
Yeah.
So it gets you in a tech chase combo.
Oh, the emulator is having a hard time.
Okay.
So this tech chase combo you see is
a fox will grab Marth and throw him to the ground.
And then from there, there's only a handful of options
that the human player has.
At that point, you can fall to the left,
fall to the right,
and no matter what you do,
Smashbot can, in reaction, figure that out.
He can cover all the options that the human player has.
So the other thing that Smashbot does is prediction.
So he's able to look forward into the future
in the game state
and know the physics of the game,
know all the attack animations of the game.
And then once you figure that out,
committed to some action, take advantage of that into the
future. So depending on whether it actually comes to
it, one neat example of this, one of them is
the rolling, right? The very moment, the very instant
that the opponent starts a roll, SmashBot knows precisely
where he's going to end up rolling and exactly
when. So he can just throw out a grab at
that exact moment. So there's no way, even in
principle, to get out of it. It's a little
bit more evident during the edge-guarding situations, when the
opponent is off the stage, where the flow charting
of, like, options the opponent has is a bit more
apparent there. So we're just going to let this
run until the timeout, basically, and I can talk
a little bit more about the other. So it
should be noted that right now, as of this
moment, the one matchup that SmashBot knows really
well is this one, which is Marth against Fox
on FD. And it shows that for a very
specific reason, right? I wanted to tackle one
problem at a time. I believe in the engineering
principle of just solving one easy problem at a
time, right? And then eventually try to add
support for that. So that's one of the things
that we're trying to do for other characters
in other stages. So one is that Marth, the
green player that's currently getting beat up,
is the, is a high-tier character. So he's a
very top-tier character amongst human players.
So you will see lots of competitive, even
professional players play that character. So
it's not simply the case that SmashBot is,
like, beating up on some low-tier character,
right? And in fact, this exact matchup, the
Marth on FD matchup, is considered almost
unwinnable for the Fox player amongst high-level
players. So this is something that, like,
if two human players were to be playing, Marth has a
massive advantage. So you can take that as an
example of, like, what, you know, is going on here.
Let me get a drink. I'll try to get some audio
after this is finished, I guess. So the, some
other parts to talk about here, the tech chase
combo is basically unavoidable. It is possible to
kind of, like, slide off the stage or something
like that. But it's very strong. So, I'll
get to that in a second. So, yeah, almost.
The, oh, we actually missed an upsmash. So
I'm not going to pretend to have created the
world's first bug-free program. The, certainly,
some instances where SmashBot will just sort
of derp off the stage or something like that.
It's been an iterative process. No. It can,
it knows about that. So, if ever you, yeah,
yeah. If ever you see it actually get hit,
it's almost surely due to what's called a shield
stab. So, you can see when Fox puts up his
shield is this big blue bubble, right? As it
turns out, the way that Smash works, the, it
only shields exactly where that blue bubble is.
And so, if you're able to, like, hit his, like,
foot that's kind of sticking out of the shield,
that will actually land. And it's hard to predict
ahead of time when that's going to happen. Also,
Fox will only shield after you've done your attack.
So, first you have to commit to the attack, and
then SmashBot will put up the shield. Which means
it's not really reliable for you to try to hit
it. It's basically like a random thing that'll
happen something like one out of a hundred
times. Where, the, like, SmashBot will try to
shield, and then it just, like, won't work.
Okay. Now.
Ah. Yeah. You can tell when somebody actually
knows how to play Mei, like, oh, yeah. Like,
you can kind of do a recovery there. Actually,
it was funny. For the first, maybe, like, six
months or something of SmashBot's existence, I
discovered that the novices did actually better than
competitors. They did better than competitors.
They did better than competitive players. And
partly due to the fact that, like, they were
just doing weird, random things that I hadn't
considered. Whereas, like, all the competitive
players, like, like, were doing things that I
had anticipated. Or, like, maybe just, I would
never consider that I would, oh, okay. Emulator's
having a hard time for a moment there. The,
particularly, like, standing there and slashing,
for, like, the first, like, six months of
SmashBot's existence, like, I don't know how I'm
going to deal with that. I'm just going to come
back to it and hope that nobody does it. So,
to give you an idea about just how competitive
SmashBot's become, it has, I've been bringing it
out in secret to the local Smash tournaments in
the Arizona scene. And I don't want to, like,
name some names of people that it has beaten
because that wouldn't entirely be fair. But
something like the last, like, 50% of its matches,
JV5 stalks the players there. Which is to say
that it doesn't take a single hit throughout the
entire match. And sometimes it will, like,
randomly take a hit. Basically, people realize
pretty quickly that you can't just play a
normal person as if it were a human being. You
basically have to try to pen test it, right? To try
to find some bug, some corner case that it's not
considering. And if you're able to do that and
then execute that, like, four times in a row in
order to, like, take four lives, then you can
beat it. But that's just sort of been an iterative
process over the last while to try to find all
those little bugs and then fix them before, you
know, bringing it out to the next thing. So,
this is really the first time that I've, like,
shown off SmashBot in any, like, major way aside
from, like, the fact that, you know, it's a
little bit of a secret. I'm kind of, like, in
secret showing some of my friends. So, it's
definitely good enough to, like, you know, show off at
this point. Though, it's worth saying that there's
still a lot of work to be done. So, yeah, in
exactly 20 seconds here, we'll get back into,
like, how this whole thing actually works or how it
began. The AI parts of it, the reverse engineering parts
of it. See if we can get one hit in in seven
seconds. Oh, no. It's still working. So, I'm
still got you. No. Cool.
It does. It's frame perfect start pressing. Really
wants to start the next match. I'll go ahead and
kill this. We, I'll do another round at the end
during questions, so. Okay. So, now a little bit
about the AI, about how it works and how it
works. How does SmashBot think, right? How does it
decide, like, what buttons to press? It's not
simply just a series of heuristics. So, the top level,
it has a four-tiered hierarchy of goals. So, at
the very top level are, like, the, what I just call
goals, right? It's the highest level of, like, what
is SmashBot trying to accomplish? What is the thing
it's trying to do? So, these are things like kill
opponent. But it's not always kill opponent, right?
Sometimes it's just, like, navigate the menu system
because it wants to, like, select its own character,
things like that, right? And so, the way that this
works is that the little bubbles on the right-hand side,
that you see here, are actual source code files, right?
These are, like, the C++ files. Then the whole point
of the file is to determine the next lowest strategy.
So, the next level is strategies. Things like bait
or sandbag. Like, if our opponent just got back from
the invincibility, then we might not want to attack them.
We just sandbag. Or bait. Try to, like, bait our
opponent into a wrong move. And so, like, kill opponent
might choose bait, for example, as a strategy. And
this basically keeps on going down and down. So, then we
are trying to, like, bait our opponent into a bad move.
We're going to, like, weave in and out of their range,
hope that they make an attack, and then punish them when
they do. And so, like, bait might then choose punish.
We say, aha, like, we know that this person has exactly
17 frames of lag, say. And so, we know that we can run
up and then give them an up smash in that time. And then
the very last level is chains. Chains are, like, button
combinations that smashers would recognize. Things like
wave dashing or dash dancing, up smash, things like that,
right? And so, these are things that we're trying to
do. So, these are the lowest possible level of abstraction
in terms of the actual, like, button press sequences. And so,
then the punish would say, okay, I'm exactly in range, and
I'm in a place where I can't up smash, so let's go ahead and
do the up smash, right? And then this is going to change
frequently every single frame. All right. So, let's talk a
little bit about reverse engineering, right? This is
something that was a lot of fun because, you know, being a
penetration tester, this is sort of up my alley. And so, I
there is an awesome melee scene of hackers. People like Dan
Salvato actually has been a huge help as well as some other
guys there. There's an entire Google spreadsheet that we
eventually made about, like, getting this sort of information.
But in terms of reverse engineering, what smash bot needs to
know is it needs to be able to figure out a picture of the
universe, right? It wants to be able to see the screen in the
same way that you do. There's no hidden information in smash,
not in a way like poker is, right? If I made a poker bot and
said that, it just plays by, like, reading your hand, lol,
like, that wouldn't be interesting at all, right? So,
there's no hidden information to smash. It's all just available
on the screen. That said, how do we actually know where all the
pieces are? So, we have to make a couple of assumptions here.
One is that the game does have the game state information
represented in some way. It has to, right? It's got to know
where the current player positions are. It's got to know
what your damage is. And so, rather than, like, parking a
camera in front of the screen and trying to, like, visualize it
that way, I knew right off the bat that would never work, right?
So, we want to be able to get information out of the game.
The only trouble is, to the game, or to the emulator, the
game is a black box. So, it doesn't actually have any idea
of what's going on inside of it. It's just a virtual machine,
basically, right? In the same way that VMware or VirtualBox
has no idea what's going on inside your, like, Windows VM,
right? It's just running opcodes and presenting virtualized
interfaces for hardware. So, un-blackboxing this black box
is the reverse engineering that is behind smash bot. In
particular, we don't want to be able to get information out of
the game. We actually care a whole lot about code, and more
than that, we care about data. So, inside of the game, there's
going to be a couple of pieces of key data, right? Things like
your exact character position, x, y, like, what character is my
opponent, what stage are we on, what damage do we have? I want
to be able to take that information, figure out where it's
stored in memory, and then ship that off to an external process.
So, it should be known that, like, when I started this almost a
year ago, none of this was, like, worked on at all. There's a
lot of trailblazing involved. So, there was also some
tools like cheat engine, but cheat engine wasn't exactly going
to do what I needed it to do. And a lot of the built-in
debugging functionality to dolphin, and there is quite a bit,
also wasn't quite going to do what I needed it to do. So, most
of the debugging functionality is about trying to, like,
disassemble code. And, again, that's not exactly what I'm
looking for. I don't necessarily care about the code, and that's
a good thing, because the game cube runs on power PC, and I
really didn't want to have to learn power PC. So, basically,
what I'm trying to do is I'm trying to figure out how to
do that. So, what I did was take memory snapshots. The game
cube is super old and only has about 24 megs of usable RAM.
There are other RAMs, there's, like, specific video memory,
there's, like, registers, but the main system RAM behind the
console is only 24 megs, which means I can just write it to a
file and then inspect it manually, basically. All the
stuff that cheat engine does, I'm just doing more or less by
hand. So, I had to make a fork of dolphin that every time it
would take a snapshot would write the entire contents of RAM
out to a file, and then I would just be doing the bin diffs on
the, like, memory instances, right? And so, I would put the
game to a known state, say, like, I'm going to put my damage
to 47, right? So, I'll have 47 damage on a particular character,
and then put the damage up to, take a snapshot, put the damage
up to 98, and then just do a search to see what regions of
memory have changed from 47 to 98, right? And that works really,
really well when the memory regions are stable, and that tends
to happen if it's, like, stack allocated, right?
.
However, not all the information is stack allocated.
In some cases, it's dynamic. So, in those cases, it gets a
little bit more complicated. So, that tends to be when there's,
like, a struct. So, all the player information, the stuff
about, like, actually what damage you are or things like
what character position you are, X, Y character positions, are
stored in a big struct that's allocated on the heap. And so,
first, you have to try to find where the struct is, right? That
tends to be pretty easy. We could just look for, like,
damages or something like that. And then, you have to search
for the memory. So, suppose that was found at a particular
address. Then, you just scan the entire RAM again to find out,
is there any regions in memory that contain that address? And
if so, that's probably a pointer to our struct. And now, we have
a stable pointer to our dynamic memory region that otherwise
we'd be moving around. So, this sounds sort of kind of easy,
and in concept, it is. In practice, this one's being a
total bitch. Some data structures make no sense. It
should be going without saying that these data structures were
never meant to be read in the way that we're reading them.
Because, of course, this is, like, a 15-year-old console game.
So, why would they have made these data structures to make any
sense? So, sometimes there's floats where there should be
integers, because it's clearly monotonically increasing, like,
value, but fuck it, they gave it a float. And that took, like,
forever to figure out. Or sometimes there's no consistency
to whether things are indexed at 0 or 1. It's just sort of, like,
figured out. So, before we go any further, I want to talk a
little bit about game programming. Because if you've done some
programming before, this is probably very different than what
you might have experienced. So, there's the concept of a frame
and a frame loop, which is very important. So, on the left there,
you can see, in real time, Marth doing his forward smash attack.
So, he's just taking his sword and throwing it down super hard,
and it looks super fast, and, in fact, looks really, really
smooth. When, in reality, that's not how it actually works.
When you slow the game way down, which you can see on the right
hand side there, you can see that it's basically just an
animated GIF. And not only is it, like, kind of choppy, but
the animations are predictable. So, at exactly frame 10, on the
10th frame of the forward smash, every single time, Marth will be
exactly in the same position every single time, right? So, the
game is basically just a finite state machine running very, very
quickly. So, the game runs at 60 frames per second, which means a
single frame lasts approximately 16.66 milliseconds. And so, the
processing looks basically like this. You start at 0. It pulls
your controller input to see, like, what has the player pressed.
It runs the game engine and produces an image on screen and
then keeps looping. And that's more or less how the game
works. Also, basically every 3D game works. And so, what's
important here is that it's not just that the game is displaying
at 60 frames per second. It's that the game engine
fundamentally runs at 60 frames per second. So, you can use this
to cause all kinds of really cool bugs. So, if you are running
very, very slowly, right, and suppose you're somebody who's
totally not Mario, and you're trying to get to some treasure,
that's past a locked door, right? If you walk slowly, you might
just kind of run into the door and not be able to get through it.
But, if you're moving super, super fast, on one frame, you might
be here. On the very next frame, you might be here. And then, on
the very next frame, you'll be there, right past the door. And
the game will have no idea that you ever collided with the door,
right? Because it never, you never touched the door. On one frame,
you were before it, and the next frame, you were behind it. So,
this leads to some really cool bugs like this. So, this is a
actually a task from Super Mario 64, where you can see exactly
that. This is the very beginning of the game, and Mario is going to
switch, go right through what is supposed to be a locked door,
just by going super, super fast. So, hopefully, some audio is here.
If not, it's not critical.
Yeah. So, basically, what you saw is that he just did some
little tricky bug thing to go super fast, and then just zips right
through some doors, right? So, that's important, not because we're
going to be doing some, like, zipping through doors, but just to
give you an idea of that the game is running with this internal
frame loop. So, the game looks a little bit more like this, where
inside the game, there's this looping thing that the emulator
actually has no idea about. So, the emulator is just the hardware,
right? It has no idea that there's this internal frame loop. That's
the game's business. And whenever it receives frames, it will go
ahead and output them.
So, in order to get the game state information out, now we've figured
out where the, like, bits are inside of the game, right? We have to
have some mechanism of exporting it out to a separate process. And so,
my first foray at this was really hilarious. So, first, I set up a
segment of shared memory between the Dolphin emulator, which was a
modified version, another fork that I made of the Dolphin emulator,
to SmashBot. So, this is a shared segment of memory. There's no,
like, input and output. It's actually just the same memory that's
shared between two processes.
So, what I had to do is write some code that took the game memory
and copied out the relevant data into a struct that's in that game.
So, had to, like, move the data out into that struct. But, of course,
I don't have any concept of when the frame is running, because the
emulator doesn't know when the frame is running. So, the natural
thing to do here is just make a spin loop. So, we have one entire
CPU doing nothing but spinning, doing absolutely nothing but copying
data into that shared memory region. So, now SmashBot has this, like,
constantly updated, real-time memory. So, we have this, like,
real-time view of all the relevant game state information. But, it
doesn't know when the frame has processed. It has to, like, trigger
per frame. And when a frame triggers is one of the pieces of game data.
So, of course, had to write a second spin loop inside of SmashBot
that would regularly check that struct. This is what computer scientists
would refer to as suboptimal. So, eventually, this was integrated into
the official Dolphin build. As the Dolphin 5.0, there was a new feature
called Memory Watcher, which does this without the terrible spin loops.
So, I would, like, super big thanks to the Dolphin guys for that.
So, now we have three parts of the whole running system. We've got the
Dolphin emulator. We have SmashBot making decisions. We're able to pipe
that data out over a named pipe, basically. But it's still not playable
at this point, because we still can't actually press buttons. And so,
that was another kind of funny instance where my initial attempt to
press buttons on, like, a virtual controller, Dolphin didn't have any
mechanism for actually doing that. But what it did have is the ability to
type on a keyboard. So, you can, like, map the A key to press the A button
or something like that, right? So, I thought, okay, great. What I'm
going to do is I'm going to write a, like, helper that uses the XOR
libraries to, like, press the button, like, on the keyboard. And it
actually, like, sort of works. It's terrible, and I would not recommend it
whatsoever. Partly because if you, like, move your focus away, it just
starts pressing buttons, like, into the random window, whatever you, like,
gain focus to, and just goes haywire, and it's, like, hard to cancel. But
also because basically all of these mechanisms are going to be buffered
input, and so there's going to be some indeterminate amount of latency
from when it presses the button to when it actually happens. And normally,
you don't care about this. If you're just a human being, like, pressing
buttons on a keyboard, it doesn't matter to you if, when you press the A
button, it doesn't happen for the next 30 milliseconds, or maybe the last
couple of buttons get buffered together. Like, you just don't care. You're
just incapable of physically noticing that. But Smashbot cares. It needs to
be able to have exactly frame perfect accuracy on all the button presses. It
needs to get there super fast. So eventually we wound up getting that
integrated in with Dolphin as well, so now we have a mechanism for pressing
buttons. So about programming. If you're anything like me, programming looks a
little bit like this, where you're more or less in a constant state of
confusion, because if you understand the problem that you're trying to, like,
trying to program, then you just solve it pretty quickly and move on to the
next problem. And so to be a programmer is to be in a constant state of
confusion, interrupted only shortly by tiny bursts of, like, epiphany and
coding things up. So if you were to walk up to me at any point while I'm
programming Smashbot, usually it's Saturday morning, eating some, like,
breakfast cereal and drinking some tea, and you say, hey, Dan, how's it going?
I'd be like, I have no idea what the fuck's going on. Nothing's working.
Nothing's working, and I have no idea why. So I wanted to give you one cool
example of what this looked like. So for the longest time in Smashbot's history,
up until maybe a couple months ago, there was just this nagging bug that I had no
idea how it worked. It was just, like, the only logical explanation for it was
that there was a gremlin inside of my computer pulling on wires. And so it
looked something like this, where, like, Smashbot would be totally cool, and then
just derp right off the stage. And I was like, what is going on here? There's
no reason for it to do this. I couldn't pinpoint in code why this was happening.
And it manifested itself in all kinds of ways. It was like, I don't know what the
hell this is. It wasn't just derping off the stage. So I implemented this entire
debugging mechanism, where I could, you give it a dash dash debug flag, and it
will take the entire game state per frame and write it out to a big CSV file. That
winds up being, like, megabytes large. It's actually the best thing I ever did in
terms of debugging, because this lets you retroactively walk through what happened
throughout the entire game and see, like, oh, yeah, it pressed this button when it
should have pressed this button, or whatever, right? So I could see in here that
sometimes, not all the time, just randomly, seemingly, I would press this button,
and it just wouldn't happen for a frame late. That was the source of the bug. I
finally figured, okay, so, like, there's, for some reason, it's pressing a button
frame late. I don't know why. It was only ever one frame late, and not all the
time. So this is super weird. I was kind of chalking it up to a dolphin bug,
maybe. There was some bug in the emulator. And so I eventually tried out this.
This is what you're going to see here is Fox doing frame perfect multishines.
This is Smashbot. It's not just doing these blindly. It's actually reacting. So
on the exactly the third frame of the jump animation, Fox is going to hit down
a little bit, and then jump out of it, and then loop through it again. So what's
important here is if he's even a single frame late on any of the inputs, he will jump
accidentally. So it looks like this. So he's going along happily doing frame
perfect multishines, and then start jumping. He's screwing it up. And then go
right back to multishining again. And then you start to notice that this is
actually cyclical. This is, like, not random. This is happening on an exact, like,
predictable basis.
So he'll do it again in just a moment here. I was like, that's weird. I do like it when
bugs are reproducible. So eventually, me and Dan Salvato, another awesome Melee
hacker, figured out that this picture of how the game input thing works was not entirely
accurate. And so what happens is that the game input and the game engine processing
are on separate threads. And they're not perfectly synced up. And so what happens is on
one frame, it'll look like this. On the very next frame, the controller input will
drift by a tiny bit. And then the next frame, the controller input will drift a tiny bit
until eventually they swap. And controller input is pulled afterwards. So in the very
beginning, you're going to press a button, right? It's zero, right? The game will process
without having read your input. Then it'll read your input and not process it until the
frame afterward, until eventually it would drift backwards. So then we put together a patch.
I should say Dan put together a patch. So then we put together a patch. I should say Dan put
together a patch. So then we put together a patch. So then we put together a patch.
We actually fixed this. We moved the controller input
routine on to the same thread as the game engine basically. So that way we patched
a 15-year-old bug in the game up to this point no one has ever noticed. So that was pretty
cool.
So some bits about the future, as it were. I wish I could have gotten this working for
DEF CON. It's like 75% working. It's running on live unmodified console. So as it turns
out, this is actually completely possible, and I was talking with Duango, who did the
task bot talk right before this about some of the parts about physically, like, how do
you send buttons, presses over to the console. But one of the more interesting stories is
actually how to get information out of the console. So remember, I want to do this on
an unmodified console, so I don't want to just, like, put some, like, leads and open
up a hardware, open up the GameCube or something like that, right? And so in order to do that,
we have to use a really fun exploit through the memory card port. So it turns out that
in Melee, you're able to give yourself a little name tag, like, what my name is, that's four
characters, and of course, since it's four characters, people name it lots of colorful
things. But if you go into the actual save file that's on the memory card port and change
your name manually to be longer than four characters, it overflows the thing. And you
can get code execution on the game. And so there's already people that have been exploiting
this and using them to make modifications to the game.
Okay.
If you've ever seen the 20XX hack pack or 20XX, actually, no, it's 20XX tournament edition
is one that uses this name tag overflow. So it's a great way of getting code execution
on the game, which we can then use to grab game state information and ship it off over
the memory card port, which is then attached over USB to a laptop. That way we can get
live frame data out of the, like, running machine. So we then put SmashBot inside of
the controller that would then, like, be pressing buttons, and you would just be
sort of looking like you're playing the game like normal. And you would never notice that
it was computer playing unless you, like, looked closely and noticed that SmashBot,
the controller, was plugged into the memory card port instead of the controller port.
Or probably both, actually.
So yeah, before we start getting back to the end part here, I want to impart on you
a little bit of Smash philosophy. So being a part of any, like, competitive scene for
sure imbues you with a certain amount of the philosophy from that game.
And so I want to share this with the hacker world.
I don't want to hear no John's.
John's.
John's.
John's.
John's.
John's.
What's a John?
It was like 2 a.m. and he was tired.
John's.
Like John's.
He'll get me on a days where I'm just not playing too well.
John's.
Just John's.
A lot of people don't know where the term came from. It just started, but I believe
it was a guy named John.
A guy in Texas. His name was John. And no matter what, every time he'd lose, he'd have
an excuse. He'd have a reason for losing.
My controller wasn't working. The stage. There's a little bit of lag on the TV.
I didn't sleep last night or I don't know why I'm not commenting.
It's too cold. My hands hurt.
I need a warm up.
Yeah, I know.
We have a, like a Swedish term. Inga-Junas. It's pretty much no John's. He used to like
using much John's back in the time.
My favorite one, I think, was I was playing somebody and they were like, someone's touching
my shoulder. And I was like, no John's.
So yeah, no John's.
And thanks a lot.
And we're hiring.
We've got something along the lines of eight minutes, so I'll go ahead and start the game
up again.
If you want to line up here, I guess we'll have to do two lines. One for playing the
game and one for questions.
And we'll do that right now.
If you want to take questions, there's a microphone right there. Otherwise, I will not be able
to hear anything you're saying.
Here, let me set up the game first, actually.
Hello?
Can we see SmashBot versus SmashBot?
Yeah, so that's a question that people actually ask a lot is, like, what would happen if
SmashBot played itself?
Or, like, the right now, there's just a small logistical problem with it, which is just
that, you know, it's not going to work.
It only knows, it's, it plays on player control report two, and it assumes its opponent is
on player control report one, and so there's just that.
But I can get that solved.
But the more interesting question is, like, what would happen if it played itself?
Or, like, what is, like, truly perfect play look like?
Let's just give it seven minutes.
Here.
So, it turns out that optimal play, I gave quite a bit of thought to this, is really,
really complicated.
And this is actually a good question.
So, in case you're sitting in the audience thinking, oh, this is really, really complicated,
like, hey, I bet I could make a, like, a better SmashBot that would, like, beat this
one, right?
Well, let me take you on a tour of what actual optimal play looks like, right?
So, first off, all projectiles can be reflected.
There's a two-frame window at which point you can reflect projectiles.
And so all of those are suboptimal.
And so the only way to, like, attack is to just basically walk forward.
And so the fastest move in the game is shine, which is Fox's down B attack.
And both, both Foxes, both bots would basically walk at each other until they're exactly
within range, and then both use their perfect one-frame move at the exact same moment.
They would clang off of each other, not hit.
And then it's a deadlock from there.
At each point, both, the optimal play for both characters is to jump and then do frame-perfect
multi-shines until the time limit runs out.
When the time limit runs out, the game goes into sudden death.
At sudden death, like, the game goes for a little while, and then Bob-ombs start falling
from the stage, sort of, like, randomly, right?
And so it would be possible to put your opponent in a position where they would be able to
either run at your attack or into the Bob-ombs, so it would be kind of, sort of, random.
But they're not actually random, right?
They just use the endgame's random number generator, which is entirely predictable.
So, back up.
The optimal strategy is not just simply run at your opponent and shine.
It's to put yourself in a position where once you deadlock your opponent into that shining,
you know that in exactly eight minutes from now, the random number generator will be seeded
such that the bombs will fall in a way such that you can put them in a disadvantageous
position.
So, before you go around thinking, I'm going to make the perfect bot, know what you're getting
yourself into.
Yes?
No and no.
I'm sorry.
The questions at the microphone.
Yeah?
Have you taught them to do taunts at the most insulting times?
Yeah.
It does do taunts.
That was, like, it didn't do taunting for the longest time.
It just sort of sat around.
But now it does frame perfect multishines in between stocks as the, like, how to taunt,
basically.
I figured that would be a pretty cool way to do it.
Yeah?
So, you mentioned that the beginner players will, like, confuse this.
So, how do you get around it?
Do you use machine learning or do you just keep on programming?
Yeah.
There is actually a separate machine learning fork of Dolphin called Phillip.
I wish I had more time to talk about it here.
That uses the Google's TensorFlow neural network library.
At first, it had a really hard time doing more than just kind of moving around.
But it's actually getting pretty cool now.
And so, one of the neat parts about SmashBot's design is that those, like, lowest level,
like, chains, like, maybe there's no need to make an AI learn how to wave dash on its
own, right?
Why don't we program that in as a primitive and then use AI to kind of choose which lowest
level primitive would be best?
And so, that's actually, like, a goal of mine for the project is to do exactly that.
This is about as far as I've taken it, like, right now.
But it's absolutely...
Actually, I should have mentioned this is an active open source project.
It's available on my GitHub.
Just github.com slash alt F4 or just Google for this, basically, and you'll find it.
Yeah.
You mentioned that with the game in the future you had plans to have this run on an
unmodified console.
Yeah.
Do you anticipate that you'll be able to overcome, like, was it strictly on the
emulator side with the drift problem with the controller that was causing the bug where...
So...
It is actually a bug in the game.
That isn't, like...
So, the game is responsible for that frame loop and the, like, controller pulling.
So, that is actually a bug in the game.
That said, we haven't been able to empirically verify that, right?
So, in theory, that bug should be present on console.
But without Smash...
There's really very difficult...
There's basically no way to know without, like, verifying that via, like, maybe some
task way of doing it.
But Smashbot would actually be the best way of verifying that.
Because it is reacting in real time to the frames rather than just, like, having a scripted
set of button presses.
So...
Do you anticipate that there will be some way to maybe overcome that so that you
can have that...
Oh, exactly.
So, we can code execution on the game, right?
So, we can just modify the running game to fix the bug.
Just patch it live.
Oh, yeah.
Yeah.
Thank you.
Hey.
First question.
Are you coming to Super SmashCon?
I'm not.
I actually only discovered that SmashCon existed after the CFP closed.
Oh, that's too bad.
So, I really would have liked to have done this over at Super SmashCon.
I'm based out of the Phoenix area.
So, if ever you want to, like, play SmashBot near me, if you want to run it yourself, just,
you know, download the source code, run it yourself.
Otherwise, I'll be around in the kind of Phoenix area.
I'm hoping to take this out to a larger tournament sometime in the near future.
But I have a, you know, busy travel schedule with work and stuff like that.
So, no promises.
The other thing was, I just wanted to say thank you so much for figuring out that
bug, the 3.5 to 5 frame thing.
Yeah.
Because I actually had an idea for a project a long time ago where I was, like,
all right, I'm going to take a high FPS camera, solder an LED to a controller,
and figure out the amount of input lag difference.
Yeah.
There's so many problems that happen in the analog world that, like, there's...
Yeah.
Yeah, it's really difficult.
This way, it doesn't matter what's going on in the screen.
SmashBot's reading the live bits out of memory.
So, the very frame that something happens, it knows about it with taking the entire
analog universe of display refresh rates out of the equation.
Yeah.
But it's so awesome that you guys figured out that bug.
And I was just wondering what went into it.
Because, like, I would have been...
Like, really freaking confused.
And I have measured...
Oh, I was really freaking confused.
Yeah.
I have measured the FPS lag.
And you can see it.
It's, like, every, like, quarter frame, something like that, it just takes longer.
And it makes no sense.
Yep.
That is absolutely correct.
So, thank you.
You bet.
Oh.
I had a couple questions.
So, can it be any...
Can it be anyone else other than Fox?
Or does it have to be Fox?
Yeah.
SmashBot plays Fox and probably will for the indefinite future.
It's clearly at this level, like, at the task level.
The best character in the game.
It's just faster than every other character.
One could make an argument for Falco, but I'm not so sure.
I mean, it is kind of an open question about what is optimal play, like, at the highest
levels.
Who knows?
Maybe, like, if you could play it this fast, Donkey Kong is, like, super broken.
I don't know.
Right?
I doubt it.
There's good reason to believe that Fox is the best character.
And so, this is my best stab at making that happen.
And what about having SmashBot play, like, three other characters at once?
Simultaneously, yeah.
So, right now, I don't know.
It only acknowledges the existence of the player one, because I just wanted to make
that work first.
I suspect that that's just a losing battle.
Like, once you actually have 3v1, at the theoretical level, you just lose.
Because even though I can frame perfect shield stuff, there's lag after the shielding.
And so, like, you could just hit me after that happens.
And one last question.
Yeah.
Is there any possible plans for other fighting games that you would use this for?
Smash is really the only game that I personally play competitively, like, at that level.
So, not for me.
But there actually are similar projects and other AIs for other games.
There are StarCraft and StarCraft II AI tournaments that actually happen.
So, there's very similar sort of endeavors in that world.
Thanks.
So, you said that this bot is supposed to be able to mimic human behavior.
It's supposed to be more, if you look at it, it's supposed to...
My question is, when I notice whenever you die, it goes left and right.
So, you're not right really, really fast.
Yeah.
Was that on purpose?
Yeah.
So, it does what's just called a dash dance, right?
It just moves, moving back and forth.
For the first, it depends on each character.
I think it's 7 frames or maybe 10 or 11 or something like that for a fox.
The first, when you start up running, you're in a dashing animation,
at which point you can dash backwards very quickly.
It's a good way of keeping mobility based.
It's something that even, like, high level players do.
But never with that exact amount of precision and that amount of speed.
Yeah.
I guess to the earlier point,
SmashBot is not intended to, like, make you feel better.
It's not meant to play like a human.
It's meant to play like a computer.
In the same way that, like, an aim bot, right, for, like, a shooter,
does not play like a human would.
And so, we're trying to break fundamentally how the game is played at that level.
So, if you're playing, like, a shooter game with, like, team-based strategy,
there's a lot of high level thoughts in terms of, like,
by getting your opponent to use, like, are you using cover?
Getting your opponent to, like,
move into the center stage.
But if you're a computer,
optimal strategy is to stand in the center of the stage,
spin 360 as fast as you can,
and then blame people in the forehead the very moment they come out, right?
And so, SmashBot's kind of taking advantage of that,
in that it's not trying to play like you do.
It's trying to play like a computer does.
Thank you.
And thank you for letting me try it out.
Absolutely.
Is that time?
Yeah.
All right.
Thanks a lot for coming out.
Thank you.