Okay. Well, welcome everybody. My name is X-Ray.
This is the DEF CON 30 Altspace VR Village for DEF CON groups.
So, welcome. And our speaker is Jim Shaver.
He's going to be talking about AWS Metadata Privilege Escalation.
Jim is a Pentester, Offensive Cloud Security Researcher,
and public speaker with 13 years of IT and security experience.
So, Jim, go ahead and take it away.
All right. I'm going to stand by the podium so I'm heard.
Today we're going to be talking about AWS Privilege Escalation.
And when I talk about Privilege Escalation,
I'm mostly talking about the API and sort of the back end of AWS.
I'm not specifically talking about, like, necessarily operating system Privilege Escalation,
although operating systems will be involved.
Okay. Next slide, please.
All right. So, some of the things we're going to be talking about today are how authentication works within AWS.
We're going to be talking about the Instant Metadata Service, IMDS.
I will just call it the Metadata Service.
I'm going to be talking about various modes of escalation,
as well as several tools and resources that you can use, some of which I have written
and some of which other people have written.
Next slide, please.
All right. So, this is what IMDS looks like.
If you're on an EC2 virtual machine, you can just curl this IP address 169.254.169.254.
You'll notice it's kind of an unusual IP address.
If you've seen or been on a network that doesn't have DHCP working properly,
you've probably gotten a 169 address.
And you can think of that IP space as an IP space that is non-routable.
And in order to avoid collisions with 10. and 172 and 192 addresses,
Amazon chose a 169 address for this non-routable interaction with the IMDS service.
Just a little bit of background on what the Metadata Service is.
It's a little bit of semi-dynamic data that the operating system uses for its own purposes,
for whatever it needs.
So, things that are included are kind of benign things like the AMI ID and MAC addresses,
network information, even the region that the virtual machine is running in.
So, the US East 1, for example.
But also other more interesting things for attackers in here, things like user data,
which is like the machine startup script, or even the machine's own identity credentials
and role-based credentials that the machine may have been granted.
And so, these credentials you can think of as sort of the machine account, quote-unquote.
Next slide, please.
All right, so this is an example of a simple, I guess, SSRF where you have a proxy parameter
that's injectable with this local IP, this 169 address, and it returns a role.
So, latest metadata, this is actually returning a text response from the metadata service
within the virtual machine that's running this web application, okay?
And what makes this kind of like the machine account, you can think of like in the Windows world,
you can give like an Active Directory machine, like domain admin, for example.
Nobody does it, but it's something that's technically possible.
It's much more common in AWS to give EC2 instances roles, which means they typically have permissions
and policies either attached or assumable or whatever through those roles.
So, if you go into latest metadata IAM security credentials, and then there's a whole bunch of stuff.
In this case, this role name is called EC2 default SSM.
If you then sort of browse to that role name, what will happen is it will show you a JSON response
that is an access key ID, secret access key, and then a token that changes every once in a while.
This is the authentication for this role for this machine, so that when the machine does certain administrative tasks,
it can authenticate using these credentials, and they kind of work like long-lived API keys for AWS,
but they're used via the short token service instead of long-lived API keys.
So, the way that you can tell that that is happening is the first four letters of the key ID are ASIA,
whereas long-lived keys are AKIA.
Okay, so we'll see another example of that later as well.
Next slide, please.
Okay, ways to get at IMDS.
So, obviously, you can imagine a bad guy wants to get at the IMDS service of an EC2 instance.
A CC2 instance is in a VPC.
Maybe it has a public interface where there's some sort of vulnerability,
or the bad guy has somehow gotten into the virtual machine.
Some of the ways that that can happen is you can be on the box.
So, if you get a shell on the box, you can just use curl, like we did in some of the previous examples.
So, in that case, you would just curl this web address, and then you would go from there.
There's also examples of command injection and SSRF, where if you are able to, you know,
render a web page or something like that, the results of a command,
then you can also display the contents of the metadata service.
There are also more novel examples that vendors and service providers haven't always thought about,
in terms of some sort of SSH key for bastion access, or if they give you, you know,
there's some sort of reverse tunnel they use for support or VPN keys that they use to connect to a box in EC2.
Even if they disable, like, SSH access to the EC2 instance,
you may disable the tunnel network traffic over SSH via SOCKS into the EC2 instance,
and you can reach a local address, even though you're routing across the internet,
you can still reach that local address over a SOCKS connection.
So, in that case, you would just, you know, have a browser or whatever tool that supported SOCKS,
and then you would tell it what SOCKS proxy to use, and you would just query the 169 address with curl
or whatever tool you're using to access the metadata service.
Next slide, please.
All right, so using the creds.
So, we've gotten this JSON response.
Basically, what we do is, this is an example of long-lived credentials.
So, you see, like, AKIA, these are obviously example credentials.
So, that's what normal, like, API keys that you might find on, like, your average developer's machine look like.
And this is an AWS credentials file that might exist on their laptop or on your laptop.
You can also, so you can also just use the tools that are on the box to interact with the AWS CLI or pull your tools onto the box, if you can do that.
But if you can get to this, you can just copy all of the data out of here, copy it to, you know, a box that you're using that has all your tools on it,
and then put it in your AWS credentials file and use it as a profile for authenticating.
So, then, when you're using tools like the AWS CLI, you just say, AWS profile, AWS session zero, which is what we have here.
And then we run whatever commands using the CLI that we have.
So, in this case, we're just doing a really basic STS get color identity, which is, if you're familiar with AWS,
this is a thing that tells you, like, information about the account that you're running in.
It's a good way to check to make sure that the credentials are valid.
Okay. Next slide, please.
Okay, so I'm not going to talk really about IAM policies because they are complicated.
There's lots of ways that policies can be associated with a user directly, either through inline policies that are directly attached to the user,
managed policies that are associated with the user, inline policies that are associated with a group,
managed policies that are associated with a group, attached or past roles, etc.
There's also service control policies and permission boundaries that are used in more advanced environments.
And there are some limitations to privilege escalation that I'm not going to get super deep into today because it's really complicated.
Even some of the tools don't even understand a lot of the nuance that's out there.
Next slide, please.
And that's because this is like the decision tree of how policies and that type of thing happen according to AWS.
And even this doesn't include all of the nuance.
So there's a very thorough decision tree around how AWS makes decisions around whether or not you or a resource in your environment has permissions to do something.
Next slide, please.
All right, so some ways that you can use the creds.
Most of the examples we're going to be using today are with the AWS CLI.
But there are also lots of other ways that you can do it.
You can use Boto3, which is the Python SDK for AWS.
It just sort of takes you the next step after the AWS CLI, allows you to chain multiple things together and write like a Python script that understands how to talk to the AWS API.
There's other SDKs as well that you could also use.
There's a pretty good tool written by Rhino Security Labs called AWS Escalate that can just sort of try to brute force and figure out whether or not you have an escalatable path to escalation within AWS.
And we're going to talk through some of those paths manually here in a little bit using the AWS CLI instead.
There's also a really good framework called Paku, also by Rhino Security Labs, that you can dockerize and all of that.
And it's basically a Python application that gives you a menu and you can sort of step it through all of the ways it can escalate in another enumeration.
It's like a Swiss Army knife, basically.
There's a couple of tools that I have written that I've thrown in here as well. One is called RedBoto.
And basically it is a set of tools that either do enumeration or do interesting things with operating systems within AWS or connect to SSM, which is basically like SCCM for AWS and other interesting things.
There's also a tool called FederateMe, which is... I'm not sure if AWS has actually fixed this as a thing, but basically it uses federation to go from credentials to an AWS console, which is the web user interface of AWS.
And sometimes it's just easier to work in the web interface than it is to work via CLI or API or with various tools. So that is just easier to just pop open a browser.
So what FederateMe does is basically you give it your credentials and it creates a signed login federation link that will pop you into the console, even as an EC2 instance or whatever.
And you have the ability to do whatever it is that that machine would be able to do if it could log into the console.
There's also another tool called EnumerateIAM, which is a good enumeration tool of different IAM permissions. IAM is the system in which you do all of your identity and access management within AWS.
A lot of tools obviously revolve around that. Also, the best resource out there on the internet with offensive security, Red Team, AWS, and other cloud information is Hacking the Cloud.
It's maintained by Nick Frechette and it's very high quality and the best resource out there for this type of information.
I have references throughout the rest of the presentation to some articles on that.
Next slide.
All right, so a couple of more easy ways to escalate that I'm not going to demonstrate because I think you can kind of imagine what they might look like.
One of the ways that you can escalate is you can pillage S3 buckets for other creds.
You might get onto a virtual machine that has some role credentials that may have access to some S3 buckets.
If you are doing sort of a gray box assessment on the environment and you have like a scout report or something like that, you can know what the role credentials have as permissions and what S3 buckets they may have access to with that report.
It may not be actually obvious to you or even possible for you to know without brute forcing what permissions that role that you have access to, what it can do.
So if it's more of a gray box assessment and you've got the capability to do a scout report with ScoutSweeter, whatever they're calling it this week, then this can be a good way to find other credentials that developers or other people have left laying around or that might have higher privilege than what you have.
It's very common for EC2 instances to be granted roles that have access to S3 buckets and nothing else because that's one of the ways that they get data in and out of the EC2 instance.
And so it's very common for them to have that access. And then it's common for, you know, them not to lock down to specific S3 buckets and they may have access to like, you know, the Terraform S3 bucket or, you know, what have you.
There's lots of ways that they can get permissive access to the environment's S3 buckets.
Another area is user data. I talked about that a little bit earlier, and it's basically a Base64 encoded script that the machine runs at startup.
And those role credentials that we've been talking about may have the ability to read the user data of other EC2 instances and other virtual machines in the environment.
And so sometimes people put things in the startup script, assuming that they as the administrators are the only people that read that stuff. And it actually doesn't take a lot of permission to be able to read an EC2 instances startup script.
So you should really treat that stuff as, you know, a sensitive thing and use other means to get secrets and that type of thing onto the box.
Next slide, please.
All right, so we're going to talk about our first IAM policy or permission that allows you to escalate.
So if you have IAM add user to group, this one's like a really basic privilege escalation.
So obviously you're not administrator. If you have the ability to add any user to any group, then, you know, you could just run the AWS CLI command, you know, IAM add user to group, user name Alice, group name administrators.
And if you're able to run that command, you've just privilege escalated the user Alice to administrator in the AWS count.
Okay, so might seem like a super obvious way to privilege escalate, but it's an example of the type of thing that I'm talking about.
Next slide, please.
One that's pretty straightforward, but also a little bit more nuanced is IAM create access key.
So in this case, you are running, you know, IAM create access key for the user name admin Bob.
So basically you're saying, I want to create an access key for this very privileged user that I know is privileged, and I want to have their access key, basically.
Okay.
Okay, next slide, please.
All right.
Pass rule and run instances.
Okay, pass rule is more complicated.
Because basically, this gets into resource based privilege escalation.
So in this case, you are not doing the privilege escalation.
In fact, you are spinning up a virtual machine that is doing the actual escalation.
Okay, so the way that pass rule works is you are given the ability to pass a role to a resource that being the virtual machine.
And then you run that instance with a role that is very permissioned, very well permissioned.
And then you have that resource, the virtual machine, run a user data script that executes some AWS commands that create an administrator user for you, for example.
Okay.
A lot of people get tripped up on this one, they think that they need to have SSH access to the EC2 instance once they spin it up.
In fact, if you specify user data, the code executes and you don't need to be on the box for it to happen.
You don't need to have a VPC that you can SSH into, as an example.
There's lots of things that make this a lot easier.
And so there are some limitations on this one.
Obviously, you need to be able to pass a role that matters and exists.
You need to be able to run arbitrary instances.
And there also needs to be an instance profile that is privileged enough that can do the thing that you want it to do for you to do.
So there's a lot of moving parts that are involved with this one.
But that is how you do that one.
Next slide, please.
There are also many other types of privilege escalation.
So there are other services within AWS that are not even commonly used by the majority of AWS customers.
So like Glue and Data Pipelines and CodeStar are all DevOps-y type things that are used by some very advanced environments.
They absolutely have methods, if configured incorrectly, that allow you to privilege escalate.
Another one that I didn't include in the slides because it worked very similarly to the EC2 instance, one that we just talked about, is Lambda.
Lambda is like the serverless functionality within AWS.
And essentially, if you have pass, roll, and create function, invoke function, maybe a couple of other things,
you can do the same thing with Lambda that we did with that EC2 instance where you can have the serverless Lambda function do all of the IAM functions that create a user or make you an administrator
or whatever it is that you use to privilege escalate.
Next slide, please.
All right, so another tool that I have not really talked about publicly, but I released about a month ago, is this thing called Red AEMM.
And basically what it is, is I wanted to solve the problem of how do you have a cloud section to a CTF, okay?
And there are cloud CTFs out there that are like flaws.cloud, and they're just sort of publicly on the internet.
And what happens with those CTFs is that you're very limited in scope in terms of what you can do.
And there's a lot of risk in setting them up because you're hooking somebody up to a real AWS account that could not only get hacked, but also cost you a lot of money if it gets hacked.
And so this incentivizes people from having cloud CTF type things.
And so one of the things that I wanted to solve the problem was how do we teach people about the metadata service, but don't hook them up to a real metadata service in ED2.
And so what I came up with was AWS actually has a project called the Amazon EC2 Mock Metadata Service.
It's a fake metadata service that is a Go application and a Docker container that emulates a real metadata service.
And it's used for evops type testing, where you need to test something that needs to interact with the metadata service, but isn't, for whatever reason, you want it to talk to a fake metadata service just because it's easier for you or it doesn't need to be in AWS or whatever the reason is.
And so I forked that and basically developed a front end that is very basic SSRF and this is an example challenge of like, hey, if you put proxy and then the domain name, this is how this proxy works.
And it's not really a proxy, it's just a thing that gets you to the metadata service as part of the challenge.
So the idea here is that this tool can be used in capture the flag exercises so that people can understand a little bit more and interact with something that acts just like a metadata service.
It doesn't expose your AWS account, so if somebody gets access to a role within here, it doesn't actually do anything.
Next slide, please.
So this is what it looks like when you exploit the SSRF that I've built in here.
So just like what we've been seeing all along, you know, you hit this address, this is not really a fake address, this is just part of the application.
So it just looks like we're exploiting an SSRF in AWS, but this is just a Docker container.
And it returns a text response just like a real metadata service, but all of this information is totally bogus.
If you pull up this AMI ID, it's an AMI ID that doesn't exist within AWS.
These are not the MAC addresses of the machine that this is running on.
All of this information is bogus.
And you can control these through a JSON configuration file, and you can also control through environment variables as well.
So you can imagine hiding a flag in the user data of this MAC metadata service, essentially, on a CTF.
And so that can be found on my GitHub. It's called redamn.
And if you have any questions about it, feel free to hit me up on LinkedIn or wherever on the internet.
And next slide.
I think that's it for me. I'm Jim Shaver. You can hit me up on LinkedIn.
I'm also iHamburglar on GitHub.
I really want to thank the volunteers for their awesome work on this conference.
The setup is amazing, and it's one of the best speaker setups that I've had as a speaker.
So it's nice to just be able to come in here and talk from my house.
So thanks, everybody, and I'm happy to take any of your questions, if any.
So I've worked a fair bit in AWS, and what I've noticed is that routinely organizations will have people who are new to AWS,
or they've just graduated and they're getting out into their first Java position, and they'll just jump straight into building things.
And IAM is sort of this thing that they get as kind of an afterthought, and they don't truly master the service until years into their effort.
Have you seen any successful attempts at getting people to train in IAM first, rather than jump straight into building things?
So I think, if I could distill down the question for those people that are not in the room,
essentially, do you get people to understand IAM and do it securely? Is that a fair representation of the question?
Yeah, try to get it right the first time, rather than a retrospective look on it.
Yeah, I think, you know, I absolutely agree with what you're saying.
I think it's really complicated. And I showed you a slide that demonstrated how stupid it is.
And I think it's very difficult to find people who really understand how it is.
I think it's the number one problem that's AWS organizations.
And I think, you know, it's a, are you training people and investing in them?
And are you hiring people that know what they're doing?
I think there's probably a lot of opportunity there for Amazon to make it easier.
Absolutely, it's one of the number one issues with an AWS account is how complicated IAM is.
And it changes. Sorry, I can't really answer that question better.
No, no, if you did, that'd be amazing. Right? I'd take that back immediately.
Yeah, I would be in a venture capitalist office right now if I had that answer to that.
Thank you for the question.
Anyone else?
All right, well, I'll be hanging out for another 10 or so minutes if people have questions.
Fair. I really appreciate the opportunity to talk today.