Give it up for Brad Dixon. Thank you. So I'm Brad Dixon. Uh I work with Carve Systems.
We have uh a lot of fun uh working with systems that involve um embedded or IOT devices uh
you know trying to do security assessments and penetration testing uh on the whole system.
And uh this is gonna be a presentation a quick one about uh uh an attack technique that we
found repeating itself uh as useful uh more times than we thought it would. And I'm gonna
describe this as as being useful because it works, it's easy, it's uh it's pretty dramatic and
it provides a teachable moment about designing more secure systems. But it is a novelty and it's
a novelty because it is uh it's it's risky, it's crude and it's perhaps redundant to a lot of
other great techniques that are out there. But it sure is fun and when you do this uh you do
this device like you can you can demo this attack to your mom and she'll just be like
oh I get it. So let's just quick get to the demo. So to set the stage here uh we've used this on
a lot of devices but this one in particular is an embedded Linux device with uh uboot as the
bootloader. There's no JTAG that's actually been turned off with micro fuses inside the CPU.
So that's gone, can't find it. JTAGulator was great but couldn't find JTAG at all. And
because this device had had some challenges before let's say um it was uh it had they had a
home grown secure bootloader added. So getting to the demo here this is the system booting up
and you'll see why we call this pin to pwn in a moment. Uh this is my trusty like 1995 uh
multimeter just jamming into the flash chips and I'll show you where in just a moment. But as
this system is booting it's doing a cryptographic checksum across the flash image. And when
that checksum doesn't match what it's expecting it does a fallback. So this system is booting a
to a secondary partition. And it's going to start that in just a moment. And as you can see,
using a serial console on these devices, you can track what's going on. I didn't put the clip in
there, but I'm again poking the flash device to get the second partition to fail as well.
And they designed this device to not respond to any serial input. But what happens is they've
misconfigured it so that if it fails twice in a row, the primary and the secondary partition,
I get a U-boot prompt. Excellent. That gets me in.
So, the primary misconfiguration was not checking their failure paths. That happens, as I found
out, more often than you would think. And there's also going to be two kinds of flash devices
you'll see in these demos. Uh, this is a parallel flash device. It comes in a standardized
package, typically, a TSOP 48 package. You can actually, uh, you can actually use it to
attack from the left or the right side. Uh, I'm attacking on the right side between two of the
data lines that feed data out of the flash device, uh, when it's, uh, called by the CPU. Um, and-
and as you saw, it's pretty low tech. You just poke it. Um, but there's been a lot of prior work
on this. Uh, glitching as a form of attack has been done numerous times, uh, and just some
really amazing stuff, like extracting cryptographic keys with transient glitches induced by a
number of manners. Um, this is a very simple way to attack a touchpad. Uh, and you can actually, uh,
there's also been other blog posts about this precise thing, using, you know, a transient
electrical fault to get a failure mode that's advantageous to doing a penetration test. Um,
and so there's some of the links you can check out. But what I want to do today is just
provide more detail so that when you sit down, pull out some devices from the closet, try
this, you might be surprised, as I was, how often this works. Um, I have to warn you, uh,
well, first of all, this is Grog. Grog's our mascot. Uh, sometimes, you know, you try elegant
techniques and they work. Sometimes you just beat stuff with rocks. Um, and, and you get what
you want that way. Um, but you can definitely break your hardware doing this. I haven't yet
destroyed anything. But, you can. And, and the way you'll do that is by exceeding, um, there's
an absolute maximum current that can go out of each IC device. And if you exceed that for too
long, um, then that device will fail. Um, and, and, and, and, and, and, and, and, and, and, and,
and, and, and The device might just be, you know, kaput. The black smoke comes
out. You can also temporarily, uh, cause a device to fail. They have, sometimes, protection
circuitry. So that if you exceed the operating conditions, that pin just shuts off.
Usually power cycling will fix that. I've had that happen a few times. Um, and, and, of
course, depending on what access you have to the device. Like if you have JTAG, you don't
need to do this. Um, if you have other means to get what you need. Use the safer means,
certainly. It, it will prevent, or, you know, prevent your device, from protection. If you
you from breaking something. Uh if there are any time travelers in the audience go back and
listen to uh Joe Grand and Joe Fitzpatrick's um 101 ways to brick your hardware. I I think I'm
adding a new manner to do so. So let's get to the details about how can you mount this attack
yourself. This is the general architecture diagram for the kinds of devices that we work
with. CPU devices, 32 or 64 bit uh processors, running Linux, using typically a boot
looter, boot loader like U-boot. Um and you're trying to interrupt the communications between
the external flash device and that CPU. Uh these can either be serial or parallel flash
devices. The reason that this works is that systems boot in stages. And what's being shown
right here is the activity on the flash bus. You don't need to you know looking at trying to
decrypt the details of this by zooming in is not helpful for this but you want to get an idea
of like the wall clock timing on it.
And figure out well when is the boot loader being loaded? Like it's shown on the left. When is
the kernel being loaded? What's the duration where the device is booting? And then where does
the user space init process kick off? And you can actually attack in two different places. The
most successful for me has been interrupting um the loading of the kernel so that you fail to a
U-boot prompt. But I'll show you an example in a second of something that was it's actually
much more surprising to me.
So let's go back to the kernel. So let's go back to the kernel. So let's go back to the kernel.
So in this example this device had actually been pretty well secured. Um this this was a device
where where JTAG wasn't going to be an option uh you know based on how they designed the
hardware. Um but we I found a point later during the init process where poking the serial flash
device caused the init process to fail and give me exactly what I wanted out of this. A root shell
on the device. Yeah!
So this kind of misconfiguration is much more rare. The the forgetting to have a a useful
failure mode for load for U-boot that's pretty common. I've seen that a lot. Um but this one was
more rare and the reason for it is uh this this uh embedded Linux system had been really
cleanly set up. It was great work. Um and they had but they had left something in there to help the
developers out. That when the primary application was running it was not working. Um and so this is a
application started up it would grab all the serial ports and throw up an authentication prompt. It
wasn't Getty. It was something that was built into their application. Um and then um but then if
their application for whatever reason failed the next step in the init process would be just run a
shell. Now another thing that happened another characteristic of this system was important to
this. Uh this system was using Busy Box. Busy Box is like a uh a what's called a multi-call binary.
It's like a Swiss army knife that does all the things that it needs to do to get it to work. But the
typical Linux system needs but does it with just one binary. And so since most of the pages
for that had already loaded before the application we caused to fail had started, when that
application failed even though I was screwing up flash for it, BusyBox was already resident
memory. I think it might have been different if it had to go load other pages because I
wouldn't have been able to time that attack very elegantly. This is also an example of a
serial flash device. These typically have an 8 pin uh pin out, very standardized, much bigger
device you know so uh you can use a multimeter probe to you know to poke at this. And I was uh
poking between the chip select which says hey flash device read me out some of that data uh and
the actual data output on that. So here's another example, this was an LT router. Um on this
device um as you can see up in the top right corner you can see it's a serial flash device.
On your top left uh there's a little dim looking uh CPU module. The flash device was on the
underside of that dim device so that's outlined in in the red box below. So to get set up for
this I had to pop that device out, flip it over, put my pin where I wanted, throw a little
blue uh tape on it and then pull that pin out just a fraction you know a couple millimeters so
that it'd be ready for the attack. I'm working on the left side of a parallel flash device,
another TSOP48 pin out. It's a
standardized pin out so you can find this pretty frequently. And I'm probing uh the chip select
uh sorry the uh command latch and address latch lines. Shorting those two out. And what those do
is when that when that command latch line is toggled it says here's a command make sure you
read this in, do what I'm asking. And then here comes an address. And just by screwing with the
logic on that you got this flash chip confused enough to do what I wanted. So this is a quicker
demo. On this device just a little boop on the back side of the device. I'm going to go ahead
and pull the pin and uh we're going to end up back at our U-boot prompt which is where we wanted
to get on this. So doing your own pin to pwn attacks uh pretty straight forward. You need to
survey the hardware. Pop that thing open. You don't need to take anything apart at the PCB
level but just the key things you want to do is find all the flash storage devices, figure out
how you can access them. You need some way to monitor the boot process. A serial console is
great because that's probably the access that you're going to get. The developer
will turn the serial port on when something goes wrong. Um but you know there's other ways that
you could do that perhaps. You know monitoring with the logic analyzer and just getting a wall
clock time on it. Um you need some data sheets to figure out on the flash storage devices that
you have um where can you probe successfully. There's some trial and error on this. Not
everywhere I tried worked on each device. So you're going to have to take a few cracks at it.
And you need a way to be able to understand when the device fails. Maybe it's LED's. Maybe it's
something on the serial console. There'll be something. Because the last thing a developer
wants is for something to go wrong in the lab and for their device to be bricked. They left a door.
Find it. Um you got after you selected your pins you're going to going to start poking at that.
This took me a try. Usually about 6 or 8 attempts per device just to get the timing right on it.
Um but you know it's something you can work through pretty quick. Um make sure you power off
between each test. If some of that uh protection circuitry in the IC gets engaged power
cycling it's going to help you out. And then monitor for a different operational state. One
that's not the normal one. Um getting U-boot prompts uh pretty common but some devices have
different failure modes like uh enabling networking ports or failing to like a USB uh device
firmware upgrade. You'll need to find those. And you need a little bit of luck on this too. So
we pulled out all the devices in our closet and we said hey let's go for this. Let's see what other
devices can do this. And you know about a little bit over 50% of the time we were able to get some
failure mode that was helpful to what we were doing. You know getting root on one of these
devices is really just the start that's kind of like day one of our project. It just helps us do
the rest of the rest of the work. Um but it's really cool to be able to demo to people to say
watch this. Boop. And you end up with what you want. Um it's it's a great demo. I highly
recommend it. But let's talk about some of the places where I was unable to get uh to get root. The
devices that have thoughtful consideration of how to fail. Those are pretty resistant. The best
ones uh were the ones that would reset if it if they were unable to read flash on it. From a
consumer perspective that device is bricked. That's probably maybe a bad thing for business. Um
but from a security perspective that's a proper reaction to it. Ways that you can use flash
that you can improve your design or uh you know give recommendations to others. You know if you
can all turn on uh watchdog timers early in the init process at the bootloader level and then
start servicing them in user space so that something interrupts in between you're gonna
confound an attacker to a degree. Um you you know who's trying to get at this device. Um and
just be very cautious. You know about shipping fail to debug mode systems. Uh those are
exploitable. You just gotta find the way to get in.
The other one is really a design um decision. Uh hide your pins, hide your traces. Um BGA or
ball grid array devices uh they have an IC with little balls underneath. That mounts directly on
the PCB and uh the PCB designer can actually take signals and route them immediately through a
via to an inner layer of the printed circuit port. When you're at an inner layer it's it's hard
to get at them. Um you know traces that are on the uh outer layers of the PCB you can actually
scrape off the solder mask and get at those. Um but you're gonna have less options. And so it
just makes it harder for an attacker. So BGA devices uh some security conscious PCB design and
PCB routing uh can make a big difference. And and the last which I I think you know everybody
should be doing when they lay out these devices to help improve security is just be very
terse about what you're doing. If you have a serial console certainly don't accept input on it
unless you you know you have a serial console. If you have a serial console certainly don't accept
input on it unless you really want that for some reason. Um and boot that thing fast. Uh you can
get like embedded Linux systems to boot you know well under a second. And it just makes it hard
for a hand timed attack like this. You could do something more elegant and crafty of course. Um
but you know that's that's that's more work an attacker. Maybe they'll move elsewhere. So that's
the last bit of this. Um Max is here. Somewhere. Max is right there. We're gonna be uh outside to
take some questions. And uh we'll see you guys next time. Bye.
And if you ask some good questions we're gonna have something for you. But uh I'm Brad
Dixon. Uh thanks so much for your attention. Appreciate it.