Anatomy of an attack: How I hacked StackOverflow

Almost two years ago, I came across a rather significant vulnerability on the StackExchange network of sites. I say stumbled because I did not try to hack the site. Circumstances opened the door for me. The vulnerability itself is quite interesting, and contains a lesson for everyone who creates and maintains websites or server infrastructure. So, here is the story of how I hacked StackOverflow ...

Initial data

At that time I worked in a small company where there was a firewall with completely draconian limitations. It cut all request and response headers that did not conform to the HTTP / 1.1 specification (and even cut valid HTTP / 1.1 headers). It played a trick on sites that rely on things like X-Requested-With . Therefore, for my “external adventures”, I set up a proxy.

I had several servers at that time, so I just put Squid on one of them. Since I understood something, I only allowed connections to the proxy with 127.0.0.1 . I raised the ssh tunnel to the server and pointed the browser to localhost as a proxy. The browser connected to the tunnel that connected to Squid on the server. Everything was great. Besides the fact that all my traffic was encrypted, I was able to use the sites normally.

For those of you who want to tell me that I did a bad thing, I want to draw your attention to the fact that I had the opportunity to do this. And not just an opportunity, I was openly told to use a proxy, since we had to work with some sites that did not work through our firewall. So I didn’t do anything wrong.
')

Attack

At that time, I often visited StackOverflow. Then he had just appeared, and there were a couple of bugs in him. One day, the site started showing me a call stack. I did not attach any importance to this, as I often met with this all over the Internet. In fact, almost every time I received an error message on a site written in ASP.NET, I saw a call stack.

And then I noticed a new menu item in the chat. This new menu item was called “admin”. Out of curiosity, I clicked on the link, believing that I would be denied access. What happened next surprised me. I got full access to everything. I had a developer console where I could see who was doing what. I had an interface for working with a database, where I could directly request anything from any database. I got full admin rights.

What happened next

The next thing I did - immediately reported it to the moderator. A few minutes later I was in a private chat with a moderator, as well as two developers. The reason was found in approximately 10 minutes. After another 10 minutes, the problem was solved superficially. A complete correction took a couple of hours. They worked great. I still have the log of that chat, and I want to say that these developers deserve all praise. They responded quickly and professionally. And solved the problem as soon as possible.

Vulnerability

If you are smart, then, most likely, guess what happened. Since I logged out through a proxy, the X-Forwarded-For header was added to my requests. The value of this header was the IP from which the proxy server was requested. But, since I connected to the proxy through an SSH tunnel, the IP was local. So Squid added X-Forwarded-For: 127.0.0.1 ...

But the most interesting thing was what the ASP showed in the logs. When they looked at the dumps of my requests, there was the REMOTE_ADDR: 127.0.0.1 entry! In their application, all header checks were implemented correctly. But IIS was incorrectly configured and it overwritten REMOTE_ADDR value X-Forwarded-For , if it was present. And thanks to such a configuration error, I was able to access the admin panel using my proxy server.

findings

Never rely on X-Forwarded-For , as we see - it is not safe. Always use REMOTE_ADDR . It is worth considering whether you need IP protection at all. Or, at least, you should not completely rely on it, and use only as an additional measure.

It is interesting to note that the developers did use proper header validation. The bottom line is that you should never blindly trust your infrastructure. This hole appeared due to the difference in configuration between the server and the application. These little things happen every day. The application assumes one, and the server - another. The problem is that such trust can completely undermine security. In this case, the developers trusted the REMOTE_ADDR value (quite justified), but the server was incorrectly configured. Of course, there are situations where you have to trust the server or other components, but you have to remember that blind trust is not a good thing. Think about it, and try to insure against such cases.

The StackOverflow team solved this question wonderfully. Quick, responsive and reasonable. They asked me for help (which I gladly provided), and behaved professionally and respectfully. They did a fantastic job. We all need to learn a lesson. Respond to reports of vulnerabilities seriously. Solve the issue professionally and quickly.

As for PHP

The most interesting thing here is that applications written in PHP may have the same vulnerability. We look at the Symfony2 Request class . Looks great. Until you notice that it uses a static variable to determine whether to trust the proxy. This means that if any part of your application wants to get headers from the proxy (for example, to write to the log), then your entire application will receive headings from the proxy after that. To see if your code is affected by a similar vulnerability, look for the $request->trustProxy() call in it. Also note that there is no reverse mechanism. You can not "stop trusting" the proxy. I think this is a big architectural miscalculation ...

Zend Framework 2 does the same. It has a class that behaves in a similar way (in terms of obtaining IP). Interestingly, in Zend Framework 1, the way to get an IP address was adequate. The calling code must explicitly indicate what it wants to receive, and by default it should be a safe option.

Conclusion

This problem was the result of a combination of two minor miscalculations. Individually, they are very easy to lose sight of. But if they converge in a certain way, you will get a very serious security threat. And the biggest lesson is that you really can't trust anything outside of your application. If you can get around this (for example, by not trusting headers and variables, like REMOTE_ADDR ), then you can make your application more secure. But above all, think about the code you write and the systems you build. And support them.