Search for vulnerabilities in habrosite

It was evening time, there was nothing to do ... It was decided to surprise the masters. But how? Banal searches for vulnerabilities on well-known sites are no longer interesting to anyone. Well, then we turn on the fantasy ... After all, users also have websites! Urgently check their resistance!

How it all happened:

It was necessary to come up with a plan of action. In principle, it was all about parsing as many user pages as possible and then parsing the pages of sites into them. Then put all this on the scanner. Something like this…

How it works?

Point one: Sparse pages.
Not everything is so simple, so you know ... On the / users page there is not as much as you want, users. It was decided to recursively go through users and take there from subscribers of other users, and there are more users and more ...

Uncovered a favorite python, imagination is included, this is written:

import urllib2, re start = " " """    ,        """ page = urllib2.urlopen("http://habrahabr.ru/users/"+start+"/subscription/followers/").read() names = re.findall('<div class="username"><a href="/users/(.*?)/">(.*?)</a></div>', page) """  , ,    ,        """ def going(names): for name in names: page = urllib2.urlopen("http://habrahabr.ru/users/"+name[0]+"/subscription/followers/").read() names = re.findall('<div class="username"><a href="/users/(.*?)/">(.*?)</a></div>', page) """       """ base = open('habrs.txt', 'a') writed = 0 for item in names: base.write(item[0]+"\r\n") writed = writed + 1 print 'Saved habrs\'s: '+str(writed) going(names) going(names) 

')
After some time of work, a sufficient number of habra people (about 10k) were sparsened. It is clear that most of these people repeat ... What to do?

Point two: Unify
Thinking, I decided that it would be unfair to ship the python. Therefore the idea about array_unique from php got into my brain. Thought - done.

The whole script is to open-close the file, but on the other hand, uniqueization itself into one function.

 <?php $arr = file_get_contents("habrs.txt"); $arr = explode("\r\n", $arr); $arr = array_unique($arr); $string = ""; foreach($arr as $name){ $string = $string.$name."\r\n"; } file_put_contents("habrs2.txt", $string); ?> 

Everything started from the console, everything works fine.
It turned out a little less than 2k of unique habrois ... So now we need to go and find their sites.

Point three: Search sites
Again, I got a python, and wrote this. It's very simple, go through the file (line by line), go to user pages, look for sites. We find, we hide in the file.

 import urllib2, re sites = "" users = file("habrs2.txt") for user in users: user = re.split("\r\n", user)[0] page = urllib2.urlopen("http://habrahabr.ru/users/"+user+"/").read() site = re.findall('<a href="(.*?)" class="url icon" target="_blank">(.*?)</a>',page) if len(site) > 0: for site, fake in site: sites += site+"\r\n" with open("sites.txt", "a") as f: f.write(sites) 

Sites turned out a decent amount, and if they did not repeat, they had an indigestible look.
We need to clean them ...

Point Four: We Clean Websites

Use preg_match in php.

 <?php $arr = file_get_contents("sites.txt"); $arr = explode("\r\n", $arr); $string = ""; foreach($arr as $name){ if (preg_match('/http:\/\/([^\/]+)\//i', $name, $matches)) { $name = $matches[1]; } elseif (preg_match('/^http:\/\/([^\/]+)$/i', $name, $matches)) { $name = $matches[1]; } $string = $string.$name."\r\n"; } file_put_contents("sites2.txt", $string); ?> 

We get a good list, ala:
yandex.ru
google.com
yahoo.com

Now you need to screw it all to the scanner, get an answer, hide it in a file ...

Point five: flight with a scanner

A firmware was written on python to manipulate the console.
I am writing from memory, because this script was written on production (BackTrack r3 on a virtual machine), and after happy finding a huge heap of vulnerabilities the whole thing was closed, without saving (virtualka in the sense), so the script itself was not saved
The point is to run the nikto.pl script for 60 seconds, and that it will have time during this time, write to the file with the name of the site (for ease of further processing).

 import os, time sites = file("sites2.txt") for site in sites: os.system("perl nikto.pl -h "+site+" | tee "+site+".txt") os.system("pidof perl | tee perlID.txt") time.sleep(60) pid = file("perlID.txt")[0] os.system("taskkill "+pid) 

After this scanner was working, it was found in files of the form: site.txt, quickly found vulnerabilities (within 60 seconds).

Point Six: Debriefing

After some time of work (left for the night) I decided to see what kind of holes there ...
Because the habrasoobshchestvo quite prosharenno in this case (in the information security in the sense), some reports were completely without pleasing the eyes of the plus sign ...

However, among the 3 hundred sites were found and "holey" sites.

Vulnerability rating

one). All sorts of open folders and files not allowed to access. (40 ~ sites)
2). Various possible SQL and XSS sites. (20 ~ sites)
3). Hazardous information exchange protocols, non-password memcached and all sorts of holes in admin areas (no check for file access, or an empty password = full password). (10 ~ sites)
four). Candid bugs (or flaws). The lack of a password for the admin panel (the script cursed as many as 12 files, and sent to see what was interesting there). (2 sites)
five). An error encountered in a single copy, and then by mistake (I think), direct access to phpmyadmin with root login and without a password (default settings). There were NO ANY vulnerabilities found on this site, so I think this is just an oversight.

Not all users were checked, not all sites, not all possibilities from all possibilities!

I hope that this post will be useful to someone (some administrator will suddenly check something on his site).
Administrators of vulnerable sites notified. I hope for positive thoughts in your heads!