We catch bots that follow links in tweets
Using CloudApp, I drew attention to the fact that the newly published links on Twitter get 18-20 hits at once. Obviously, these are robots, and I decided them to count.
I created an empty html file on my server and posted a link to it on Twitter. After that, I collected the values of the User-Agents that accessed this link. Moreover, I myself deleted the tweet almost immediately.
Lit services and products:
')
If you want to block them, then www.botsvsbrowsers.com is useful .
Ps. When posting a link to a “heavy” link on your website, keep in mind that each retweet will have +20 hits.
I created an empty html file on my server and posted a link to it on Twitter. After that, I collected the values of the User-Agents that accessed this link. Moreover, I myself deleted the tweet almost immediately.
Lit services and products:
- js-kit.com
- Googlebot
- www.oneriot.com
- www.postrank.com
- kmbot (some chinese bot - cl.ly/1pcT )
- Twingly Recon ( http://www.twingly.com )
- www.linksalpha.com
- tweetmeme.com
- www.ellerdale.com/crawler.html
- Topsy Butterfly ( http://labs.topsy.com/butterfly )
| IP | Referer |
|---|---|
| 38.113.234.181 | Voyager / 1.0 (twice) |
| 128.242.241.133 | Twitterbot / 0.1 |
| 204.236.175.30 | JS-Kit URL Resolver, js-kit.com (twice) |
| 66.249.71.218 | Mozilla / 5.0 (compatible; Googlebot / 2.1; + http: //www.google.com/bot.html) |
| 216.24.142.45 | Mozilla / 5.0 (Windows; U; Windows NT 5.1; en-US; rv: 1.9.1.7) Gecko / 20091221 Firefox / 3.5.7 OneRiot / 1.0 (http://www.oneriot.com) |
| 74.123.148.48 | Mozilla / 4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1) |
| 65.52.17.163 | Mozilla / 4.0 (compatible; MSIE 7.0; Windows NT 6.0) |
| 204.236.206.79 | PostRank / 2.0 (postrank.com) |
| 204.236.202.14 | Mozilla / 5.0 (compatible; kmbot-62c5 / 0.0; + http: //knowmore.com/bots) |
| 65.52.2.3 | Mozilla / 4.0 (compatible; MSIE 7.0; Windows NT 6.0) |
| 79.99.6.106 | Twingly recon |
| 174.129.146.212 | PycURL / 7.18.2 |
| 72.14.212.81 | AppEngine-Google; (+ http: //code.google.com/appengine; appid: linksalpha) |
| 89.151.116.54 | Mozilla / 5.0 (compatible; MSIE 6.0b; Windows NT 5.0) Gecko / 2009011913 Firefox / 3.0.6 TweetmemeBot |
| 70.37.65.108 | Mozilla / 4.0 (compatible; MSIE 7.0; Windows NT 6.0) |
| 64.13.147.188 | Mozilla / 5.0 (compatible; abby / 1.0; + http: //www.ellerdale.com/crawler.html) |
| 75.101.235.29 | - |
| 74.112.128.62 | Mozilla / 5.0 (compatible; Butterfly / 1.0; + http: //labs.topsy.com/butterfly/) Gecko / 2009032608 Firefox / 3.0.8 |
| 174.129.89.199 | Python-urllib / 2.5 |
')
If you want to block them, then www.botsvsbrowsers.com is useful .
Ps. When posting a link to a “heavy” link on your website, keep in mind that each retweet will have +20 hits.
Source: https://habr.com/ru/post/100620/
All Articles