Whois: practical user guide
The article talks about the work of the whois protocol, existing client solutions and the features of communication with different whois servers (as well as the choice of the right whois server). Its main task is to help in writing scripts to get whois information for IP addresses and domains.
What is and why you need whois can be read, for example, here: http://en.wikipedia.org/wiki/Whois .
In a few words, whois (from the English "who is" - "who is") is a network protocol based on the TCP protocol. Its main purpose is to obtain in a text form registration data on the owners of IP addresses and domain names (mainly their contact information). A domain record usually contains the name and contact information of the “registrant” (domain owner) and “registrar” (organization that registered the domain), the names of the DNS servers, the date of registration, and the expiration date. The IP address records are grouped by range (for example, 8.8.8.0 - 8.8.8.255) and contain information about the organization to which this range has been delegated.
')
Often, whois is used to check if the domain name is free or already registered. Theoretically, this can be done simply by opening the domain in the browser, but in practice the registered name is not always used.
It is worth noting that all contact information is entered at the time of registration, and over time they may become outdated. Each registrar has its own procedure for updating them, in addition, the process itself may take some time (although this usually happens within 24 hours).
The protocol implies a client-server architecture and is used to access public database servers (as a rule, IP registrars and domain names themselves). In some cases, a whois server for a specific top-level domain contains a complete database of all registered domains. In other cases, such a whois server contains only the most basic information and sends it to specific servers of specific registrars whois.
The remaining details will be reviewed in the course of the article itself. In fact, there is a lot of information, so especially for those who want to get only a general idea, without delving into technical details, there is its “squeeze”: you can skip straight to the last section of the article, called “ Short Results ”.
There is a software product, an Internet portal that provides various information about IP addresses and domains, including whois information. From time immemorial, a third-party utility jwhois was used for these purposes, and no one has penetrated into its work. However, after migrating the product to the new version of FreeBSD, it suddenly turned out that jwhois stopped working: for most domains an absolutely illogical error “Unable to connect to remote host” began to be generated. Google did not help, and I was ready to start debugging the C-shny code, when it turned out that jwhois doesn’t suit us at all because of its license (GPL v3). In general, the problem arose of finding some alternative solution.
To my surprise, there were no sane alternatives available. Most of the forums just recommended using the same jwhois. Of course, several programs were found (including several libraries in the native Python language for our product), but most of them were rejected literally after the very first test for a domain in the “ua” zone. In general, all solutions looked frankly written “on the knee” and absolutely unsuitable for a serious product. The only library that at the time was credible was written in Ruby (which didn't suit us at all) and had an awesome source code size and different configuration files for each particular whois server.
There was still a standard Unix-s utility, and the so-called "whois", but her work left much to be desired.
In general, the only option was to sit down to read the specifications for the protocol and write the solution yourself.
The result was a module in Python for 1000 lines of code, in the process of writing which I consistently stepped on all the accompanying rakes, and, ultimately, this article, which tells about all these “rakes” (yes, if anything, the code proprietary).
Looking ahead, I note that from the height of the acquired experience, both jwhois and Ruby Whois ( http://www.ruby-whois.org ) now also look "leaving much to be desired."
All whois work is described in RFC 3912 ( http://tools.ietf.org/html/rfc3912 ) and takes up as many as 4 pages. In a few words, it all comes down to the following: open a TCP connection on port 43 to the desired whois server, send a request in a specific format (which can be anything for a particular whois server), finish it with "\ r \ n" and get the result, format which for a particular whois server can also be anything. Closing the server connection means the end of the result. Everything! In other words, each whois server determines the format of communication at its own discretion. This is not to mention the fact that it is absolutely not obvious where to get the necessary whois server for a specific domain or IP address.
Of course, I naively hoped that I would find on the Internet a sea of ​​practical articles in which all this would be written in detail, but as a result I did not find one. Basically, it all came down to chewing on the scarce information from the RFC and the names of several of the most famous whois servers. Although some scattered information still managed to find.
In general, I had to look at the Unix code of whois, as well as jwhois and Ruby Whois - their principle of operation will be discussed further.
So, what is the job of the Unix whois?
Note: additional functionality that did not interest me (not related to domains and IP addresses) is hereinafter omitted.
As you can see, everything is extremely simple. And, in fact, in many cases it works. For many domains, the <top-level domain> .whois-servers.net is an alias of the real whois server (but for many it isn’t, and the request is naturally broken). Also, whois.arin.net for many IP addresses for which he himself is not responsible, nevertheless, knows the correct regional server and sends it to him correctly (however, in some cases, he still does not know or sends to him in an arbitrary format, not necessarily with the mention of the name of the whois server itself, what the program is counting on). Most whois servers really understand the format of "<domain or IP address> \ r \ n", but exceptions are not limited to two servers. And, of course, the link to another whois server does not necessarily have to be strictly in the format of “Whois Server: <server name>”.
In addition, often the result is not quite what we expect. For example:
It turns out that the whois server knows about a number of domains that are similar to google.com and does not know which one to choose.
Or another example:
As you can see, the IP address was first delegated to one organization within a certain range, and later redelegated to another organization within an already smaller subrange. And the whois server again does not know what kind of information interests us.
In other words, the program relies on the most general principle of the whois protocol and is absolutely not ready for any exceptions. For discovery of domain whois servers, whois-servers.net is used, which is currently not the most efficient way.
Now let's see how the most popular whois utility works. Unlike Unix-based whois, it does not do any discovery, but relies entirely on a kilometer configuration file (about a thousand lines!), In which whois servers for all known top-level domains, for a number of specific second-level domains (which have own whois servers) and even for specific ranges of IP addresses. Needless to say, in today's rapidly changing world, this configuration file will require daily updates and is still unlikely to ever be 100% relevant? In the file header there is a link to the repository, from which you can download the latest version, and it turned out that it is dated April 2011! During this time, many new whois servers have appeared, and some of the old ones no longer work. I am sure that a whole range of new IP ranges has been delegated - especially for IPv6.
In addition to the actual names of whois servers, the file in a special format describes all known exceptions for request formats, as well as the format of links to other whois servers (much more than in the Unix-whois).
Also found another interesting functionality. For many top-level domains, whois actually do not have servers, or access to them is limited, but whois information is available through the web on their website. So, jwhois can send a GET or POST request, get the result and then bite out the necessary information from HTML. The address of the pages, the names of the form parameters and the format of the result for each case are also hardcoded in the configuration file. Naturally, in many cases, these data are already outdated, and “scraping” does not work. Some sites simply added a captcha to their forms - perhaps, just against similar programs.
Returning to the examples above, jwhois correctly yields the result for google.com, but with 8.8.8.8 it is no better than Unix-whois.
In general, as it turned out, jwhois is not such a good program, as it was thought. Relying only on a hard-boiled list of servers (even if it was regularly updated) is clearly not the best idea.
In fact, Ruby Whois works the same way as jwhois, with the only difference that updates to the configuration come out almost every week. Their changelog is replete with updates like "Update whois.nic.ms to the new response format", "whois.coza.net.za became whois.registry.net.za" or "Added .AX definition and parser". In addition, the configuration is not a single file, but a whole tree of files, where for each whois server the query format and result format are punctually recorded, as well as a sample server response for existing and not existing domains (probably for unit tests).
If this program analyzed the ancient Egyptian papyrus, then, of course, it would make sense to describe in detail the format of all known types of documents of each of the pharaonic dynasties. But doing this for whois servers, which change almost every day, is probably not the most rewarding activity. Somehow it’s a pity that the authors of the project are clearly not an easy task to support it.
At that time, I already had the names of several “tricky” whois servers (and an understanding of where they could be taken from) - there was no information in the Ruby Whois configuration about them.
I did not have the opportunity (and special desire) to run Ruby Whois, so I don’t know if he could cope with “8.8.8.8” or not.
Another program that I was advised in the comments to the article. It is written in Perl and is a wrapper for the Net :: Whois :: Raw module. Both can be downloaded here: http://search.cpan.org/~despair/Net-Whois-Raw-2.43/ . Last updated August 2012.
The program uses the same hardcode method: there is a huge configuration file (this time, two thousand lines), in which the names of whois servers and everything else are wired.
The main disadvantage is the same: just do not bother. There are no many whois servers in the configuration; accordingly, it is impossible to obtain information about a whole range of domains. Well, the program also failed with "8.8.8.8".
By this time, I already had some ideas about how the “correct” whois should work, and I set about developing the initial version. All further "knowledge" was gained experimentally as a result of a variety of whois queries and analysis of their results. Fortunately, much of this could be automated.
I would also like to mention the site http://whois.domaintools.com - the best whois web service that I managed to find. Naturally, I did not have the opportunity to see its source code, but in many cases a comparison of its results with mine was a good “tip”. I will not voice assumptions about how it works, but in fact it showed much better results than the above-mentioned programs (as I mentioned, I judge the possibilities of Ruby Whois only by code).
I would reduce working with whois to solving three fundamental problems:
So let's get started.
First, a few comments.
Whois information may not be available for all top-level domains. Some countries whois do not provide information for their domains in principle (for example, the DPRK). In addition, some of the African countries do not have their whois servers (perhaps they simply have no money or have eaten all the programmers).
For some top-level whois domains, information is available only on the registrar's site. In some cases, you need to enter a captcha, in others it can be implemented programmatically. However, I completely abandoned the idea of ​​“scraping”. Its correct operation requires a large amount of information (the address of the page, the names of the form fields, the format of the received HTML page, etc.), which I strongly wanted to avoid. In addition, this information requires constant updating, as it may change even more often than the names of whois servers.
Well, and most importantly, if whois information is displayed to site visitors, you can simply provide a link to the registrar's site and allow the user to fill out the form himself. In most cases, the whois form is presented either on the main page, or you can go to the page with one click. This is a much more reliable way than relying on “scraping”, which in any case will not be able to cope with captcha. Where to get the link to the website of the registrar will be described below.
Also, some whois servers may ban the IP addresses of users who send too many requests.
So where do you get the name of the correct whois server?
You can take it from several sources:
1. There is such a wonderful whois server whois.iana.org, owned by IANA ( http://en.wikipedia.org/wiki/Internet_Assigned_Numbers_Authority ). It contains the latest information on all top-level domains. For example (hereinafter all the examples are using Unix-whois):
Pay attention to the following lines:
This is actually the whois server and the registrar site! For most top-level domains, IANA returns the completely current name of the whois server. I specifically checked the changes in Ruby Whois over the past few months, and all of them were taken from whois.iana.org.
In my module, I cache the result from IANA for 24 hours (for a longer period of time there is no special meaning, and the data can change).
Despite all the above, for some top-level domains, IANA for some reason does not provide information about whois servers. Perhaps the registrars of some countries do not advertise them specifically. But not scary, we have other ways.
2. If you look closely, for very many top-level domains, the names of their whois servers look either as whois.nic. <Top level domain> (more common), or as whois. <Top level domain> (less common). For example:
Moreover, even if IANA indicates some other whois server (for example, whois.registro.br), in many cases an alias built according to this scheme (for our example, whois.nic.br) will still work.
Thus, if IANA did not return anything to us, for any top-level domain, we easily get the names of two potential whois servers:
Practice has shown that this method identifies existing whois servers for a dozen top-level domains for which IANA does not return information.
Quite often, if we are looking for a third-level domain (for example, russia.edu.ru), then the whois server responsible for the top-level domain does not contain the necessary information. For example:
This is due to the fact that for many second-level domains (in our example, edu.ru) a completely different organization is responsible, which has its own whois server.
In such cases, jwhois and Ruby Whois meticulously hardcodes the name of the whois server for each second level domain they know with their own server. Naturally, it is absolutely unrealistic to keep track of all second-level domains, especially since there is no centralized source of such information.
But again, if you look closely, most of these whois servers are of the form whois. <Second-level domain> (more common) or whois.nic. <Second-level domain> (less common). For example:
How to deal with those whois servers that are called differently? For example, in the Ruby Whois configuration, you can see:
From the very beginning, I was mentally prepared for the necessity of hard-coding several whois servers and in my own code, however, going through absolutely all the similar servers mentioned in the jwhois and Ruby Whois configurations, it turned out that all of them are also available also for whois aliases . <second level domain> or whois.nic. <second level domain>! For our example:
In the end, I did not have to hardcode a single whois server at all (of course, except for whois.iana.org).
3. In addition to the above, the name of the whois server may still look like whois. <Server of the registrar site>. For example:
In some cases, when IANA returns only the registrar's site without a whois server, the name built according to this scheme is correct.
4. As already mentioned, whois-servers.net contains aliases to whois servers only for a fairly limited number of top-level domains, but in some cases when all of the above does not help, an alias of the form <top level domain> .whois-servers.net is quite workable (for example, for "ps").
Thus, for each domain that we want to find, we will have a whole list of potential whois servers with a greater or lesser probability of efficiency. For example, for russia.edu.ru we will have:
Which one to choose? In fact, there is no need to choose anything, since it is not difficult to poll all found whois servers in order of priority. Naturally, if the result is found, then it is no longer necessary to poll the following servers. In addition, if one of these whois servers does not actually exist (and among the “fictional” ones there will, naturally, be the majority), then this information can be easily cached.
I would like to tell about the cache separately.
Whois server from IANA and whois servers found by links (more on that below) are considered “active” by default. When the active server does not respond (meaning, the server is not available in principle, and not just did not find the necessary information for a particular domain and returned an error), its status changes to “temporarily unavailable”. When a temporarily unavailable server does not respond, it is blocked for one minute, then two, four, eight, etc., until this period is more than two weeks. In this case, the server status changes to “inactive”, and it is blocked for exactly two weeks. If an inactive server does not respond after two weeks, it remains inactive and again blocked for two weeks. If a temporarily unavailable or inactive server suddenly responded, its status changes to active. All “fictional” whois servers and servers of the form <top level domain> .whois-servers.net are considered inactive by default. That is, if they do not respond, they are immediately blocked for two weeks.
This scheme allows you to filter all non-existing whois servers and at the same time does not allow for a long time to block servers that for one reason or another have become temporarily unavailable.
It should be noted that the next time we look for a domain in the edu.ru zone, two whois servers will be considered active at once:
However, the fact that whois.edu.ru is active indicates that the first whois server most likely does not contain the information we need (otherwise it would not reach whois.edu.ru, and it would not become active). Therefore, it would be advisable to put it first in priority and interrogate whois servers in this order:
If we, for example, look for daily.lviv.ua (also a third-level domain), then the result for it will most likely be returned by the very first whois server that is returned to IANA (whois.ua). Therefore, the whois.lviv.ua server will not become active (perhaps it does not exist - I did not check it), and the next time we will look again at the domain in the lviv.ua zone on whois.ua.
In other words, after a while, requests to non-existing or incorrect whois servers will be minimized.
Often the result that the whois server issued is incomplete, but it contains a link to another whois server that contains more complete information. For example:
As you can see, the information is not so much, but there is a link to another whois server:
If we now turn to it, we get:
Thus, in order to get the most complete information, we should look for links to other whois servers as a result. Most often they look like “whois server: <name>”, but often the link is also in a completely unpredictable place (for example, in the middle of the text).
My module is looking for links rather aggressively, taking for a whois server any string that looks like a hostname and contains the word “whois” (as we found out, “extra” whois servers are not a problem). If we found the link in the “whois server: <name>” format, the server name no longer has to contain the word “whois” (in practice, this may not even be a hostname, but simply an IP address). All found whois servers are by default considered active.
The truth is there is one pitfall. The result may contain the name of your own whois server, and if it happened that we applied to it for some alias, it would look like a link to another whois server. For example:
The whois.jprs.jp is mentioned here, the alias of which, in fact, is whois.jp. That is, if we now send a request to whois.jprs.jp, we will get the same result. , whois - whois — , .
, - : -, , -, .
, : , , (, whois ). However, not all so simple. , — , , , . whois , , , - . : whois ! , .
«X», , "#" "%" (), , , «trim» . ( set Python), , .
IDN ( «.»), , . «.» «xn--80aealotwbjpid2k.xn--p1ai» whois.iana.org «xn--p1ai»:
IP . whois :
, IP ( IPv4, IPv6) . - IP ( IPv6), IANA. For example:
- IP , , , . whois , whois.iana.org, - whois .
ARIN, IP ( IP ). , IP ARIN whois . IANA , .
, AfriNIC — whois . IP ARIN, RIPE APNIC. whois , IP . For example:
ARIN , 213.154.64.0 RIPE. RIPE, , 213.154.64.0 — 213.154.95.255 AfriNIC.
. whois.lacnic.net , - IP , , , . , LACNIC. , : whois (RIPE AfriNIC) IP , . , 10000 , , , AfriNIC. LACNIC , , , IP , . , - IP AfriNIC, , LACNIC , , . LACNIC .
whois , . . ARIN whois «ReferralServer: <>». IP , , , whois , , , LACNIC RIPE.
, , - whois . ?
43, "< IP >\r\n" , . , , ( , , ).
, RFC 3912 "\r\n", whois , "\n" "\r\n" ! whois whois.nic.org.mt.
, whois :
VeriSign — whois.verisign-grs.com. ? "<>\r\n" `To single out one record, look it up with «xxx»` `look them up with "=xxx" to receive a full display`, VeriSign . «domain <>\r\n», whois VeriSign .
whois.arin.org "<IP >\r\n", :
whois.nic.name . For example:
( , ). :
:
whois , :
, whois - . We try:
, , «name», .
whois ( ). , - . , "-B" whois.ripe.net «notify», «changed» «e-mail», :
telnet, Unix- whois .
whois . :
, , . , .
, ( , , 99% ). whois , , ( ).
:
, , whois, , rwhois. .
rwhois («Referral Whois») : http://en.wikipedia.org/wiki/Whois#Referral_Whois . , , whois , , . 4321. , rwhois 1994- , .
rwhois , ARIN ( - whois ). , «rwhois». , ARIN , : «whois://» «rwhois://». rwhois , , , "<IP >\r\n" . .
Referral Whois RFC 2167 ( http://tools.ietf.org/html/rfc2167 ). 170. , Joel- Spolsky: « - - , , . , ».
- , Google . , telnet- , , , . http://projects.arin.net/rwhois/ C GPL v2 . 100 rwhois 1.0 1.5 — . rwhois , , . jwhois rwhois, ( , ).
, rwhois , subpocess rwhois , .
: , rwhois . , «error querying rwhois server». Google, , . , jwhois .
, , , : «rwhois» whois! , «, rwhois 1.0 rwhois 1.5?» "<IP >\r\n" ! , , .
, « » rwhois , , whois, rwhois . Joel Spolsky .
, - rwhois - : . , , / . Rwhois . .
, rwhois - rwhois , whois . , whois .
— . : whois . . whois , : IP ( ).
, :
IP whois , :
, ( ):
, . , , , .
ARIN (, 8.8.8.8), :
#end #start , ( 8.8.8.0 — 8.8.8.255).
:
1. , IP ( whois ).
2. HTML ( whois , ).
3. whois :
: whois , ( ), ( jwhois http://whois.domaintools.com ). () , () , . .
. , .
whois RFC 3912 ( http://tools.ietf.org/html/rfc3912 ), whois «». , : TCP 43 whois , , "\r\n" . whois . , whois IP .
whois Unix- «whois», , . jwhois Ruby Whois — , , whois . , , ( , jwhois 2011- ).
, whois , , whois .
, whois :
, , whois . whois :
, whois . , whois , ( ), .
, whois , whois . , «whois server: <>», -. , whois , whois , ( , whois ).
IP . whois :
IP ( IPv4, IPv6) . - IP , whois.iana.org. whois , «» whois «». ARIN, IP . , whois , whois .
, whois.ripe.net whois.afrinic.net IP , . , whois.lacnic.net , , whois (, «» whois.ripe.net whois.afrinic.net).
whois "< IP >\r\n", whois ( " ? ").
whois , rwhois («Referral Whois») 4321 ( http://en.wikipedia.org/wiki/Whois#Referral_Whois http://tools.ietf.org/html/rfc2167 ). , rwhois (, http://projects.arin.net/rwhois/ ). , , rwhois, whois! rwhois .
whois - , . , whois . , , " ? ".
, whois IP , . whois HTML . whois utf8, whois iso-2022-jp euc-kr .
, whois , . , , , .
What is whois?
What is and why you need whois can be read, for example, here: http://en.wikipedia.org/wiki/Whois .
In a few words, whois (from the English "who is" - "who is") is a network protocol based on the TCP protocol. Its main purpose is to obtain in a text form registration data on the owners of IP addresses and domain names (mainly their contact information). A domain record usually contains the name and contact information of the “registrant” (domain owner) and “registrar” (organization that registered the domain), the names of the DNS servers, the date of registration, and the expiration date. The IP address records are grouped by range (for example, 8.8.8.0 - 8.8.8.255) and contain information about the organization to which this range has been delegated.
')
Often, whois is used to check if the domain name is free or already registered. Theoretically, this can be done simply by opening the domain in the browser, but in practice the registered name is not always used.
It is worth noting that all contact information is entered at the time of registration, and over time they may become outdated. Each registrar has its own procedure for updating them, in addition, the process itself may take some time (although this usually happens within 24 hours).
The protocol implies a client-server architecture and is used to access public database servers (as a rule, IP registrars and domain names themselves). In some cases, a whois server for a specific top-level domain contains a complete database of all registered domains. In other cases, such a whois server contains only the most basic information and sends it to specific servers of specific registrars whois.
The remaining details will be reviewed in the course of the article itself. In fact, there is a lot of information, so especially for those who want to get only a general idea, without delving into technical details, there is its “squeeze”: you can skip straight to the last section of the article, called “ Short Results ”.
A little background
There is a software product, an Internet portal that provides various information about IP addresses and domains, including whois information. From time immemorial, a third-party utility jwhois was used for these purposes, and no one has penetrated into its work. However, after migrating the product to the new version of FreeBSD, it suddenly turned out that jwhois stopped working: for most domains an absolutely illogical error “Unable to connect to remote host” began to be generated. Google did not help, and I was ready to start debugging the C-shny code, when it turned out that jwhois doesn’t suit us at all because of its license (GPL v3). In general, the problem arose of finding some alternative solution.
To my surprise, there were no sane alternatives available. Most of the forums just recommended using the same jwhois. Of course, several programs were found (including several libraries in the native Python language for our product), but most of them were rejected literally after the very first test for a domain in the “ua” zone. In general, all solutions looked frankly written “on the knee” and absolutely unsuitable for a serious product. The only library that at the time was credible was written in Ruby (which didn't suit us at all) and had an awesome source code size and different configuration files for each particular whois server.
There was still a standard Unix-s utility, and the so-called "whois", but her work left much to be desired.
In general, the only option was to sit down to read the specifications for the protocol and write the solution yourself.
The result was a module in Python for 1000 lines of code, in the process of writing which I consistently stepped on all the accompanying rakes, and, ultimately, this article, which tells about all these “rakes” (yes, if anything, the code proprietary).
Looking ahead, I note that from the height of the acquired experience, both jwhois and Ruby Whois ( http://www.ruby-whois.org ) now also look "leaving much to be desired."
What is the problem?
All whois work is described in RFC 3912 ( http://tools.ietf.org/html/rfc3912 ) and takes up as many as 4 pages. In a few words, it all comes down to the following: open a TCP connection on port 43 to the desired whois server, send a request in a specific format (which can be anything for a particular whois server), finish it with "\ r \ n" and get the result, format which for a particular whois server can also be anything. Closing the server connection means the end of the result. Everything! In other words, each whois server determines the format of communication at its own discretion. This is not to mention the fact that it is absolutely not obvious where to get the necessary whois server for a specific domain or IP address.
Of course, I naively hoped that I would find on the Internet a sea of ​​practical articles in which all this would be written in detail, but as a result I did not find one. Basically, it all came down to chewing on the scarce information from the RFC and the names of several of the most famous whois servers. Although some scattered information still managed to find.
In general, I had to look at the Unix code of whois, as well as jwhois and Ruby Whois - their principle of operation will be discussed further.
Unix whois
So, what is the job of the Unix whois?
- If we are looking for a domain, then we define a top-level domain for it (for example, “ru”) and send a request to a whois server like ru.whois-servers.net. If the result obtained contains a string of the form “Whois Server: <server name>”, then send the request also to this new server and combine the results.
- If we are looking for an IP address, then send a request to whois.arin.net (whois server responsible for North America). If the result is mentioned by one of the regional servers known to us (Latin American, European, Pacific and African), then we also send a request to this new server and combine the results.
- For most whois servers, the request is sent in the format "<domain or IP address> \ r \ n". For two whois servers (whois.denic.de and whois.dk-hostmaster.dk), the request is sent in its own special format.
- The user has the opportunity to specify the whois server to which to contact.
Note: additional functionality that did not interest me (not related to domains and IP addresses) is hereinafter omitted.
As you can see, everything is extremely simple. And, in fact, in many cases it works. For many domains, the <top-level domain> .whois-servers.net is an alias of the real whois server (but for many it isn’t, and the request is naturally broken). Also, whois.arin.net for many IP addresses for which he himself is not responsible, nevertheless, knows the correct regional server and sends it to him correctly (however, in some cases, he still does not know or sends to him in an arbitrary format, not necessarily with the mention of the name of the whois server itself, what the program is counting on). Most whois servers really understand the format of "<domain or IP address> \ r \ n", but exceptions are not limited to two servers. And, of course, the link to another whois server does not necessarily have to be strictly in the format of “Whois Server: <server name>”.
In addition, often the result is not quite what we expect. For example:
$ whois google.com GOOGLE.COM.ZZZZZZZZZZZZZZZZZZZZZZZZZZZ.LOVE.AND.TOLERANCE.THE-WONDERBOLTS.COM GOOGLE.COM.ZZZZZZZZZZZZZZZZZZZZZZZZZZ.HAVENDATA.COM GOOGLE.COM.ZZZZZZZZZZZZZ.GET.ONE.MILLION.DOLLARS.AT.WWW.UNIMUNDI.COM GOOGLE.COM.ZZZZZ.GET.LAID.AT.WWW.SWINGINGCOMMUNITY.COM GOOGLE.COM.ZOMBIED.AND.HACKED.BY.WWW.WEB-HACK.COM ... GOOGLE.COM.AU GOOGLE.COM.AR GOOGLE.COM.ALL.THE.PEOPLE.WHO.SPAM.THE.WHOIS.ARE.SERIOUSLY.ANNOYING.SOMEPONY.COM GOOGLE.COM.AFRICANBATS.ORG GOOGLE.COM.9.THE-WONDERBOLTS.COM GOOGLE.COM.1.THE-WONDERBOLTS.COM GOOGLE.COM To single out one record, look it up with "xxx", where xxx is one of the of the records displayed above. If the records are the same, look them up with "=xxx" to receive a full display for each record. It turns out that the whois server knows about a number of domains that are similar to google.com and does not know which one to choose.
Or another example:
$ whois 8.8.8.8 Level 3 Communications, Inc. LVLT-ORG-8-8 (NET-8-0-0-0-1) 8.0.0.0 - 8.255.255.255 Google Incorporated LVLT-GOOGL-1-8-8-8 (NET-8-8-8-0-1) 8.8.8.0 - 8.8.8.255 As you can see, the IP address was first delegated to one organization within a certain range, and later redelegated to another organization within an already smaller subrange. And the whois server again does not know what kind of information interests us.
In other words, the program relies on the most general principle of the whois protocol and is absolutely not ready for any exceptions. For discovery of domain whois servers, whois-servers.net is used, which is currently not the most efficient way.
jwhois
Now let's see how the most popular whois utility works. Unlike Unix-based whois, it does not do any discovery, but relies entirely on a kilometer configuration file (about a thousand lines!), In which whois servers for all known top-level domains, for a number of specific second-level domains (which have own whois servers) and even for specific ranges of IP addresses. Needless to say, in today's rapidly changing world, this configuration file will require daily updates and is still unlikely to ever be 100% relevant? In the file header there is a link to the repository, from which you can download the latest version, and it turned out that it is dated April 2011! During this time, many new whois servers have appeared, and some of the old ones no longer work. I am sure that a whole range of new IP ranges has been delegated - especially for IPv6.
In addition to the actual names of whois servers, the file in a special format describes all known exceptions for request formats, as well as the format of links to other whois servers (much more than in the Unix-whois).
Also found another interesting functionality. For many top-level domains, whois actually do not have servers, or access to them is limited, but whois information is available through the web on their website. So, jwhois can send a GET or POST request, get the result and then bite out the necessary information from HTML. The address of the pages, the names of the form parameters and the format of the result for each case are also hardcoded in the configuration file. Naturally, in many cases, these data are already outdated, and “scraping” does not work. Some sites simply added a captcha to their forms - perhaps, just against similar programs.
Returning to the examples above, jwhois correctly yields the result for google.com, but with 8.8.8.8 it is no better than Unix-whois.
In general, as it turned out, jwhois is not such a good program, as it was thought. Relying only on a hard-boiled list of servers (even if it was regularly updated) is clearly not the best idea.
Ruby whois
In fact, Ruby Whois works the same way as jwhois, with the only difference that updates to the configuration come out almost every week. Their changelog is replete with updates like "Update whois.nic.ms to the new response format", "whois.coza.net.za became whois.registry.net.za" or "Added .AX definition and parser". In addition, the configuration is not a single file, but a whole tree of files, where for each whois server the query format and result format are punctually recorded, as well as a sample server response for existing and not existing domains (probably for unit tests).
If this program analyzed the ancient Egyptian papyrus, then, of course, it would make sense to describe in detail the format of all known types of documents of each of the pharaonic dynasties. But doing this for whois servers, which change almost every day, is probably not the most rewarding activity. Somehow it’s a pity that the authors of the project are clearly not an easy task to support it.
At that time, I already had the names of several “tricky” whois servers (and an understanding of where they could be taken from) - there was no information in the Ruby Whois configuration about them.
I did not have the opportunity (and special desire) to run Ruby Whois, so I don’t know if he could cope with “8.8.8.8” or not.
pwhois
Another program that I was advised in the comments to the article. It is written in Perl and is a wrapper for the Net :: Whois :: Raw module. Both can be downloaded here: http://search.cpan.org/~despair/Net-Whois-Raw-2.43/ . Last updated August 2012.
The program uses the same hardcode method: there is a huge configuration file (this time, two thousand lines), in which the names of whois servers and everything else are wired.
The main disadvantage is the same: just do not bother. There are no many whois servers in the configuration; accordingly, it is impossible to obtain information about a whole range of domains. Well, the program also failed with "8.8.8.8".
What's next?
By this time, I already had some ideas about how the “correct” whois should work, and I set about developing the initial version. All further "knowledge" was gained experimentally as a result of a variety of whois queries and analysis of their results. Fortunately, much of this could be automated.
I would also like to mention the site http://whois.domaintools.com - the best whois web service that I managed to find. Naturally, I did not have the opportunity to see its source code, but in many cases a comparison of its results with mine was a good “tip”. I will not voice assumptions about how it works, but in fact it showed much better results than the above-mentioned programs (as I mentioned, I judge the possibilities of Ruby Whois only by code).
I would reduce working with whois to solving three fundamental problems:
- Determining the correct whois server;
- Sending the correct request to the server;
- Analysis of the result.
So let's get started.
How to determine the correct whois server for the domain?
First, a few comments.
Whois information may not be available for all top-level domains. Some countries whois do not provide information for their domains in principle (for example, the DPRK). In addition, some of the African countries do not have their whois servers (perhaps they simply have no money or have eaten all the programmers).
For some top-level whois domains, information is available only on the registrar's site. In some cases, you need to enter a captcha, in others it can be implemented programmatically. However, I completely abandoned the idea of ​​“scraping”. Its correct operation requires a large amount of information (the address of the page, the names of the form fields, the format of the received HTML page, etc.), which I strongly wanted to avoid. In addition, this information requires constant updating, as it may change even more often than the names of whois servers.
Well, and most importantly, if whois information is displayed to site visitors, you can simply provide a link to the registrar's site and allow the user to fill out the form himself. In most cases, the whois form is presented either on the main page, or you can go to the page with one click. This is a much more reliable way than relying on “scraping”, which in any case will not be able to cope with captcha. Where to get the link to the website of the registrar will be described below.
Also, some whois servers may ban the IP addresses of users who send too many requests.
So where do you get the name of the correct whois server?
You can take it from several sources:
1. There is such a wonderful whois server whois.iana.org, owned by IANA ( http://en.wikipedia.org/wiki/Internet_Assigned_Numbers_Authority ). It contains the latest information on all top-level domains. For example (hereinafter all the examples are using Unix-whois):
$ whois -h whois.iana.org ru domain: RU organisation: Coordination Center for TLD RU address: 8, Zoologicheskaya str. address: Moscow 123242 address: Russian Federation contact: administrative name: .RU domain Administrative group organisation: Coordination Center for TLD RU address: 8, Zoologicheskaya str. address: Moscow 123242 address: Russian Federation phone: +7 499 254 88 94 fax-no: +7 499 254 89 63 e-mail: ru-adm@cctld.ru contact: technical name: Technical Center of Internet organisation: Technical Center of Internet address: 8, Zoologicheskaya str. address: Moscow 123242 address: Russian Federation phone: +7 495 737 92 95 fax-no: +7 495 737 06 84 e-mail: ru-tech@tcinet.ru nserver: A.DNS.RIPN.NET 193.232.128.6 2001:678:17:0:193:232:128:6 nserver: B.DNS.RIPN.NET 194.85.252.62 2001:678:16:0:194:85:252:62 nserver: D.DNS.RIPN.NET 194.190.124.17 2001:678:18:0:194:190:124:17 nserver: E.DNS.RIPN.NET 193.232.142.17 2001:678:15:0:193:232:142:17 nserver: F.DNS.RIPN.NET 193.232.156.17 2001:678:14:0:193:232:156:17 ds-rdata: 14072 8 2 DFFBFE59FBBD3289D0C3819F05F94610A1E03B556D64540A2CC5F8C4158A00E7 whois: whois.tcinet.ru status: ACTIVE remarks: Registration information: <span>http</span>://www.cctld.ru/en created: 1994-04-07 changed: 2012-12-21 source: IANA Pay attention to the following lines:
whois: whois.tcinet.ru remarks: Registration information: <span>http</span>://www.cctld.ru/en This is actually the whois server and the registrar site! For most top-level domains, IANA returns the completely current name of the whois server. I specifically checked the changes in Ruby Whois over the past few months, and all of them were taken from whois.iana.org.
In my module, I cache the result from IANA for 24 hours (for a longer period of time there is no special meaning, and the data can change).
Despite all the above, for some top-level domains, IANA for some reason does not provide information about whois servers. Perhaps the registrars of some countries do not advertise them specifically. But not scary, we have other ways.
2. If you look closely, for very many top-level domains, the names of their whois servers look either as whois.nic. <Top level domain> (more common), or as whois. <Top level domain> (less common). For example:
- whois.nic.fr
- whois.nic.it
- whois.biz
Moreover, even if IANA indicates some other whois server (for example, whois.registro.br), in many cases an alias built according to this scheme (for our example, whois.nic.br) will still work.
Thus, if IANA did not return anything to us, for any top-level domain, we easily get the names of two potential whois servers:
- whois.nic. <top level domain>
- whois. <top level domain>
Practice has shown that this method identifies existing whois servers for a dozen top-level domains for which IANA does not return information.
Quite often, if we are looking for a third-level domain (for example, russia.edu.ru), then the whois server responsible for the top-level domain does not contain the necessary information. For example:
$ whois -h whois.tcinet.ru russia.edu.ru No entries found for the selected source(s). Last updated on 2013.01.04 01:41:36 MSK This is due to the fact that for many second-level domains (in our example, edu.ru) a completely different organization is responsible, which has its own whois server.
In such cases, jwhois and Ruby Whois meticulously hardcodes the name of the whois server for each second level domain they know with their own server. Naturally, it is absolutely unrealistic to keep track of all second-level domains, especially since there is no centralized source of such information.
But again, if you look closely, most of these whois servers are of the form whois. <Second-level domain> (more common) or whois.nic. <Second-level domain> (less common). For example:
- whois.za.net
- whois.eu.org
- whois.nic.priv.at
How to deal with those whois servers that are called differently? For example, in the Ruby Whois configuration, you can see:
- ".ae.org", "whois.centralnic.com"
- ".edu.ru", "whois.informika.ru"
From the very beginning, I was mentally prepared for the necessity of hard-coding several whois servers and in my own code, however, going through absolutely all the similar servers mentioned in the jwhois and Ruby Whois configurations, it turned out that all of them are also available also for whois aliases . <second level domain> or whois.nic. <second level domain>! For our example:
- whois.ae.org
- whois.edu.ru
In the end, I did not have to hardcode a single whois server at all (of course, except for whois.iana.org).
3. In addition to the above, the name of the whois server may still look like whois. <Server of the registrar site>. For example:
$ whois -h whois.iana.org br whois: whois.registro.br remarks: Registration information: <span>http</span>://registro.br/ In some cases, when IANA returns only the registrar's site without a whois server, the name built according to this scheme is correct.
4. As already mentioned, whois-servers.net contains aliases to whois servers only for a fairly limited number of top-level domains, but in some cases when all of the above does not help, an alias of the form <top level domain> .whois-servers.net is quite workable (for example, for "ps").
Thus, for each domain that we want to find, we will have a whole list of potential whois servers with a greater or lesser probability of efficiency. For example, for russia.edu.ru we will have:
- whois.tcinet.ru (IANA result for “ru”)
- whois.edu.ru << whois server we need
- whois.nic.edu.ru
- whois.nic.ru
- whois.ru
- whois.cctld.ru ("fictional" server based on the website address of the registrar)
- ru.whois-servers.net (in fact, working alias for whois.tcinet.ru)
Which one to choose? In fact, there is no need to choose anything, since it is not difficult to poll all found whois servers in order of priority. Naturally, if the result is found, then it is no longer necessary to poll the following servers. In addition, if one of these whois servers does not actually exist (and among the “fictional” ones there will, naturally, be the majority), then this information can be easily cached.
I would like to tell about the cache separately.
Whois server from IANA and whois servers found by links (more on that below) are considered “active” by default. When the active server does not respond (meaning, the server is not available in principle, and not just did not find the necessary information for a particular domain and returned an error), its status changes to “temporarily unavailable”. When a temporarily unavailable server does not respond, it is blocked for one minute, then two, four, eight, etc., until this period is more than two weeks. In this case, the server status changes to “inactive”, and it is blocked for exactly two weeks. If an inactive server does not respond after two weeks, it remains inactive and again blocked for two weeks. If a temporarily unavailable or inactive server suddenly responded, its status changes to active. All “fictional” whois servers and servers of the form <top level domain> .whois-servers.net are considered inactive by default. That is, if they do not respond, they are immediately blocked for two weeks.
This scheme allows you to filter all non-existing whois servers and at the same time does not allow for a long time to block servers that for one reason or another have become temporarily unavailable.
It should be noted that the next time we look for a domain in the edu.ru zone, two whois servers will be considered active at once:
- whois.tcinet.ru
- whois.edu.ru
However, the fact that whois.edu.ru is active indicates that the first whois server most likely does not contain the information we need (otherwise it would not reach whois.edu.ru, and it would not become active). Therefore, it would be advisable to put it first in priority and interrogate whois servers in this order:
- whois.edu.ru << whois server we need
- whois.tcinet.ru
- whois.nic.edu.ru
- whois.nic.ru
- whois.ru
- whois.cctld.ru
- ru.whois-servers.net
If we, for example, look for daily.lviv.ua (also a third-level domain), then the result for it will most likely be returned by the very first whois server that is returned to IANA (whois.ua). Therefore, the whois.lviv.ua server will not become active (perhaps it does not exist - I did not check it), and the next time we will look again at the domain in the lviv.ua zone on whois.ua.
In other words, after a while, requests to non-existing or incorrect whois servers will be minimized.
Links to additional whois servers
Often the result that the whois server issued is incomplete, but it contains a link to another whois server that contains more complete information. For example:
$ whois -h whois.verisign-grs.com 'domain google.com' Domain Name: GOOGLE.COM Registrar: MARKMONITOR INC. Whois Server: whois.markmonitor.com Referral URL: <span>http</span>://www.markmonitor.com Name Server: NS1.GOOGLE.COM Name Server: NS2.GOOGLE.COM Name Server: NS3.GOOGLE.COM Name Server: NS4.GOOGLE.COM Status: clientDeleteProhibited Status: clientTransferProhibited Status: clientUpdateProhibited Status: serverDeleteProhibited Status: serverTransferProhibited Status: serverUpdateProhibited Updated Date: 20-jul-2011 Creation Date: 15-sep-1997 Expiration Date: 14-sep-2020 As you can see, the information is not so much, but there is a link to another whois server:
Whois Server: whois.markmonitor.com If we now turn to it, we get:
$ whois -h whois.markmonitor.com google.com Registrant: Dns Admin Google Inc. Please contact contact-admin@google.com 1600 Amphitheatre Parkway Mountain View CA 94043 US dns-admin@google.com +1.6502530000 Fax: +1.6506188571 Domain Name: google.com Registrar Name: Markmonitor.com Registrar Whois: whois.markmonitor.com Registrar Homepage: <span>http</span>://www.markmonitor.com Administrative Contact: DNS Admin Google Inc. 1600 Amphitheatre Parkway Mountain View CA 94043 US dns-admin@google.com +1.6506234000 Fax: +1.6506188571 Technical Contact, Zone Contact: DNS Admin Google Inc. 2400 E. Bayshore Pkwy Mountain View CA 94043 US dns-admin@google.com +1.6503300100 Fax: +1.6506181499 Created on..............: 1997-09-15. Expires on..............: 2020-09-13. Record last updated on..: 2012-01-29. Domain servers in listed order: ns2.google.com ns4.google.com ns3.google.com ns1.google.com Thus, in order to get the most complete information, we should look for links to other whois servers as a result. Most often they look like “whois server: <name>”, but often the link is also in a completely unpredictable place (for example, in the middle of the text).
My module is looking for links rather aggressively, taking for a whois server any string that looks like a hostname and contains the word “whois” (as we found out, “extra” whois servers are not a problem). If we found the link in the “whois server: <name>” format, the server name no longer has to contain the word “whois” (in practice, this may not even be a hostname, but simply an IP address). All found whois servers are by default considered active.
The truth is there is one pitfall. The result may contain the name of your own whois server, and if it happened that we applied to it for some alias, it would look like a link to another whois server. For example:
$ whois -h whois.jp newsmap.jp [ JPRS database provides information on network administration. Its use is ] [ restricted to network administration purposes. For further information, ] [ use 'whois -h whois.jprs.jp help'. To suppress Japanese output, add'/e' ] [ at the end of command, eg 'whois -h whois.jprs.jp xxx/e'. ] ... The whois.jprs.jp is mentioned here, the alias of which, in fact, is whois.jp. That is, if we now send a request to whois.jprs.jp, we will get the same result. , whois - whois — , .
, - : -, , -, .
, : , , (, whois ). However, not all so simple. , — , , , . whois , , , - . : whois ! , .
«X», , "#" "%" (), , , «trim» . ( set Python), , .
IDN ( «.»), , . «.» «xn--80aealotwbjpid2k.xn--p1ai» whois.iana.org «xn--p1ai»:
whois: whois.tcinet.ru whois IP ?
IP . whois :
- whois.arin.net (ARIN, )
- whois.apnic.net (APNIC, )
- whois.ripe.net (RIPE, )
- whois.afrinic.net (AfriNIC, )
- whois.lacnic.net (LACNIC, )
, IP ( IPv4, IPv6) . - IP ( IPv6), IANA. For example:
$ whois -h whois.iana.org 12af:: inet6num: 1000:0:0:0:0:0:0:0/4 descr: Reserved by IETF remarks: <span>http</span>://tools.ietf.org/html/rfc4291 source: IANA - IP , , , . whois , whois.iana.org, - whois .
ARIN, IP ( IP ). , IP ARIN whois . IANA , .
, AfriNIC — whois . IP ARIN, RIPE APNIC. whois , IP . For example:
$ whois -h whois.iana.org 213.154.64.0 refer: whois.ripe.net inetnum: 213.0.0.0 - 213.255.255.255 organisation: RIPE NCC status: ALLOCATED whois: whois.ripe.net changed: 1993-10 source: IANA ARIN , 213.154.64.0 RIPE. RIPE, , 213.154.64.0 — 213.154.95.255 AfriNIC.
. whois.lacnic.net , - IP , , , . , LACNIC. , : whois (RIPE AfriNIC) IP , . , 10000 , , , AfriNIC. LACNIC , , , IP , . , - IP AfriNIC, , LACNIC , , . LACNIC .
whois , . . ARIN whois «ReferralServer: <>». IP , , , whois , , , LACNIC RIPE.
?
, , - whois . ?
43, "< IP >\r\n" , . , , ( , , ).
, RFC 3912 "\r\n", whois , "\n" "\r\n" ! whois whois.nic.org.mt.
, whois :
- whois.arin.org: «n + <IP >\r\n»
- whois.denic.de: "-C UTF-8 -T dn,ace <>\r\n"
- whois.dk-hostmaster.dk: "--show-handles <>\r\n"
- whois.nic.name: «domain = <>\r\n»
- whois , VeriSign: «domain <>\r\n»
VeriSign — whois.verisign-grs.com. ? "<>\r\n" `To single out one record, look it up with «xxx»` `look them up with "=xxx" to receive a full display`, VeriSign . «domain <>\r\n», whois VeriSign .
whois.arin.org "<IP >\r\n", :
Level 3 Communications, Inc. LVLT-ORG-8-8 (NET-8-0-0-0-1) 8.0.0.0 - 8.255.255.255 Google Incorporated LVLT-GOOGL-1-8-8-8 (NET-8-8-8-0-1) 8.8.8.0 - 8.8.8.255 whois.nic.name . For example:
$ whois -h whois.nic.name 'domain = dns.name' Domain Name ID: 1102376DOMAIN-NAME Domain Name: DNS.NAME Sponsoring Registrar ID: 44REGISTRAR-NAME Sponsoring Registrar: GoDaddy.com, LLC Domain Status: clientDeleteProhibited Domain Status: clientRenewProhibited Domain Status: clientTransferProhibited Domain Status: clientUpdateProhibited Registrant ID: 1337930CONTACT-NAME Admin ID: 1337932CONTACT-NAME Tech ID: 1337931CONTACT-NAME Billing ID: 1337933CONTACT-NAME Name Server ID: 1017197HOST-NAME Name Server: NS3.AFRAID.ORG Name Server ID: 1017198HOST-NAME Name Server: NS4.AFRAID.ORG Name Server ID: 2249HOST-NAME Name Server: NS1.AFRAID.ORG Name Server ID: 2250HOST-NAME Name Server: NS2.AFRAID.ORG Created On: 2004-09-04T19:14:53Z Expires On: 2013-09-04T19:14:53Z Updated On: 2012-09-03T09:10:57Z ( , ). :
Sponsoring Registrar ID: 44REGISTRAR-NAME :
$ whois -h whois.nic.name 'registrar = 44REGISTRAR-NAME' Registrar ID: 44REGISTRAR-NAME Registrar Name: GoDaddy.com, LLC Registrar URL: <span>www</span>.godaddy.com Registrar Status: ok Registrar Address: 14455 North Hayden Rd, Suite 226 Registrar City: Scottsdale Registrar State/Province: Arizona Registrar Country: UNITED STATES Registrar Postal Code: 85260 Registrar Phone Number: +1.4805058800 Registrar Fax Number: +1.4805058865 Registrar E-mail: registrar_routine@godaddy.com Admin ID: 1269602CONTACT-NAME Admin Organization: Go Daddy Software, Inc. Admin Name: Tim Ruiz Admin Address: 14455 N Hayden Suite 226, , Admin City: Scottsdale Admin State/Province: AZ Admin Country: UNITED STATES Admin Postal Code: 85260 Admin Phone Number: +1.4805058800 Admin Fax Number: +1.4805058865 Admin Email: registrar_routine@godaddy.com ... whois , :
Registrar URL: <span>www</span>.godaddy.com , whois - . We try:
$ whois -h whois.godaddy.com dns.name Registered through: GoDaddy.com, LLC (<span>http</span>://www.godaddy.com) Domain Name: DNS.NAME Created on: 04-Sep-04 Expires on: 04-Sep-13 Last Updated on: 03-Sep-12 Registrant: Mike Taylor 9 Ludford Grove Sale, M33 4DP United Kingdom Administrative Contact: Taylor, Mike admin@crazyemail.net 9 Ludford Grove Sale, M33 4DP United Kingdom +7.745095404 Technical Contact: Taylor, Mike admin@crazyemail.net 9 Ludford Grove Sale, M33 4DP United Kingdom +7.745095404 ... , , «name», .
whois ( ). , - . , "-B" whois.ripe.net «notify», «changed» «e-mail», :
$ telnet whois.ripe.net 43 Trying 193.0.6.135... Connected to whois.ripe.net. Escape character is '^]'. % This is the RIPE Database query service. % The objects are in RPSL format. % % The RIPE Database is subject to Terms and Conditions. % See http://www.ripe.net/db/support/db-terms-conditions.pdf -B 213.180.204.0 <<<<< "-B" % Information related to '213.180.204.0 - 213.180.204.255' inetnum: 213.180.204.0 - 213.180.204.255 netname: YANDEX-213-180-204 descr: Yandex enterprise network country: RU admin-c: YNDX1-RIPE tech-c: YNDX1-RIPE remarks: INFRA-AW status: ASSIGNED PA mnt-by: YANDEX-MNT changed: artem@yandex-team.ru 20091116 <<<<< source: RIPE ... telnet, Unix- whois .
whois . :
obj = socket.socket(socket.AF_INET, socket.SOCK_STREAM) try: ... res = '' while True: buf = obj.recv(4096) if buf: res += buf else: break return res except Exception: return '' , , . , .
, ( , , 99% ). whois , , ( ).
:
obj = socket.socket(socket.AF_INET, socket.SOCK_STREAM) ... res = '' while True: try: buf = obj.recv(4096) except socket.error: break if buf: res += buf else: break return res , , whois, , rwhois. .
Referral Whois
rwhois («Referral Whois») : http://en.wikipedia.org/wiki/Whois#Referral_Whois . , , whois , , . 4321. , rwhois 1994- , .
rwhois , ARIN ( - whois ). , «rwhois». , ARIN , : «whois://» «rwhois://». rwhois , , , "<IP >\r\n" . .
Referral Whois RFC 2167 ( http://tools.ietf.org/html/rfc2167 ). 170. , Joel- Spolsky: « - - , , . , ».
- , Google . , telnet- , , , . http://projects.arin.net/rwhois/ C GPL v2 . 100 rwhois 1.0 1.5 — . rwhois , , . jwhois rwhois, ( , ).
, rwhois , subpocess rwhois , .
: , rwhois . , «error querying rwhois server». Google, , . , jwhois .
, , , : «rwhois» whois! , «, rwhois 1.0 rwhois 1.5?» "<IP >\r\n" ! , , .
, « » rwhois , , whois, rwhois . Joel Spolsky .
, - rwhois - : . , , / . Rwhois . .
, rwhois - rwhois , whois . , whois .
?
— . : whois . . whois , : IP ( ).
, :
IP whois , :
- «0.0.0.0 — 255.255.255.255» «0::/0» ( )
- «network range is not allocated to apnic» ( APNIC)
- «address block not managed by the ripe» «does not operate any networks using» ( RIPE)
- «afrinic whois server» «ripe database query service» ( LACNIC)
- «whois.arin.net» «allocated to arin», , ,
, ( ):
- , «http(s)://» (, «12:45:51»). , , .
- , "#" "%" ().
- IP , , , , . : whois.eu.
- «» . 5 , «».
- 3 «» , .
- «» , , ( , ), «name» «address», .
- .
, . , , , .
ARIN (, 8.8.8.8), :
$ whois -h whois.arin.net 'n + 8.8.8.8' # start NetRange: 8.0.0.0 - 8.255.255.255 CIDR: 8.0.0.0/8 OriginAS: NetName: LVLT-ORG-8-8 NetHandle: NET-8-0-0-0-1 Parent: NetType: Direct Allocation RegDate: 1992-12-01 Updated: 2012-02-24 Ref: <span>http</span>://whois.arin.net/rest/net/NET-8-0-0-0-1 ... # end # start NetRange: 8.8.8.0 - 8.8.8.255 CIDR: 8.8.8.0/24 OriginAS: NetName: LVLT-GOOGL-1-8-8-8 NetHandle: NET-8-8-8-0-1 Parent: NET-8-0-0-0-1 NetType: Reassigned RegDate: 2009-09-21 Updated: 2009-09-21 Ref: <span>http</span>://whois.arin.net/rest/net/NET-8-8-8-0-1 ... # end #end #start , ( 8.8.8.0 — 8.8.8.255).
:
1. , IP ( whois ).
2. HTML ( whois , ).
3. whois :
- whois ".jp", «iso-2022-jp» ( , ).
- whois ".kr" «krnic.net», «euc-kr» ( , ).
- «utf8». , «iso-8859-1».
: whois , ( ), ( jwhois http://whois.domaintools.com ). () , () , . .
. , .
whois RFC 3912 ( http://tools.ietf.org/html/rfc3912 ), whois «». , : TCP 43 whois , , "\r\n" . whois . , whois IP .
whois Unix- «whois», , . jwhois Ruby Whois — , , whois . , , ( , jwhois 2011- ).
, whois , , whois .
, whois :
- whois ;
- ;
- .
, , whois . whois :
- whois whois.iana.org. " whois -h whois.iana.org < > " , , whois . IANA , whois «» whois.< www>.
- whois whois.nic.< > whois.< >.
- whois . , whois.< > whois.nic.< >.
- < >.whois-servers.net.
, whois . , whois , ( ), .
, whois , whois . , «whois server: <>», -. , whois , whois , ( , whois ).
IP . whois :
- whois.arin.net (ARIN, )
- whois.apnic.net (APNIC, )
- whois.ripe.net (RIPE, )
- whois.afrinic.net (AfriNIC, )
- whois.lacnic.net (LACNIC, )
IP ( IPv4, IPv6) . - IP , whois.iana.org. whois , «» whois «». ARIN, IP . , whois , whois .
, whois.ripe.net whois.afrinic.net IP , . , whois.lacnic.net , , whois (, «» whois.ripe.net whois.afrinic.net).
whois "< IP >\r\n", whois ( " ? ").
whois , rwhois («Referral Whois») 4321 ( http://en.wikipedia.org/wiki/Whois#Referral_Whois http://tools.ietf.org/html/rfc2167 ). , rwhois (, http://projects.arin.net/rwhois/ ). , , rwhois, whois! rwhois .
whois - , . , whois . , , " ? ".
, whois IP , . whois HTML . whois utf8, whois iso-2022-jp euc-kr .
Instead of conclusion
, whois , . , , , .
Source: https://habr.com/ru/post/165869/
All Articles