Release 2.2.0.

README.md

@@ -37,6 +37,10 @@ The manual (including install instructions) can be found in the doc/ directory.
 	* Will detect and warn about any changes in parsed data compared to previous runs
 	* Guarantees that previously working WHOIS parsing doesn't unintentionally break when changing code
 
+## Important update notes
+
+*2.2.0 and up*: The internal workings of `get_whois_raw` have been changed, to better facilitate parsing of WHOIS data from registries that may return multiple partial matches for a query, such as `whois.verisign-grs.com`. This change means that, by default, `get_whois_raw` will now strip out the part of such a response that does not pertain directly to the requested domain. If your application requires an unmodified raw WHOIS response and is calling `get_whois_raw` directly, you should use the new `never_cut` parameter to keep pythonwhois from doing this post-processing. As this is a potentially breaking behaviour change, the minor version has been bumped.
+
 ## It doesn't work!
 
 * It doesn't work at all?

doc/usage.zpy

@@ -123,7 +123,7 @@ To start using pythonwhois, use `import pythonwhois`.
 	
 	These are the fields that any contact dict may contain. If certain information for a contact was not found, the corresponding key will be absent.
 	
-	! Note that any of these fields may consist of multiple lines, although the `address` field is the only one that is **likely** to consist of multiple lines.
+	! Note that any of these fields **may** consist of multiple lines, although the `address` field is the only one that is **likely** to consist of multiple lines.
 	
 	handle::
 		The NIC handle for the contact.
@@ -160,7 +160,7 @@ To start using pythonwhois, use `import pythonwhois`.
 
 ## When you need more control...
 
-^ pythonwhois.net.get_whois_raw(**domain**[, **server=""**, **rfc3490=True**])
+^ pythonwhois.net.get_whois_raw(**domain**[, **server=""**, **rfc3490=True**, **never_cut=False**])
 
 	Retrieves the raw WHOIS data for the specified domain, and returns it as a list of responses (one element for each WHOIS server queried). This method will keep following redirects, until it ends up at the right server (and all responses it picks up in the meantime, will be included). Raises `pythonwhois.shared.WhoisException` if no root server for the TLD could be found.
 	
@@ -171,7 +171,10 @@ To start using pythonwhois, use `import pythonwhois`.
 		**Optional.** The WHOIS server to query. When not specified, it will default to the appropriate WHOIS server for the TLD.
 
 	rfc3490::
-                **Optional.** If set to `True` a given domain will be encoded through the **toASCII** method as documented in RFC3490 before its submission to the whois service. If the domain isn't given in unicode, the method will handle the decoding by itself.
+		**Optional.** If set to `True`, a given domain will be encoded through the `toASCII` method as documented in {http://www.ietf.org/rfc/rfc3490.txt}(RFC3490) before its submission to the WHOIS service. If the domain isn't supplied in unicode, the method will handle the decoding by itself.
+		
+	never_cut::
+		**Optional.** If set to `True`, pythonwhois will never strip out data from the raw WHOIS responses, **even** if that data relates to a partial match, rather than the requested domain.
 
 
 ^ pythonwhois.net.get_root_server(**domain**)

pythonwhois/net.py

@@ -2,11 +2,13 @@ import socket, re
 from codecs import encode, decode
 from . import shared
 
-def get_whois_raw(domain, server="", previous=[], rfc3490=True):
+def get_whois_raw(domain, server="", previous=[], rfc3490=True, never_cut=False):
 	# Sometimes IANA simply won't give us the right root WHOIS server
 	exceptions = {
 		".ac.uk": "whois.ja.net",
-		".ps": "whois.pnina.ps"
+		".ps": "whois.pnina.ps",
+		# The following is a bit hacky, but IANA won't return the right answer for example.com because it's a direct registration.
+		"example.com": "whois.verisign-grs.com"
 	}
 	
 	if rfc3490:
@@ -33,7 +35,14 @@ def get_whois_raw(domain, server="", previous=[], rfc3490=True):
 	else:
 		request_domain = domain
 	response = whois_request(request_domain, target_server)
-	new_list = [response] + previous
+	if never_cut:
+		# If the caller has requested to 'never cut' responses, he will get the original response from the server (this is
+		# useful for callers that are only interested in the raw data). Otherwise, if the target is verisign-grs, we will
+		# select the data relevant to the requested domain, and discard the rest, so that in a multiple-option response the
+		# parsing code will only touch the information relevant to the requested domain. The side-effect of this is that
+		# when `never_cut` is set to False, any verisign-grs responses in the raw data will be missing header, footer, and
+		# alternative domain options (this is handled a few lines below, after the verisign-grs processing).
+		new_list = [response] + previous
 	if target_server == "whois.verisign-grs.com":
 		# VeriSign is a little... special. As it may return multiple full records and there's no way to do an exact query,
 		# we need to actually find the correct record in the list.
@@ -41,6 +50,8 @@ def get_whois_raw(domain, server="", previous=[], rfc3490=True):
 			if re.search("Domain Name: %s\n" % domain.upper(), record):
 				response = record
 				break
+	if never_cut == False:
+		new_list = [response] + previous
 	for line in [x.strip() for x in response.splitlines()]:
 		match = re.match("(refer|whois server|referral url|whois server|registrar whois):\s*([^\s]+\.[^\s]+)", line, re.IGNORECASE)
 		if match is not None:
@@ -57,7 +68,7 @@ def get_root_server(domain):
 		if match is None:
 			continue
 		return match.group(1)
-	raise shared.WhoisException("No root WHOIS server found for TLD.")
+	raise shared.WhoisException("No root WHOIS server found for domain.")
 	
 def whois_request(domain, server, port=43):
 	sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)

setup.py

@@ -1,7 +1,7 @@
 from setuptools import setup
 
 setup(name='pythonwhois',
-      version='2.1.4',
+      version='2.2.0',
       description='Module for retrieving and parsing the WHOIS data for a domain. Supports most domains. No dependencies.',
       author='Sven Slootweg',
       author_email='pythonwhois@cryto.net',

test/data/example.com

@@ -0,0 +1,25 @@
+% IANA WHOIS server
+% for more information on IANA, visit http://www.iana.org
+% This query returned 1 object
+
+domain:       EXAMPLE.COM
+
+organisation: Internet Assigned Numbers Authority
+
+created:      1992-01-01
+source:       IANA
+
+
+--
+   Domain Name: EXAMPLE.COM
+   Registrar: RESERVED-INTERNET ASSIGNED NUMBERS AUTHORITY
+   Whois Server: whois.iana.org
+   Referral URL: http://res-dom.iana.org
+   Name Server: A.IANA-SERVERS.NET
+   Name Server: B.IANA-SERVERS.NET
+   Status: clientDeleteProhibited
+   Status: clientTransferProhibited
+   Status: clientUpdateProhibited
+   Updated Date: 14-aug-2013
+   Creation Date: 14-aug-1995
+   Expiration Date: 13-aug-2014

test/target_default/example.com

@@ -0,0 +1 @@
+{"status": ["clientDeleteProhibited", "clientTransferProhibited", "clientUpdateProhibited"], "updated_date": ["2013-08-14T00:00:00"], "contacts": {"admin": null, "tech": null, "registrant": null, "billing": null}, "nameservers": ["A.IANA-SERVERS.NET", "B.IANA-SERVERS.NET"], "expiration_date": ["2014-08-13T00:00:00"], "creation_date": ["1992-01-01T00:00:00"], "raw": ["% IANA WHOIS server\n% for more information on IANA, visit http://www.iana.org\n% This query returned 1 object\n\ndomain:       EXAMPLE.COM\n\norganisation: Internet Assigned Numbers Authority\n\ncreated:      1992-01-01\nsource:       IANA\n\n", "   Domain Name: EXAMPLE.COM\n   Registrar: RESERVED-INTERNET ASSIGNED NUMBERS AUTHORITY\n   Whois Server: whois.iana.org\n   Referral URL: http://res-dom.iana.org\n   Name Server: A.IANA-SERVERS.NET\n   Name Server: B.IANA-SERVERS.NET\n   Status: clientDeleteProhibited\n   Status: clientTransferProhibited\n   Status: clientUpdateProhibited\n   Updated Date: 14-aug-2013\n   Creation Date: 14-aug-1995\n   Expiration Date: 13-aug-2014\n"], "whois_server": ["whois.iana.org"], "registrar": ["RESERVED-INTERNET ASSIGNED NUMBERS AUTHORITY"]}

test/target_normalized/example.com

@@ -0,0 +1 @@
+{"status": ["clientDeleteProhibited", "clientTransferProhibited", "clientUpdateProhibited"], "updated_date": ["2013-08-14T00:00:00"], "contacts": {"admin": null, "tech": null, "registrant": null, "billing": null}, "nameservers": ["a.iana-servers.net", "b.iana-servers.net"], "expiration_date": ["2014-08-13T00:00:00"], "creation_date": ["1992-01-01T00:00:00"], "raw": ["% IANA WHOIS server\n% for more information on IANA, visit http://www.iana.org\n% This query returned 1 object\n\ndomain:       EXAMPLE.COM\n\norganisation: Internet Assigned Numbers Authority\n\ncreated:      1992-01-01\nsource:       IANA\n\n", "   Domain Name: EXAMPLE.COM\n   Registrar: RESERVED-INTERNET ASSIGNED NUMBERS AUTHORITY\n   Whois Server: whois.iana.org\n   Referral URL: http://res-dom.iana.org\n   Name Server: A.IANA-SERVERS.NET\n   Name Server: B.IANA-SERVERS.NET\n   Status: clientDeleteProhibited\n   Status: clientTransferProhibited\n   Status: clientUpdateProhibited\n   Updated Date: 14-aug-2013\n   Creation Date: 14-aug-1995\n   Expiration Date: 13-aug-2014\n"], "whois_server": ["whois.iana.org"], "registrar": ["Reserved-internet Assigned Numbers Authority"]}