Another implementation of the field "city" for Django

“One more” - because it seems to me that I am missing something and, in fact, there is a good, but unknown to me solution “out of the box”. However, here is my recipe:

Data

First of all, the question arose - where to get a list of cities. Historically, I opted for a geonames.org with a creative commons license. OpenStreetMaps was abandoned because of possible “patent” claims from Google, where Geonames takes some of the data. (However, at http://www.openstreetmap.org/ it is used as an alternative source of information.)
')
Considering the superficiality of interests (in this case, just a city was needed, like a profile field, without geo-operations like searching for occurrences in a region, etc.), you need to convert geonames data into a simple, suitable format.
I did not waste time searching for a beautiful solution and made everything simple using SQL and an additional database:

1. Create a geonames database and import data there ( http://forum.geonames.org/gforum/posts/list/732.page )
You can stop at this and use them in this form, or you can transform it to subordinate it to your logic.
2. Transform. I do not pretend to the only correct decision, I cut some of the technical issues of the country:

-- Country import insert into common_country(id,name,code,population,latitude,longitude,alternatenames) select geonameid as id,name,country as code, population,latitude,longitude,alternatenames from geonames.geoname gn where (gn.fcode IN ('PCLI','PCLIX','PCLD','PCLS','PCLF','PCL','TERR')); delete from common_country where id in (2077507,2170371,7910334,7909807); create unique index common_country_idx on common_country (id); create index common_country_code_idx on common_country(code); -- South Korea Fix update common_country set alternatenames = concat(alternatenames,"Korea, Republic of") where id = 1835841; -- City import insert into common_city(id,name,country_id,alternatenames,latitude,longitude, adm) select gn.geonameid as id, gn.name, c.id as country_id, gn.alternatenames, gn.latitude, gn.longitude, admin1 as adm from geonames.geoname gn left join common_country as c on gn.country=c.code where (gn.fcode in ('PPL','PPLC','PPLA','PPLA2','PPLA3','PPLA4')); 

The obvious minus is the need to write functionality for updating data. The solutions I found are sharpened for PostgreSQL.

Search tools

MySQL's full-text native search did not cope with a quick search for so many localities, so I used Sphinxsearch (I think any other Solr will do)
Index and source config:

 source geo_city { type = mysql sql_host = localhost sql_user = citylist sql_pass = citylist sql_db = citylist sql_port = sql_query_pre = SET NAMES utf8 sql_query_post = sql_query = \ SELECT id, name, country_id, alternatenames, latitude, longitude\ FROM geo_city sql_query_info = SELECT * FROM `geo_city` WHERE `id` = $id # ForeignKey's sql_attr_uint = country_id } index common_city { source = geo_city path = /var/lib/sphinxsearch/data/geo_city docinfo = extern morphology = none stopwords = min_word_len = 2 charset_type = utf-8 min_prefix_len = 2 min_infix_len = 0 prefix_fields = name, alternatenames enable_star = 1 } 

Representation

We will need:
django-selectable
django-sphinx
django-profiles - for my particular case with a profile

We put both applications, add them to INSTALLED_APPS.
Django-sphinx requires an API version number in settings.py

for Sphinx 0.9.9:

 SPHINX_API_VERSION = 0x116 

create a model for the city and country in the application responsible for the geology / model
(geo in my case)

 class Country(models.Model): name = models.CharField(max_length=200) code = models.CharField(max_length=10) population = models.IntegerField() latitude = models.FloatField() longitude = models.FloatField() alternatenames = models.CharField(max_length=2000, blank=True, default='') def __str__(self): return unicode(self.name).encode('utf-8') def __unicode__(self): return unicode(self.name) class City(models.Model): name = models.CharField(max_length=200) country = models.ForeignKey(Country) alternatenames = models.CharField(max_length=2000, blank=True, default='') latitude = models.FloatField(default=0) longitude = models.FloatField(default=0) adm = models.CharField(max_length=200) search = SphinxSearch(weights={ 'name': 100, 'alternatenames': 80 }) def __str__(self): return unicode(self.name).encode('utf-8') def __unicode__(self): return unicode(self.name) 

If the city field is needed for a profile, we add it and the model to the application that deals with profiles, I have this usermanage:

 class CustomUserProfile(models.Model): user = models.ForeignKey(User, unique=True) full_name = models.CharField(max_length=200, blank=True) city = models.ForeignKey(City, blank=True, null=True) country = models.ForeignKey(Country, blank=True, null=True, editable=False) date_registered = models.DateField(editable=False, auto_now_add=True) 

and profile form:

 class UserProfileForm(forms.ModelForm): ''' Form to edit profile''' full_name = forms.CharField(widget=forms.TextInput()) city = selectable.AutoCompleteSelectField( label='City please', lookup_class = common.lookups.CityLookup, required=False, ) def clean_city(self): """ Convert city code to city object """ city_id = int(self.data["city_1"].encode("utf8")) city = City.objects.get(id=city_id) return city class Meta: model = CustomUserProfile exclude = ("user",) 

and in settings.py

 AUTH_PROFILE_MODULE = 'usermanage.CustomUserProfile' 

more about setting up django-profiles here

django-selectable requires setting up urls.py

 (r'^selectable/', include('selectable.urls')), 

Let's create lookup.py and a method in it for the implementation of queries to the database and initial filling of the field:

 class CityLookup(LookupBase): model = City item = None def get_query(self, request, term): qs = self.model.search.query(term + "*") return qs def get_queryset(self): return None def get_item_id(self, item): return item.id def get_item_value(self, item): if (not self.item): return smart_unicode(item) return smart_unicode(self.item.name) def get_item_label(self, item): return u"%s, %s" % (item.name, item.country) def get_item(self, value): item = None if value: try: item = City.objects.get(id=value) self.item = item except IndexError: pass return item try: registry.register(CityLookup) except: pass 

In the forms.py profile you need to import lookups and selectable fields / widgets

 import selectable.forms as selectable import geo.lookups</code> 

For django-selectable, jquery libraries are needed, add them to base.html , first download jquery and jquery-ui (jquery.dj.selectable.js - goes to django-selectable)

 <script type="text/javascript" src="/js/jquery/jquery.min.js"></script> <script type="text/javascript" src="/js/jquery/jquery-ui.min.js"></script> <script type="text/javascript" src="/js/jquery/jquery.dj.selectable.js"></script> 

If everything went well, then the output will be something like this:



or even such (the alternatenames field contains variants of names in different languages)



A logical continuation, I see the addition to the model is admin1 (administrative unit of the first level) for separating settlements with the same name, jemeter tests for measuring performance and writing a reusable application, at least having improved MySQL django-geonames, however at the last Kyiv.py I unexpectedly I realized that now the project's path lies in the direction of geodjango, PostgreSQL and PostGIS, since the role of the city in the profile becomes more than just informational.

PS If someone is interested in a test project-demo, lay out.

References:
List of potential OSM data sources

github and bitbucket pages describing the relevant projects.

Thanks for attention.