Visualization and work with historical data: interactive maps and linked data of the knowledge base for studying history

Sometimes you look around and it seems that the modern world outside of IT does not exist. However, there are areas of human life that are very poorly affected by computerization. One of these areas is history. And as a science, and as a training course. Of course, computer work is hardly ever replaced by historians tinkering with archives. But to study the history of static maps drawn in the textbook, and building the order of events carefully writing the dates in ascending order on paper is exactly the last century. However, there are not so many tools for visual study of history and it is very difficult to find them.

If you want to find out what kind of interactive historical maps there are, where you should look at the presentation of events in the form of timelines and how to make complex requests to Wikipedia like “all statesmen who worked in Europe in 1725” - read on.

How it all began: at the summer school, we undertook to make an interactive map of historical events based on Wikipedia. I do not give a direct link to the project, because the project is very raw (a team of 4 beautiful tenth-graders worked on it, but you have a lot of time in 3 weeks), and also because the server has a habit of “falling” without habra effect.
We wanted to display events that took place in different historical eras on the map - and this partly turned out: we have a map of battles with their descriptions. At that time, when we were engaged in this project, we knew only about a couple of interactive historical atlases, and none of them showed events on the map.
')
I believe that these maps are so few because everyone faces the same problems as we: historical data is not structured. There are no machine-readable databases from which you can download information about important historical events. Historians, if they add databases, describe in them, as a rule, only their narrow subject area - such as the map of fortifications of the Roman Empire. This may be interesting and useful for historians, but it is unlikely that ordinary people can derive much benefit from such a map. The second problem is the complete lack of data on the borders of countries in a historical perspective. You can find hundreds of atlases of ancient eras, but you will have to transfer the coordinates of the borders from the atlases manually. The third problem is the absence of any standards for describing historical data. Even the normal date description format does not exist, standard data types and formats break down about years before Christ. What can we say about different calendars or inaccurately certain dates? ..

The problems of lack of machine-readable historical data are still waiting to be resolved (we are working on it, join, there will be enough work for everyone). But still, some projects cope with this in their own way ...

As popular wisdom says: "After you have broken the device, study the instructions." Already after we made our map, I managed to find several other projects with interactive maps and other ways to visualize history and extract historical data. But it took me a completely indecent amount of time to dig up these resources in the depths of the Internet, so I decided to collect everything I found in one place.

The first category is interactive historical maps . These are not my dream cards, but quite working products. There are quite a few of them (and I don’t list here even very narrowly specialized ones), but there are only a couple of really good ones, alas. Separately, it is sad that there are no localized projects among them, which means that it is difficult to teach Russian-speaking schoolchildren according to them.

• Chronoas is the most personalized map, but also having very wide possibilities for visualization. It is complicated to study it yourself, so watch the video about its capabilities. It is beautiful and strong. The map shows historical events of different types with supporting information, which allows you to get acquainted with the history without leaving the map.
  
  Information on the map has been extracted, including from Wikipedia and wikidata. Historically, the map is inaccurate, as reported by many users familiar with the history of China. But the project has the beginnings of wiki-editing maps, so sometime the errors will be corrected.
  
  From the introductory video, you can also learn about the rather wide possibilities of visualizing statistical information (such as population, professed religions, etc.) about different eras. Not all of these visualizations are simple and clear, but the very possibility of doing so is great.
  
  
• There is a map of Running Reality with a very detailed marking of the territories. The project wants to describe the story down to the history of the streets and for this it allows wiki editing of the map (as I understand it, not in the web version). They have a rather poor visualization of historical data, but they have a very competent data model, which allows describing alternative branches of history (which is useful when historians have several hypotheses about how it really was). They write that the web map is much younger and has been cut back in features compared to standalone, and I did not test the standalone version (it did not start). However, it is as free as the web. If you can run it, write your feedback in the comments.
  
  
• I found the geacron map a long time ago. It is drawn by historians by sources and atlases, which means it is likely that it reflects history more accurately than others. But the interactivity of this map is seriously lacking. In addition to the map mode, the site has a timeline for historically significant periods. Sadly, but prioritized by real historians. One of the problems of the previous cards is that there important events and walk-throughs are on an equal footing. Geacron avoids this by manually overseeing data.
  
  
• Map Spacetime with event search by category. Not incendiary, but done well (and already against the background of the number of similar cards close to zero ...) And this is Wikipedia and Wikidata again.
  
  
• Proprietary Atlas CENTENNIA without a web version. It seems to me that, in the video clips like “1000 years of European history in five minutes,” this map is usually used.
  
  
• Timemaps is a rather weak geacron clone, but it may be more convenient for someone.
• upd: The history of urbanization - an animated map on which the times of the emergence of cities.
• upd: World population history - map of population over time. It also shows all sorts of things like life expectancy, the level of greenhouse gases and so forth. Some important milestones in the history of mankind
• upd2: Wordology - a set of very simple interactive maps for different periods of history. Probably handmade. Detailing is minimal, interactivity also does not shine.

The second category is different. These are interesting near-historical projects that I found along the way.

• Historic timelines on Histropedia . I don’t like such a presentation style as the time axis, but a) for lack of the best visualization tools, you can use them, b) these timelines are really well made and convenient, c) these timelines can be edited, and you can also create your own, d ) you can create timelines not with your hands, but with a request for wikidans, d) quite a lot of timelines have already been done for you, and it's nice to study them.
  
  
• Wikijourney is a map with geotagged wiki articles about these places. It is assumed that it is used for sights, but in Wikipedia there are articles on almost every street in Moscow and on every metro station - so I see a rather everyday list of “sights” around me. On the aforementioned Chronoas , by the way, there are also photos on the map that are somehow related to the place-time. The reference to time, however, is rather conditional: how old is this photographing? ..
  
  
• Visualization data tools. The last half century there is the science of "Digital humanities" - computer methods of humanitarian studies. I would say that this science is barely warm, judging by how little has been done so far ... but nonetheless. So, for historians, philologists, archaeologists and other experts developed a number of visualization tools. For the most part, it is a visualization of all connections between objects. In the graph, on the map, in the tag cloud, in the time perspective, etc.
  For example, Stanford developed a number of similar tools (I stumbled upon mentioning their tool Palladio several times, apparently this is their main tool).
  There is also a project NodeGoat - they are well sharpened for visual data linked (see below). Here, for example, their battle map based on data from wikidata and dbpedia. The map looks great, although it’s not very convenient to travel through links to linked objects. By the way, if you click, for example, on a point with events that “happened” in the very center of Russia, you will see a common problem with all the maps made by parsing information: the incorrect assignment of an event to a place and time.

The third category is my favorite; behind it is the future, definitely. Linked data .
Marked graphs of knowledge or semantic networks, this is all. The most powerful technology for compiling complex search queries. It has been developing for a long time, but it hasn’t yet become a nation. The main reason for this is the complexity of use and, especially, the complexity of the study: there are few materials, and almost all materials are designed for programmers. I have compiled a small selection of good and accessible training materials that will allow an ordinary person to master this tool in a couple of hours. This is not fast, but during this time your “google-fu” will increase significantly.

The technology of semantic networks is adopted by all major search and information systems. In particular, many are now learning to translate natural language into formalized queries for such a graph. Surely, the investigating authorities and intelligence services use this (given that one of the most popular knowledge graphs is done on the CIA Factbook). You can think of a million ways to use this technology in any analytical work: for the state, for business, for science, and even for household planning.

Maybe in a few years the search engines will learn how to decipher a part of your questions in natural language and answer them. But you yourself can use all the power of this tool now and get much more flexibility than any search engine will give you. So, training materials:

• There is a great tutorial “Using SPARQL to access Linked Open Data” (on The Programming Historian website) on what linked data is and why they are needed. I believe that every educated person should learn the basics of SPARQL, just as every person should be able to google. This is literally about how to build complex and powerful search queries (see examples below). You may not use it every day, but when the next task of searching and analyzing information that requires a month of manual work comes, you will know how to avoid it.
  
  Frankly speaking, despite the good presentation, the material is still quite complex: the format of the RDF data, ontologies and the SPARQL query language. Until I found this article, I could only admire how cool people use it, but did not understand how to make it work. The Programming Historian provides sophisticated material on very clear examples and shows how to use it.
  
  Their site , by the way, is not without its name already. They teach historians to use computational tools and programming for research. Because a little bit of programming makes any work easier.
  
  
• A good introductory video tutorial for 15 minutes on how to make requests to wikidans, and then visualize them in histropedia. A purely practical exercise, after which it will be clear to you which buttons to poke in order to create your own query and see the result in a digestible form. I recommend after this tutorial to watch this video and then start practicing.
  
  
• Examples of requests to get the power of the tool. Feel free to click "Run." In the query window, you can direct the identifiers with the mouse - in the pop-up hint you will be shown what lies behind the mysterious wdt: P31 and wd: Q12136. So: a request that returns all women mayors of major cities or composers who have voiced more than 100 films . Or a request that I came up with for my own training - about the causes of death of film actors, in addition to diseases .
  
  
• A couple of good introductory habr posts about the knowledge base: one and two .
  
  
• A good example of where linked data is used is PanamaPapers . To explore such a hefty array of documents, links were made there in a format similar to RDF. Visualize links as a graph. Made imperfectly, but it makes sense to look in order to touch this structure with your hands.
  
  
• Data providers. Almost all of the maps, as well as Histropedia, use either the rapidly developing Wikidata wiki project or the project to automatically retrieve data from DBpedia's Wikipedia as data sources. These projects aim to make computer-readable data sources constantly updated by the community. There are still more conservative data sources supported by museums - about collections of objects of art and archeology, dictionaries of geographical names and biographies, biological ontologies. And surely a lot of other things. Google by the words "SPARQL endpoint".

I hope that this entry will help you not only to satisfy your curiosity, and to captivate your acquaintances of schoolchildren with the visibility of history, but also to awaken your imagination about new tools and historical databases. Works in the field of historical informatics is an uncultivated field. Join us, gentlemen!

upd: Recalled and added two more maps about the history of urbanization and human development. And one more very simple card made by hands.