Knowledge Base. Part 2. Freebase: making requests to the Google Knowledge Graph
More than a year ago, Google announced that from now on their search used the mysterious Knowledge Network (the official translation of the Knowledge Graph). Probably, not everyone knows that a significant part of the Network data is available for use by everyone and is available through a well-described API. This part is the Freebase knowledge base maintained by Google and enthusiasts. In this article, we first fool around a bit, and then try to make some simple queries in the MQL language.
This article is the second of the Knowledge Base series. Keep for updates.
- Part 1 - Introduction
- Part 2 - Freebase: making queries to the Google Knowledge Graph
- Part 3 - Dbpedia - Linked Data World Core
- Part 4 - Wikidata - Semantic Wikipedia
Google Knowledge Graph in terms of the average user
One of the visible manifestations of the Google Knowledge Graph are dashboards that briefly describe the object you are looking for. They often arise when searching for personalities, a little less often - geographical names. They often arise for queries specified in English in the English interface, but we will stick to the Russian language where it is possible.
For example, the query Roger Waters gives the following result:
Click on the links in the infobox and pay attention to the URL - it uses the stick parameter, the contents of which is some type identifier
&stick=H4sIAAAAAAAAAONg[VuLQz9U3]< >AAAAWhen the Knowledge Graph first appeared, it allowed the uninitiated to demonstrate a little street magic , for example, to add a & stick-parameter from Marilyn Monroe to a request from Stephen King:
')
Now this opportunity is covered, and nowhere is it for us - it’s better to look at something useful. For example, recently it became possible to compare several objects using the keyword vs:
Google promises to add many more goodies related to smart search and answers to questions, and the Knowledge Graph will be one of the pillars on which this intelligence keeps. What is especially beautiful for us is that a piece of Knowledge Graph is open for use by everyone.
Freebase - GNG subgraph
Let's start with a historical excursion. Metaweb began working on its knowledge base in 2005. The way Freebase was filled with data was most like Dbpedia : the lion’s share of knowledge presented in Freebase was Wikipedia data. The difference from Dbpedia was, firstly, the ability to correct the entered data manually, and secondly, that Freebase did not disdain other data sources . Unlike the DBpedia team, Metaweb representatives did not care much about publishing scientific articles (although they started recently, here’s an interesting list) and admitted that the main component code, graphd, is unlikely to ever see the light of the day.
In 2010, the company Metaweb was bought by Google, but, judging by the Freebase mailing list, the search giant did not interfere too much in the affairs of the newly-purchased team. After the release of the colorful video, in which Google rips competitors as pioneering truth with the help of their new intellectual semantic technologies, representatives of Metaweb ( and then Google ) confirmed that Freebase is a very important part of the Knowledge Network, along with Wikipedia and the CIA facts base . During the big work day on the unification of all Google APIs, the Freebase program interface did not make much changes, and its description was simply transferred to developers.google.com . In order to ask something from the knowledge base, we still use the MQL query language (pronounced “mikl”, Metaweb Query Language). For the cause!
First request and editor
Let's start with a simple question: ask a fact for Freebase, for example the date of birth of Leonardo da Vinci:
www.googleapis.com/freebase/v1/mqlread?query={"/type/object/id":"/en/leonardo_da_vinci","/people/person/date_of_birth":null}We get a completely correct result:
{ "result": { "/type/object/id": "/en/leonardo_da_vinci", "/people/person/date_of_birth": "1452-04-15" } } In order to make it easier for us to practice, we will use the query editor , kindly provided by Freebase.
This editor is terribly convenient and has an excellent feature for auto-completion of queries - in case of difficulty, just press Ctrl + Enter and you will get excellent contextual prompts. The bottom panel of the editor contains useful tools, which are described in detail in the manual . When studying independently, we especially recommend looking at the examples button, which contains sample queries that clarify many of the features of MQL.
Well, here is our request, but the answer is:
| Request | Answer |
|---|---|
| |
Let's sort this request in more detail. We specified an object identifier in Freebase using the term
id . All objects have identifiers, and the word id is an abbreviation of /type/object/id . There are many other /type/object properties that all Freebase entities possess, they will be discussed a bit later.An object having the identifier
/en/leonardo_da_vinci may have the property /people/person/date_of_birth , the values ​​of which we do not know. Instead, we put the special word null instead of this value, in place of which Freebase will write the value from the base in the answer.It is easy to notice that the request and the response to it are symmetrical.
Complex query
Now, in order to have more questions, we will create a rather complicated MQL query and explain it briefly. After that, it will be possible to begin a detailed study of the structure of Freebase and an overview of the possibilities of the language.
So, here is our query (taken from the MQL manual):
| Request | Answer |
|---|---|
| [ { |
Let us briefly describe what kind of MQL tools are used in this query.
First, as you can see, the entire query is wrapped in the
[ { } ] construct, which means that you expect an array of objects as a result, not just one object, as is the case with { } .Lines 2-4 should not cause any problems: we are looking for an object like an album (
/music/album ), we want to get its name and we are not interested in albums called " Greatest Hits ".In lines 5-8 and 11-15, the operator appears OR
|= - we are interested in albums whose release date is 1978 or 1979. We now turn to the genre:"genre" : [ ] ,
"a: genre" : "New Wave" ,
"b: genre | =" : [
"Punk Rock" ,
"Post-punk" ,
"Progressive rock"
] ,
The first line says that we want to get a list of the genres of these albums in the request response. To do this, we added an empty list
[ ] to the query. Then we say that we are interested only in albums, in the genres of which New Wave are indicated. Finally, lines 23-24 contain MQL directives: I’m only interested in two results (
limit ) and I want to sort them by name ( sort ).JSON to MQL
MQL-requests and responses to them are JSON-objects, so for the youngest (or those who do not belong to web developers) I will talk about JSON.
General information about JSON
JSON (JavaScript Object Notation) is a language created for exchanging data in a key-value format. Initially, JSON was used to serialize JavaScript objects, but quickly became language independent and, due to its simplicity, became very loved and respected by programmers in different languages ​​and platforms.
The simplest JSON object is an empty object. It is written as follows:
Now let's create an object that stores information about Leonardo da Vinci. For a start we will restrict ourselves to his name. To do this, enclose the key and value in quotes, separated by a colon
Add a few facts about Leonardo, separating them with a comma:
Now it is necessary to determine what kind of profession Da Vinci had. And there were many of these professions: a sculptor, an artist, an architect, and many more. In order to assign several values ​​to a single key, JSON uses lists of values ​​— the values ​​enclosed in square brackets, separated by commas:
One more thing about JSON that you should know is embedded objects. They are made extremely simple: after the key, you simply insert a new set of key-value pairs in curly braces. In the case of Leonardo, we can try to display data on the place of birth of Leonardo - Ankiano village, located in Italy. We will say that the key "place_of_birth" corresponds to some object with the name Anchiano, located in Italy:
The simplest JSON object is an empty object. It is written as follows:
{} Now let's create an object that stores information about Leonardo da Vinci. For a start we will restrict ourselves to his name. To do this, enclose the key and value in quotes, separated by a colon
{ "name" : "Leonardo di ser Piero da Vinci" } Add a few facts about Leonardo, separating them with a comma:
{ "name" : "Leonardo di ser Piero da Vinci", "date_of_birth": "1453-04-15", } Now it is necessary to determine what kind of profession Da Vinci had. And there were many of these professions: a sculptor, an artist, an architect, and many more. In order to assign several values ​​to a single key, JSON uses lists of values ​​— the values ​​enclosed in square brackets, separated by commas:
{ "name" : "Leonardo di ser Piero da Vinci", "date_of_birth": "1453-04-15", "profession": [ "Architect", "Engineer", "Anatomist", "Inventor", "Artist", "Sculptor" ], } One more thing about JSON that you should know is embedded objects. They are made extremely simple: after the key, you simply insert a new set of key-value pairs in curly braces. In the case of Leonardo, we can try to display data on the place of birth of Leonardo - Ankiano village, located in Italy. We will say that the key "place_of_birth" corresponds to some object with the name Anchiano, located in Italy:
{ "name" : "Leonardo di ser Piero da Vinci", "date_of_birth": "1453-04-15", "profession": [ "Architect", "Engineer", "Anatomist", "Inventor", "Artist", "Sculptor" ], "place_of_birth": { "name": "Anchiano", "containedby": "Italy", }, } Not quite json
In general, MQL queries are not valid JSON objects. MQL is a superset of JSON and all sorts of freedoms are allowed in it. One of the ideas of Metaweb products is that programs should be able to forgive users for errors and errors that they make. This idea is in other languages ​​and programs, and first of all - in the World Wide Web - it's okay that some parts of html are written with errors, you need to try to display the document anyway.
Here, for example, is a valid JSON request that searches for people with a rare and valuable profession:
{ "id": "/en/pope", "/people/profession/people_with_this_profession": [{ "name": null, "limit": 4 }] } We can remove quotes and the query will continue to work:
{ id: /en/pope, /people/profession/people_with_this_profession: [{ name: null, limit: 4 }] } Closing the parentheses and separating pairs with a colon is also not necessary, so here’s an example of a very ugliness:
id /en/pope /people/profession/people_with_this_profession [{ name null limit 4 Freebase device
The official manual gives a very good introduction to how the data is stored inside Freebase. This is not too important for us, because the fours of objects used in Freebase are completely hidden behind the object paradigm. If you are interested, you can refer to the corresponding page of the manual.
So, Freebase allows us to think about what objects lie inside it. Each object is delimited by curly brackets
{ } and consists of property-value pairs, separated by colons. The objects that Freebase issues as answers to MQL queries are valid JSON objects, but they are not similar to OOP paradigm objects. It is best to think of them as unordered sets of pairs.As a property (that is, what stands up to a colon) in MQL there can be identifiers. As values ​​can be identifiers, literals, arrays, and finally nested objects. Freebase has rules by which identifiers should be built. The identifier consists of a namespace and a key, separated by a forward slash
/ . For example, consider the identifier /people/person/date_of_birth - in it date_of_birth is the key, and /people/person is the namespace.IDs are unique. They are not required to carry a semantic load, but it is often easy to understand what is being said by object identifier.
Universal properties of objects
All Freebase objects have the following reserved (generic properties):
- name -
/type/object/name - key -
/type/object/key - id -
/type/object/id - type (usually more than one) -
/type/object/type - creation time -
/type/object/timestamp - creator -
/type/object/creator - access mode -
/type/object/permission - global id -
/type/object/guid - machine id -
/type/object/mid
Here we will look at the properties that are most often used in MQL queries: names, identifiers and types.
Identifiers
There are a lot of identifiers in Freebase. The most important of them is
/type/object/guid , given once and for all. There is its short form /type/object/mid . Well, usually /type/object/id is used in queries - it is often human-readable. The most important thing is that there are no two objects with the same identifiers. For example, take a look at how many people called Adam Smith: an article on the English Wikipedia . Only the moral philosopher Adam Smith wears a proud identifier /en/adam_smith . All other Adams Smith will be identified differently, whether they are politicians ( /en/adam_smith_1965 ), footballers ( /en/adam_smith_huddersfield ), or anyone else.You can enter the identifier in the search field on Freebase.com and get the property property page:
Property / type / object / name
Each object has a name. The name is not a unique thing, the object usually has several names - one for each language. The most interesting thing is that it doesn’t complicate queries at all - you will notice that when you request names you will only receive a name in the language set in Freebase as the current one. So you can treat objects of type as with normal strings.
Property / type / object / type
This property specifies the type of the object. One object can have several types - usually it happens.
If you specified the
type property in the query, then you can omit the namespace for this type. What properties are of the type of /film/director ? Of course, those that are in the namespace of this type, that is, those that begin with /film/director . Consider, for example, a request for all films made by Stanley Kubrick. On the left, the request is shown in abbreviated form, which we will use further, and on the right, what it might look like if the Metaweb developers were not so kind to us.| Request | Request in full form |
|---|---|
| |
Secondly, all properties from the namespace / type / object can be omitted - that is why we have the right to write simply
id , name , type , etc. Why? Because all objects in Freebase are of type Object.Different types of MQL queries
We have already sorted out quite a lot of requests, but have not yet focused on the language itself. First of all, let's look at how the required values ​​are requested in MQL. There are the following cases:
- I need to request one literal value. For example, the date of birth of a person
- I need to request an array of values. For example the list of albums of the musical group.
- I need to request one object with its main properties: identifier, main type and name
- I need to request an array of objects
- I need to know everything about the object
Single value request
If you want Freebase to return you an object of the same structure as the request object, but with an unknown field filled in, then in the request you need to substitute
null instead of this field. We have seen quite a few such examples, here is another one. Ask musician Keith Emerson where he came from:| Request | Answer |
|---|---|
| |
Request array of values
If we try to use
null to query all the albums of a band, we get an error. If you are expecting an array of objects, use square brackets [] . Freebase fills this array with comma-separated strings. There are a lot of examples with albums of musical groups in the official manual, and we will find a list of books written by Hawking:| Request | Answer |
|---|---|
| There are many more |
If, on the contrary, you request an array instead of a single value — there is no problem with this — Freebase will convert the results and return an array with a single value.
Request for objects
Well, but for my application I’m interested to know not only the names of the Hawking books, but also the dates of their release, and the pictures would not hurt! This is also possible. The fact is that the array of books that we received in the last query only looks like an array of strings. In fact, this is an array of objects, just Freebase collapses objects into strings, leaving only their
name property.Also England, where our musician comes from, is not just a string “England”, but a full-fledged object. To get an object representation in a query, use the
{ } construct, like this: { "name": "Keith Emerson", "type": "/music/artist", "origin": { } } As a result, we will be given the most important information about the object: its identifier, name and type list:
query result
{ "result": { "origin": { "id": "/en/england", "name": "England", "type": [ "/common/topic", "/location/location", "/film/film_subject", "/book/book_subject", "/location/administrative_division", "/film/film_location", "/location/uk_constituent_country", "/user/xleioo/winning_night/option_list", "/location/statistical_region", "/location/dated_location", "/symbols/name_source", "/m/04kp2w0", "/user/robert/military/military_power", "/symbols/flag_referent", "/user/skud/flags/topic", "/user/skud/names/topic", "/user/robert/military/topic", "/base/petbreeds/topic", "/m/04mp17s", "/user/xleioo/winning_night/topic", "/organization/organization_scope", "/base/charities/geographic_scope", "/sports/sports_team_location", "/base/thoroughbredracing/thoroughbred_racehorse_origin", "/user/tsegaran/random/taxonomy_subject", "/base/authors/country_of_origin", "/base/authors/topic", "/biology/breed_origin", "/fictional_universe/fictional_setting", "/government/political_district", "/olympics/olympic_participating_country", "/base/summermovies2009/topic", "/base/leicester/topic", "/base/popstra/location", "/location/country", "/base/england/topic", "/base/ontologies/ontology_instance", "/event/speech_topic", "/user/skud/legal/treaty_signatory", "/government/governmental_jurisdiction", "/base/masterthesis/topic", "/user/jamie/default_domain/yacht_racing/yacht_racing_country", "/base/horticulture/cultivar_origin", "/base/horticulture/topic", "/base/localfood/food_producing_region", "/base/localfood/topic", "/sports/sport_country", "/base/todolists/topic", "/base/tagit/concept", "/food/beer_country_region", "/periodicals/newspaper_circulation_area", "/location/uk_statistical_location", "/base/biblioness/bibs_location", "/base/biblioness/bibs_topic", "/base/aareas/schema/gb/constituent_country", "/base/aareas/schema/administrative_area", "/base/uncommon/topic", "/base/schemastaging/statistical_region_extra", "/people/place_of_interment", "/base/allthingsnewyork/topic", "/base/events/topic", "/base/events/geographical_scope", "/base/tonyfranksbuckley/topic", "/base/piratesofthewirralpeninsula/topic", "/military/military_combatant", "/military/military_post", "/organization/organization_member" ] }, "name": "Keith Emerson", "type": "/music/artist" } } Subqueries
You can get any other information about the country of origin of our musician, thus forming an inline request. For example, I would like to know what language is spoken in the country where Emerson is from. Notice, I add a type for the country to get hints from the query editor:
| Request | Answer |
|---|---|
| |
: , , ?
| Answer | |
|---|---|
| |
. , , . , .
limit , :| Answer | |
|---|---|
| |
— . , : ,
"*" : []| Answer | |
|---|---|
| |
, , , Freebase :
| Design | |
|---|---|
| . default- : value ( ) name ( ). , ! |
| . , , |
| |
| . , |
| . {}, |
| - , |
| . . |
Enough to start. , MQL , , , . Acre, , Semantic MediaWiki. Dbpedia Wikidata. ?
Source: https://habr.com/ru/post/177217/
All Articles