Dynamic ontology. How the Palantir engineers explain this to the CIA, the NSA and the military
Palantir is the fourth-longest privately owned company in Silicon Valley (after Uber, Xiaomi and Airbnb). While Palantir collects information about everything, we collect information about him. Together with Edison, we continue to investigate the capabilities of the Palantir platform.
ITishnies thought of how to effectively “monetize mathematics and algorithms” (Segalovich, Bakunov), PayPal Mafia thought of how to monetizegadgets of Feanor philosophy (the capitalization of Palantir is $ 20 billion).
')
In a ten-minute lecture, an employee of Palantir will talk about the central concept of their system - a dynamic ontology.
0:00 Hi, I'm Asher Sinensky, Palantir Technology Deployment Engineer. I will talk about dynamic ontology.
0:08 Obviously, now, these two words look rather vague to you, I hope that by the end of the conversation you will understand what meaning we are putting into them.
0:17 Before getting down to business, I will explain: many people have problems with the word ontology. What do we mean by this word?
0:24 If you look at the roots of this word, it comes from the Greek “ontos” (being) and “logia” (studying something). In essence, ontology is the categorization of the world.
0:34 There are a lot of terms that people use to describe this: taxonomy, data model schematizer. But we use it, in a broader sense, as an idea that we really categorize the world in some way.
0:43 The idea of ​​building an ontology for exploring the world is not new. The first who approved this idea was a man named Plato. The idea of ​​Platonov's realism, basically, is that there are real things, and there is our idea of ​​things.
1:02 In his model, the things of the real world are a manifestation of certain patterns from the ideal world.
1:07 He talked about the universal and the particular; about ontology and being itself; about the form and copy (perhaps an idea and embodiment?). Computer scientists have a class and an object.
1:15 In general, this is the idea that there is a concept of something and this is something. And what makes it an apple brings us closer to what the apple concept is (correct me, people who are familiar with Plato’s philosophy).
1:29 Well, this is a cool philosophy, but the question now is how to use it, how it will help us in creating a tool that could be useful in the information community.
1:36 A great example of a convenient ontological structure is the periodic table of Mendeleev. Here we have the number ten (shows arsenic, atomic mass), it says something about weight, about mass, but what we see here is not the “ten” itself, in our real world, conceptualizing what “ten” should mean.
1:56 Another well-known example of ontology, or rather taxonomy: this is a linear taxonomy, with Latin names for animals, and this is a good example of hierarchical ontology.
2:06 Here you can move down to the genus panthera (lat. Panther), and you can use the division into species with Latin names: leo, tigris, pardus. You can create objects "lion" "tiger" "leopard".
2:20 Obviously, panthera pardus is not a leopard itself, it is a concept of what a leopard is.
2:25 Well, and you know, there is a living leopard in the real world and panthera pardus in ontology.
2:29 These same concepts can be applied to areas that are more meaningful if we are talking about a person. For example, here is a model of human ontology (person): we started with objects, passed to entities, further to living entities, then to people.
2:43 If we want to characterize a person further, then we can add such concepts as: pilot, lawyer, and doctor — these concepts are united by the general idea of ​​singling out subspecies. We'll talk more about this later.
2:53 So, how can you model in ontology in practice? How would you structure your ontology?
2:58. People tried several ways. I will list the four most pronounced approaches.
3:03 Top tied to the objects. In object models, for example, the taxonomy of animals by the Latin name, we have certain objects that are concepts that must correspond to objects in the real world. You can add a little depth, saturation, adding features, and thus get objects with features.
3:21 For an example, we can take the Periodic Table, where objects have weight and number of protons.
3:28 What is missing in this table is the indication of a relationship. Some relationships can be understood by analyzing the structure of the table, but you cannot know, for example, whether these two substances form an alloy.
3:40 Another way you can model the world is objects and relationships, that is, you have objects and how they relate to each other.
3:47 And here the signs that were on the periodic table are missing. You can get an idea of ​​the signs if you rely on connections between two objects, for example, one of the data object objects, and if you want to say that Mike Fikri (a fictional villain from one of the first commercials about Palantir) blue eyes, you create a connection between the objects "Mike Fikri" and "blue eyes".
4:08 This is going to be a little weird, but you can get a pretty complete picture of the object.
4:13 Finally, the most revealing way of using objects, connections, and attributes. As you can guess, this is the way we use in Palantir.
4:22 Let's take a little deeper look at Paltar's ontology, how it applies.
5:08 (The ontology permeates almost every function in the Palantir workspace. Therefore, having a properly formed ontology is critical for effective analysis.)
5:15 I said that Palantir is a dynamic ontology, let's talk about what I mean by dynamics.
5:21 The first thing we have in mind when speaking about dynamics is that the ontology in Palantir is not hard coded. For example, the axial layer, the layer of interpretations, the superstructure above the databases, the analysis and user interaction layer are performed this way.
5:33 There are some concepts that are tightly coded: object, attribute, relationship. Objects are also strictly divided into documents, entities and events. And this is all, - all ontology, given in advance. In fact, it is more than the ontological structure to which you are attached and which you are accustomed to use.
5:50 Why soft coding? Why not code ontology hard? I will talk about a couple of disadvantages of this approach.
6:00 The problem is that using a hard-coded ontology, you inevitably end up in one of two camps.
6:06 You have either a very generalized ontology, or a very special one, and you have to choose where you want to be.
6:12 In generalized ontology, you are not actually modeling anything. You have objects, signs and relationships, and you can put anything inside such an ontology, but, in general, that's all, there is nothing to model there.
6:21 Some try to create very specialized ontologies where you can model everything you want. The problem here is that semantics becomes over-defined / over-defined.
6:30 What do we mean by overdetermined semantics? This is a situation where it becomes overly difficult to understand how something should be modeled.
6:35 You can introduce the concept of "notice" (citation - statement, summons, mention), and then one will say: "Oh, the notice is when I get a parking ticket" - and the other that the notice comes when a person gets a bonus.
6:48 If you need to model similar things in a super-defined ontology using the “notification”, it becomes difficult to separate them.
6:54 If you use soft coding, a spectrum appears between our camps, and, starting with a general model, you will reach a certain point that meets your objectives (sweet spot).
7:11 Another significant advantage of dynamic ontology is that it is a flexible system: you can model many different concepts, or one concept in many different ways.
7:19 Flexibility - let's see how it works. Answer the question: "How to simulate a person's occupation?"
7:28 The slide says: "The working function of a person can be classified through the definition of objects."
First, take a look at the object model of human employment. We have classified the working functions as objects, and you have already seen this slide.
7:40 We have a man upstairs and a pilot, a lawyer, a doctor downstairs, so this looks like a model of animals with genera and species.
7:47 What are the advantages and disadvantages of this method? The disadvantage is that you cannot make a person a lawyer and a doctor at the same time (like a lion and a tiger), you have to choose.
7:56 The advantage is that you have a lot of signs specific to a doctor or lawyer, and you can use these signs.
8:04 Medical specialization is obviously different from legal, and you can easily use this difference when working with objects.
8:12 The slide says: “The working functions of a person can be classified by defining the signs of his employment”
You can use the signs to get an idea of ​​a person’s employment.
8:17 We can say that a person may have a sign of "employment" and different values ​​of this sign.
8:24 You can be a doctor, a pilot, a lawyer, and if you have an idea about the multiplicity of a trait, you can choose several values. In Palantir it is possible: doctor and pilot, pilot and lawyer, lawyer and cook.
8:35 The drawback here is that you will have to try to add to the lawyer the information that he is from such a high school without linking this information to this feature for good.
8:45 Finally, you can get an idea of ​​a person’s career using relationships. The reason that someone is called a doctor is not because he graduated from a medical school, the reason is that this person treats patients.
8:58 So you can say that person 1 is a doctor in relation to person 2, person 1 is a pilot because he controls the plane, person 1 is a lawyer because he provides legal services to person 3. So, in fact, career.
9:08 So what we see here are three career modeling concepts. You cannot do this if you use a hard-coded ontology, such as the norms of society.
9:21 I talked about flexibility, about soft coding, but not all of these things mean dynamic. So what is the real dynamics of the Palantir ontology?
9:31 The fact is that in Palantir, the ontology can be changed after it has been deployed. Thus, an ontology can develop along with how the organization’s vision of the world is developing.
9:41 A few examples of the changes you can make:
- any object, feature or relationship can be deleted or added;
- the functionality of objects, signs and relationships can be changed - and this is probably the most important.
9:54 This is important because the way most people interact with ontology is to try to understand what different signs are responsible for and other things.
10:05 Here's a slide, for example, a list of some of the things you can change:
10:08 These changes have a significant impact on how people will interact with ontology and how they will be analyzed.
10:15 Now let's quickly summarize. With the help of a dynamic ontology, you can model everything your organization needs in your area of ​​interest. This ontology is not hardcode, but softcode.
(For assistance in preparing the article, special thanks to Alexey Vorsin, Russian expert on the Palantir system)
More about Palantir:
ITishnies thought of how to effectively “monetize mathematics and algorithms” (Segalovich, Bakunov), PayPal Mafia thought of how to monetize
')
In a ten-minute lecture, an employee of Palantir will talk about the central concept of their system - a dynamic ontology.
0:00 Hi, I'm Asher Sinensky, Palantir Technology Deployment Engineer. I will talk about dynamic ontology.
0:08 Obviously, now, these two words look rather vague to you, I hope that by the end of the conversation you will understand what meaning we are putting into them.
0:17 Before getting down to business, I will explain: many people have problems with the word ontology. What do we mean by this word?
0:24 If you look at the roots of this word, it comes from the Greek “ontos” (being) and “logia” (studying something). In essence, ontology is the categorization of the world.
0:34 There are a lot of terms that people use to describe this: taxonomy, data model schematizer. But we use it, in a broader sense, as an idea that we really categorize the world in some way.
0:43 The idea of ​​building an ontology for exploring the world is not new. The first who approved this idea was a man named Plato. The idea of ​​Platonov's realism, basically, is that there are real things, and there is our idea of ​​things.
1:02 In his model, the things of the real world are a manifestation of certain patterns from the ideal world.
1:07 He talked about the universal and the particular; about ontology and being itself; about the form and copy (perhaps an idea and embodiment?). Computer scientists have a class and an object.
1:15 In general, this is the idea that there is a concept of something and this is something. And what makes it an apple brings us closer to what the apple concept is (correct me, people who are familiar with Plato’s philosophy).
1:29 Well, this is a cool philosophy, but the question now is how to use it, how it will help us in creating a tool that could be useful in the information community.
1:36 A great example of a convenient ontological structure is the periodic table of Mendeleev. Here we have the number ten (shows arsenic, atomic mass), it says something about weight, about mass, but what we see here is not the “ten” itself, in our real world, conceptualizing what “ten” should mean.
1:56 Another well-known example of ontology, or rather taxonomy: this is a linear taxonomy, with Latin names for animals, and this is a good example of hierarchical ontology.
2:06 Here you can move down to the genus panthera (lat. Panther), and you can use the division into species with Latin names: leo, tigris, pardus. You can create objects "lion" "tiger" "leopard".
2:20 Obviously, panthera pardus is not a leopard itself, it is a concept of what a leopard is.
2:25 Well, and you know, there is a living leopard in the real world and panthera pardus in ontology.
2:29 These same concepts can be applied to areas that are more meaningful if we are talking about a person. For example, here is a model of human ontology (person): we started with objects, passed to entities, further to living entities, then to people.
2:43 If we want to characterize a person further, then we can add such concepts as: pilot, lawyer, and doctor — these concepts are united by the general idea of ​​singling out subspecies. We'll talk more about this later.
2:53 So, how can you model in ontology in practice? How would you structure your ontology?
2:58. People tried several ways. I will list the four most pronounced approaches.
3:03 Top tied to the objects. In object models, for example, the taxonomy of animals by the Latin name, we have certain objects that are concepts that must correspond to objects in the real world. You can add a little depth, saturation, adding features, and thus get objects with features.
3:21 For an example, we can take the Periodic Table, where objects have weight and number of protons.
3:28 What is missing in this table is the indication of a relationship. Some relationships can be understood by analyzing the structure of the table, but you cannot know, for example, whether these two substances form an alloy.
3:40 Another way you can model the world is objects and relationships, that is, you have objects and how they relate to each other.
3:47 And here the signs that were on the periodic table are missing. You can get an idea of ​​the signs if you rely on connections between two objects, for example, one of the data object objects, and if you want to say that Mike Fikri (a fictional villain from one of the first commercials about Palantir) blue eyes, you create a connection between the objects "Mike Fikri" and "blue eyes".
4:08 This is going to be a little weird, but you can get a pretty complete picture of the object.
4:13 Finally, the most revealing way of using objects, connections, and attributes. As you can guess, this is the way we use in Palantir.
4:22 Let's take a little deeper look at Paltar's ontology, how it applies.
5:08 (The ontology permeates almost every function in the Palantir workspace. Therefore, having a properly formed ontology is critical for effective analysis.)
5:15 I said that Palantir is a dynamic ontology, let's talk about what I mean by dynamics.
5:21 The first thing we have in mind when speaking about dynamics is that the ontology in Palantir is not hard coded. For example, the axial layer, the layer of interpretations, the superstructure above the databases, the analysis and user interaction layer are performed this way.
5:33 There are some concepts that are tightly coded: object, attribute, relationship. Objects are also strictly divided into documents, entities and events. And this is all, - all ontology, given in advance. In fact, it is more than the ontological structure to which you are attached and which you are accustomed to use.
5:50 Why soft coding? Why not code ontology hard? I will talk about a couple of disadvantages of this approach.
6:00 The problem is that using a hard-coded ontology, you inevitably end up in one of two camps.
6:06 You have either a very generalized ontology, or a very special one, and you have to choose where you want to be.
6:12 In generalized ontology, you are not actually modeling anything. You have objects, signs and relationships, and you can put anything inside such an ontology, but, in general, that's all, there is nothing to model there.
6:21 Some try to create very specialized ontologies where you can model everything you want. The problem here is that semantics becomes over-defined / over-defined.
6:30 What do we mean by overdetermined semantics? This is a situation where it becomes overly difficult to understand how something should be modeled.
6:35 You can introduce the concept of "notice" (citation - statement, summons, mention), and then one will say: "Oh, the notice is when I get a parking ticket" - and the other that the notice comes when a person gets a bonus.
6:48 If you need to model similar things in a super-defined ontology using the “notification”, it becomes difficult to separate them.
6:54 If you use soft coding, a spectrum appears between our camps, and, starting with a general model, you will reach a certain point that meets your objectives (sweet spot).
7:11 Another significant advantage of dynamic ontology is that it is a flexible system: you can model many different concepts, or one concept in many different ways.
7:19 Flexibility - let's see how it works. Answer the question: "How to simulate a person's occupation?"
7:28 The slide says: "The working function of a person can be classified through the definition of objects."
First, take a look at the object model of human employment. We have classified the working functions as objects, and you have already seen this slide.
7:40 We have a man upstairs and a pilot, a lawyer, a doctor downstairs, so this looks like a model of animals with genera and species.
7:47 What are the advantages and disadvantages of this method? The disadvantage is that you cannot make a person a lawyer and a doctor at the same time (like a lion and a tiger), you have to choose.
7:56 The advantage is that you have a lot of signs specific to a doctor or lawyer, and you can use these signs.
8:04 Medical specialization is obviously different from legal, and you can easily use this difference when working with objects.
8:12 The slide says: “The working functions of a person can be classified by defining the signs of his employment”
You can use the signs to get an idea of ​​a person’s employment.
8:17 We can say that a person may have a sign of "employment" and different values ​​of this sign.
8:24 You can be a doctor, a pilot, a lawyer, and if you have an idea about the multiplicity of a trait, you can choose several values. In Palantir it is possible: doctor and pilot, pilot and lawyer, lawyer and cook.
8:35 The drawback here is that you will have to try to add to the lawyer the information that he is from such a high school without linking this information to this feature for good.
8:45 Finally, you can get an idea of ​​a person’s career using relationships. The reason that someone is called a doctor is not because he graduated from a medical school, the reason is that this person treats patients.
8:58 So you can say that person 1 is a doctor in relation to person 2, person 1 is a pilot because he controls the plane, person 1 is a lawyer because he provides legal services to person 3. So, in fact, career.
9:08 So what we see here are three career modeling concepts. You cannot do this if you use a hard-coded ontology, such as the norms of society.
9:21 I talked about flexibility, about soft coding, but not all of these things mean dynamic. So what is the real dynamics of the Palantir ontology?
9:31 The fact is that in Palantir, the ontology can be changed after it has been deployed. Thus, an ontology can develop along with how the organization’s vision of the world is developing.
9:41 A few examples of the changes you can make:
- any object, feature or relationship can be deleted or added;
- the functionality of objects, signs and relationships can be changed - and this is probably the most important.
9:54 This is important because the way most people interact with ontology is to try to understand what different signs are responsible for and other things.
10:05 Here's a slide, for example, a list of some of the things you can change:
- add / edit label generators;
- add / change aliases;
- add / change parsers;
- add / change control parameters;
- add / change icons;
- add / change approximate values;
- add / modify feature limits;
- add / modify link constraints.
10:08 These changes have a significant impact on how people will interact with ontology and how they will be analyzed.
10:15 Now let's quickly summarize. With the help of a dynamic ontology, you can model everything your organization needs in your area of ​​interest. This ontology is not hardcode, but softcode.
(For assistance in preparing the article, special thanks to Alexey Vorsin, Russian expert on the Palantir system)
More about Palantir:
- Cybercontrol. How can Palantir catch snowdens?
- Palantir: how to detect a botnet
- Palantir and money laundering
- Palantir: arms trade and the spread of the pandemic
- Palantir, PayPal Mafia, special services, world government
- Palantir 101. What is allowed to ordinary mortals to know about the second most abrupt private company in Silicon Valley
Source: https://habr.com/ru/post/281118/
All Articles