Why can't Ukrainian guys succeed in an effective gesture translator?
After an interesting publication about the invention for the deaf-mute (See “Ukrainian students created gloves that translate sign language into speech” ), for the authors of which I was sincerely glad, I expressed my doubts in the comments about the practical benefits of such innovation at the moment, being sure that today there’s hardly any effective tool that can really help to understand sign language.
But it will not work out exactly for the same reason, which prevents the creation of high-quality natural-language machine translator. And this reason lies exactly in the fact that a computer, as we know, is not trained to understand the meaning of a language message, whether it is presented, though in a verbal form, though in an epistolary (textual) form, or even in the form of gestures.
The machine is able to perceive only algorithmized things, for example, some rules from the field of linguistics. But with their help it is impossible to close the SENSE SPACE, which is NADLINGUISTIC and therefore NOT FORMALIZED within the framework of linguistic rules.
')
In order to understand (extract) the Meaning from any language message, including gestural, to a recipient person (seeing, hearing, reading ...) speech, text, gestures ... you need to have in your consciousness (memory) a certain BASE of a certain number of Behavior Models Images (MPO), which should be enough to recognize the DESIGN laid down in this language message (speech, text, gestures ...) by the author. So, these MPOs make up the Knowledge Base (BZ) of people. At the same time, the quality and quantity of MPOs (in English Models of Image Behaviors => MIBs) for different language groups vary greatly, as a result of which people of different language cultures do not understand each other.
Sign language is no exception. When ordinary people look at the communication of deaf and dumb, they do not understand them for exactly the same reason - due to the lack in our memory of those IGOs that the participants have conversations in sign language.
And how do these IGOs arise? Yes, in the process of learning. In our childhood, for example, they show our mother and say “mother.” We remember the Image [mom] and its language matching. Then we are shown to the uncle who is walking and they say: “Uncle is coming” and we remember this model of behavior in our virtual memory. This is how the IGOs are formed in our mind. Therefore, in order to understand messages in any language, we need to understand THROUGH their correspondences - graphic, sound, tactile, or gestures, which particular IGOs are formed in these messages. If there is no MPO, then there is no understanding, no extraction of meaning. And, if there are no CONFORMATIONS, then, moreover, there can be no understanding.
Here we may be puzzled by the question: “What, then, does the machine need, so that it can extract Meaning from the language message?” The answer is very simple - yes, you need exactly the same thing as a person - the Knowledge Base.
And what, if modern computers do not have a Knowledge Base? After all, we constantly read about various programs that supposedly use BRs and even extract some Meanings and Knowledge from something there? No, unfortunately, what is meant by the "Knowledge Base", in fact, are not. These are all the same two-dimensional table databases. Knowledge did not smell there.
Well, then what is needed to create such BRs or what criteria criteria should they, these BRs, meet? The answer here may be.
First, we need the Universe Model (MM), without which it is not possible to extract Meaning from a language message.
Secondly, you need to decide on the format of the IGO - it must be DYNAMIC. For example, MPS can be represented as shown here .
Thirdly, the machine must RECOGNIZE the IGOs from language messages. This requires the technology of heuristic-semantic processing of gestures (text, speech ...).
From what is listed, today there is only the third position, and that is in a state of development.
As for the first and second conditions, the approaches to them only just appear.
But now, when these three conditions-requirements will be fulfilled together, then we can again return to the remarkable Ukrainian invention.
But it will not work out exactly for the same reason, which prevents the creation of high-quality natural-language machine translator. And this reason lies exactly in the fact that a computer, as we know, is not trained to understand the meaning of a language message, whether it is presented, though in a verbal form, though in an epistolary (textual) form, or even in the form of gestures.
The machine is able to perceive only algorithmized things, for example, some rules from the field of linguistics. But with their help it is impossible to close the SENSE SPACE, which is NADLINGUISTIC and therefore NOT FORMALIZED within the framework of linguistic rules.
')
In order to understand (extract) the Meaning from any language message, including gestural, to a recipient person (seeing, hearing, reading ...) speech, text, gestures ... you need to have in your consciousness (memory) a certain BASE of a certain number of Behavior Models Images (MPO), which should be enough to recognize the DESIGN laid down in this language message (speech, text, gestures ...) by the author. So, these MPOs make up the Knowledge Base (BZ) of people. At the same time, the quality and quantity of MPOs (in English Models of Image Behaviors => MIBs) for different language groups vary greatly, as a result of which people of different language cultures do not understand each other.
Sign language is no exception. When ordinary people look at the communication of deaf and dumb, they do not understand them for exactly the same reason - due to the lack in our memory of those IGOs that the participants have conversations in sign language.
And how do these IGOs arise? Yes, in the process of learning. In our childhood, for example, they show our mother and say “mother.” We remember the Image [mom] and its language matching. Then we are shown to the uncle who is walking and they say: “Uncle is coming” and we remember this model of behavior in our virtual memory. This is how the IGOs are formed in our mind. Therefore, in order to understand messages in any language, we need to understand THROUGH their correspondences - graphic, sound, tactile, or gestures, which particular IGOs are formed in these messages. If there is no MPO, then there is no understanding, no extraction of meaning. And, if there are no CONFORMATIONS, then, moreover, there can be no understanding.
Here we may be puzzled by the question: “What, then, does the machine need, so that it can extract Meaning from the language message?” The answer is very simple - yes, you need exactly the same thing as a person - the Knowledge Base.
And what, if modern computers do not have a Knowledge Base? After all, we constantly read about various programs that supposedly use BRs and even extract some Meanings and Knowledge from something there? No, unfortunately, what is meant by the "Knowledge Base", in fact, are not. These are all the same two-dimensional table databases. Knowledge did not smell there.
Well, then what is needed to create such BRs or what criteria criteria should they, these BRs, meet? The answer here may be.
First, we need the Universe Model (MM), without which it is not possible to extract Meaning from a language message.
Secondly, you need to decide on the format of the IGO - it must be DYNAMIC. For example, MPS can be represented as shown here .
Thirdly, the machine must RECOGNIZE the IGOs from language messages. This requires the technology of heuristic-semantic processing of gestures (text, speech ...).
From what is listed, today there is only the third position, and that is in a state of development.
As for the first and second conditions, the approaches to them only just appear.
But now, when these three conditions-requirements will be fulfilled together, then we can again return to the remarkable Ukrainian invention.
Source: https://habr.com/ru/post/147527/
All Articles