Geekly Articles each Day
📜 ⬆️ ⬇️

Method of formalized models as an alternative to neural networks

The dominance of statistical methods, especially neural networks, pushes me to this stuffing - yes, I will classify them in this way. On the one hand, I have nothing against them, but at the same time, there is a clear bias, sometimes even neural networks are almost identical with the vague concept of artificial intelligence, although is it worse than SVM, HMM, etc. In the matter of processing natural languages, I have always been a supporter of linguistic methods as opposed to statistical ones, but their essential drawback is felt - the complexity of manual modeling compared to machine learning. Or maybe machine learning is also applicable to linguistic methods?


I do not undertake to answer this question in this description, but I propose some reflections. I note that the main idea came to me a long time ago and was framed in the form of patent No. 2392660, half of which has almost expired. But what if we slightly expand the solution in this patent?


What I thought up before


A patent describes a semantic model of a natural language, but some modification allows you to create a semantic model of images. So if you select a common part, then the potential scope expands.


I will begin with the previously invented semantic model. In this case, it is a generalized semantic, but not ontological. But at the same time multilevel, approximately as words, sentences, paragraphs, chapters, etc., only with other units for a natural language and other units for other cases, but more on that later. So, one level includes (quote from previously published articles):



Now a little more about what it is, on the example of natural languages. Since the same phrase can be expressed in different words, the division into words does not make sense in the semantic model. The situation is similar with complex sentences, but it makes sense to select simple ones; clauses - their well-established name. Consider the semantic model inside the clause.


The concepts of lexemes and semantic classes are also known. Often, lexemes correspond to words, but in general, the ratio is many to many. The lexemes themselves are polysemantic, but the semantic class captures a specific value. I will not do a review of markup methods of semantic classes now, if necessary, I will leave for later - a lot has already been written on this topic. For the lexemes with semantic classes themselves there are different names, here let them be called denotations.


There is a need to separate them (denotation) into two types: objects and predicates. The first ones themselves have a certain meaning, the second ones require the addition of other denotates, here we will call them (complementary denotates) actants.


Denotats still do not solve the problem of the possibility of expressing meaning in different words, so we divide them into basic and composite ones, in more detail about the criteria in the preface to the "New Explanatory Dictionary of Synonyms" and other works by the same author. It is the division of the composite denotations on the base to solve this problem. Actually, at this level, i.e. within simple sentences, the set of basic predicates will be the set of connecting elements.


Denotaty objects in pure text, ie, without drawings, diagrams, formulas, listings, etc., will be of two types - discrete and continuous, the second - mostly directly numbers. The sets of basic denotation objects will, of course, be endless.


A little running ahead, it should be noted that the composite denotations include differential and characteristic components — we use their name here, in this case they are the only one-place operators. They make it possible to distinguish logical (complete) negation from linguistic, where negation extends only to differential components.


In this case, classifiers will refer only to object denotations, and they will be the semantic classes mentioned earlier. It is assumed the use of parallel hierarchies and the presence of the upper universal class. Hierarchies themselves can begin with any semantic classes, but they belong to the universal semantic class completely. Restrictions on compatibility will apply only to composite predicate denotations (see below) and for specific actants include many combinations of semantic classes and hierarchies. That is, for a particular actant, the restriction includes a potentially infinite set of semantic classes, expressed by a finite set of combinations of semantic classes and hierarchies. In this case, the actant can only be an object belonging to one of the appropriate semantic classes. The actants of the basic predicates of the always universal semantic class.


The sets of composite elements (also quoted from previously published articles):



The binding elements are also predicate denotates, but in this case are composite. Non-dwelling elements can be both composite objects, or extend to the entire clause or fragments corresponding to the phrases, the main requirement is the indication of all the actors.


Each composite predicate, combining basic predicates and objects, can impose restrictions on the subsets of the semantic classes of its actants in the direction of detailing - this set of restrictions is, of course, infinite.


It is advisable to present a composite predicate in conjunctive normal form, where the disjunctive members have a single operator, whether they are differential and characteristic. In accordance with them, linguistic negation may lead to a logical one. In addition, it is assumed the possibility of a general logical simplification, for example, the exclusion of identical objects united by a logical operator. The second group of substitution rules will relate to basic predicates, such as the possibility of excluding opposing actions, for example, verbalization of the noun and substantivization of the verb in the meaning of action. The third group of rules is simultaneously to basic predicates and objects, implying the possibility of excluding joining predicates for actants of a composite predicate when an object corresponds to a given set of semantic classes.


A few words about the other sets. For the Clause level, I find it difficult to drill down on these sets, only the fact that they will constitute a natural language text is obvious. In this case, the composite object denotates of the level described above, which express clauses completely, will be the basic objects of this level.


For images, different approaches are all the more possible, atomic elements can be numerical sets for filter characteristics or texels in the case of textures, in other cases there are also options. Some possible solutions for the remaining semantic models will follow.


What now


Next, it makes sense to return to the original question of the application of machine learning. It is clear that compiling a dictionary of objects, semantic classes, etc. manually is extremely laborious. But modern methods of machine learning in the case of natural languages ​​are able to solve the problem not only of word matching, but also of statistical detection of denotations, which in the case of a neural network remain a black box. At the same time, other statistical methods, such as clustering, provide clear results.


The infinite sets, of course, are such only in theory, in practice this means that machine learning is designed to constantly update their composition. Since this is a semantic model, not an ontological one, maintaining relevance is also one of the tasks.


Now some thoughts on the feasibility of implementation. Take, for example, the semantic vectors offered by Yandex, and recall the notion of a corpus of texts . Suppose there is a certain corpus of queries in the context of Yandex algorithms and a set of certain semantic matrices of documents, although I myself do not fully understand what kind of details are in all details of the data structure. But, I suppose, there is data on the words included in documents and queries, which means that it is possible to compare the presence of words in documents and queries with classical search tools, and to compare relevance based on semantic vectors. What exactly to use - words, N-grams or other models for clustering - is a rather open question, but one way or another it is possible to distinguish semantic classes based on cluster analysis.


One more statistical model needs to be obtained in the form of a directed asymmetric graph of connections between words, i.e., to what extent one word can replace another in relevant queries. The base denotates will obviously have a pronounced asymmetrical weight of the arcs. The specific algorithm for traversing the resulting graph to identify the base denotates is rather an open question, especially considering the large computational complexity.


The options mentioned use the previously proposed neural networks as an intermediate link, but if we consider that many clustering algorithms are iterative in themselves, then simplification is definitely possible. For now I’ll leave some questions open.


What's the point


A few words about why all this may be needed. By and large, these are the typical advantages of an explicit model over the "black box" of the neural network. Firstly, it is the possibility of optimization. If the search is based on the neural network, then the denotations are without clear boundaries, which makes it difficult to optimize the equivalent of any step, be it the markup of semantic classes or the comparison of basic denotations. Secondly, when the existing concepts are updated, in the case of a neural network, the set of changes remains all the more uncertain from the point of view of interpretation, which makes it difficult to debug in the case of retraining. Thirdly, if the search task is replaced, for example, with the reference task, the complexity of identifying that part of the neural network that deals directly with the search task after the initial semantic analysis also remains. If I am mistaken - correct.


')

Source: https://habr.com/ru/post/340778/

More articles:


All Articles