We translate the recipe into the multicooker language
Strawberries quartered and other features of machine translation: to take another step towards automated kitchens, you need to teach the technique to understand the recipes written in human language and in human logic. To do this, they need to transform. Under the cut, the text is a reflection on the problem that was born over a cup of coffee during the discussion of a related topic.
Smart kitchen appliances facilitate the cooking process, but people still do most of the work: washing, cutting and putting food into them, mixing, etc. And how I would like to just say: “Pot, cook”, and that it all washed itself, cut, mixed, and the user only needs to wait for the beep and go and eat.
Now automatic kitchens only appear on an industrial scale, but it will not come very soon to the household level. To do this, you need to solve several groups of tasks, and these solutions should be available.
The first group of tasks is to give the kitchen the opportunity to act autonomously from the user.
This task, for example, can be solved by introducing a manipulator system , a conveyor into the kitchen;
')
The second group of tasks is to introduce computer vision and a sensor system in the kitchen to automatically determine the degree of readiness of a dish or its part.
The third group of tasks is the management interface. The task can be implemented through the application on smartphones, as well as voice assistants.
And finally, the fourth group - the task of the kitchen logic of behavior. How to teach the kitchen to cook? For example, a robot cook teaches a person - a professional cook. Wearing sensors on his hands, he slowly shows how to. The machine then reproduces its actions, down to the smallest detail, right up to the shaking hands. Of course, during processing, extra movements must be removed, but if the programmer misses something, the robot cook will one day stop to scratch his non-existent nose.
In addition, in order to teach the robot one recipe, the movement must be repeated several times. Even if we teach the robot to separate movements that it can then combine, then the time for such training will be extremely long. Not to mention the strictest compliance with the recipe and, apparently, calibrated ingredients: allowing the robot to pick out potato eyes is a separate complex engineering task. And after cooking, you still have to wash the stove and dishes.
If you go back to the situation now - i.e. to take into account the economic component of the issue, the difficulties of development, etc., it becomes clear that in the foreseeable future we can only teach smart home appliances to read the recipe and use it. To do this, convert the recipe into a code. But classic software Java or C is hardly suitable here - they can describe a recipe, but if the user wants to make any changes to it, then he will have to either contact tech support or sign up for programming courses. Besides, the classic code is too formal. For example, if the ingredients say “dried mushrooms”, then fresh ones will not go into cooking. Further complicates the task of the fact that the recipes of national cuisines are presented in different natural languages.
And I would like to give the user the opportunity to find an interesting recipe on the Web, download it, and tell the robot kitchen to correctly interpret it and apply it. The maximum task is the ability of the kitchen itself, based on the culinary preferences of the user, to search and offer options. Good cooking recipes should not only include a list of raw ingredients and cooking methods, but also the environmental conditions in which products grow. The dish belongs to the national cuisine according to many criteria, including specific cooking methods, used cookware, tools, etc. Transforming all this data into the proper digital form or, most likely, its interpretation will be a difficult task.
So which languages ​​may be suitable for recipes?
Existing technologies of formalized knowledge representation fall into several groups of structures.
These include the Semantic Web approach with its OWL and RDF technologies. Online collections of related data sets using the Semantic Web tools are also known as Linked Data. Recently an article devoted to this concept appeared on Habré, so we will not focus on it. Many digitization projects are built around the idea of ​​using some higher ontology , which can be expanded by engineers in a particular area of ​​expertise.
Consider the case of the use and application of these tools for digitizing national cuisines. The main idea of ​​RDF is that the Internet is moving away from storing information that is perceived only by man, becoming a worldwide network of interacting processes. As the name implies, RDF is the basis for expressing information about resources. First of all - about web documents and various organizations. Its formalism is based on the idea of ​​static classes and properties. The question arises: how reasonable it is to treat a recipe as an entity, and not as a complex process with arguments, timing, incoming subprocesses, etc. For the recipe this will not be enough.
The second approach can be considered the initiative of Schema.org - joint activities of the Internet community to create, maintain and promote schemes for structuring data on the Internet. This initiative aims to provide a standardized dictionary for common metadata published on web resources. Cooking-related web resources can use “Recipe” class field metadata stored by reference . Below is an example of a recipe code provided by Google:
The dictionary and format adopted by Schema.org are primarily focused on the presentation of high-level metadata in web documents. However, a properly normalized semantic graph requires a much more explicit representation of concepts. Most of the string values ​​in Schema.org fields are natural texts that require human cognitive interpretation. Such text cannot be directly read by digital systems without special natural language processing tools, which are often error prone. As an illustration, Google’s machine translation will fit a single line item in the list of ingredients in the above recipe example:
Translation into Russian gives a completely wrong meaning:
with “quartered”, used in the sense of “housing”, as in “Our troops were stationed in Boston”, and not “Cut into four parts”.
The absence of clearly defined object roles and the explicit identification of methods makes the task of machine translation of a recipe much more difficult.
For our futuristic task for digital recipes - a scenario in which a robotic machine could follow the instructions for preparing a dish, a simple line of text cannot be a solution.
Consider a typical ingredient description:
It is clear that this text line contains a lot of classifying information: raw ingredient as a class, a specific sort of apples, number in pieces, a list of methods that should be applied to each part in order to use the ingredient normally: removing seeds, peel, cutting an object into pieces forms.
The formalism of Schema.org is not expressive enough to write the cooking process in some kind of program code. To properly digitize recipes for robotic cooking, we need to separate the ingredient description from the logic
The traditional way to create a recipe is to start with the components and operations that need to be performed. This order and style of description is known as imperative or procedural. The declarative or functional style of describing the logic of a process usually starts from the top of the execution pyramid, the expected useful result we want to achieve.
Consider the following simplified view of the classic Russian recipe for mushroom soup. Arrows denote subprocesses (sometimes alternative) necessary for the parent process.
Cognitive interpretation can be a dead end for the moment when we try to define a recipe. The term “recipe” has several contextual meanings. It can be defined in a general sense as a method of obtaining the desired result. When used in the context of cooking, it means a set of instructions for cooking a culinary dish. Thus, this concept can be considered as an object with certain properties, such as the necessary ingredients and time. Alternatively, it can be considered as a technological process, which has some initial data, goes through a series of steps that must be performed, and leads to certain results. The recipe also includes the time it takes to complete the steps and a description of the necessary utensils.
Since 2006, in order to develop an optimal formalism for expressing the complex semantics of natural language, the Knowdy project has been created that focuses on the management of graph data.
Knowdy is a software project with an open source research team of linguists in St. Petersburg that develops a super-fast database of graphs that allows you to work directly and effectively with conceptual graphs, bypassing any intermediate representations, such as SQL tables. The database engine is implemented in C and can be used both as a network service and as a standalone library for the embedded environment.
After several years of research and development, the development team came up with a special data format for Knowdy DB, called GSL (acronym General Semantics Language). GSL is optimized for compact storage of concept graphs. It is used to store data, send messages and exchange information. This format is not overly verbose, like XML, and a bit more compact than JSON. The language takes on some of the features of Lisp S-expressions, but with a lot of semantic modification, since you need to keep in mind that columns are not lists. The GSL description has a special meaning that allows users to express not only multi-level groupings, but also CRUD operations within the database storage system.
In the GSL description, the process is encoded as a first-class function, which can be called or can be anonymous, supports inheritance from the base function, has arguments and subprocesses that can work in parallel. The processes below describe some of the logic of the same recipe for Russian mushroom soup.
R4S uses the third approach. It allows you to submit a recipe with the necessary degree of formalism, to describe both the essence and the action. The recipe recorded in this way in the database can be quickly loaded and applied by the technique. In devices of our development, where recipes are supported, they are recorded in GSL format. The plans are to train the neural network to record recipes of the peoples of the world, which will simplify the exchange of recipes and will provide a technological reserve for high-grade automated kitchens.
We often encounter seemingly trivial tasks, the implementation of which is not so simple. In your opinion, which language can be used to write recipes, on the one hand, accessible to users from all over the world, and on the other, understood and interpreted by machines?
Automated kitchens. Necessary stages
Smart kitchen appliances facilitate the cooking process, but people still do most of the work: washing, cutting and putting food into them, mixing, etc. And how I would like to just say: “Pot, cook”, and that it all washed itself, cut, mixed, and the user only needs to wait for the beep and go and eat.
Now automatic kitchens only appear on an industrial scale, but it will not come very soon to the household level. To do this, you need to solve several groups of tasks, and these solutions should be available.
The first group of tasks is to give the kitchen the opportunity to act autonomously from the user.
This task, for example, can be solved by introducing a manipulator system , a conveyor into the kitchen;
')
The second group of tasks is to introduce computer vision and a sensor system in the kitchen to automatically determine the degree of readiness of a dish or its part.
The third group of tasks is the management interface. The task can be implemented through the application on smartphones, as well as voice assistants.
And finally, the fourth group - the task of the kitchen logic of behavior. How to teach the kitchen to cook? For example, a robot cook teaches a person - a professional cook. Wearing sensors on his hands, he slowly shows how to. The machine then reproduces its actions, down to the smallest detail, right up to the shaking hands. Of course, during processing, extra movements must be removed, but if the programmer misses something, the robot cook will one day stop to scratch his non-existent nose.
In addition, in order to teach the robot one recipe, the movement must be repeated several times. Even if we teach the robot to separate movements that it can then combine, then the time for such training will be extremely long. Not to mention the strictest compliance with the recipe and, apparently, calibrated ingredients: allowing the robot to pick out potato eyes is a separate complex engineering task. And after cooking, you still have to wash the stove and dishes.
The recipe is code
If you go back to the situation now - i.e. to take into account the economic component of the issue, the difficulties of development, etc., it becomes clear that in the foreseeable future we can only teach smart home appliances to read the recipe and use it. To do this, convert the recipe into a code. But classic software Java or C is hardly suitable here - they can describe a recipe, but if the user wants to make any changes to it, then he will have to either contact tech support or sign up for programming courses. Besides, the classic code is too formal. For example, if the ingredients say “dried mushrooms”, then fresh ones will not go into cooking. Further complicates the task of the fact that the recipes of national cuisines are presented in different natural languages.
And I would like to give the user the opportunity to find an interesting recipe on the Web, download it, and tell the robot kitchen to correctly interpret it and apply it. The maximum task is the ability of the kitchen itself, based on the culinary preferences of the user, to search and offer options. Good cooking recipes should not only include a list of raw ingredients and cooking methods, but also the environmental conditions in which products grow. The dish belongs to the national cuisine according to many criteria, including specific cooking methods, used cookware, tools, etc. Transforming all this data into the proper digital form or, most likely, its interpretation will be a difficult task.
So which languages ​​may be suitable for recipes?
Rdf
Existing technologies of formalized knowledge representation fall into several groups of structures.
These include the Semantic Web approach with its OWL and RDF technologies. Online collections of related data sets using the Semantic Web tools are also known as Linked Data. Recently an article devoted to this concept appeared on Habré, so we will not focus on it. Many digitization projects are built around the idea of ​​using some higher ontology , which can be expanded by engineers in a particular area of ​​expertise.
Consider the case of the use and application of these tools for digitizing national cuisines. The main idea of ​​RDF is that the Internet is moving away from storing information that is perceived only by man, becoming a worldwide network of interacting processes. As the name implies, RDF is the basis for expressing information about resources. First of all - about web documents and various organizations. Its formalism is based on the idea of ​​static classes and properties. The question arises: how reasonable it is to treat a recipe as an entity, and not as a complex process with arguments, timing, incoming subprocesses, etc. For the recipe this will not be enough.
Schema.org
The second approach can be considered the initiative of Schema.org - joint activities of the Internet community to create, maintain and promote schemes for structuring data on the Internet. This initiative aims to provide a standardized dictionary for common metadata published on web resources. Cooking-related web resources can use “Recipe” class field metadata stored by reference . Below is an example of a recipe code provided by Google:
<script type="application/ld+json"> { "@context": "<a href=" http: //schema.org/">http://schema.org/</a>", "@type": "Recipe", "name": "Strawberry-Mango Mesclun Recipe", "image": [ "<a href=" https: //example.com/photos/1x1/photo.jpg">https://example.com/photos/1x1/photo.jpg</a>" ], "author": { "@type": "Person", "name": "scoopnana" }, "datePublished": "2008-03-03", "description": "Mango, strawberries, and sweetened dried cranberries are a vibrant addition to mixed greens tossed with an oil and balsamic vinegar dressing. ", "aggregateRating": { "@type": "AggregateRating", "ratingValue": "5", "reviewCount": "52" }, "prepTime": "PT15M", "totalTime": "PT14M", "recipeYield": "12 servings", "nutrition": { "@type": "NutritionInformation", "servingSize": "1 bowl", "calories": "319 cal", "fatContent": "20.2 g" }, "recipeIngredient": [ "1/2 cup sugar", "3/4 cup canola oil", "1 teaspoon salt", "1/4 cup balsamic vinegar", "8 cups mixed salad greens", "2 cups sweetened dried cranberries", "1/2 pound fresh strawberries, quartered", "1 mango — peeled, seeded, and cubed", "1/2 cup chopped onion", "1 cup slivered almonds" ], "recipeInstructions": "\n1. Place the sugar, oil, salt, and vinegar in a jar with a lid.Seal jar, and shake vigorously to mix.\n2.In a large bowl, mix salad greens, sweetened dried cranberries, strawberries, mango, and onion.To serve, toss with dressing and sprinkle with almonds. " } </script> The dictionary and format adopted by Schema.org are primarily focused on the presentation of high-level metadata in web documents. However, a properly normalized semantic graph requires a much more explicit representation of concepts. Most of the string values ​​in Schema.org fields are natural texts that require human cognitive interpretation. Such text cannot be directly read by digital systems without special natural language processing tools, which are often error prone. As an illustration, Google’s machine translation will fit a single line item in the list of ingredients in the above recipe example:
'fresh strawberries, quartered' Translation into Russian gives a completely wrong meaning:
' , ' with “quartered”, used in the sense of “housing”, as in “Our troops were stationed in Boston”, and not “Cut into four parts”.
The absence of clearly defined object roles and the explicit identification of methods makes the task of machine translation of a recipe much more difficult.
For our futuristic task for digital recipes - a scenario in which a robotic machine could follow the instructions for preparing a dish, a simple line of text cannot be a solution.
Consider a typical ingredient description:
'8 Granny Smith apples — peeled, cored and sliced' It is clear that this text line contains a lot of classifying information: raw ingredient as a class, a specific sort of apples, number in pieces, a list of methods that should be applied to each part in order to use the ingredient normally: removing seeds, peel, cutting an object into pieces forms.
The formalism of Schema.org is not expressive enough to write the cooking process in some kind of program code. To properly digitize recipes for robotic cooking, we need to separate the ingredient description from the logic
The third approach is cognitive recipe modeling.
The traditional way to create a recipe is to start with the components and operations that need to be performed. This order and style of description is known as imperative or procedural. The declarative or functional style of describing the logic of a process usually starts from the top of the execution pyramid, the expected useful result we want to achieve.
Consider the following simplified view of the classic Russian recipe for mushroom soup. Arrows denote subprocesses (sometimes alternative) necessary for the parent process.
Cognitive interpretation can be a dead end for the moment when we try to define a recipe. The term “recipe” has several contextual meanings. It can be defined in a general sense as a method of obtaining the desired result. When used in the context of cooking, it means a set of instructions for cooking a culinary dish. Thus, this concept can be considered as an object with certain properties, such as the necessary ingredients and time. Alternatively, it can be considered as a technological process, which has some initial data, goes through a series of steps that must be performed, and leads to certain results. The recipe also includes the time it takes to complete the steps and a description of the necessary utensils.
Since 2006, in order to develop an optimal formalism for expressing the complex semantics of natural language, the Knowdy project has been created that focuses on the management of graph data.
Project knowdy
Knowdy is a software project with an open source research team of linguists in St. Petersburg that develops a super-fast database of graphs that allows you to work directly and effectively with conceptual graphs, bypassing any intermediate representations, such as SQL tables. The database engine is implemented in C and can be used both as a network service and as a standalone library for the embedded environment.
After several years of research and development, the development team came up with a special data format for Knowdy DB, called GSL (acronym General Semantics Language). GSL is optimized for compact storage of concept graphs. It is used to store data, send messages and exchange information. This format is not overly verbose, like XML, and a bit more compact than JSON. The language takes on some of the features of Lisp S-expressions, but with a lot of semantic modification, since you need to keep in mind that columns are not lists. The GSL description has a special meaning that allows users to express not only multi-level groupings, but also CRUD operations within the database storage system.
In the GSL description, the process is encoded as a first-class function, which can be called or can be anonymous, supports inheritance from the base function, has arguments and subprocesses that can work in parallel. The processes below describe some of the logic of the same recipe for Russian mushroom soup.
{!proc prepare mushroom soup mix [_gloss {ru }] {is cooking by boiling} {arg cut-mushrooms {do prepare mushroom mix}} {arg cut-potatoes {do prepare potato mix}} {arg cut-onions {do prepare onion mix}} {do _put [_gloss {ru .}] {obj _all} {target-loc container}}} {!proc prepare mushroom mix [_gloss {ru }] {arg clean-mushrooms {do clean mushrooms}} {do _cut [_gloss {ru .}] {obj clean-mushrooms} {form slice {size 1.5 {unit cm}}}}} R4S uses the third approach. It allows you to submit a recipe with the necessary degree of formalism, to describe both the essence and the action. The recipe recorded in this way in the database can be quickly loaded and applied by the technique. In devices of our development, where recipes are supported, they are recorded in GSL format. The plans are to train the neural network to record recipes of the peoples of the world, which will simplify the exchange of recipes and will provide a technological reserve for high-grade automated kitchens.
We often encounter seemingly trivial tasks, the implementation of which is not so simple. In your opinion, which language can be used to write recipes, on the one hand, accessible to users from all over the world, and on the other, understood and interpreted by machines?
Source: https://habr.com/ru/post/425023/
All Articles