Artificial Intelligence in the real world
The development of complex artificial intelligence systems in the laboratory is a difficult task. But what happens when such systems enter the real world and begin to interact with real people? This is what researchers are trying to figure out, including Dr. Fernando Diaz, senior research manager at Microsoft Research Montreal. Today, Dr. Diaz will share his thoughts and answer our questions about artificial intelligence and its impact on society.
Together with colleagues, Fernando is trying to understand how AI systems affect society, getting into the real world, and how to cope with the consequences.
')
He will also talk about how technology can shape musical tastes, and explain why it is today so important to teach students who are educated in computer science, not only algorithms, but also ethical principles.
When I run an experiment, I ask myself: which experiments on users are considered ethical, and which ones are not? How widely do you need to inform people that they have become participants in the experiment? How to recognize and eliminate the bias of data used by machine learning technologies? This comes to mind first. In the following years, many other questions will arise, such as how to develop AI systems in such a way as to show respect for users.
You are listening to the Microsoft Research podcast. Here we introduce you to cutting-edge technology research and the scientists behind it. With you lead Gretchen Huizinga (Gretchen Huizinga).
The development of complex artificial intelligence systems in the laboratory is a difficult task. But what happens when such systems get into the real world and start interacting with real people? This is exactly what researchers like Dr. Fernando Diaz, senior research manager at Microsoft Research Montreal, are trying to figure out. Together with colleagues, Fernando is trying to understand how AI systems affect society, getting into the real world, and how to cope with the consequences.
Today, Dr. Diaz will share his thoughts and answer our questions about artificial intelligence and its impact on society. He will also talk about how technology can shape musical tastes, and explain why it is today so important to teach students who are educated in computer science, not only algorithms, but also ethical principles. This and much more is in the new release of the Microsoft Research podcast.
Fernando Diaz, welcome to the podcast.
Thank.
You are a senior research manager at Microsoft Research Montreal and are working in the field of artificial intelligence, search, and information. But you also learn the principles of fairness, accountability, transparency, and ethics (Fairness, Accountability, Transparency and Ethics, FATE). That is, if we speak in general (we will move on to the specifics later): what makes you wake up in the morning? What serious questions are you looking for answers to, what important problems do you want to solve?
Many of the systems we build are highly successful. Information search, web search, computer vision - all these technologies have been developed for many years. Today they are actively filling the mass market, and people are starting to use them every day. However, some IT professionals have not thought through how to design these technologies: in what social context they will be used.
And in this case, I'm just trying to understand what social prerequisites existed for the creation of these systems, how the social context in which they work affects not only our indicators, for example, the accuracy and completeness of the returned data, but also society as a whole. . It seems to me that this issue is coming to the fore for IT professionals, since many of these technologies, developed in isolation, are only now beginning to enter the market.
So, you are an IT specialist, you have investigated the algorithms for obtaining information, machine learning and statistical methods. Recently, however, you are interested in the interaction of artificial intelligence technologies with society, in particular the consequences of their wide distribution or, as some say, their release. Why are you interested right now? What worries you? What spurred your interest in this area?
Great question. At first I, of course, enrolled in the magistracy, received a degree. I studied all these systems, so to speak, on an abstract level, experimenting with static data obtained offline. Shortly after graduation, I got to the industrial research laboratory. Here I worked together with production workers, we were engaged in the practical implementation of technologies that I studied at the university.
And then I began to understand: when we scale these algorithms and provide them to real users, most of the basic assumptions put forward in the laboratory are completely inapplicable in reality. For me, this was a kind of final test of all my research, a return to the basic principles and an attempt to understand what the problem is, how I can accurately assess the results and achieve specific indicators.
That is, you have already worked in Microsoft Research, then you left, and then returned again. You started in New York, and now you have moved to Montreal. Why are you back?
After university, I began research work in Montreal and for a number of reasons I was forced to leave there. But living there, I realized that in this city - as in Canada as a whole - the tradition of research in the field of IT and machine learning is quite strong. And in my heart I really always wanted to come back here to participate in this work. And when I had the opportunity to return to the Microsoft Research lab in Montreal, I gladly took it. Especially when you consider that the laboratory is fully engaged in developments in the field of artificial intelligence. In Montreal, there have been very active research in this area, and I wanted to be a part of all this, to make my own contribution.
Let's say a few words about Montreal. This city has become a real Mecca in all that relates to artificial intelligence, and the SMR Montreal laboratory has a very specific task - to teach cars to read, think and communicate in the same way as people do. Tell us how far you have come along and how your own research correlates with the work of the Montreal laboratory.
I think a special AI research lab was created because a lot of questions arose regarding the development of such systems, and they haven’t yet been answered. And I think that this requires not only specialists in the processing of natural languages, not only experts in dialogic training or in stimulating training. In fact, they should all work closely together. And it seems to me that this is what makes our laboratory truly unique.
My task today is to come to the laboratory, talk to specialists as much as possible and tell them how these systems can behave when real people start interacting with them. As I said earlier, such systems are quite easy to develop in isolation from reality, in isolation. But when their practical implementation begins, it turns out that too many assumptions were made during the experiments. Now I am forming a team whose task is to anticipate the emergence of such questions, optimize the development of the system, increase its stability in conditions, say, differences between the population groups with whom we interact, or variations within the knowledge base from which I draw information.
What team do you want to form?
I'm trying to create a kind of "sister" of the FATE group, which we organized in New York a few years ago. We will focus on the social consequences of integrating artificial intelligence into society. Our team will include experts not only in IT, but also in related disciplines, such as sociology. For IT professionals to better understand the implications for society, we need sociological experts, anthropologists, and so on. They will be able to tell a lot of useful things about such things that we have not yet evaluated or taken into account.
Yes, let's talk about this in more detail. The application of FATE principles in conducting various studies in the field of artificial intelligence and machine learning today is of paramount importance. As you have said, the reason is that not all controversial issues can be properly studied in the laboratory. Along with the planned result, completely unexpected, shocking consequences may arise. So, the researchers of this community have different specialization and education. What is the contribution of each specialist when it comes to the principles of honesty, accountability, transparency and ethics?
Yes, sociologists have a much better understanding of various aspects of the application of technology in general, they know about the possible positive and negative consequences, about how people react to certain tools offered by us. Specialists with a law degree will be able to comment on the political background of individual technologies being developed and will help us to better understand the concept of “honesty,” for example.
IT professionals, in turn, better understand the essence of the systems being developed, are able to translate concepts such as “honesty” into a viable concept and incorporate it into the system. However, the presence of very different points of view on the same problem is simply necessary in order to design systems more efficiently.
Let's go back to what you did in the past and what you continue to work on now: information access systems, search engines and information retrieval. In the document you have compiled, you speak about the existence of a certain gap between the study of such systems and their practical implementation, but at the same time you make a provocative statement that employees of educational institutions will cope better with some problems than industry technical specialists. What are these problems and why do you think so?
Let's look at the situation in the field of access to information research. There are scientific employees of educational institutions who have done a lot for the good of society, but today many studies, say in the field of web search, are carried out by large search giants who have data, information about users, and so on. And in most cases, researchers do not have access to such data, a platform for conducting experiments. Therefore, the research possibilities are clearly unequal.
And in my article, I wrote that research officers of educational institutions do not have a large amount of data, but they have the opportunity to attract diverse specialists, which the search giants cannot do. The university has sociologists and experts in other sciences. There are teachers of economic disciplines. All these potential “accomplices” of research will help to look at the problem more broadly, to study it from different points of view, instead of resting on one single one, which some search giant sticks to.
I think that generating data sets is just one of the strategies. Another approach, or type of scientific platform, not available for educational institutions, is experiments. I can conduct A / B tests. You can set up controlled experiments involving a large sample of users that is not available in the data set.
Yes it's true.
And yet, it seems to me, it is worth exploring how, in fact, we provide educational institutions with access to their resources for conducting such controlled experiments.
Interesting.
All this happened randomly, haphazardly, and it seems to me that this is what we, the industry researchers, need to think about: how to make access to such features more simple and convenient.
So, let's go back to the data. Let's say a few words about them. Specialists who were engaged in machine learning agreed that it is not enough to have just “big data” - I specifically say “big data” in quotes. Among other things, high-quality and objective data are required. We know that all these big data sets lack objectivity to one degree or another.
And we need to fix it. Today, there is a lot of talk about how to increase the objectivity of data through, for example, search engine audits, equality algorithms and the like. How to do it?
One of the reasons for our concern about data bias: a model trained on the basis of this data will be biased when deployed. That is, the most important thing is to be able to determine how objectively artificial intelligence acts. And if he acts biased, you need to go back and retrain the algorithm or add restrictions to it that will not allow you to adopt bias from the data. My work today is mainly focused on evaluation and measurement.
We want to understand the users accessing the system, understand what they need, and evaluate whether the system is objectively or non-objectively taking into account who these users are and which population group they belong to. This requires a wealth of experience, accumulated by specialists in obtaining information, who, since the beginning of such research in the 50s of the 20th century, managed to think through all the estimation and measurement algorithms. This is what makes it possible to find a natural balance between auditing, evaluating and obtaining information.
As we have said, non-objectivity is something like a buzzword among researchers in the field of data processing and analysis and artificial intelligence. However, you say that in addition to bias there are other social problems that need to be addressed. What are these problems, and how can research help solve them?
Yes, I really think non-objectivity is a very important issue, but why did I even start talking about the social context of using artificial intelligence? Because I believe that bias is just one of the social problems that we can identify. Of course there are others. Obviously, one of them is related to transparency. How can I make the decisions made by the algorithm transparent for users, make them feel that they can control the situation, participate in the work of the algorithm?
The second problem is the cultural background of the algorithms. This all happens in the context of, say, the selection of recommendations of films or music. For example, I create a great system for selecting music recommendations for users. What will be the consequences of the deployment of this algorithm for culture, if I know, for example, that by adding individual performers to a recommendation, you can form someone’s musical tastes in a certain way? What will it mean to create or preserve culture in the long run?
There is another aspect of this problem: the algorithms for selecting music recommendations can have a significant impact on the authors or performers themselves. As an IT specialist, I can say that this is the best algorithm for selecting music recommendations. And I'm going to bring it to the market. But we, IT professionals, did not even think about how this algorithm will affect the authors. For me personally, this is especially important.
How, then, are you going to do research that will take all this into account?
Let's go back to the example with a selection of music recommendations. Imagine that you communicate closely with musicians and understand how important this is for them. How will they know that they are excluded from the system? How does it feel that their life is governed by a system of selection of recommendations, and they absolutely can not affect it? As an IT specialist, I just need to sit at the table with sociologists and anthropologists, experts in mass media, in order to better understand what such a significant group of the population is like musicians.
And then I, an IT specialist, will be able to sit down and think about how to create an algorithm that will satisfy the needs of both listeners and musicians. Now it seems to me that such a formulation sounds too simplistic. That is why I want a specialist in other disciplines to tell me: “Fernando, you know, you didn’t think about this, about that, or about that.”
Given the nature of your research and the results obtained, can you say that we have something to worry about? Are there things that keep you awake?
Personally, I am concerned about the fact that many of the technologies developed in our research community go out into the world, go into mass circulation in a matter of days or weeks. And this is done by people who themselves have not conducted any experiments. There is nothing terrible about open science, but I think we should learn more and understand better the implications of using algorithms before introducing them somewhere. And it also worries me that we are releasing very quickly more and more new algorithms, not fully appreciating and understanding their consequences.
Laboratory Microsoft Research is famous for its close cooperation with educational institutions. And I know that you pay great attention to education. Let's talk about what happens with education in terms of the principles of FATE - honesty, accountability, transparency and ethics. Tell us how you see the future of educational programs in this area.
You know, when I entered the magistracy ... or even earlier, while studying at the IT department, we were practically not taught the principles of ethics and did not talk about the implications for society of the technologies that we are developing. , , , . , . , .
-, , , , , . , , . , - , . - , , . , , , .
— — . , , , , .
: « , : , — ? , ? , ?». . , , .
, , « ». , , , , . , , . , , . , .
Yes. , , , , , . , , -. . , , .
, : ? : , , , , . , ?
, , … , , . , , . , , , , .
, , . , , , , , — . .
, , , (FATE) IT, . - .
12 AI-Community « ». .
-, , , « » .
, !
check in
Together with colleagues, Fernando is trying to understand how AI systems affect society, getting into the real world, and how to cope with the consequences.
')
He will also talk about how technology can shape musical tastes, and explain why it is today so important to teach students who are educated in computer science, not only algorithms, but also ethical principles.
Interview
When I run an experiment, I ask myself: which experiments on users are considered ethical, and which ones are not? How widely do you need to inform people that they have become participants in the experiment? How to recognize and eliminate the bias of data used by machine learning technologies? This comes to mind first. In the following years, many other questions will arise, such as how to develop AI systems in such a way as to show respect for users.
You are listening to the Microsoft Research podcast. Here we introduce you to cutting-edge technology research and the scientists behind it. With you lead Gretchen Huizinga (Gretchen Huizinga).
The development of complex artificial intelligence systems in the laboratory is a difficult task. But what happens when such systems get into the real world and start interacting with real people? This is exactly what researchers like Dr. Fernando Diaz, senior research manager at Microsoft Research Montreal, are trying to figure out. Together with colleagues, Fernando is trying to understand how AI systems affect society, getting into the real world, and how to cope with the consequences.
Today, Dr. Diaz will share his thoughts and answer our questions about artificial intelligence and its impact on society. He will also talk about how technology can shape musical tastes, and explain why it is today so important to teach students who are educated in computer science, not only algorithms, but also ethical principles. This and much more is in the new release of the Microsoft Research podcast.
Fernando Diaz, welcome to the podcast.
Thank.
You are a senior research manager at Microsoft Research Montreal and are working in the field of artificial intelligence, search, and information. But you also learn the principles of fairness, accountability, transparency, and ethics (Fairness, Accountability, Transparency and Ethics, FATE). That is, if we speak in general (we will move on to the specifics later): what makes you wake up in the morning? What serious questions are you looking for answers to, what important problems do you want to solve?
Many of the systems we build are highly successful. Information search, web search, computer vision - all these technologies have been developed for many years. Today they are actively filling the mass market, and people are starting to use them every day. However, some IT professionals have not thought through how to design these technologies: in what social context they will be used.
And in this case, I'm just trying to understand what social prerequisites existed for the creation of these systems, how the social context in which they work affects not only our indicators, for example, the accuracy and completeness of the returned data, but also society as a whole. . It seems to me that this issue is coming to the fore for IT professionals, since many of these technologies, developed in isolation, are only now beginning to enter the market.
So, you are an IT specialist, you have investigated the algorithms for obtaining information, machine learning and statistical methods. Recently, however, you are interested in the interaction of artificial intelligence technologies with society, in particular the consequences of their wide distribution or, as some say, their release. Why are you interested right now? What worries you? What spurred your interest in this area?
Great question. At first I, of course, enrolled in the magistracy, received a degree. I studied all these systems, so to speak, on an abstract level, experimenting with static data obtained offline. Shortly after graduation, I got to the industrial research laboratory. Here I worked together with production workers, we were engaged in the practical implementation of technologies that I studied at the university.
And then I began to understand: when we scale these algorithms and provide them to real users, most of the basic assumptions put forward in the laboratory are completely inapplicable in reality. For me, this was a kind of final test of all my research, a return to the basic principles and an attempt to understand what the problem is, how I can accurately assess the results and achieve specific indicators.
That is, you have already worked in Microsoft Research, then you left, and then returned again. You started in New York, and now you have moved to Montreal. Why are you back?
After university, I began research work in Montreal and for a number of reasons I was forced to leave there. But living there, I realized that in this city - as in Canada as a whole - the tradition of research in the field of IT and machine learning is quite strong. And in my heart I really always wanted to come back here to participate in this work. And when I had the opportunity to return to the Microsoft Research lab in Montreal, I gladly took it. Especially when you consider that the laboratory is fully engaged in developments in the field of artificial intelligence. In Montreal, there have been very active research in this area, and I wanted to be a part of all this, to make my own contribution.
Let's say a few words about Montreal. This city has become a real Mecca in all that relates to artificial intelligence, and the SMR Montreal laboratory has a very specific task - to teach cars to read, think and communicate in the same way as people do. Tell us how far you have come along and how your own research correlates with the work of the Montreal laboratory.
I think a special AI research lab was created because a lot of questions arose regarding the development of such systems, and they haven’t yet been answered. And I think that this requires not only specialists in the processing of natural languages, not only experts in dialogic training or in stimulating training. In fact, they should all work closely together. And it seems to me that this is what makes our laboratory truly unique.
My task today is to come to the laboratory, talk to specialists as much as possible and tell them how these systems can behave when real people start interacting with them. As I said earlier, such systems are quite easy to develop in isolation from reality, in isolation. But when their practical implementation begins, it turns out that too many assumptions were made during the experiments. Now I am forming a team whose task is to anticipate the emergence of such questions, optimize the development of the system, increase its stability in conditions, say, differences between the population groups with whom we interact, or variations within the knowledge base from which I draw information.
What team do you want to form?
I'm trying to create a kind of "sister" of the FATE group, which we organized in New York a few years ago. We will focus on the social consequences of integrating artificial intelligence into society. Our team will include experts not only in IT, but also in related disciplines, such as sociology. For IT professionals to better understand the implications for society, we need sociological experts, anthropologists, and so on. They will be able to tell a lot of useful things about such things that we have not yet evaluated or taken into account.
Yes, let's talk about this in more detail. The application of FATE principles in conducting various studies in the field of artificial intelligence and machine learning today is of paramount importance. As you have said, the reason is that not all controversial issues can be properly studied in the laboratory. Along with the planned result, completely unexpected, shocking consequences may arise. So, the researchers of this community have different specialization and education. What is the contribution of each specialist when it comes to the principles of honesty, accountability, transparency and ethics?
Yes, sociologists have a much better understanding of various aspects of the application of technology in general, they know about the possible positive and negative consequences, about how people react to certain tools offered by us. Specialists with a law degree will be able to comment on the political background of individual technologies being developed and will help us to better understand the concept of “honesty,” for example.
IT professionals, in turn, better understand the essence of the systems being developed, are able to translate concepts such as “honesty” into a viable concept and incorporate it into the system. However, the presence of very different points of view on the same problem is simply necessary in order to design systems more efficiently.
Let's go back to what you did in the past and what you continue to work on now: information access systems, search engines and information retrieval. In the document you have compiled, you speak about the existence of a certain gap between the study of such systems and their practical implementation, but at the same time you make a provocative statement that employees of educational institutions will cope better with some problems than industry technical specialists. What are these problems and why do you think so?
Let's look at the situation in the field of access to information research. There are scientific employees of educational institutions who have done a lot for the good of society, but today many studies, say in the field of web search, are carried out by large search giants who have data, information about users, and so on. And in most cases, researchers do not have access to such data, a platform for conducting experiments. Therefore, the research possibilities are clearly unequal.
And in my article, I wrote that research officers of educational institutions do not have a large amount of data, but they have the opportunity to attract diverse specialists, which the search giants cannot do. The university has sociologists and experts in other sciences. There are teachers of economic disciplines. All these potential “accomplices” of research will help to look at the problem more broadly, to study it from different points of view, instead of resting on one single one, which some search giant sticks to.
I think that generating data sets is just one of the strategies. Another approach, or type of scientific platform, not available for educational institutions, is experiments. I can conduct A / B tests. You can set up controlled experiments involving a large sample of users that is not available in the data set.
Yes it's true.
And yet, it seems to me, it is worth exploring how, in fact, we provide educational institutions with access to their resources for conducting such controlled experiments.
Interesting.
All this happened randomly, haphazardly, and it seems to me that this is what we, the industry researchers, need to think about: how to make access to such features more simple and convenient.
So, let's go back to the data. Let's say a few words about them. Specialists who were engaged in machine learning agreed that it is not enough to have just “big data” - I specifically say “big data” in quotes. Among other things, high-quality and objective data are required. We know that all these big data sets lack objectivity to one degree or another.
And we need to fix it. Today, there is a lot of talk about how to increase the objectivity of data through, for example, search engine audits, equality algorithms and the like. How to do it?
One of the reasons for our concern about data bias: a model trained on the basis of this data will be biased when deployed. That is, the most important thing is to be able to determine how objectively artificial intelligence acts. And if he acts biased, you need to go back and retrain the algorithm or add restrictions to it that will not allow you to adopt bias from the data. My work today is mainly focused on evaluation and measurement.
We want to understand the users accessing the system, understand what they need, and evaluate whether the system is objectively or non-objectively taking into account who these users are and which population group they belong to. This requires a wealth of experience, accumulated by specialists in obtaining information, who, since the beginning of such research in the 50s of the 20th century, managed to think through all the estimation and measurement algorithms. This is what makes it possible to find a natural balance between auditing, evaluating and obtaining information.
As we have said, non-objectivity is something like a buzzword among researchers in the field of data processing and analysis and artificial intelligence. However, you say that in addition to bias there are other social problems that need to be addressed. What are these problems, and how can research help solve them?
Yes, I really think non-objectivity is a very important issue, but why did I even start talking about the social context of using artificial intelligence? Because I believe that bias is just one of the social problems that we can identify. Of course there are others. Obviously, one of them is related to transparency. How can I make the decisions made by the algorithm transparent for users, make them feel that they can control the situation, participate in the work of the algorithm?
The second problem is the cultural background of the algorithms. This all happens in the context of, say, the selection of recommendations of films or music. For example, I create a great system for selecting music recommendations for users. What will be the consequences of the deployment of this algorithm for culture, if I know, for example, that by adding individual performers to a recommendation, you can form someone’s musical tastes in a certain way? What will it mean to create or preserve culture in the long run?
There is another aspect of this problem: the algorithms for selecting music recommendations can have a significant impact on the authors or performers themselves. As an IT specialist, I can say that this is the best algorithm for selecting music recommendations. And I'm going to bring it to the market. But we, IT professionals, did not even think about how this algorithm will affect the authors. For me personally, this is especially important.
How, then, are you going to do research that will take all this into account?
Let's go back to the example with a selection of music recommendations. Imagine that you communicate closely with musicians and understand how important this is for them. How will they know that they are excluded from the system? How does it feel that their life is governed by a system of selection of recommendations, and they absolutely can not affect it? As an IT specialist, I just need to sit at the table with sociologists and anthropologists, experts in mass media, in order to better understand what such a significant group of the population is like musicians.
And then I, an IT specialist, will be able to sit down and think about how to create an algorithm that will satisfy the needs of both listeners and musicians. Now it seems to me that such a formulation sounds too simplistic. That is why I want a specialist in other disciplines to tell me: “Fernando, you know, you didn’t think about this, about that, or about that.”
Given the nature of your research and the results obtained, can you say that we have something to worry about? Are there things that keep you awake?
Personally, I am concerned about the fact that many of the technologies developed in our research community go out into the world, go into mass circulation in a matter of days or weeks. And this is done by people who themselves have not conducted any experiments. There is nothing terrible about open science, but I think we should learn more and understand better the implications of using algorithms before introducing them somewhere. And it also worries me that we are releasing very quickly more and more new algorithms, not fully appreciating and understanding their consequences.
Laboratory Microsoft Research is famous for its close cooperation with educational institutions. And I know that you pay great attention to education. Let's talk about what happens with education in terms of the principles of FATE - honesty, accountability, transparency and ethics. Tell us how you see the future of educational programs in this area.
You know, when I entered the magistracy ... or even earlier, while studying at the IT department, we were practically not taught the principles of ethics and did not talk about the implications for society of the technologies that we are developing. , , , . , . , .
-, , , , , . , , . , - , . - , , . , , , .
— — . , , , , .
: « , : , — ? , ? , ?». . , , .
, , « ». , , , , . , , . , , . , .
Yes. , , , , , . , , -. . , , .
, : ? : , , , , . , ?
, , … , , . , , . , , , , .
, , . , , , , , — . .
, , , (FATE) IT, . - .
12 AI-Community « ». .
-, , , « » .
, !
check in
Source: https://habr.com/ru/post/423233/
All Articles