Future spyware analyst - you

Crowdsourcing It can be used everywhere: on election races, to create a super-encyclopedia , to develop tools for better evaluation of films for the Netflix service. And American spy agencies hope that crowdsourcing will help them predict the next most important events in the world. New sharp-eyed Washington intelligence analyst can become - you! Initial testing of the system will begin this week.

On Friday, Applied Research Associates, Inc. will launch Aggregative Contingent Estimation System (ACES ) , a web site that will allow members of the public to try crowdsourcing for intelligence prediction. Funded by the IARPA organization, a forge of advanced intelligence community development, ACES offers users to try their hand at forecasting, as well as honing the existing forecasting skills. As ARA and IARPA hope, the information obtained will help to determine whether the “crowd” can become a suitable magic ball of predictions for the world of intelligence. Through the site, the project will check whether it is possible to apply car-outsourcing forecasting for the intelligence community.

“We try to use the individual opinion of each for the benefit of each other and try to determine which aspects will be most important and lead to more accurate predictions.” - says Dr. Dirk Varnaar, lead ARA researcher for the ACES project.

The idea behind “collective intelligence” is simple, and boils down to a specific task: there is a certain amount of useful information distributed among representatives of various groups of people, therefore the collection of which will bring more accurate answers, even more precisely, experts.
')
But ARA is going to offer something slightly different from ordinary crowdsourcing. While in many similar systems, the opinions of all participants are equally valued, ACES will , over time, look for representatives who make more accurate forecasts, and evaluating their opinions more weighty.

The tools that resort to the ingenuity of the “crowd” to solve complex problems show prospects for many years. Researchers used rumors on Twitter to fairly accurately determine the box office for new films. Tools like Ushaidi , non-profit software that allows users to create maps of incidents, resources, people who, in the midst of a crisis, allowed rescuers to find victims of disasters, such as the earthquake in Haiti .

ACES is a bit like a regular online survey. Users will be interviewed about what events in politics, economy, science, society and security can occur, and why they suggest this. Then their answers are grouped, and it is determined which group is more accurate in the forecasts. This is not a prediction market such as the famous InTrade , which allows users to bet on anything they can think about. Commercial goals are not pursued - only opinions are important.

Success for ACES will take place if we manage to attract various groups of people and support their participation in the system. But online is home to a large number of crowdsourcing tools and prediction exchanges, creating fierce competition for the attention of an independent predictor. In addition to InTrade, there are many similar projects. The Iowa Electronic Exchange, developed by the University of Iowa (The University of Iowa's Iowa Electronic Market) , since about 1988, has allowed users to bid on presidential elections or on Federal Reserve Policy. Cinema-goers use the Hollywood Stock Exchange website to bet on the box office for new films or other movie-related events.

Dirk Varnaar says that he hopes that some of the ACES functions created to research analytical skills for the intelligence community will exert the interest of ordinary users - “We think that people will compete with each other, and perhaps learn how to become better at prediction. "

To achieve this, ARA is working on a tool designed to help analysts calibrate their estimates of plausible forecasts. For example, ACES will be able to show that the probability of an action predicted by you is 60%, but ultimately, through the site, you will find out that the probability will tend to 40%. The site will be the key to your confidence, thus providing a feedback between your predictions and the ability to make subsequent adjustments.

Other features planned for the site will add a social dimension to the forecasting process. There are some options for identifying facts that indicate the exchange of information between project participants, which can spoil the accuracy of forecasts and turn the “wise crowd” into a “stupid crowd”. Varnaar also mentioned that the jury will not have the effect of social influence - “It is not yet clear whether cooperation among the participants will be beneficial or harmful.” In any case, he plans to find out what methods ACES can use to jointly predict solutions to complex problems.

Also, as with math problems, ACES wants you to provide a point after your prediction to prove its probability. ARA has yet to establish how participants can specifically develop and work on their forecasts. But some approaches may lead to the fact that some participants will make predictions, just in order to vote for them.

“We believe that there is a way of allowing us to basically get the arguments of [a particular event] and divide the problems among themselves so that people start thinking a little deeper than they could,” says Varnaar.

Appeals to the “crowd” method turned out to be quite popular among security agencies. In-Q-Tel, the investment and venture department of the CIA, has recently been developing in the market forecasting, aimed at predicting computer security events . DARPA , the cousin-botanist of IARPA , belonging to the Ministry of Defense (Defense Department) , even last year hid a little cash in a secluded place to develop an intelligence, tracking and reconnaissance system to increase the accuracy of his analyzes. Even the Marine Corps (The Navy) turned to crowdsourcing through multiplayer online games , to hunt for the best ideas of the fight against piracy ( naval raids - note of the translator ).

So, how will the ACES be applied in the intelligence community? Documents provided by ARA mention National Intelligence. Forecasts (National Intelligence Estimates (NIEs)) , as an area of ​​potential application. NIEs are estimates collected by various “who-do-not-exist” agencies and leading analysts in specific areas. Right now, ACES is still in the process of developing and adding new ideas that, precisely or at least theoretically, will provide more potential for use. Completed by ACES as a forecasting tool, complemented by the NIE , will allow consumers to get a more accurate picture of their views on the problem.

But as far as ACES is accurate, we still have to find out. It will take a long time before the answers become useful - the development of the project will last 4 years.

Photo: Focus Features