Friday format: IaaS and science - how it works

/ photo by Guilherme Yagui CC

The amount of data collected in various fields of science is constantly growing, which allows researchers to build realistic models and carry out accurate simulations based on them. However, every year it requires more and more computing power.
')
Cloud technologies and IaaS provide users with resources that meet the requirements of the task: the necessary amount of memory and storage, the required number of processors. Due to this, research teams of any size are able to solve problems without investing huge amounts of money in computer infrastructure.

All this helps a lot when conducting research. As an example, the University of São Paulo is the largest university in Brazil, which was already discussed in one of our previous posts . In 2012, the leadership of the university made a decision on the implementation of the project “Cloud USP” In the course of the work, it was planned to form 6 of 150 separate university data centers, and collect corporate, research and educational environments into a massive private cloud.

When the project was implemented, USP acquired the opportunity to conduct research, being at a great distance from the object being studied, and students - the opportunity to study online. More than 150 thousand people got access to lectures, mail, a digital library, as well as museum collections.

“The cloud allows researchers to achieve results much faster, which contributes to the rapid penetration of information technology at the university,” explains Antonio Roque Dechen, executive vice president of management and professor at the Agricultural College Luis de Queiroz at the University of São Paulo. “It speeds up research and development, providing safe and mobile access to particularly important educational tools.”

Mankind is gradually aware of the full potential of cloud computing, and therefore seeks to apply this technology to solve major scientific and industrial problems. Therefore, later in the article we will look at several areas in which IaaS technology is effectively used.

Physics

One of the common problems in conducting large-scale research in physics is the management of data sets. To solve this problem, cloud computing is suitable, with the help of which users get remote access to arrays of information and distributed computing resources. For example, IaaS-clouds can be effectively used to process experimental data from high-energy physics.

A group of scientists from Canada developed a distributed cloud system using IaaS clusters in Canada and the United States. The user of such a system can write batch jobs for an analytical virtual machine and transfer them to the central scheduler. The system will automatically prepare one of the virtual machines in the cloud and launch a user application on it, which, in turn, will have free access to the central database with calibration data.

The virtual machine has the installed software BaBar, which simulates the collisions of charged particles: it measures their movement trajectories and energy. Tests have shown that the system is capable of effectively performing hundreds of batch tasks at the same time, and its potential is not limited to this.

Astronomy

Astronomy is a science adjacent to physics, and it also generates terabytes of data. Processing them each time brings us closer to an understanding of the structure of the universe. Cloud computing is also very common in this area.

For example, in the "clouds" modeling of collisions of galaxies is carried out using the GADGET application. It is specially designed for simulations on parallel computing systems and uses tree-based algorithms to evaluate the influence of gravitational forces on closely spaced particles.



/ NASA's photo Earth Observatory CC

Also worth noting is the mission of the Kepler space telescope launched by NASA in 2009. Equipped with a supersensitive photometer, it was created to search for planets like the Earth outside the solar system. By the beginning of 2014, they had discovered 3.5 thousand candidates for the planet, of which more than 1 thousand were confirmed by various scientific groups of researchers.

Kepler measures with great accuracy the intensity of light coming from distant stars and detects its change when a planet passes over a star's disk. The analysis of such signals requires the calculation of periodograms and the assessment of their significance, and this is impossible without serious computational resources.

Cloud technologies allow you to parallelize calculations, and speed up data processing. For example, performing a task on a cluster of 128 Dell PowerEdge 1950 machines allowed us to increase the productivity of algorithms hundreds of times.

As another example, the system developed by Canadian scientists is worth mentioning. They combined the CANFAR (Canadian Advanced Network for Astronomical Research) cloud computing system with Skytree advanced machine learning software, thereby creating the first cloud-based data mining system used in astronomy.

More than 500 processor cores and several hundred terabytes of reliable storage are now available. Virtual machines are able to perform large-scale calculations and operate with millions of objects, but this is far from the limit of the CANFAR + Skytree system.

Robotics

Analytical company Gartner in 2015 published its study of the “maturity cycle” of developing technologies. On the graph, technologies are distributed according to how large their adoption is by the majority.

The new document says that unmanned vehicles and the Internet of things are at the peak of high expectations. However, one of the main technological and advanced directions remains robotics.

The full potential of robots is not fully revealed, but clouds will soon help with this. The story goes back to the early 1990s. With the advent of the first browser, Mosaic, a professor and students from the University of Southern California began to develop the idea of ​​camera webcasts.

At the same time, the team decided to move away from the concept of passive observation of what is happening and create a robot that cares for a garden with live plants. For these purposes, an industrial manipulator equipped with a camera, an irrigation system and a nozzle for collecting seeds was adapted. Roborvoru was installed in the center of a three-meter flower bed, and users could control it using a specially designed graphic interface. “Telesad”, this name was given to the project, became the first active device working on the network.

Since then, robotics has progressed far enough. At the moment there are hundreds of research laboratories in which more than 5 million service robots that clean up homes and offices, and more than 3 thousand robots helping surgeons in operating rooms around the world have been developed.

But so far it is impossible to create a robot that would put things in the house in its place. Such work is difficult for them. This problem was addressed by Andrew Ng during his speech at Stanford University.



The problem lies in the fact that he is not able to remember all the objects of everyday life - there will always be something with which he is not familiar. The new remote control from the TV, the new toy baby, new slippers.

However, a possible solution already exists: you need to connect an electronic assistant to a wireless network, so that he will have access to an extensive repository of information on the Internet. The cloud robot can receive data directly from data centers. Moreover, this will simplify the hardware stuffing of the electronic assistant, since all the important algorithmic operations will be processed in the data center. Several research groups are already working in this direction.

Cloud technology is the key to a new generation of robots. Take, for example, the Google car, which, when moving, accesses a huge company database with maps and images from space, comparing the information obtained with sensor data and surveillance cameras.

Until recently, robots were considered autonomous systems with limited computing power and memory. Cloud robotics also offers an alternative when robots exchange data and code over wireless networks.

That's all for today. Cloud technologies penetrate into many other scientific fields, for example, chemistry, biology, genetics, and geography. We plan to talk about this in the second part of this post. Subscribe to our Habrablog , not to miss our new publications, friends.