How to manage cloud resources with Python? We raise clusters on demand through several lines of code
The Simple Azure library allows you to manage cloud resources, including the creation, management, and deletion of virtual machines in a cloud environment. You can use this library for any purpose: from deploying a sandbox for Dev & Test purposes to locating and managing solutions in commercial operation.
Using Simple Azure, you can easily raise ipython notebook and ipython clusters in a cloudy environment, deploy ready virtual machines to choose from hundreds of VMDepot presented in the catalog .
Below is a brief introduction and examples of using Simple Azure for simple tasks and tasks for deploying an IPython cluster in the cloud.
Video :
')
Change virtual machine to choose from:
This code example selects an OpenSUSE based Azure Data Science Core virtual machine from the VMDepot community directory with a customized environment for working with big data, HPC and the following packages: cython, ipython, matplotlib, networkx, nltk, nodejs, numpy, pandas, pytables, redis, scikit-image, scikit-learn, scipy, statsmodels, sympy.
Simple Azure supports commands in the Linux shell. For example, you can create StarCluster type virtual machine clusters with the following command:
This guide describes how to deploy an IPython cluster in the cloud in a few steps.
Step one:
Step two - create three nodes for a cluster with ADSC:
Step three - import the IPython plugin:
SSH setup:
Step four - configure the master and engine nodes:
Name binding in the IPython plugin:
Install SSH connection to the nodes:
Creating an IPython profile:
Starting IPController on master:
Setting engine nodes:
The last step is to execute ipengine on each engine node, so that they start interacting with the master:
More guides can be found at the following link . The details of the installation and configuration of the library are described here . You can get the source code of the library and join the development project on GitHub . The latest news and information about the project you can always find on the official website of Simple Azure .
Using Simple Azure, you can easily raise ipython notebook and ipython clusters in a cloudy environment, deploy ready virtual machines to choose from hundreds of VMDepot presented in the catalog .
Below is a brief introduction and examples of using Simple Azure for simple tasks and tasks for deploying an IPython cluster in the cloud.
Installation
$ pip install simpleazure $ simpleazure-cluster start mycluster Deploying Virtual Machines
from simpleazure.simpleazure import SimpleAzure as saz azure = saz() azure.get_config() azure.create_vm() Video :
')
Change virtual machine to choose from:
azure.set_image(label="Ubuntu Server 12.04.2 LTS") Placement of several cars
azure = saz() azure.get_config() azure.create_cluster(num=5) my-cluster-vm-0-87412 {'request_id': '88c94c00288d42acaf877783f09c4558'} my-cluster-vm-1-61293 {'request_id': 'abfd563c2c4f4926872b6b1dba27a93b'} my-cluster-vm-2-96085 {'request_id': '29b55f6cb5e94cfdbf244a7c848c854d'} my-cluster-vm-3-46927 {'request_id': 'b1a3446ebafe47a295df4c9d1b7d743c'} Hosting machines from the VMDepot community directory
This code example selects an OpenSUSE based Azure Data Science Core virtual machine from the VMDepot community directory with a customized environment for working with big data, HPC and the following packages: cython, ipython, matplotlib, networkx, nltk, nodejs, numpy, pandas, pytables, redis, scikit-image, scikit-learn, scipy, statsmodels, sympy.
azure = saz() azure.get_config() q = azure.get_registered_image(name="Azure-Data-Science-Core") azure.set_image(image=q) azure.create_vm() Command line support
Simple Azure supports commands in the Linux shell. For example, you can create StarCluster type virtual machine clusters with the following command:
$ simpleazure-cluster start mycluster
Enable IPython Cluster
This guide describes how to deploy an IPython cluster in the cloud in a few steps.
Step one:
from simpleazure.simpleazure import SimpleAzure as saz azure = saz() azure.get_config() Step two - create three nodes for a cluster with ADSC:
adsc = azure.get_registered_image(name="Azure-Data-Science-Core") azure.set_image(image=adsc) azure.set_location("West Europe") azure.create_cluster(3) Step three - import the IPython plugin:
from simpleazure.plugin import ipython ipy = ipython.IPython() SSH setup:
ipy.set_username(azure.get_username()) ipy.set_private_key(azure.get_pkey()) Step four - configure the master and engine nodes:
from simpleazure import config master = config.get_azure_domain(azure.results['master']) engines = [ config.get_azure_domain(x) for x in azure.results.keys()] Name binding in the IPython plugin:
ipy.set_master(master) ipy.set_engines(engines) Install SSH connection to the nodes:
ipy.init_ssh() ipy.connect_nodes() Creating an IPython profile:
ipy.create_profile() Starting IPController on master:
ipy.run_ipcontroller() Setting engine nodes:
ipy.copy_pkey_to_nodes() # <- Temporary function to distribute id_rsa private key to node(s) ipy.copy_json2engines() The last step is to execute ipengine on each engine node, so that they start interacting with the master:
ipy.run_ipengine() ipy.apply_ipcluster(azure) Additional Information
More guides can be found at the following link . The details of the installation and configuration of the library are described here . You can get the source code of the library and join the development project on GitHub . The latest news and information about the project you can always find on the official website of Simple Azure .
Additional links
Try Azure for free for 30 days
Take a training course at the Microsoft Virtual Academy (MVA)
- Application Development on the Microsoft Platform
- Microsoft Azure and SQL Azure Databases security overview
- Azure Software Service Management
- Azure immersion for application development and infrastructure services
- Introduction to the development of modern applications with Azure for beginners
- or Azure cloud development courses
- or many other free MVA courses !
Source: https://habr.com/ru/post/229083/
All Articles