📜 ⬆️ ⬇️

Dive into Ethereum development. Part 5: Oraclize

Access to large files and various external dynamic data is often a very important part of a decentralized application. At the same time, Ethereum does not provide for the mechanism of circulation to the outside itself - smart contracts can read and write only within the framework of the blockchain itself. In this article we will consider Oraclize, which just gives the opportunity to interact with the outside world by querying virtually any Internet resources. A related topic is IPFS, briefly mentioning it.



IPFS


IPFS is a distributed file system with content addressing. This means that the content of any file added there is considered a unique hash. The same hash is then used to search and retrieve this content from the network.
The main information is already described in this article and in several others, so there is no point in repeating.

Why use IPFS in conjunction with Ethereum?


Any volumetric content stored on the blockchain is too expensive and harmful to the network. Therefore, the best option is to save any link to the file that is located in the off-line repository, not necessarily IPFS. But IPFS has several advantages:
')

Among the shortcomings we can mention that since there is no central server, then for the availability of the files it is necessary that at least one of these files be “distributed”. But if you have a specific file, then it is easy to connect to the distributors — launch the ipfs daemon and add the file via ipfs add .

The technology is very well suited to the ideology of decentralization; therefore, considering Oraclize now, we will more than once encounter the use of IPFS in different oracle mechanisms.

Oraclize


To perform almost any useful work, a smart contract needs to receive new data. However, there is no built-in ability to execute a request from the blockchain to the outside world. You can of course add everything that is required by transactions manually, but it is impossible to verify where this data came from and their accuracy. Plus, you may need to organize additional infrastructure for the rapid updating of dynamic data, such as exchange rates. And updates with a fixed interval will lead to gas overspending.

Therefore, the service provided by Oraclize comes in handy: in a smart contract you can send a request to almost any API or resource on the Internet, be sure that the data came from the specified resource unchanged, and use the result in the same smart contract.

Oraclize is not only an Ethereum service, similar functionality is provided to other blockchains, but we will only describe a bundle with Ethereum.

Beginning of work


All you need to get started is to add yourself to the project one of the oraclizeAPI files from the repository . You only need to choose the appropriate compiler for your version (solc): oraclizeAPI_0.5.sol for versions starting at 0.4.18, oraclizeAPI_0.4.sol for versions from 0.4.1, oraclizeAPI_pre0.4.sol for everything older, support This version has already been discontinued. If you use truffle, do not forget to rename the file to usingOraclize - it requires that the name of the file and the contract match.

By including the appropriate file in your project, inherit the contract from usingOraclize . And you can start using Oracle, which boils down to two main things: sending a request using the oraclize_query helper, and then processing the result in the __callback function. The simplest smart contract (to get the current price of the broadcast in dollars) may look like this:

 pragma solidity 0.4.23; import "./usingOraclize.sol"; contract ExampleContract is usingOraclize { string public ETHUSD; event updatedPrice(string price); event newOraclizeQuery(string description); function ExampleContract() payable { updatePrice(); } function __callback(bytes32 myid, string result) { require (msg.sender == oraclize_cbAddress()); ETHUSD = result; updatedPrice(result); } function updatePrice() payable { if (oraclize_getPrice("URL") > this.balance) { newOraclizeQuery("Oraclize query was NOT sent, please add some ETH to cover for the query fee"); } else { newOraclizeQuery("Oraclize query was sent, standing by for the answer.."); oraclize_query("URL", "json(https://api.coinmarketcap.com/v1/ticker/ethereum/?convert=USD).0.price_usd"); } } } 

The function that sends the request is updatePrice . You can see that first there is a check that oraclize_getPrice(“URL”) greater than the current balance of the contract. This is done because the call to oraclize_query has to be paid, the price is calculated as the sum of the fixed commission and the payment for the gas to call the callback. “URL” is the designation of one of the types of data sources, in this case it is a simple https request, then we will consider other options. Answers on request can be pre-parsed as json (as in the example) and in several other ways (we will look further). In __callback , a string with the answer is returned. At the very beginning, it is checked that the call passed from the trusted address oraclize

All options for using oraclize are based on the same scheme, only data sources and the ability to add authentication to __callback . Therefore, in future examples, we will cite only significant differences.

Price of use


As already mentioned, additional air is paid for oraclize requests, and it is removed from the balance of the contract, and not the calling address. The only exception is the first request from each new contract, it is provided free of charge. It is also interesting that in test networks the same mechanics is preserved, but the payment goes on the air of the corresponding network, that is, in testnets, the requests are actually free.

It has already been mentioned that the request price is made up of two quantities: a fixed commission and a callback call charge. The fixed commission is determined in dollars, and the amount of air is calculated from the current rate. The Commission depends on the source of the data and additional supporting mechanisms, which we will dwell on. The current price table looks like this:


As you can see, the price per URL request is a few cents. Is it a lot or a little? To do this, let's look at how much the second part costs - gas for a call to a callback.
This works according to the following scheme: the amount of air needed to pay for a fixed amount of gas at a fixed price is transferred in advance from the contract together with the request. This amount should be enough to fulfill the callback, and the price should be adequate to the market, otherwise the transaction will not work or will hang for a very long time. At the same time, it is clear that it is not always possible to know the amount of gas in advance, and therefore the fee must be in stock (the stock is not returned). The default values ​​are 200 thousand gas limit at 20 gwei. This is enough for an average callback with several entries and some kind of logic. Although the price of 20 gwei may seem too high at the moment (at the time of writing, the average is 4 gwei), but at the moments of the influx of transactions, the market price may suddenly jump and be even higher, therefore, in general, these values ​​are close to the actual ones. So, with such values ​​and the price of air in the region of $ 500, payment for gas will approach $ 2, so we can say that the fixed commission takes an insignificant part.

If you know what you are doing, then there is an option to change the limit and price of gas, thus significantly saving on requests.

The gas price can be set by a separate function - oraclize_setCustomGasPrice(< wei>) . After the call, the price is saved and used in all subsequent requests.
The limit can be set in the oraclize_query request oraclize_query , specifying it with the last argument, for example:

 oraclize_query("URL", "<>", 50000); 

If you have complicated logic in __callback and gas is spent more than 200k, then you will definitely need to set a limit that covers the worst case of gas consumption. Otherwise, if the limit is exceeded, __callback simply roll back.

By the way, recently, oraclize has information that requests can be paid outside the blockchain, which will allow you not to spend the entire limit or return the balance (and the payment is not from the contract). We have not had to use it yet, but oraclize offers to contact them at info@oraclize.it, if this option is interesting. So keep in mind.

How does it work


Why, inheriting from the usual smart contract, we get functionality that was not originally supported by the blockchain mechanisms? Actually service OrakLayz consists not only of contracts with functions-helpers. The main job of getting data is external service. Smart contracts form applications for access to external data and put them in the blockchain. External service - monitors new blocks of the blockchain, and if it detects an application - executes it. Schematically, this can be represented as:


Data sources


In addition to the considered URL , oraclize provides 4 more options (which you saw in the section on prices): WolframAlpha , IPFS , random and computation . Consider each one of them.

1. URL


The example already considered uses this data source. This is the source for HTTP requests to various APIs. In the example was the following:

 oraclize_query("URL", "json(https://api.coinmarketcap.com/v1/ticker/ethereum/?convert=USD).0.price_usd"); 

This is the receipt of the price of the broadcast, and since api provides the json string with the data set, the request is wrapped in a json parser and returns only the field we need. In this case, this is GET, but the source URL also supports POST requests. The type of request is automatically determined by the optional argument. If there is a valid json there as in this example:

 oraclize_query("URL", "json(https://shapeshift.io/sendamount).success.deposit", '{"pair":"eth_btc","amount":"1","withdrawal":"1AAcCo21EUc1jbocjssSQDzLna9Vem2UN5"}') 

then the request is processed as POST (the api used is described here , if interested)

2. WolframAlpha


This data source allows you to access the WolframAlpha service, which can provide answers to various requests for facts or calculations, for example

 oraclize_query(“WolframAlpha”, “president of Russia”) 

will return Vladimir Putin , and the request

 oraclize_query(“WolframAlpha”, “solve x^2-4”) 

returns x = 2 .
As you can see, the result was incomplete, because the symbol ± was lost. Therefore, before using this source, you need to check that the value of a specific request can be used in a smart contract. In addition, authentication is not supported for answers, therefore oraclize themselves recommend using this source only for testing.

3. IPFS


As you can guess, allows you to get the contents of the file in IPFS on the multihash. The timeout for receiving content is 20 seconds.

 oraclize_query(“IPFS”, “QmTL5xNq9PPmwvM1RhxuhiYqoTJcmnaztMz6PQpGxmALkP”) 

will return Hello, Habr! (if the file with such contents is still available)

4. random


Random number generation works in the same way as other sources, but if you use oraclize_query , you need time-consuming preparation of arguments. To avoid this, you can use the helper function oraclize_newRandomDSQuery(delay, nbytes, customGasLimit) , specifying only the execution delay (in seconds), the number of bytes generated and the throttle limit for the __callback call.
Using random has a couple of features to keep in mind:



5. computation


It is the most flexible of the sources. It allows you to write your own scripts and use them as a data source. Calculations take place on AWS. For execution, you need to describe the Dockerfile and put it together with arbitrary additional files in a zip-archive, and upload the archive to IPFS. Execution must meet the following conditions:


For an example of how this is done, consider how to perform the simplest join of the passed strings and return the result.

Dockerfile:

 FROM ubuntu:16.04 MAINTAINER "info@rubyruby.ru" CMD echo "$ARG0 $ARG1 $ARG2 $ARG3" 

Environment Variables ARG0 , ARG1 , etc. - These are the parameters passed along with the request.
Add pre-file to the archive, start the ipfs server and add this archive there

 $ zip concatenation.zip Dockerfile $ ipfs daemon & $ ipfs add concatenation.zip QmWbnw4BBFDsh7yTXhZaTGQnPVCNY9ZDuPBoSwB9A4JNJD 

The resulting hash is used to send the request via oraclize_query in the smart contract:

 oraclize_query("computation", ["QmVAS9TNKGqV49WTEWv55aMCTNyfd4qcGFFfgyz7BYHLdD", "s1", "s2", "s3", "s4"]); 

The argument is an array, in which the first element is the multi-cache archive, and all the rest are parameters that fall into the environment variables.

If you wait for the query, then in __callback will come the result s1 s2 s3 s4 .

Helper parsers and subqueries


From the response returned by any source, you can pre-select only the required information using a number of helpers, such as:

1. JSON parser


You saw this method in the very first example, where from the result that coinmarketcap returns, only the price was returned:

 json(https://api.coinmarketcap.com/v1/ticker/ethereum/?convert=USD).0.price_usd 

The use case is pretty obvious, going back to the example:

 [ { "id": "ethereum", "name": "Ethereum", "symbol": "ETH", "rank": "2", "price_usd": "462.857", "price_btc": "0.0621573", "24h_volume_usd": "1993200000.0", "market_cap_usd": "46656433775.0", "available_supply": "100800968.0", "total_supply": "100800968.0", "max_supply": null, "percent_change_1h": "-0.5", "percent_change_24h": "-3.02", "percent_change_7d": "5.93", "last_updated": "1532064934" } ] 

Since this is an array, we take the element 0 , and from it the price_usd field

2. XML


Use similar to JSON, for example:

 xml(https://informer.kovalut.ru/webmaster/getxml.php?kod=7701).Exchange_Rates.Central_Bank_RF.USD.New.Exch_Rate 

3. HTML


You can parse XHTML with XPath. For example, get a market cap with etherscan:

 html(https://etherscan.io/).xpath(string(//*[contains(@href, '/stat/supply')]/font)) 

We MARKET CAP OF $46.148 BillionB

4. Binary helper


Allows you to cut pieces from raw data using the slice (offset, length) function. That is, for example, we have a file with the contents of “abc”:

 echo "abc" > example.bin 

Put it on IPFS:

 $ ipfs add example.bin added Qme4u9HfFqYUhH4i34ZFBKi1ZsW7z4MYHtLxScQGndhgKE 

And now let's cut out 1 character from the middle:

 binary(Qme4u9HfFqYUhH4i34ZFBKi1ZsW7z4MYHtLxScQGndhgKE).slice(1, 1) 

In the answer we get b

As you may have noticed, in the case of the binary helper, IPFS was used, not the URL source. In fact, parsers can be applied to any sources, let's say it is not necessary to apply JSON to what the URL returns, you can add such content to the file:

 { "one":"1", "two":"2" } 

Add it to IPFS:

 $ ipfs add test.json added QmZinLwAq5fy4imz8ZNgupWeNFTneUqHjPiTPX9tuR7Vxp 

And then disassemble like this:

 json(QmZinLwAq5fy4imz8ZNgupWeNFTneUqHjPiTPX9tuR7Vxp).one 

We get 1

And a particularly interesting use case is to combine any data sources and any parsers in one query. This is possible using a separate nested data source. We use the newly created file in a more complex query (the addition of values ​​in two fields):

 [WolframAlpha] add ${[IPFS] json(QmZinLwAq5fy4imz8ZNgupWeNFTneUqHjPiTPX9tuR7Vxp).one} to ${[IPFS] json(QmZinLwAq5fy4imz8ZNgupWeNFTneUqHjPiTPX9tuR7Vxp).two} 

We get 3
The query is formed as follows: specify the nested data source, then for each query add the source name in front of it in square brackets, and further enclose all subqueries in ${..} .

Testing


Oraclize provides a useful query validation service without the need for smart contracts. Just go, choose a data source, a verification method and see what will return in __ callback if you send the corresponding requests

For local verification in conjunction with a smart contract, you can use a special version of the Remix IDE that supports oraclize requests.

And to check locally with ganache, you will need an ethereum bridge , which will deploy smart oraclize contracts into your testnet. To test, first add the following line to your contract constructor:

 OAR = OraclizeAddrResolverI(0x6f485C8BF6fc43eA212E93BBF8ce046C7f1cb475); 

run

 ganache-cli 

Then

 node bridge --dev 

Wait for the contracts to go through and be tested. In the output of the node bridge it will be possible to see the sent requests and the received responses.

Another help not only for testing, but also for real use - the ability to monitor requests here . If you request a public network, you can use the hash of the transaction in which the request is executed. If you use authentication, then keep in mind that they are guaranteed to be sent only to mainnet, for other networks it can arrive 0. If the request was on the local network, then you can use the id of the request, which is returned by oraclize_query . By the way, this id is always recommended to be saved, for example in a similar mapping:

 mapping(bytes32=>bool) validIds; 

During the request, mark the id sent as true :

 bytes32 queryId = oraclize_query(<...>); validIds[queryId] = true; 

And then in __callback to check that the request with such id was not processed yet:

 function __callback(bytes32 myid, string result) { require(validIds[myid] != bytes32(0)); require(msg.sender == oraclize_cbAddress()); validIds[myid] = bytes32(0); <...> 

This is necessary because the __callback per request can be called more than once due to the peculiarities of the operation of the Oraclize mechanisms.

Authentication


In the table with sources you could see that different sources can support different types of confirmations, and different commissions may be charged. This is a very important part of oraclize, but a detailed description of these mechanisms is a separate topic.

The most commonly used mechanism, at least by us, is TLSNotary with storage in IPFS. Storage in IPFS is more efficient because the __callback does not return the proof itself (maybe around 4-5 kilobytes), but a much smaller multi-cache. To set this type, add a line in the constructor:

 oraclize_setProof(proofType_TLSNotary | proofStorage_IPFS); 

We can only say that this type, roughly speaking, protects us from the unreliability of data obtained from Oraclize. But Oraclize uses Amazon servers, which act as an auditor, so they only have to trust.

Read more here .

Conclusion


Oraclize provides tools that significantly increase the number of use cases for smart contracts, as well as IPFS, which can be seen in several variations of Oracle requests. The main problem is that we again use external data that is subject to the threats from which the blockchain was supposed to protect: centralization, blocking capabilities, code changes, result substitution. But while this is all inevitable, and the option of obtaining data is very useful and viable, you just need to be aware of why the use of the blockchain was introduced into the project and whether the appeal reduces external unreliable sources to zero advantage.

If you are interested in some topics on the development of Ethereum not yet disclosed in these articles - write in the comments, perhaps we will reveal in the following.

Ethereum development:
Part 1: introduction
Part 2: Web3.js and gas
Part 3: user application
Part 4: warmth and debug in truffle, ganache, infura

Source: https://habr.com/ru/post/417681/


All Articles