“Production environment out of your control”: Rian Lewis about testing blockchain projects

It may seem that now it is too late to discuss cryptocurrencies and blockchain projects: they say everything possible was said a couple of years ago, and then the overestimated expectations were not met, the hype was asleep and the topic became irrelevant.

But in fact, right now you can talk about her seriously. At the peak of the hype, it was difficult to break through the cries of "INVEST IN OUR ICO YET NOT LATER" to something more reasonable, and in the signal-to-noise ratio, the second component went off scale. But now, when the hype subsided and the lovers of quick money switched to something else, it became possible to talk normally. And when the question “what to invest money in” stopped eclipsing everything else, it became easier to touch on technical aspects.
')
Rian Lewis (known, for example, by a small service CountMyCrypto ) sees the blockchain-ecosystem from both the enthusiast's angle and the technical specialist's angle: it is also interested in “what happens” and testing the blockchain projects. And we decided to ask her questions first about the first, and then go to the second.


- In 2018, when many turned away from cryptocurrency, you remain their enthusiast. Do you think that in the long run they have a bright future?

- Absolutely. Know the concept of "Hype cycle" from Gartner? There is a typical cycle: first, unjustified expectations are placed on a new technology, then they are disappointed in it, and after all this, it develops and becomes truly in demand. In my opinion, we are now at the lowest point of frustration. This is partly due to the current cryptocurrency crisis. As you know, the price trend is now downward. Nevertheless, there is a widespread opinion with which I agree that it is in this phase that important work takes place.

Now there is much less speculation than in 2017 and the second half of 2016. Projects are slowly but surely developing. In parallel with the fall in prices, interest in ICO is falling, but at the same time the number of new startups does not decrease: companies are simply switching to private financing. There are still promising ideas.

In my opinion, this is a positive trend, because thanks to it there was a filtering of people with unrealistic designs. I think you know that a significant part of modern technologies in this area does not scale well, and many startups that collected large sums of money were based on extremely unrealistic expectations. Now is a period of consolidation, and, from my point of view, this is for the better. We are working on very interesting projects, they simply do not tell about them every day in the news.

Specifically, in the case of Ethereum, a decrease in the number of transactions seems to me to be a healthy sign. The Ethereum blockchain architecture is not designed for the number of decentralized applications that began to emerge last year after the release of CryptoKitties. One of the IT evangelists from Parity recently even half in jest asked to stop deploying new DApps under Ethereum. Many were indignant at hearing this, but, in essence, he was right. We cannot continue to deploy more and more decentralized applications on Ethereum without thinking about the architecture of these applications. It is necessary to try to perform maximum actions outside the chain, since the current situation cannot last for a long time.

In general, a fairly high level of activity is now observed, but due to the fact that all new blockchains launch their main networks, the number of transactions is distributed among a large number of blockchains. In addition, the popularity of blockchains without tokenization is growing - examples include Hyperledger and Corda. Corda was created in the consortium R3, it can be used as a private blockchain. So now there are a number of blockchains that are not public and tokenized, they can not see the current activity. And, of course, there are blockchains within various organizations created by different implementations of Blockchain as a Service like SAP or those offered by Microsoft. So, I think, the level of activity is now quite high, just not all of this activity is visible to the public.

- A few years ago, many relished the news that even a Bitcoin conference stopped accepting payment in bitcoins due to increased commissions and processing time. What now with cryptocurrencies as a form of payment?

- As an example, I can cite the Hackers Congress Paralelní Polis conference in Prague, where I will now be - they only accept payment in Bitcoins and other cryptocurrencies. I have already bought a ticket for bitcoins, and while I am there, I will also buy all the food for them. Nevertheless, you raised a really important problem: because of the rise in prices for bitcoin, people used it for some time not as a deal, but as an investment. All the time it was said that it was e-gold, so it was bought for storage, and not for transactions. In 2013, I bought quite a lot of beer and coffee for bitcoins - then it cost a few pounds or a few euros, and now the same amounts in bitcoins can be sold for a pretty decent amount of money.

In fact, a lot here depends on the individual. For example, I have a girlfriend from Russia who does not have a bank account, for the last four years she has been using only cryptocurrency for ethical reasons. There are a small number of such people, an example of which proves that it is possible in principle to live like this. But a significant part of those companies that used to accept bitcoins have now stopped doing this. This is partly due to the fact that during a surge of interest in Bitcoins in some cafe or bar they could train an employee to use the application, and then this employee left, and the next cashier no longer knew how to work with the application. It is a pity that it happened so, because these skills are very simple. To a large extent, everything here is tied to the preservation of existing knowledge. Now quite a lot is being done in order to integrate cryptocurrency and fiat money, simplifying payment.

In general, we are now in some intermediate phase. At first there was a lot of enthusiasm due to the fact that people expected to pay for everything only with cryptocurrency. Then this enthusiasm dried up and many were repelled by the instability of the courses. However, I believe that the time is not far away when people will use many different cryptocurrencies. It is also worth remembering the attempts of various companies to introduce some censorship - it is known, for example, that PayPal closes people's accounts for political reasons. As a result, awareness of the need for censored money is growing stronger.

- Three years ago, they loved to repeat “the blockchain's possibilities are not limited to bitcoin, it can have a lot of other uses,” and they tried to use it in various fields. And what did life show now? Was it useful in practice somewhere else?

- Yes. One of the most famous recent projects was created by Samsung for the transport of goods. You may know that every freighter coming from China to Europe generates about two kilograms of documents. It takes a huge amount of resources, since it is not only paper, but also hundreds of people who sign, stamp, fill out these documents, and so on. All this is necessary because people do not trust each other: the company receiving the goods wants to make sure that it is not fake; the authorities want to make sure that the company does not import anything illegal; authorities of different countries do not trust each other. This gave rise to the idea of ​​using the blockchain to solve this problem of lack of confidence, especially when applied to the transport of goods. And recently, a consortium led by Samsung conducted the first shipment from Asia to Europe, registered using the blockchain.

Thus, there are already successful projects based on the blockchain, but most of them are still at an experimental stage. True, some companies are gradually increasing their activity - for example, Power Ledger, it distributes electricity and records small transactions using the blockchain. More recently, they expanded their field of activity to three cities in Australia and seem to offer their services even outside of Australia. Some banks use blockchains to enter into transactions.

When we talk about blockchains outside Bitcoin, you need to take some caution. A few years ago, the mood was popular that we needed a blockchain without a bitcoin, but I think people now realized that public blockchains need some motivation using tokens. But in private blockchains or consortium blockchains there are really many different activities not related to bitcoin.

- Can you give a negative example, when it seemed “the blockchain is very useful here,” but life has shown that this is not so?

- It is difficult to answer objectively. I think you will not find such a case that someone publicly declared: “here the blockchain does not work.” But there were situations when firms had to make a 180-degree turn.

In Berlin, there is an interesting startup called SatoshiPay, they are engaged in micropayments in publishing. Their plan was to allow writers to monetize their content: for a small fee in bitcoins, the reader opens a small part of the page, then the next one and so on. At first, the company quickly gained fame and was able to raise funds, but then the fee for transactions in Bitcoin became too high, and all their plans were stalled because of this. The case did not end in complete failure, but they had to switch from Bitcoin to Stellar. Another example I can give concerns the markets for predictions. These are some betting sites that were originally based on Ethereum and which later had to be abandoned. In both of the examples described, the idea of ​​using the blockchain was quite reasonable, just the wrong specific choice was made.

There are other examples of sharp turns. I can’t name specific names, but there were rumors about startups using the blockchain to create medical records. At first, this seemed to be a promising direction, but then questions arose related to the legal side of the case: is it permissible to have an immutable data structure for storing information that may need to be changed in case of changes in legislation or for ethical reasons, that is, if a person succeeds in not including information in your medical booklet.

I think there were many situations when companies tried to use the blockchain simply because it was fashionable, and not because they had a specific problem of lack of trust that could be solved with the help of the blockchain. I think the difficulties are now experiencing exactly this kind of projects. But as far as I know, usually these difficulties are solved by switching from one blockchain to another, or, in the interests of saving time, switching from a public blockchain for the whole project to a private blockchain with proof of authority.

- Let us turn to more technical issues. You have a post about ten common misconceptions about blockchain / cryptocurrency. It is focused on the general public, and most of our readers already know what has been described there, but it may not know anything else. Are there any misconceptions about the blockchain common among the IT audience?

- I would not use the word "delusion", because anyone who is engaged in technology and at least a little familiar with the blockchain will have a very good idea of ​​the basics of this technology. I think it would be more correct to speak not about getting rid of delusions, but about some new knowledge, additional training in the development and testing of public blockchains (private blockchains are much closer to what programmers usually do).

I think it's harder to get used to the fact that the production environment is completely out of the control of the developer. It is not easy to predict what will be the performance, and simulate the necessary conditions. For me, this was the main difficulty when I wrote an application for Ethereum with my friends two years ago.

Everything went smoothly with us, there was a well-established pipeline, the application was successfully launched in the virtual blockchain, which is included in the set of tools provided by Ethereum. We deployed the application to Testnet, and it worked fine there. But when deployed to a production network, it becomes much more difficult to predict how the application will behave with the user. For example, there are numerous and unpredictable delays. Many serious problems are not found in the test environment, but only in production.

Suppose you have a contract in Ethereum, and this contract, among other things, dynamically creates wallets for users. But another contract of the same application may be launched before the creation of wallets begins, and this will happen due to the fact that the corresponding block will be obtained first. Thus, in front of you there will be many exploitable traps that can often not be detected before production. Therefore, it is not enough for you to test the application, you need to predict in advance how it will behave in production.

For example, a very serious bug was recently discovered in the Parity Wallet multi-signature wallet. The wallet, in essence, was incapacitated due to the fact that an outsider had access to a function that was supposed to be private. Thus, he managed to disable the entire contract. This situation should not have happened, but since it has happened, nothing can be done, because this code has already appeared on thousands of machines and can not be rolled back. Therefore, when thinking about the architecture and testing, you should think about the worst possible scenarios and how to mitigate the possible damage in a situation where you can not revoke the code.

Usually when writing a website as a last resort, you can always turn off your server. Of course, it will cost you money, but the opportunity is there. And in the case of a public blockchain, you cannot even do this: once the expanded code can no longer be returned.

- When you think about testing cryptocurrency projects, the first thing you remember is stories with giant thefts. Do I understand correctly that security is the most important consideration here?

- Absolutely. According to the Metcalfe effect, the network with the most users will be the most successful and popular in the future. Due to this effect, of all public blockchains, the most development is conducted on Ethereum. For her, created a lot of tools and libraries. For example, there is a startup OpenZeppelin, which provides access to various libraries with the most basic functionality - for example, they have SafeMath library, which prevents overflows. Thus, they take care of some very simple things — for example, make sure that you include open source libraries for security in your project.

Being responsible for other people's money is always hard, but the threat is not only theft. In the case of the most serious hacks, it was often not theft that took place, but the access to money was denied: this was the case, for example, in the case of the bug in the Parity wallet I already mentioned. Money does not go anywhere, but the owner can not get it. From a security point of view, this is a completely new threat that must also be borne in mind. You should not think about whether someone can steal the money, but about whether the system is safe, whether someone can cancel the contract.

When I talk about testing blockchains for security, I always try to convey the idea that vulnerabilities can be not only in smart contracts. They may also be caused by the fact that the developers are focusing entirely on the smart contract, and because of this, they lose sight of the application as a whole. Of course, this is less of a role if you simply provide a database or API. However, many people are developing web applications or mobile applications based on blockchains, and there can be very significant vulnerabilities due to the fact that developers are not thinking properly. For example, last year there was a crack of the Italian cryptobirth BitGrail currency Nano. On the back end, there were no problems interacting with the wallets of various currencies. But I still can not believe that no one noticed the mistake made by the developer of the frontend. When a user withdraws money from his wallet, a check with the account balance was created on the client side, and because of this, any attacker with the most basic knowledge of Chrome tools could withdraw money, changing their balance each time to a higher amount. Many took advantage of this, and there was a major drain of funds.

Thus, trying to make the blockchain as safe as possible, developers often ignore elementary security, say, the frontend. And this is not the first vulnerability of the frontend, which the attackers took advantage of. Interestingly, the development teams in question are all fairly experienced;but in general, the process of software development life cycle is not going the way they plan. In addition, testers are experts on only a few projects with blockchains. Most of these projects are still experimental, and because of this, many have not yet had time to recruit the usual team for commercial development. Therefore, testing often suffers. This is unfortunate because many public blockchains have very good testing tools.

- And besides the emphasis on security, in general, testing is similar to testing "normal" projects?

“I think the same skills are needed everywhere.” The only difference is that you need to learn how to work with a production environment that is out of your control and badly predictable. Obviously, this requires a very good knowledge of the technical side of things.

But if your work already requires you to automate tests, test APIs, test performance, then all these skills will be useful to you. The only difference is that you need to think about user expectations and user experience. It must be borne in mind that the general public is not yet familiar with the specifics of this technology, and it is necessary to inform users that nothing can be done with the stored information, and that a certain transaction can take an arbitrary amount of time. It helps shape user expectations.

Thus, the tester needs to think more about the quality of the product as a whole than about finding specific bugs. Here, I think, some people skills are needed to predict how the user will interact with the application.

Therefore, I believe that developers of public blockchains should have experience in using these blockchains and an understanding of the basics of this system. Just reading about it is not enough. When I hold meetings and answer questions, I always demonstrate what I am saying in practice. Previously, I used the Bitcoin wallet for this purpose, and after Bitcoin went up, I began to use Litecoin. In order to learn, I ask you to install this wallet, show how the transaction is carried out and how to verify its implementation through the Block Explorer. Even such a five-minute experience helps to understand that you are dealing with a public and transparent network.

All other necessary skills are common to any kind of testing: the ability to understand code, use tools like Docker, Jenkins, and so on.

- That is, a person who is already working as a tester will not take much time to go on testing blockchain projects?

- Exactly. I think the most important thing, besides good technical skills, is interest in the subject. The difficulty is that this area is changing very quickly - although this is a feature rather than a bug. If you are comfortable when the situation remains approximately the same and you have been doing the same thing for a year or two, this area is most likely not suitable for you. But, on the other hand, if you like the rapid pace of changes in development, then, most likely, it will be interesting.

- It looks like that on time the invested money in cryptocurrency returned these investments many times, but the testers who invested their time lost it: the acquired knowledge quickly became irrelevant. This is true?

- I think this can be answered on the basis of general considerations on how technology is developing. The knowledge of a person who received a degree in computer science 10 years ago does not remain static during this time, because projects now work in a completely different way than they did then. For example, then there were no containers, it was possible to find projects without version control, and continuous integration was used only in the most advanced projects. So if you want to participate in any interesting projects, in any case you will have to accept the changes. Although indeed, in blockchains, the situation is changing faster than in other areas.

But if the developer is counting on consistency, then, most likely, his career will not be too long - the exception may be very large old projects that can not afford any significant changes. Now you need to be flexible and able to learn. Recall, for example, the transition from SQL to NoSQL or Redis. These technologies did not exist before, and over the past 5-6 years, testers began to expect them to understand them.

- I think that people who are thinking about testing blockchains can stop the fear that “once the hype has collapsed, there are no vacancies there”. But since, as you say, there are still a lot of startups, is the labor market also active?

- I think we are now at the beginning of the growth of demand for testers. Basically, this is not demand from startups, but from banks, exchanges, and various internal projects.

Of course, it is likely that this technology will be much less common than we are now expecting. But as an example, we can look at a cluster of interrelated technologies that has arisen around artificial intelligence and machine learning. There are constantly looking for new employees; Previously, it was mainly researchers and developers, now there is a demand, including for testers. I think the blockchain will evolve along a similar trajectory.

The idea of ​​a data structure that does not need an external security system is attractive, and people see how it can help reduce costs and how much new business models it can generate. Despite all the changes, I think that this basic principle will continue. Perhaps few are now ready to specialize in the blockchain, but at least you can add it to your toolkit. For example, working in big data, I only recently began to notice the demand for testers with experience with Hadooop. Projects with this technology existed before, but there were not looking for people with specialized experience. I believe that the blockchain will develop in the same way. Those experimental technologies that are widely known today will be the most promising in the labor market tomorrow.

— , . , , : ( , )?

- The main technical problem, in my opinion, is scaling. There are two different approaches to its solution. According to one of them, part of the activity should be taken out of the main chain using, say, trust channels - for example, the Lightning Network for Bitcoin and Litecoin, Plasma and Raiden for Ethereum. The second approach is to change the functioning of the blockchains themselves. For example, in Ethereum there will be very significant changes over the next few months - the reward system will change, a proof-of-stake will be added to the proof-of-work. This is a mechanism of consensus, which perform the same function, but work a little differently. In addition, there is a fairly promising idea of ​​blockchain segmentation, that is, splitting data into small pieces, which can then be re-assembled.

I believe that this technology will show itself best of all just when the scaling problem is resolved, and there will be public networks with thousands of nodes that anyone can join. Distributed ledgers for private companies and consortia are, of course, interesting, but they do not reveal the full potential of these confidence-building mechanisms to the extent that public blockchains do.

- And the last question is not quite serious, but it is curious to hear the answer. When cryptocurrencies were at their peak, everyone just talked about buying them, and after the fall they no longer talk. But since now the courses are lower and the period of calm is, then, probably, it is right now that you need to run and buy? What do you advise as an expert?

- I never advise people to buy a cryptocurrency worth more than you are mentally willing to lose. I myself buy small amounts of different currencies. And I would say that now it is indeed wiser to buy than at the end of last year. But always keep in mind that you can lose what you bought. I do not think bitcoin will drop to zero, and yet the probability of this is non-zero.

“Great advice.” Thank you very much for your answers!

. - : , , , . Heisenbug 2018 Moscow , 6-7 . , .