Paul Graham: The Other Road Ahead, Part 1

“For Viaweb, we often did three to five versions a day.”
- Paul Graham, developer, investor, essayist.

I was curious to get acquainted with the forecast of the founder of the most influential business incubator in Silicon Valley (Y combinator). 15 years after the publication of Paul Graham's essay, thanks to Edison and great people from Habr, hands reached the translation. For those who are interested in how the birth of the new product took place and how the three programmers were butting with the giants of the industry, welcome to Cat.


Paul Graham, Robert Morris and Trevor Blackwell in 1995
')
The next chapter of the book "Hackers and Artists" (which is almost completely translated into Russian)

Another road to the future

September 2001
Original - The Other Road Ahead
(For translating the plus in karma knagaev )

(This article explains why much of the next generation software can be implemented on the server side, what it will mean for programmers, and why this new type of software will be a good opportunity for startups. The sources of the article were dialogues at BBN Labs.)

In the summer of 1995, I and my friend Robert Morris (Robert Morris) decided to launch a startup. The PR campaign before the Netscape IPO was then in full swing, and the press was full of e-commerce talk. At that time, the network already had about thirty real electronic stores, and all of them were made by hand. If soon a lot of online stores were to appear in the network, there was a need for software for their development, and we decided to write one.

At first, somewhere around a week, we thought it would be a regular PC application. But then we had the idea to develop an application that will run on our web server, using the browser as a user interface. We tried to rewrite the application to work on the Web, and it became clear that this is the right direction. If we wrote our software for execution on the server, it was more convenient for both users and us.

It turned out that it was a great plan. Now, like Yahoo Store, this software is the most popular designer of online stores, with about 14 thousand users working with it.



When we started developing Viaweb, almost no one understood what we mean when we say that the software will work on the server. That was until Hotmail started a year later, and people started using it. Now everyone knows that this is a reasonable approach. Now there is a name for what we were then: an application provider (Application Service Provider or ASP).

I think that a significant part of the next generation software will be developed on this model. Even Microsoft, who can lose the most, seems to see the inevitability of transferring applications from desktops to the server. If such a transfer occurs, it means a completely different world for programmers. This article describes the surprises that we encounter as the pioneers of this world. And once software moves to servers, what I describe here is our future.

New device?

When we look back at the era of desktop software, I think we should be struck by its inconvenience, just as we are now struck by the difficulties that owners of first cars faced. For the first twenty or thirty years, you had to become a car expert to use a car. But the cars had so many advantages that many of those who were not auto experts also wanted to have them.

Computers are now at the same stage of development. In order to use a personal computer, you have to study much more than you want to know what is going on inside it. However, computers are owned by more than half of households in the States. My mother uses a computer for email correspondence and account management. About a year ago, she was alarmed when she received a letter from Apple in which she was offered a discount when buying a new version of the operating system. Obviously, something is wrong if a sixty-five-year-old woman who wants to use only e-mail and bills is forced to think about installing new versions of the operating system. Ordinary users should not even be aware of the terms “operating system”, let alone “device driver” or “patch”.

Now there is another way to install software that will not force users to become system administrators. Web applications are programs that run on web servers and use web pages as the user interface. For the average user, this new type of software will be simpler, cheaper, more portable, more reliable, and often more powerful than an application on a personal computer.

With web applications, most users will not have to think about anything other than the applications themselves. All complex and changing components will be located somewhere on the server and serviced by specially trained specialists. And what's more, you may not need a computer at all to use the application. All you need is something with a keyboard, screen and web browser. Possibly with wireless internet. Perhaps this will be your mobile phone. Whatever it is, it will be household appliances: it will cost about $ 200, and they will choose on the basis of appearance. You will pay more for Internet access than for equipment, just as it is with phones now [1].

The response time of a server to a user is about a tenth of a second, so users of heavy interactive software (for example, Photoshop) will still want the calculations to run on the local machine. But if you look at why most people use computers, then you will agree that the response time per second will not be a problem for them. My mom doesn't really need a personal computer, and most people, too.

User benefits

Not far from my house there is a car with a sticker “death is better than inconvenience”. Most people in most cases choose something that requires less effort. If web applications win, it will be because they are more convenient. And for both users and developers.

To use a clean web application, all you need is a web browser with Internet access. Therefore, you can use the web application from anywhere. When you install software on your PC, you can only use it on this PC. Worse, your files are locked in this PC. The inconvenience of this model is becoming more and more obvious to people who use networks.

The turning point has become web-based email applications. Millions realized that they should have access to mail from anywhere. And if you are reading your mail, then why not the calendar too? If you are discussing a document with your colleagues, why not edit it? Why should your data be tied to a specific computer?

The idea of ​​“your computer” disappears and is replaced by “your data”. You must have access to your data from any computer. Or even from any device, and the device does not have to be a computer.

Such devices do not store data, they should be like phones. In fact, they can become telephones, or vice versa, telephones will become such devices. And since they become smaller, there is another reason not to store data on them: what you carry with you can be lost or stolen. Leaving a PDA in a taxi is like a disk crash, only in this case, your data falls into someone's hands, and not just erased.

With web-based software, neither data nor applications are stored on the client. Therefore, you do not need to install anything to use. And when the installation is not needed, then do not worry that something will be installed wrong. There can be no incompatibility between the application and the operating system on your device, because the application is not running on your operating system.

Since the installation is not required, try the software before you buy will be easy and simple standard approach. It should be possible to test the application for free just by visiting the site where it is sold. The Viaweb website can be presented as one big pointer to a suggestion for test-drive users.

After the demo version, the subscription to the service should not require anything except filling out a short form (the shorter the better). And this should be the last thing that is required of the user. Using web applications, you should receive updates at no additional cost, without any action, or even without notice.

Upgrades will no longer be as shocking as they are now. Over time, applications will quietly become more powerful. This will require some programming effort. They will need to design applications so that updates do not affect users. This is a new task, but there are ways to solve it.

With web applications, everyone uses the same version, and bugs can be fixed immediately after detection. Therefore, web applications should have significantly fewer bugs compared to applications on the PC. I doubt that Viaweb had more than ten bugs at the same time. This is several orders of magnitude better than in the case of PC applications.

Web applications can be shared by several people. This advantage of cooperative software is obvious, but I can argue users will want this in most applications as soon as they realize that this is possible. For example, it will often be useful to allow two users to edit one document. Viaweb allows several users to edit the site at the same time, more so because the design is correct, than because we expected users to want it, but it turned out that many wanted it.

When you use the web application, your data will be better protected. Disk crashes will not be a thing of the past, but users will no longer hear about it. This will occur in server centers. And companies offering web applications will make backups - not only because they will have competent system administrators who know what to worry about, but also because the service provider losing user data will be in big trouble. When people lose their data because of a disk breakdown, they cannot go mad, because they can only be angry with themselves. When the company loses their data, they will be much angrier.

And finally, web applications should be less vulnerable to viruses. If the terminal does not start anything except the browser, there is less chance of getting a virus, and there is no local data that can be damaged either. A program that attacks the servers themselves will find them very well protected [2].

Web applications will be less annoying users. If you look into the thoughts of the average Windows user, you will find there is a huge unmet need for such software. If you make such software, it can be a powerful advantage.

City code

For developers, the most obvious difference between web software and “ordinary” software is that web software does not consist of one single piece of code. It will be a set of programs of various types, rather than a single large binary. So the design of web applications will be similar to the design of the city, not the design of the building: in addition to buildings, you will need roads, road signs, maintenance, police and fire services, as well as plans for the development and elimination of disasters.

In Viaweb, the software package consisted of relatively large applications, with which users directly work, from programs that use these applications, from programs that constantly work in the background to identify problems, from programs that try to restart what failed, from programs that run from time to time to collect statistics or build search indexes, from programs that are started manually for garbage collection or for moving / restoring data, from programs that mimic users (for measuring performance or finding errors), from programs for diagnosing network crashes, from programs that perform backups, from interfaces to external services, from programs that run an impressive collection of dials showing real-time server statistics (not only necessary only for users, but also for us), changes (including bugfixes) in open source software and from a huge number of configuration files and settings. After we bought Yahoo, Trevor Blackwell (Trevor Blackwell) wrote a spectacular program to transfer stores to new servers located in another part of the country, without turning them off. Programs send us paging messages, send faxes and email to users, collect credit card processing transactions, and interact with other programs through sockets, pipes, http requests, ssh, udp packets, shared memory and files. Part of Viaweb even consists of the “absence” of programs, since one of the key Unix security principles is not to allow running extra tools that can be used to hack servers.

And this is not limited to software. We spent a lot of time developing the server configuration. We ourselves have built our servers from the components - both in order to save money, and in order to get exactly what we want. We had to think about whether the bandwidth provider had enough bandwidth. We have consistently bypassed all RAID vendors.

But equipment is not just something to take care of. When you manage it, you can provide more to users. In the case of personal software, you can determine the minimum hardware configuration, but you will not be able to give more later. If you are administering servers, you can add all users the functions of paging, faxing, phone control or credit card processing, etc., simply by installing the appropriate equipment. We are constantly looking for new opportunities to add various functions based on equipment, not only to please users, but also to stand out compared to competitors who cannot control the equipment directly (because they sell personal software or provide their web applications via the Internet providers).

Since the software in web applications will be a software package, rather than a single executable file, they can be written in several programming languages. When you write personal software, you are practically forced to write in the same language in which the operating system is written, that is, C or C ++. Therefore, these languages ​​are considered (especially by non-technical employees, for example, managers or vice-presidents) as languages ​​for "serious" development. But this is just a stereotype prevailing in the development of personal software. For applications hosted on the server, you can use any language you want. [3] Now many high-end programmers use languages ​​very far from C and C ++: Perl, Python, or even Lisp.

For server software, no one can tell you which language to use, because you control the system as a whole, down to the hardware level. Programming languages ​​are suitable for various tasks. You can use the language that best suits your needs. And when you have competitors, “you can” turns into “must”, because if you do not take advantage of this advantage, they will take advantage.

Most of our competitors use C and C ++, which makes their software noticeably worse, because (among other problems) they cannot work with the lack of state of CGI scripts. If you are going to change something, all changes should occur on the same page with the “Change” button below. Since I used Lisp, which many people still see as a language for research, we were able to make the Viaweb editor look like a desktop application.

Versions

One of the most important changes in this new world will be the order of release of new software versions. In the world of personal software, the release of a version is a big shock when everyone in the company sweats and strains to squeeze one giant piece of code. Obvious comparisons suggest themselves for both the release process and its result.

With the application on the server, you can do the refinement in much the same way as in the program you write for yourself. You release versions as a series of phased changes instead of a sudden big bang. Typically, a desktop software company can release one or two versions in a year. For Viaweb, we often did three to five versions per day.

When you go to this new model, you understand how much in the process of software development depends on the order of release versions. Many of the most disgusting problems that can be seen in the personal software development business are related to the disastrous nature of the release of versions.

When you release only one new version per year, you work with wholesale bugs. Some time before the release date you are collecting a new version, in which half of the code has already been torn out and replaced, with a bunch of new bugs. After that, the quality control team proceeds to their calculation, and programmers begin to eliminate them from the list. Usually they do not reach the end of the list, and in fact no one knows where this end is. This is how to clean the pond from the stones. You never know exactly what is happening inside the application. At best, you end up with some probabilistic characteristic of quality.

In the case of an application on a server, most of the changes are small and consistent. So by themselves reduce the possibility of bugs. It also means that you know exactly what needs to be tested most carefully when you release a new version: what was being done last. You finish the release version with a much more confident control of the situation. In general, you know exactly what is happening in the code. You do not memorize the source, but when you read them, you do it as a pilot reading the instruments on the panel, and not as a detective trying to unravel a mystery puzzle.

Personal software brings a certain fatalism to the bugs. You know that you supply it with bugs, and even determine the mechanisms for their elimination (for example, patch patches). So why bother that they will be a little more? And soon you discover that all the elements that you know about are not functional. Apple this year faced this situation. They felt pressure in relation to the release of a new version of the operating system, which had already been postponed four times, but some components (CD and DVD support) were not yet ready. And the solution? They released a version without these components, and users will install them later.

You are not obliged to run applications on web servers until they start working, and you can publish them as soon as you are done.
Experienced colleagues might think: yes, it sounds great if you say that you are not obliged to release software until it works, but what will happen if you promised to release a new version by a certain date? With web applications, you are not obligated to make such promises, because there are no versions as such. Your application changes gradually and continuously. Some changes may be more significant than others, but the release approach is not harmonious with web applications.

If anyone remembers Viaweb, this may sound strange, since we have always announced new versions. This was done solely for the purpose of PR. The business press, as we understood, operates with version numbers. They give you a reversal if the version is updated, that is, if the first digit in the version number has increased, and only a paragraph, if the version is for modifications, where the changes in the number are after the period.

Some of our competitors offered personalized software with valid version numbers. For such versions, this itself, which seems to us an obvious fact of their backwardness, makes it possible to obtain all possible informational support. We did not want to miss this opportunity, and also began to assign version numbers to our applications. When we wanted to advertise, we collected a list of all the developments that were added from the previous "version", sculpted a new version number on the app and issued a press release informing about the availability of the new version. It is amazing that no one called us about this.

By the time we were bought, we did it three times, and were on version 4. Version 4.1, if the memory fails. After Viaweb became the Yahoo Store, there was no urgent need for popularization, and although the system continued to evolve, the very idea with version numbers was quietly buried.

Bugs

Another powerful technical advantage of web applications is that you can reproduce most of the bugs. User data is right on your disk. If someone fails in your application, you are not required to try to guess what happened there, as it would be in the case of personal software: you can reproduce the failure right during a telephone conversation with them. You may even already be aware of this if your application has a crash alert service.

Web applications work around the clock, so the code you added immediately goes to the meat grinder. Bugs appear quickly.

Developers are often blamed for debugging their software on users. And I want to speak up for this. For web applications, this is actually a good idea, because the bugs are smaller and they are light. When you frequently release new versions, you get a lot less bugs. And when you can instantly reproduce the error and instantly release a fix, you can find and fix most of the bugs as soon as they prove themselves. We have never had at the same time such a number of bugs, which would be enough for us to think about a full-fledged system of bug tracking.

Of course, you are required to test improvements before releasing them into the industrial system, so there should not be serious bugs in the release. The few that will inevitably slip will manifest themselves only in non-standard conditions for a small group of users who have time to face these bugs while no one has yet had time to complain. And if you eliminate the errors immediately, then as a result, the average user will see a lot less bugs. I think that the average Viaweb user has never seen errors at all.

Eliminating fresh bugs is much easier than eliminating old ones. As a rule, it is much easier to look for an error in the code you just wrote. When an error occurs, you can often determine what is wrong even before you get into the code, because it is still in the subconscious. Fixing a bug in the code that you wrote six months ago (average time if the version is released once a year) will require much more effort. And since you no longer feel the code as well, most likely, you will not fix it in the best way, or even add other bugs. [four]

When bugs are caught in the early stages, you get fewer composite bugs. Composite bugs are two separate bugs that interact with each other: you go down the stairs, and when you cling to the railing, they remain in your hand. In software, this kind of bugs is the most difficult to detect, and besides, it tends to bring worse consequences. [5] The traditional “break everything and eliminate bugs” approach itself generates many composite bugs. Continuous release of small versions does not lead to this. Floors are cleaned up all the time from any lost small things that can later get stuck in something.

This technique helps functional programming. OP means to avoid side effects. This is something more suitable for scientific research than for commercial software, but in the case of web applications it turns into a really useful idea. The whole program is difficult to write in a purely functional code, but you can write individual blocks in this style. Such parts of your application are easier to test because they have no state, and this is very convenient in the case when you constantly make and test small changes. I wrote most of Viaweb's editor in this style, and we created our own scripting language RTML as a purely functional language.

Those who work in the personal software business will find it hard to believe, but in Viaweb, bugs become almost a game. Since most of the bugs in the industrial version are detected in exceptional cases, users who find them are advanced.

Advanced users are more patient with bugs, especially if bugs appear in the process of adding any new feature of the application that they asked to add. In fact, since the bugs are very rare, and you have to do some complicated actions to see them, advanced users are often proud of such findings. They call in support rather in the mood of a victor than in anger, as if gaining points.

Support

When you can reproduce errors, it changes your approach to support. In most development companies, support is offered as a way to make users feel better. Either they call you about a known bug, or they just do something wrong, and you try to figure out exactly what. In these cases, you do not learn anything special. So you start to consider support calls as a pain in the ass, and you want to protect your developers from them as much as possible.

Not so it is arranged in Viaweb. Viaweb support is free because we want to hear from our customers. If any of them have a problem, we want to know about it immediately in order to reproduce the error and issue a fix.

So in Viaweb, developers have always been in close contact with support. The support staff sat ten meters away from the programmers, and they knew that they could disturb at any moment with a message about a new bug. We could suspend the board meeting to correct a serious mistake.

Everyone liked our approach to support. Buyers were delighted. Just imagine this feeling when you call the hotline and you are listened to as a source of very important news. Support staff liked it because it meant that they could help users, instead of reading scripts to them. And programmers liked it, because they could reproduce errors, instead of getting vague reports about it from second-hand.

Our on-the-fly error recovery policy has changed the relationship between support staff and high-end programmers. Most companies in support work low-paid "watchmen", and programmers are small copies of God the Father, the creators of the Universe. Whatever the error reporting procedure, it is most likely directed in one direction: the support officer, who received information about the error, writes them in some form and after some time sends (possibly through the quality service) to programmers who put them in the work plan. This was completely wrong on Viaweb. Within a minute after receiving the error message from the user, the support officer could already stand next to the programmer and hear from him “Damn, you're right, this is a bug.” The support staff was very pleased to hear “you are right” from the super-specialist. They are used to bringing us bugs with the same facial expression that happens to a cat carrying you just killed mouse. It also made them more cautious in assessing the importance of a mistake, because their reputation was at stake.

After we bought Yahoo, the support staff moved away from the programmers. Only then did we feel that they were actually employees of the quality service and to some extent marketing. In addition to catching bugs, they were the custodians of knowledge about obscure bug-like things that confused users.[6] They were also in some way a focus group of user representatives; we could ask them which of the two new features users want more, and they always turned out to be right.

To be continued.

[1] Realizing that a large share of the money in the provision of services, companies that develop thin clients, usually try to combine hardware solutions with online solutions. This approach is not very good, partly due to the fact that you need two different firms, one of which will develop the hardware, and the second to provide online services, and partly because users hate this idea. Gilette can afford to sell razors cheaply and earn money later on blade sales, but a razor is a much smaller investment than a web terminal. Companies selling mobile phones have enough revenue from the sale of devices, and they are not trying to get all the revenue from the telephone connection. This should probably be a model for Internet devices. If someone just sells a beautiful little box with a web browser,which you can use with any Internet provider, every man in the street will buy one for himself.

[2] Security is always afraid of mistakes, more than any other design decisions, but the nature of the server software forces developers to pay even more attention to protection against hacking. Server compromise can cause so much damage that application providers (who don’t want to burn out) have to take care of security.

[3] In 1995, when we started Viaweb, Java applets were considered as a technology that everyone would use to develop server applications. We applets seemed outdated idea. Download to run the program on the client? Easier to get to the logical end and run the program on the server. We spent a little time on the applets, but it is vanishingly small compared to other startups that were lured into this tar pit. Few were able to get out of there alive, or Microsoft could not escape, throwing Java support out of its latest versions of IE.

[4] This idea is from Trevor Blackwell (Trevor Blackwell), who added “the cost of creating software grows faster than its size. It is possible mainly because of the need to fix old bugs, and the increase in cost will be closer to linear if all the bugs are found immediately. ”

[5] The bug that is most difficult to find can be a kind of composite bug when the action of one bug is compensated by the action of another . When you fix one bug, the other manifests itself. But it seems that the correction is the cause of the error, as this is the last change you just made.

[6] In Viaweb, we once organized a competition to describe the worst problem in our software. Two support staff played first prize among themselves, pointing out such things that I still remember with a shudder. We immediately eliminated these errors.

---

Hackers and artists