How GitHub is released

Yac 2013 visited Jason Rudolph from GitHub. I think his report on the API was one of the most interesting at the conference. Yandex promised to post entries in the network, so I advise you to watch it at your leisure for everyone who has not seen.

But this is not about the report. The picture shows the release schedule for GitHub production .


')
When I heard the number, I could not believe my ears. GitHub has hundreds of updates per week. There are about forty developers and no QA in the team.

Fortunately, Jason, after the report, was still near the stage for some time and I was able to question him with a passion for how they do it.

Hubot lives in GitHub . At first it was just4fun chat bot. Over time, he learned to open the door in the office and ... run the display on the production. In GitHub, any developer from any feature-brand can start the calculation with a simple chat message. I often do a lot of release management. When I heard about this order the hair on my hands involuntarily stirred. In fact, to lay out a "beaten" release will not give you anyway. The layout is as follows:

• If the strum was not dealt with the master - to smite him
• Run tests
• Run Migration
• Post changes in staff-only mode
• If everything is good, turn on "public mode"
• Check logs and twitter for errors / WTF tweets
• If everything is ok, you can use the feature in the master

DB Migrations

The GitHub database is large enough and migrations can take a long time. Therefore, they are performed separately, before posting the update. For all the time Jason will not remember that I had to roll back. Despite this, developers always write backward migrations.

Tests

When Jason came to GitHub, there were a lot of tests and they ran for about two minutes. Now they are "very much", but they are executed in 50 seconds. GitHub was able to achieve this through parallelization. All tests are performed on 64 cores. After each run, the environment analyzes which test packs were run longer than the others and automatically redistributes the tests so that everything runs for approximately the same time. GitHub does not use Selenium, but they have unit tests, integration and system end-to-end tests. In general, a separate DevOps department is responsible for automating this entire kitchen, so developers can focus on features.

Staff-only

All new features fall first into the so-called staff-only mode. Employees of GitHub will be able to touch them, for others the feature will be “pretend” that it is not there. Staff-only is implemented in the code of GitHub itself. The same idea is used in the iOS division of Facebook. They have all the features have a toggle switch on / off. This helps to release a release even if it is necessary to urgently "return some change back."

Twitter

Jason jokes that GitHub has the best team of testers - their customers. If “something went wrong” anomalous activity immediately appears on Twitter. This Yac made me take another look at Twitter: it seems that this service can still be used for something more useful than “Mom, I poked.”

Professional ethics

Getting to work at GitHub is not an easy task. Each candidate, even if someone from the team recommended him, is studied with passion. The advantage of GitHub is that if a candidate has an account, they get a lot of information about the applicant. Attention is paid not only to the quality of the code, but also to whether the person supports his project, how quickly he responds to the bug reports, and whether he cooperates with the community. There are many criteria, but if it is short, then GitHub employs people, whose main goal is to produce a cool product. Not the coolest code, design, ui, algorithm, namely “product”. It should be important for a person to “deliver” (to deliver) new functionality on time and efficiently. This, perhaps, is what distinguishes the most "foreign" developers from our compatriots. I am deeply convinced that product orientation is great and we need to learn this from Western colleagues.