Introduction

The second part of the review will be devoted to the HockeyApp service, the first part of the review can be found here .

Apparently, Miscrosoft decided to keep up with Apple, as a result of which, in early December, it was announced that HockeyApp was acquired (more details on the HockeyApp blog and on the Microsoft website ). Unlike Apple, Microsoft did not cut the functionality of the service or limit the list of supported platforms. The press release states that the service will continue its work without changes, and in the future it will be developed, the details of which we will learn later.

Evaluation system: services are evaluated on a 10-point scale for each of the sections (Registration and Integration, Main Functionality, Additional Functionality, Continuous Integration). The total score will determine the winner (the final conclusion will be included in the last part of the review).
')

HockeyApp

Registration and Integration

Registration in the service is simple, with mandatory confirmation by e-mail. A free account does not allow the full use of the service - only participation as a tester is available, or the use of the service without restrictions as part of another team (organization account). In order to evaluate the service, it is possible to enable the Trial mode - for 1 month (and the trial period will start only from the moment the profile of the first application is created, and not from the day of registration). The cost of a paid account starts at $ 10 per month, more details can be found here . From the advantages it can be noted that for any type of paid accounts there are no restrictions on the size of the stored data and the number of users associated with the account. The difference between different tariff plans in the number of applications and the number of accounts with the type “Owner” (only for tariff plans of the category “Business”).

The only wish for the account creation process: to make more clear the result of choosing the option “I'm developer”:

Now, when it is not selected by default, the following explanation can be taken as a list of features that are already available, and if you select the option, something else is expected to appear. In fact, the default account only allows you to view a list of available applications (and, of course, install them), add your devices and leave reviews.

Separately, I would like to note that the service website has a special guide for those developers who need to migrate a project that uses TestFlight of the old version (before switching to Apple): http://support.hockeyapp.net .

Integration with the SDK is also quite simple, but of course far from ideal, which, in my opinion, is currently the solution used by Fabric ( Crashlytics ) - we'll talk about it in the final - fourth - part of the review.





Rating: 7/10.

Main functionality

Note (updated June 15, 2015): speaking of the distribution of iOS applications, you need to remember that at the moment all services except Apple's TestFlight still have to take into account the limit on the number of devices available using AdHoc provisional profiles. Consequently, the maximum of the devices used is 100 for each family of devices (this condition was changed on June 8, 2015, after announcing at WWDC that all developers programs are combined into one), which also includes devices used directly by developers. The biggest limitation is that the list of registered devices can be “zeroed out” only once a year, after the subscription to the iOS Developer Program is renewed (it is possible later that this condition will be changed).

The next step after activating a new HockeyApp account (of any type) is to register the device, which will be used to install test versions of applications:

The service offers several ways:

• Follow the link in the mobile browser on the device
• reading a QR code (in which the same link is encoded)
• add ID manually

An interesting point: during the initial registration of the test device, I did not receive a notification by mail that a new device was added and is available for use. However, later, when registering a new device for another account, the service automatically determined that a device with such a UDID is already included in the application’s provision profile and a notification was sent to me with the opportunity to invite a new user to my team:

Another general note: the HockeyApp website often uses special “pop-ups” to report the result of an action. For example, after inviting a new user, a new (temporary) block will appear at the top of the page with the message:


This method can be considered convenient (there is no need to manually close pop-up windows with notifications), but you need to get used to it, because the notification block is inserted inside the container with the main content of the page and when scrolling it can hide under the navigation bar, that is, become invisible.

The service also allows you to import a list of users from a text file:

The content of the file can be as minimalistic (only an e-mail address) or with additional data (name and role).

Note. The role is encoded as follows:

• “1” means Developer
• “2” - Member
• “3” - Tester

You can assign the role of “Owner” (or any other) on the user management page. Learn more about roles here and here .

A new build can be sent either manually via the service website:

Or in semi-automatic mode, if a desktop application is installed - in this case, it will suffice to archive the project (Product-Archive) - then select Upload in the pop-up pop-up:



And then finish the publication, if necessary, setting additional settings in the window that appears:

Note: Release type - maybe Store - in this case, only debugging symbols are uploaded to the service site - the application itself cannot be downloaded from the service client, but the collection and analysis of crash reports will be performed.

Important: the assembly number should always increase! Since, even if the version changes, the build number cannot be used in a value less than what was used before. If this rule is broken, the new build will be considered “old” and, as a result, the automatic notification of the availability of the new version will not work and the “wrong” build will not be at the top of the list in the version history information ( more ) .

Even if the desktop application was not launched when the archive was being prepared, all archived versions can be downloaded from the “Upload New Build” section:

Release notes can be in plain text format or with additional formatting using the popular Markdown markup language.

When downloading a new version of an application through a service site, it is possible to restrict access to this version (using the Restrict downloads option - in the desktop application this option is inactive).

You can also choose to notify all users (to whom this version of the application will be available) or to notify only individual users (when using the desktop application, you can only choose between two options: send an alert or not send):

Note: possible bug: after downloading a new version without notification, the Notify button (Manage version, Status tab) does not lead to any actions.

Also, when downloading, you can indicate that this version is mandatory - in this case, the user will not be able to continue using the older version of the application - a notification about the availability of the update will be shown every time and cannot be waived (provided that the application uses the service framework).

The initial installation of the HockeyApp web application on a mobile device is performed in several steps:



HockeyApp installed web application icon:



The ability to install previous versions of the application is there, but this functionality is not very convenient. It is not available from the application page on the History tab, and in order to use it, go to the Overview section through the menu (top right), select the Versions subsection there, find the version you need, open the page with detailed information and only install there. .



The functionality of sending feedback to HockeyApp is somewhat better implemented in Apple TestFlight, as it allows you to attach up to three attachments to a message (but this is only possible on tablets):



The attachments themselves are placed in the AWS cloud storage, so theoretically it is possible to send both video files and simple screenshots with no restrictions on file size.

Important: switching between the Details and Feedback tabs on the feedback page causes it to reboot. Keep this in mind in order not to lose the text of the typed message.

Immediately after sending a review, it is displayed on the Feedback tab, but it is stored there only temporarily; all messages can be viewed on mobile devices:

• on iPad: Dashboard - Feedback tab
• on iPhone / iPod Touch: detailed application information - top right menu - Overview - Feedback

Note: the ability to view sent messages also depends on the user's role and device type.

The existing differences in the functional for devices of different types I refer to the shortcomings of the service. Perhaps this drawback is temporary and in the future the functionality and interface will be as unified as possible.

Also, the inadequacies of the current implementation include the inability to track changes in the status of a response (Open / Waiting / Resolved / Ignored) for all users except those in the Owner group. This functionality is available to them in the usual / "desktop" version of the website - for users of other groups, the service does not even e-mail notification.

Important: the biggest drawback, in my opinion, is the fact that reviews are not associated with the version of the application, so it must be specified manually if it matters.

Finally, as a plus, I would like to mention the possibility to leave comments on reviews (again, with the support of attachments when working with the service on the tablet).

Rating: 8/10.

Additional functionality

I think all developers agree with the thesis “it’s better to see once than hear a hundred times” in relation to the software testing process. Indeed, often the information provided by testers is either incomplete or erroneous, since it is partially based on guesswork. Therefore, the usual reviews is not enough, and in this case, additional functionality comes to the aid of developers, in particular, “symbolization” of reports about the application crash.

This functionality in HockeyApp is implemented at a very good level. Sending crash reports works right out of the box (of course, if the application uses the service framework). Reports can be sent in a fully automatic mode (after restarting the application), or with a request to send with confirmation by the user (the mode is selected once and can be reset by reinstalling the application).



If a desktop application was used when sending a new version, or debugging symbols (* .dSYM) were additionally loaded with * .ipa, then the service automatically “symbolizes”, allowing you to uniquely identify the class, method and line number that caused the error:

The service also conducts grouping of the received reports, helping to assess the scale / frequency of the problem and therefore prioritize the correction of detected errors (in the screenshot above, the number of reports in the group can be seen in the Count column, in addition to the principles of grouping, you can find out here ).

Moreover, if access to the project repository is configured in the service's desktop application, you can see the “quote” with the code in the detailed information for the crash report, as well as open the class in Xcode:

Additionally, it is possible to download a report of a fall manually, but this can only be done on the website of the service:

Note: recommendations for solving problems with “symbolization” can be found here .

In addition, the service provides the ability to integrate with the bug tracking system (for example, Basecamp or JIRA, the full list is here ).

After the connection with such a system is established, the service can add new entries (tickets) in the automatic mode:


Note: By default, the auto create option is disabled. It is also possible to set the minimum number of reports received in the group, after which the automation is triggered.

Manually creating a new ticket based on error data is only possible on the service website:

As an analytical data, the service offers a graphical representation of information on the number of crash reports, downloads, installations, as well as statistics on the use of a particular version of the application:


The source code for the client part of the SDK github.com/bitstadium is available to everyone - this way the service demonstrates its transparency for Enterprise clients.

Additional links:

• The data collected by the service can be exported via a special API.
• Knowledge base

Rating: 9/10.

Continuous integration

Notes:

1. The following nuances of using services as part of continuous integration (continuous integration) in this review imply that it will be made on the basis of the solution proposed by Apple. That is, with the help of Mac OS X Server and Xcode bots - in detail on the organization of such integration, I will not stop, perhaps it will be a topic for a separate article. Those interested can familiarize themselves with the topic, for example, by referring to Apple’s official guide .

2. Unfortunately at the moment, setting up Apple's CI does not always go smoothly. As a rule, there are problems with signing identity and provision profiles, to solve them in a simple case, it is enough to install the latest version of OS X Server (at the time of publication 4.0.3 for OS X Yosemite) and add an Apple Developer account in the Xcode settings. If the problems remain, then in my case it took:

• add used certificate and key to keychain System
• Copy the used provision profile to the folder “/ Library / Developer / XcodeServer / ProvisioningProfiles /” (error message “No provisioning profiles containing the following signing identities was found”)
• add an exception for codesign in the properties for the key (otherwise, if the bot is configured to create an archive, the integration cannot be completed and will be stopped with the error “Creating archive failed: xcodebuild exited with status 15”, more )

The script for use with the Xcode bots can be in two versions: using curl or using the special utility puck (it can be installed together with the desktop version of the HockeyApp client, details ):

In the second case, you need to create a symlink for the puck: “sudo ln -s / usr / local / bin / puck / usr / bin / puck”, otherwise the bot will not be able to execute the necessary script.



The script itself is added in the last step of the bot creation / configuration wizard:

puck -download=true -notify=true -force=true -submit=auto -app_id=APP_ID -api_token=API_TOKEN "${XCS_ARCHIVE}" APP_ID -      API_TOKEN -       “API Tokens” 

As additional parameters you can specify (including):

 -notes=<text> release notes text (single line only!) -notes_path=<path> absolute path to release notes file -tags=<comma-separated string> restrict version to tags -teams=<comma-separated string> restrict version to team IDs -users=<comma-separated string> restrict version to user IDs 

A complete list of parameters can be obtained by running “puck -help”.

Rating: 9/10.

Total cumulative score for all sections: 33 points.

Part 1. Apple TestFlight . To be continued...