In this article I want to talk about the approach to building Unity-projects on android and ios via Gitlab on my own collectors with macOS.
I work in a small gamedev company, and the task of automating the assembly appeared due to the following problems:
To solve these problems, ready-made solutions have already been created: Unity Cloud Build, TeamCity, Jenkins, Gitlab CI, Bitbucket Pipelines.
The first of them, although prepared for the assembly of Unity-projects, but does not allow to automate the work with certificates, and for each project they have to start manually. TeamCity and Jenkins require customization of the project in the admin area (this makes configuration a bit harder for developers), installing additional software on a separate server and supporting it. As a result, the simplest and fastest in implementation remained two options - Gitlab and Bitbucket.
At the time of solving the problem, Bitbucket Pipelines had not yet announced, so it was decided to use Gitlab.
To implement the approach, the following steps were performed:
Projects that are built on the picker we store on Gitlab. The free version of the service does not limit the repositories themselves and their number.
For each project, a runner is turned on (a service that executes commands from a gitlab server) running on a poppy.
The configuration for the collector lies in the project root in the form of a .gitlab-ci.yml file. It describes the application id, the required signing identity (keystore for android and account name for ios), the required version of Unity, the branch, the launch mode: manual or automatic, and the command that starts the build (if necessary, gitlab supports much more parameters, documentation ) .
variables: BUNDLE: com.banana4apps.evolution SIGNING: banana4apps UNITY_VERSION: 2017.1 build:android: script: - buildAndroid.sh $BUNDLE $SIGNING $UNITY_VERSION only: - releaseAndroid when: manual build:ios: script: - buildIOS.sh $BUNDLE $SIGNING $UNITY_VERSION only: - releaseIOS when: manual
Gitlab CI works with shared and own runners ( documentation ). The free version limits the number of hours of use of shared runners, but allows you to use your own runners unlimitedly. Shared runners run on linux, so iOS applications cannot be assembled on them (but it will work to launch Unity, there was an article about this in Habré). Because of this, I had to raise runners on my own poppies. In the example above, the runner runs the buildAndroid.sh or buildIOS.sh script (depending on the branch), which describes the preparatory steps, the launch of Unity and the notification of the build result.
The process of setting up a runner is well described in the documentation and comes down to running gitlab-runner install
and gitlab-runner start
.
After that, the necessary Unity versions are installed on the Mac.
For each of the platforms, due to differences in the build process, I had to write my own script. But the algorithm is the same:
A feature of the Unity project build is that Unity in batch mode allows you to perform only the static method of the class present in the project. Therefore, the build script “throws” into the project a class with methods for running the build:
public class CustomBuild { static string outputProjectsFolder = Environment.GetEnvironmentVariable("OutputDirectory"); static string xcodeProjectsFolder = Environment.GetEnvironmentVariable("XcodeDirectory"); static void BuildAndroid() { BuildTarget target = BuildTarget.Android; EditorUserBuildSettings.SwitchActiveBuildTarget(target); PlayerSettings.applicationIdentifier = Environment.GetEnvironmentVariable("AppBundle"); PlayerSettings.Android.keystoreName = Environment.GetEnvironmentVariable("KeystoreName"); PlayerSettings.Android.keystorePass = Environment.GetEnvironmentVariable("KeystorePassword"); PlayerSettings.Android.keyaliasName = Environment.GetEnvironmentVariable("KeyAlias"); PlayerSettings.Android.keyaliasPass = Environment.GetEnvironmentVariable("KeyPassword"); BuildPipeline.BuildPlayer(GetScenes(), string.Format("{0}/{1}.apk" , outputProjectsFolder, PlayerSettings.applicationIdentifier), target, options); } static void BuildIOS() { BuildTarget target = BuildTarget.iOS; EditorUserBuildSettings.SwitchActiveBuildTarget(target); PlayerSettings.applicationIdentifier = Environment.GetEnvironmentVariable("AppBundle"); PlayerSettings.iOS.appleDeveloperTeamID = Environment.GetEnvironmentVariable("GymTeamId"); BuildPipeline.BuildPlayer(GetScenes(), xcodeProjectsFolder, target, options); } // static string[] GetScenes() { var projectScenes = EditorBuildSettings.scenes; List<string> scenesToBuild = new List<string>(); for (int i = 0; i < projectScenes.Length; i++) { if (projectScenes[i].enabled) { scenesToBuild.Add(projectScenes[i].path); } } return scenesToBuild.ToArray(); } }
The Environment.GetEnvironmentVariable method gets the value of environment variables that were previously specified in bash scripts.
Sample build script for Android
GREEN='\033[0;32m' RED='\033[0;33m' NC='\033[0m' # No Color export COMMIT=$(git log -1 --oneline —no-merges) if [ "$1" = "" ]; then echo -e "${RED}You must provide application Id${NC}" exit 1 fi export ANDROID_HOME=/Library/Android export OutputDirectory=./ export AppBundle=$1 if [ "$2" = "account1" ]; then export KeystoreName="$CI_DATA_PATH/keystores/account1.keystore" export KeystorePassword="..." export KeyAlias="..." export KeyPassword="..." elif [ "$2" = "account2" ]; then export KeystoreName="$CI_DATA_PATH/keystores/account2.keystore" export KeystorePassword="..." export KeyAlias="..." export KeyPassword="..." else echo "${RED}No keystore config found for $2${NC}" exit 1 fi echo -e "${GREEN}BundleId: ${AppBundle}${NC}" echo -e "${GREEN}Signing: ${KeyAlias}${NC}" # mkdir -p $CI_PROJECT_DIR/Assets/Editor && cp $CI_DATA_PATH/CustomBuild.cs "$_" # Unity if [ "$3" = "5.5" ]; then /Applications/Unity5.5/Unity.app/Contents/MacOS/Unity -buildTarget android -projectPath $CI_PROJECT_DIR -batchmode -executeMethod CustomBuild.Android -quit -logFile /dev/stdout -username "..." -password "..." elif [ "$3" = "2017.1" ]; then /Applications/Unity2017.1/Unity.app/Contents/MacOS/Unity -buildTarget android -projectPath $CI_PROJECT_DIR -batchmode -executeMethod CustomBuild.Android -quit -logFile /dev/stdout -username "..." -password "..." else /Applications/Unity5.6.4/Unity.app/Contents/MacOS/Unity -buildTarget android -projectPath $CI_PROJECT_DIR -batchmode -executeMethod CustomBuild.Android -quit -logFile /dev/stdout -username "..." -password "..." fi # , apk export APK="${CI_PROJECT_DIR}/${OutputDirectory}/${AppBundle}.${CI_BUILD_ID}.apk" echo "Testing apk exists: ${APK}..." if [ -f ${APK} ]; then echo -e "${GREEN}BUILD FOR ANDROID SUCCESS${NC}" # apk aws s3 cp ${APK} s3://ci-data/android/${AppBundle}.${CI_BUILD_ID}.apk --grants read=uri=http://acs.amazonaws.com/groups/global/AllUsers echo "<html><title>Download apk: ${AppBundle}</title><body><a href=\"https://ci-data.s3.amazonaws.com/android/${AppBundle}.${CI_BUILD_ID}.apk\">Install<br><br><strong>${AppBundle}</strong><br><small>${COMMIT}<br>(build ${CI_BUILD_ID} - android)</small></a></body></html>" >> ${CI_PROJECT_DIR}/download.html # html aws s3 cp ${CI_PROJECT_DIR}/download.html s3://ci-data/android/${AppBundle}.${CI_BUILD_ID}.html --grants read=uri=http://acs.amazonaws.com/groups/global/AllUsers # Slack ${CI_DATA_PATH}/notifySlack.sh android success "https://ci-data.s3.amazonaws.com/android/${AppBundle}.${CI_BUILD_ID}.html" exit 0 else echo -e "${RED}BUILD FOR ANDROID FAILED${NC}" ${CI_DATA_PATH}/notifySlack.sh android failure exit 1 fi
Sample build script for iOS
Building projects is carried out in two steps: the formation of the Xcode project from Unity and the assembly of the Xcode project. Developers can not directly influence the Xcode project, which introduces restrictions: you can not directly change the project settings, build information.
Also, the build feature on iOS is that test devices must be registered in the provisioning profile of the application. And in order to build an Xcode project, you need to create a certificate, provisioning profile and application id in the Apple developer console before assembly.
To automate this process is used fastlane . This tool creates and synchronizes certificates, profiles and allows you to upload builds and meta-information to itunes connect.
When building Unity projects without access to Xcode, there are nuances:
GREEN='\033[0;32m' RED='\033[0;33m' NC='\033[0m' # No Color export COMMIT=$(git log -1 --oneline --no-merges) if [ "$1" = "" ]; then echo -e "${RED}You must provide application Id${NC}" exit 1 fi if [ "$2" = "account1" ]; then # fastlane export AccountName="account email" export AccountDesc="account description" export FastlanePassword="..." export GymExportTeamId="..." export FastlaneRepository="fastlane-keys.git" export ProduceTeamName="team name" else echo "${RED}No keystore config found for $2${NC}" exit 1 fi echo -e "${GREEN}BundleId: ${AppBundle}${NC}" echo -e "${GREEN}Account: ${AccountDesc} (${2})${NC}" # mkdir -p $CI_PROJECT_DIR/Assets/Editor && cp $CI_DATA_PATH/CustomBuild.cs "$_" # Unity if [ "$3" = "5.5" ]; then /Applications/Unity5.5/Unity.app/Contents/MacOS/Unity -buildTarget ios -projectPath $CI_PROJECT_DIR -batchmode -executeMethod CustomBuild.IOS -quit -logFile /dev/stdout -username "..." -password "..." elif [ "$3" = "2017.1" ]; then /Applications/Unity2017.1/Unity.app/Contents/MacOS/Unity -buildTarget ios -projectPath $CI_PROJECT_DIR -batchmode -executeMethod CustomBuild.IOS -quit -logFile /dev/stdout -username "..." -password "..." else /Applications/Unity5.6.4/Unity.app/Contents/MacOS/Unity -buildTarget ios -projectPath $CI_PROJECT_DIR -batchmode -executeMethod CustomBuild.IOS -quit -logFile /dev/stdout -username "..." -password "..." fi # , Unity XCode XCODE_FILES="${CI_PROJECT_DIR}/${XcodeDirectory}" if [ -d ${XCODE_FILES} ]; then # Apple Developer Console export PRODUCE_APP_IDENTIFIER=${AppBundle} export PRODUCE_APP_NAME=${AppBundle} export PRODUCE_USERNAME=${AccountName} export PRODUCE_SKU=${AppBundle} # skip_itc itunes connect - adhoc fastlane produce --app_version "1.0" --language "English" --skip_itc # code signing keys and profiles cd "${CI_PROJECT_DIR}/${XcodeDirectory}" rm -f Matchfile echo "git_url \"${FastlaneRepository}\"" >> Matchfile echo "app_identifier [\"${AppBundle}\"]" >> Matchfile echo "username \"${AccountName}\"" >> Matchfile # , export MATCH_PASSWORD='...' # , # force_for_new_devices true , developer console fastlane match adhoc --force_for_new_devices true # Gymfile XCode project Ad-Hoc rm -f Gymfile echo "export_options(" >> Gymfile echo " manifest: {" >> Gymfile echo " appURL: \"https://ci-data.s3.amazonaws.com/ios/${AppBundle}.${CI_BUILD_ID}.ipa\"," >> Gymfile echo " displayImageURL: \"https://ci-data.s3.amazonaws.com/ios-icon.png\"," >> Gymfile echo " fullSizeImageURL: \"https://ci-data.s3.amazonaws.com/ios-icon-big.png\"" >> Gymfile echo " }," >> Gymfile echo ")" >> Gymfile fastlane gym --scheme "Unity-iPhone" --export_method ${GYM_EXPORT_METHOD} --xcargs "DEVELOPMENT_TEAM=\"${GYM_EXPORT_TEAM_ID}\" PROVISIONING_PROFILE_SPECIFIER=\"match AdHoc ${AppBundle}\" CODE_SIGN_IDENTITY=\"iPhone Distribution: ${AccountDesc}\"" -o "${CI_PROJECT_DIR}/" -n "${AppBundle}.${CI_BUILD_ID}.ipa" # S3 export IPA="${CI_PROJECT_DIR}/${AppBundle}.${CI_BUILD_ID}.ipa" ls -l "${CI_PROJECT_DIR}/${XcodeDirectory}/*.ipa" echo "Testing ipa exists: ${IPA}..." if [ -f ${IPA} ]; then echo -e "Begin uploading to S3..." aws s3 cp ${CI_PROJECT_DIR}/${AppBundle}.${CI_BUILD_ID}.ipa s3://ci-data/ios/${AppBundle}.${CI_BUILD_ID}.ipa --grants read=uri=http://acs.amazonaws.com/groups/global/AllUsers aws s3 cp ${CI_PROJECT_DIR}/manifest.plist s3://ci-data/ios/${AppBundle}.${CI_BUILD_ID}.plist --grants read=uri=http://acs.amazonaws.com/groups/global/AllUsers echo "<html><title>Download ipa: ${AppBundle}</title>" >> ${CI_PROJECT_DIR}/download.html echo "<body><a href=\"itms-services://?action=download-manifest&url=https://ci-data.s3.amazonaws.com/ios/${AppBundle}.${CI_BUILD_ID}.plist\">Install<br><br><strong>${AppBundle}</strong><br><small>${COMMIT}<br>(build ${CI_BUILD_ID} - iOS)</small></a></body></html>" >> ${CI_PROJECT_DIR}/download.html aws s3 cp ${CI_PROJECT_DIR}/download.html s3://ci-data/ios/${AppBundle}.${CI_BUILD_ID}.html --grants read=uri=http://acs.amazonaws.com/groups/global/AllUsers ${CI_DATA_PATH}/notifySlack.sh ios ad-hoc "https://ci-data.s3.amazonaws.com/ios/${AppBundle}.${CI_BUILD_ID}.html" echo -e "${GREEN}BUILD AD-HOC FOR IOS SUCCESS${NC}" exit 0 else echo -e "${RED}BUILD AD-HOC FOR IOS FAILED${NC}" ${CI_DATA_PATH}/notifySlack.sh ios failure exit 1 fi else echo -e "${RED}BUILD FOR IOS FAILED${NC}" ${CI_DATA_PATH}/notifySlack.sh ios failure exit 1 fi
GREEN='\033[0;32m' RED='\033[0;33m' NC='\033[0m' # No Color export COMMIT=$(git log -1 --oneline --no-merges) if [ "$1" = "" ]; then echo -e "${RED}You must provide application Id${NC}" exit 1 fi if [ "$2" = "account1" ]; then # fastlane export AccountName="account email" export AccountDesc="account description" export FastlanePassword="..." export GymExportTeamId="..." export FastlaneRepository="fastlane-keys.git" export ProduceTeamName="team name" else echo "${RED}No keystore config found for $2${NC}" exit 1 fi echo -e "${GREEN}BundleId: ${AppBundle}${NC}" echo -e "${GREEN}Account: ${AccountDesc} (${2})${NC}" # mkdir -p $CI_PROJECT_DIR/Assets/Editor && cp $CI_DATA_PATH/CustomBuild.cs "$_" # Unity if [ "$3" = "5.5" ]; then /Applications/Unity5.5/Unity.app/Contents/MacOS/Unity -buildTarget ios -projectPath $CI_PROJECT_DIR -batchmode -executeMethod CustomBuild.IOS -quit -logFile /dev/stdout -username "..." -password "..." elif [ "$3" = "2017.1" ]; then /Applications/Unity2017.1/Unity.app/Contents/MacOS/Unity -buildTarget ios -projectPath $CI_PROJECT_DIR -batchmode -executeMethod CustomBuild.IOS -quit -logFile /dev/stdout -username "..." -password "..." else /Applications/Unity5.6.4/Unity.app/Contents/MacOS/Unity -buildTarget ios -projectPath $CI_PROJECT_DIR -batchmode -executeMethod CustomBuild.IOS -quit -logFile /dev/stdout -username "..." -password "..." fi # , Unity XCode XCODE_FILES="${CI_PROJECT_DIR}/${XcodeDirectory}" if [ -d ${XCODE_FILES} ]; then # Apple Developer Console and Itunes Connect export PRODUCE_APP_IDENTIFIER=${AppBundle} export PRODUCE_APP_NAME=${AppBundle} export PRODUCE_USERNAME=${AccountName} export PRODUCE_SKU=${AppBundle} fastlane produce --app_version "1.0" --language "English" # code signing keys and profiles cd "${CI_PROJECT_DIR}/${XcodeDirectory}" rm -f Matchfile echo "git_url \"${FastlaneRepository}\"" >> Matchfile echo "app_identifier [\"${AppBundle}\"]" >> Matchfile echo "username \"${AccountName}\"" >> Matchfile # , export MATCH_PASSWORD='...' # fastlane match appstore # XCode fastlane gym --scheme "Unity-iPhone" --xcargs "DEVELOPMENT_TEAM=\"${GymExportTeamId}\" PROVISIONING_PROFILE_SPECIFIER=\"match AppStore ${AppBundle}\" CODE_SIGN_IDENTITY=\"iPhone Distribution: ${AccountDesc}\"" -o "${CI_PROJECT_DIR}/" -n "${AppBundle}.${CI_BUILD_ID}.ipa" # itunes connect export IPA="${CI_PROJECT_DIR}/${AppBundle}.${CI_BUILD_ID}.ipa" ls -l "${CI_PROJECT_DIR}/${XcodeDirectory}/*.ipa" echo "Testing ipa exists: ${IPA}..." if [ -f ${IPA} ]; then rm -f Deliverfile echo "app_identifier \"${AppBundle}\"" >> Deliverfile echo "username \"${AccountName}\"" >> Deliverfile echo "ipa \"${IPA}\"" >> Deliverfile echo "submit_for_review false" >> Deliverfile echo "force true" >> Deliverfile fastlane deliver echo -e "${GREEN}BUILD FOR IOS SUCCESS${NC}" exit 0 else echo -e "${RED}BUILD FOR IOS FAILED${NC}" ${CI_DATA_PATH}/notifySlack.sh ios failure exit 1 fi else echo -e "${RED}BUILD FOR IOS FAILED${NC}" ${CI_DATA_PATH}/notifySlack.sh ios failure exit 1 fi
--xcargs "DEVELOPMENT_TEAM = \" $ {GymExportTeamId} \ "PROVISIONING_PROFILE_SPECIFIER = \" match AppStore $ {AppBundle} \ "CODE_SIGN_IDENTITY = \" iPhone Distribution: $ {AccountDesc} \ "" GREEN='\033[0;32m' RED='\033[0;33m' NC='\033[0m' # No Color export COMMIT=$(git log -1 --oneline --no-merges) if [ "$1" = "" ]; then echo -e "${RED}You must provide application Id${NC}" exit 1 fi if [ "$2" = "account1" ]; then # fastlane export AccountName="account email" export AccountDesc="account description" export FastlanePassword="..." export GymExportTeamId="..." export FastlaneRepository="fastlane-keys.git" export ProduceTeamName="team name" else echo "${RED}No keystore config found for $2${NC}" exit 1 fi echo -e "${GREEN}BundleId: ${AppBundle}${NC}" echo -e "${GREEN}Account: ${AccountDesc} (${2})${NC}" # mkdir -p $CI_PROJECT_DIR/Assets/Editor && cp $CI_DATA_PATH/CustomBuild.cs "$_" # Unity if [ "$3" = "5.5" ]; then /Applications/Unity5.5/Unity.app/Contents/MacOS/Unity -buildTarget ios -projectPath $CI_PROJECT_DIR -batchmode -executeMethod CustomBuild.IOS -quit -logFile /dev/stdout -username "..." -password "..." elif [ "$3" = "2017.1" ]; then /Applications/Unity2017.1/Unity.app/Contents/MacOS/Unity -buildTarget ios -projectPath $CI_PROJECT_DIR -batchmode -executeMethod CustomBuild.IOS -quit -logFile /dev/stdout -username "..." -password "..." else /Applications/Unity5.6.4/Unity.app/Contents/MacOS/Unity -buildTarget ios -projectPath $CI_PROJECT_DIR -batchmode -executeMethod CustomBuild.IOS -quit -logFile /dev/stdout -username "..." -password "..." fi # , Unity XCode XCODE_FILES="${CI_PROJECT_DIR}/${XcodeDirectory}" if [ -d ${XCODE_FILES} ]; then # Apple Developer Console and Itunes Connect export PRODUCE_APP_IDENTIFIER=${AppBundle} export PRODUCE_APP_NAME=${AppBundle} export PRODUCE_USERNAME=${AccountName} export PRODUCE_SKU=${AppBundle} fastlane produce --app_version "1.0" --language "English" # code signing keys and profiles cd "${CI_PROJECT_DIR}/${XcodeDirectory}" rm -f Matchfile echo "git_url \"${FastlaneRepository}\"" >> Matchfile echo "app_identifier [\"${AppBundle}\"]" >> Matchfile echo "username \"${AccountName}\"" >> Matchfile # , export MATCH_PASSWORD='...' # fastlane match appstore # XCode fastlane gym --scheme "Unity-iPhone" --xcargs "DEVELOPMENT_TEAM=\"${GymExportTeamId}\" PROVISIONING_PROFILE_SPECIFIER=\"match AppStore ${AppBundle}\" CODE_SIGN_IDENTITY=\"iPhone Distribution: ${AccountDesc}\"" -o "${CI_PROJECT_DIR}/" -n "${AppBundle}.${CI_BUILD_ID}.ipa" # itunes connect export IPA="${CI_PROJECT_DIR}/${AppBundle}.${CI_BUILD_ID}.ipa" ls -l "${CI_PROJECT_DIR}/${XcodeDirectory}/*.ipa" echo "Testing ipa exists: ${IPA}..." if [ -f ${IPA} ]; then rm -f Deliverfile echo "app_identifier \"${AppBundle}\"" >> Deliverfile echo "username \"${AccountName}\"" >> Deliverfile echo "ipa \"${IPA}\"" >> Deliverfile echo "submit_for_review false" >> Deliverfile echo "force true" >> Deliverfile fastlane deliver echo -e "${GREEN}BUILD FOR IOS SUCCESS${NC}" exit 0 else echo -e "${RED}BUILD FOR IOS FAILED${NC}" ${CI_DATA_PATH}/notifySlack.sh ios failure exit 1 fi else echo -e "${RED}BUILD FOR IOS FAILED${NC}" ${CI_DATA_PATH}/notifySlack.sh ios failure exit 1 fi
Interface for viewing the build logs:
Thus, the resulting system is easy to use, allows you to add checks and validations from the server (code style, tests), while managers see links to assemblies in Slack and there are no problems with assembling on iOS.
From minuses - its support is necessary for adding new versions of Unity, signing identity and ensuring the health of poppies.
At the moment we have two runners (about two years), more than 4000 assemblies have passed through the system. The build speed depends on the runner’s characteristics and the number of assets in the project, because they are re-imported every time and it varies from 3 to 30 minutes for Android and 10 to 60 for iOS.
Source: https://habr.com/ru/post/358448/
All Articles