Work with external resources in Unity3D

Introduction

Hello dear readers, today it will be about working with external resources in the Unity 3d environment.

By tradition, to begin with, we will define what it is and why we need it. So, what are these external resources. As part of game development, such resources can be all that is required for the operation of the application and should not be stored in the final build of the project. External resources can be located either on the hard disk of the user's computer or on an external web server. In general, such resources are any file or data set that we load into our already running application. If we talk in the framework of Unity 3d, then they can be:

• Text file
• Texture file
• Audio file
• Byte array
• AssetBundle (archive with assets of Unity 3d project)

Below, we will take a closer look at the built-in mechanisms for working with these resources, which are present in Unity 3d, and also write simple managers to interact with the web server and download resources to the application.
')
Note : The article uses code using C # 7+ and is designed for the Roslyn compiler used in Unity3d in versions 2018.3+.

Unity 3d features

Prior to the 2017 version of Unity, one mechanism was used to work with server data and external resources (excluding the self-written ones) that was included in the engine - this is the WWW class. This class allowed using various http commands (get, post, put, etc.) in a synchronous or asynchronous form (via Coroutine). Work with this class was fairly simple and straightforward.

IEnumerator LoadFromServer(string url) { var www = new WWW(url); yield return www; Debug.Log(www.text); } 

Similarly, you can get not only text data, but also others:

• Byte array - www.bytes
• Texture - www.texture
• Audio - www.GetAudioClip ()
• Asset - www.assetBundle

However, starting with version 2017, Unity has a new system for working with the server, represented by the UnityWebRequest class, which is located in the Networking namespace. Prior to Unity 2018, it existed along with the WWW , but in the latest version of the WWW engine, it was not recommended, and will later be completely removed. Therefore, the following discussion will deal only with UnityWebRequest (hereinafter UWR).

Working with the UWR in general is similar to the WWW at its core, but there are also differences, which will be discussed further. Below is a similar example of loading text.

 IEnumerator LoadFromServer(string url) { var request = new UnityWebRequest(url); yield return request.SendWebRequest(); Debug.Log(request.downloadHandler.text); request.Dispose(); } 

The main changes introduced by the new UWR system (in addition to changes in the principle of operation inside) are the ability to assign handlers to download and download data from the server itself, read more here . By default, these are the UploadHandler and DownloadHandler classes. Unity itself provides a set of extensions of these classes for working with various data, such as audio, textures, assets, etc. Let us consider the work with them.

Work with resources

Text

Working with text is one of the easiest options. Above has been described a way to download it. Rewrite it a bit using the creation of a direct http Get request.

 IEnumerator LoadTextFromServer(string url, Action<string> response) { var request = UnityWebRequest.Get(url); yield return request.SendWebRequest(); if (!request.isHttpError && !request.isNetworkError) { response(uwr.downloadHandler.text); } else { Debug.LogErrorFormat("error request [{0}, {1}]", url, request.error); response(null); } request.Dispose(); } 

As you can see from the code, DownloadHandler is used here by default. The text property is a getter that converts a byte array to text in UTF8 encoding. The main use of loading text from a server is getting a json file (a serialized representation of data in text form). You can get this data using the Unity JsonUtility class.

 var data = JsonUtility.FromJson<T>(value); // T  ,    . 

Audio

To work with audio, you need to use a special method to create a UnityWebRequestMultimedia.GetAudioClip request, and to get a presentation of the data you need to work in Unity, you need to use DownloadHandlerAudioClip . In addition, when creating a request, you must specify the type of audio data, represented by the AudioType enumeration, which specifies the format (wav, aiff, oggvorbis, etc.).

 IEnumerator LoadAudioFromServer(string url, AudioType audioType, Action<AudioClip> response) { var request = UnityWebRequestMultimedia.GetAudioClip(url, audioType); yield return request.SendWebRequest(); if (!request.isHttpError && !request.isNetworkError) { response(DownloadHandlerAudioClip.GetContent(request)); } else { Debug.LogErrorFormat("error request [{0}, {1}]", url, request.error); response(null); } request.Dispose(); } 

Texture

Loading textures is similar to that for audio files. The request is created using UnityWebRequestTexture.GetTexture . To get the data in the right form for Unity, DownloadHandlerTexture is used.

 IEnumerator LoadTextureFromServer(string url, Action<Texture2D> response) { var request = UnityWebRequestTexture.GetTexture(url); yield return request.SendWebRequest(); if (!request.isHttpError && !request.isNetworkError) { response(DownloadHandlerTexture.GetContent(request)); } else { Debug.LogErrorFormat("error request [{0}, {1}]", url, request.error); response(null); } request.Dispose(); } 

Assetbundle

As mentioned earlier, a bundle is, in fact, an archive with Unity resources that can be used in an already running game. These resources can be any project assets, including scenes. The exception is C # scripts, they can not be transferred. To load AssetBundle, use a request that is created using UnityWebRequestAssetBundle.GetAssetBundle. To get the data in the right form for Unity, DownloadHandlerAssetBundle is used.

 IEnumerator LoadBundleFromServer(string url, Action<AssetBundle> response) { var request = UnityWebRequestAssetBundle.GetAssetBundle(url); yield return request.SendWebRequest(); if (!request.isHttpError && !request.isNetworkError) { response(DownloadHandlerAssetBundle.GetContent(request)); } else { Debug.LogErrorFormat("error request [{0}, {1}]", url, request.error); response(null); } request.Dispose(); } 

Main problems and solutions when working with a web server and external data

Above, simple ways of application interaction with the server were described in terms of downloading various resources. However, in practice, everything is much more complicated. Consider the main problems that accompany the developers and dwell on ways to solve them.

Not enough space

One of the first problems when downloading data from a server is the possible lack of free space on the device. It often happens that the user uses old devices for games (especially on Android), as well as the size of downloaded files can be quite large (hi PC). In any case, this situation must be properly handled and the player should be informed in advance that there is not enough space and how much. How to do it? The first thing you need to know the size of the file being downloaded, this is done by means of the UnityWebRequest.Head () request. Below is the code to get the size.

 IEnumerator GetConntentLength(string url, Action<int> response) { var request = UnityWebRequest.Head(url); yield return request.SendWebRequest(); if (!request.isHttpError && !request.isNetworkError) { var contentLength = request.GetResponseHeader("Content-Length"); if (int.TryParse(contentLength, out int returnValue)) { response(returnValue); } else { response(-1); } } else { Debug.LogErrorFormat("error request [{0}, {1}]", url, request.error); response(-1); } } 

Here it is important to note one thing, for the request to work properly, the server should be able to return the size of the content, otherwise (as, in fact, to display progress) the wrong value will be returned.

After we received the size of the downloaded data, we can compare it with the size of free disk space. For the latter, I use the free plugin from the Asset Store .

Note : You can use the Cache class in Unity3d, it can show free and occupied space in the cache. However, it is worth considering the moment that these data are relative. They are calculated based on the size of the cache itself, by default it is 4GB. If the user has more free space than the cache size, then there will be no problems, but if this is not the case, then the values ​​may assume incorrect values ​​relative to the real state of affairs.

Internet access check

Very often, before downloading something from the server, it is necessary to handle the situation of lack of access to the Internet. There are several ways to do this: from pinging an address to a GET request to google.com. However, in my opinion, the most correct and giving a fast and stable result is downloading from a small server (the same one from which files will download). How to do this is described above in the text section.
In addition to checking the very fact of having access to the Internet, it is also necessary to determine its type (mobile or WiFi), because the player is unlikely to want to download a few hundred megabytes on mobile traffic. This can be done through the Application.internetReachability property.

Caching

Next, and one of the most important problems, is the caching of downloaded files. Why do we need this caching:

1. Saving traffic (do not download already downloaded data)
2. Providing work in the absence of the Internet (you can show the data from the cache).

What do you need to cache? The answer to this question is everything, all the files that you download need to be cached. How to do this, consider below, and start with simple text files.
Unfortunately, in Unity there is no built-in mechanism for caching text, as well as textures and audio files. Therefore, for these resources, you need to write your system, or not to write, depending on the needs of the project. In the simplest version, we simply write the file to the cache and in the absence of the Internet we take the file from it. In a slightly more complicated version (I use it in projects), we send a request to the server, which is returned by json with the versions of files that are stored on the server. You can write and read files from the cache using the C # File class or any other method convenient and accepted by your team.

 private void CacheText(string fileName, string data) { var cacheFilePath = Path.Combine("CachePath", "{0}.text".Fmt(fileName)); File.WriteAllText(cacheFilePath, data); } private void CacheTexture(string fileName, byte[] data) { var cacheFilePath = Path.Combine("CachePath", "{0}.texture".Fmt(fileName)); File.WriteAllBytes(cacheFilePath, data); } 

Similarly, retrieving data from the cache.

 private string GetTextFromCache(string fileName) { var cacheFilePath = Path.Combine(Utils.Path.Cache, "{0}.text".Fmt(fileName)); if (File.Exists(cacheFilePath)) { return File.ReadAllText(cacheFilePath); } return null; } private Texture2D GetTextureFromCache(string fileName) { var cacheFilePath = Path.Combine(Utils.Path.Cache, "{0}.texture".Fmt(fileName)); Texture2D texture = null; if (File.Exists(cacheFilePath)) { var data = File.ReadAllBytes(cacheFilePath); texture = new Texture2D(1, 1); texture.LoadImage(data, true); } return texture; } 

Note : why the same UWR is not used for loading textures from a url of the form file: //. At the moment, there is a problem with this, the file just simply does not load, so I had to find a workaround.

Note : I do not use AudioClip direct download in projects; I store all such data in AssetBundle. However, if necessary, this is easily done using the functions of the AudioClip class GetData and SetData.

Unlike simple resources for AssetBundle , Unity has a built-in caching mechanism. Consider it in more detail.

Basically, this mechanism can use two approaches:

1. Using CRC and version numbers
2. Using Hash Values

In principle, you can use any of them, but I decided for myself that Hash is the most acceptable, because I have my own version system and it takes into account not only the AssetBundle version, but also the application version, since often the bundle may not be compatible with the version presented in stores.

So, how is caching performed:

1. We request a bundle file from the server manifest (this file is created automatically when it is created and contains a description of the assets that it contains, as well as the values ​​of hash, crc, size, etc.). The file has the same name as the bundle plus the .manifest extension.
2. Get from hash value hash128
3. Create a request to the server to get AssetBundle, where, in addition to the url, we specify the resulting value hash128

In the above example, when requested by the Unity server, it first looks to see if there is a file in the cache with the specified hash128 value, if there is, it will be returned, if not, the updated file will be loaded. To manage all the cache files in Unity there is a class Caching , with which we can find out if there is a file in the cache, get all cached versions, and also delete unnecessary ones, or clear it completely.

Note : why such a strange way of getting hash values? This is due to the fact that getting the hash128 in the manner described in the documentation requires downloading the entire bundle and then getting the AssetBundleManifest asset from it and from there hash values. The disadvantage of this approach is that the whole AssetBundle is swinging, and we just need it not to be. Therefore, we first download only the manifest file from the server, pick up the hash128 from it, and only then, if we need to download the bundle file, while pulling out the hash128 value will have to be done through the interpretation of the lines.

Work with resources in editor mode

The last problem, or rather the issue of ease of debugging and development, is working with loaded resources in editor mode, if there are no problems with regular files, then with bundles is not so simple. You can, of course, do their builds every time, upload them to the server and launch the application in the Unity editor and watch how everything works, but it sounds like a “crutch” even by description. With this we need to do something and for this we will help the class AssetDatabase .

In order to unify the work with the bundles, I made a special wrapper:

 public class AssetBundleWrapper { private readonly AssetBundle _assetBundle; public AssetBundleWrapper(AssetBundle assetBundle) { _assetBundle = assetBundle; } } 

Now we need to add two modes of work with assets depending on whether we are in the editor or in the build. For the build, we use wrappers on AssetBundle class functions , and for the editor, we use the AssetDatabase class mentioned above.


Note : the code uses the TaskManager class, it will be discussed below, if briefly, this is a wrapper for working with Coroutine .

In addition to the above, it is also useful during development to look at what we downloaded and what is now in the cache. For this purpose, you can use the option to install your own folder, which will be used for caching (you can also write downloaded text and other files to the same folder):

 #if UNITY_EDITOR var path = Path.Combine(Directory.GetParent(Application.dataPath).FullName, "_EditorCache"); #else var path = Path.Combine(Application.persistentDataPath, "_AppCache"); #endif Caching.currentCacheForWriting = Caching.AddCache(path); 

Writing a network request manager or working with a web server

Above, we covered the main aspects of working with external resources in Unity, now I would like to dwell on the implementation of the API, which summarizes and unifies all the above. And first we will focus on the network request manager.

Note : hereinafter the wrapper is used over Coroutine in the form of TaskManager class. I wrote about this wrapper in another article .

Let's get the corresponding class:

 public class Network { public enum NetworkTypeEnum { None, Mobile, WiFi } public static NetworkTypeEnum NetworkType; private readonly TaskManager _taskManager = new TaskManager(); } 

The static NetworkType field is required for an application to receive information about the type of Internet connection. In principle, this value can be stored anywhere, I decided that in the Network class it belongs to it.


As can be seen from the code, the way the request is processed is changed, compared to the code in the previous sections. This is done to show the progress of data loading. Also, all sent requests are saved in the list so that, if necessary, they can be canceled.


Similarly, functions for texture, audio, text, byte array are created.

Now you need to ensure that the server sends data through the Post command. Often you need to send something to the server, and depending on what it is, get an answer. Add the appropriate functions.




Similarly, methods are added for texture, audio file, text, etc.


This is where our network request manager is complete. Of necessity, each game subsystem that requires working with a server can create its own class instances.

We write the manager of loading of external resources

In addition to the class described above, in order to fully work with external data, we need a separate manager who will not only download data, but also notify the application about the start of loading, completion, progress, lack of free space, and also deal with caching issues.


, . , Network, .

, . AssetBundle, .



, :

• DONT_USE_SERVER_IN_EDITOR
• - , , - ( _externalResourcesStorage ) , , ( , ), , null
• , Caching , , ( )
• , , , , - ( ). , ( )

: -. , , , - , , .

/ : GetJson, GetTexture, GetText, GetAudio ..


TaskManager , , , . .

: , Coroutine , async/await , , ( ).

Conclusion

. , . , ( f2p ), , , , , .

, :
assetstore.unity.com/packages/tools/simple-disk-utils-59382
habr.com/post/352296
habr.com/post/282524