Binary Serialization in Unity 3D / Visual Studio Application
In the process of developing a plug-in for Unity 3D, it was necessary to make storage of a relatively large amount of data. In my case, this is the storage of node data for visual programming (also applicable to the save game implementation). The storage method must meet the specified requirements:
As a result, I stopped at binary serialization. This method meets all specified requirements, but makes it impossible to view and edit already serialized data in a text editor. But this is not a problem, since the program for editing is intended for this.
The first task was to do serialization and de-serialization of data in the program. I wrote a simple program that will edit and serialize the data of the nodes in the custom Nodes class as (ID, Object) in the Dictionary <short, data> collection. There will be a lot of objects, so the node ID will be stored in a 16-bit short data type.
Class Nodes, to begin with will be the easiest. Mark it as Serializable.
(program code at the end of the article)
')
The newly created node should be added to the first free position in the collection, for this I used the code:
Two steps that need to be performed at this stage are to force the serializer to work with our NodesV1 class and take into account that the data structure of the serializable / deserialized object will change (it will change more than once during the development process).
The second step is not necessary, but if you change the structure, deserialize the file with the previous structure does not work (but in some cases, if you add new data to the end, then the old file usually passes without problems).
First you need a class that will work on serialization / deserialization, in it we will force the serializer to work with our class.
We will save the file to the Unity project folder: Assets \ Resources. It is from the Resources folder that it will work correctly on Unity reading a file on mobile devices, etc.
Now step two, resolve the issue with the deserializer version. In the first two bytes of the binary file, we will write the version of the serializer. During deserialization, we read the version, remove the two bytes and start the corresponding version deserializer. The serializer version will be determined by the numbers at the end of the class name (NodesV1 - version ”1”).
Add a version check:
Now we will create several nodes in the program and start the serialization. We still need the resulting file.
Now check if it works. Suppose our structure has changed, we have added the Permission (Perm) variable. Create a class with a new structure:
We change the NodesV1 class to NodesV2 in the program code. At startup, we also initialize a new variable:
Now the fun part. In the file with the old data structure, there is no Perm variable, and we need to deserialize according to the old structure and return it to the new one.
In each case, there will be its own handling of this situation, but I will simply create this collection with false values.
Let's change the version verification code in the deserializer:
After the changes, the deserialization of the file with the old structure is successful.
Create a C # script that deserializes the binary and will display the name and text of the node in the GUI. You can also change this data and serialize it back.
As you can see, the class code that handles the serialization is the same, only instead:
we will use:
If the script is not planned to run on mobile devices (or similar), you can not touch anything, just correct the path:
After saving the objects with the Save button, you need to collapse and expand Unity so that the updated binary file is imported and updated.
Now you can change and save the binary file in the program and in the unit:
- High processing speed;
- High level of data compression;
- The ability to store their classes and structures;
- Reading / writing in Unity, as well as in a separate program (Visual Studio Application, C #);
- Work with old versions of the saved data (when changing the structure);
- Should not require the presence of additionally installed packages and other software from users;
- Work on mobile devices;
- Language: C #.
As a result, I stopped at binary serialization. This method meets all specified requirements, but makes it impossible to view and edit already serialized data in a text editor. But this is not a problem, since the program for editing is intended for this.
Program
The first task was to do serialization and de-serialization of data in the program. I wrote a simple program that will edit and serialize the data of the nodes in the custom Nodes class as (ID, Object) in the Dictionary <short, data> collection. There will be a lot of objects, so the node ID will be stored in a 16-bit short data type.
Class Nodes, to begin with will be the easiest. Mark it as Serializable.
[Serializable()] public class NodesV1 { public Dictionary<short, string> Name;// public Dictionary<short, string> Text;// } (program code at the end of the article)
')
The newly created node should be added to the first free position in the collection, for this I used the code:
short CalcNewItemIndex() { short Index = -1; // while (Nodes.Name.ContainsKey(++Index)); // return Index; // } Serialization
Two steps that need to be performed at this stage are to force the serializer to work with our NodesV1 class and take into account that the data structure of the serializable / deserialized object will change (it will change more than once during the development process).
The second step is not necessary, but if you change the structure, deserialize the file with the previous structure does not work (but in some cases, if you add new data to the end, then the old file usually passes without problems).
First you need a class that will work on serialization / deserialization, in it we will force the serializer to work with our class.
Serialization / deserialization class code
public class SaveLoad_Data { BinaryFormatter bformatter = new BinaryFormatter(); public void Save(object data, string filepath)// { Stream stream = File.Open(filepath + ".txt", FileMode.Create);// BinaryFormatter bformatter = new BinaryFormatter(); bformatter.Binder = new ClassBinder();// bformatter.Serialize(stream, data);//C stream.Close();// } public object Load(string filepath)// { byte[] data = File.ReadAllBytes(filepath + ".txt");// MemoryStream stream = new MemoryStream(data);// bformatter.Binder = new ClassBinder();// NodesV1 _NodesV1 = (NodesV1)bformatter.Deserialize(stream);// stream.Close();// return _NodesV1;// } } public sealed class ClassBinder : SerializationBinder // { public override Type BindToType(string assemblyName, string typeName) { if (!string.IsNullOrEmpty(assemblyName) && !string.IsNullOrEmpty(typeName)) { Type typeToDeserialize = null; assemblyName = Assembly.GetExecutingAssembly().FullName; typeToDeserialize = Type.GetType(String.Format("{0}, {1}", typeName, assemblyName)); return typeToDeserialize; } return null; } } Starting serialization / deserialization
NodesV1 Nodes;// , private void Form1_Load(object sender, EventArgs e) // { Nodes = new NodesV1();// , // Nodes.Name = new Dictionary<short, string>(); Nodes.Text = new Dictionary<short, string>(); } private void button1_Click(object sender, EventArgs e)// { SaveLoad_Data _SaveNodes = new SaveLoad_Data();// _SaveNodes.Save(Nodes, @"C:\UnityProjects\Blueprint_Editor_Plugin\Assets\Resources\HabrSerialisText");// } private void button2_Click(object sender, EventArgs e)// { SaveLoad_Data _LoadNodes = new SaveLoad_Data();// Nodes = (NodesV1)_LoadNodes.Load(@"C:\UnityProjects\Blueprint_Editor_Plugin\Assets\Resources\HabrSerialisText"); // } We will save the file to the Unity project folder: Assets \ Resources. It is from the Resources folder that it will work correctly on Unity reading a file on mobile devices, etc.
Now step two, resolve the issue with the deserializer version. In the first two bytes of the binary file, we will write the version of the serializer. During deserialization, we read the version, remove the two bytes and start the corresponding version deserializer. The serializer version will be determined by the numbers at the end of the class name (NodesV1 - version ”1”).
Add a version check:
public class SaveLoad_Data { BinaryFormatter bformatter = new BinaryFormatter(); public void Save(object data, string filepath)// { int Version;// BinaryFormatter bformatter = new BinaryFormatter(); bformatter.Binder = new ClassBinder();// MemoryStream streamReader = new MemoryStream(); bformatter.Serialize(streamReader, data);//C Version = Convert.ToInt32(data.GetType().ToString().Replace("HabrSerialis.NodesV", ""));// byte[] arr = streamReader.ToArray();// byte[] versionBytes = BitConverter.GetBytes(Version);// byte[] result = new byte[arr.Length + 4]; // // , . int - 4 Array.Copy(arr, 0, result, 4, arr.Length);// Array.Copy(versionBytes, 0, result, 0, versionBytes.Length);// File.WriteAllBytes(filepath + ".txt", result);// streamReader.Close();// } public object Load(string filepath)// { byte[] back = File.ReadAllBytes(filepath + ".txt");// int versionBack = BitConverter.ToInt32(back, 0);// byte[] data = new byte[back.Length - 4]; // Array.Copy(back, 4, data, 0, back.Length - 4);// MemoryStream stream = new MemoryStream(data);// bformatter.Binder = new ClassBinder();// if (versionBack == 1)// 1 { NodesV1 _NodesV1 = (NodesV1)bformatter.Deserialize(stream);// 1 stream.Close();// return _NodesV1; } return null; } } public sealed class ClassBinder : SerializationBinder { public override Type BindToType(string assemblyName, string typeName) { if (!string.IsNullOrEmpty(assemblyName) && !string.IsNullOrEmpty(typeName)) { Type typeToDeserialize = null; assemblyName = Assembly.GetExecutingAssembly().FullName; typeToDeserialize = Type.GetType(String.Format("{0}, {1}", typeName, assemblyName)); return typeToDeserialize; } return null; } } Now we will create several nodes in the program and start the serialization. We still need the resulting file.
Now check if it works. Suppose our structure has changed, we have added the Permission (Perm) variable. Create a class with a new structure:
[Serializable()] public class NodesV2 { public Dictionary<short, string> Name; public Dictionary<short, string> Text; public Dictionary<short, bool> Perm; } We change the NodesV1 class to NodesV2 in the program code. At startup, we also initialize a new variable:
Nodes.Perm = new Dictionary<short, bool>(); Now the fun part. In the file with the old data structure, there is no Perm variable, and we need to deserialize according to the old structure and return it to the new one.
In each case, there will be its own handling of this situation, but I will simply create this collection with false values.
Let's change the version verification code in the deserializer:
if (versionBack == 1)// 1 { NodesV1 _NodesV1 = (NodesV1)bformatter.Deserialize(stream);// 1 stream.Close();// NodesV2 NodeV2ret = new NodesV2();// NodeV2ret.Name = _NodesV1.Name; // NodeV2ret.Text = _NodesV1.Text; // NodeV2ret.Perm = new Dictionary<short, bool>(); // 1 Perm foreach (KeyValuePair<short, string> name in NodeV2ret.Name) { NodeV2ret.Perm.Add(name.Key, false);// false } return NodeV2ret; // } else if (versionBack == 2)// 2 - ( ) { NodesV2 _NodesV2 = (NodesV2)bformatter.Deserialize(stream);// stream.Close();// return _NodesV2; } After the changes, the deserialization of the file with the old structure is successful.
Unity
Create a C # script that deserializes the binary and will display the name and text of the node in the GUI. You can also change this data and serialize it back.
Unity script code
using UnityEngine; using System; using System.Collections; using System.Collections.Generic; using System.IO; using System.Runtime.Serialization.Formatters.Binary; using System.Runtime.Serialization; using System.Reflection; public class HabrSerialis : MonoBehaviour { NodesV2 Nodes; SaveLoad_Data _LoadNodes; void Start() { Nodes = new NodesV2(); _LoadNodes = new SaveLoad_Data(); Nodes = (NodesV2)_LoadNodes.Load("HabrSerialisText"); } float Offset; void OnGUI() { Offset = 100; for (short i = 0; i < Nodes.Name.Count; i++) { Nodes.Name[i] = GUI.TextField(new Rect(Offset, 100, 100, 30), Nodes.Name[i]); Nodes.Text[i] = GUI.TextArea(new Rect(Offset, 130, 100, 200), Nodes.Text[i]); Offset += 120; } if (GUI.Button(new Rect(10, 10, 70, 30), "Save")) { _LoadNodes.Save(Nodes, "HabrSerialisText"); } } } [Serializable()] public class NodesV1 { public Dictionary<short, string> Name; public Dictionary<short, string> Text; } [Serializable()] public class NodesV2 { public Dictionary<short, string> Name; public Dictionary<short, string> Text; public Dictionary<short, bool> Perm; } public class SaveLoad_Data { private int Version; BinaryFormatter bformatter = new BinaryFormatter(); public void Save(object data, string filepath)// { BinaryFormatter bformatter = new BinaryFormatter(); bformatter.Binder = new ClassBinder();// MemoryStream streamReader = new MemoryStream(); bformatter.Serialize(streamReader, data);//C Version = Convert.ToInt32(data.GetType().ToString().Replace("NodesV", ""));// byte[] arr = streamReader.ToArray(); byte[] versionBytes = BitConverter.GetBytes(Version);// byte[] result = new byte[arr.Length + 4]; // // , . int - 4 Array.Copy(arr, 0, result, 4, arr.Length);// Array.Copy(versionBytes, 0, result, 0, versionBytes.Length);// File.WriteAllBytes("Assets/Resources/" + filepath + ".txt", result);// streamReader.Close();// } public object Load(string filepath)// { TextAsset asset = Resources.Load(filepath) as TextAsset;// byte[] back = asset.bytes; int versionBack = BitConverter.ToInt32(back, 0);// byte[] data = new byte[back.Length - 4]; // Array.Copy(back, 4, data, 0, back.Length - 4);// Stream stream = new MemoryStream(data);// bformatter.Binder = new ClassBinder();// //////////////////////////////////////////////////////// if (versionBack == 1)// 1 { NodesV1 _NodesV1 = (NodesV1)bformatter.Deserialize(stream);// 1 stream.Close();// NodesV2 NodeV2ret = new NodesV2();// NodeV2ret.Name = _NodesV1.Name; // NodeV2ret.Text = _NodesV1.Text; // NodeV2ret.Perm = new Dictionary<short, bool>(); // 1 Perm foreach (KeyValuePair<short, string> name in NodeV2ret.Name) { NodeV2ret.Perm.Add(name.Key, false);// } return NodeV2ret;// } else if (versionBack == 2)// 2 - ( ) { NodesV2 _NodesV2 = (NodesV2)bformatter.Deserialize(stream);// stream.Close();// return _NodesV2; } ////////////////////////////////////////////////////////////// return null; } } public sealed class ClassBinder : SerializationBinder { public override Type BindToType(string assemblyName, string typeName) { if (!string.IsNullOrEmpty(assemblyName) && !string.IsNullOrEmpty(typeName)) { Type typeToDeserialize = null; assemblyName = Assembly.GetExecutingAssembly().FullName; typeToDeserialize = Type.GetType(String.Format("{0}, {1}", typeName, assemblyName)); return typeToDeserialize; } return null; } } As you can see, the class code that handles the serialization is the same, only instead:
byte[] back = File.ReadAllBytes(filepath + ".txt"); we will use:
TextAsset asset = Resources.Load(filepath) as TextAsset; byte[] back = asset.bytes; If the script is not planned to run on mobile devices (or similar), you can not touch anything, just correct the path:
byte[] back = File.ReadAllBytes("Assets/Resources/" + filepath + ".txt"); After saving the objects with the Save button, you need to collapse and expand Unity so that the updated binary file is imported and updated.
Source code of the program
using System; using System.Collections.Generic; using System.ComponentModel; using System.Data; using System.Drawing; using System.Linq; using System.Text; using System.Windows.Forms; using System.IO; using System.Runtime.Serialization.Formatters.Binary; using System.Runtime.Serialization; using System.Reflection; namespace HabrSerialis { public partial class Form1 : Form { public Form1() { InitializeComponent(); } ///////////////////////////////////////////////////////////////////////////////////////////////// short _SelectedNodeID = 0; // NodesV2 Nodes;// , private void Form1_Load(object sender, EventArgs e) // { Nodes = new NodesV2();// , // Nodes.Name = new Dictionary<short, string>(); Nodes.Text = new Dictionary<short, string>(); Nodes.Perm = new Dictionary<short, bool>(); } private void button1_Click(object sender, EventArgs e)// { SaveLoad_Data _SaveNodes = new SaveLoad_Data();// _SaveNodes.Save(Nodes, @"C:\UnityProjects\Blueprint_Editor_Plugin\Assets\Resources\HabrSerialisText"); } private void button2_Click(object sender, EventArgs e)// { SaveLoad_Data _LoadNodes = new SaveLoad_Data(); Nodes = (NodesV2)_LoadNodes.Load(@"C:\UnityProjects\Blueprint_Editor_Plugin\Assets\Resources\HabrSerialisText"); UpdateList();// } /////////////////////////////////////////////////////////////////////////////////// private void listBox1_SelectedIndexChanged(object sender, EventArgs e) // { _SelectedNodeID = (short)listBox1.SelectedIndex; if (Nodes.Name.ContainsKey(_SelectedNodeID))// { textBox1.Text = Nodes.Name[_SelectedNodeID];// 1 textBox2.Text = Nodes.Text[_SelectedNodeID];// 2 } } /////////////////////////////////////////////////// private void button3_Click(object sender, EventArgs e)// () { short _NewNodeID = CalcNewItemIndex(); Nodes.Name.Add(_NewNodeID, "New Node name");// Nodes.Text.Add(_NewNodeID, "New Node Text");// Nodes.Perm.Add(_NewNodeID, false);// UpdateList();// listBox1.SelectedIndex = _NewNodeID; } /////////////////////////////////////////////////// private void textBox2_TextChanged(object sender, EventArgs e) { Nodes.Text[_SelectedNodeID] = textBox2.Text;// } /////////////////////////////////////////////////// private void textBox1_TextChanged(object sender, EventArgs e)// { Nodes.Name[_SelectedNodeID] = textBox1.Text;// listBox1.Items[_SelectedNodeID] = "ID: " + _SelectedNodeID + " " + textBox1.Text;// } /////////////////////////////////////////////////// short CalcNewItemIndex()// { short Index = -1; while (Nodes.Name.ContainsKey(++Index)); return Index; } /////////////////////////////////////////////////// void UpdateList()// { listBox1.Items.Clear(); foreach (KeyValuePair<short, string> node in Nodes.Name) { listBox1.Items.Add("ID: " + node.Key + " " + node.Value); } } } } ////////////////////////////////////////////////// [Serializable()] public class NodesV1 { public Dictionary<short, string> Name; public Dictionary<short, string> Text; } [Serializable()] public class NodesV2 { public Dictionary<short, string> Name; public Dictionary<short, string> Text; public Dictionary<short, bool> Perm; } public class SaveLoad_Data { private int Version; BinaryFormatter bformatter = new BinaryFormatter(); public void Save(object data, string filepath) { BinaryFormatter bformatter = new BinaryFormatter(); bformatter.Binder = new ClassBinder();// MemoryStream streamReader = new MemoryStream(); bformatter.Serialize(streamReader, data);//C Version = Convert.ToInt32(data.GetType().ToString().Replace("NodesV", ""));// byte[] arr = streamReader.ToArray(); byte[] versionBytes = BitConverter.GetBytes(Version);// byte[] result = new byte[arr.Length + 4]; // // , . int - 4 Array.Copy(arr, 0, result, 4, arr.Length);// Array.Copy(versionBytes, 0, result, 0, versionBytes.Length);// File.WriteAllBytes(filepath + ".txt", result);// streamReader.Close();// } public object Load(string filepath) { byte[] back = File.ReadAllBytes(filepath + ".txt");// int versionBack = BitConverter.ToInt32(back, 0);// byte[] data = new byte[back.Length - 4]; // Array.Copy(back, 4, data, 0, back.Length - 4);// MemoryStream stream = new MemoryStream(data);// bformatter.Binder = new ClassBinder();// //////////////////// //////////////////////////////////// if (versionBack == 1)// 1 { NodesV1 _NodesV1 = (NodesV1)bformatter.Deserialize(stream);// 1 stream.Close();// NodesV2 NodeV2ret = new NodesV2();// NodeV2ret.Name = _NodesV1.Name; // NodeV2ret.Text = _NodesV1.Text; // NodeV2ret.Perm = new Dictionary<short, bool>(); // 1 Perm foreach (KeyValuePair<short, string> name in NodeV2ret.Name) { NodeV2ret.Perm.Add(name.Key, false);// } return NodeV2ret; } else if (versionBack == 2)// 2 - ( ) { NodesV2 _NodesV2 = (NodesV2)bformatter.Deserialize(stream);// stream.Close();// return _NodesV2; } ////////////////////////////////////////////////////////////// return null; } } public sealed class ClassBinder : SerializationBinder { public override Type BindToType(string assemblyName, string typeName) { if (!string.IsNullOrEmpty(assemblyName) && !string.IsNullOrEmpty(typeName)) { Type typeToDeserialize = null; assemblyName = Assembly.GetExecutingAssembly().FullName; typeToDeserialize = Type.GetType(String.Format("{0}, {1}", typeName, assemblyName)); return typeToDeserialize; } return null; } } Now you can change and save the binary file in the program and in the unit:
Source: https://habr.com/ru/post/242069/
All Articles