Greetings, Habrozhiteli!

Today we will speak about a world about which most of you do not know, but at the same time many excellent development engineers and big money are spinning there. Yes, oddly enough, it will be about Minecraft.

Minecraft is a sandbox game and on a multiplayer server there is an acute problem of griffery (from English griefing - sabotage) when players demolish other people's buildings. On servers, this problem is handled differently. In public, the plugin is used in “privat”, for the rest, everything is built on trust.

Another way to prevent a grifering is the ban of all griefers. And in order to calculate them, you have to log install and remove blocks. Actually, the process of creating such a log system will be discussed further.

Database selection

So, here we have an array of data and it would be good to save it somewhere. Smart people have long invented the database. Personally, my requirements for the database were as follows:

• Quick insert
• Maximum data compression
• Ability from Java without root-rights to deploy without unnecessary gestures

The last item appeared due to the fact that not all hosting companies have the opportunity to get root access or install any package. In addition, I did not want to complicate the installation procedure, but to stop at “I threw and forgot”.

I could not find a database that would satisfy all the criteria, so I decided to make my mini-database in Java.

Optimize hard disk space

The main problem of the game, as many believe, is that all its calculations occur in one thread. This is a real pain for server holders. To parallelize the initially single-threaded architecture - you need to try.

Therefore, the logging itself had to be taken out in a separate thread. And so that the system does not choke on Event'ov in the queue, add support for workers. The number of workers is customizable.

As a result, it turned out that the event itself is intercepted in the main tick, then sent to the stream, which is busy distributing tasks between workers. There we receive the file in which we need to register our event and transfer it to the worker who is attached to this file. And the IO operation itself takes place in a worker.

Optimize hard disk space

A large number of events can lead to the fact that the logs will weigh more than the world itself. We can not allow this, so we will think.

Initially, the line in the log file looked like this:

[2001-07-04T12:08:56.235-0700]Player PLACE <blockid> to 128,128,128

At a quick glance, you can see that 2001-07-04T12: 08: 56.235-0700 can be shortened to Timestamp, and PLACE or REMOVE to the '+' and '-' symbols, respectively. Well, remove nafig 'to':

[123454678]Player + <blockid> 128,128,128

It is not difficult to notice that the nickname and blockid will often be repeated in the log. Correspondingly, they can be put into a separate file, and only id should be written to the log

[123454678]1 + 1 128,128,128

As a result, I came to the conclusion that only numbers and one character remained in the log line. We will save a lot of space if we remove the delimiters (spaces) and the numbers will be written as bytes, not as characters. Actually, this led me to the decision to use byte logs.

The byte string itself now looks like this:

Total we have 35 bytes per line fixed (1 byte to separate the lines).
At first, it was tempting to leave 34 bytes, but since the recording is in one file, in the case of a fixed length, if one line is broken, the entire file will become unreadable.

The path for the logs:: /ssave}/{world/dimension}/*.bytelog

Line structure for blockname to id:

Total: ~ 21 bytes per block
File Name: blockmap.bytelog

Line structure for nickname to id:

Total: ~ 10 bytes per player
File Name: nickmap.bytelog

Memory optimization

To quickly map blockname and nickname to id, I had to keep the contents of both files in memory. Java cannot store primitive types in HashMap, so each Integer will cost us ~ 50 bytes in memory, which is a lot.

The trove library will help us solve this problem.

 private final TObjectIntHashMap uuidToId = new TObjectIntHashMap(); 

But each character takes about 2 bytes. We can reduce memory consumption with an ASCIString samopisny file in which characters are stored in byte [], and not in char [].

Testing

In testing byte serialization and deserialization, there is nothing unusual, but for testing components that require multithreaded access, we had to use the framework from Google Thread Weaver. The usual test using this framework looks like this:

 public class NickMapperAsyncTest extends TestCase { private volatile NickMapper nickMapper; public void testNickMapper() { final AnnotatedTestRunner runner = new AnnotatedTestRunner(); runner.runTests(this.getClass(), NickMapper.class); } @ThreadedBefore public void before() throws IOException { nickMapper = new NickMapper(); } @ThreadedMain public void main() { nickMapper.getOrPutUser(new ASCIString("2")); nickMapper.getOrPutUser(new ASCIString("LionZXY")); nickMapper.getOrPutUser(new ASCIString("3")); } @ThreadedSecondary public void secondary() { nickMapper.getOrPutUser(new ASCIString("2")); nickMapper.getOrPutUser(new ASCIString("LionZXY")); nickMapper.getOrPutUser(new ASCIString("3")); } @ThreadedAfter public void after() { final int first = nickMapper.getOrPutUser(new ASCIString("LionZXY")); final int second = nickMapper.getOrPutUser(new ASCIString("2")); final int third = nickMapper.getOrPutUser(new ASCIString("3")); assertEquals(3, nickMapper.size()); assertEquals(Integer.MIN_VALUE + 3, Collections.max(Arrays.asList(first, second, third)).intValue()); } } 

The framework knocks from both threads with a different order, which allows you to catch the most nasty bugs in asynchronous code.

Conclusion

So far, by the number of downloads, it will be clear whether to further develop this mod and idea. From rough plans for the future:

• Add the ability to delete old and irrelevant logs automatically
• Add file compression

Links

• Github
• Curseforge

 @Config(modid = FastLogBlock.MODID) @Config.LangKey("fastlogblock.config.title") public class LogConfig { @Config.Comment("Enable handling event") public static boolean loggingEnable = true; @Config.Comment("Filepath from minecraft root folder to block log path") public static String logFolderPath = "blocklog"; @Config.Comment("Path to nickname mapper file from logFolderPath") public static String nickToIntFilePath = "nicktoid.bytelog"; @Config.Comment("Path to block mapper file from logFolderPath") public static String blockToLongFilePath = "blocktoid.bytelog"; public static HashConfig HASH_CONFIG = new HashConfig(); @Config.Comment("File splitter type. SINGLE for single-file strategy, BLOCKHASH for file=HASH(BlockPos) strategy") public static FileSplitterEnum fileSplitterType = FileSplitterEnum.BLOCKHASH; @Config.Comment("Utils information for migration") public static int logSchemeVersion = 1; @Config.Comment("Utils information for migration") public static int writeWorkersCount = 4; @Config.Comment("Regular expression for block change event ignore") public static String[] ignoreBlockNamesRegExp = new String[]{"<minecraft:tallgrass:*>"}; @Config.Comment("Permission level for show block log.") public static boolean onlyForOP = true; public static class HashConfig { @Config.Comment("Max logfile count") public final int fileCount = 16; @Config.Comment("Pattern for log filename. %d - file number. Default: part%d.bytelog") public final String fileNamePattern = "part%d.bytelog"; } @Mod.EventBusSubscriber(modid = FastLogBlock.MODID) private static class EventHandler { /** * Inject the new values and save to the config file when the config has been changed from the GUI. * * @param event The event */ @SubscribeEvent public static void onConfigChanged(final ConfigChangedEvent.OnConfigChangedEvent event) { if (event.getModID().equals(FastLogBlock.MODID)) { ConfigManager.sync(FastLogBlock.MODID, Config.Type.INSTANCE); } } } }