KTV. New JSON

In my development I had to run through several stages in several directions: Java → Objective C → Swift, Web → Enterprise → Mobile, XML → JSON. I walked this way for more than 15 years, long and attentively lingering at each stage. Need to go further. For mobile applications, you can think of something (probably, so far you don’t want to), languages ​​are generally a dime a dozen, nothing more interesting than JSON has been invented. Why change it?

Then I will tell you why I do not really like JSON for some applications and how, in my opinion, it can be improved to make it a little more convenient.

I should immediately note that I do not consider KTV as a replacement for JSON. And in no case do I consider it for use in JavaScript. It will be uncomfortable and wrong. On the other hand, the situation when the system of describing objects of JavaScript is used in other languages ​​for working with typed data is also strange, and I want to correct it.
')
If you are wondering what was the original reason for creating such a format - you can read about S2 and the reasons for creating the KTV format

Pros and cons of JSON

JSON itself is good.

• Plain. Special thanks for the lack of comments. If they were left, there would start to push the meta-information (because JSON is very poor) and the notation would quickly turn into a trash.
• Expressive. Not like XML, but enough for a wide range of tasks.
• Much shorter in writing than XML.

JSON's diseases grow from the history of creation. It was necessary for eval(-) in JavaScript to produce an object with which it is convenient to work. Unfortunately, evals in other languages ​​usually work differently (if they are present at all), and the languages ​​themselves are different. Using this format with non-javascript, its flaws are visible.

• Quotes. According to the standard (as a standard I take the text from here: http://www.json.org ) all names should be in quotes. If we use JSON, for example, to transfer data over the network, then this is overkill.
• Lack of typing. More precisely, there are types, but only the string / number / true / false / null. And objects with arrays. Everything. There are no dates, no whole / fractional numbers. There is no possibility to mark objects with types in order to make the model easier to understand.
• Lack of standards for recording objects. This, for example, can lead to arrays with mixed objects inside. When you have this parsit - it hurts.
• Missing links. If you record the hierarchical structure of objects in JSON, then there are regular references to the same object from different places. JSON does not allow to refer to the first use. It is necessary either to invent something, or to repeat objects as a whole, which has a bad effect on the size of the structure and on the complexity of parsing.

I tried to find a format that would work as JSON (ideally - would be its superset), but at the same time I was able to solve some or all of the problems listed above in a standard way. Found YAML.

Yaml

Yaml is good. It is almost a superset of JSON, well defined ( http://www.yaml.org/spec/1.2/spec.html ), easy to read. But at the same time:

• nesting of structures is determined by indents. In the case of the transfer of information, it would be convenient to remove them. Tab lovers are also not respected, indents are made only with spaces.
• YAML is overly complex. A large number of special characters, agreements on the structure make support for YAML'a difficult, parsers - complex (YAML 1.2 has appeared for 7 years already, but, according to news from the site, parsers still support mostly previous versions).

In general, YAML "did not go." In addition to his popular formats, I do not know. If suddenly you know - write, I will look with pleasure.

It's time to introduce what I got.

KTV

Key-Type-Value. I call this format.

Let's write a simple object in JSON:

 { "name": "Alex", "coolness": 3.1415, "isAProgrammer": true } 

If you rewrite the same on KTV, you get:

 { name: Alex coolness: 3.1415 isAProgrammer: true } 

It would seem that nothing has changed. But let's take a look:

• quotes cleaned up. They are now required only if there is a non-trivial string in the key or value. Normal situations do not require quotes.
• commas were removed at the end of lines. They (or semicolons) can be put if we write several definitions in one line. But in the traditional record they can be omitted.

Both the first and second examples are both correct KTV files. This is very important, since it allows you to stuff standard JSON into the KTV parser, and it will eat it with pleasure. And you can take advantage of amenities, types for example.

Where are the types? In the above case, types are inferred from values. “Alex” is a lowercase literal, so the name type is “string”, the coolness is “double”, and isAProgrammer is “bool”. In addition to these types, there is also “null” (aka “nil”, hello to colleagues), “int” and “color”. Let's write the same example, but completely indicating the types:

 { name(string): Alex coolness(double): 3.1415 isAProgrammer(bool): true } 

Nowhere and nested objects / arrays. With additions:

• Both objects and arrays can have types. For an array, type is the generic type of each object in the array. You cannot put two different types (string / number, or different objects) into an array.
• The type of an array or object may also indicate the class to which this object or array must be cast during parsing. For example:
  property(font): {name:..., size:...}
  may denote a font object, and
  property(point): [2, 3]
  can be converted to a point object when recognized. In principle, a point can also be made an object (with x, y perperty), but why? After all, we already know where there is.

There are links, the ability to refer to any other proppeyu. Something like this:

 { property: value another: @property } 

This allows you to reduce the recording of some branched objects, or simply set constants, referring to them in other parts of the file. The first feature is useful in REST if the structure of objects is nontrivial. The second is in the configuration files.

The idea of ​​mixins is also borrowed from other languages. They allow one object (or objects) to be attached to another object. You can imagine it as inheritance, but inheritance is from the PLO, where behavior is inherited, there are only properties. So it is better to mix.

 { object: { a: ..., b: ...} object2(+object): { c: ..., d: ... } } 

In this example, after parsing, object2 will contain both its own properties and the properties of the object. That is, a, b, c, and d.

Comments

I could not resist and organized comments. They can be set either using // or #. Both types of comments are lowercase, no block comments.

With the second type (which is octotorp) there is a slight problem. The fact is that the color literal is supposed to be set in the standard CSS form, that is, #rrggbbaa, which conflicts with the current version of writing comments. So, perhaps, in the future, only the C-variant (//) will remain.

Work with format

For me, this format, and the parser of this format is needed in two places:

• to store style information in S2 (development of the Ångström Style System )
• so that you can improve communication between your own Swift servers and applications on it.

Therefore, I consider working with the format for Swift. In principle, the format is quite simple and it should not be difficult to organize its parser in any language.

Swift problems now do not allow you to really properly load / serialize objects from / to KTV. Therefore, the process of working with the format assumes the following:

• Model classes are created. Like regular classes (including annotated @objc ) or structures. Each class must implement the KTVGenerated protocol.
• With the help of the generator, extensions are created to these classes that can load and serialize objects.
• We use the generated to convert KTV into objects and back (or to / from JSON).

A model structure might look like this:

 public struct RootObject: KTVGenerated { var string:String //   var stringOrNil:String? // optional var int:Int //  var stringArray:[String] //  var stringDictionary:[String:Int] //   var object:ChildObject? //      private var _privateString:String //      private(set) var _privateSetString:String //  —  let _constantString:String //      } 

It is seen that all the main features are supported:

• Types (they must be written explicitly so that the generator can correctly create class extensions.
• Collections (arrays and associative arrays).
• Object hierarchies (links to other model classes)
• Property that does not participate in serialization (these are constants, private properties and properties with private setters)

The history of creating such a parser is a topic for a separate article. Tasks of simultaneously maintaining structures and classes, ways of specifying attributes without the presence of attributes themselves, using SourceKit instead of reflection for analyzing files, the need for custom mappers (for example, for parsing non-standard dates). The parser itself is now in the process of active development, and is constantly changing. But, if there is enough interest - lay out a working draft, discuss options.

Benefit from the next data format

https://xkcd.com/927/
JSON is a so-so format for transferring data between applications. This is the format for describing objects in JavaScript. And we constantly use it for completely different purposes, cramming both in and out of business.

It seems to me that it is time to move on to something more adapted to the tasks performed. To the new Swift with its strict typing, to the complex data structures that often have to be transferred to the mobile clients from the server. We need a format whose syntax can be checked in the IDE, during the compilation process and during the parsing. And, it seems to me that KTV, without changing cardinally the structure, practically without complicating the already known format, adds a few handy little things.

I spread the experimental source code with a small description. There are more interesting techniques for parsing the format on Swift, rather than the parser. But suddenly it will be interesting to anyone to see: http://github.com/bealex/KTV

Most likely, I missed a lot or not. Maybe someone has a similar experience? Write in the comments, or e-mail: alex@jdnevnik.com, discuss!