jParser makes it easy to read data structures from binary files with javascript.

• You describe the structure once; it is analyzed automatically.
  
• The process of analyzing data can be expanded with handwritten functions What simplifies the analysis of non-trivial files.
  
• jParser works both in the browser and in NodeJS, because it works on the basis of jDataView .

API

Elementary structures:

• Unsigned integers: uint8, uint16, uint32
  
• Signed : int8, int16, int32
  
• Fractional floating point: float32, float64
  
• String: char, string (len)
  
• Array: array (type, len)
  
• Position: tell, skip (len), seek (pos), seek (pos, func)

JParser methods:
')

• parse (value) - starts the analysis, can be used recursively. The behavior depends on the type of argument:
  
  
  • function: calls the function;
    
  • line: dereference (the specified name is taken from the value of the structure);
    
  • array: a function call whose name is the first element of the array, and the subsequent elements are arguments;
    
  • object: returns an object with the same field names, but parsed values.
  
  
• tell () - returns the current position (offset) in the analyzed file.
  
• skip (count) - move forward by count in the file (skip them).
  
• seek (position) - go to the specified offset in the analyzed file.
  
• seek (position, callback) - go to the specified offset, do a callback () there, then return to the previous position.
  
• current - The current object being analyzed. Use to access the results already achieved during the analysis.

JParser Constructor:

• new jParser (data, structure)
  
  
  • data is a jDataView with data to be analyzed. You can pass a string of bytes (String), you can pass an ArrayBuffer or a buffer to Node.js - these data types are converted to jDataView automatically.
    
  • structure - an object with a description of all data structures.

Examples

Common C Structure

You can describe C-like data structures. This is a javascript object in which the fields are data names and their values ​​are types.

var parser = new jParser(file, { header: { fileId: 'int32', recordIndex: 'int32', hash: ['array', 'uint32', 4], fileName: ['string', 256], } }); parser.parse('header'); // { // fileId: 42, // recordIndex: 6002, // hash: [4237894687, 3491173757, 3626834111, 2631772842], // fileName: ".\\Resources\\Excel\\Items_Weapons.xls" // } 

Links

Structures may contain other structures. Use the structure name as a string to refer to its description. The following example is part of the file structure that contains the model for World of Warcraft:

 nofs: { count: 'uint32', offset: 'uint32' }, animationBlock: { interpolationType: 'uint16', globalSequenceID: 'int16', timestamps: 'nofs', keyFrame: 'nofs' }, uvAnimation: { translation: 'animationBlock', rotation: 'animationBlock', scaling: 'animationBlock' } 

Helper Functions

It will be easy for you to define new elementary types. You can use existing kinds of constructions, such as objects ( float3 in the example below) or arrays ( float4 in the example). If you want to define a more complex type, it is always possible to define a new function that relies on the this.parse method for the primary analysis ( hex32 and string0 in the following example):

 float3: { x: 'float32', y: 'float32', z: 'float32' }, float4: ['array', 'float32', 4], hex32: function () { return '0x' + this.parse('uint32').toString(16); }, string0: function (length) { return this.parse(['string', length]).replace(/\0+$/g, ''); } 

Feedback

If the size of the array is not known in advance, you can put in its place in the structure a function that returns an integer. In this function, you can refer to the object currently being analyzed, through this.current , to read its already received fields.

 image: { width: 'uint8', height: 'uint8', pixels: [ 'array', ['array', 'rgba', function () { return this.current.width; }], function () { return this.current.height; } ] } 

Advanced data analysis

A nice feature of jParser is that complex processing algorithms can be expressed by functions within the description of the data structure. This allows you to analyze complex files without separating the structure description from the code of its handlers.

 entryHeader: { start: 'int32', count: 'int32' }, entry: function (type) { var that = this; var header = this.parse('entryHeader'); var res = []; this.seek(header.start, function () { for (var i = 0; i < header.count; ++i) { res.push(that.parse(type)); } }); return res; }, name: { language: 'int32', text: ['string', 256] }, file: { names: ['entry', 'name'] } 

Beginning of work

On the engine NodeJS

Just use npm to install jParser - and you are ready :-)

 npm install jParser 

 var fs = require('fs'); var jParser = require('jParser'); fs.readFile('file.bin', function (err, data) { var parser = new jParser(data, { magic: ['array', 'uint8', 4] }); console.log(parser.parse('magic')); }); 

In the browser

I patched jQuery , provided I downloaded binary files in the best binary format. Connect this patched jQuery code, as well as jDataView and jParser - and you are ready :-)

 <script src="https://raw.github.com/vjeux/jDataView/master/jquery/jquery-1.7.1-binary-ajax.js"></script> <script src="https://raw.github.com/vjeux/jDataView/master/src/jdataview.js"></script> <script src="https://raw.github.com/vjeux/jParser/master/src/jparser.js"></script> <script> $.get('file.bin', function (data) { var parser = new jParser(data, { magic: ['array', 'uint8', 4] }); console.log(parser.parse('magic')); }, 'dataview'); </script> 

Precautions

This tool relies on an undocumented JavaScript feature in its work: bypassing the fields of an object works in the order in which they are specified. Keep in mind that in Chrome and in Opera, this implicit rule does not work for fields with numeric names.

Follow these two rules for this library to work in all existing implementations of the JavaScript language:

• Do not begin the field name with a number in the data structure description.
  
• Do not override the field with the name once specified, do not place another field of the same name in the same object.

Demo

ICO analysis . This is a simple example of analyzing a binary file on the NodeJS engine. It shows how many typical problems that arise when parsing binary data are solved.

• ico.js - data structure for jParser.
  
• ico.json - the result of the analysis.

Tar unpacker . This is a simple example of parsing a binary file inside a browser.

• tar.html - jParser structure.

A means of displaying models of World of Warcraft . Uses jParser to read the binary code of the three-dimensional model, then WebGL to display it.

• m2.js - jParser structure.
  
• model.json - the result of the analysis.

Diablo 3 internal files

• convert.coffee - structure for jParser. Using a censcript to write down the file structure is even easier.
  
• Examples of processed files:
  
  
  • Items_Weapons.json
    
  • Quest / ProtectorOfTristam.json
    
  • TreasureClass / SkeletonKing