Strelki.js - another library for working with arrays

When programming in JavaScript, there is often a problem of choosing the optimal presentation of data in a program: arrays, hashes, hashes arrays, arrays hashes, etc. The same data can be loaded into various combinations of structures, but the difficulty of choosing is usually how to find a compromise between the simplicity of the code for accessing this data, the speed of operation and the amount of memory required.



The article talks about my attempt to find a universal solution.

Let us, for example, need to display some data from two related tables:
')


The standard approach usually consists of the following steps:

1. On the server, write a SQL query with JOIN
2. On the server, add a function for it that returns an array of objects and make it accessible via routes
3. On the client, add an AJAX call to the server, and draw the result to a table

The disadvantages of the standard approach are seen in the following:

1. The SQL query and the wrapper function must take into account possible collisions of column names, i.e. one cannot simply make a " SELECT * ".
2. The server response will contain a large number of duplicate records from related tables. In our example, the record with the key “sales” from the departments table will be transmitted twice.
3. When linking a large number of tables, we will either get long keys, which will increase the useless consumption of memory and traffic to transfer these keys, or the column names in the SQL query must be manually transferred, which will lead to additional costs when making changes to the database structure.
4. The number of API functions for obtaining data from related tables can significantly exceed the number of tables, which leads to bloat of the code, and, as a result, to costs.

Non-standard approach - get the tables separately and link them to the client. Sometimes this can be done easily. For example, in the structure above, you can load the “departments” table into the hash, and access by “id”. But more often this cannot be done, and one has to use various search functions like Array.find or Array.indexOf.

Working on my next project, and once again puzzling over how I better organize the data to avoid the shortcomings of both approaches, I decided that it was time to cut this node once and for all.

The approach in which the server gives us normalized tables, and then we associate them in JavaScript code, seemed more attractive to me. It lacked only a tool to easily bind them. I put aside all things and sat down to write it.

Requirements have the following:

• The tool should allow me to create arbitrary indexes on an array, and keep them up to date.
• The tool should be able to search in an array by index, if possible without slow operations of enumerating elements.
• The tool should be able to connect arrays by indexed fields in accordance with a certain declaration object, something like a JOIN operator in SQL, but without parsing queries and all the power offered by SQL.

This is how Strelki.js appeared, and so far the only class in it is IndexedArray.

So, create a new IndexedArray:

var emp = new StrelkiJS.IndexedArray(); 

Add data to it:

  emp.put({ id: "001", first_name: "John", last_name: "Smith", dep_id: "sales", address_id: "200" }); emp.put({ id: "002", first_name: "Ivan", last_name: "Krasonov", dep_id: "sales", address_id: "300" }); 

Let's see what's inside:



Under the hood, IndexedArray is a hash (this.data) where object references are stored. As a hash key, the “id” field of the saved item is used, which must be unique. Since many modern server frameworks also use the “id” field in a similar way, this restriction should not be a problem.

In addition, IndexedArray has a hash of this.indexData. The keys of this hash contain the name of the field being indexed, and the values ​​are hashes with ids of the corresponding elements of the main hash. We don't have any indexes yet, so this.indexData is empty.

Add an index:

 emp.createIndex("dep_id"); 

Let's see this.indexData:



this.indexData now contains the key “dep_id”, which contains index data as nested hashes.

Look for something on the index:

 > emp.findIdsByIndex("dep_id","sales") < ["001", "002"] 

Unlike functions like Array.find, index search does not use data iteration, but only hashes, which allows achieving high speed. I haven’t done the measurements yet, but it should work quickly.

Add more data:

 emp.put({ id: "003", first_name: "George", last_name: "Clooney", dep_id: "hr", address_id: "400" }); emp.put({ id: "004", first_name: "Dev", last_name: "Patel", dep_id: "board", address_id: "500" }); 

Find the elements by index, and create a new IndexedArray from them:

 var sales_emp = emp.where("dep_id","sales"); 

Create and fill another IndexedArray:

 var adr = new StrelkiJS.IndexedArray(); adr.put({ id: "200", address: "New Orleans, Bourbon street, 100"}); adr.put({ id: "300", address: "Moscow, Rojdestvensko-Krasnopresnenskaya Naberejnaya"}); adr.put({ id: "500", address: "Bollywood, India"}); 

Array binding

To describe the relationship of this IndexedArray with any other object serves as follows:

 { from_col: "address_id", //    IndexedArray to_table: adr, //     to_col: "id", // "id",        type: "outer", // "outer"  LEFT OUTER JOIN,  null  INNER JOIN join: // null          ,    JOIN- } 

Attach adr to emp JOIN:

 var res = emp.query([ { from_col: "address_id", // name of the column in "emp" table to_table: adr, // reference to another table to_col: "id", // "id", or other indexed field in "adr" table type: "outer", // "outer" for LEFT OUTER JOIN, or null for INNER JOIN //join: [ // optional recursive nested joins of the same structure // { // from_col: ..., // to_table: ..., // to_col: ..., // ... // }, // ... //], } ]) 

A similar SQL statement would look like this:

 SELECT ... FROM emp LEFT OUTER JOIN adr ON emp.address_id = adr.id 

The result will look like this:

 [ [ {"id":"001","first_name":"John","last_name":"Smith","dep_id":"sales","address_id":"200"}, {"id":"200","address":"New Orleans, Bourbon street, 100"} ], [ {"id":"002","first_name":"Ivan","last_name":"Krasonov","dep_id":"sales","address_id":"300"}, {"id":"300","address":"Moscow, Rojdestvensko-Krasnopresnenskaya Naberejnaya"} ], [ {"id":"003","first_name":"George","last_name":"Clooney","dep_id":"hr","address_id":"400"}, null ], [ {"id":"004","first_name":"Dev","last_name":"Patel","dep_id":"board","address_id":"500"}, {"id":"500","address":"Bollywood, India"} ] ] 

As you can see, array binding has evolved from functions with cycles and iterations to a simple declaration of links.

Restrictions

IndexedArray does not store copies of objects, but only pointers to them (hence the name Strelki). Therefore, if the object was placed in IndexedArray by the put () method and then changed, the information in the indexes may become incorrect. To avoid this situation, you must remove the object from IndexedArray using the del () method before changing.

Linking can be carried out only by the indexed field, or by the "id" field.

Some methods of the IndexedArray object (for example, length ()) require the construction of an “id” key array. In this case, the key array is stored in the object for possible reuse. Each time the array changes (methods put (), del (), etc.), the array of keys is reset. Therefore, the alternation of methods that create and then zero out an array of keys can lead to performance problems on large data sets.

Plans

StrelkiJS created to facilitate the writing of the main project KidsTrack, which I wrote on Habré earlier. Therefore, all decisions on the new functionality are dictated by the needs of the parent project. In the near future - to make more convenient access to the columns in the results of the JOIN,

Where can I download

Github: github.com/amaksr/Strelki.js
Sandbox: www.izhforum.info/strelkijs