The idea of ​​implementing an I / O package in Java

Perfection is achieved not when there is nothing left to add,
and when nothing can be taken away.
Antoine de Saint-Exupery, Wind, sand and stars, 1939

Often you have to design and develop I / O packages for Java applications. On the one hand there is java.io, which is more than enough. However, in practice it is rarely possible to get along with a set of standard classes and interfaces.

The article provides a practical example of an idea for implementing I / O packages on the Java platform.
')

Formulation of the problem

For clarity, consider an example. Suppose you want to develop an I / O package for the matrix library. At the same time, it is necessary to consider that:

• Matrices (types / classes) can be countless, for example - dense, sparse;
• There may also be many input and output formats, for example, MatrixMarket , XML;

MatrixMarket format

For the MatrixMarket format, the following matrix representations will be:

0 2 3 0 

For a dense matrix:

 %%MatrixMarket matrix array real general 2 2 0 2 3 0 

For a sparse matrix:

 %%MatrixMarket matrix coordinate real general 2 2 2 0 1 2 1 0 3 

Implementation

Thus, the implementation of an I / O package must be so flexible that when expanding the system (adding a new type of matrix, such as a block or adding a new format, such as CSV), it is not necessary to completely rewrite the package — it would be sufficient only to implement an additional class — a new class. matrix or new format.

The attentive reader will notice the obvious similarity of the described problem, with the problem solved by the bridge pattern. This is indeed the case and the article can be taken as an example of the implementation of the Bridge pattern for an I / O package.

Returning to the basics of design patterns , you can briefly describe the bridge pattern as a separation of abstraction and implementation. In our case - the separation of the matrix type (dense, block) from the format (XML, MatrixMarket). This is achieved by introducing two interfaces - an abstraction interface and an implementation interface. The abstraction interface should high-level describe the behavior of the package, for example, the methods readMatrix (), writeMatrix (). While the implementation interface should describe low-level moments, such as readMatrixElement (), writeMatrixElement (), etc. Then in the simplest case, the class diagram for an I / O package looks like this.



The high-level writeMatrix () method is a sequence of low-level calls:

• writeMatrixHeader () - write matrix type information;
• writeMatrixMeta () - write matrix dimension information;
• writeMatrixElement () - write information about the matrix element.

It turns out that the bridge pattern solves the problem described earlier, thanks to the separation of implementation and abstraction. But in most cases, the objects that I / O packages work with already implement the serialization mechanisms (Srializable, Externalizable). In our case, the Matrix interface already expands the Externalizable interface. Why Externalizable and not Serizliable can be found in this or this (author's work) research. In short - Externalizble works many times faster due to the reduced number of calls in the JVM / Reflection.

And so the readExternal / writeExternal methods for a dense matrix look like this:

 public void writeExternal(ObjectOutput out) throws IOException { out.writeInt(rows); out.writeInt(columns); for (int i = 0; i < rows; i++) { for (int j = 0; j < columns; j++) { out.writeDouble(self[i][j]); } } } public void readExternal(ObjectInput in) throws IOException, ClassNotFoundException { rows = in.readInt(); columns = in.readInt(); self = new double[rows][columns]; for (int i = 0; i < rows; i++) { for (int j = 0; j < columns; j++) { self[i][j] = in.readDouble(); } } } 

The attentive reader will say: “This is exactly like the Bridge pattern!” And will be absolutely right. The ObjectOutput and ObjectInput interfaces implement the template idea as implementation interfaces. Then the question arises - “Why would you produce classes like MatrixReader / MarixWriter and write duplicate readExterna () l / writeExternal () methods in them?”. That's right - no need. Moreover, DRY (Do not Repeat Yourself) methodology reminds us of this.

In this case, we will try to revise the proposed implementation of the package, given that java.io already contains implementation interfaces - ObjectInput / ObjectOutput. Those. we only need to implement the classes MMOutoutStream / MMInputStream (MM = MatrixMarket) to use them instead of the standard classes for serialization ObjectInputStream / ObjectOutputStream. Then the use will be very transparent:

 //  ObjectOutput mmos = new MMOutputStream(“file.mm”); mmos.writeObject(a); mmos.close(); //  ObjectInput mmis = new MMInputStream(“file.mm”); Matrix b = (Matrix) mmis.readObject(); 

The code above is easily transformed into serialization code. For this, it suffices to replace the classes MM * with Object *. (MMOutputStream -> ObjectOutputStream).

There is only one unsolved problem. The problem of separation of logical blocks of the file. In our case, the file is divided into:

• Header - header containing the type of the matrix;
• Meta - meta information containing the dimension of the matrix;
• Data - the data.

In the previous package architecture, separate methods were presented that allow recording this information separately. However, ObjectOutput / ObjectInput interfaces obviously do not contain such methods. Those. methods in standard classes are lower level.

To solve this problem, the author proposes to use special markers (bytes) denoting the boundaries of each of the blocks — the header (HEADER_MARKER), Meta-information (META_MARKER) and the element (ELEMENT_MARKER).

Then the writeExternal () / readExternal () methods will look like this:

 @Override public void writeExternal(ObjectOutput out) throws IOException { out.writeInt(rows); out.writeInt(columns); out.writeByte(META_MARKER); //   META for (int i = 0; i < rows; i++) { for (int j = 0; j < columns; j++) { out.writeDouble(self[i][j]); out.writeByte(ELEMENT_MARKER); //   ELEMENT } } } @Override public void readExternal(ObjectInput in) throws IOException, ClassNotFoundException { rows = in.readInt(); columns = in.readInt(); in.readByte(); //   META self = new double[rows][columns]; for (int i = 0; i < rows; i++) { for (int j = 0; j < columns; j++) { self[i][j] = in.readDouble(); in.readByte(); //   ELEMENT } } } 

On the one hand, the recording of additional several bytes will not affect the performance of the application or the readability of the code, on the other hand, it gives an additional opportunity to implement external streams with support for various formats.

From a stream point of view, in our case MMInputStream / MMOutputStream, the record will look like this:

• Write to the buffer, any data received via the writeDouble () / writeInt () methods;
• When one of the markers is received, perform the operations associated with it based on the value buffer.

The following is the main part of the implementation of the MMOutputSteam class:

 public class MMOutputStream extends OutputStream implements ObjectOutput { @Override public void writeByte(int v) throws IOException { switch (v) { case HEADER_MARKER: writeHeader(); break; case META_MARKER: writeMeta(); break; case ELEMENT_MARKER: writeElement(); break; } } @Override public void writeInt(int v) throws IOException { put(String.valueOf(v)); } @Override public void writeDouble(double v) throws IOException { put(String.format(Locale.US, "%.12f", v)); } @Override public void writeObject(Object obj) throws IOException { if (matrix instanceof SparseMatrix) { put(SPARSE_HEADER); } else if (matrix instanceof DenseMatrix) { put(DENSE_HEADER); } writeHeader(); matrix.writeExternal(this); flush(); } private void writeHeader() throws IOException { out.write("%%MatrixMarket "); out.write(buffer[0] + " "); out.write(buffer[1] + " "); out.write("real general"); out.newLine(); } private void writeMeta() throws IOException { dumpBuffer(); out.newLine(); } private void writeElement() throws IOException { dumpBuffer(); out.newLine(); } private void put(String value) { buffer[length++] = value; } private void dumpBuffer() throws IOException { for (int i = 0; i < length; i++) { out.write(buffer[i] + " "); } } } 

Summary

The proposed implementation of the package I / O in Java is quite a good application of the Bridge to existing hierarchies in the Java API. The author hopes that the idea described in the article will become another convenient tool at the disposal of readers and will push them to additional reasoning on this topic.

*

The example described in the article is part of an open library for solving problems of linear algebra - la4j . The considered implementation of the idea can be found in the la4j.io package. The current version supports only the format MatrixMarket.

PS author topic = author la4j.