Databases in MIDP Part 1: The Concept of Record Management System
One of the key components of MIDP is the Record Management System (RMS). This is an API that provides the ability to store data locally in the device’s memory. For most MIDP-compatible phones, this is the only way to store data - only a small number of devices support access to a regular file system. It is easy to guess that a full understanding of the RMS mechanism is necessary for writing any application that requires the storage of local data.
This is the first article of the cycle, which will address the most common problems related to the use of RMS in applications, for example, interaction with external data sources, such as relational databases. To begin, we learn what RMS can offer us, and write some simple debuggers.
From the name it is clear that RMS is a system for managing records. A record is a data item. RMS does not impose any restrictions on the contents of the record; it can contain a number, a string, an array, an image — anything that can be represented as a sequence of bytes. If you can encode the available data into a binary format (and decode it back), then you can save it in the record, if, of course, they fit into the size limit imposed by the system.
')
Many newcomers to RMS are puzzled by the notion of writing. They ask: “where are the fields?”, Wondering how the system divides the individual records into separate data sequences. The answer is simple: the RMS record contains no fields. More precisely, the record contains one binary field of arbitrary length. The function of interpreting the contents of the record rests entirely with the application. RMS provides storage and a unique identifier, nothing more. This creates difficulties for applications, but keeps the RMS simple and flexible, which is quite important for the MIDP subsystem.
At the API level, entries are simply arrays of bytes.
A record store is an ordered collection of records. Each entry belongs to the repository and is accessible only through it. The storage guarantees atomic reading and writing of data, preventing their damage.
When a record is created, the repository assigns it a unique integer identifier (record ID). The first record gets id 1, the second one gets 2, and so on. This is not an index: when you delete a record, the remaining elements are not renumbered.
The name is used to identify the repository inside the MIDlet. The name can contain from 1 to 32 unicode characters and must be unique within the MIDlet that created the repository. In MIDP 1.0, repositories cannot be used by more than one application. MIDP 2.0 optionally allows this, in which case the storage is identified not only by the name, but also by the name and manufacturer of the application that created the storage.
In addition, the repository contains information about the last modified date and version. Applications can also bind to the repository a data change event handler in it.
At the API level, storage is represented by an instance of the javax.microedition.rms.RecordStore class. All classes and interfaces of RMS are defined in the javax.microedition.rms package.
The amount of memory available for storing records differs across devices. The MIDP specification requires a backup of at least 8 KB of memory for permanent data storage. At the same time the size of one record is not limited. RMS provides methods for determining the size of the record size, the total storage size and the size of free memory. Remember that permanent memory is common for all applications, use it sparingly.
Any MIDlet using RMS must specify the minimum storage size in bytes necessary for its operation. For this, the MIDlet-Data-Size attribute must be set in the jar-file manifest and in the jad-file. Do not specify there too much value - some devices may prohibit the installation of the application if there is not enough free space. In practice, most devices allow applications to go beyond the starting data size.
Please note that some MIDP implementations require you to specify additional attributes related to the required memory size. This should be stated in the device documentation.
Operations with permanent memory are usually performed more slowly than with operational. In particular, on some platforms, data recording can take a long time. To improve performance, use caching of frequently used data in RAM. In order not to slow down the response of the user interface, do not perform RMS operations in the flow of event processing of the MIDlet.
RMS operations are thread-safe. However, flows need to be coordinated among themselves, as when working with any shared resource. This also applies to simultaneously running midlets using the same storage.
In general, the RMS API methods throw a few exceptions (in addition to standard exceptions like java.lang.IllegalArgumentException). These exceptions are defined in the javax.microedition.rms package:
Note that for brevity, we will ignore exception handling in some examples (because of its simplicity).
We will devote the rest of the article to the main operations with records through the RMS API. Some of them are presented in the RMSAnalyzer class, intended for analyzing repositories. You can use it as a debugging tool in your projects.
The list of storages can be obtained using RecordStore.listRecordStores (). This static method returns an array of strings, each of which is the name of the storage that belongs to the midlet. If no storage is created, null is returned.
The RMSAnalyzer.analyzeAll () method uses listRecordStores () to call analyze () for each repository:
Please note that the array contains only the names of the repositories created by our midlet. The MIDP specification does not contain any way to get a list of all the repositories of the other MIDlets. MIDP 1.0 does not allow access to foreign repositories. In MIDP 2.0, an application can mark a repository as shareable, but other midlets can use it only if they know its name.
RecordStore.openRecordStore () is used to open (and sometimes create) a repository. This static method returns an instance of the RecordStore object, as seen from this version of RMSAnalyzer.analyze ():
The second parameter of the openRecordStore () method indicates whether the store will be created if it does not exist. In MIDP 2.0, the following openRecordStore () form is used to open a repository created by another application:
The manufacturer and name of the midlet must match the ones specified in the manifest.
After the work is finished, close the repository by calling RecordStore.closeRecordStore (), as in the analyze () method above.
The RecordStore instance is unique in the midlet: after it is opened, all subsequent openRecordStore () calls with the same arguments will return a reference to the same object. This copy is common to all midlets in the collection.
Each instance of RecordStore counts how many times a repository has been opened. It will not close until closeRecordStore () has been called the same number of times. After the storage is closed, an attempt to use it will result in a RecordStoreNotOpenException.
To create a repository (not available to other midlets), you need to call openRecordStore (), setting the second parameter to true:
To perform the initial initialization of the repository, check the value of getNextRecordID () - if it is equal to 1, there are no records in the repository:
An alternative way is to check the value returned by getNumRecords ():
To create a shared storage (only in MIDP 2.0), use the following openRecordStore () call option with four parameters:
If the second parameter is true and the storage does not exist, the last two parameters control its access mode and the ability to write. Access mode determines whether other applications can use this storage. Two options are possible: RecordStore.AUTHMODE_PRIVATE (only our application has access) or RecordStore.AUTHMODE_ANY (any application has access). The writable flag determines whether another application will have write access — if writable = false, it can only read data.
The storage owner application can change these settings at any time using RecordStore.setMode ():
In fact, it is best to create a private repository, and open access only after its initialization.
Recall that records are arrays of bytes. To add a new record to the public storage, use the RecordStore.addRecord () method:
You can create an empty entry by passing null as the first parameter. The second and third parameters specify the starting position of the reading and the number of bytes to be saved. If successful, the record id is returned, otherwise an exception is thrown (for example, RecordStoreFullException).
At any time, you can update the record using RecordStore.setRecord ():
You can find out what id will be assigned to the next record being added, using the RecordStore.getNextRecordID () method. All existing entries have id less than this.
In the second part, we will look at how to convert objects and other data into an array of bytes.
To read entries, use RecordStore.getRecord () in one of two forms. In the first variant, this method creates an array of the required length and writes data to it:
In the second variant, the data is copied to the already created array, starting from the specified position, and the number of bytes written is returned:
The array must be of sufficient length to contain the data, otherwise java.lang.ArrayIndexOutOfBoundsException will occur. To determine the required size of the array is used RecordStore.getRecordSize (). In fact, the first form of getRecord () is equivalent to the following:
The second form is useful if you loop through many records in a loop, as the number of memory requests is reduced. For example, you can use it with getNextRecordID () and getRecordSize () to search through all the records in the repository:
However, it is better to use RecordStore.enumerateRecords () for this. We will consider this method in the third part of a series of articles.
The function RecordStore.deleteRecord () is used to delete records:
After deleting a record, any attempt to use it results in an InvalidRecordIDException.
You can delete the entire repository using RecordStore.deleteRecordStore ():
Storage cannot be removed while it is open by any application. Only the MIDlet that created it can delete the repository.
There are only a few RMS operations left, all of which are methods of the RecordStore class:
A MIDlet can also monitor changes to the repository by registering a handler with addRecordListener (); removeRecordListener () is designed to remove it. In the third part, these methods will be discussed in more detail.
We end this article with the source code of the RMSAnalyzer class, our repository analyzer. For the analysis you need to run the following code:
By default, the output is redirected to System.out and looks like this:
This format is useful when testing with the J2ME Wireless Toolkit. When testing on a real device, you can send the analyzer output to a serial port or even over the network. To do this, you need to create a new class with the RMSAnalyzer.Logger interface and pass it an instance to the RMSAnalyzer constructor.
The article ends with a J2ME Wireless Toolkit project called RMSAnalyzerTest, demonstrating the use of the analyzer: pastebin.com/n36QLuAs
The rest of the article in English can be seen here . Does it make sense to continue their translation?
This is the first article of the cycle, which will address the most common problems related to the use of RMS in applications, for example, interaction with external data sources, such as relational databases. To begin, we learn what RMS can offer us, and write some simple debuggers.
Key Concepts
Records
From the name it is clear that RMS is a system for managing records. A record is a data item. RMS does not impose any restrictions on the contents of the record; it can contain a number, a string, an array, an image — anything that can be represented as a sequence of bytes. If you can encode the available data into a binary format (and decode it back), then you can save it in the record, if, of course, they fit into the size limit imposed by the system.
')
Many newcomers to RMS are puzzled by the notion of writing. They ask: “where are the fields?”, Wondering how the system divides the individual records into separate data sequences. The answer is simple: the RMS record contains no fields. More precisely, the record contains one binary field of arbitrary length. The function of interpreting the contents of the record rests entirely with the application. RMS provides storage and a unique identifier, nothing more. This creates difficulties for applications, but keeps the RMS simple and flexible, which is quite important for the MIDP subsystem.
At the API level, entries are simply arrays of bytes.
Record Storage
A record store is an ordered collection of records. Each entry belongs to the repository and is accessible only through it. The storage guarantees atomic reading and writing of data, preventing their damage.
When a record is created, the repository assigns it a unique integer identifier (record ID). The first record gets id 1, the second one gets 2, and so on. This is not an index: when you delete a record, the remaining elements are not renumbered.
The name is used to identify the repository inside the MIDlet. The name can contain from 1 to 32 unicode characters and must be unique within the MIDlet that created the repository. In MIDP 1.0, repositories cannot be used by more than one application. MIDP 2.0 optionally allows this, in which case the storage is identified not only by the name, but also by the name and manufacturer of the application that created the storage.
In addition, the repository contains information about the last modified date and version. Applications can also bind to the repository a data change event handler in it.
At the API level, storage is represented by an instance of the javax.microedition.rms.RecordStore class. All classes and interfaces of RMS are defined in the javax.microedition.rms package.
RMS aspects
Data Size Limits
The amount of memory available for storing records differs across devices. The MIDP specification requires a backup of at least 8 KB of memory for permanent data storage. At the same time the size of one record is not limited. RMS provides methods for determining the size of the record size, the total storage size and the size of free memory. Remember that permanent memory is common for all applications, use it sparingly.
Any MIDlet using RMS must specify the minimum storage size in bytes necessary for its operation. For this, the MIDlet-Data-Size attribute must be set in the jar-file manifest and in the jad-file. Do not specify there too much value - some devices may prohibit the installation of the application if there is not enough free space. In practice, most devices allow applications to go beyond the starting data size.
Please note that some MIDP implementations require you to specify additional attributes related to the required memory size. This should be stated in the device documentation.
Work speed
Operations with permanent memory are usually performed more slowly than with operational. In particular, on some platforms, data recording can take a long time. To improve performance, use caching of frequently used data in RAM. In order not to slow down the response of the user interface, do not perform RMS operations in the flow of event processing of the MIDlet.
Thread safety
RMS operations are thread-safe. However, flows need to be coordinated among themselves, as when working with any shared resource. This also applies to simultaneously running midlets using the same storage.
Exceptions
In general, the RMS API methods throw a few exceptions (in addition to standard exceptions like java.lang.IllegalArgumentException). These exceptions are defined in the javax.microedition.rms package:
- InvalidRecordIDException - the operation could not be performed because an invalid record id was passed.
- RecordStoreFullException - Out of available memory.
- RecordStoreNotFoundException - the specified store does not exist.
- RecordStoreNotOpenException - the application is trying to use a repository that has been closed.
- RecordStoreException - the superclass of previous exceptions, is also thrown when common errors are not covered by them.
Note that for brevity, we will ignore exception handling in some examples (because of its simplicity).
Using RMS
We will devote the rest of the article to the main operations with records through the RMS API. Some of them are presented in the RMSAnalyzer class, intended for analyzing repositories. You can use it as a debugging tool in your projects.
Storage search
The list of storages can be obtained using RecordStore.listRecordStores (). This static method returns an array of strings, each of which is the name of the storage that belongs to the midlet. If no storage is created, null is returned.
The RMSAnalyzer.analyzeAll () method uses listRecordStores () to call analyze () for each repository:
public void analyzeAll() { String[] names = RecordStore.listRecordStores(); for( int i = 0; names != null && i < names.length; ++i ) { analyze( names[i] ); } } Please note that the array contains only the names of the repositories created by our midlet. The MIDP specification does not contain any way to get a list of all the repositories of the other MIDlets. MIDP 1.0 does not allow access to foreign repositories. In MIDP 2.0, an application can mark a repository as shareable, but other midlets can use it only if they know its name.
Opening and closing the repository
RecordStore.openRecordStore () is used to open (and sometimes create) a repository. This static method returns an instance of the RecordStore object, as seen from this version of RMSAnalyzer.analyze ():
public void analyze( String rsName ) { RecordStore rs = null; try { rs = RecordStore.openRecordStore( rsName, false ); analyze( rs ); // } catch( RecordStoreException e ) { logger.exception( rsName, e ); } finally { try { rs.closeRecordStore(); } catch( RecordStoreException e ){ // } } } The second parameter of the openRecordStore () method indicates whether the store will be created if it does not exist. In MIDP 2.0, the following openRecordStore () form is used to open a repository created by another application:
... String name = "mySharedRS"; String vendor = "EricGiguere.com"; String suite = "TestSuite"; RecordStore rs = RecordStore.openRecordStore( name, vendor, suite ); ... The manufacturer and name of the midlet must match the ones specified in the manifest.
After the work is finished, close the repository by calling RecordStore.closeRecordStore (), as in the analyze () method above.
The RecordStore instance is unique in the midlet: after it is opened, all subsequent openRecordStore () calls with the same arguments will return a reference to the same object. This copy is common to all midlets in the collection.
Each instance of RecordStore counts how many times a repository has been opened. It will not close until closeRecordStore () has been called the same number of times. After the storage is closed, an attempt to use it will result in a RecordStoreNotOpenException.
Creation of storage
To create a repository (not available to other midlets), you need to call openRecordStore (), setting the second parameter to true:
... // RecordStore rs = null; try { rs = RecordStore.openRecordStore( "myrs", true ); } catch( RecordStoreException e ){ // } ... To perform the initial initialization of the repository, check the value of getNextRecordID () - if it is equal to 1, there are no records in the repository:
if( rs.getNextRecordID() == 1 ){ // } An alternative way is to check the value returned by getNumRecords ():
if( rs.getNumRecords() == 0 ){ // , } To create a shared storage (only in MIDP 2.0), use the following openRecordStore () call option with four parameters:
boolean writable = true; rs = RecordStore.openRecordStore( "myrs", true, RecordStore.AUTHMODE_ANY, writable ); If the second parameter is true and the storage does not exist, the last two parameters control its access mode and the ability to write. Access mode determines whether other applications can use this storage. Two options are possible: RecordStore.AUTHMODE_PRIVATE (only our application has access) or RecordStore.AUTHMODE_ANY (any application has access). The writable flag determines whether another application will have write access — if writable = false, it can only read data.
The storage owner application can change these settings at any time using RecordStore.setMode ():
rs.setMode( RecordStore.AUTHMODE_ANY, false ); In fact, it is best to create a private repository, and open access only after its initialization.
Adding and modifying entries
Recall that records are arrays of bytes. To add a new record to the public storage, use the RecordStore.addRecord () method:
... byte[] data = new byte[]{ 0, 1, 2, 3 }; int recordID; recordID = rs.addRecord( data, 0, data.length ); ... You can create an empty entry by passing null as the first parameter. The second and third parameters specify the starting position of the reading and the number of bytes to be saved. If successful, the record id is returned, otherwise an exception is thrown (for example, RecordStoreFullException).
At any time, you can update the record using RecordStore.setRecord ():
... int recordID = ...; // ID byte[] data = new byte[] { 0, 10, 20, 30 }; rs.setRecord( recordID, data, 1, 2 ); // 10, 20 ... You can find out what id will be assigned to the next record being added, using the RecordStore.getNextRecordID () method. All existing entries have id less than this.
In the second part, we will look at how to convert objects and other data into an array of bytes.
Reading records
To read entries, use RecordStore.getRecord () in one of two forms. In the first variant, this method creates an array of the required length and writes data to it:
... int recordID = .... // ID byte[] data = rs.getRecord( recordID ); ... In the second variant, the data is copied to the already created array, starting from the specified position, and the number of bytes written is returned:
... int recordID = ...; // ID byte[] data = ...; // int offset = ...; // int numCopied = rs.getRecord( recordID, data, offset ); ... The array must be of sufficient length to contain the data, otherwise java.lang.ArrayIndexOutOfBoundsException will occur. To determine the required size of the array is used RecordStore.getRecordSize (). In fact, the first form of getRecord () is equivalent to the following:
... byte[] data = new byte[ rs.getRecordSize( recordID ) ]; rs.getRecord( recordID, data, 0 ); ... The second form is useful if you loop through many records in a loop, as the number of memory requests is reduced. For example, you can use it with getNextRecordID () and getRecordSize () to search through all the records in the repository:
... int nextID = rs.getNextRecordID(); byte[] data = null; for( int id = 0; id < nextID; ++id ) { try { int size = rs.getRecordSize( id ); if( data == null || data.length < size ) { data = new byte[ size ]; } rs.getRecord( id, data, 0 ); processRecord( rs, id, data, size ); // - } catch( InvalidRecordIDException e ){ // } catch( RecordStoreException e ){ handleError( rs, id, e ); // } } ... However, it is better to use RecordStore.enumerateRecords () for this. We will consider this method in the third part of a series of articles.
Deleting records and repositories
The function RecordStore.deleteRecord () is used to delete records:
... int recordID = ...; rs.deleteRecord( recordID ); ... After deleting a record, any attempt to use it results in an InvalidRecordIDException.
You can delete the entire repository using RecordStore.deleteRecordStore ():
... try { RecordStore.deleteRecordStore( "myrs" ); } catch( RecordStoreNotFoundException e ){ // } catch( RecordStoreException e ){ // } ... Storage cannot be removed while it is open by any application. Only the MIDlet that created it can delete the repository.
Other operation
There are only a few RMS operations left, all of which are methods of the RecordStore class:
- getLastModified () returns the time of the last modification of the repository in the same format as System.currentTimeMillis ().
- getName () returns the name of the repository.
- getNumRecords () returns the number of records in the repository.
- getSize () returns the full size of the repository in bytes, including the length of the records and service fields necessary for the system to organize it.
- getSizeAvailable () returns the size of the free space in bytes. The actual available size may be smaller due to the extra bytes used by the repository to store each record.
- getVersion () returns the version number of the repository. This is a positive integer increasing by one each time the data changes.
A MIDlet can also monitor changes to the repository by registering a handler with addRecordListener (); removeRecordListener () is designed to remove it. In the third part, these methods will be discussed in more detail.
RMSAnalyzer class
We end this article with the source code of the RMSAnalyzer class, our repository analyzer. For the analysis you need to run the following code:
... RecordStore rs = ...; // RMSAnalyzer analyzer = new RMSAnalyzer(); analyzer.analyze( rs ); ... By default, the output is redirected to System.out and looks like this:
=========================================
Record store: recordstore2
Number of records = 4
Total size = 304
Version = 4
Last modified = 1070745507485
Size available = 975950
Record #1 of length 56 bytes
5f 62 06 75 2e 6b 1c 42 58 3f _b.ukBX?
1e 2e 6a 24 74 29 7c 56 30 32 ..j$t)|V02
5f 67 5a 13 47 7a 77 68 7d 49 _gZ.Gzwh}I
50 74 50 20 6b 14 78 60 58 4b PtP kx`XK
1a 61 67 20 53 65 0a 2f 23 2b .ag Se./#+
16 42 10 4e 37 6f .B.N7o
Record #2 of length 35 bytes
22 4b 19 22 15 7d 74 1f 65 26 "K.".}te&
4e 1e 50 62 50 6e 4f 47 6a 26 N.PbPnOGj&
31 11 74 36 7a 0a 33 51 61 0e 1.t6z.3Qa.
04 75 6a 2a 2a .uj**
Record #3 of length 5 bytes
47 04 43 22 1f GC".
Record #4 of length 57 bytes
6b 6f 42 1d 5b 65 2f 72 0f 7a koB.[e/rz
2a 6e 07 57 51 71 5f 68 4c 5c *n.WQq_hL\
1a 2a 44 7b 02 7d 19 73 4f 0b .*D{.}.sO.
75 03 34 58 17 19 5e 6a 5e 80 u.4X..^j^?
2a 39 28 5c 4a 4e 21 57 4d 75 *9(\JN!WMu
80 68 06 26 3b 77 33 ?h.&;w3
Actual size of records = 153
-----------------------------------------
This format is useful when testing with the J2ME Wireless Toolkit. When testing on a real device, you can send the analyzer output to a serial port or even over the network. To do this, you need to create a new class with the RMSAnalyzer.Logger interface and pass it an instance to the RMSAnalyzer constructor.
The article ends with a J2ME Wireless Toolkit project called RMSAnalyzerTest, demonstrating the use of the analyzer: pastebin.com/n36QLuAs
The rest of the article in English can be seen here . Does it make sense to continue their translation?
Source: https://habr.com/ru/post/85856/
All Articles