Attempting to speed up writing to flash

Introduction

Many were faced with the problem that you need to quickly write data to a flash-ku, but a vile piece of hardware a small device limited us in speed. Of course, manufacturers do not stand still, but to increase speed, you definitely need to buy the N-th, model, and the old one just starts gathering dust on the shelf.

Thinking about it, I started thinking: how can you speed up recording without buying a new device. The fact that I came up with, and why I did not succeed and this post will be.

General description of the idea

First you should think: what is a file? The course of my thoughts was simple: the file is a sequence of bytes. A byte is a sequence of bits.

Next, it is worth considering how the flash is arranged. The amateur’s view is as follows: there is a matrix of cells (as far as I know: 1 cell - 1 field effect transistor). In one cell, you can write 1 bit of information. The speed at which information is written depends on the speed at which the controller can set the cell to the desired state.

We go further and understand that it will be possible to increase the speed only if you do not write some of the information in the cell. The last statement, at first glance, seems absurd, but let's recall the file structure. In fact, information is a random set of some numbers. And where there is an accident, there is a probability. Here the amateur again woke up in me, and I assumed that the number 1 and 0 in the file is about the same, and the probability of their appearance = 0.5.

But what if, during the recording, we skip those cells that are set to the state we need? This means that the write speed should increase slightly. To do this, we need to set the cells in a state that we would know. I tried to alternate 1 and 0.

Implementation

To implement, we need to know a few things:

• Write speed to flash drive
• Reading speed from a flash drive (not really needed, but you can find out for a complete picture)
• Bit comparison rate

Go ahead.

Generate random files

Let's write a small program that will generate a random 1Mb file. Hereinafter we will use the C and C ++ cocktail (well, it happened).

#include <iostream> #include <fstream> #include <stdlib.h> #include <time.h> #define fsize 1048576 using namespace std; int main(){ fstream f; f.open("rand_file", ios::out); srand(time(0)); for(int i=0; i<fsize; ++i){ f<<(char)(rand()%255); } f.close(); return 0; } 

I think everything is clear. We generate random bytes and write them to a file.

Speed ​​measurement

Again we start to warm up the compiler, and, we feed it the following code:

 #include <iostream> #include <fstream> #include <sys/time.h> #include <fcntl.h> using namespace std; inline long long gettimeus() { struct timeval tv; gettimeofday( &tv, NULL ); return (long long) tv.tv_sec * 1000000LL + (long long) tv.tv_usec; } int main(int argc, char **argv) { int in = open(argv[1], O_RDONLY); int out = open(argv[2], O_WRONLY | O_CREAT); int size = lseek(in, 0, SEEK_END); lseek(in, 0, SEEK_SET); if (size > 0){ char *fin = new char[size]; long long tmIn = gettimeus(); for(int i=0; i<size; ++i) read (in, &(fin[i]), 1); tmIn = gettimeus() - tmIn; cout<<"Time read: "<<tmIn<<endl; long long tmOut = gettimeus(); for(int i=0; i<size; ++i) write(out, &(fin[i]), 1); tmOut = gettimeus() - tmOut; cout<<"Time write: "<<tmOut<<endl; long long tmIF = gettimeus(); int s = size*8; for(int i=0; i<s; ++i) if (true == false); tmIF = gettimeus() - tmIF; cout<<"Time if: "<<tmIF<<endl; delete [] fin; } close(in); close(out); return 0; } 

Here it is worth clarifying some points:

The gettimeus () function is necessary for counting time, but Windows users most likely will not work, because they do not have sys / time.h. Happy owners of linux distributions should not feel any difficulties while compiling. Next, a little code. The program takes two arguments:

1. File to read.

2. File to write.

And alternately trying to read one file, and write to another file, simultaneously measuring the time spent on these actions. For these purposes, the system calls read () and write () are used. Well, as a bonus, the time of comparisons is measured.

Run the program 10 times and average the resulting values:

Read	Write	If
296127	1082888.7	24024.9

As you would expect, the fastest operation is the comparison, and the slowest is the recording. For further calculations, we move on to other time units. To do this, we divide all the times at the time of comparison. As a result, we get:

Comparison - 1.

Record - 45.07.

Reading - 12.33.

Everything. Now nothing prevents us from starting the simulation.

Performance evaluation

The idea of ​​evaluation is simple: we overtake the file into memory, we overtake each byte into a bit representation. And start counting rewrite time.

 #include <iostream> #include <fstream> #include <sys/time.h> #include <fcntl.h> #include <stdio.h> #include <string.h> using namespace std; const double WRITE = 45.07; const double READ = 12.33; const double IF = 1.0; bool *translateToBitArray(const char *array, int size); int main(int argc, char **argv) { if (argc > 1){ int in = open(argv[1], O_RDONLY); int size = lseek(in, 0, SEEK_END); lseek(in, 0, SEEK_SET); if (size > 0){ char *fin = new char[size]; bool *bFOut = new bool[size*8]; //    for(int i=0; i<size; ++i) read (in, &(fin[i]), 1); bFOut[0] = true; for(int i=1; i<size*8; ++i) //     bFOut[i] = !bFOut[i-1]; bool *bArr = translateToBitArray(fin, size);//     double timeLinear = 0; double timeOpt = 0; timeLinear = WRITE*size*8; //    bool flag = bFOut[0]; //  for(int i=0; i<size*8; ++i){ //  timeOpt += IF; if (bArr[i] != flag){ bFOut[i] = bArr[i]; timeOpt += WRITE; } flag = !flag; } // flag = false; for(int i=0; i<size*8; ++i) if (bArr[i] != bFOut[i]){ flag = true; break; } if (flag) printf("ERROR\n"); printf("Time linear = %.6lf\nTime opt = %.6lf\n",timeLinear, timeOpt); delete [] fin; delete [] bFOut; } close(in); } return 0; } bool *translateToBitArray(const char *array, int size) { bool *bArr = new bool[size*8]; bool *tmp = new bool[8]; int pos = 0; for(int i=0; i<size; ++i){ unsigned char c = array[i]; int ppp = 8; fill_n(tmp, 8, 0); while (c != 0){ tmp[ppp--] = (c & 1); c = c >> 1; } memcpy(&(bArr[pos]), tmp, 8); pos += 8; } delete [] tmp; return bArr; } 

No checks, no tips. If it works, well, if not, we will fly out and that's it. Run the program 10 times on different files (first random, and then real ones).

Simulation results

As a result, we get the acceleration by writing ~ 1.71 times with respect to the normal recording.

A spoon of tar

Unfortunately, nothing (except for the inheritance) in the world is free, and, to speed up the recording, you have to pay by slowing down data deletion, since it is necessary to maintain the alternation of bits. Plus the complication of disaster recovery procedures, etc.

findings

The conclusions are disappointing.

I have not found a single way to write individual bits to a flash drive. Of course, there is nothing surprising about it (why then do we need a controller if the OS had to perform its functions). The complication of the controller's functions is also not foreseen, since occasional failures (read: “Vasya pulled the flash drive out before the controller managed to restore the structure”) can lead to unreliable recording and mistrust to the manufacturer.

Of course, there are pluses too:

1. Increase the record almost 2 times.
2. The manufacturer does not need to develop “mega-new” technologies of ultrafast cells, but only complicate the recording procedure (and, of course, selling drives is much more expensive).
3. It is possible (but not a fact) that the cell service life will increase.

Something like that. Waiting for your comments.