Hi, Habr! I am involved in the development of an ECM system. And in a small series of articles I want to share our experience and history of developing my React Data Grid (hereinafter referred to as a grid), namely:
Our system has a web application in which users work with lists of documents, search results, directories. Moreover, the lists can be either small (10 employees) or very large (50,000 contractors). To display these lists, we developed our grid:
When we first started developing a web application, I wanted to find a ready library for displaying a grid that can do everything we need: sort and group records, drag and stretch columns, work with multiple selections, filter and calculate totals by columns, portionwise upload data from the server and display tens of thousands of records.
Let me explain the last requirement “to display tens of thousands of records”. In grids this requirement is implemented in several ways: paging, infinity scrolling, virtual scrolling.
The paging and infinity scrolling approaches are common on web sites, you use them every day. For example, paging on Google:
Or infinity scrolling in the same Google in pictures, where the next portion of pictures is loaded when you scroll through the first portion:
But virtual scrolling (I will call virtual scrolling hereafter) is rarely used on the web, its main difference from infinity scrolling is the ability to quickly scroll very large lists to any place. In this case, only the data visible to the user will be loaded and displayed.
For our web application, we wanted to use virtual scrolling. I agree that scrolling to any place in the list of 10,000 entries is rather a fictional case. However, arbitrary scrolling within 500–1000 records is a live case.
When virtual scrolling is implemented, the software management API of this scrolling is often implemented. This is a very important feature. Software scrolling is used, for example, to position a highlighted entry in the middle of the screen when opening a directory:
Let's return to the requirements. What else we needed:
In general, there were many requirements.
Not long examining existing libraries, we stumbled upon DevExtreme JavaScript Data Grid. According to the functional requirements, this grid covered all our needs and had a very presentable appearance. However, the technological requirements are not suitable (not react, not redux, not flexbox). At that time, DevExtreme had no react grid.
Well, let not react, we decided, for that the grid is beautiful and functional, we will use it. And they added the library to their project. It turned out we added 3 MB of scripts.
For a couple of weeks, we integrated the grid into our web application and enhanced the basic functionality:
In the course of screwing up the grid, two serious problems emerged and a whole heap of less serious ones.
Making friends with DevExtreme JavaScript Data Grid with redux is very difficult. We managed to manage the settings of the columns and the selection of records through redux, but storing the batch-loaded data in redux and performing CRUD operations on them through redux is unrealistic. I had to make a crutch that, bypassing redux, manipulated the grid data. The crutch turned out to be complex and fragile. It was the first alarm bell that the grid does not suit us, but we continued to screw it.
No virtual scrolling management API. We couldn’t refuse software scrolling management, we had to override the DevExtreme sources and find the internal scrolling management API. Of course, this API had a mountain of restrictions, because it was designed for internal use. As a result, we achieved that the internal API more or less worked on our cases, but, again bypassing redux, and again a bunch of crutches.
Less serious problems surfaced constantly, because the standard DevExtreme JavaScript Data Grid functionality weren’t completely matched, and we tried to correct it:
Although the DevExtreme grid in terms of functionality contained everything we needed, but almost all the standard functionality wanted to be rewritten. During its use, hundreds of lines of difficult to understand code were added, which tried to solve problems of interaction with redux and react, it was difficult to use a non-react grid in the react application.
After some time using DevExtreme, it was decided to abandon it. Throw out all the hacks, complex code, as well as 3 MB of DevExtreme scripts. And find or write a new grid.
This time, we were more attentive to the study of existing grids. MS Fabric DetailsList, ReactVirtualized Grid, DevExtreme React Grid, Telerik Grid, KendoUI Grid were studied.
The requirements remain the same, but they have already formed into a list that is understandable to us.
Technology Requirements:
Requirements for functionality:
By this time, the first version of DevExtreme React Grid has already appeared, but we immediately discarded it for the following reasons:
Analysis of existing solutions showed that there is no “silver bullet”. Grid, which would close all our requirements, does not exist. It was decided to write our grid, which we will develop in terms of functionality in the direction we need, and be friends with the technologies needed by our product.
Grid development started with prototypes, where we tried the most difficult topics for us:
The most difficult was to make a virtual scrolling. In a big way, it is made in one of 3 ways:
1. Page-based virtualization
Data is drawn in chunks - pages. When scrolling, visible pages are added, invisible ones are deleted. The page consists of 20-60 lines (usually the size is customized). In this way, the products went: DevExtreme JavaScript Data Grid, MS Fabric DetailsList.
2. Line Virtualization
Only visible lines are drawn. As soon as the line leaves the screen, it is immediately deleted. This way went the products: ReactVirtualized Grid, DevExtreme React Grid, Telerik Grid.
3. Canvas
All lines and their contents are drawn using Canvas. So did Google Docs.
When developing the grid, we made prototypes for all three variants of virtualization (even for Canvas). And they chose paged virtualization.
Line-by-line virtualization had problems with rendering speed in the prototype. As soon as the contents of the lines became complicated (a lot of text, highlighting, trimming, icons, a large number of columns, and flexbox everywhere), it became expensive to add / delete lines several times a second. Of course, the results also depend on the browser (we did support, including for ie11, edge):
The Canvas version was very seductive in rendering speed, but time consuming. It was suggested to draw everything: text, text wrap, text trimming, highlighting, icons, dividing lines, selection, indents. Make a reaction to pressing the mouse buttons on the Canvas, highlighting the lines when you hover the cursor. At the same time, on top of the Canvas should be put some Dom-elements (showing hints, "pop-up actions" above the line). Still needed to solve the problem of blur text and icons in the Canvas. All this is long and difficult to do. Although we mastered the prototype. In this case, any customization of rows and cells in the future we would have resulted in greater complexity.
The selected page-based virtualization had advantages over line-by-line, which determined its choice:
A separate question is the choice of page size. Above, I wrote that the size is customizable and is usually 20-60 lines. A large page is drawn for a long time, a small one leads to frequent display of a “white screen” when scrolling. Experimentally, the page size was 25 lines. However, for ie11, the size has been reduced to 5 lines. It feels like the interface in IE is more responsive if you draw many small pages with small delays than one large one with a large delay.
Page-based virtualization needed to be implemented using react. For this, several tasks had to be solved:
Task 1. How to add / delete pages through react when scrolling?
To solve this problem introduced the concept:
A model is information on which to build a presentation. A view is a React component.
In essence, the task of virtualization after that was reduced to manipulating page models: keeping a list of page models, adding and deleting models when scrolling. And according to the list of models through react, build / rebuild the mapping:
In the course of implementation, the rules for working with page models were formed:
Task 2. How to display scollbar?
Virtual scrolling assumes that a scrollbar is available, which takes into account the sizes of the list and allows scrolling to any place:
How to display such a scollbar? The simplest solution is to draw an invisible div of the desired size instead of real data. And already on top of this div display the visible pages:
Task 3. How to monitor the size of the viewport?
Viewport is the visible data area of ​​the grid. Why keep track of its size? To calculate the number of pages to display to the user. Suppose we have a small page size (5 lines) and a large screen resolution (1920x1080). How many pages should be displayed to the user to close the entire viewport?
You can solve this problem if you know the height of the viewport and the height of one page. Now let's complicate the task, suppose the user dramatically changes the scale in the browser - sets 50%:
The situation with the scale shows that it is not enough to know the size of the viewport once, the size must be monitored. And now we’ll completely complicate the task: html elements don’t have a resize event to which you can subscribe and track the size. Resize is only for the window object.
The first thing that comes to mind is to use a timer and constantly poll the height of the html element. But there is an even better solution that we saw in DevExtreme JavaScript Data Grid: create an invisible iframe, stretch it to the grid size and subscribe to the resize event in iframe.contentWindow:
PS This is not the end. In the next article I will tell how we made friends with our redux grid.
To get a full-fledged virtual scrolling, many other tasks had to be solved. But those described above were themselves interesting. Here are a few more tasks that came up too:
If you have questions about the implementation, you can write them in the comments.
Source: https://habr.com/ru/post/457504/
All Articles