How we created a new product, removing 130 thousand lines of code from the old

To begin with, about 15 years ago (in the early 2000s) we needed to create our own editor of text documents. Why your own? Yes, because not quite text documents and not WYSIWYG editor. You will say that this is not a document editor at all. And what then is a document editor? What features should it contain? And how did it occur to us to make such a bike? That's how...

Origins

Once upon a time, during the heyday of editors ed, vi, em, and other antiquities, there was a real need to edit text files. Basically, various configs and source code, and not documents in their modern sense. At that time, there were no Unicode encodings yet and the whole variety of text files was probably expressed only in contrast to cr + lf from simple lf as a character for the end of a line. And then the text was really text, from 32 to 127th ASCII character of the table and the upper half of the codes, which could denote different international characters (in each country their own). We’ll just keep silent about the mbcs encoding, because it’s not about encodings, but about the documents themselves.

And suddenly, Apple appeared with its editors LisaWrite for Apple Lisa, released in 1983, and MacWrite for Apple Macintosh, released in 1984, which were already real WYSIWYG editors.
')
To tell you the truth, WYSIWYG appeared another 10 years earlier, in the form of the Bravo editor, developed in the well-known Xerox PARC for Alto computers, but these machines were never released to the mass market. And other home computers that were popular in the 1970s simply lacked the graphics capabilities needed to display documents in WYSIWYG mode.

What gave the world the principle of WYSIWYG? People began to work on computers with documents as if they were creating them on paper. They used various formatting, alignment and text styles, inserted figures, formulas and diagrams into the text, made tables and generally designed the document as they pleased, since the WYSIWYG editor provided the output of all this beauty to print exactly as it looks. on the screen (as far as accuracy is achievable here, this is a separate conversation, because the monitor and the printer have completely different dpi, but the WYSIWYG editor tries as much as it can to bring the drawing of the document on the screen to what is then printed on the paper).

Let's skip the whole long way of the emergence and maturity of editors such as Windows Write, WordPad, Word (from Word 1.0 to Word 2016 ).

Now we are used to that if we are talking about editing documents (not source files or configs, but various office and other documents), then a WYSIWYG editor is simply necessary and you cannot live without it. But is it true? For example, your goal is to create some similar documents based on forms. Or do you keep some large database of similar documents (for example, specific contracts or tenders). And here WYSIWYG will only interfere, allowing you to create some arbitrariness in the documents and adding formatting elements that are not supposed to be added to the document according to a previously accepted agreement. only the text itself is important. For example, you can agree that you cannot use multi-column text, you cannot use footnotes in the text and footers, and also not on top of the text of the image and 3D-inscriptions.

Something like this was in our case. But the situation was complicated by several other factors. In particular, in addition to the text itself, it was necessary to maintain specific markup of documents. And each element of this markup should have some kind of behavior. Approximately as if the element [spoiler] knew how to fold. And the [url] element was able to send us on the link by click. Only such elements were not 2 and not 3, but a couple of dozen. Each with their own behavior. And the documents themselves were ordinary text, but in addition with pictures, formulas and tables. That is, in fact, we needed an editor who can build on top of a regular RTF document containing text, pictures, tables, impose special markup, and somehow tricky to work with it.

It was possible to go here along the path of some specialized macros for the Word, but
a) the Word is not always available at hand, you need to buy licenses for it
b) with the advent of new versions of MS Office, migration problems from one version of Word to another might appear
c) I want to be able to absolutely complete control over the behavior of the editor
d) as already written, WYSIWYG interferes. We needed an editor to create documents that were completely identical in terms of the design possibilities used in them. Those. it was agreed in advance that all documents should contain text typed in such and such a font of such and such size, with such and indented paragraphs, etc. Under such conditions, the functionality of setting the size and type of the font, flying over the text of inscriptions, various indents and alignments, including, for example, multi-column text, would only interfere with users.

Result

No sooner said than done. Editor created. For 15 years, about 45 person-years were spent on the development of the editor, since On average, 3 people worked full time on the project. The editor's architecture turned out to be so specific and tailored to our tasks, and therefore this editor cannot be called a text editor in a general sense. And in this form it could be used only within our company. Now the editor has precisely passed through the middle of his life cycle, has acquired a certain architecture, has been filled with a large number of functions, and further work on it involves mainly refactoring and supporting the necessary changes.

Reincarnation

For a long time I wondered whether our editor could somehow be brought outside to a more open market. After all, so much time has been spent and so many useful things have been done that it is strange not to apply at least some of them in some other project with a different purpose. Since in our editor many things were implemented concerning the general principles of working with documents, it was decided to try to remove everything specific from the project and leave only those functions that are inherent in the “ordinary” text editor. And see what comes out of it.

I want to warn in advance that all the functions of the editor, including drawing the document in the editing window, are implemented completely from scratch by our code. Those. this is not some rich edit that just loads the text. Calculating paragraph positions, dividing a paragraph into lines, drawing a carriage, selection - everything is only manually, only hardcore. Those. in principle, we can do anything with this editor.

Without delaying matters indefinitely and without too much doubt, I called the new editor TexEd (from the phrase Text Editor). The code of the current editor was taken and everything that opens “our internal functions” outside (in the application interface) was removed from it. The implementation code for these functions has also been deleted as far as possible. Before the start of functional cuts, the project code was 430 thousand lines of our code (1 million 300 thousand, if you count the used third-party components together with the code).

After discarding or hiding a specific functional, the size of the code was 297 thousand lines, i.e. Approximately 1/3 of the project was thrown off. Of course, it was possible to reject more, but the source can be cleaned from the heritage later; this “extra” unused code does not affect the appearance and work of the editor.

The great advantage of the editor is that in fact it is a single exe-file and several text files with directories and settings. Not like monstrous office packages.

A large reservoir of functions contained various automatic processing (for example, the formation of a table of contents based on text analysis, without taking into account styles, the meaning of the text, identifying structural units, etc.), as well as checking the compliance of documents with certain design rules. But since these were our internal rules, not applicable to other users, this functionality was mostly cut out from the editor (if it was returned, it was already in some new, revised form).

The editor is not demanding of resources and can open 100-megabyte (and more) documents, including with tables, and quite smartly working with them (which is not true of MS Office and Open Office, which can be hung up on such documents, at least in page layout mode). The limitation on the volume of opened documents is that at the moment we support only 32-bit build, i.e. the application cannot technically allocate more than 4 GB of memory.

What functions are left in the editor?

1) Format support:
- RTF, WordML 2003 (XML), DOCX, HTML - native reading and writing (we independently read and write the format in the part of the subset of the supported design elements)
- ODT - we read through Open / Libre Office by converting it with funds to one of the formats that we support natively
- DOC - we read through the Word converter as well by converting to one of the formats that we support natively. To do this, simply install the free Office 2007 Compatibility Pack (FileFormatConverters.exe), while MS Office itself does not need to be installed.
- TXD - its own binary editor format, supporting all the same elements that we support in other formats. Why not? And as a clipboard format when copying pieces of documents inside our editor is convenient.
- XLS - somehow read through Excel
2) Support for editing documents containing:
- Text
- Tables (without nesting, when reading some formats, nested tables are simply embedded in a cell of a more external table, they are not supported in RTF)
- Drawings and formulas (to edit the formulas, you must install the MathType program, you can have a free trial version, it will work, but the set of formula elements will be limited)
3) Printing documents (with the ability to preview)
4) Document Comparison (File Menu - Compare Files) is a cool thing. can compare documents with spreadsheets, this is not Tortoise Diff! ;)
5) Check Russian and English spelling, including user dictionaries
6) Search and replace
7) Background autosave documents
8) highlighting of words and substrings in words in an arbitrary user directory and quick transitions to such highlighted words
9) Insert templates of standard phrases from the toolbar (the phrases themselves are customized by the user)
10) Record and play key macros (keystroke sequences on the keyboard, not Visual Basic macros :))
11) Several standard treatments, just as an example of what can be and how it works

What else can be interesting in the editor, you ask? Yes, a lot of things. For example, when deleting huge (several hundred megabytes) pieces of large documents, the editor can warn you that the free RAM is running out (due to the formation of the Undo list for Ctrl + Z) and will go into the memory saving mode, which will limit the number of Undo actions the list. Agree, an original and interesting opportunity, and to some it saved some nerve cells when editing such large documents on a machine with 1-2 GB of RAM. And this is just one example. Under the hood of the editor there are quite a lot of interesting things that could be re-launched into “production”, if there were interested users. For example, we have a full-fledged pseudographic table editor (typed in monospaced font using pseudographic characters), which allows working with such tables as with conventional dynamic Word tables, i.e. change the widths of rows and columns, set the text alignment inside the cells, etc., while the table remains pseudographic. At the time of DOS, the word and deed text editor (Word & Deed, wd.exe) was able to do that. I disabled this functionality because considered that in 2017, users who have documents with pseudographic tables, probably no longer exist or their percentage is very small. But if it is interesting and demanded by someone, you can turn it back on.

The rest you can see for yourself in the demo version.

Editor Restrictions

There may be a large sheet, because it depends on what to compare and what is considered a restriction, since This whole project is one big limitation, but this is also good in certain cases.

1) Only 32-bit version.
2) Not cross-platform, works only under Windows.
3) Few format elements supported.
4) The current version does not support the OOXML format (the new Office XML format, starting in 2010).
5) There is no WYSIWYG mode with pagination displayed (this breakdown can only be seen in the print preview).
6) There is no task fonts. Totally. But it is not difficult to add, because Internally, the editor supports working with different fonts and displaying them on the screen.
7) I repent that the editor icon was taken from a set of Linux icons from designer Everaldo Coelho . If we redraw something later, there was simply no time for it.
8) The editor is written in Delphi :-) At the moment it is going under the latest versions of Delphi (specifically in 10.1 Berlin, but you can also in Tokyo - no problems). And what else was there to start writing such a Rich-GUI Windows Application in the early 2000s. It's even a little fun, because The principle of security through obscurity, since there are few Delphi developers, i.e. for example, even if the source is leaked, it will not be so easy to use them (for example, to create a competing product or even for some unfriendly purposes) :-)
9) The editor alone cannot replace the entire OpenOffice and MS Office, because does not contain, for example, functions for working with tables (like Excel) or creating presentations (like PowerPoint).
10) There is no documentation (user manual)

You can download the full-featured (if this term is generally applicable in the current situation) demo version of the editor (0.1) from GitHub as an archive, which can be unpacked to any place. When you first start the editor with administrator rights, installs the protection service, do not worry, this is not a virus or a Trojan, honestly, honestly. The service is needed to determine the parameters of the computer hardware, on which the calculation of the hash for the registration key is based. Without this service, when launching an application under UAC (with the privileges of a regular user, that is, not an administrator), it is impossible to reach the necessary hardware parameters, there are not enough rights.

This version of the editor works without registration (without a key) until the end of 2017. I limited it purely for reasons of “to be”, and not because I think that at the end of 2017 someone else will use this low-functional first alpha version.

Future

Future plans:

1) RTF (selection of arbitrary text fonts as in Word)
2) interface multilingualism (and ortho checks?)
3) Normal support lists (multi-level, etc.)
4) Some support for review mode
5) Setting paragraph indents and changing column widths on a ruler
6) WYSIWYG in edit mode (paginated)
7) Own built-in formula editor, not tied to MathType
8) From the editor, you can make the library automate the processing of documents from other applications using the Com Automation mechanism, just like Word, only much faster, simpler and more convenient. This has already been implemented, but the API is still very meager, I will not give it, so as not to embarrass anyone.

What other features would be worth adding to the product first? Please write in the comments!

Why all this? I sincerely hope that the project of such an editor can be used very well. Those. we are now looking for our new users to implement what they need for them. No, we do not plan to engage in custom development (to cut something for one specific customer, if it is of course not Beeline or Sberbank level :-) Ie we want to make a new finished product that will be useful to a large circle of users thanks to convenient settings and specialized functions that are absent in ordinary text editors.

PS Since the functions were sawed out in a hurry (it took about 10 days, 2-3 hours a day), something could break somewhere, which we could not notice. You can create Issues on GitHub.

→ Temporary home project: GitHub

UPD! Laid out version 0.1.1 , which does not install and does not use the service of protection / registration.

You can delete an already installed service on the command line with the command
sc delete TexEdProtectService
after which you need to remove the TexEdProtectService.exe file from the system32 directory of the Windows installation directory.

And finally, a small survey: