How we made Rich Text Editor with support for co-editing under Android

“Mobilizing” workflows in companies means that more and more collaboration features are being transferred to the phone or tablet. For Wrike , as a cross-platform project management service, it is important that the functionality of the mobile application is completely complete, convenient and does not restrict users to work. And when the task arose to create Rich Text Editor with the support of joint editing of the description of the tasks, we, assessing the capabilities of existing WebView components, decided to go our own way and implemented our own native tool.




First, a little about the history of the product. One of the basic functions of Wrike was initially mail integration. From the very first version of the task it was possible to create and update via e-mail, and then work on them together with other employees. The body of the letter turned into a description of the problem, and all further discussion went in the comments to it.
')
Since you can use HTML formatting in the mail, in earlier versions of the product we used CKEditor to further work with the description of the task. But in a collaborative environment, this is very inconvenient - it is necessary to block all or part of the document so that the prepared description of the task does not get erased by someone else. As a result, we decided to delve into the practice of Operation Transformation (OT) and make a tool for real collaboration. In this article I will not consider in detail the theory and implementation of OT for rich text documents, there are already enough materials about this. I will consider only the difficulties faced by our team when developing a mobile application.

Joint editing on a smartphone - but why?

Perhaps there is no need, unless, of course, this is a key function of your product. In addition to the common goal to provide maximum basic functionality on all platforms, there were a number of more specific reasons why we had to think about it:

1. An OT implementation requires you to store the document in a specific format that supports co-editing. In the case of plain text, there is no special format — it may just be a string. But in the case of Rich Text (formatted text), the storage format becomes more complicated.
2. We need a way to save changes made by the mobile client without breaking the document and creating a conflict with the changes that other users could make at the same time. These are tasks that are just solved by OT algorithms.
3. Since we need to transfer the OT algorithm to the mobile platform in order to fulfill the conditions of clause 2, then full-fledged collaborative editing does not require much additional effort.

So, we have a rich text description of the problem as the basic functionality, the need to support a specific document format and synchronization protocol, so we will take up the search for a solution.

Implementation options

With the implementation of the component to work together, the experience was already there, but with how to transfer it to Android, you had to figure it out. Much depended on the requirements for the editor and there were, by and large, two of them:

1. Support for basic formatting, lists, inserting pictures and tables,
2. An API that allows you to make and track changes both in the text itself and in its formatting.

Method 1: Use an Existing Component from a Web Product

Indeed, we could use a component that we already have and wrap it in a webview. One of the advantages is simplicity of integration, since virtually all the code of the editor is in scripts, and the Android / iOS developer need only implement the WebView wrapper.

It quickly became clear that the existing component from the main application, working with the ContentEditable document, is very unstable, depending on the version of the OS and the vendor. Exotic bugs in some places went off scale, but mostly they surfaced around the functions of selecting and entering text, as well as the disappearing focus and keyboard.

To get around the problems of ContentEditable, we tried to use CodeMirror as a front-end for the editor, while it works much better and more stable on Android, since it processes all keyboard events and rendering itself. There were, of course, disadvantages, but as a quick workaround, it worked very well until there was a well-known change in handling keystroke events in IME - this problem is discussed in some detail here . If in a nutshell - when using LatinIME, it does not send an event for KEYCODE_DEL.

What does this mean for the user? When you click on Delete, nothing happens, that is, the editor works correctly, you can enter text, apply formatting ... only the text cannot be deleted, no matter how absurd it sounds. The only solution to this problem, among other things, included the following code:

@Override public InputConnection onCreateInputConnection(EditorInfo outAttrs) { BaseInputConnection baseInputConnection = new BaseInputConnection(this, false) { @Override public boolean sendKeyEvent(KeyEvent event) { if (needsKeyboardFix() && event.getAction() == KeyEvent.ACTION_MULTIPLE && event.getKeyCode() == KeyEvent.KEYCODE_UNKNOWN) { passUnicodeCharToEditor(event); return true; } return super.sendKeyEvent(event); } @Override public boolean deleteSurroundingText(int beforeLength, int afterLength) { if ((Build.VERSION.SDK_INT >= Build.VERSION_CODES.ICE_CREAM_SANDWICH) && (Build.VERSION.SDK_INT < Build.VERSION_CODES.KITKAT) && (beforeLength == 1 && afterLength == 0)) { // Send Backspace key down and up events return super.sendKeyEvent(new KeyEvent(KeyEvent.ACTION_DOWN, KeyEvent.KEYCODE_DEL)) && super.sendKeyEvent(new KeyEvent(KeyEvent.ACTION_UP, KeyEvent.KEYCODE_DEL)); } else { return super.deleteSurroundingText(beforeLength, afterLength); } } }; outAttrs.inputType = InputType.TYPE_NULL; return baseInputConnection; } 

InputType.TYPE_NULL at the same time translated the IME into a “simplified” view, signaling that the InputConnection works in a limited mode, which means no copy / paste, autocorrect / autocomplete, or text input using gestures, but it also allows you to handle all keyboard events .

As a result, in the latest implementation of the editor, which used the web interface, there were the following disadvantages:

• slow download speed;
• lack of access to advanced features of IME (copy / paste, autocomplete / autocorrect, gesture input);
• in some cases, unstable work, due to the different implementation of WebView on different versions of the API and modifications of this component by some vendors;
• Usually, WebView does not last long in memory, especially on devices with a small amount of memory, and if the application is minimized and after some time restarted, in most cases WebView will have to be initialized again;
• numerous crutches in the code, the number of which only increased over time.

Realizing that supporting such an implementation of the editor is not easy, and given the described shortcomings and limitations, it was decided to develop a native component that would give the opportunity to work with formatted text.

Method 2: native implementation

For the native implementation it is necessary to solve two problems:

1. UI editor, that is, the display of text with regard to formatting and editing.
2. Work with the document format, change tracking, and data exchange with the server.

In order to solve the first problem, you do not need to reinvent the wheel - Android provides the necessary tools, namely the EditText component and the Spannable interface, which describes the labeling of the text.

The second problem is solved by transferring OT algorithms from JavaScript to Java, and the process here is quite transparent.

Rich Text Display in EditText

Android has a wonderful Spannable interface that allows you to set text markup. The process of creating markup is quite simple - you need to use a special class SpannableStringBuilder, which allows you to set / change the text, and set styles for specified sections of text through the method

 setSpan(Object what, int start, int end, int flags). 

The first parameter just sets the style. It must be an instance of a class that implements one or more interfaces from the android.text.style package: CharacterStyle, UpdateAppearance, UpdateLayout, ParagraphStyle, etc. The set of default styles is quite wide - from changing the character format (StyleSpan, UnderlineSpan), setting the text size (RelativeSizeSpan) and changing its position (AlignmentSpan) to supporting images (ImageSpan) and clickable text (ClickableSpan).

The last parameter sets the flags, the role of which will be described below. For example, this is how you can change the color of the entire text:

 SpannableStringBuilder ssb = new SpannableStringBuilder(text); ssb.setSpan(new ForegroundColorSpan(Color.BLUE), 0, text.length(), Spannable.SPAN_EXCLUSIVE_EXCLUSIVE); textView.setText(ssb, TextView.BufferType.SPANNABLE); 

So, at the entrance there is text in a certain format, and at the exit you need to get its representation in the form of a Spannable object and pass it to EditText. In our case, the document comes from the server in a special format in the form of an attributed string — you need to parse this string using our OT library and apply attributes to the specified text areas. Depending on the style, you need to set the correct flag so that the text marking meets the user's expectations.

If you mark a style with the SPAN_EXCLUSIVE_INCLUSIVE flag, it will be applied to the text entered at the end of the interval, but will not be applied at the beginning. For example, there is an interval [10, 20] for which the style UnderlineSpan + SPAN_EXCLUSIVE_INCLUSIVE is set. In this case, when you enter text at position 9, the UnderlineSpan style will not be applied to it, but if you start entering text at position 20, the interval that covers the style will expand and become [10, 21]. Naturally, this is useful for inline formatting (bold / italic / underline, etc.).

When using the SPAN_EXCLUSIVE_EXCLUSIVE flag, the style interval is limited at both ends. This is suitable, for example, for links - if you start to insert text immediately after the link, the link style should not be applied to it.

Using the SPAN_EXLUSIVE_INCLUSIVE and SPAN_EXCLUSIVE_EXCLUSIVE flags, you can control the formatting behavior when entering text depending on the user's expectations. For example, if you have turned on the Bold formatting mode, then the entered text should remain bold. And if you made a link, then appending text at the end should not extend the boundaries of the link.

BulletSpan can be used to display the list items, but it is suitable only for unnumbered lists. If numbering is necessary, then you can write your own class that implements the LeadingMarginSpan and UpdateAppearance interfaces, drawing the list indicator at your discretion in the drawLeadingMargin method.

Custom Style Processing

It is clear that the editor should give the user the ability to apply formatting, this includes:

1. Adding a new style to the selected text.
2. Insert a new style at the cursor position,
3. Apply current style when editing.

First of all, you need somewhere to place the buttons for styles supported by the editor. Putting them in the Activity toolbar was not practical until the release of Android Marshmallow. By default, the same toolbar is used for the context menu when text is selected, and thus it is impossible to select a style for the selected text. Therefore, you can put them on the toolbar at the bottom of the screen. When you click on the style button, you must decide on the current state of the editor and either apply the style to the selected text, or remember this style as temporary in the cursor position.

 private void onApplyInlineAttributeToSelection(int selectionStart, int selectionEnd, TextAttribute attribute) { int selectionStart = mEditText.getSelectionStart(); int selectionEnd = mEditText.getSelectionEnd(); if (!mEditText.hasSelection()) { // if there's no selection, insert/delete empty span for the appropriate attribute, // but only in case the cursor is present if (selectionStart == selectionEnd && selectionStart != -1) { if (mTempAttributes == null || mTempAttributes.getPos() != selectionStart) { mTempAttributes = new TempAttributes(selectionStart); } Set<Object> attributeSpans = getAttributeSpans(selectionStart, selectionEnd, attribute); if (attributeSpans.size() > 0) { attribute.nullify(); } mTempAttributes.addAttribute(attribute); } return; } if (attribute == null) { return; } boolean changed = applyInlineAttributeToSelection(selectionStart, selectionEnd, attribute); // if nothing changed, then there's no need to build any changesets and send updates to server if (!changed) { return; } // ... } 

mTempAttributes is an instance of the TempAttributes class. It defines a set of attributes in a given position, selected by the user. This variable is reset either after use or when the cursor position changes.

 static class TempAttributes { private final int mPos; private final Map<AttributeName, TextAttribute> mAttributeMap = new HashMap<>(); public TempAttributes(int pos) { mPos = pos; } public int getPos() { return mPos; } public Collection<TextAttribute> getAttributes() { return mAttributeMap.values(); } public void addAttribute(TextAttribute attribute) { AttributeName name = attribute.getAttributeName(); TextAttribute oldAttribute = mAttributeMap.get(name); if (oldAttribute != null && !oldAttribute.isNull()) { attribute.nullify(); } mAttributeMap.put(name, attribute); } } 

If the user presses a button corresponding to a certain style on the toolbar, but no text is selected, in this case, you must save this style as “temporary” at the current cursor position and apply it when entering text at this position. More on this below.

When the text has been selected, it is necessary to determine whether this style already exists in the selected interval or not. If not or is partially, then it is necessary to combine all existing spans and cover the interval with this style completely. If there is, then remove the corresponding span from the interval, breaking it if necessary.

Example 1
There is a text: Quick brown fox .
It has 2 span-a: bold [0,4] and bold [12,14]. If the user selects all the text and applies the bold style to it, then in the end he should cover the entire interval. To do this, you can either delete both spans and add a new bold [0, 14], or delete the second one and extend the first one to the end of the interval.

Example 2
There is a text: Quick brown fox .
It has one span: bold [0, 14]. If the user selects the text [4, 12] and selects the bold style in the toolbar, then the style must be removed from the interval, since it is fully present in the selection. To do this, split the interval into two parts: shorten the entire interval [0, 14] before the start of the selection ([0, 4]) and add a new interval from the end of the selection to the end of the text ([4, 12]).

Tracking document changes

In order to correctly track user changes and feed them to the OT algorithm, the editor must be able to track them. The TextWatcher interface is used for this - every time some changes occur in the EditText, the beforeTextChanged, onTextChanged and afterTextChanged methods of this interface are sequentially called, allowing you to determine where has changed.

 private boolean mIgnoreNextTextChange = false; private int mCurrentPos; private String mOldStr = null; private String mNewStr = null; // ... public void ignoreNextTextChange(boolean ignore) { mIgnoreNextTextChange = ignore; } public void beforeTextChanged(CharSequence s, int start, int count, int after){ if (mIgnoreNextTextChange) { return; } mOldStr = null; mCurrentPos = start; if (s.length() > 0 && count > 0) { mOldStr = s.subSequence(start, start + count).toString(); } } public void onTextChanged(CharSequence s, int start, int before, int count) { if (mIgnoreNextTextChange) { return; } mNewStr = null; if (s.length() > 0 && count > 0) { mNewStr = s.subSequence(start, start + count).toString(); } } public void afterTextChanged(Editable s) { // ... } 

It is important to note that during the initial installation of text into the editor via setText (CharSequence), TextWatcher will also receive a notification about this, so the programmatic installation of the text turns into:

 mEditTextWatcher.ignoreNextTextChange(true); mEditText.setText(builder); mEditTextWatcher.ignoreNextTextChange(false); 

The mOldStr and mNewStr variables store the old line and the new line, respectively, mCurrentPos indicates the position from which the changes occurred. For example, if a user added the character “a” at position 10, then

 mOldStr = null; mNewStr = "a"; mCurrentPos = 10; 

However, there is a small nuance - when inserting text due to autocorrection, these values ​​may include the beginning of a word. For example, if the text starts with the word “Text” and the user replaces the third character with “s”, then the IME can report this change as:

 mOldStr = "Tex"; mNewStr = "Tes"; mCurrentPos = 0; 

In this case, you need to cut off the same sequence of characters from the beginning of the line.

In the end, using TextWatcher, you can unambiguously determine what exactly happened - whether the text was replaced, deleted or added. If the user adds text to the position or replaces part of the existing text with text from the buffer, you must apply those attributes that are at the cursor position to the added text. To do this, find all Spannable objects at the cursor position, while not forgetting to exclude those that have become empty (s.getSpanStart (span) == s.getSpanEnd (span)), deleting Spannable objects themselves and filtering only by inline attributes (bold, italic, etc.). Additionally, the attributes that correspond to the styles selected by the user on the toolbar (mTempAttributes) are added.

 public void afterTextChanged(Editable s) { // ... Object[] spans = s.getSpans(mCurrentPos, mCurrentPos, Object.class); Map<Object, TextAttribute> spanAttrMap = new LinkedHashMap<>(); for (Object span : spans) { TextAttribute attr = AttributeManager.attributeForSpan(span); if (attr != null) { spanAttrMap.put(span, attr); } } if (!TextUtils.isEmpty(mOldStr)) { Iterator<Map.Entry<Object, TextAttribute>> iterator = spanAttrMap.entrySet().iterator(); while (iterator.hasNext()) { Map.Entry<Object, TextAttribute> entry = iterator.next(); Object span = entry.getKey(); TextAttribute attr = entry.getValue(); // ... if (s.getSpanStart(span) == s.getSpanEnd(span)) { s.removeSpan(span); iterator.remove(); } } } // ... Set<TextAttribute> attributes = new HashSet<>(); if (!TextUtils.isEmpty(mNewStr)) { // determine all inline attributes at current position for (Map.Entry<Object, TextAttribute> entry : spanAttrMap.entrySet()) { TextAttribute attr = entry.getValue(); if (AttributeManager.isInlineAttribute(attr)) { attributes.add(attr); } } } if (mCallbacks != null) { mCallbacks.onTextChanged(mCurrentPos, mOldStr, mNewStr, attributes); } } 

As a result, there is a position in which changes occurred, the old and new texts in this position are known, as well as inline attributes that need to be applied to the new text. After that, you can add additional processing. For example, if a user inserts a line break at the end of the last list item, you can insert a new list item at the current cursor position to continue the list. In the end, a list of changes is compiled from these data and sent to the server.

It is worth noting that when tracking changes in the editor, it is good practice to use wrappers for all default styles. For example, instead of UnderlineSpan use the class CustomUnderlineSpan, which is inherited from UnderlineSpan, but at the same time no methods in it are redefined. Such an approach will allow for the class to unambiguously separate “their” styles from those used by EditText. For example, if AutoCorrect support is enabled, then when editing the word EditText adds the UnderlineSpan style to it, and the word is visually underlined at the time of editing.

About compatibility with different API versions

On versions of APIs prior to Android KitKat, there is a problem with overlaying spannable text when editing. It is solved by disabling TextView hardware acceleration (perhaps there are other ways to fix this — suggestions in the comments are most welcome):

 mEditText.setLayerType(View.LAYER_TYPE_SOFTWARE, null); 

However, in this form, the TextView cannot be placed in the ScrollView, since the entire View will be rendered in memory (“View too large to fit into drawing cache”), so you need to enable scrolling in the TextView itself.

 mEditText.setVerticalScrollBarEnabled(true); mEditText.setScrollBarStyle(View.SCROLLBARS_OUTSIDE_OVERLAY); 

Conclusion

Having troubled with the implementation of the editor on a webview and realizing the deadlock of this approach, we were able to develop a native component that solves the difficult but quite interesting task of co-editing text. This allowed us to improve the usability of the application and increase the productivity of our users. The resulting result can be estimated by downloading our application from Google Play .