We turn html into native components

Good day! We, the mobile developers of surfingbird, decided to try to write a small series of articles about the difficulties we face in the process of developing mobile applications (android, ios), and how we solve them. We decided to dedicate the first post to the webview problem. Immediately, I’ll make a reservation that we have solved this problem somewhat cardinally ... In order to make it clearer, we will have to tell a few words about the actual thing we are doing. We aggregate content from various sources (parsim original articles), select a significant part (content) and, based on user ratings and any complex algorithms, we recommend them to the end user and, of course, simply display them in a more convenient way.

In mobile applications, we strive not only to clear pages from layout elements and annoying pop-up windows, but also to optimize content for consumption on mobile devices.

But when using webview to display content, we are faced with a number of difficulties. This component is hard to customize and quite heavy and even, I would say, buggy. The day came when we realized that we no longer wanted to see the webview at all. But getting rid of it, given that the content is given in html, turned out to be not so simple. Therefore, we decided to turn html into native components.
')


I'll try to briefly describe the principle before moving on to code examples.

1. We clean html from styles and javascripts
2. As a reference point we use links to images and iframe.
3. All that is before and between the links to the images is the text that we render using the textview
4. Directly images - render using imageview
5. For Iframe, we analyze the content and video render as clickable pictures on the video, and others render as links or, in extreme cases, insert them into the webview container (for example, links to audio from soundcloud)
6. The resulting array of components is placed in the listview and adapter (in fact, already in recyclerView, but at the time of this writing it was a listview)

First of all, you need to clear the html from all junk in the form of javascript and css. For these purposes, we used the HtmlCleaner library. At the same time create an array of all the images that are found in the content (we will need it later):

final ArrayList<Link> links = new ArrayList<Link>(); HtmlCleaner mHtmlCleaner = new HtmlCleaner(); CleanerTransformations transformations = new CleanerTransformations(); TagTransformation tt = new TagTransformation("img", "imgs", true); transformations.addTransformation(tt); mHtmlCleaner.setTransformations(transformations); //clean html = mHtmlCleaner.getInnerHtml(mHtmlCleaner.clean(parsed_content)); TagNode root = mHtmlCleaner.clean(html); root.traverse(new TagNodeVisitor() { @Override public boolean visit(TagNode tagNode, HtmlNode htmlNode) { if (htmlNode instanceof TagNode) { TagNode tag = (TagNode) htmlNode; String tagName = tag.getName(); if ("iframe".equals(tagName)) { if (tag.getAttributeByName("src") != null) { Link link = parseTag(tag, "iframe"); if (link != null) { links.add(link); } } } if ("imgs".equals(tagName)) { String src = tag.getAttributeByName("src"); //ico if (src != null && !src.endsWith("/") && !src.toLowerCase().endsWith("ico")) { Link link = parseTag(tag, "img"); if (link != null) { links.add(link); } } } } return true; } }); 

Here we replace the img tags with imgs ^ _ ^, firstly, so that the textview does not have the temptation to render the images, secondly, to then find all the links to the images and replace them with imageview.

Since we decided to display the pictures natively, it would not be bad at the same time to enlarge them, so that medium pictures, for example, more than 1/3 of the screen would become full screen of the smartphone, small pictures would become larger, and very small ones could be completely neglected (as The rule is the icons of links on social networks):

 public Link parseTag(TagNode tag,String type) { final String src = tag.getAttributeByName("src"); final String width = tag.getAttributeByName("width"); final String height = tag.getAttributeByName("height"); int iWidth=0, iHeight=0; try { iWidth = Integer.parseInt(width.split("\\.")[0]); iHeight = Integer.parseInt(height.split("\\.")[0]); } catch (Exception e) {} //   1/3  -   if (iWidth>((displayWidth*1)/3) && iHeight>0) { iHeight = (displayWidth * iHeight)/iWidth; iWidth = displayWidth; } //   if (iWidth>45 && iHeight>45) { int scaleFactor = 1; if (iWidth<displayWidth/3) { //  2   scaleFactor = 2; } if (iHeight>=4096 || iWidth>=4096 || src.endsWith("gif")) { type = "iframe"; } return new Link(type, src, iWidth*scaleFactor, iHeight*scaleFactor,""); } return null; } 

Actually, half the work has already been done. Now it remains to go through the array of links to the images, find the content before the image and paste it into the textview, then insert the picture.
To do this, we created an ArrayList into which we will actually place the content itself, indicating its type (text, picture, iframe).

Some pseudocode:

  private ArrayList<Link> data = new ArrayList<Link>();; for(int i=0;i<links.size();i++) { final Link link = links.get(i); if (link.type.equals("txt")) continue; int pos = html.indexOf(link.src); String abzats = ""; if (pos>0) { abzats = html.substring(0, pos); int closeTag = html.indexOf(">",pos)+1; if (closeTag>0) { html = html.substring(closeTag); } if (!TextUtils.equals("", abzats)) { data.add(new Link("txt","",0,0,abzats)); } } //add text if (link.type.equals("img")) { //add image data.add(link); } //add iframe if (link.type.equals("iframe")) { data.add(link); } } data.add(new Link("txt","",0,0,html)); 

At this place we have a magnificent array, with content broken into types. All that's left is to render it. And for the rendering of arrays, it is difficult to find something more beautiful than the usual listview + adapter:
This is what the getView code looks like in the adapter:

 if (link.type.equals("txt")) { // return getTextView(activity, link.txt); } if (link.type.equals("img")) { //  } ... //, textview public TextView getTextView(Context context,String txt){ TextView textView = new TextView(activity); textView.setMovementMethod(LinkMovementMethod.getInstance()); textView.setText(Html.fromHtml(txt)); textView.setTextSize(TypedValue.COMPLEX_UNIT_SP,fontSize); textView.setPadding(UtilsScreen.dpToPx(8),0,UtilsScreen.dpToPx(8),0); textView.setAutoLinkMask(Linkify.ALL); textView.setLineSpacing(0, 1.4f); ColorStateList cl = null; try { XmlResourceParser xpp = context.getResources().getXml(R.xml.textview_link_color_selector); cl = ColorStateList.createFromXml(context.getResources(), xpp); textView.setLinkTextColor(cl); } catch (Exception e) { textView.setLinkTextColor(Color.parseColor("#6fb304")); } return textView; } 

So, the text is rendered as html using a textview, the pictures are converted into regular pictures, but optimized for the resolution of the device. Only the pain with the iframe remains. We analyze its content, and if this is a link to youtube, for example, we generate a picture with a placeholder video, on which we open the youtube application. In general, everything is already quite simple:

  String youtubeVideo = ""; if (link.src.contains("lj-toys") && link.src.contains("youtube") && link.src.contains("vid=")) { try { youtubeVideo = link.src.substring(link.src.indexOf("vid=") + 4, link.src.indexOf("&", link.src.indexOf("vid=") + 4)); } catch (Exception e) { e.printStackTrace(); } } //http://www.youtube.com/embed/ZSPyC6Uv9xw if (link.src.contains("youtube") && link.src.contains("embed/")) { try { youtubeVideo = link.src.substring(link.src.indexOf("embed/") + 6); } catch (Exception e) { e.printStackTrace(); } } if (!youtubeVideo.equals("")) { //new RelativeLayout RelativeLayout relativeLayout = new RelativeLayout(activity); RelativeLayout.LayoutParams layoutParams = new RelativeLayout.LayoutParams( RelativeLayout.LayoutParams.WRAP_CONTENT, RelativeLayout.LayoutParams.WRAP_CONTENT); ImageView imageView = new ImageView(activity); imageView.setLayoutParams(layoutParams); relativeLayout.addView(imageView); imageView.setBackgroundColor(Color.parseColor("#f8f8f8")); if (link.width>0 && link.height>0) { aq.id(imageView).width(link.width, false).height(link.height, false); } String youtubeVideoImage = youtubeVideo; if (youtubeVideoImage.contains("?")) { //params youtubeVideoImage = youtubeVideoImage.substring(0, youtubeVideoImage.indexOf("?")); } if (link.width>0) { aq.id(imageView).image("http://img.youtube.com/vi/" + youtubeVideoImage + "/0.jpg", true, false, link.width, 0, null, AQuery.FADE_IN_NETWORK); } else { aq.id(imageView).image("http://img.youtube.com/vi/" + youtubeVideoImage + "/0.jpg"); } ImageView imageViewPlayBtn = new ImageView(activity); relativeLayout.addView(imageViewPlayBtn); RelativeLayout.LayoutParams playBtnParams = new RelativeLayout.LayoutParams( ViewGroup.LayoutParams.WRAP_CONTENT, ViewGroup.LayoutParams.WRAP_CONTENT); playBtnParams.addRule(RelativeLayout.CENTER_IN_PARENT, RelativeLayout.TRUE); imageViewPlayBtn.setLayoutParams(playBtnParams); aq.id(imageViewPlayBtn).image(R.drawable.play_youtube); final String videoId = youtubeVideo; aq.id(relativeLayout).clickable(true).clicked(new View.OnClickListener() { @Override public void onClick(View v) { try { Intent intent = new Intent(Intent.ACTION_VIEW, Uri.parse("vnd.youtube:" + videoId)); intent.putExtra("VIDEO_ID", videoId); activity.startActivity(intent); } catch (Exception e) { activity.startActivity(new Intent(Intent.ACTION_VIEW, Uri.parse("http://www.youtube.com/watch?v=" + videoId))); } } }); return relativeLayout; 

We shot a small video with a demonstration of the application at work, but of course it's better to download the application and try it yourself .

Perhaps this method may seem somewhat radical to someone, but we are pleased with the end result. Pages began to load faster, look native, consistent and easy to read on any device. Plus, a lot of interesting opportunities are opened, native photo preview, font setting, opening video in the native application and of course there are no problems with different versions and often strange webview behavior.