How Opera “repairs” websites

Perhaps most often, Opera browser developers are blamed for their integrity when it comes to adhering to web standards. Oddly enough. The main argument at the same time sounds like this: “Well, here others can adapt to the imperfection of the Internet, how are you better?”. In fact - almost nothing. True, behind this “almost” lies such a simple thing as the desire to make the Internet world a better place. The usual everyday logic suggests that going on about the unfair web developers to change the situation is unlikely to succeed, but too principled approach does not help the matter. Only one way out - look for a middle ground. And I must say that the Norwegian programmers were probably the closest to this compromise solution. Moreover, their method in many articles goes head and shoulders ahead of all other market participants. How can this be? We read the translation of the article by Hollvord Steen.


How Opera “repairs” websites
')
It is not difficult to guess that the Internet can simply get out of control if each web site requires an individual display mode. Imagine that you are driving a car, while you need to tune something in the engine or change the wheels every time you turn a corner or drive onto a new street. The current state of browser development is somewhat similar to this situation.

Virtually every modern browser today has built-in features that allow it to bypass all the bumps and potholes in the web standards left by website developers:

- IE8 displays websites from the list of incompatible in IE7 mode
- Google Chrome seems to be another browser for Hotmail.
- Safari contains some secret undocumented hacks, specific to some sites
- Firefox probably has only an extension to switch the User Agent , which is very popular with a certain audience.
- Opera has a built-in browser identification switch and corrective browser.js

Process each site separately? It sounds crazy. In fact, this is simply impossible. For reference: there are billions of sites, and they all differ from each other. Moreover, they are constantly changing , because every day millions of lines of code are added to them or edited. Who can handle it? In fact, this problem should be solved by standards, isn't it?

Standards

I commend the standardization efforts, the active work of the evangelists of The Web Standards Project and everyone else who promotes web standards. It is possible that you will be able to change the future - more and more sites are being validated and get rid of the "non-standard" legacy of the past. The power is with you!

It should be noted that Opera also has several teams that are trying to change the world for the better, engaging in learning or sharing their experiences . Our Open the Web department constantly contacts website owners with tips on cross-browser compatibility and standards compliance.

But the browser represents the interests of the user. Therefore, we must do everything possible to go through all the intricacies of non-standard code of web sites, our browser could provide the user with the most correct result . No matter how sad it is to realize, but in spite of the fact that we are not guilty of incorrect websites, this is often our headache. And we have to solve it now, and not in 2022 .

What we actually do.

Observing how other browser vendors with varying success perform the same task, I think the time has come for us to share our experience in the difficult task of fixing websites — at least to show all the strengths and weaknesses of our approach to the problem.

Strengths

Experience

Compared to other browsers, Opera has the most experience in adding patches that are specific to each website, because we started doing this in the days of Opera 8.01 . I also take the liberty to say that our solution is the most advanced. A set of custom JavaScript custom commands create a simple but powerful API for modifying and correcting websites. With the methods of obtaining and assigning variables, as well as using the DOM interface, we can change the behavior of Opera in accordance with most of the standard operations performed by the script. Sometimes we add the missing style or even delete some objects and rules, which lead to incorrect display of pages. As a result, if a website checks the browser version, we can often simply find and fix the line of code that does it! Some illustrative examples:

“We can add one missing class name when AOL Japan incorrectly detects the browser and sends the wrong styles, and also fix the error in AOL webmail when they use the erroneous spelling of dojoType instead of dojotype.
- We can take advantage of the elegant call postMessage () when E * Trade is trying to violate the security policy in terms of navigating the frame of another web site while it is loading encoded content.
- We can also redefine what browser we are introducing to the website and which page display mode (adapted to the website or conforming to standards) should be used for this website through loadable settings.

Bug fixes

With the ability to change websites and Opera, we can sometimes even fix our own bugs.

- When the menu on weather.com ceases to be displayed due to improper handling of mouse events, we can fix it.
- When search results on Google Codesearch are displayed in white font on a white background, we can apply a different style.

Just think: Opera is a browser that can, without interfering with the user, correct their mistakes. It looks like a time machine: we can fix our old flaws even in those versions that were already loaded by the user before these errors were detected.

In the long run, this is a very important and subtle property of browser.js: it allows web developers to reduce the number of Opera-specific code corrections. The truth is, it should be noted that both the IE8 update problem and our own experience suggest that working on our old bugs is one of our biggest problems. We did the wrong things, and the websites also became wrong in response. We fixed our browser, but the world wide web never forgets our old mistakes .

What is the difference between the mechanism using browser.js? When we release a new version of the browser with bug fixes, the corresponding pieces of browser.js code are imperceptibly marked as obsolete. What worked with browser.js now starts to work by itself due to browser fixes, making the Opera upgrade process smoother for users and web developers, and also much simpler for us. Sometimes I even seem to feel the huge envy of the IE8 development team across the Atlantic.

Another interesting point is that we can thus add new standards to the browser with minimal problems. For example, if the <input required> tag in the HTML5 specification breaks the Barnes & Noble entry process because the username field doesn’t really need to be filled in, when you click the “create account” button, we can first apply our corrections for registration to be successful, and only then meet with the specification editor and discuss with him over a cup of coffee how we can make the new technology more compatible with the existing Internet.

Transparency

From the very beginning, one of the main problems was that our corrections will allow web developers not to search and correct errors in their code. Moreover, in this way we would even contribute to the appearance of errors, continuing to keep adjustments while the website is already fixed.

To ensure maximum transparency, all corrections are documented , and we also try to keep the file as readable and well commented as possible (of course, not to the detriment of the total size of the downloaded data). In addition, each adjustment made is displayed in the Opera Error Console, telling the user what is happening.

Coverage

Adjusting websites is a cross platform task. But this term too poorly describes the actual state of affairs. Do you know that we are adjusting websites for the Nintendo Wii ? And for DSi ? For Opera on Windows Mobile, Symbian, BREW and Archos portable media players? Besides the jokes, we even apply the adjustment for Opera Mini !

- When editing emails with complicated formatting is required to create email messages on Hotmail and Yahoo mail , we can simulate its support even on devices with very limited hardware capabilities for which this function is impossible in principle
- If the processing of fancy animations on Facebook takes too much time on the Opera Mini server - and sending content to the client is possible only when everything is done - we can simply cut it down and at least complete the most important Facebook functions requested by the end user

It looks like an ambitious task, perhaps even more ambitious than it seems from the outside. Devices and platforms have very different requirements and are replete with specific problems. For example:

“On the Wii game console, we have to limit the minimum font size in order to maintain readability — but this often leads to unpredictable changes in the width, layout, and positioning of web page elements, resulting in a result that is simply not possible on a desktop browser. Therefore, we are adjusting several websites with elements that are too hard to maintain the ability to use slightly larger fonts.
- Using the on-screen keyboard on the Wii and a variety of mobile phones means changing the size of the browser window every time you need to enter text in the form field - the keyboard also requires some space on the display. But, for example, the British Airways website updates the checkout page with each window size change , which makes buying tickets using a virtual keyboard a nontrivial task ...
- ... or when eBay in France tries to count keystrokes to make sure that you are not a rogue and insert your email address in the "Re-enter E-mail" registration form - this also will not work with on-screen keyboards. Most of them simply do not support keyboard events. Come on eBay - I really did it. I'm not kidding - REAL! On a device for which the cut-paste operation is in principle IMPOSSIBLE! ..

When dealing with such a range of problems, the true and magical potential of browser.js becomes even more obvious.

Infrastructure

In order to support all this farming, we have several small servers that send exactly the file that is intended for it at the request of the client. We have a database of adjustments and a set of metadata containing records about which platforms and devices these adjustments can apply. But did you know that we also have a special “spyware” script that constantly checks websites — have the errors we corrected there been corrected? If such a miracle still happens, it would be nice to know about it. I love the system sound that accompanies each deletion of the next “outdated” adjustment to the Trash.

Weak sides

Time

So many opportunities, so little time ... Analyzing websites and making adjustments takes a lot of time. We don’t even come close to the impressive list of IE8-incompatible websites, which contains more than 2,400 entries (in fact, this list’s research reminds Deja Vu: there are so many familiar names, so many websites that need to be corrected or contact for one reason or another).

However, our list is noticeably shorter - fortunately! Technical analysis with the level of detail that is required for our website adjustment system can be very time consuming, so I hope that we can keep our list still much smaller than IE, and as a result, we can better focus focus on the quality of the adjustments. However, we are ready for a wider coverage: in our database there are more than 600 entries and there is still plenty of space - we are waiting for your messages about incorrectly displayed web sites.

Performance

I just said that there is a place for new adjustments. But what scale can be achieved before it significantly affects the performance of the browser?

Obviously, we can maintain the growth of the database for quite a long time. Some of our recent experiments have shown that we can handle up to 10,000 different websites, slowing Opera down by only a fraction of a percent. I also sincerely believe that by the time our database of records has grown to 10,000, you will already buy a more productive computer ...

Security

Is it safe? Can a virus or a “hostile” server fake your browser.js and thereby instantly gain control of your use of Opera?

The short answer is no.

A more detailed answer is that it would be possible if we had not taken appropriate protective measures, but in fact we are engaged in them very actively. Each file contains a digital signature and Opera checks whether the file is authentic and has not been modified. Such a signature is very, very difficult to falsify. Therefore, as soon as a virus or a “hostile” server changes the file, Opera will immediately stop using it and download a more recent version from our server.

findings

One of the things that struck me most about browser.js is how quickly the global network is changing. Paying such close attention to certain websites, we see that in general the Internet is improving , and it happens much faster than I could have expected. And in spite of, perhaps, your fears, we often see that websites improve and become more loyal to Opera browsers after we release corrections to them!

This means that site adjustment works . This works because it improves compatibility, thus giving users a real choice of browser and — however fantastic that may sound — as a direct impact, it gives web developers confidence that their websites are compatible with Opera and with many other browsers.

Welcome to the future - created from and with the help of standards, open technologies and website adjustments .

about the author
Hallvord RM Steen works with Opera’s “nuclear” team to test quality, test and web compatibility. He is the developer of the browser.js file and can break all sites around the world with a little typo.

Thanks:
This translation was made possible thanks to melnichuck and arestov