Illusions of XML / XSLT Technologies
Sometimes in the computer world there are surges of interest in a particular technology. Bursts are not random, but clearly supported by the manufacturers of these technologies. This is not surprising, because it is difficult to sell the same thing, it is easier to sell something new or old, but the name is different. Nothing is selling as well as a function that is not in the previous model. Why is the consumer so arranged? And the consumer's opinion is trite exploited, desire is simply imposed on him. Large software vendors very often exhaust the market and need a constant change of technology in order to sell updates and simply increase the prices of programs. Well, it's easier to keep away from competitors, assuring that we have the best and latest technologies.
So it happened with XML. After all, XML - is, in general, nothing new. XML is a simplified subset of SGML , which dates back to IBM’s 1960 GML. XML, in fact, simply standardized the format of information exchange and that's it.
But a miracle happened, we received XML and an object appeared for advertising, and manufacturers began to declare at every corner that they already had XML databases and everything was saturated with XML.
But why look far back, here it is AJAX and here it is an object for advertising and sales. Now only the lazy one does not say that he is with AJAX.
')
In general, this is the world. Not all new technologies are bad and you don’t need to be taken to bayonets. But we need to be able to choose the tools for certain decisions and to make these decisions correctly for our business or for solving our business problems.
In my article “Programming as an art,” I already mentioned the enthusiasm of programmers for technological fashion. And as I have already said, I myself am a programmer, and it was common for me and my colleagues to get involved. And in this article I will talk about our errors related to the use of XML / XSLT technologies. Perhaps this experience will be useful to someone and will help him find the right place to use these technologies. Although we can firmly say that this article will cause much criticism in my address.
Our company has developed several software products. But the first products did not find a big response in the market, as they contained a number of technological and marketing errors.
One of these products is Bitrix Info-Portal. One of the most difficult products we created. Designed for the Microsoft platform based on ASP technologies with the MSSQL database, it was designed to create web solutions on the MS platform.
In the first version of this product (in 1999 or 2000, it seems) we created our own template template, in which the instructions were in the correct tags and looked something like #DATA () #. It had simple conditions and cycles: Well, who did not go through this? Yes, everyone who once made the system, which was supposed to be transferred to someone to use and configure, made their own template engine.
After the introduction of the first projects came dissatisfaction with such a template. All the time you need to add features, you need to teach developers to your language, there are no normal tools for debugging and development: Well, in general, nonsense. I am always surprised when I still meet products that use their template engines. It's like picking your nose with your toes, uncomfortable and you do not reach the goal. There were options for using someone else's template engines, but this was a change of sewing for soap, we did not accept this option and went to look for an industrial solution.
Indeed, many books and textbooks were written in which programmers and designers were taught that the best way to create a templating system or to abstract the appearance (presentation) of data is to drive everything into XML, skip it through XSLT, and get HTML at the output. Those. XSLT will be the best template engine for you.
It was argued that projects with XSLT templates should be simpler and more convenient than any other templates and they will be more convenient to manage. Much has been said about safety, portability, and ease of development of such projects.
Everyone took it literally and started making similar products. And of course, we also heard and believed that our future is XML / XSLT technologies.
They accomplished the feat by forcing XSLT templates to work fast enough, they invested a lot of effort, time and money in developing the technology ... The largest catalogs of products contained 70 thousand products.
But customers voted ruble and clarified.
What we got in the end:
1. The illusion of simplicity of XSLT templates.
Projects using XML / XSLT turned out to be very difficult to support clients and our partners. The cost of ownership of such projects is very high and tends to increase as the project grows. There are very few XSLT specialists. Technological pattern can be corrected only by a specialist of sufficiently high qualification. XPATH is also not very convenient to select data. Thus, the illusion of simplicity for clients and convenience in project management was dispelled.
2. The illusion of controllability and flexibility.
XSLT templates for the most part are not enough to write serious business logic in the public part of the site. Why shift serious business logic to a template with XSLT? Yes, it turns out in some applications that the data are the same, but it depends on the user or a number of conditions what he will see in what form. And this is a template and it needs logic. XSLT did not develop as a full-fledged programming language; only simple conditional representations and limited logic can be done on it. It is not possible to use the full potential of modern programming languages and libraries (graphics, presentation, service functions, etc.).
3. The illusion of performance, low-cost placement and scalability.
As developers do not try, the performance of XML / XSLT systems remains very low, despite the best efforts of the industry. Yes, and how to squeeze this performance? First, the data from the SQL database is converted to XML (and this is a large text file due to its structure). Then the XML data is loaded into the XML parser already in the server part, where they take up even more memory for XPATH operation, generating indexes on the XML data at the time of loading, etc. Next, XSLT goes through a huge array of data, receiving again the text that takes up memory.
The real performance solution is only visible in multi-level caching, which is not always possible, or undesirable, or just expensive, both in development and in use.
Sometimes they argue, they say, why do you need a lot of data in XML? Well, to make a site template or just a forum, for example, on XSLT, in XML there should be almost all data: user authorization, statistics data, catalog, product or article branches: Some solution lies in managing and requesting the necessary data in XML directly from Xslt. But still, data has to be extracted significantly more, conditions are needed more, memory and queries have to be done more, well, the template itself becomes more complicated and actually becomes a full-fledged software application that kills the original goal, making the template simple.
Well, as a result, projects on this technology are very difficult to place on conventional hosting, it is almost always dedicated machines. The need for scaling up such projects arises rather quickly and requires considerable financial effort.
4. The illusion of the convenience of data abstraction and appearance.
One of the advantages that everyone is chasing using XML / XSLT technologies is to achieve a high-quality abstraction of appearance. But only after the creation of the first templates, everyone understands that the abstraction has turned out, but no one needs it anymore. The XSLT template is very complex to represent the appearance, especially of a modern AJAX application. Adjusting such a pattern requires a lot of effort. A complete change of design requires a complete rewriting of all templates, which, based on the complexity of creating XSLT, is even more expensive. Thus, it turns out that the price of owning XML / XSLT technology is very high for both developers and their clients.
5. Inability to create libraries and reuse work results
Created templates or an XSLT application cannot be combined into a library in order to simplify work with a typical operation in the next template. Programmers are extremely limited in the transfer of their developments to their colleagues. Identified errors have to be tracked in several templates.
6. Non-linear increase in development complexity during application development.
This is really strange, but if you need to dynamically develop an application, scale it up, the complexity of developing applications with XSLT grows completely disproportionately to the resulting functionality. Applications are hard to debug. All this also leads to the need to involve more and more qualified programmers.
And what is the result of the heroic efforts of developers to master the technology?
High costs of ownership and development, ever-increasing costs of scaling, low demand for products, expensive services, switching to an expensive price group, reducing the number of orders, slowing down the development of products, lack of a working partner network ...
Does this mean that we should abandon XML altogether? Of course not. XML / XSLT is a very beautiful technology (programmer phrase, right?) And will continue to evolve. XML works great for communication between projects (RSS, Yandex.Market, CommerceML, and others). For small templates and fragments, XSLT is quite effectively used today for XML processing.
.NET fully supports XML in all its manifestations. XSLT transformation is also supported: if needed, apply, no need — no. But nobody insists on using XML / XSLT.
There are even very vivid examples when the largest projects use XML / XSLT. As far as I know, Yandex uses this combination for its services. And for me the best result of efficiency is the financial result. I will not argue, but I will assume that the development cost in relation to the development speed for Yandex is quite high, obviously, within the limits of the permissible for this project, but still not as comfortable as we would like. But this is just my guess. I do not spend analogies with Yandex, it is inappropriate. We made a lottery product, and they make internal services for themselves and have the ability to use all the tools for optimizing and configuring the application, they serve it only with their team.
Of course, there is always a temptation to say that the MSXML we have chosen was bad and that there are new and better parsers, more optimal platforms. Maybe. But this is not the case, the issue is in the economics of the project and in the reasons I have indicated above. As a result, we concluded that the use of XML / XSLT turned out to be unreasonably expensive and inexpedient in the circulation product.
What is this article for? Choose your tools deliberately, do not be guided only by fashion and the reassurance of technology developers of its incredible efficiency. If you make a lottery product like us, consider the economy of partners and customers. If the developer needs to know the programming language of the platform (ASP, JAVA, PHP ...) and use XSLT in templates and XPATH to search for data, this alone can make a project unfavorable to use, not to mention a lot of others. reasons.
In general, nobody canceled the head.
So it happened with XML. After all, XML - is, in general, nothing new. XML is a simplified subset of SGML , which dates back to IBM’s 1960 GML. XML, in fact, simply standardized the format of information exchange and that's it.
But a miracle happened, we received XML and an object appeared for advertising, and manufacturers began to declare at every corner that they already had XML databases and everything was saturated with XML.
But why look far back, here it is AJAX and here it is an object for advertising and sales. Now only the lazy one does not say that he is with AJAX.
')
In general, this is the world. Not all new technologies are bad and you don’t need to be taken to bayonets. But we need to be able to choose the tools for certain decisions and to make these decisions correctly for our business or for solving our business problems.
In my article “Programming as an art,” I already mentioned the enthusiasm of programmers for technological fashion. And as I have already said, I myself am a programmer, and it was common for me and my colleagues to get involved. And in this article I will talk about our errors related to the use of XML / XSLT technologies. Perhaps this experience will be useful to someone and will help him find the right place to use these technologies. Although we can firmly say that this article will cause much criticism in my address.
Our company has developed several software products. But the first products did not find a big response in the market, as they contained a number of technological and marketing errors.
One of these products is Bitrix Info-Portal. One of the most difficult products we created. Designed for the Microsoft platform based on ASP technologies with the MSSQL database, it was designed to create web solutions on the MS platform.
In the first version of this product (in 1999 or 2000, it seems) we created our own template template, in which the instructions were in the correct tags and looked something like #DATA () #. It had simple conditions and cycles: Well, who did not go through this? Yes, everyone who once made the system, which was supposed to be transferred to someone to use and configure, made their own template engine.
After the introduction of the first projects came dissatisfaction with such a template. All the time you need to add features, you need to teach developers to your language, there are no normal tools for debugging and development: Well, in general, nonsense. I am always surprised when I still meet products that use their template engines. It's like picking your nose with your toes, uncomfortable and you do not reach the goal. There were options for using someone else's template engines, but this was a change of sewing for soap, we did not accept this option and went to look for an industrial solution.
Indeed, many books and textbooks were written in which programmers and designers were taught that the best way to create a templating system or to abstract the appearance (presentation) of data is to drive everything into XML, skip it through XSLT, and get HTML at the output. Those. XSLT will be the best template engine for you.
It was argued that projects with XSLT templates should be simpler and more convenient than any other templates and they will be more convenient to manage. Much has been said about safety, portability, and ease of development of such projects.
Everyone took it literally and started making similar products. And of course, we also heard and believed that our future is XML / XSLT technologies.
They accomplished the feat by forcing XSLT templates to work fast enough, they invested a lot of effort, time and money in developing the technology ... The largest catalogs of products contained 70 thousand products.
But customers voted ruble and clarified.
What we got in the end:
1. The illusion of simplicity of XSLT templates.
Projects using XML / XSLT turned out to be very difficult to support clients and our partners. The cost of ownership of such projects is very high and tends to increase as the project grows. There are very few XSLT specialists. Technological pattern can be corrected only by a specialist of sufficiently high qualification. XPATH is also not very convenient to select data. Thus, the illusion of simplicity for clients and convenience in project management was dispelled.
2. The illusion of controllability and flexibility.
XSLT templates for the most part are not enough to write serious business logic in the public part of the site. Why shift serious business logic to a template with XSLT? Yes, it turns out in some applications that the data are the same, but it depends on the user or a number of conditions what he will see in what form. And this is a template and it needs logic. XSLT did not develop as a full-fledged programming language; only simple conditional representations and limited logic can be done on it. It is not possible to use the full potential of modern programming languages and libraries (graphics, presentation, service functions, etc.).
3. The illusion of performance, low-cost placement and scalability.
As developers do not try, the performance of XML / XSLT systems remains very low, despite the best efforts of the industry. Yes, and how to squeeze this performance? First, the data from the SQL database is converted to XML (and this is a large text file due to its structure). Then the XML data is loaded into the XML parser already in the server part, where they take up even more memory for XPATH operation, generating indexes on the XML data at the time of loading, etc. Next, XSLT goes through a huge array of data, receiving again the text that takes up memory.
The real performance solution is only visible in multi-level caching, which is not always possible, or undesirable, or just expensive, both in development and in use.
Sometimes they argue, they say, why do you need a lot of data in XML? Well, to make a site template or just a forum, for example, on XSLT, in XML there should be almost all data: user authorization, statistics data, catalog, product or article branches: Some solution lies in managing and requesting the necessary data in XML directly from Xslt. But still, data has to be extracted significantly more, conditions are needed more, memory and queries have to be done more, well, the template itself becomes more complicated and actually becomes a full-fledged software application that kills the original goal, making the template simple.
Well, as a result, projects on this technology are very difficult to place on conventional hosting, it is almost always dedicated machines. The need for scaling up such projects arises rather quickly and requires considerable financial effort.
4. The illusion of the convenience of data abstraction and appearance.
One of the advantages that everyone is chasing using XML / XSLT technologies is to achieve a high-quality abstraction of appearance. But only after the creation of the first templates, everyone understands that the abstraction has turned out, but no one needs it anymore. The XSLT template is very complex to represent the appearance, especially of a modern AJAX application. Adjusting such a pattern requires a lot of effort. A complete change of design requires a complete rewriting of all templates, which, based on the complexity of creating XSLT, is even more expensive. Thus, it turns out that the price of owning XML / XSLT technology is very high for both developers and their clients.
5. Inability to create libraries and reuse work results
Created templates or an XSLT application cannot be combined into a library in order to simplify work with a typical operation in the next template. Programmers are extremely limited in the transfer of their developments to their colleagues. Identified errors have to be tracked in several templates.
6. Non-linear increase in development complexity during application development.
This is really strange, but if you need to dynamically develop an application, scale it up, the complexity of developing applications with XSLT grows completely disproportionately to the resulting functionality. Applications are hard to debug. All this also leads to the need to involve more and more qualified programmers.
And what is the result of the heroic efforts of developers to master the technology?
High costs of ownership and development, ever-increasing costs of scaling, low demand for products, expensive services, switching to an expensive price group, reducing the number of orders, slowing down the development of products, lack of a working partner network ...
Does this mean that we should abandon XML altogether? Of course not. XML / XSLT is a very beautiful technology (programmer phrase, right?) And will continue to evolve. XML works great for communication between projects (RSS, Yandex.Market, CommerceML, and others). For small templates and fragments, XSLT is quite effectively used today for XML processing.
.NET fully supports XML in all its manifestations. XSLT transformation is also supported: if needed, apply, no need — no. But nobody insists on using XML / XSLT.
There are even very vivid examples when the largest projects use XML / XSLT. As far as I know, Yandex uses this combination for its services. And for me the best result of efficiency is the financial result. I will not argue, but I will assume that the development cost in relation to the development speed for Yandex is quite high, obviously, within the limits of the permissible for this project, but still not as comfortable as we would like. But this is just my guess. I do not spend analogies with Yandex, it is inappropriate. We made a lottery product, and they make internal services for themselves and have the ability to use all the tools for optimizing and configuring the application, they serve it only with their team.
Of course, there is always a temptation to say that the MSXML we have chosen was bad and that there are new and better parsers, more optimal platforms. Maybe. But this is not the case, the issue is in the economics of the project and in the reasons I have indicated above. As a result, we concluded that the use of XML / XSLT turned out to be unreasonably expensive and inexpedient in the circulation product.
What is this article for? Choose your tools deliberately, do not be guided only by fashion and the reassurance of technology developers of its incredible efficiency. If you make a lottery product like us, consider the economy of partners and customers. If the developer needs to know the programming language of the platform (ASP, JAVA, PHP ...) and use XSLT in templates and XPATH to search for data, this alone can make a project unfavorable to use, not to mention a lot of others. reasons.
In general, nobody canceled the head.
Source: https://habr.com/ru/post/18569/
All Articles