Future Activities - inter-device sessions
This post is a translation of this article written today .
The post attracted me primarily because the author comes to the idea of a shared data repository, and touches upon the cross-device trend, which cracked the latest changes in Mac / iOS.
So, writes Chani Armitage.
')
I tried to write this post for many months, and usually everything ended with a complete stupor, or some other misunderstanding. So, to score - I will start writing, and then we'll see what happens.
What I wanted to talk about was sessions, XSMP, and Wayland.
Activities (KDE Activities) use the XSMP, X Session Management Protocol, and Xbox Session Management Protocol. They use it to save and restore groups of windows. Previously, it was used only for resuming a session at login. In fact, this protocol is much better than it is usually thought of, and it has served well for decades — but times are changing. If we want to move forward and do such cool things as common sessions for several devices, we need something new. Even in Activities, since they are now expanding the limitations of XSMP, there are many hidden hideous crutches that are not needed.
So, the key point is that if we want to replace the old, but reliable, technology with something new, then we want this something new to be not just better, but much better. Something that is really worth moving over. Small problems like the lack of XSMP autosave are easily solved; but there is a big problem that XSMP is process based. Session keys are mapped to individual processes, and when you need to recover, this happens by calling a specific binary to which the session key is passed as an argument. This is the root of evil; for starters, when the process has a lot of windows (such as a console), it is impossible to properly scatter these windows in several sessions - therefore, Activities should do any nastiness to hide it from the user. Secondly, an even greater problem is that such a method is ridiculously intolerable and opaque. What if the program's binary has moved somewhere? What if the user began to use an alternative application to replace the old one? And, of course, the same programs are not installed on the user's cell phone as on the stationary computer! Even if he has a phone on meego, and he uses Calligra on all his mobile and non-mobile devices, the binary on the phone will be called differently. : P
And what should we do? Let's use resources! :) We will store data in a standard place in a standard format. To see the session manager, instead of seeing “in this session are / usr / bin / firefox, / usr / bin / konsole and / usr / bin / okular”, he saw “in this session there are web pages X, Y and Z that last time on this device opened with firefox, with X and Y grouped in the same window; two terminals for which konsole was last used; and foo.pdf, opened on page 3, which last time opened on this device with okular. ” Then, if okular is not available, it can request the program to open pdf from the system, and therefore, when the user sends a session to the phone, it is absolutely unimportant that another pdf reader is used on the phone, or even that the phone browser does not support tabs. In addition, if instead of 3 pages open 20, the phone may say “oyoi, this is somehow a bit too much; I will open only the first three, and I will display links to all the others in a special place. ” Or, for example, I can think: “why did I create an activity called 'some kind of garbage'? What the hell is there? ”, And I can get an answer without even having to download this session.
The other side of the coin is that the little things decide. I recently heard that OSX added strong session support; they learned from the example of the iPhone and applied their knowledge to the desktop. Now there is a system for which it is absolutely unimportant if the application suddenly falls off - the operating system can kill it whenever it wants - support for the sessions is so good that the application will be restored to the same state it was before. It would be very nice to see support for sessions of similar quality in Linux too; In KDE applications, XSMP support is already pretty good, but it can be even better (and before that, the autosave protocol should go a long way).
Now, XSMP replacement is not a completely new idea. XSMP dwarfs didn’t like it for a long time, although I don’t think that any of them took steps to create an alternative (and for them, XSMP still works). When I spoke with the guys from the Nepomuk team many months ago about the sessions transferred between devices, it turned out that they already thought about it - but they decided that the political part of such a project would be too complicated. They may still be right - we'll see. But it's worth a try. :)
So what's the difficulty, you ask? Here's the problem I'm stuck with in developing XSMP for Activities: having a new protocol means that you need to convince applications to support this protocol. XSMP support, although not without problems, has recently become widespread and has caused quite a bit of resistance. The new protocol ... well, I’m pretty sure that they will use it in KDE, but I want it to become part of applications on gtk, on pure qt, on meego, and even on proprietary handicrafts. And now this is a real test! :)
So, how do we survive this challenge? To begin with, we will explain to everyone what kind of buns he will add - I hope I described it quite well above. Secondly, we will provide application developers with a well-designed, easily used API, this should cause a reaction of developers who are really doing something; Recently, I saw a person at IRC swearing at XSMP :) Thirdly, as soon as we have enough sympathizers to know that the result is worth trying, we will show them a reliable implementation that will actually work (without critical bugs). Seeing is believing, right? Nobody wants to spend precious time on creating code for a system that does not exist yet, so as soon as we make a minimal set of features, sport a couple of applications, we will immediately show it all. The fourth is the secret weapon :) Weyland is gaining popularity, and when applications start switching to it, they will be able to switch to our protocol too, right? Wayland does not have its own session protocol; our protocol can be one.
So, who is with me? There are only a few days left until the Desktop Summit, and, suddenly, I am its organizer (yahhoo!), And I would like to discuss there the topics described above. In the end, this is a great place for something that aims to be portable between devices, desktops, and everything you want.
TL; DR: A resource-based session protocol allows us to do really cool things, so replacing XSMP is well worth the effort.
Now, I want to delve into the technical details (I have enough material to write a whole work on this topic, if not this damn bad writer syndrome ...)
...
And what are we, technical details. What really needs to be implemented? What API do we need?
There are three interrelated parts. The first is shared storage. It comes first, because XSMP doesn’t say anything about how session data is stored; applications can be immediately started using this data, or continue to be run by the XSMP server (this can be a good solution during the migration phase. It will be a long time. Nothing much changes fast.)
Okay ... I haven't really studied the vaults. You can store it in nepomuk / tracker / zeitgeist, or in text files like kconfig, or even in the registry (eek) - it’s really only important that the system copes well with its tasks, and that all applications on the device use it only. It may turn out that different backends are suitable for different devices - I have not figured it out yet, and I would like to hear some information about the positive and negative sides of different approaches. Applications will regularly update certain data — page size, position in a film, etc. — so I would like to hear more about performance considerations.
At a minimum, I made a rough sketch of the data format. The main part of the resources will be some URI (usually local or http) and position within the document. The URI becomes the key, and the position is one of the fields ... Other fields will be the windows in which this resource is located (after all, the windows are still a logical unit of window managers, and their position and size are important session data) that was last used applications, own data fields for the program (preferably, the smaller, the better), and, possibly, data from other devices (to help with solutions that the device cannot handle by itself, or in the case of transferring the session to some devices where these data will be it be meaning). Session data for windows will be recorded, perhaps by the window manager itself ... Actually, I need a clean board and also programmers to come up with good data structures, which is why I am going to do it in Berlin. And I am sure that even these decisions will change during the implementation :) For a start, it is important to build at least some kind of resource-based structure that will work.
The second part is the session manager. This is a program — most likely a demon, but not necessarily — that loads applications when you open a session and tells them to close when necessary. If there are any interactive processes (“do you want to save this document?”), This should also be processed. This is what ksm-server is doing for XSMP now. In fact, it will not take much time, for the most part it will go on reading and writing configuration files, but everything is complicated by the ability of applications to request communication with the user (suspending session closure) or even cancel it entirely.
The third part is the transparent API, the part that most developers will see. I want the API to be good . I would also like to have backward and forward compatibility; The XSMP protocol has never changed, except to fix implementation bugs or design errors, but the new protocol will change. Most applications will have a shared dynamically linked library, but should we worry about statically linked? Fortunately, we came up with a design that does a pretty good job with such cases. I remember the dbus; He describes a bus format that allows applications to always communicate in the same language, but there is a drawback - they can never improve this format. : /
Speaking of dbus, it is one of the things that interests me, how should the session manager and applications communicate with each other? I think the repository should be written so that it can be accessed as directly as possible, for efficiency reasons (simultaneous recording conflicts will be vanishingly rare, if at all, and I expect the same from concurrent write and read conflicts). We can expect something like a sharp decrease in performance if the file system buffer is too small; in the worst case, imagine thousands of applications starting to update their state every 10 seconds without any synchronization; it may be that more than one write operation occurs per second. Fortunately, applications that will make such regular updates will be a little, one or two (you can watch only one movie, or even when reading - to listen to background music from just one source), but still this is something to think about later .
Oops, I have slightly deviated; Let's return to communication with the session manager. When I updated ksmserver and kwin to support sub-sessions (that is, activities), I got stuck in a quagmire with timeouts in dbus. Most likely, I was not the only one who walked this path, but I do not know of any alternative. XSMP uses some kind of ancient protocol called ICE, which seems to be tightly tied to X11; There are many protocols with this name, so googling for it is quite unpleasant. In any case, I want to break the X11 string, so that it can be easily done with Wayland - what do they use to communicate there?
Continuing to talk further (my hands are already tired of writing, despite the fact that I take regular breaks), the API is exactly defined by the capabilities it provides. I think that from the first version you need to throw out all the possibilities, except for the most vital ones; such as communication with the user and cancellation of the session. I tend to cancel. Now 2011, not 1990; applications should behave correctly when it is their time to close. Even kate now has paging files to recover from crashing; they can be used to resume the session. The only negative is that they are not so portable ... but I think this problem can be postponed;) Hell, I have to postpone even the ability to add one window to N sessions at the same time; this complicates everything so much that I am no longer sure that this is really a worthwhile feature for Activities, but at the same time, this opportunity should still be available under the hood, realized by creating additional sessions.
So what opportunities do we really need?
- the application or window manager should record the size and position of the windows. If in the end the application will be engaged in this, it should happen absolutely automatically, with the help of the library, and not something the application developer should worry about.
- applications need to record the fact that they display the resource in a particular window
- they also need to record that they have stopped displaying it :)
- either a window or a resource must be associated with a particular session at the time of display
- applications need to talk when they need to close
- applications should be able to recover themselves from session data
- applications must have a method of storing their own non-standard data in the session
This is just the basics, copying the capabilities of XSMP. To do something really cool, we have to do this:
- tell the application exactly which windows to close if these windows are distributed between several sessions
- tell the application whether it should resume the entire session or only part of it
- allow and encourage applications to store common portable session data (such as a position in a document) in a standard place, and so that any application can use this data
I am sure that I missed a couple of things, but you already understood the idea :)
In fact, there is a fourth part: synchronization between devices. How will the two devices exchange their session data? Sending a list of resources and related data sounds pretty simple, but there are a lot of details that still need to be understood - what resources need to be copied to other devices, how to handle changing the URI (file /home/chani/Documents/foo.pdf on the stationary computer will be on the phone somewhere in a very different place) when trying to synchronize and how to resolve conflicts ... :) I am sure there are people who understand much more than me in all this. And, as a bonus, the resulting synchronization code should be ready for transfer between distributions, and even, someday, between most different backends for data storage.
Realization of all this will take time, adaptation will take even longer. This is a project for many years. But, if people really want something, all this is completely realizable.
The post attracted me primarily because the author comes to the idea of a shared data repository, and touches upon the cross-device trend, which cracked the latest changes in Mac / iOS.
So, writes Chani Armitage.
')
I tried to write this post for many months, and usually everything ended with a complete stupor, or some other misunderstanding. So, to score - I will start writing, and then we'll see what happens.
What I wanted to talk about was sessions, XSMP, and Wayland.
Activities (KDE Activities) use the XSMP, X Session Management Protocol, and Xbox Session Management Protocol. They use it to save and restore groups of windows. Previously, it was used only for resuming a session at login. In fact, this protocol is much better than it is usually thought of, and it has served well for decades — but times are changing. If we want to move forward and do such cool things as common sessions for several devices, we need something new. Even in Activities, since they are now expanding the limitations of XSMP, there are many hidden hideous crutches that are not needed.
So, the key point is that if we want to replace the old, but reliable, technology with something new, then we want this something new to be not just better, but much better. Something that is really worth moving over. Small problems like the lack of XSMP autosave are easily solved; but there is a big problem that XSMP is process based. Session keys are mapped to individual processes, and when you need to recover, this happens by calling a specific binary to which the session key is passed as an argument. This is the root of evil; for starters, when the process has a lot of windows (such as a console), it is impossible to properly scatter these windows in several sessions - therefore, Activities should do any nastiness to hide it from the user. Secondly, an even greater problem is that such a method is ridiculously intolerable and opaque. What if the program's binary has moved somewhere? What if the user began to use an alternative application to replace the old one? And, of course, the same programs are not installed on the user's cell phone as on the stationary computer! Even if he has a phone on meego, and he uses Calligra on all his mobile and non-mobile devices, the binary on the phone will be called differently. : P
And what should we do? Let's use resources! :) We will store data in a standard place in a standard format. To see the session manager, instead of seeing “in this session are / usr / bin / firefox, / usr / bin / konsole and / usr / bin / okular”, he saw “in this session there are web pages X, Y and Z that last time on this device opened with firefox, with X and Y grouped in the same window; two terminals for which konsole was last used; and foo.pdf, opened on page 3, which last time opened on this device with okular. ” Then, if okular is not available, it can request the program to open pdf from the system, and therefore, when the user sends a session to the phone, it is absolutely unimportant that another pdf reader is used on the phone, or even that the phone browser does not support tabs. In addition, if instead of 3 pages open 20, the phone may say “oyoi, this is somehow a bit too much; I will open only the first three, and I will display links to all the others in a special place. ” Or, for example, I can think: “why did I create an activity called 'some kind of garbage'? What the hell is there? ”, And I can get an answer without even having to download this session.
The other side of the coin is that the little things decide. I recently heard that OSX added strong session support; they learned from the example of the iPhone and applied their knowledge to the desktop. Now there is a system for which it is absolutely unimportant if the application suddenly falls off - the operating system can kill it whenever it wants - support for the sessions is so good that the application will be restored to the same state it was before. It would be very nice to see support for sessions of similar quality in Linux too; In KDE applications, XSMP support is already pretty good, but it can be even better (and before that, the autosave protocol should go a long way).
Now, XSMP replacement is not a completely new idea. XSMP dwarfs didn’t like it for a long time, although I don’t think that any of them took steps to create an alternative (and for them, XSMP still works). When I spoke with the guys from the Nepomuk team many months ago about the sessions transferred between devices, it turned out that they already thought about it - but they decided that the political part of such a project would be too complicated. They may still be right - we'll see. But it's worth a try. :)
So what's the difficulty, you ask? Here's the problem I'm stuck with in developing XSMP for Activities: having a new protocol means that you need to convince applications to support this protocol. XSMP support, although not without problems, has recently become widespread and has caused quite a bit of resistance. The new protocol ... well, I’m pretty sure that they will use it in KDE, but I want it to become part of applications on gtk, on pure qt, on meego, and even on proprietary handicrafts. And now this is a real test! :)
So, how do we survive this challenge? To begin with, we will explain to everyone what kind of buns he will add - I hope I described it quite well above. Secondly, we will provide application developers with a well-designed, easily used API, this should cause a reaction of developers who are really doing something; Recently, I saw a person at IRC swearing at XSMP :) Thirdly, as soon as we have enough sympathizers to know that the result is worth trying, we will show them a reliable implementation that will actually work (without critical bugs). Seeing is believing, right? Nobody wants to spend precious time on creating code for a system that does not exist yet, so as soon as we make a minimal set of features, sport a couple of applications, we will immediately show it all. The fourth is the secret weapon :) Weyland is gaining popularity, and when applications start switching to it, they will be able to switch to our protocol too, right? Wayland does not have its own session protocol; our protocol can be one.
So, who is with me? There are only a few days left until the Desktop Summit, and, suddenly, I am its organizer (yahhoo!), And I would like to discuss there the topics described above. In the end, this is a great place for something that aims to be portable between devices, desktops, and everything you want.
TL; DR: A resource-based session protocol allows us to do really cool things, so replacing XSMP is well worth the effort.
Now, I want to delve into the technical details (I have enough material to write a whole work on this topic, if not this damn bad writer syndrome ...)
...
And what are we, technical details. What really needs to be implemented? What API do we need?
There are three interrelated parts. The first is shared storage. It comes first, because XSMP doesn’t say anything about how session data is stored; applications can be immediately started using this data, or continue to be run by the XSMP server (this can be a good solution during the migration phase. It will be a long time. Nothing much changes fast.)
Okay ... I haven't really studied the vaults. You can store it in nepomuk / tracker / zeitgeist, or in text files like kconfig, or even in the registry (eek) - it’s really only important that the system copes well with its tasks, and that all applications on the device use it only. It may turn out that different backends are suitable for different devices - I have not figured it out yet, and I would like to hear some information about the positive and negative sides of different approaches. Applications will regularly update certain data — page size, position in a film, etc. — so I would like to hear more about performance considerations.
At a minimum, I made a rough sketch of the data format. The main part of the resources will be some URI (usually local or http) and position within the document. The URI becomes the key, and the position is one of the fields ... Other fields will be the windows in which this resource is located (after all, the windows are still a logical unit of window managers, and their position and size are important session data) that was last used applications, own data fields for the program (preferably, the smaller, the better), and, possibly, data from other devices (to help with solutions that the device cannot handle by itself, or in the case of transferring the session to some devices where these data will be it be meaning). Session data for windows will be recorded, perhaps by the window manager itself ... Actually, I need a clean board and also programmers to come up with good data structures, which is why I am going to do it in Berlin. And I am sure that even these decisions will change during the implementation :) For a start, it is important to build at least some kind of resource-based structure that will work.
The second part is the session manager. This is a program — most likely a demon, but not necessarily — that loads applications when you open a session and tells them to close when necessary. If there are any interactive processes (“do you want to save this document?”), This should also be processed. This is what ksm-server is doing for XSMP now. In fact, it will not take much time, for the most part it will go on reading and writing configuration files, but everything is complicated by the ability of applications to request communication with the user (suspending session closure) or even cancel it entirely.
The third part is the transparent API, the part that most developers will see. I want the API to be good . I would also like to have backward and forward compatibility; The XSMP protocol has never changed, except to fix implementation bugs or design errors, but the new protocol will change. Most applications will have a shared dynamically linked library, but should we worry about statically linked? Fortunately, we came up with a design that does a pretty good job with such cases. I remember the dbus; He describes a bus format that allows applications to always communicate in the same language, but there is a drawback - they can never improve this format. : /
Speaking of dbus, it is one of the things that interests me, how should the session manager and applications communicate with each other? I think the repository should be written so that it can be accessed as directly as possible, for efficiency reasons (simultaneous recording conflicts will be vanishingly rare, if at all, and I expect the same from concurrent write and read conflicts). We can expect something like a sharp decrease in performance if the file system buffer is too small; in the worst case, imagine thousands of applications starting to update their state every 10 seconds without any synchronization; it may be that more than one write operation occurs per second. Fortunately, applications that will make such regular updates will be a little, one or two (you can watch only one movie, or even when reading - to listen to background music from just one source), but still this is something to think about later .
Oops, I have slightly deviated; Let's return to communication with the session manager. When I updated ksmserver and kwin to support sub-sessions (that is, activities), I got stuck in a quagmire with timeouts in dbus. Most likely, I was not the only one who walked this path, but I do not know of any alternative. XSMP uses some kind of ancient protocol called ICE, which seems to be tightly tied to X11; There are many protocols with this name, so googling for it is quite unpleasant. In any case, I want to break the X11 string, so that it can be easily done with Wayland - what do they use to communicate there?
Continuing to talk further (my hands are already tired of writing, despite the fact that I take regular breaks), the API is exactly defined by the capabilities it provides. I think that from the first version you need to throw out all the possibilities, except for the most vital ones; such as communication with the user and cancellation of the session. I tend to cancel. Now 2011, not 1990; applications should behave correctly when it is their time to close. Even kate now has paging files to recover from crashing; they can be used to resume the session. The only negative is that they are not so portable ... but I think this problem can be postponed;) Hell, I have to postpone even the ability to add one window to N sessions at the same time; this complicates everything so much that I am no longer sure that this is really a worthwhile feature for Activities, but at the same time, this opportunity should still be available under the hood, realized by creating additional sessions.
So what opportunities do we really need?
- the application or window manager should record the size and position of the windows. If in the end the application will be engaged in this, it should happen absolutely automatically, with the help of the library, and not something the application developer should worry about.
- applications need to record the fact that they display the resource in a particular window
- they also need to record that they have stopped displaying it :)
- either a window or a resource must be associated with a particular session at the time of display
- applications need to talk when they need to close
- applications should be able to recover themselves from session data
- applications must have a method of storing their own non-standard data in the session
This is just the basics, copying the capabilities of XSMP. To do something really cool, we have to do this:
- tell the application exactly which windows to close if these windows are distributed between several sessions
- tell the application whether it should resume the entire session or only part of it
- allow and encourage applications to store common portable session data (such as a position in a document) in a standard place, and so that any application can use this data
I am sure that I missed a couple of things, but you already understood the idea :)
In fact, there is a fourth part: synchronization between devices. How will the two devices exchange their session data? Sending a list of resources and related data sounds pretty simple, but there are a lot of details that still need to be understood - what resources need to be copied to other devices, how to handle changing the URI (file /home/chani/Documents/foo.pdf on the stationary computer will be on the phone somewhere in a very different place) when trying to synchronize and how to resolve conflicts ... :) I am sure there are people who understand much more than me in all this. And, as a bonus, the resulting synchronization code should be ready for transfer between distributions, and even, someday, between most different backends for data storage.
Realization of all this will take time, adaptation will take even longer. This is a project for many years. But, if people really want something, all this is completely realizable.
Source: https://habr.com/ru/post/125488/
All Articles