Ilya Dronov: how LiveJournal fights spam
Ilya Dronov, director of product development SUP, answers questions from habraludey and LJ users. Almost all interview questions are about the poor performance of Livejournal, the prevalence of spam and the lack of the necessary functionality.
“Why are you fighting spam so badly?” If you give people the opportunity to ban users themselves, it will be much easier and better for everyone. Or, for example, you can ban on IP.
- At the moment, users already have the opportunity to ban other users in their journal, but this is done by robots by setting up a large number of accounts and using the “one comment per diary” rule. Recently, we have expanded this opportunity by distributing such a list to all mutual friends: if several of your mutual friends ban a specific user for spam, then this user will not be able to harm you. Naturally, this only applies to the ban for spam.
As for the IP address, such a mechanism is already available and available to members of the Conflict Commission, but we try to use it as rarely as possible for two reasons: 1) It is very easy to change the IP address, because there are a lot of public proxy servers; 2) it is likely to block the IP address of a public IP from which many regular users log in to LiveJournal. Not to mention the fact that there are countries in which LiveJournal is closed, and they just have to use similar systems.
- When can I delete all entries from the diary? With a choice, of course, for example, place a checkmark and erase a hundred records at once?
- Unfortunately, now you can not delete entries through the "list" tools of the service itself. Most likely, this is available through third-party applications, since this feature is provided in the API. I heard you, we will add this option to the journal editing page (http://www.livejournal.com/editjournal.bml) in one of the next releases.
')
- It’s no secret that you introduce some functions only for Cyrillic users. Do you specifically want to be connected as much as possible? What is the difference between Cyrillic and non-Cyrillic users?
- This is due to different perceptions of LiveJournal in different regions and countries. For example, in the Cyrillic segment, there is the concept of ranking users and records, LiveJournal is perceived as a platform that allows you to speak publicly or share something with as many people as possible. It's quite another thing in the USA: their diaries are a purely personal space for friends (although they keep them publicly), and the appearance in the comments of unfamiliar users sometimes causes mixed reactions to users. Therefore, they have a much larger weight of the community.
The concept of “Cyrillic services” is a tribute to history, we are going to abandon it in the future, we will replace it with reference to the region of residence. That is, the part of the functionality and the content being promoted will depend on which country you are located in your profile.
- My LiveJournal is constantly slowing down, comments come late. Are you planning to fix this?
- We are constantly working on this, the main reasons for slow work are two.
First, problems with the communication channel. All LiveJournal servers are located in the United States, and the speed at which data is returned (despite the fact that LiveJournal itself generates pages on average in less than a second) is highly dependent on their workload or on their failures. Secondly, the sharply increased traffic to the server in connection with the occurrence of any events - both positive and negative. All of the above problems we solve and are constantly working to increase the power reserve. Over the past few years, traffic has only grown - we are still alive and continue to grow. This proves that we are working on solving problems, just not all users notice this.
- I am not subscribed to Cyrillic services and I do not have spam ...
- Spam does not depend on a subscription to Cyrillic services, here is another reason. Perhaps this is a special mode of comments, or you are not a popular enough user. Yes, spammers are very selective now.
- Will anything be changed in the design of LiveJournal? Nightmare design, just nightmarish!
- Everything will be changed in the near future.
- Spam at hand SUP? Why aren't you fighting him?
- Spam can not be in the hands of the company for two reasons: firstly, it gives useless load on our servers and, secondly, annoys users. We fight spam, just the methods we use have a slightly delayed, but much stronger effect.
- All spam often contains the same link, up to parameters. Is it really so difficult to kill this evil on takeoff?
- Blocking one link is easy, but the second one will follow. Block the second - followed by the third and so on ad infinitum, until the links just start wrapping cuts, which will lead to the complication of dealing with them. In fact, spam with a link is of two types - SEO and traffic. The narrow throat of the first type is the appearance of such links for search engines, for the second - coverage of the audience. Both of them overlap with the hiding of comments, which means lack of effectiveness and leads to the lack of benefits of the spread of such spam. I understand that the traces of this spam still annoy the owners of diaries with rubbish in the comments, but we recently solved this problem by introducing a special section of suspicious comments where they can be removed at once.
- It seems to me that your top does not work correctly. There are a variety of "illegal" techniques, cheating visits for money, and people can push through unpopular and uninteresting people records. If you sort the top "by views" and "by visitors", then the posts are sorted for the last day, if we click to sort "by rating", then for some reason the top of the received ratings for all the time of LiveJournal will appear. Does it mean that positive or negative “assessments” do not affect anything?
- Ratings at the moment really do not affect the rating, except that you can sort by them as a separate entity. We have already implemented many anti-cheat mechanisms and are constantly developing them, and soon we plan to introduce several more - they will allow users to influence getting to the top, which, therefore, will become less dependent on attendance - which, as you correctly noticed, you can buy.
- How to get rid of frandospamier? Terribly enraged.
- Protection against mass franding was introduced a few days before the New Year, do you still notice it? If so, please contact me so that we can find leaks and shortcomings of the algorithm. You can leave a comment in my journal igrick.livejournal.com or write me a private message.
- A friend often writes notes backdating (through semagic), that is, not wanting to write all the records at once for one day, scatters them on the dates of the events described. And these entries in the friends feed are not displayed at all, they never appear there at all. Why is this happening and how to fix it?
- Friendlenta is sorted by user dates (that is, not by the date the record was created, but by the date specified in the record) - this is how it should be. We are currently working on a mechanism that will allow displaying posts on a friend by creation date, as well as the ability to create pending entries.
- Why not add the domains of the services for the shortened entry of the URL in the LJ-shny "black" list so that this dirty trick does not appear in the messages and comments?
- Because they are used by normal users, as well as various kinds of integration with Twitter - both ours, "relatives", and third-party.
- Why did you cut LJTimes to everyone? Beta versions are usually offered to try, rather than impose.
- In fact, in the first version, before the introduction of the display shutdown completely, it was possible to remove LJTimes by minimizing it to the minimum state. Users are often conservative and first accept innovations "in the bayonets" - now we find a lot of positive reviews about LJTimes, many of which for the first time turned off the Cyrillic services, turned them on a little later.
- When will your innovations be normal, meaningful? Integration with Facebook and Twitter is more from Facebook and Twitter, you can integrate the devil of bald in Facebook, Learn is not much different, for example, I do not know the API Learn. I am silent, that you are 11 years old, and you are still on the old code, during this time you could build a space ship!
- Yes, the service is over 11 years old, but do not forget that SUP Media has been in possession for 4 years. LiveJournal has both an API and everything you need to integrate both LiveJournal to yourself and something in LiveJournal. Few people know about this, but we do not plan to promote it until we make this product understandable to a large audience. As for “significant innovations” - this is always a subjective view, for some, some innovations are significant, for someone they do not make any sense. You will not please everyone at once, but we have big plans, I think everyone will find in them something interesting for themselves.
- In the code LiveJournal remained fragments, written by Fitzpatrick?
- Yes, and quite a lot.
- How many developers are there in the company? How many of them are working on the LiveJournal engine?
- There are more than 50 developers in the company, about 20 are currently working on LiveJournal, including ServerSide, JavaScript and HTML.
- Why such a big staff turnover? One gets the impression that every two or three years new people are engaged in the development of LiveJournal.
- In LiveJournal there is no large turnover of personnel, on the contrary, most developers work in the company for a long time.
- In which areas are you planning to focus when modifying the engine? What are the most fundamental changes to be made in the coming years?
- Basically, on the architecture, it is she who is the bottleneck of the system - the current one is simply not designed for the existing growth dynamics. Of the most cardinal, I would point out the interface of the service, since it will be the most noticeable change for users.
- Is it possible to call LiveJournal a Web 2.0 service? How extensively are Ajax, HTML5 used in it?
- Every month we are using AJAX more and more, but we still cannot call LiveJournal Web 2.0 a service.
reference
Ilya Dronov, SUP Product Development Director. He graduated from the Moscow Institute of Energy Industry (Moscow State Institute of Radio Engineering, Electronics and Automation), faculty of computers and systems. He has been working in the field of Internet development and programming since 1999. During this time, he headed a company that creates web solutions for small and medium-sized businesses, and also headed the Internet development department of Renaissance Insurance.
“Why are you fighting spam so badly?” If you give people the opportunity to ban users themselves, it will be much easier and better for everyone. Or, for example, you can ban on IP.
- At the moment, users already have the opportunity to ban other users in their journal, but this is done by robots by setting up a large number of accounts and using the “one comment per diary” rule. Recently, we have expanded this opportunity by distributing such a list to all mutual friends: if several of your mutual friends ban a specific user for spam, then this user will not be able to harm you. Naturally, this only applies to the ban for spam.
As for the IP address, such a mechanism is already available and available to members of the Conflict Commission, but we try to use it as rarely as possible for two reasons: 1) It is very easy to change the IP address, because there are a lot of public proxy servers; 2) it is likely to block the IP address of a public IP from which many regular users log in to LiveJournal. Not to mention the fact that there are countries in which LiveJournal is closed, and they just have to use similar systems.
- When can I delete all entries from the diary? With a choice, of course, for example, place a checkmark and erase a hundred records at once?
- Unfortunately, now you can not delete entries through the "list" tools of the service itself. Most likely, this is available through third-party applications, since this feature is provided in the API. I heard you, we will add this option to the journal editing page (http://www.livejournal.com/editjournal.bml) in one of the next releases.
')
- It’s no secret that you introduce some functions only for Cyrillic users. Do you specifically want to be connected as much as possible? What is the difference between Cyrillic and non-Cyrillic users?
- This is due to different perceptions of LiveJournal in different regions and countries. For example, in the Cyrillic segment, there is the concept of ranking users and records, LiveJournal is perceived as a platform that allows you to speak publicly or share something with as many people as possible. It's quite another thing in the USA: their diaries are a purely personal space for friends (although they keep them publicly), and the appearance in the comments of unfamiliar users sometimes causes mixed reactions to users. Therefore, they have a much larger weight of the community.
The concept of “Cyrillic services” is a tribute to history, we are going to abandon it in the future, we will replace it with reference to the region of residence. That is, the part of the functionality and the content being promoted will depend on which country you are located in your profile.
- My LiveJournal is constantly slowing down, comments come late. Are you planning to fix this?
- We are constantly working on this, the main reasons for slow work are two.
First, problems with the communication channel. All LiveJournal servers are located in the United States, and the speed at which data is returned (despite the fact that LiveJournal itself generates pages on average in less than a second) is highly dependent on their workload or on their failures. Secondly, the sharply increased traffic to the server in connection with the occurrence of any events - both positive and negative. All of the above problems we solve and are constantly working to increase the power reserve. Over the past few years, traffic has only grown - we are still alive and continue to grow. This proves that we are working on solving problems, just not all users notice this.
- I am not subscribed to Cyrillic services and I do not have spam ...
- Spam does not depend on a subscription to Cyrillic services, here is another reason. Perhaps this is a special mode of comments, or you are not a popular enough user. Yes, spammers are very selective now.
- Will anything be changed in the design of LiveJournal? Nightmare design, just nightmarish!
- Everything will be changed in the near future.
- Spam at hand SUP? Why aren't you fighting him?
- Spam can not be in the hands of the company for two reasons: firstly, it gives useless load on our servers and, secondly, annoys users. We fight spam, just the methods we use have a slightly delayed, but much stronger effect.
- All spam often contains the same link, up to parameters. Is it really so difficult to kill this evil on takeoff?
- Blocking one link is easy, but the second one will follow. Block the second - followed by the third and so on ad infinitum, until the links just start wrapping cuts, which will lead to the complication of dealing with them. In fact, spam with a link is of two types - SEO and traffic. The narrow throat of the first type is the appearance of such links for search engines, for the second - coverage of the audience. Both of them overlap with the hiding of comments, which means lack of effectiveness and leads to the lack of benefits of the spread of such spam. I understand that the traces of this spam still annoy the owners of diaries with rubbish in the comments, but we recently solved this problem by introducing a special section of suspicious comments where they can be removed at once.
- It seems to me that your top does not work correctly. There are a variety of "illegal" techniques, cheating visits for money, and people can push through unpopular and uninteresting people records. If you sort the top "by views" and "by visitors", then the posts are sorted for the last day, if we click to sort "by rating", then for some reason the top of the received ratings for all the time of LiveJournal will appear. Does it mean that positive or negative “assessments” do not affect anything?
- Ratings at the moment really do not affect the rating, except that you can sort by them as a separate entity. We have already implemented many anti-cheat mechanisms and are constantly developing them, and soon we plan to introduce several more - they will allow users to influence getting to the top, which, therefore, will become less dependent on attendance - which, as you correctly noticed, you can buy.
- How to get rid of frandospamier? Terribly enraged.
- Protection against mass franding was introduced a few days before the New Year, do you still notice it? If so, please contact me so that we can find leaks and shortcomings of the algorithm. You can leave a comment in my journal igrick.livejournal.com or write me a private message.
- A friend often writes notes backdating (through semagic), that is, not wanting to write all the records at once for one day, scatters them on the dates of the events described. And these entries in the friends feed are not displayed at all, they never appear there at all. Why is this happening and how to fix it?
- Friendlenta is sorted by user dates (that is, not by the date the record was created, but by the date specified in the record) - this is how it should be. We are currently working on a mechanism that will allow displaying posts on a friend by creation date, as well as the ability to create pending entries.
- Why not add the domains of the services for the shortened entry of the URL in the LJ-shny "black" list so that this dirty trick does not appear in the messages and comments?
- Because they are used by normal users, as well as various kinds of integration with Twitter - both ours, "relatives", and third-party.
- Why did you cut LJTimes to everyone? Beta versions are usually offered to try, rather than impose.
- In fact, in the first version, before the introduction of the display shutdown completely, it was possible to remove LJTimes by minimizing it to the minimum state. Users are often conservative and first accept innovations "in the bayonets" - now we find a lot of positive reviews about LJTimes, many of which for the first time turned off the Cyrillic services, turned them on a little later.
- When will your innovations be normal, meaningful? Integration with Facebook and Twitter is more from Facebook and Twitter, you can integrate the devil of bald in Facebook, Learn is not much different, for example, I do not know the API Learn. I am silent, that you are 11 years old, and you are still on the old code, during this time you could build a space ship!
- Yes, the service is over 11 years old, but do not forget that SUP Media has been in possession for 4 years. LiveJournal has both an API and everything you need to integrate both LiveJournal to yourself and something in LiveJournal. Few people know about this, but we do not plan to promote it until we make this product understandable to a large audience. As for “significant innovations” - this is always a subjective view, for some, some innovations are significant, for someone they do not make any sense. You will not please everyone at once, but we have big plans, I think everyone will find in them something interesting for themselves.
- In the code LiveJournal remained fragments, written by Fitzpatrick?
- Yes, and quite a lot.
- How many developers are there in the company? How many of them are working on the LiveJournal engine?
- There are more than 50 developers in the company, about 20 are currently working on LiveJournal, including ServerSide, JavaScript and HTML.
- Why such a big staff turnover? One gets the impression that every two or three years new people are engaged in the development of LiveJournal.
- In LiveJournal there is no large turnover of personnel, on the contrary, most developers work in the company for a long time.
- In which areas are you planning to focus when modifying the engine? What are the most fundamental changes to be made in the coming years?
- Basically, on the architecture, it is she who is the bottleneck of the system - the current one is simply not designed for the existing growth dynamics. Of the most cardinal, I would point out the interface of the service, since it will be the most noticeable change for users.
- Is it possible to call LiveJournal a Web 2.0 service? How extensively are Ajax, HTML5 used in it?
- Every month we are using AJAX more and more, but we still cannot call LiveJournal Web 2.0 a service.
reference
Ilya Dronov, SUP Product Development Director. He graduated from the Moscow Institute of Energy Industry (Moscow State Institute of Radio Engineering, Electronics and Automation), faculty of computers and systems. He has been working in the field of Internet development and programming since 1999. During this time, he headed a company that creates web solutions for small and medium-sized businesses, and also headed the Internet development department of Renaissance Insurance.
Source: https://habr.com/ru/post/124152/
All Articles