Adventure out of the blue

How Spotify can assist in learning about demons, RFCs, networks and the promotion of open source. Or what happens if it fails to pay, and some premium buns really want to.

Start

On the third day, it was noticed that Spotifay advertises on the basis of the country’s ip-address. It was also noted that in some countries advertising was not delivered at all. For example, in RB . And then the “ingenious” plan for turning off advertising in a non-premium account was ripe.

A little about Spotify

Generally speaking, Spotify has a strange policy. Our brother has to be pretty perverted to buy a premium: change the location in the profile to overseas, look for a suitable gift card, which can only be paid by paypal, who has been freaking out and wants a bunch of documents lately. In general, also an adventure, but of a different order. Although, most of this is done for the sake of the mobile version, I am not interested in such. Therefore, all of the following will help only in the case of the desktop version. Moreover, there will be no extension of functions. Just cutting off some extras.

And what is difficult?

And I thought so, writing the data socks-proxy in the Spotify config. The problem turned out to be that authentication in socks with login and password does not work for them. Plus, the developers regularly twist something around a proxy: either by allowing, then prohibiting, or breaking it, which gives rise to whole panels of discussions on off.sayte.

It was decided not to rely on unstable functions and find something more reliable and curious.

Somewhere here the reader should ask: why not take ssh with the -D key and deal with it? And, in general, it will be right. But, first, it still needs to be demonized and befriended with autossh, so as not to think about the torn connections. And secondly: it is too simple and boring.

In order

As usual, let's go from left to right, top to bottom and describe everything we need to implement our "simple" idea.

First you need a proxy

And at once there are many alternatives:

• you can just go and get out of the open proxilists. Cheap (more precisely for free), but absolutely unreliable and the lifetime of such proxies tends to zero. Therefore, it would be necessary to find / write the proxy list parser, filter them according to the desired type and country, and the substitution question of the found proxy in Spotify remains open (well, unless you send a custom wrapper for the binary to HTTP_PROXY so that the rest of the traffic will not be sent there) ).
• You can buy a similar proxy and rid yourself of most of the problems described above. But at the price of a proxy, you can immediately buy a premium on Spotify, and this is not sports as part of the original task.
• Raise yours. As you probably guessed, this is our choice.

Purely by chance it may turn out that you have a friend with a server in the Republic of Belarus or another small country. This need to use and roll on it the desired proxy. Special connoisseurs can be content with a friend using a DD-WRT or similar software. But there is a wonderful world there and in the framework of this story this world clearly does not fit.

So, our options: Squid does not inspire, and I don’t want HTTP proxy, this protocol is too much around. And in the field of SOCKS nothing sensible except Dante has yet been delivered. Therefore, we take it.

Manul on installing and configuring Dante do not wait. It is simply googling and is of no particular interest. In the minimum configuration it is necessary to distribute all sorts of client pass , socks pass , correctly register interfaces and do not forget to add socksmethod: username . In this form, for authentication, the system will be taken from users of the system. And the part about security: prohibition of access to localhost, restriction on users, and so on - this is purely individual, depending on personal paranoia.

Deploy proxy face to network

Performance in two acts.

Act one

With the proxy figured out, now we need to access it from the global web. If you have a machine with a white IP in the desired country, then you can safely skip this item. We have no such (we, as mentioned above, are hosted by friends at home) and the nearest white ip is somewhere in Germany, so we will study the network.

So yes, the attentive reader will again ask: why don't you take an existing service like ngrok or something similar? And he will be right again. But this is a service, again it needs to be demonized, it can also cost money, and in general it is not sports. Therefore, we will create bicycles from scrap materials.

Task: there is a proxy somewhere far behind the NAT, it is necessary to hang it on one of the VPS ports, which has a white IP and is located on the edge of the world.

It is logical to assume that this is solved either by forwarding the port (which is implemented through the aforementioned ssh ), or by combining hardware into a virtual network through a VPN. We are able to work with ssh , autossh boring to take, so let's take OpenVPN.

DigitalOcean has a great deal on this case. I have nothing to add to it. And the resulting config can be pretty easily made friends with the OpenVPN client and systemd . It is enough to put it (config) in /etc/openvpn/client/ and remember to change the extension to .conf . After that, pull openvpn-client@openvpn-client-config-name.service service, do not forget to enable it for it and rejoice that everything flew.

Of course, you need to disable any traffic redirection to the newly created VPN, because we don’t want to cut the speed on the client machine due to driving the traffic through the half-ball.

And yes, you need to register a static ip-address on the VPN server for our client. This will need a little further on the narrative. To do this, enable ifconfig-pool-persist , edit ipp.txt that comes with OpenVPN and enable client-config-dir, plus edit the configuration of the desired client, adding ifconfig-push with the correct mask and the desired IP address.

Act Two

Now we have a machine in the "network" that is turned into a person on the Internet and can be used for personal gain. Namely, to redirect some traffic through it.

So, a new task: you need to wrap traffic arriving on one of the VPS ports with a white ip so that this traffic flew into the newly connected virtual network and the answer could return from there.

Solution: of course iptables ! When will you have such a great opportunity to practice with him?

The configuration is needed quite soon, for three hours, a hundred expletives and a handful of spent nerves, for debugging networks is a very specific procedure.

First, you need to enable traffic redirection in the kernel. This thing is called ipv4.ip_forward and is included a little differently depending on the OS and network manager.

Secondly, you need to select the port on the VPS and wrap all the traffic going to it in the virtual subnet. This can be done, for example, like this:

 iptables -t nat -A PREROUTING -p tcp -i eth0 --dport 8080 -j DNAT --to-destination 10.8.0.2:8080 

Here we redirect all TCP traffic coming to port 8080 of the external interface to the machine with ip 10.8.0.2 and the same port 8080.

Those who want dirty details of netfilter , iptables and routing in general, it is absolutely necessary to contemplate this or that .

So, now we have packages fly into the virtual subnet and ... remain there. More precisely, the response from the socks-proxy flies back through the default gateway on the machine with Dante and the addressee drops it, because in networks it is not accepted to send a request to one IP, and to receive a response from the other. Therefore, you need to conjure further.

So, now you need to redirect all packets from the proxy back to the virtual subnet in the direction of the VPS with white IP. Here the situation is a bit worse, since simply iptables is not enough for us, because if we correct the destination address before routing ( PREROUTING ), then our package will not fly to the Internet, and if we do not correct it, the package will go to the default gateway . So, you need to do the following: remember about the mangle chain, in order to mark packets using iptables and wrap them in a custom routing table, which will send them where they should.

No sooner said than done:

 iptables -t mangle -A OUTPUT -p tcp --sport 8080 -j MARK --set-mark 0x80 ip rule add fwmark 0x80 table 80 ip route add default via 10.8.0.1 dev tun0 table 80 

We take outgoing traffic, mark everything that flies from the port on which the proxy sits (8080 in our case), redirect all marked traffic to the routing table with number 80 (in general, the number does not depend on anything, just wanted to) and add the only rule , by which all packets that fall into this table fly into the VPN subnet.

Fine! Now the packets fly back towards the VPS ... and die there. Because the VPS does not know what to do with them. Therefore, if you do not bother, you can simply take and redirect all traffic arriving from the virtual subnet back to the Internet:

 iptables -t nat -A POSTROUTING -s 10.8.0.0/24 -o eth0 -j SNAT --to-source 172.42.1.10 

Here, everything that comes from subnet 10.8.0.0 with mask 255.255.255.000 is wrapped in source-NAT and flies away to the default interface, which is turned to the Internet. It is important to note that this thing will work only if we transparently forward the port, that is, the incoming port on the VPS coincides with the port of our proxy. Otherwise, you will need to suffer a little more.

Somewhere now everything should start working. And there will be only a little: do not forget to make sure that all the iptables and route configs do not continue after the restart. For iptables there are special files like /etc/iptables/rules.v4 (in the case of Ubuntu), and for routs everything is a bit more complicated. I pushed OpenVPN scripts into the up/down scripts, although, I think, it could have been more decently done.

Wrap traffic from the application to the proxy

So, we have a proxy with authentication in the right country, accessible by a static white IP address. It remains to use it and redirect traffic from Spotify there. But there is a nuance, as mentioned above, the login-password for proxy in Spotifay does not work, so we will look for how to twist.

For a start, let's remember about proxifier . Great stuff, only costs like a starship ($ 40). We can buy a premium again for this money and be done with it. Therefore, we will look for more free and open analogues for poppy (yes, we want to listen to music on a poppy). Discover a whole one tool: proximac . And happily let's go poking him.

But the joy will be short-lived, because it turns out that you need to enable debug mode and custom kernel extensions in MacOS, file down a simple config and understand that this tool has exactly the same problem as Spotify: it cannot be authenticated with a login password. socks-proxy.

Somewhere here it's time to get crazy and still buy a premium ... but no! Let's try to ask to fix, etozh open source! We make a ticket . And in response, we get a sentimental story that the only Maintainer no longer has a MacBook and fuck you, and not a fix.

Frustrated again. But then we will remember the youth and C, turn on debug mode in Dante, dig a hundred kilobytes of logs, go to RFC1927 for information about the SOCKS5 protocol, deal with Xcode and find the problem. It is enough to fix one character in the list of method codes that the client offers for authentication and everything starts working like a clock. We rejoice, collect release binaries, do pool-requests and go to sunset go to the next item.

Automate it

Once Proximac has earned, it must be demonized and forgotten about it. For this, a whole one initialization system is suitable, which is in MacOS, namely launchd .

We quickly find the manual and we understand that it is not at all systemd and there is almost a scoop and xml . No beautiful configs for you, no commands like status , restart , daemon-reload . Only hardcore start-stop , list-grep , unload-load and many more oddities. Overcoming all this we write plist , we load. Does not work. We study the debugging of the demon, debazhim, we understand that even the PATH did not deliver normal to the ENV , we swear, we import (by adding /sbin and /usr/local/bin ) and finally enjoy autostart and stable operation.

Exhale

What is the result? Adventure Week, a knee-type zoo of services that is dear to the heart and does what is required of it. A bit of knowledge in dubious technical areas, a drop of open source and a smile on his face from the thought "I did it!"

PS: this is not a call for a boycott of capitalists, for saving on matches or for total cunning, but just an indication of the possibilities of research and development where they, in general, do not expect them.