We download videos from YouTube
This story originates in the distant 2006 year ... I was then at the beginning of my programming career and worked in one govnokontore developing nouneym company. Then we had several orders for sites with similar functionality: it was necessary to make a portal to which people could upload video / pictures / music and then view this content, comment, and at the same time watch ads bringing profit to a good uncle to the owner. Everything is good, but to lure people to the site you need to have a lot of content there already. Thus it would give the impression that the site has been working for a long time and successfully. And the customer sets a grand goal: to fill the newly created portal with youtube.com video. Not knowing what was waiting for me, I gladly set to work ...
All of course it was not so easy. Surely you know that YouTube never gave direct links to your files. Those links that can be copied from the page with the player will not give the desired result. In the best case, you will be confronted with the fact that, having finally received the desired "direct" link to the file, you will receive the 403rd error (no access rights to the file). Then, in 2006, I stopped at this after several days of searching and googling. By the way, then youtube was not yet the acquisition of a great and terribly good Google (thanks to anmipo for the correction). I decided that the possibility of downloading videos from YouTube simply does not exist. As it turned out, I was wrong ...
Several years have passed ... I returned to the topic of video downloading quite accidentally already in 2012. I came across one site where you could convert video files from one format to another online and there you could also download videos from YouTube simply by submitting a link to this video. An old topic has surfaced in my memory. By the way, I was just studying the Android platform. I decided to write an application for android, which would allow the user to download a favorite video to your phone.
In the end, I began to study again and try to understand what I was doing wrong the first time ...
I will not give here the source code of the application because it will be too boring to read. If someone wants to know the details of the implementation, you can write to me directly.
')
So, let's get down to business and consider the process of downloading videos from YouTube step by step.
A link to a video, such as www.youtube.com/watch?v=JCYIwiT0X98&feature=g-all-esi&context=G265c44fFAAAAAAAAHAA
Need to get meta-information about the video.
This is done by a request to www.youtube.com/get_video_info?video_id=JCYIwiT0X98
Video ID is taken from the source data, this is the parameter v = JCYIwiT0X98
As a result, we get a text stream of approximately the following type:
dl.dropbox.com/u/26865519/get_video_info_example
Get links to download videos.
The links to the video are contained in the url_encoded_fmt_stream_map parameter. The format of this parameter is as follows:
url_encoded_fmt_stream_map = Url encoded (url = url 1, url = url 2, ..., url = url n)
That is, other URLs are encoded in the URL, for this purpose it was necessary to zenkodit. If we decode our links, we get the following:
These are all URLs of our video in various formats. The itag parameter is responsible for the format. Here are the possible values for this parameter:
It would seem that we are already close to downloading, just enough to take the necessary URL and that's it. But no ... You need tojump a little more with a tambourine to work.
For example, take the first URL and decode it:
If we now insert it into the browser string, we get our favorite 403rd error. In order to get a valid link to the stream, you need to discard all parameters after id and add "& title = Our title" to the end, that is, in our case we get:
Here it is - a welcome link to the stream! :-) By the way, the title for the last parameter is taken from step 1, the most recent parameter from the meta information.
And finally, I give a link to an application for Android that demonstrates the theory shown here in practice:
market.android.com/details?id=com.youtubedownloader
UPD: Corrected the place where it is written about the "creation of Google"
All of course it was not so easy. Surely you know that YouTube never gave direct links to your files. Those links that can be copied from the page with the player will not give the desired result. In the best case, you will be confronted with the fact that, having finally received the desired "direct" link to the file, you will receive the 403rd error (no access rights to the file). Then, in 2006, I stopped at this after several days of searching and googling. By the way, then youtube was not yet the acquisition of a great and terribly good Google (thanks to anmipo for the correction). I decided that the possibility of downloading videos from YouTube simply does not exist. As it turned out, I was wrong ...
Several years have passed ... I returned to the topic of video downloading quite accidentally already in 2012. I came across one site where you could convert video files from one format to another online and there you could also download videos from YouTube simply by submitting a link to this video. An old topic has surfaced in my memory. By the way, I was just studying the Android platform. I decided to write an application for android, which would allow the user to download a favorite video to your phone.
In the end, I began to study again and try to understand what I was doing wrong the first time ...
I will not give here the source code of the application because it will be too boring to read. If someone wants to know the details of the implementation, you can write to me directly.
')
So, let's get down to business and consider the process of downloading videos from YouTube step by step.
Initial data
A link to a video, such as www.youtube.com/watch?v=JCYIwiT0X98&feature=g-all-esi&context=G265c44fFAAAAAAAAHAA
First step
Need to get meta-information about the video.
This is done by a request to www.youtube.com/get_video_info?video_id=JCYIwiT0X98
Video ID is taken from the source data, this is the parameter v = JCYIwiT0X98
As a result, we get a text stream of approximately the following type:
dl.dropbox.com/u/26865519/get_video_info_example
Second step
Get links to download videos.
The links to the video are contained in the url_encoded_fmt_stream_map parameter. The format of this parameter is as follows:
url_encoded_fmt_stream_map = Url encoded (url = url 1, url = url 2, ..., url = url n)
That is, other URLs are encoded in the URL, for this purpose it was necessary to zenkodit. If we decode our links, we get the following:
url = http% 3A% 2F% 2Fo-o.preferred.uarnet-kbp1.v11.lscache8.c.youtube.com% 2Fvideoplayback% 3Fsparams% 3Did% 252Cexpire% 252Cip% 252Cipbits% 252Citag% 252Csource% 252Crateby%%% 252Cipbits% 252Citag% 252Csource% 252Cipag%% 252Cipbits% 252Citag% 252Cippits% 252Cipag%% 2FV% 3D916601% 252C900203% 26itag% 3D45% 26ip% 3D91.0.0.0% 26signature% 3D84EA8CE08129B8AAF41D1F7898420BED2E2C3438.57CDB11E07D00A54C99F905DE0DA77AEBC0E9038% 26sver% 3D3% 26ratebypass% 3Dyes% 26source% 3Dyoutube% 26expire% 3D1328198927% 26key% 3Dyt1% 26ipbits% 3D8% 26cp% 3DU0hRTVFUVl9HUUNOMV9JTlpKOldSRTY2Tk9hSmU5% 26id% 3D242608c224f45fdf & quality = hd720 & fallback_host = tc.v11.cache8.c.youtube.com & type = video% 2Fwebm% 3B + codecs% 3D% 22vp8.0% 2C + vorbis% 22 & itag = 45, url = http% 3A% 2F% 2Fo- o.preferred.uarnet-kbp1.v15.lscache1.c.youtube.com% 2Fvideoplayback% 3pcfparams% 3Did% 252Cexpire% 252Cip% 252Cipbits% 252Citag% 252Csource% 252Cratebypass% 252Ccp% 26fexp% 3D966p6pcpc%% 252Cratebypass% 252Ccp% 26fexp% 3D9666p6p6pcits% 252Cratebypass% 252Ccpp% 26fexp% 3D966606p6p4p4p6p4p6pcp%% 252Crateagpass% 252Ccpp% 252Crateagpass% 252Cs% 3D91.0.0.0% 26signature% 3D1AC166895886593943D1346017A82762AFF47E3C.59C7B7E9030474710D768847DD0433DE42D0A19% 26sver% 3D3% 26ratebypass% 3Dyes% 26source% 3Dyoutube% 26expire% 3Ds% % Ancrontexa.com angelsa.com ancestors.com ancestorstex.com % 22 & itag = 22, url = http% 3A% 2F% 2Fo-o.preferred.uarnet-kbp1.v21.lscache1.c.youtube.com% 2Fvideoplayback% 3Fsparams% 3Did% 252Cexpire% 252Cip% 252Cipbits% 252Citag% 252Csource% % 252Ccp% 26fexp% 3D916601% 252C900203% 26itag% 3D44% 26ip% 3D91.0.0.0% 26signature% 3D0E18ABF397609CD176F02A026E6827BF89EDEEC1.AF7A313EFC85510A908760E20F3B88A69F9BC871% 26sver% 3D3% 26ratebypass% 3Dyes% 26source% 3Dyoutube% 26expire% 3D1328198927% 26key% 3Dyt1% 26ipbits% 3D8 % 26cp% 3DU0hRTVFUVl9HUUNOMV9JTlpKOldSRTY2pcpcpcpcpcpcfcfcfcfgfgfhhhhhhhhfgfgfhhhhhhhhhhhhhhhhhhtgtglhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh as Hellftyhhhhhhhhfhhhhhhhhhhhhhhhhhhhh ...% asus_ehhhhhhhhhhhhhhhhhhhhhhhh ...% asleep% 3d224pcp% % 2F% 2Fo-o.preferred.uarnet-kbp1.v8.lscache6.c.youtube.com% 2Fvideoplayback% 3Fsparams% 3Did% 252Cexpire% 252Cip% 252Cipbits% 252Citag% 252Csource% 252Calgorithm% 252Cburs t% 252Cfactor% 252Ccp% 26fexp% 3D916601% 252C900203% 26algorithm% 3Dthrottle-factor% 26itag% 3D35% 26ip% 3D91.0.0.0% 26burst% 3D40% 26sver% 3D3% 26signature% 3D3946508038978141EA2FE1E8A691C237497B20A8.5C4F1CAA8BFF22B061A304281162831BBDF8F3C1% 26source% 3Dyoutube% 26expire% 3D1328198927% 26key% 3Dyt1% 26ipbits% 3D8% 26factor% 3D1.25% 26cp development 3DU0hRTVFUVl9HUUNOMV9JTlpKOldSRTY2pk9hSmU5% 26id% 3D242608c224f45fdf & quality = wide & fallback_pahp4p2ldcp2dgt2dkt2pcd2dcp2ldp4ld2pcd2dcp2ldp4ld2pcd2dcp2dcp2dcp2dcp2dcp2dcp2dcd2dcp2dcd2dcp2dcp2dcp2dcp2dcp2dcp2dcp2dcp2dcp2dcd2gc 3A% 2F% 2Fo-o.preferred.uarnet-kbp1.v22.lscache3.c.youtube.com% 2Fvideoplayback% 3Fsparams% 3Did% 252Cexpire% 252Cip% 252Cipbits% 252Citag% 252Csource% 252Cratebypass% 252Ccp% te red 00 9 9 9 9 9 9 9 26itag% 3D43% 26ip% 3D91.0.0.0% 26signature% 3D25CE0D32BEF98C0B631656DCBCE204F44DB0678F.810337E9249E7AF4B5D06592FC3664AE81A054C7% 26sver% 3D3% 26ratebypass% 3Dyes% 26source% 3Dyoutube% 26expire% 3D1328198927% 26key% 3Dyt1% 26ipbits% 3D8% 26cp% 3DU0hRTVFUVl9HUUNOMV9JTlpKOldSRTY2Tk9hSmU5% 26id% 3D242608c224f45fdf & quality = medium & fallback_host = tc.v22.cache3.c.you tube.com & type = video% 2Fwebm% 3B + codecs% 3D% 22vp8.0% 2C + vorbis% 22 & itag = 43, url = http% 3A% 2F% 2Fo-o.preferred.uarnet-kbp1.v7.lscache6.c. youtube.com% 2Fvideoplayback% 3Fsparams% 3Did% 252Cexpire% 252Cip% Mmv eajl ngk naug qa qaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa al i ali eksi alekti% 252Calagám% 252 fact fact 3D 3D 3D fact fact fact% 3D% 9% 0% 26burst% 3D40% 26sver% 3D3% 26signature% 3D639B6C6CE89DA078D0F236F2E6EF25D6699E92B3.8E643652167A642565D7CF46356702AB4240B039% 26source% 3Dyoutube% 26expire% 3D1328198927% 26key% 3Dyt1% 26ipbits% 3D8% 26factor% 3D1.25% 26cp% 3DU0hRTVFUVl9HUUNOMV9JTlpKOldSRTY2Tk9hSmU5% 26id% 3D242608c224f45fdf & quality = medium & fallback_host = tc. v7.cache6.c.youtube.com & type = video% 2Fx-flv & itag = 34, url = http% 3A% 2F% 2Fo-o.preferred.uarnet-kbp1.v3.lscache3.c.youtube.com% 2Fvideoplayback% 3Fsparams% 3Did% 252Cexpire% 252Cip% 252Cipbits% 252Citag% 252Csource% 252Cratebypass% 252Ccp% 26fexp% 3D916601% 252C900203% 26itag% 3D18% 26ip% 3D91.0.0.0% of the accountant accountant accountant 1A918357D275F04CDDE1AFF% 26sver% 3D3% 26ratebypass% 3Dyes% 26source% 3Dyoutube% 26expire% 3D1328198927% 26key% 3Dyt1% 26ipbits% 3D8% 26cp% 3DU0hRTVFUVl9HUUNOMV9JTlpKOldSRTY2Tk9hSmU5% 26id% 3D242608c224f45fdf & quality = medium & fallback_host = tc.v3.cache3.c.youtube.com & type = video% 2Fmp4% 3B + codecs% 3D% 22avc1.42001E% 2C + mp4a.40.2% 22 & itag = 18, url = http% 3A% 2F% 2Fo-o.preferred.uarnet-kbp1.v23.lscache1.c.youtube.com% 2Fvideoplayback% 3Fsparams% 3Did% 252Cexpire% 252Cip% 252Cipbits chart% 252Citag% 252Csource% 252Calgorithm% 252Cpchpchp% 252Cfactor% 252Ccpp% 26fexp% 3D916601% bpchpchp% 25fexp% 3D916601% 3D40% 26sver% 3D3% 26signature% 3D8CED45631A15BD0E90EDB77AEFEEE2C421F122F1.9C1BC70FB6A2ED45D3491A5A082A96F029A228CB% 26source% 3Dyoutube% 26expire% 3D1328198927% 26key% 3Dyt1% 26ipbits% 3D8% 26factor% 3D1.25% 26cp% 3DU0hRTVFUVl9HUUNOMV9JTlpKOldSRTY2Tk9hSmU5% 26id% 3D242608c224f45fdf & quality = small & fallback_host = tc.v23.cache1. c.youtube.com & type = video% 2Fx-flv & itag = 5 &
These are all URLs of our video in various formats. The itag parameter is responsible for the format. Here are the possible values for this parameter:
itag = 5 - FLV 320 x 240 itag = 34 - FLV 640 x 360 itag = 35 - FLV 854 x 480 itag = 18 - MP4 640 x 360 itag = 22 - MP4 1280 x 720 itag = 37 - MP4 1920 x 1080 itag = 38 - MP4 4096 x 1714 itag = 43 - WEBM 640 x 360 itag = 44 - WEBM 854 x 480 itag = 45 - WEBM 1280 x 720
It would seem that we are already close to downloading, just enough to take the necessary URL and that's it. But no ... You need to
For example, take the first URL and decode it:
http://oo.preferred.uarnet-kbp1.v15.lscache1.c.youtube.com/videoplayback?sparams=id%2Cexpire%2Cip%2Cipbits%2Citag%2Csource%2Cratebypass % 2Ccp & fexp = 916,601% 2C900203 & itag = 22, & ip = 91.0.0.0 & signature = 1AC166895886593943D1346017A82762AFF47E9C.59C7B7E9030474710D768847DDD0433DE42D0A19 & sver = 3 & ratebypass = yes & source = youtube & expire = 1328198927 & key = yt1 & ipbits = 8 & cp = U0hRTVFUVl9HUUNOMV9JTlpKOldSRTY2Tk9hSmU5 & id = 242608c224f45fdf & quality = hd720 & fallback_host = tc.v15.cache1.c.youtube.com & type = video / mp4; codecs = "avc1.64001F, mp4a.40.2" & itag = 22
If we now insert it into the browser string, we get our favorite 403rd error. In order to get a valid link to the stream, you need to discard all parameters after id and add "& title = Our title" to the end, that is, in our case we get:
http://oo.preferred.uarnet-kbp1.v15.lscache1.c.youtube.com/videoplayback?sparams=id%2Cexpire%2Cip%2Cipbits%2Citag%2Csource%2Cratebypass % 2Ccp & fexp = 916,601% 2C900203 & itag = 22, & ip = 91.0.0.0 & signature = 1AC166895886593943D1346017A82762AFF47E9C.59C7B7E9030474710D768847DDD0433DE42D0A19 & sver = 3 & ratebypass = yes & source = youtube & expire = 1328198927 & key = yt1 & ipbits = 8 & cp = U0hRTVFUVl9HUUNOMV9JTlpKOldSRTY2Tk9hSmU5 & id = 242608c224f45fdf & title =% D0% 9F% D1% 8B% D0% BB% D1% 8C + % D0% BD% D0% B0 +% D0% 91% D0% B8% D0% B1% D0% BB% D0% B8% D0% B8 ++% D1% 80% D0% B6% D0% B0% D0% B2% D1 % 87% D0% B8% D0% BD% D0% B0 +% D0% BD% D0% B0 +% D1% 81% D0% B5% D1% 80% D0% B4% D1% 86% D0% B5
Here it is - a welcome link to the stream! :-) By the way, the title for the last parameter is taken from step 1, the most recent parameter from the meta information.
And finally, I give a link to an application for Android that demonstrates the theory shown here in practice:
market.android.com/details?id=com.youtubedownloader
Sources
- www.google.com
- Browser plugin code for downloading videos from YouTube: userscripts.org/scripts/review/25105
- meyerweb.com/eric/tools/dencoder - a useful tool for encoding / decoding urls online
UPD: Corrected the place where it is written about the "creation of Google"
Source: https://habr.com/ru/post/137496/
All Articles