"Transparent" Squid with HTTPS filtering resources without certificate substitution (x86, x64 - universal instruction)
Hello! The last article about transparent HTTPS proxying using Squid was quite successful. A lot of feedback came in the mail about the successful installation of this system. But there were also letters asking for help. The problems were quite solvable. But not so long ago, a colleague approached me with a request for assistance in installing this system on x64 architecture (Debian). Here we are puzzled. Firstly, it turned out that the previous article was unsuitable for this due to the lack of necessary source codes in the Debian repository (there is now 3.5.10 there). It was not possible to find the Debian sources needed in the first article, and checkinstall produced strange errors. Secondly, I wanted a more universal solution that would work without problems on x64, x86, and (if possible) on other distributions. A solution was found. It turned out a small addition to the previous article + some clarifications. This instruction allows you to compile both x86 and x64 versions of Squid and create the appropriate packages. The instruction will be divided into several points and sub-items. If interested, go under the cat:
We go in order.
one)
a) To get started, prepare for the assembly of packages:
Do not forget to go to the folder where you will collect the source so as not to rid yourself of Home.
b) Next, download the libressl:
c) And now we collect:
2) Now you can install libressl:
After installation, you must configure the use of LibreSSL by default:
Check to see if Libressl was installed:
If the console exhaust is similar, then everything worked out. Go ahead.
3) At the queue Libecap.
a) You need to edit sources.list, including the source code from the testing branch (this is necessary, since we need to compile a new libecap, which in turn is required to build Squid):
Update the package cache:
And now we download the necessary sources from Testing:
Next, collect libecap:
b) Remove the old one and install the new one:
')
4) Sked to the compilation of Squid itself.
a) Download thelatest and most efficient Squid snapshot:
Unpack:
b) Download patch for bio.cc, and patch:
5) And this stage is one of the most responsible. It is necessary to configure Squid with the necessary options. In the previous article, the debian / rules file was used, but we will compile Squid in this instruction using make, and we will create packages using checkinstall. Therefore, there will be more options. And here are some:
Be extremely attentive. We are more interested, as in the previous article, three options: --enable-ssl, --enable-ssl-crtd, --with-openssl. The remaining options can be changed in accordance with your preferences (if you want to change them, be sure to read the documentation on configuration).
6) Now we got to compiling.
a) Compile.
b) Ambiguous stage. It is necessary to create the / usr / share / squid / and / usr / share / squid / icons directories, otherwise the following stage will not be executed due to the absence of these folders (why checkinstall does not create them, I did not understand, unfortunately):
c) And now we create installation packages:
7) We are approaching the final. Install Squid:
8) Try running squid:
And we see a big FIG! Wow ... We try in the old-fashioned way:
And we also see a big FIG. Why? Because checkinstall did not include the Squid service files in the package. No problem. Create the necessary systemd service yourself:
With the following content:
Enable the created service
9) Yes, you are right, that's not all. Since we compiled completely original sources (with the exception of the bio.cc patch), we have installed the configuration files like squid.conf.default, mime.conf.default, etc. Of course, Squid did not hear about them. Rename them to Squid's readable view:
10) And that's not all =) You must manually create a folder for Squid logs and assign the appropriate rights to it:
11) And here it is - the final stage. Start Squid'a and check the status of the service!
If the console's exhaust looks similar, or rather there are no errors in it and the line “Active: active (running)” is sure to be present, then you have successfully installed Squid with support for HTTPS transparent proxying! Congratulations!
If you don't want to compile anything, you can download an archive with ready-made deb packages (x64 version!). If you install from ready-made packages, then you will need steps: 2, 3 (b), 7, 8, 9, 10, 11 .
I also want to note that checkinstall allows you to create rpm packages, and you can use this. The only thing you need to build all the packages with the help of checkinstall, but I think there will be no problems with this, since the main and most difficult thing is already assembled by checkinstall.
Squid's configuration file with the right directives, job description, etc. read in the previous article !
UPD 12/11/15: At the request of the user nikitasius checked Cloudflare. Comrade nikitasius wrote in the comments:
UPD 12/14/15: I want to share great news with my colleagues! I found a way to make new versions of Squid work without special dances with a tambourine! It is necessary that the clients and Squid settings have the same DNS! In my case, the Bind is spinning on the squid gateway. He appointed his clients, and Kalmar, with a directive:
UPD 12/16/15: Centos packages!
UPD 04/25/16: A request for those who have tested the x64 build write whether everything works. There are reports of glitches, in particular, a heavy load on the CPU after some time has passed and a subsequent security. I myself can not check yet
UPD 04.05.18: wrote a new article where the problem of glitches of new versions of Squid + is solved in the logs beautiful domain names instead of ip addresses + RKN bypass blocking.
We go in order.
one)
a) To get started, prepare for the assembly of packages:
apt-get install git fakeroot checkinstall build-essential devscripts patch apt-cache policy squid3 apt-get build-dep squid3 apt-get build-dep libecap2 apt-get install libssl-dev libgnutls28-dev Do not forget to go to the folder where you will collect the source so as not to rid yourself of Home.
b) Next, download the libressl:
wget http://ftp.openbsd.org/pub/OpenBSD/LibreSSL/libressl-2.1.6.tar.gz tar -xzvf libressl-2.1.6.tar.gz cd libressl-2.1.6 c) And now we collect:
./configure make checkinstall --pkgname libressl --pkgversion 2.1.6 2) Now you can install libressl:
dpkg -i libressl_2.1.6-1_amd64.deb ldconfig After installation, you must configure the use of LibreSSL by default:
mv /usr/bin/openssl /usr/bin/openssl-1 update-alternatives --install /usr/bin/openssl openssl /usr/bin/openssl-1 10 update-alternatives --install /usr/bin/openssl openssl /usr/local/bin/openssl 50 update-alternatives --config openssl Check to see if Libressl was installed:
openssl version LibreSSL 2.1.6 If the console exhaust is similar, then everything worked out. Go ahead.
3) At the queue Libecap.
a) You need to edit sources.list, including the source code from the testing branch (this is necessary, since we need to compile a new libecap, which in turn is required to build Squid):
deb-src http://ftp.de.debian.org/debian/ testing main contrib non-free Update the package cache:
apt-get update And now we download the necessary sources from Testing:
apt-get source libecap3/testing Next, collect libecap:
cd libecap-1.0.1/ dpkg-buildpackage -us -uc -nc -d b) Remove the old one and install the new one:
apt-get purge libecap2 libecap3_1.0.1-2_amd64.deb libecap3-dev_1.0.1-2_amd64.deb ')
4) Sked to the compilation of Squid itself.
a) Download the
wget http://www.squid-cache.org/Versions/v3/3.5/squid-3.5.8.tar.gz Unpack:
tar -xf squid-3.5.8.tar.gz cd squid-3.5.8 b) Download patch for bio.cc, and patch:
wget -O bug-4330-put_cipher_by_char-t1.patch http://bugs.squid-cache.org/attachment.cgi?id=3216 patch -p0 -i bug-4330-put_cipher_by_char-t1.patch » patching file src/ssl/bio.cc 5) And this stage is one of the most responsible. It is necessary to configure Squid with the necessary options. In the previous article, the debian / rules file was used, but we will compile Squid in this instruction using make, and we will create packages using checkinstall. Therefore, there will be more options. And here are some:
./configure --build=x86_64-linux-gnu \ --prefix=/usr \ --includedir=${prefix}/include \ --mandir=${prefix}/share/man \ --infodir=${prefix}/share/info \ --sysconfdir=/etc \ --localstatedir=/var \ --libexecdir=${prefix}/lib/squid \ --srcdir=. \ --disable-maintainer-mode \ --disable-dependency-tracking \ --disable-silent-rules \ --datadir=/usr/share/squid \ --sysconfdir=/etc/squid \ --mandir=/usr/share/man \ --enable-inline \ --disable-arch-native \ --enable-async-io=8 \ --enable-storeio=ufs,aufs,diskd,rock \ --enable-removal-policies=lru,heap \ --enable-delay-pools \ --enable-cache-digests \ --enable-icap-client \ --enable-follow-x-forwarded-for \ --enable-auth-basic=DB,fake,getpwnam,LDAP,NCSA,NIS,PAM,POP3,RADIUS,SASL,SMB \ --enable-auth-digest=file,LDAP \ --enable-auth-negotiate=kerberos,wrapper \ --enable-auth-ntlm=fake,smb_lm \ --enable-external-acl-helpers=file_userip,kerberos_ldap_group,LDAP_group,session,SQL_session,unix_group,wbinfo_group \ --enable-url-rewrite-helpers=fake \ --enable-eui \ --enable-esi \ --enable-icmp \ --enable-zph-qos \ --enable-ecap \ --disable-translation \ --with-swapdir=/var/spool/squid \ --with-logdir=/var/log/squid \ --with-pidfile=/var/run/squid.pid \ --with-filedescriptors=65536 \ --with-large-files \ --with-default-user=proxy \ --enable-ssl \ --enable-ssl-crtd \ --with-openssl \ --enable-linux-netfilter \ 'CFLAGS=-g -O2 -fPIE -fstack-protector-strong -Wformat -Werror=format-security -Wall' \ 'LDFLAGS=-fPIE -pie -Wl,-z,relro -Wl,-z,now' \ 'CPPFLAGS=-D_FORTIFY_SOURCE=2' \ 'CXXFLAGS=-g -O2 -fPIE -fstack-protector-strong -Wformat -Werror=format-security' Be extremely attentive. We are more interested, as in the previous article, three options: --enable-ssl, --enable-ssl-crtd, --with-openssl. The remaining options can be changed in accordance with your preferences (if you want to change them, be sure to read the documentation on configuration).
6) Now we got to compiling.
a) Compile.
make b) Ambiguous stage. It is necessary to create the / usr / share / squid / and / usr / share / squid / icons directories, otherwise the following stage will not be executed due to the absence of these folders (why checkinstall does not create them, I did not understand, unfortunately):
mkdir -p /usr/share/squid/icons c) And now we create installation packages:
checkinstall --pkgname squid --pkgversion 3.5.8 7) We are approaching the final. Install Squid:
dpkg -i squid_3.5.8-1_amd64.deb Spoiler header
Yes, that's right, it turned out just one file, while in the previous article there were several of them, as is customary in Debian.
8) Try running squid:
systemctl start squid And we see a big FIG! Wow ... We try in the old-fashioned way:
service squid start And we also see a big FIG. Why? Because checkinstall did not include the Squid service files in the package. No problem. Create the necessary systemd service yourself:
touch /etc/systemd/system/squid.service nano /etc/systemd/system/squid.service With the following content:
## Copyright (C) 1996-2015 The Squid Software Foundation and contributors ## ## Squid software is distributed under GPLv2+ license and includes ## contributions from numerous individuals and organizations. ## Please see the COPYING and CONTRIBUTORS files for details. ## [Unit] Description=Squid Web Proxy Server After=network.target [Service] Type=simple ExecStart=/usr/sbin/squid -sYC -N ExecReload=/bin/kill -HUP $MAINPID KillMode=process [Install] WantedBy=multi-user.target Spoiler header
In fact, this very service is in the archive with the source code for Squid. For what reasons, Checkinstall did not include it in the package, it is not known.
Enable the created service
systemctl enable squid 9) Yes, you are right, that's not all. Since we compiled completely original sources (with the exception of the bio.cc patch), we have installed the configuration files like squid.conf.default, mime.conf.default, etc. Of course, Squid did not hear about them. Rename them to Squid's readable view:
cp /etc/squid/squid.conf.default /etc/squid/squid.conf cp /etc/squid/mime.conf.default /etc/squid/mime.conf cp /etc/squid/cachemgr.conf.default /etc/squid/cachemgr.conf cp /etc/squid/errorpage.css.default /etc/squid/errorpage.css 10) And that's not all =) You must manually create a folder for Squid logs and assign the appropriate rights to it:
mkdir /var/log/squid chown proxy /var/log/squid 11) And here it is - the final stage. Start Squid'a and check the status of the service!
systemctl start squid systemctl status -l squid ● squid.service - Squid Web Proxy Server Loaded: loaded (/etc/systemd/system/squid.service; enabled) Active: active (running) since 2015-12-04 23:32:04 YEKT; 2min 41s ago Main PID: 590 (squid) CGroup: /system.slice/squid.service ├─590 /usr/sbin/squid -sYC -N └─591 (logfile-daemon) /var/log/squid/access.log 04 23:32:04 squidX64 squid[590]: Max Swap size: 0 KB 04 23:32:04 squidX64 squid[590]: Using Least Load store dir selection 04 23:32:04 squidX64 squid[590]: Current Directory is / 04 23:32:04 squidX64 squid[590]: Finished loading MIME types and icons. 04 23:32:04 squidX64 squid[590]: HTCP Disabled. 04 23:32:04 squidX64 squid[590]: Pinger socket opened on FD 16 04 23:32:04 squidX64 squid[590]: Squid plugin modules loaded: 0 04 23:32:04 squidX64 squid[590]: Adaptation support is off. 04 23:32:04 squidX64 squid[590]: Accepting HTTP Socket connections at local=[::]:3128 remote=[::] FD 14 flags=9 04 23:32:05 squidX64 squid[590]: storeLateRelease: released 0 objects If the console's exhaust looks similar, or rather there are no errors in it and the line “Active: active (running)” is sure to be present, then you have successfully installed Squid with support for HTTPS transparent proxying! Congratulations!
If you don't want to compile anything, you can download an archive with ready-made deb packages (x64 version!). If you install from ready-made packages, then you will need steps: 2, 3 (b), 7, 8, 9, 10, 11 .
I also want to note that checkinstall allows you to create rpm packages, and you can use this. The only thing you need to build all the packages with the help of checkinstall, but I think there will be no problems with this, since the main and most difficult thing is already assembled by checkinstall.
Squid's configuration file with the right directives, job description, etc. read in the previous article !
Thanks to Tatiana Illarionova and the Squid developers for helping to create this recipe!
UPD 12/11/15: At the request of the user nikitasius checked Cloudflare. Comrade nikitasius wrote in the comments:
Apparently for sites that for Cloudflare such a system will not work correctly ...So, checked. I added one of his domains from Cloudflare to the blacklist of HTTPS blocking, the browser does not enter it, but the browser quietly enters other domains that are registered in the certificate. So, Cloudflare check passed
They usually have a pack of domains in certificates.
If example.com is in the same certificate with freepron.cum, the squid will work on freepron, if the latter is in the local bath, right?
UPD 12/14/15: I want to share great news with my colleagues! I found a way to make new versions of Squid work without special dances with a tambourine! It is necessary that the clients and Squid settings have the same DNS! In my case, the Bind is spinning on the squid gateway. He appointed his clients, and Kalmar, with a directive:
dns_nameservers 127.0.0.1 . After that, everything worked successfully. Tested on Squid 4.0.3, compiled WITHOUT Libressl!UPD 12/16/15: Centos packages!
UPD 04/25/16: A request for those who have tested the x64 build write whether everything works. There are reports of glitches, in particular, a heavy load on the CPU after some time has passed and a subsequent security. I myself can not check yet
UPD 04.05.18: wrote a new article where the problem of glitches of new versions of Squid + is solved in the logs beautiful domain names instead of ip addresses + RKN bypass blocking.
For those who think that this is MITM =)
In the survey, the last item is of course comic =) If you CAREFULLY read the previous article, then it will immediately become clear that there is no attack on the user, since there is no substitution of certificates !!!
Source: https://habr.com/ru/post/272733/
All Articles