Geekly Articles each Day
📜 ⬆️ ⬇️

Accelerate Drupal: Pressflow + Nginx + Varnish

This article shows in some detail how you can switch to the development of Drupal websites with a serious stress tolerance and the ability to handle large traffic.

This is my first experience with a similar setup, but as will be seen further from the statistics, it copes quite well with its main task - accelerating the site’s work. It will be interesting to hear and see the settings, additional materials from all who faced similar tasks, as there is still little free and high-quality information on this topic regarding the Drupal system in runet.

For a long time, I used a bunch of Drupal + Nginx with the default server settings for development:
')
server {
listen 62.xxx.xx.xx:80;
server_name mysite.com www.mysite.com;
rewrite>^(/manager/.*)$>https://$host$1>permanent;
location ~* ^/(webstat/|awstats|webmail/|myadmin/|manimg/) {
proxy_pass 62.xxx.xx.xx:8080;
proxy_redirect mysite.com:8080/ /;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Real-IP $remote_addr;
}
location / {
proxy_pass mysite.com:8080;
proxy_redirect mysite.com:8080/ /;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Real-IP $remote_addr;
}
location ~* ^.+\.(jpg|jpeg|gif|png|svg|js|css|mp3|ogg|mpe?g|avi|zip|gz|bz2?|rar)$ {
root /home/pathto/drupal613;
access_log /home/httpd-logs/mysite.com.access.log;
error_page 404 = @fallback;
}
location @fallback {
proxy_pass 62.xxx.xx.xx:8080;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Real-IP $remote_addr;
}
}



The essence of these settings is simple - unload the Apache and leave only php processes on it, and throw all the statics (pictures, files) on Nginx. My apache is configured for port 8080, and Nginx for 80. Nginx on the frontend analyzes what it requests and throws out static data itself or throws everything on the Apache. This approach offloads the server and slightly improves performance.

To analyze the site on Drupal, I used the excellent service LoadImpact.com , which threw me this report:

image

Not long to analyze, to understand the more traffic on the site, the more you have to wait for a response from the server. This is bad. I found the solution in the following bundle:

Pressflow + Nginx + Varnish



Pressflow is a Drupal distribution with integrated improvements in performance, extensibility, efficiency, and testing.

Each Pressflow version is an API equivalent to the same Drupal version. For example, Pressflow 6 is compatible with all Drupal 6 modules. Pressflow 6 also has an integrated SimpleTest system from Drupal 7 and a patch to support CDN.


Here are the links for Pressflow downloads:
fourkitchens.com/pressflow-makes-drupal-scale/downloads
launchpad.net/pressflow/+download

Varnish is the key to speeding up your site.

Varnish is open source software that is standardized and requires little resources.


Download link Varnish:
www.varnish-cache.org/releases

The idea of ​​this bundle is not much more complicated: on the request to the server, we throw out the statics through Nginx, otherwise we follow the cached data to Varnish + Pressflow, and to Apache if we need something from PHP.

Step 1: Install Pressflow

Installing Pressflow in general is no different from installing Drupal :)
fourkitchens.com/pressflow-makes-drupal-scale/installation

Step 2: Customize Pressflow

Go to mysite.com/admin/settings/performance and enable caching. That's all.

Step 3: Install Varnish

If you are far from the concepts of SSH and Linux, try creating a request to your hosters to install Varnish. I used this manual under FreeBSD www.varnish-cache.org/installation/freebsd . Everything is quite simple.

Step 4: Customize Varnish

This is one of the most difficult points for setting up the entire system:

1. Add the launch of the varnish deamon and varnish log to the /etc/rc.conf configuration file:
varnishd_enable = "YES"
varnishlog_enable = "YES"
2. Change default settings for varnish deamon:
${varnishd_enable:="NO"}
${varnishd_pidfile:="/var/run/${name}.pid"}
${varnishd_listen:=":8081"}
${varnishd_admin:="localhost:8090"}
${varnishd_backend:="localhost:8080"}
${varnishd_config:="/usr/local/etc/varnish/default.vcl"}
${varnishd_storage:="file,/usr/local/varnish.cache,1G"}
${varnishd_hash:="classic,16383"}
${varnishd_user:="www"}
${varnishd_group:="www"}
As mentioned earlier, on my server Nginx listens to port 80, and Apache 8080. Varnish will also wiretap port 8081, and from it again we will go to Apache, that is, port 8080.

Here:
-varnishd_config - path to default settings file for Varnish behavior
-varnishd_storage - size and path to the cache file

3. Configure Varnish to work with Pressflow:
backend default {
.host = "127.0.0.1";
.port = "8080";
.connect_timeout = 600s;
.first_byte_timeout = 600s;
.between_bytes_timeout = 600s;
}

sub vcl_recv {
if (req.request != "GET" &&
req.request != "HEAD" &&
req.request != "PUT" &&
req.request != "POST" &&
req.request != "TRACE" &&
req.request != "OPTIONS" &&
req.request != "DELETE") {
/* Non-RFC2616 or CONNECT which is weird. */
return (pipe);
}

if (req.request != "GET" && req.request != "HEAD") {
/* We only deal with GET and HEAD by default */
return (pass);
}

// Remove has_js and Google Analytics cookies.
set req.http.Cookie = regsuball(req.http.Cookie, "(^|;\s*)(__[az]+)=[^;]*", "");

// To users: if you have additional cookies being set by your system (eg
// from a javascript analytics file or similar) you will need to add VCL
// at this point to strip these cookies from the req object, otherwise
// Varnish will not cache the response. This is safe for cookies that your
// backed (Drupal) doesn't process.
//
// Again, the common example is an analytics or other Javascript add-on.
// You should do this here, before the other cookie stuff, or by adding
// to the regular-expression above.

// Remove a ";" prefix, if present.
set req.http.Cookie = regsub(req.http.Cookie, "^;\s*", "");
// Remove empty cookies.
if (req.http.Cookie ~ "^\s*$") {
unset req.http.Cookie;
}

if (req.http.Authorization || req.http.Cookie) {
/* Not cacheable by default */
return (pass);
}

// Skip the Varnish cache for install, update, and cron
if (req.url ~ "install\.php|update\.php|cron\.php") {
return (pass);
}

// Normalize the Accept-Encoding header
// as per: varnish-cache.org/wiki/FAQ/Compression
if (req.http.Accept-Encoding) {
if (req.url ~ "\.(jpg|png|gif|gz|tgz|bz2|tbz|mp3|ogg)$") {
# No point in compressing these
remove req.http.Accept-Encoding;
}
elsif (req.http.Accept-Encoding ~ "gzip") {
set req.http.Accept-Encoding = "gzip";
}
else {
# Unknown or deflate algorithm
remove req.http.Accept-Encoding;
}
}

// Let's have a little grace
set req.grace = 30s;

return (lookup);
}

sub vcl_hash {
if (req.http.Cookie) {
set req.hash += req.http.Cookie;
}
}

// Strip any cookies before an image/js/css is inserted into cache.
sub vcl_fetch {
if (req.url ~ "\.(png|gif|jpg|swf|css|js)$") {
// This is for Varnish 2.0; replace obj with beresp if you're running
// Varnish 2.1 or later.
unset beresp.http.set-cookie;
}
}

sub vcl_error {
// Let's deliver a friendlier error page.
// You can customize this as you wish.
set obj.http.Content-Type = "text/html; charset=utf-8";
synthetic {"
<?xml version="1.0" encoding="utf-8"?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
"} obj.status " " obj.response {"
< type="text/css">
#page {width: 400px; padding: 10px; margin: 20px auto; border: 1px solid black; background-color: #FFF;}
p {margin-left:20px;}
body {background-color: #DDD; margin: auto;}
</>

<1>Page Could Not Be Loaded</1>
We're very sorry, but the page could not be loaded properly. This should be fixed very soon, and we apologize for any inconvenience.
< /> <4>Debug Info:</4>
<>
Status: "} obj.status {"
Response: "} obj.response {"
XID: "} req.xid {"
</>
<>< href="http://www.varnish-cache.org/">Varnish</></>



"};
return (deliver);
}
3. Configure Ngnix to work with Varnish:
server {
listen 62.xxx.xx.xx:80;
server_name mysite.com www.mysite.com;
rewrite ^(/manager/.*)$>https://$host$1>permanent;
rewrite>^(/manager/.*)$>https://$host$1>permanent;
location ~* ^/(webstat/|awstats|webmail/|myadmin/|manimg/) {
proxy_pass 62.xxx.xx.xx:8080;
proxy_redirect mysite.com:8080/ /;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Real-IP $remote_addr;
}
location / {
proxy_pass 62.xxx.xx.xx:8081;
proxy_redirect mysite.com:8081/ /;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Real-IP $remote_addr;
}
location ~* ^.+\.(jpg|jpeg|gif|png|svg|js|css|mp3|ogg|mpe?g|avi|zip|gz|bz2?|rar)$ {
root /home/cross/data/www/mysite.com;
access_log /home/httpd-logs/mysite.com.access.log;
error_page 404 = @fallback;
}
location @fallback {
proxy_pass 62.xxx.xx.xx:8080;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Real-IP $remote_addr;
}
}
Step 5: Rebooting the server

Now you need to make sure that Varnish is running - try running the 'top' command on the terminal and check that the varnish process is running.
Make sure Varnish + Pressflow works - run the varnishlog command under SSH and try to open your website mysite.com , the varnishlog should show the HTTP headers after that, this will mean that everything works fine.

After all these settings, I once again with loadimpact drove the test and got:

image

Now the situation is much better, the delay from the server is stable and with increasing traffic does not bring down the server.

These simple settings will launch Pressflow + Nginx + Varnish on your server to speed up Drupal.

Also, the way things turned out, Nginx has special settings for Drupal - wiki.nginx.org/Drupal .

Additional links:

Source: https://habr.com/ru/post/107094/

More articles:


All Articles