"My Website Crashed During the Holidays: How I Fixed Magento 2.3.4 Overload on a VDS"
Listen, the situation is classic to the point of teeth-grinding. The website was working, working, and then — BAM. Holiday sales, marketers gleefully rubbing their hands, expecting a flood of customers, and the server collapses from the load. This happened to me on a VDS with 39 GB of RAM and 12 cores. It seemed like more than enough resources, but the php-fpm process, in conjunction with MySQL, was devouring all the memory, and the site was crashing.
I almost tore my hair out then. I restarted services, added 20 GB of swap — it helped for an hour, then back to square one. I had to dive into the settings and figure out what was wrong.
In this article, I'll share how I fixed it all. No fluff, just what actually worked.
Where It All Began
The server seemed decent:
Yes, the versions aren't the newest, I know. But Magento 2.3.4 only works properly with PHP 7.2, and an upgrade would have entailed upgrading the entire CMS, which I didn't have time for.
Before the holiday sales, the system lived peacefully. But when the marketers turned on the advertising and users started pouring in, hell broke loose. php-fpm in tandem with MySQL was consuming all the RAM, and the site was going down. Restarts helped for a couple of hours, then the same thing all over again. I added a 20 GB swap file — same story.
I had to recall how to configure nginx and php-fpm.
Where to Find Configs
Before changing anything, you need to understand where to change it. I sometimes get confused myself, so let me lay it out clearly.
Nginx
Nginx configs are located in /etc/nginx. The main file is nginx.conf, but usually, site settings are moved to separate files in /etc/nginx/conf.d/. There, each site has its own .conf file.
Another important point: check the Include directive in your configs. My previous admin added include /var/www/www/nginx.conf, which allowed changing settings directly from the site directory, similar to .htaccess in Apache. If you have something like this — keep in mind that after editing these files, you also need to restart nginx.
PHP
First, check the version:
bash
php -v
I have 7.2, and the configs are in /etc/php/7.2/. There are several folders:
To find out exactly which configuration file is being used, I put a phpinfo() script in the site's root and checked the output for the config path.
Enabling HTTP/2
The first thing I discovered was that HTTP/2 wasn't enabled on the server. This protocol significantly speeds up website loading by multiplexing and compressing headers. For Magento, where there are a lot of simultaneous requests, this is especially important.
It's enabled simply. In the site's configuration file (in /etc/nginx/conf.d/), find the server block and add http2 to the listen directive:
nginx
server {
listen 443 ssl http2;
ssl on;
...
}
After making changes, reload nginx:
bash
systemctl reload nginx
OPcache: To Enable or Not
OPcache caches compiled PHP bytecode in memory. This really speeds things up, but there are nuances with Magento.
When installing modules, Magento performs a recompile. If OPcache is enabled, problems can arise. You either have to disable the cache before installation or use Varnish instead of Magento's built-in caching.
OPcache is enabled with a single line in php.ini (for fpm, it's in /etc/php/7.2/fpm/php.ini):
ini
opcache.enable=1
By default, 128 MB of memory is allocated for the cache. You can increase it:
ini
opcache.memory_consumption=256
I temporarily disabled OPcache on my server. The reason: the site is under active development, new modules are frequently installed, and the constant hassle with the cache became tiresome. Magento's built-in caching is currently working, but in the future, I plan to look into Varnish.
Blocking Annoying Bots
Bots are a separate pain. They hammer the site with requests, consume resources, and offer no benefit. Some completely ignore robots.txt.
It's fixed simply — we block them by User-Agent in nginx. Add this to your server block:
nginx
if ($http_user_agent ~* SemrushBot|semrush|PetalBot|petalbot|MJ12Bot|AhrefsBot|bingbot|DotBot|LinkpadBot|SputnikBot|statdom.ru|WebDataStats|Jooblebot|Baiduspider|openstat.ru) {
return 403;
}
This isn't the complete list. I added to it by looking at access.log — there you can see who's actually crawling.
Custom Maintenance Page for Magento
The standard "website under maintenance" page in Magento looks pretty sad. I wanted to create my own, with a nice design, but without unnecessary load on the server.
Add this to your site's config:
nginx
set $MAGE_ROOT /var/www/www;
set $maintenance off;
if (-f $MAGE_ROOT/maintenance.enable) {
set $maintenance on;
}
if ($remote_addr ~ (188.xx.yy.zz|188.aa.bb.cc)) {
set $maintenance off;
}
if ($maintenance = on) {
return 503;
}
location /maintenance { }
error_page 503 @maintenance;
location @maintenance {
root $MAGE_ROOT;
rewrite ^(.*)$ /maintenance.html break;
}
How it works:
I put the maintenance.html itself in the site's root. To avoid extra requests on the server, I embedded all CSS and SVG images directly into this file.
PHP-FPM Pool Configuration
Now for the most important part — configuring php-fpm. The pool config is usually located somewhere in /etc/php/version/fpm/pool.d/. For me, it was www.conf.
After all the experiments, this is the resulting config:
ini
[www]
user = www-data
group = www-data
listen = /var/run/php/php7.2-fpm.sock
listen.owner = www-data
listen.group = www-data
php_admin_value[disable_functions] = exec,passthru,shell_exec,system
php_admin_flag[allow_url_fopen] = on
php_value[max_input_vars] = 300000
pm = dynamic
pm.max_children = 259
pm.start_servers = 48
pm.min_spare_servers = 24
pm.max_spare_servers = 248
pm.max_requests = 2000
request_terminate_timeout = 6000
php_admin_value[memory_limit] = 8192M
chdir = /
slowlog = /var/log/php-slow.log
request_slowlog_timeout = 30s
Let me explain what each line does.
How to Calculate max_children
Just blindly copying a value like 259 is a bad idea. I used a script that helped me estimate the approximate values.
First, find out how much memory one php-fpm process consumes on average. For this, there's a command:
bash
ps --no-headers -o "rss,cmd" -C php-fpm7.2 | awk '{ sum+=$1 } END { printf ("Average process size: %d MB\n", sum/NR/1024) }'
This will show the average memory usage in MB.
Next, take all available server memory (let's say 39 GB = 39936 MB), subtract memory for the system, MySQL, nginx, and divide by the average process size. This gives you an approximate max_children.
Mine came out to around 250. I rounded it up to 259 and kept a buffer.
The Outcome
After all this voodoo dancing, the server finally stopped crashing. Memory stayed around 11 GB, even when the load increased. Before that, php-fpm was eating all 39 GB and crashing.
Now I have a working config, the slow log shows bottlenecks, bots aren't pounding the site, and HTTP/2 has slightly sped up loading.
The moral of the story: don't blindly copy configs from the internet. You need to understand what each line does and choose values that fit your specific load. And definitely enable the slow log — without it, you're like a blind kitten.
Listen, the situation is classic to the point of teeth-grinding. The website was working, working, and then — BAM. Holiday sales, marketers gleefully rubbing their hands, expecting a flood of customers, and the server collapses from the load. This happened to me on a VDS with 39 GB of RAM and 12 cores. It seemed like more than enough resources, but the php-fpm process, in conjunction with MySQL, was devouring all the memory, and the site was crashing.
I almost tore my hair out then. I restarted services, added 20 GB of swap — it helped for an hour, then back to square one. I had to dive into the settings and figure out what was wrong.
In this article, I'll share how I fixed it all. No fluff, just what actually worked.
Where It All Began
The server seemed decent:
- Ubuntu
- 39 GB RAM
- 12 cores
- PHP 7.2 + PHP-FPM
- MySQL 5.7
- CMS Magento 2.3.4
Yes, the versions aren't the newest, I know. But Magento 2.3.4 only works properly with PHP 7.2, and an upgrade would have entailed upgrading the entire CMS, which I didn't have time for.
Before the holiday sales, the system lived peacefully. But when the marketers turned on the advertising and users started pouring in, hell broke loose. php-fpm in tandem with MySQL was consuming all the RAM, and the site was going down. Restarts helped for a couple of hours, then the same thing all over again. I added a 20 GB swap file — same story.
I had to recall how to configure nginx and php-fpm.
Where to Find Configs
Before changing anything, you need to understand where to change it. I sometimes get confused myself, so let me lay it out clearly.
Nginx
Nginx configs are located in /etc/nginx. The main file is nginx.conf, but usually, site settings are moved to separate files in /etc/nginx/conf.d/. There, each site has its own .conf file.
Another important point: check the Include directive in your configs. My previous admin added include /var/www/www/nginx.conf, which allowed changing settings directly from the site directory, similar to .htaccess in Apache. If you have something like this — keep in mind that after editing these files, you also need to restart nginx.
PHP
First, check the version:
bash
php -v
I have 7.2, and the configs are in /etc/php/7.2/. There are several folders:
- apache2 — if I were using Apache with mod_php (but I'm not)
- cli — settings for console PHP
- fpm — this is what we need. It contains php-fpm and PHP configs.
- mods-available — PHP extensions. In each .ini file, you can enable/disable an extension by commenting out the extension line.
To find out exactly which configuration file is being used, I put a phpinfo() script in the site's root and checked the output for the config path.
Enabling HTTP/2
The first thing I discovered was that HTTP/2 wasn't enabled on the server. This protocol significantly speeds up website loading by multiplexing and compressing headers. For Magento, where there are a lot of simultaneous requests, this is especially important.
It's enabled simply. In the site's configuration file (in /etc/nginx/conf.d/), find the server block and add http2 to the listen directive:
nginx
server {
listen 443 ssl http2;
ssl on;
...
}
After making changes, reload nginx:
bash
systemctl reload nginx
OPcache: To Enable or Not
OPcache caches compiled PHP bytecode in memory. This really speeds things up, but there are nuances with Magento.
When installing modules, Magento performs a recompile. If OPcache is enabled, problems can arise. You either have to disable the cache before installation or use Varnish instead of Magento's built-in caching.
OPcache is enabled with a single line in php.ini (for fpm, it's in /etc/php/7.2/fpm/php.ini):
ini
opcache.enable=1
By default, 128 MB of memory is allocated for the cache. You can increase it:
ini
opcache.memory_consumption=256
I temporarily disabled OPcache on my server. The reason: the site is under active development, new modules are frequently installed, and the constant hassle with the cache became tiresome. Magento's built-in caching is currently working, but in the future, I plan to look into Varnish.
Blocking Annoying Bots
Bots are a separate pain. They hammer the site with requests, consume resources, and offer no benefit. Some completely ignore robots.txt.
It's fixed simply — we block them by User-Agent in nginx. Add this to your server block:
nginx
if ($http_user_agent ~* SemrushBot|semrush|PetalBot|petalbot|MJ12Bot|AhrefsBot|bingbot|DotBot|LinkpadBot|SputnikBot|statdom.ru|WebDataStats|Jooblebot|Baiduspider|openstat.ru) {
return 403;
}
This isn't the complete list. I added to it by looking at access.log — there you can see who's actually crawling.
Custom Maintenance Page for Magento
The standard "website under maintenance" page in Magento looks pretty sad. I wanted to create my own, with a nice design, but without unnecessary load on the server.
Add this to your site's config:
nginx
set $MAGE_ROOT /var/www/www;
set $maintenance off;
if (-f $MAGE_ROOT/maintenance.enable) {
set $maintenance on;
}
if ($remote_addr ~ (188.xx.yy.zz|188.aa.bb.cc)) {
set $maintenance off;
}
if ($maintenance = on) {
return 503;
}
location /maintenance { }
error_page 503 @maintenance;
location @maintenance {
root $MAGE_ROOT;
rewrite ^(.*)$ /maintenance.html break;
}
How it works:
- $MAGE_ROOT — the path to your site.
- By default, maintenance mode is off.
- If the maintenance.enable file appears in the root, maintenance mode turns on.
- For the specified IPs (e.g., an administrator's), maintenance mode doesn't trigger.
- When maintenance mode is on, everyone goes to maintenance.html.
I put the maintenance.html itself in the site's root. To avoid extra requests on the server, I embedded all CSS and SVG images directly into this file.
PHP-FPM Pool Configuration
Now for the most important part — configuring php-fpm. The pool config is usually located somewhere in /etc/php/version/fpm/pool.d/. For me, it was www.conf.
After all the experiments, this is the resulting config:
ini
[www]
user = www-data
group = www-data
listen = /var/run/php/php7.2-fpm.sock
listen.owner = www-data
listen.group = www-data
php_admin_value[disable_functions] = exec,passthru,shell_exec,system
php_admin_flag[allow_url_fopen] = on
php_value[max_input_vars] = 300000
pm = dynamic
pm.max_children = 259
pm.start_servers = 48
pm.min_spare_servers = 24
pm.max_spare_servers = 248
pm.max_requests = 2000
request_terminate_timeout = 6000
php_admin_value[memory_limit] = 8192M
chdir = /
slowlog = /var/log/php-slow.log
request_slowlog_timeout = 30s
Let me explain what each line does.
- pm = dynamic — means the number of processes will change depending on the load. There's also static (fixed number) and ondemand (processes are created on demand). Dynamic is the golden mean.
- pm.max_children — this is the maximum number of processes a pool can create. If you have 1000 simultaneous requests and max_children is set to 200, 800 people will be waiting. Too high a value can crash the server if the processes consume all memory.
- pm.start_servers — how many processes are created when FPM starts. I set it to 48.
- pm.min_spare_servers — the minimum number of processes that are kept in memory, waiting for requests. If the load drops, FPM won't kill processes below this value.
- pm.max_spare_servers — the maximum number of idle processes. If there are more, the excess are killed.
- pm.max_requests — each process is restarted after handling 2000 requests. This helps combat memory leaks. If a process starts consuming memory and doesn't release it, it will at least restart after some time.
- request_terminate_timeout — if a request takes longer than 6000 seconds, it's simply killed. Magento sometimes needs a lot of time, so I set it with a buffer.
- memory_limit — 8 GB per process. Sounds insane, but Magento knows how to eat memory in chunks.
- slowlog and request_slowlog_timeout — if a script runs for longer than 30 seconds, it's logged. You can then check the logs to see what's causing the slowdown.
How to Calculate max_children
Just blindly copying a value like 259 is a bad idea. I used a script that helped me estimate the approximate values.
First, find out how much memory one php-fpm process consumes on average. For this, there's a command:
bash
ps --no-headers -o "rss,cmd" -C php-fpm7.2 | awk '{ sum+=$1 } END { printf ("Average process size: %d MB\n", sum/NR/1024) }'
This will show the average memory usage in MB.
Next, take all available server memory (let's say 39 GB = 39936 MB), subtract memory for the system, MySQL, nginx, and divide by the average process size. This gives you an approximate max_children.
Mine came out to around 250. I rounded it up to 259 and kept a buffer.
The Outcome
After all this voodoo dancing, the server finally stopped crashing. Memory stayed around 11 GB, even when the load increased. Before that, php-fpm was eating all 39 GB and crashing.
Now I have a working config, the slow log shows bottlenecks, bots aren't pounding the site, and HTTP/2 has slightly sped up loading.
The moral of the story: don't blindly copy configs from the internet. You need to understand what each line does and choose values that fit your specific load. And definitely enable the slow log — without it, you're like a blind kitten.
