Magento 2 Crashing due to MySQL / Cron Jobs

We have come across a few instances now where a module that was not written correctly, and that made use of the Magento Cron framework caused MySQL to stop responding. When MySQL stops responding then it basically crashes Magento.

Tell tale signs in various logs point to the end effect, but not the root cause. You’ll see stuff like MySQL went away… MySQL connection lost… or Magento reports with lines like this:

{"0":"SQLSTATE[HY000] [2002] No such file or directory","1":"#0 \/var\/www\/magento\/vendor\/magento\/zendframework1\/library\/Zend\/Db\/Adapter\/Pdo\/Mysql.php(111):

Watching top from the command line points to MySQL consuming way too much CPU and Memory.

The root cause is actually NOT MySQL. MySQL just can’t keep up and ends up exhausting memory, cpu, and even drive space.

Most of the time it comes back to Magento’s use of cron!

Quickly checking the cron_schedule table in the database will help spot the problem and the offending module(s).

You’ll see a lot jobs stuck in the pending state, and perhaps thousands of records. A quick temporary relief is to just empty the table of all jobs, but this may not fix the core issue.

Best way to tackle this issue is to identify the offending module and disable it. Then reach out to the author and ask for a fix. Simply using another module like MageMojo’s Cron Module to circumvent Magento’s Cron system will let authors of buggy code continue to impact hundreds of systems without accountability.

Holding software vendors to a higher standard is important, because the Magento community as a whole depends on high-quality plugins.

Too Aggressive Cron Policy [Feb 2019]

Our cron policy for aMiSTACX may have been too aggressive, and lead to flooding, or when a poorly written module is involved help cascade into a memory issue.

Our default policies for Magento will be corrected to be inline with Magento’s recommendation of */1 [Every minute]:

*/1 * * * * www-data /usr/bin/php /var/www/magento/bin/magento cron:run | grep -v “Ran jobs by schedule” >> /var/www/magento/var/log/magento.cron.log
*/1 * * * * www-data /usr/bin/php /var/www/magento/update/cron.php >> /var/www/magento/var/log/update.cron.log
*/1 * * * * www-data /usr/bin/php /var/www/magento/bin/magento setup:cron:run >> /var/www/magento/var/log/setup.cron.log

When you change the cron from the crontab, make sure you reload it:

sudo /etc/init.d/cron reload

Glad to shed some light on a common, but yet mysterious cause of Magento gremlins.

 

Leave a comment