How to improve data imports for Magento 2

How to Improve Magento 2 Import Speed

A lot of new customers are anxious to just spin up an aMiSTACX stack and start importing data to their new Magento 2 flavor. Only to find that the import is either very slow or doesn’t work at all.

TIPS on Improving Import Performance

First step is to review the admin stack guide that came with the stack. This is found as a PDF version in /home/ubuntu/ and also as an HTML version in the utility directory.

  • Do a test run with your import data. If 8 records won’t import, then 1000 won’t import either.
  • Avoid using the Magento GUI for importing. If the CLI is an option, go with that over using the Magento GUI.
  • Temporarily increase your instance size for importing. A t3-2xlarge is a great starting point because it has 4 procs and plenty of memory.
  • Temporarily increase your RDS database size [If using a LAP/LEP] to at least a db-t3-medium. Again, because memory and two procs.
  • Give Magento more memory. For Apache this is in the .user.ini files in the document root and magento/pub. [See stack guide.] 3-4Gs should be plenty.
  • If using a LAMP/LEMP increase MySQL pool size to 50-70% RAM. [You should do this anyway.]
  • Temporarily disable any monitoring server agents [AWS CW Agent], Elasticsearch service, or any other Magento modules or logging that may consume server resources.
  • Temporarily set indexing to schedule and not on save. You don’t want Magento indexing to run while you are importing.
  • Remove anything in front of the server like a CDN or LoadBalancer. This will help troubleshooting if you run into issues with timeouts.

Overall these simple steps should help you achieve your import goals. If you run into issues, check the Apache/NGINX logs, and Magento logs of possible culprits.

~ Lead_Robot

Comments are closed.