Script to check directory for changed or missing files

This script was needed to monitor product images on a Magento 2 store. We noticed that sometimes after importing product updates, images which had been working were now missing.

The Magento products were imported using a custom extension from a CSV file which included fields for product images. Images specified in the CSV fields needed to be uploaded to a staging folder on the website so the import process could find and move them to the correct path for Magento to use.

The CSV was created using a back-end process which got product details from a database and added all images found in the back-end master directory to each product.

If an image had been manually added to a product using the Magento admin pages instead of the CSV it would work fine. However, if the image hadn’t also been uploaded to the staging directory, the next time that same product was updated from the CSV, the problem occurred. Any images not found in the staging directory would be removed from the Magento product DB and filesystem.

This script was scheduled as a daily cron job to monitor the Magento image directory and notify by email if any images had been deleted (or added) since the last time it ran.

Note that this Magento 2 installation used an additional non-standard location to store extra product images (pub/media/catalog/history).

Tested on Ubuntu 18.0.4 LTS

#!/usr/bin/env bash

# based on a script from
# https://stackoverflow.com/questions/25053061/bash-shell-script-to-find-out-the-missing-file-newly-added-file-in-a-directory

# Directory you want to watch
WATCHDIR=/var/www/vhosts/site.co.uk/httpdocs/pub/media/catalog

# Sub-directory 1 you want to exclude
EXCLUDEDIR=/var/www/vhosts/site.co.uk/httpdocs/pub/media/catalog/product/cache

# Sub-directory 2 you want to exclude
EXCLUDEDIR2=/var/www/vhosts/site.co.uk/httpdocs/pub/media/catalog/category

# Sub-directory 3 you want to exclude
EXCLUDEDIR3=/var/www/vhosts/site.co.uk/httpdocs/pub/media/catalog/tmp

# Name of the file that will keep the list of the files when you last checked it
LAST=/var/www/vhosts/site.co.uk/scripts/MissingImageCheckWithHistoryLast.log

# Name of the file that will keep the list of the files you are checking now
CURRENT=/var/www/vhosts/site.co.uk/scripts/MissingImageCheckWithHistoryCurrent.log

# Name of the file to store text for an email
EMAILTEXT=/var/www/vhosts/site.co.uk/scripts/MissingImageCheckWithHistoryEmail.txt

# The first time we create the log file
touch $LAST

find $WATCHDIR -path $EXCLUDEDIR -prune -o -path $EXCLUDEDIR2 -prune -o -path $EXCLUDEDIR3 -prune -type f -o -print > $CURRENT

diff $LAST $CURRENT > /dev/null 2>&1

# If there is no difference exit
if [ $? -eq 0 ]
then
    echo "No changes"
else
    # Else, list the files that changed

    echo "List of new files" >$EMAILTEXT
    diff $LAST $CURRENT | grep '^>' >>$EMAILTEXT
    echo "List of files removed" >>$EMAILTEXT
    diff $LAST $CURRENT | grep '^<' >>$EMAILTEXT

    echo "$(cat $EMAILTEXT)"

    # Send a notification
    mail -r "from@example.com" -s "Website images check" to@example.com <$EMAILTEXT

    # Lastly, move CURRENT to LAST
    mv $CURRENT $LAST
fi

Example of the email notification

List of new files
List of files removed
< /var/www/vhosts/site.co.uk/httpdocs/pub/media/catalog/product/1/8/test2.txt