قالب وردپرس درنا توس
Home / Tips and Tricks / Perform a task when a new file is added to a directory in Linux

Perform a task when a new file is added to a directory in Linux



  Bash Shell on Ubuntu PC Concept
Fatmawati Achmad Zaenuri / Shutterstock.com

This tutorial will show you how to use (notify) Linux file system events to be notified when a file is in one Directory is displayed. You could use these as triggers to automate common tasks on your system.

We will write a script that monitors a directory and processes new files added. Each file is compressed and moved to another directory as soon as it is recognized. The script uses the inotify subsystem through a utility called inotify-tools. But first we install the tool and experiment.

Installing inotify-tools and gzip

Use apt-get to install this package on your system if you are using Ubuntu or any other Debian-based distribution system. Instead, use the package management tool of your Linux distribution on other Linux distributions.

  sudo apt-get install inotify-tools gzip 

Experimenting with inotify-tools

Let's start by looking at a directory and the occurrence of events when they are new files arrive. We will use a tool called inotifywatch which is part of inotify-tools. Create a new directory named "incoming":

  mkdir incoming 

Start monitoring this directory by running the following command:

  inotifywatch -v incoming 

This instructs inotify all File system events in the incoming folder "directory. With the option -v the tool can provide additional information about what it does. We did not specify a timeout option (-t ), and the command collects events until we exit with CTRL + C. At this point, our terminal should look something like this:

Open a new terminal window (or tab) and change to the incoming directory. Use the touch command to create a new file named "newfile".

  cd incoming /
Touch Newfile

Now go back to the first terminal window and stop inotifywatch by pressing CTRL + C.

The console displays a table of events that indicates an instance of "create" open "," attrib "and" close_write ". These four events occurred when we created a new file with touch, set its file access attributes, opened it to write a final null character, and then closed it again. These are just a few of the many events that can be monitored on a file system using inotify-tools. The full list can be found on the main page for inotifywatch.

For our purposes we are only interested in two events:

  • "create" – when a file is created in the target directory.
  • "moving_to" – when a file is moved from another location to the destination directory.

Let's try inotifywatch again, but this time with the instruction to monitor only these two events. Run this command in the first terminal window off:

  inotifywatch -v -e create -e moves_incoming 

In the second terminal window or on the tab, try to create a new file, change the contents, and then move the file to a different location to the destination directory. All these commands are executed from the home directory.

  touch incoming / created
Echo Testing123 >> detailed / created
Touch / tmp / created2
mv / tmp / created2 incoming /

Return to the first terminal window and stop inotifywatch by pressing CTRL + C. The following output appears:

Only two events were counted: create a file named "created.txt" and move an existing file named "created2.txt". Everything else, like changing "created.txt", was ignored.

Watching a directory and performing a task

Now that we know what events should follow, we can use another tool named inotifywait . to block execution until a file is created in or moved to our destination directory. We use the same arguments as in inotifywatch and also determine how to format the filename for our task.

Before we start, we need a directory that stores files that have already been processed. Create a directory called "processing":

  mkdir processing 

Then create a new script named "watch-incoming.sh" and add the following content:

  #! / Bin / bash

GOAL = ~ / incoming /
PROCESSED = ~ / edited /

inotifywait -m -e created -e moved -to -format "% f" $ TARGET 
| while reading FILENAME
do
Echo $ FILENAME recognized, moved and zipped
mv "$ TARGET / $ FILENAME" "$ PROCESSED / $ FILENAME"
gzip "$ PROCESSED / $ FILENAME"
done

The script executes the command [otowwait with the option -m . This will change the command monitor indefinitely. Each time a new event is detected, the file name is passed to the read command and inserted in the "FILENAME" variable. The block under the while loop is executed by first moving the file to the "processed" directory and then zipping it. The original file is replaced with the compressed file, and the file name ends in ".gz".

Let's give execute permissions to this script and execute it from our home directory.

  chmod u + x watch-incoming. Sch
./watch-incoming.sh[19659011*Openthesecondterminalwindowandopenanewfileinthe'InboxDirectory'tolistthecontentsofthedirectory'incoming'and'processing'toseetheresultoftherecognizedevent:

The raw text file that we copied to the incoming directory was detected by Script, copied to "processed" and then compressed with gzip.

We can now do some interesting tasks as we can search for new files that arrive in a directory. For example, we could add a watermark to image files, compress raw videos in mp4 format, and even upload any new file we see into an Amazon S3 bucket. This script is a good starting point for rolling your own workflows and automating common tasks on your system.


Source link