Regular expression based process termination in linux.

Even though Unix-based systems are generally stable, some processes (usually non-kernel) could benefit from a kill from time to time. GUI applications may hang, some batch programs may have memory leaks, there are many things that can go wrong. Fortunately Linux provides a set of tools that may help in such situations, you can use the ps command to list active processes. It reports a snapshot of a process granting you information about the process id (PID), parent process id (PPID), priority, memory usage, current state and more. There are also kill  and pkill commands which may terminate a process identified by a specific id or executable name respectively.

If you are running multiple instances of a program, and you want to terminate only a few of them its hard to apply the presented commands - pkill will terminate all instances (that is not desired), while kill will require you to obtain PIDs (this requires some work). It may be easier to locate the mentioned processes by their command line arguments, execution paths or other run parameters, literally or using regular expressions.

The following script does the job:

if [ "$1" ]
then
    ps lax  | tr -s ' ' | cut -d ' '  -f 3,13- | \
    egrep "$1" | cut -d ' ' -f 1 | \ 
    xargs kill -9 2>/dev/null
else
    echo "Usage: $0 <extended regular expresion>"
fi

First a process list is obtained and adjusted for further processing (the text output is so lousy...). Using cut preserves only the third column (PID) and everything beyond the 13-th (whitespace separated application name with additional parameters). Next we match the output with a provided extended regular expression (ERE), be warned though - the tested string starts with the process ID so starting the ERE with a "^" is a bad idea (starting with "^[0-9]+" may work, but you'll eventually end up with restarting your system :-)).

Cheers!

~KR

Find duplicate (redundant) files: bash / linux

Recently I was implementing MT940-extract parsers for a variety of banks. Thousands of files, each containing hundreds of entries. Sometimes the entries had unique identification numbers... sometimes they had not. 

Problems occurred when, due to some random events, the extract storage started to contain duplicate files. As a result many redundant entries were loaded by the parser (this had major consequences on the whole processing).

I have implemented a few mechanisms to prevent this situation, one of them is a linux shell script that locates duplicate files in a selected subdirectory (unlimited depth):

  1 if [ -z $1 ]
  2 then
  3     echo "This script finds duplicate files in the selected directory"
  4     echo "Usage: ./find_duplicate.sh <base dir>"
  5     exit
  6 fi 
  7
  8 all_duplicate=$(find $1 | \
  9     egrep "\.[a-zA-Z0-9]+$" | \
 10     xargs md5sum 2>/dev/null | sed 's/ $/\n/g' | \
 11     sed 's/  /;/g' | sort | uniq  -w32 -D)
 12
 13 last_hash=""
 14
 15 for file in $all_duplicate
 16 do
 17     cur_hash=$(echo $file | cut -d ";" -f1)
 18     if [ "$cur_hash" = "$last_hash" ]
 19     then
 20         echo $(echo $file | cut -d ";" -f2)
 21     fi 
 22     last_hash=$cur_hash
 23 done

So lines 8-11 produce a list of all duplicate files. Since we only want to locate the redundant files, further processing is needed. In the second phase we iterate over the sorted "hash;filename" array and print out file names that have a predecessor with the same hash value thus leaving only a single file name unprinted ( within a group of duplicates that is).

This script ain't perfect, for example it will not work on file names that contain white spaces... anyway, who uses white spaces to name files? :-)

Feel free to correct/modify/share this code!