programming bash

How to run multiple parallel commands in Bash

Often I need to use all my four CPU's cores to speed up processing.

This bash script can execute N parallel processes. Each argument is a command.

trap will kill all subprocesses when SIGINT is catched.
wait $PID_LIST is waiting each process to complete. When all processes have completed, the program exits.

#!/bin/bash

for cmd in "$@"; do {
  echo "Process \"$cmd\" started";
  $cmd & pid=$!
  PID_LIST+=" $pid";
} done

trap "kill $PID_LIST" SIGINT

echo "Parallel processes have started";

wait $PID_LIST

echo
echo "All processes have completed";

Save this script as parallel_commands and make it executable.

This is how to use this script:

parallel_commands "cmd arg0 arg1 arg2" "other_cmd arg0 arg2 arg3"

Examples:

Start 4 parallel sleep and waits until all commands ("sleep 4" is the longer) finishes.

parallel_commands "sleep 1" "sleep 2" "sleep 3" "sleep 4"

Start 5 parallel imagemagick for cr2 to jpg conversion

parallel_commands "convert 1.cr2 1.jpg" "convert 2.cr2 2.jpg" "convert 3.cr2 3.jpg" "convert 4.cr2 4.jpg"

Ti è piaciuto l'articolo? Condividilo! Commentalo!

comments powered by Disqus