Parallel Processing in Shell

Level 5244 keystrokes

Each file is processed in a background job (&), so the four jobs run at the same time; their PIDs are kept in an array and waited for one by one. Because the jobs run in parallel, the order of their lines may differ.

Shell code

# Simulated work for one file: about 1 second
1process_file() {
2 echo "Processing $1..."
3 sleep 1
4 echo "Done: $1"
5}
# Files to process
6files=("a.txt" "b.txt" "c.txt" "d.txt")
# Empty array for the background PIDs
7pids=()
8for f in "${files[@]}"; do
# & starts the call in the background and moves on immediately
9 process_file "$f" &
# $! is that job's PID; += appends it to the array
10 pids+=("$!")
11done
12for pid in "${pids[@]}"; do
# Wait for each job in turn
13 wait "$pid"
14done
# Reached only after all jobs have finished
15echo "All complete"

Output

Processing a.txt...
Processing b.txt...
Processing c.txt...
Processing d.txt...
Done: a.txt
Done: b.txt
Done: c.txt
Done: d.txt
All complete
Practice typing this code

Practice typing this code. Results are not recorded in the rankings.

Symbols in this code

The symbols you type in this Shell code and how often they appear.

  • "× 24
  • .× 7
  • $× 7
  • (× 4
  • )× 4
  • {× 3
  • }× 3
  • =× 3
  • _× 2
  • ;× 2

More Shell examples at level 5

← Shell typing practice