Pipeline Processing in Shell

Level 5161 keystrokes

A pipeline finds the 10 IP addresses with the most requests in an access log: it keeps lines starting with an IP, extracts the IP, counts duplicates, sorts by count and formats the result. The output depends on the log.

Shell code

# Send the log into the pipeline; a line ending in | continues on the next line
1cat access.log |
# Keep only lines that start like an IP address (digits.digits)
2 grep -E '^[0-9]+\.[0-9]+' |
# Print the first field, the client IP
3 awk '{print $1}' |
# Sort so identical IPs are next to each other (uniq needs this)
4 sort |
# Collapse duplicates and prefix each IP with its count
5 uniq -c |
# Sort numerically by count, largest first
6 sort -rn |
# Keep the top 10
7 head -n 10 |
# Read each "count ip" pair and print it as "ip: count requests"
8 while read count ip; do
9 echo "$ip: $count requests"
10 done

Output

192.168.1.1: 1542 requests
192.168.1.2: 892 requests
10.0.0.5: 456 requests
Practice typing this code

Practice typing this code. Results are not recorded in the rankings.

Symbols in this code

The symbols you type in this Shell code and how often they appear.

  • |× 7
  • -× 6
  • '× 4
  • $× 3
  • .× 2
  • "× 2
  • [× 2
  • ]× 2
  • +× 2
  • ;× 1

More Shell examples at level 5

← Shell typing practice