Recipe 5.5 Traversing a Hash

5.5.1 Problem

You want to perform an action on each entry (i.e., each key-value pair) in a hash.

5.5.2 Solution

Use each with a while loop:

while(($key, $value) = each(%HASH)) {
    # do something with $key and $value
}

Or use keys with a foreach loop, unless the hash is potentially very large:

foreach $key (keys %HASH) {
    $value = $HASH{$key};
    # do something with $key and $value
}

5.5.3 Discussion

Here's a simple example, iterating through the %food_color hash from the introduction:

# %food_color per the introduction
while(($food, $color) = each(%food_color)) {
    print "$food is $color.\n";
}
Banana is yellow.
Apple is red.
Carrot is orange.
Lemon is yellow.

foreach $food (keys %food_color) {
    my $color = $food_color{$food};
    print "$food is $color.\n";
}
Banana is yellow.
Apple is red.
Carrot is orange.
Lemon is yellow.

We didn't really need the $color variable in the foreach example, because we use it only once. Instead, we could have written:

print "$food is $food_color{$food}.\n"

Every time each is called on the same hash, it returns the "next" key-value pair. We say "next" because the pairs are returned in the order the underlying lookup structure imposes on them, which appears to be no order at all. When each runs out of hash elements, it returns the empty list ( ), whose assignment tests false and terminates the while loop.

The foreach example uses keys, which constructs an entire list containing every key from the hash before the loop even begins executing. The advantage to using each is that it gets the keys and values one pair at a time. If the hash contains many keys, not having to preconstruct a complete list of them can save substantial memory. The each function, however, doesn't let you control the order in which pairs are processed.

Using foreach and keys to loop over the list lets you impose an order. For instance, if we wanted to print the food names in alphabetical order:

foreach $food (sort keys %food_color) {
    print "$food is $food_color{$food}.\n";
}
Apple is red.
Banana is yellow.
Carrot is orange.
Lemon is yellow.

This is a common use of foreach. We use keys to obtain a list of keys in the hash, and then we use foreach to iterate over them. The danger is that if the hash contains a large number of elements, the list returned by keys will use a lot of memory. The trade-off lies between memory use and the ability to process the entries in a particular order. We cover sorting in more detail in Recipe 5.10.

Because keys, values, and each all share the same internal data structures, be careful about mixing calls to these functions or prematurely exiting an each loop. Each time you call keys or values, the current location for each is reset. This code loops forever, printing the first key returned by each:

while ( ($k,$v) = each %food_color ) {
    print "Processing $k\n";
    keys %food_color;               # goes back to the start of %food_color
}

Modifying a hash while looping over it with each or foreach is, in general, fraught with danger. The each function can behave differently with tied and untied hashes when you add or delete keys from a hash. A foreach loops over a pregenerated list of keys, so once the loop starts, foreach can't know whether you've added or deleted keys. Keys added in the body of the loop aren't automatically appended to the list of keys to loop over, nor are keys deleted by the body of the loop deleted from this list.

Example 5-1 reads a mailbox file and reports the number of messages from each person. It uses the From: line to determine the sender. (It isn't clever in this respect, but we're showing hash manipulation, not mail-file processing.) Supply the mailbox filename as a command-line argument, or use a "-" to indicate you're piping the mailbox to the program. (When Perl opens a file named "-" for reading using fewer than three arguments to open, this means to use the current standard input.)

Example 5-1. countfrom

  #!/usr/bin/perl
  # countfrom - count number of messages from each sender
  $filename = $ARGV[0] || "-";          # "-" means standard input
  open(FILE, "< $filename")                    or die "Can't open $filename : $!";
  while(<FILE>) {
      if (/^From: (.*)/) { $from{$1}++ }
  }
  foreach $person (sort keys %from) {
      print "$person: $from{$person}\n";
  }