099. Collections Module (Counter, Defaultdict)

Rohan wants to know which process NAMES appear most often across 1,000 snapshots, and which user accounts own the most memory-hungry processes.

Manual approach he starts with:

name_counts = {}
for snap in snapshots:
    for proc in snap["processes"]:
        name = proc["name"]
        if name in name_counts:
            name_counts[name] += 1
        else:
            name_counts[name] = 1

# Sort to find top 5
top = sorted(name_counts.items(), key=lambda x: x[1], reverse=True)[:5]

With Counter:

from collections import Counter

name_counts = Counter(
    proc["name"]
    for snap in snapshots
    for proc in snap["processes"]
)
top = name_counts.most_common(5)

For grouping processes by user:

# Manual — requires KeyError guard:
by_user = {}
for proc in snapshot["processes"]:
    u = proc["username"]
    if u not in by_user:
        by_user[u] = []
    by_user[u].append(proc)

# With defaultdict:
from collections import defaultdict
by_user = defaultdict(list)
for proc in snapshot["processes"]:
    by_user[proc["username"]].append(proc)

defaultdict(list) creates an empty list for any new key automatically — the if key not in dict guard disappears entirely.

Counter arithmetic: c1 + c2 merges two counters. c1 - c2 subtracts (dropping zero/negative counts). c1 & c2 intersection. Useful for comparing two snapshots: snapshot2_counts - snapshot1_counts shows what’s new.

💡 Fun fact: The collections module has been in Python since version 2.4 (2004). Counter was added later in Python 2.7 / 3.1 (2009) — before that, developers used dict.get(key, 0) + 1 or setdefault. The module also includes deque (a double-ended queue with O(1) appends on both sides) and namedtuple (lightweight immutable records).

⚠️ Watch out: Accessing a missing key in a defaultdict silently creates it with the default value. This means len(d) or iterating d.keys() may include keys you never explicitly added — just because you accessed them. If you want read-only access, use d.get(key) instead of d[key].

🤔 Think about it: Counter returns 0 for missing keys instead of raising KeyError. How does this change the way you’d write code that checks if an element was counted? Can you think of a case where this silent-zero behavior could hide a bug?

Learning objectives

Use Counter to count occurrences in an iterable
Use most_common() to find top N elements without manual sorting
Use defaultdict to simplify grouping patterns and eliminate KeyError guards

Key concepts

Counter
defaultdict
collections module
most_common

Try it

Concept detail

The collections module provides specialized containers that extend dict and list.

Counter — counts occurrences of elements: from collections import Counter c = Counter([“a”, “b”, “a”, “c”, “a”]) # Counter({“a”: 3, “b”: 1, “c”: 1}) c[“a”] → 3 (missing keys return 0, not KeyError) c.most_common(2) → [(“a”, 3), (“b”, 1)] Counter(“hello”) → Counter({“l”: 2, “h”: 1, “e”: 1, “o”: 1})

Counter arithmetic: c1 + c2 — add counts c1 - c2 — subtract (drops zero/negative) c1 & c2 — intersection (min of each count) c1 | c2 — union (max of each count)

defaultdict — dict that auto-creates missing values: from collections import defaultdict d = defaultdict(list) d[“missing”].append(1) # no KeyError — creates empty list first d[“missing”].append(2) d[“missing”] → [1, 2]

d = defaultdict(int) d[“count”] += 1 # starts at 0 automatically

d = defaultdict(set) d[“key”].add(“value”) # starts as empty set

Other useful collections: deque(maxlen=N) — O(1) append/pop from both ends; fixed-size rolling window OrderedDict — remembers insertion order (less needed in Python 3.7+) namedtuple — lightweight immutable record: Point = namedtuple(“Point”, [“x”, “y”])

Solution

from collections import Counter, defaultdict

def word_frequency(text):
    return Counter(text.lower().split())

def top_n_words(text, n):
    freq = Counter(text.lower().split())
    return freq.most_common(n)

def group_by_length(words):
    groups = defaultdict(list)
    for word in words:
        groups[len(word)].append(word)
    return dict(groups)

Tests

def test_word_frequency():
    freq = word_frequency("the cat sat on the mat the")
    assert freq["the"] == 3
    assert freq["cat"] == 1

def test_word_frequency_type():
    from collections import Counter
    freq = word_frequency("hello world")
    assert isinstance(freq, Counter)

def test_top_n_words():
    result = top_n_words("a b a c a b", 2)
    assert result[0][0] == "a"
    assert result[0][1] == 3
    assert len(result) == 2

def test_group_by_length():
    groups = group_by_length(["cat", "dog", "elephant", "ox", "bat"])
    assert set(groups[3]) == {"cat", "dog", "bat"}
    assert groups[8] == ["elephant"]
    assert groups[2] == ["ox"]