149. Collections + Logging + Functools

The app works. Time to make it production-quality.

🤔 Socratic question: Why do companies pay engineers just to “add logging”? Why is print() considered unprofessional? Because when something breaks at 3am on a production server with 10 million users, you need to know exactly what happened — and print() statements don’t have timestamps, levels, or go to log files.

🤯 Scale check: Google processes more than 8.5 billion searches per day. Every single one generates a log entry. Google has engineers whose entire job is just making logging fast enough. You’re learning the same concepts they use — just at a much smaller scale.

Three stdlib tools that separate amateur code from professional code:

1. logging instead of print() 📝

import logging

logging.basicConfig(level=logging.INFO,
                    format='%(asctime)s %(name)s %(levelname)s %(message)s')
logger = logging.getLogger('ram_manager')

logger.debug('Taking snapshot')           # only visible at DEBUG level
logger.info('RAM: %.1f%%', pct)           # informational
logger.warning('RAM high: %.1f%%', pct)   # visible by default
logger.error('LLM call failed: %s', e)   # always visible

2. Counter for frequency tracking 📊

from collections import Counter

offender_counts: Counter = Counter()

def record_offenders(processes, threshold_mb=100):
    names = [p['name'] for p in processes if p['rss_mb'] > threshold_mb]
    offender_counts.update(names)          # increment all at once
    top = offender_counts.most_common(3)   # [(name, count), ...]
    logger.info('Top offenders: %s', top)

3. @wraps for decorator identity 🎭

from functools import wraps

def retry(fn):
    @wraps(fn)                    # preserves fn.__name__ and __doc__
    def wrapper(*args, **kwargs):
        ...
    return wrapper

Without @wraps, every decorated function shows up as 'wrapper' in logs and tracebacks. Debugging becomes guesswork.

💡 Real-world: Every production Python service at companies like Stripe, Airbnb, and Uber uses logging (not print), Counter for metrics, and decorators for retry logic. This chapter is your graduation from “it works on my machine” to “it works in production.”

💡 Fun fact: Python’s logging module was added in Python 2.3 (2003), modeled after Java’s Log4J library. The Counter class was added in Python 2.7 (2010) — before that, developers used defaultdict(int) or wrote manual counting loops. functools.wraps was added in Python 2.5 (2006) after enough people complained that decorators were breaking help() and debug output.

⚠️ Watch out: Forgetting @wraps(fn) is a silent bug that only shows up under debugging. Every decorated function will appear as 'wrapper' in tracebacks, help() output, and log messages. When your retry fires and logs “wrapper failed on attempt 2”, you have no idea which function it was wrapping. Always add @wraps(fn) inside every decorator.

🤔 Think about it: logger.warning('RAM over %d%%', threshold) uses %-style formatting, not an f-string. Why? Because logging is lazy: it only formats the string if the message will actually be emitted. If the log level is set to ERROR, logger.debug('expensive: %s', big_obj) never calls str(big_obj). What’s the performance impact when you log thousands of debug messages?

🔧 Aryan wants to add logging to the RAM manager, count how often each process hits the threshold, and add retry logic to the LLM call. He uses print() for logging, a plain dict for counting (with manual default handling), and copy-pastes the retry logic instead of wrapping it cleanly.

Learning objectives

Replace print() with logging.getLogger() and logger.warning()
Use Counter(iterable) for frequency counting
Use defaultdict(list) for grouping without KeyError
Apply @wraps(fn) inside decorators to preserve identity
Configure logging once with basicConfig at app startup

Key concepts

logging.getLogger(name) — named logger
logger.debug/info/warning/error() — leveled output
Counter(iterable) — frequency dict
defaultdict(type) — dict with auto default
functools.wraps — preserve name and doc

Try it

import time
<h1 id="bulb-hint-010-xp-use-logger-logging-getlogger-ram-manager-and-logger-warning-print-has-no-timestamps-or-log-levels">💡 : hint : -010 xp Use logger = logging.getLogger(‘ram_manager’) and logger.warning() — print() has no timestamps or log levels</h1>
<p>def check_ram(threshold: int) -> bool:
print(f’Checking RAM…‘)
over = True  # pretend RAM is high
if over:
print(f’WARNING: RAM over {threshold}%’)
return over</p>
<h1 id="bulb-hint-010-xp-use-counter-process-names-counter-counts-elements-in-one-line-with-no-manual-dict-needed">💡 : hint : -010 xp Use Counter(process_names) — Counter counts elements in one line with no manual dict needed</h1>
<p>def count_offenders(process_names: list[str]) -> dict:
counts = {}
for name in process_names:
if name not in counts:
counts[name] = 0
counts[name] += 1
return counts</p>
<h1 id="bulb-hint-010-xp-add-wraps-fn-inside-the-decorator-without-it-all-decorated-functions-show-as-wrapper-in-tracebacks">💡 : hint : -010 xp Add @wraps(fn) inside the decorator — without it all decorated functions show as ‘wrapper’ in tracebacks</h1>
<p>def retry(fn):
def wrapper(*args, **kwargs):
for attempt in range(3):
try:
return fn(*args, **kwargs)
except Exception as e:
if attempt == 2:
raise
time.sleep(1)
return wrapper

Concept detail

stdlib Toolkit: logging, Counter, wraps

logging — proper output for real apps

import logging

# Configure once at app startup
logging.basicConfig(
    level=logging.DEBUG,
    format='%(asctime)s %(name)s %(levelname)s %(message)s'
)

# Use named loggers everywhere else
logger = logging.getLogger('ram_manager')
logger.debug('Snapshot taken: %s', snapshot)
logger.info('RAM at %.1f%%', percent)
logger.warning('RAM over threshold: %d%%', threshold)
logger.error('LLM call failed: %s', err)

collections.Counter — frequency counting

from collections import Counter

offenders = ['chrome', 'python', 'chrome', 'chrome', 'slack']
counts = Counter(offenders)
# Counter({'chrome': 3, 'slack': 1, 'python': 1})

counts['chrome']        # 3
counts['missing']       # 0 (no KeyError)
counts.most_common(2)   # [('chrome', 3), ('slack', 1)]
counts.total()          # 5

collections.defaultdict — dict with auto default

from collections import defaultdict

# list default — no 'if key not in d' needed
history = defaultdict(list)
history['chrome'].append(82.0)
history['chrome'].append(87.5)
# {'chrome': [82.0, 87.5]}

# int default — counters without Counter
hits = defaultdict(int)
hits['chrome'] += 1

functools.wraps — preserve function identity

from functools import wraps

def retry(fn):
    @wraps(fn)   # copies __name__, __doc__, __annotations__
    def wrapper(*args, **kwargs):
        for _ in range(3):
            try:
                return fn(*args, **kwargs)
            except Exception:
                pass
        raise RuntimeError('All retries failed')
    return wrapper

@retry
def call_api(prompt: str) -> str:
    '''Call the LLM API.'''
    ...

call_api.__name__   # 'call_api' ✓ (not 'wrapper')
call_api.__doc__    # 'Call the LLM API.'

Solution

import time
import logging
from collections import Counter, defaultdict
from functools import wraps

# logging.getLogger — named logger, configured once
logger = logging.getLogger('ram_manager')

def check_ram(threshold: int) -> bool:
    logger.debug('Checking RAM...')
    over = True  # pretend RAM is high
    if over:
        logger.warning('RAM over %d%%', threshold)
    return over

def count_offenders(process_names: list[str]) -> Counter:
    # Counter is a dict subclass that counts hashable objects
    return Counter(process_names)

def retry(fn):
    @wraps(fn)   # preserves __name__, __doc__ of wrapped function
    def wrapper(*args, **kwargs):
        for attempt in range(3):
            try:
                return fn(*args, **kwargs)
            except Exception:
                if attempt == 2:
                    raise
                time.sleep(1)
    return wrapper

Tests

import logging
from collections import Counter
from functools import wraps
import pytest

def test_count_offenders_returns_counter():
    names = ['chrome', 'python', 'chrome', 'chrome', 'python']
    result = count_offenders(names)
    assert result['chrome'] == 3
    assert result['python'] == 2

def test_count_offenders_is_counter_type():
    result = count_offenders(['a', 'b', 'a'])
    assert isinstance(result, Counter), 'Use Counter, not plain dict'

def test_retry_preserves_function_name():
    @retry
    def my_func():
        '''Does RAM stuff.'''
        pass
    assert my_func.__name__ == 'my_func', 'Use @wraps to preserve __name__'

def test_retry_preserves_docstring():
    @retry
    def my_func():
        '''Does RAM stuff.'''
        pass
    assert my_func.__doc__ == 'Does RAM stuff.'

def test_retry_retries_on_exception():
    calls = []
    @retry
    def flaky():
        calls.append(1)
        if len(calls) < 3:
            raise ValueError('not yet')
        return 'ok'
    result = flaky()
    assert result == 'ok'
    assert len(calls) == 3

def test_check_ram_uses_logger():
    import inspect
    src = inspect.getsource(check_ram)
    assert 'logger' in src, 'Use logger.warning() not print()'
    assert 'print' not in src, 'Replace print() with logger calls'