← Home

105. Csv Module

In-memory CSV parsing with csv.DictReader and io.StringIO

105. Csv Module

Rohan wants to export the top processes from each snapshot into a CSV so his manager can open it in Excel.

He starts with manual string formatting — immediately messy when process names contain commas:

# Wrong — breaks if name has a comma, no quoting, no header
lines = ["name,pid,rss_mb"]
for proc in snapshot["processes"]:
    lines.append(f"{proc['name']},{proc['pid']},{proc['rss_mb']}")

The right way — csv.DictWriter handles quoting and escaping:

import csv, io

output = io.StringIO()
writer = csv.DictWriter(output, fieldnames=["name", "pid", "rss_mb"])
writer.writeheader()
writer.writerows(snapshot["processes"])
csv_string = output.getvalue()

When reading a CSV back (e.g., from an uploaded file), csv.DictReader gives dict access by column name:

reader = csv.DictReader(io.StringIO(csv_string))
for row in reader:
    print(row["name"], row["rss_mb"])   # column name access
    # Values are always strings — convert as needed
    rss = float(row["rss_mb"].strip())

io.StringIO wraps a string so it behaves like a file object. This is how you use csv in Pyodide (browser Python) and in unit tests — no real files needed.

csv.reader vs csv.DictReader: reader gives each row as a list — you access by index row[0]. DictReader uses the first row as column headers and gives each subsequent row as a dict — you access by name row["name"]. DictReader is almost always what you want.

💡 Fun fact: CSV (Comma-Separated Values) is one of the oldest data interchange formats — it predates JSON by decades and was used for data exchange with mainframes in the 1970s. Despite its simplicity, CSV has no formal standard (RFC 4180 is merely an informational memo). This is why there are so many edge-case headaches: fields with commas, quoted fields, newlines inside fields, encoding issues.

⚠️ Watch out: All values from csv.DictReader are strings — even numbers. row["Score"] is always "95", never 95. You must call int() or float() yourself. And if the CSV has leading or trailing whitespace around values (like " 95"), you must call .strip() before conversion or int(" 95") will raise ValueError.

🤔 Think about it: io.StringIO makes a string behave like a file. This is useful for testing. Could you use io.StringIO to avoid writing a CSV to disk when you just want to pass it to another function that expects a file object? What other Python concepts rely on this “duck typing” approach where any object with the right interface works?

Learning objectives

  • Use csv.DictReader to access CSV columns by name
  • Use io.StringIO to parse CSV data from a string without a real file
  • Strip whitespace from CSV values before type conversion
  • Compute statistics (average, max) from parsed CSV data

Key concepts

  • csv.reader — rows as lists
  • csv.DictReader — rows as dicts with header keys
  • io.StringIO — string as file object
  • .strip() — whitespace removal before int()

Try it

Concept detail

csv Module and io.StringIO

Python’s csv module parses comma-separated data. Two main reader types:

csv.reader vs csv.DictReader

import csv, io

data = "Name,Score\nAlice,95\nBob,87\n"

# csv.reader — rows are lists:
for row in csv.reader(io.StringIO(data)):
    print(row)  # ['Name', 'Score'], ['Alice', '95'], ...
    # Access by index: row[0], row[1]

# csv.DictReader — rows are dicts using header as keys:
for row in csv.DictReader(io.StringIO(data)):
    print(row)  # {'Name': 'Alice', 'Score': '95'}, ...
    # Access by name: row['Name'], row['Score']

io.StringIO — In-Memory File

io.StringIO wraps a string so it behaves like a file object. This is essential in Pyodide and anywhere you receive data as a string (API responses, test fixtures) but a function expects a file:

import io
fake_file = io.StringIO("line1\nline2\n")
for line in fake_file:
    print(line.strip())

Always Strip Whitespace Before Conversion

CSV values often have leading/trailing spaces:

score = int(row["Score"].strip())  # safe — "  95" → "95" → 95
score = int(row["Score"])          # crashes on " 95"
ToolUse case
csv.readerSimple CSVs, access by column index
csv.DictReaderCSVs with headers, access by column name
io.StringIOTreat a string as a file object
.strip()Remove whitespace from values before conversion

Solution

import csv
import io

def parse_grades(csv_string: str) -> list:
    """Parse a CSV string into a list of dicts with Name and Score."""
    # csv.DictReader uses the header row as keys automatically
    reader = csv.DictReader(io.StringIO(csv_string))
    rows = []
    for row in reader:
        name = row["Name"]
        # strip whitespace before int() conversion
        score = int(row["Score"].strip())
        rows.append({"name": name.strip(), "score": score})
    return rows

def compute_average(csv_string: str) -> float:
    """Compute the average score from a grade CSV string."""
    grades = parse_grades(csv_string)
    if not grades:
        return 0.0
    total = sum(g["score"] for g in grades)
    return round(total / len(grades), 2)

def top_student(csv_string: str) -> str:
    """Return the name of the student with the highest score."""
    grades = parse_grades(csv_string)
    if not grades:
        return ""
    best = max(grades, key=lambda g: g["score"])
    return best["name"]

Tests

GRADES_CSV = "Name,Score,Grade\nAlice, 95,A\nBob, 87,B+\nCharlie, 72,C\n"
SINGLE_CSV = "Name,Score,Grade\nDiana, 100,A+\n"
SPACED_CSV = "Name,Score,Grade\nEve,   88,B+\nFrank,   76,C+\n"

def test_parse_grades_returns_list():
    result = parse_grades(GRADES_CSV)
    assert isinstance(result, list), "parse_grades should return a list"

def test_parse_grades_correct_count():
    result = parse_grades(GRADES_CSV)
    assert len(result) == 3, f"Expected 3 students, got {len(result)}"

def test_parse_grades_name_access():
    result = parse_grades(GRADES_CSV)
    names = [r["name"] for r in result]
    assert "Alice" in names, f"Expected 'Alice' in names, got {names}"

def test_parse_grades_score_is_int():
    result = parse_grades(GRADES_CSV)
    for r in result:
        assert isinstance(r["score"], int), f"score should be int, got {type(r['score'])}"

def test_parse_grades_strips_whitespace():
    # Values with leading spaces like " 95" must be stripped before int()
    result = parse_grades(SPACED_CSV)
    scores = [r["score"] for r in result]
    assert 88 in scores, f"Expected 88 in scores after stripping, got {scores}"

def test_compute_average_correct():
    # (95 + 87 + 72) / 3 = 84.67
    result = compute_average(GRADES_CSV)
    assert result == 84.67, f"Expected 84.67, got {result}"

def test_compute_average_single():
    result = compute_average(SINGLE_CSV)
    assert result == 100.0, f"Expected 100.0, got {result}"

def test_top_student_correct():
    result = top_student(GRADES_CSV)
    assert result == "Alice", f"Expected 'Alice', got '{result}'"

Resources