141. Json + Csv + Pathlib (App V0.4)

“I want to look at last week’s RAM history.” Time to save to disk.

🤔 Socratic question: Every app you use stores data. WhatsApp messages, your Spotify history, your game saves — where does it all go? Files. Even databases are files at the bottom. This chapter: you learn to write and read files like a professional.

He wants timestamped JSON reports and CSV exports. Three traps he falls into:

Trap 1: str(snapshot) writes Python repr, not valid JSON.

# Wrong
f.write(str(snapshot))            # {'percent': 80.1} — not valid JSON

# Right
json.dump(snapshot, f, indent=2)  # {"percent": 80.1} — valid JSON

Trap 2: Every report overwrites the last because the filename is always report.json.

ts = datetime.now().strftime('%Y%m%d_%H%M%S')
path = report_dir / f'report_{ts}.json'   # unique every second

Trap 3: The directory doesn’t exist yet.

report_dir.mkdir(parents=True, exist_ok=True)  # create it (idempotent)

For CSV, he learns csv.DictWriter — no manual row formatting needed:

writer = csv.DictWriter(f, fieldnames=['name', 'pid', 'rss_mb'])
writer.writeheader()
writer.writerows(snapshot['processes'])

Finding the latest report is just a sorted glob:

files = sorted(report_dir.glob('report_*.json'))
latest = files[-1]   # alphabetical = chronological (YYYYMMDD prefix)

🤯 Mind-blown moment: JSON was invented in 2001 by Douglas Crockford. He literally said he “discovered” it rather than invented it — it was already there in JavaScript. Now it’s the lingua franca of the entire internet. Every API, every config file, every data exchange uses it. You’re learning the format that powers the modern web.

💡 Real-world: GitHub stores every issue, PR, and comment as JSON. Twitter/X stores every tweet as JSON. Your AWS bill? JSON. When you write json.dump(), you’re using the same format that runs the internet.

💡 Fun fact: datetime.strftime() has been in Python since version 1.5 (1997). The format codes come from C’s strftime() function, which dates to 1971. The %Y%m%d_%H%M%S pattern produces sortable filenames because lexicographic order matches chronological order — the same trick used by every log rotation system in the world.

⚠️ Watch out: str(snapshot) produces Python repr, not JSON. Single quotes 'percent' vs double quotes "percent", True vs true, None vs null — Python and JSON are different. If you write str(snapshot) to a file and try to parse it with json.load(), you get JSONDecodeError. Always use json.dump().

🤔 Think about it: Why does csv.DictWriter need newline='' when opening the file? What would happen on Windows without it? Why does CSV even have this problem when JSON doesn’t?

💾 Aryan wants to save RAM snapshots so he can review trends later. His save functions use hardcoded paths, write Python repr instead of JSON, and every report overwrites the last because there’s no timestamp in the filename.

Learning objectives

• Use mkdir(parents=True, exist_ok=True) before writing files
• Build timestamped filenames with strftime
• Use json.dump (not str()) for valid JSON output
• Use csv.DictWriter with writeheader() and writerows()
• Use Path.glob() to find the latest report file

Key concepts

• Path.mkdir(parents=True, exist_ok=True) — create directory safely
• strftime(‘%Y%m%d_%H%M%S’) — sortable timestamp string
• json.dump(data, file, indent=2) — write pretty JSON
• csv.DictWriter(file, fieldnames=) — write dict rows as CSV
• Path.glob(‘pattern’) — find files matching a pattern

Try it

Concept detail

App v0.4 — Saving timestamped reports

~/.ram_manager/reports/
  report_20240315_143022.json
  report_20240315_143022.csv
  report_20240315_150000.json
  report_20240315_150000.csv

Key patterns

# Timestamped filename
ts = datetime.now().strftime('%Y%m%d_%H%M%S')
path = report_dir / f'report_{ts}.json'

# Find the latest report
files = sorted(report_dir.glob('report_*.json'))
latest = files[-1]   # alphabetical sort = chronological (YYYYMMDD prefix)

# Write JSON
with open(path, 'w') as f:
    json.dump(snapshot, f, indent=2)

# Write CSV
with open(path, 'w', newline='') as f:
    writer = csv.DictWriter(f, fieldnames=['name', 'pid', 'rss_mb'])
    writer.writeheader()
    writer.writerows(processes)

Solution

import psutil
import json
import csv
from pathlib import Path
from datetime import datetime

REPORT_DIR = Path.home() / '.ram_manager' / 'reports'

def take_snapshot(n: int = 10) -> dict:
    mem = psutil.virtual_memory()
    processes = []
    for proc in psutil.process_iter(['pid', 'name', 'memory_info']):
        try:
            rss_mb = proc.info['memory_info'].rss / 1e6
            processes.append({'name': proc.info['name'], 'pid': proc.info['pid'], 'rss_mb': rss_mb})
        except (psutil.NoSuchProcess, psutil.AccessDenied):
            pass
    return {
        'percent': mem.percent,
        'used_gb': mem.used / 1e9,
        'total_gb': mem.total / 1e9,
        'processes': sorted(processes, key=lambda p: p['rss_mb'], reverse=True)[:n],
        'timestamp': datetime.now().isoformat(),
    }

def save_report(snapshot: dict, report_dir: Path = REPORT_DIR) -> tuple[Path, Path]:
    report_dir.mkdir(parents=True, exist_ok=True)
    ts = datetime.now().strftime('%Y%m%d_%H%M%S')

    json_path = report_dir / f'report_{ts}.json'
    with open(json_path, 'w') as f:
        json.dump(snapshot, f, indent=2)

    csv_path = report_dir / f'report_{ts}.csv'
    if snapshot['processes']:
        with open(csv_path, 'w', newline='') as f:
            writer = csv.DictWriter(f, fieldnames=['name', 'pid', 'rss_mb'])
            writer.writeheader()
            writer.writerows(snapshot['processes'])

    return json_path, csv_path

def load_latest_report(report_dir: Path = REPORT_DIR) -> dict:
    files = sorted(report_dir.glob('report_*.json'))
    if not files:
        raise FileNotFoundError(f'No reports in {report_dir}')
    return json.loads(files[-1].read_text())

def main():
    snapshot = take_snapshot(n=10)
    json_path, csv_path = save_report(snapshot)
    print(f"RAM: {snapshot['percent']:.1f}%")
    print(f"Saved: {json_path.name}  {csv_path.name}")

if __name__ == '__main__':
    main()

Tests

import json
import csv
import inspect
import tempfile
import re
from pathlib import Path

SNAP = {
    'percent': 72.5, 'used_gb': 11.6, 'total_gb': 16.0,
    'timestamp': '2024-01-01T10:00:00',
    'processes': [{'name': 'chrome', 'pid': 812, 'rss_mb': 1800.0}],
}

def test_save_report_creates_json():
    with tempfile.TemporaryDirectory() as d:
        json_path, csv_path = save_report(SNAP, report_dir=Path(d))
        assert json_path.exists()
        loaded = json.loads(json_path.read_text())
        assert loaded['percent'] == 72.5

def test_save_report_creates_csv_with_header():
    with tempfile.TemporaryDirectory() as d:
        _, csv_path = save_report(SNAP, report_dir=Path(d))
        assert csv_path.exists()
        lines = csv_path.read_text().splitlines()
        assert 'name' in lines[0], 'First line of CSV must be a header row'
        assert 'chrome' in lines[1]

def test_save_report_filename_has_timestamp():
    with tempfile.TemporaryDirectory() as d:
        json_path, _ = save_report(SNAP, report_dir=Path(d))
        assert re.search(r'\d{8}_\d{6}', json_path.name), (
            'Filename must contain a sortable YYYYMMDD_HHMMSS timestamp'
        )

def test_save_report_uses_json_dump():
    src = inspect.getsource(save_report)
    assert 'json.dump' in src, 'Use json.dump(), not str()'

def test_load_latest_report():
    snap = {'percent': 99.0, 'processes': [], 'used_gb': 1.0, 'total_gb': 1.0, 'timestamp': ''}
    with tempfile.TemporaryDirectory() as d:
        save_report(snap, report_dir=Path(d))
        loaded = load_latest_report(report_dir=Path(d))
        assert loaded['percent'] == 99.0

def test_load_latest_report_uses_glob():
    src = inspect.getsource(load_latest_report)
    assert 'glob' in src, 'Use Path.glob() to find the latest timestamped report'

Resources

• pathlib — Python Docs
• json — Python Docs
• csv — Python Docs
• Python JSON guide — Real Python
• Working with files in Python — Corey Schafer (YouTube)