140. Json + Csv + Pathlib + Datetime

“I want to look at last week’s RAM history.” Time to save to disk.

🤔 Socratic question: Every app you use stores data. WhatsApp messages, your Spotify history, your game saves — where does it all go? Files. Even databases are files at the bottom. This chapter: you learn to write and read files like a professional.

He wants timestamped JSON reports and CSV exports. Three traps he falls into:

Trap 1: str(snapshot) writes Python repr, not valid JSON.

# Wrong
f.write(str(snapshot))            # {'percent': 80.1} — not valid JSON

# Right
json.dump(snapshot, f, indent=2)  # {"percent": 80.1} — valid JSON

Trap 2: Every report overwrites the last because the filename is always report.json.

ts = datetime.now().strftime('%Y%m%d_%H%M%S')
path = report_dir / f'report_{ts}.json'   # unique every second

Trap 3: The directory doesn’t exist yet.

report_dir.mkdir(parents=True, exist_ok=True)  # create it (idempotent)

For CSV, he learns csv.DictWriter — no manual row formatting needed:

writer = csv.DictWriter(f, fieldnames=['name', 'pid', 'rss_mb'])
writer.writeheader()
writer.writerows(snapshot['processes'])

Finding the latest report is just a sorted glob:

files = sorted(report_dir.glob('report_*.json'))
latest = files[-1]   # alphabetical = chronological (YYYYMMDD prefix)

🤯 Mind-blown moment: JSON was invented in 2001 by Douglas Crockford. He literally said he “discovered” it rather than invented it — it was already there in JavaScript. Now it’s the lingua franca of the entire internet. Every API, every config file, every data exchange uses it. You’re learning the format that powers the modern web.

💡 Real-world: GitHub stores every issue, PR, and comment as JSON. Twitter/X stores every tweet as JSON. Your AWS bill? JSON. When you write json.dump(), you’re using the same format that runs the internet.

💡 Fun fact: datetime.strftime() has been in Python since version 1.5 (1997). The format codes come from C’s strftime() function, which dates to 1971. The %Y%m%d_%H%M%S pattern produces sortable filenames because lexicographic order matches chronological order — the same trick used by every log rotation system in the world.

⚠️ Watch out: str(snapshot) produces Python repr, not JSON. Single quotes 'percent' vs double quotes "percent", True vs true, None vs null — Python and JSON are different. If you write str(snapshot) to a file and try to parse it with json.load(), you get JSONDecodeError. Always use json.dump().

🤔 Think about it: Why does csv.DictWriter need newline='' when opening the file? What would happen on Windows without it? Why does CSV even have this problem when JSON doesn’t?

💾 Aryan wants to save RAM snapshots to disk so he can review them later. His save function opens files in the wrong mode, uses string concatenation instead of pathlib, and forgets to add a timestamp so every report overwrites the last.

Learning objectives

Use pathlib.Path for cross-platform file paths
Create directories with mkdir(parents=True, exist_ok=True)
Write JSON with json.dump(indent=2)
Write CSV with csv.DictWriter, writeheader(), writerows()
Generate sortable timestamps with datetime.strftime()

Key concepts

Path.home() / ‘subdir’ — cross-platform home path
json.dump(data, f, indent=2) — write readable JSON
csv.DictWriter — write dicts as CSV rows
datetime.now().strftime() — sortable timestamp
newline=‘’ — required for csv on Windows

Try it

Concept detail

Saving Data with json, csv, pathlib, and datetime

pathlib — modern file paths

from pathlib import Path

# Build paths safely (works on all OS)
report_dir = Path.home() / '.ram_manager' / 'reports'
report_dir.mkdir(parents=True, exist_ok=True)

path = report_dir / 'snapshot.json'
path.write_text('hello')          # write string
content = path.read_text()        # read string
path.write_bytes(b'...')          # write bytes
path.exists()                     # bool
path.parent                       # parent directory

json — save and load structured data

import json

data = {'percent': 72.5, 'processes': [{'name': 'chrome', 'rss_mb': 1800}]}

# Write
with open('snapshot.json', 'w') as f:
    json.dump(data, f, indent=2)

# Read back
with open('snapshot.json') as f:
    loaded = json.load(f)

# To/from string
text = json.dumps(data, indent=2)
data = json.loads(text)

csv — save tabular data

import csv

processes = [
    {'name': 'chrome', 'pid': 812, 'rss_mb': 1800.0},
    {'name': 'python', 'pid': 421, 'rss_mb': 670.0},
]

# Write
with open('report.csv', 'w', newline='') as f:
    writer = csv.DictWriter(f, fieldnames=['name', 'pid', 'rss_mb'])
    writer.writeheader()
    writer.writerows(processes)

# Read back
with open('report.csv') as f:
    reader = csv.DictReader(f)
    rows = list(reader)

datetime — timestamps

from datetime import datetime

now = datetime.now()
ts = now.strftime('%Y%m%d_%H%M%S')   # '20240315_143022'
iso = now.isoformat()                  # '2024-03-15T14:30:22.123456'

# Build timestamped filenames
filename = f'report_{ts}.json'

Solution

import json
import csv
from pathlib import Path
from datetime import datetime

REPORT_DIR = Path.home() / '.ram_manager' / 'reports'

def save_json(snapshot: dict, filename: str) -> Path:
    REPORT_DIR.mkdir(parents=True, exist_ok=True)
    path = REPORT_DIR / filename
    with open(path, 'w') as f:
        json.dump(snapshot, f, indent=2)
    return path

def save_csv(processes: list[dict], filename: str) -> Path:
    REPORT_DIR.mkdir(parents=True, exist_ok=True)
    path = REPORT_DIR / filename
    if not processes:
        return path
    with open(path, 'w', newline='') as f:
        writer = csv.DictWriter(f, fieldnames=processes[0].keys())
        writer.writeheader()
        writer.writerows(processes)
    return path

def timestamped_filename(base: str, ext: str) -> str:
    ts = datetime.now().strftime('%Y%m%d_%H%M%S')
    return f'{base}_{ts}.{ext}'

Tests

import json
import csv
import inspect
import tempfile
from pathlib import Path

def test_save_json_creates_valid_json():
    with tempfile.TemporaryDirectory() as d:
        global REPORT_DIR
        orig = REPORT_DIR
        REPORT_DIR = Path(d)
        try:
            snapshot = {'percent': 72.5, 'total_gb': 16.0, 'used_gb': 11.6}
            path = save_json(snapshot, 'test.json')
            assert path.exists()
            loaded = json.loads(path.read_text())
            assert loaded['percent'] == 72.5
        finally:
            REPORT_DIR = orig

def test_save_csv_writes_header():
    with tempfile.TemporaryDirectory() as d:
        global REPORT_DIR
        orig = REPORT_DIR
        REPORT_DIR = Path(d)
        try:
            procs = [
                {'name': 'chrome', 'pid': 812, 'rss_mb': 1800.0},
                {'name': 'python', 'pid': 421, 'rss_mb': 670.0},
            ]
            path = save_csv(procs, 'procs.csv')
            assert path.exists()
            lines = path.read_text().splitlines()
            assert 'name' in lines[0], 'First line must be a header row containing "name"'
            assert 'chrome' in lines[1]
        finally:
            REPORT_DIR = orig

def test_timestamped_filename_includes_base():
    name = timestamped_filename('ram_report', 'json')
    assert name.startswith('ram_report_')
    assert name.endswith('.json')

def test_timestamped_filename_includes_datetime_stamp():
    name = timestamped_filename('snap', 'csv')
    # Should contain a numeric timestamp portion between the base and extension
    import re
    assert re.search(r'\d{8}_\d{6}', name), (
        'Filename must include a sortable datetime stamp like 20240315_143022'
    )

def test_save_json_uses_json_dump():
    src = inspect.getsource(save_json)
    assert 'json.dump' in src, 'Must use json.dump(), not str()'