Python JSON Practical Guide: Datetime Serialization and Streaming Large Files

In Python backends and data pipelines, JSON looks easy until production constraints appear: datetime fields, non-ASCII text, and multi-GB payloads. This guide focuses on the patterns that stay reliable under real traffic.

We will cover:

• • the real boundary between loads/dumps and load/dump

• • robust datetime serialization with custom encoders

• • readable CJK output using ensure_ascii=False

• • streaming strategies for large JSON data

• • trade-offs between json, orjson, and ujson

---

1) loads/dumps vs load/dump

• • json.dumps(obj): Python object -> JSON string

• • json.loads(text): JSON string -> Python object

• • json.dump(obj, file): Python object -> file

• • json.load(file): file -> Python object

A simple memory aid: s means string.

---

2) Fixing datetime serialization errors

The common error:

from datetime import datetime
import json

json.dumps({"now": datetime.now()})
# TypeError: Object of type datetime is not JSON serializable

Use a custom encoder and keep the policy centralized:

from datetime import datetime, date
import json

class DateTimeEncoder(json.JSONEncoder):
    def default(self, obj):
        if isinstance(obj, datetime):
            return obj.isoformat()
        if isinstance(obj, date):
            return obj.isoformat()
        return super().default(obj)

payload = {"now": datetime.now(), "today": date.today()}
print(json.dumps(payload, cls=DateTimeEncoder, ensure_ascii=False, indent=2))

Use strftime(...) if your APIs require a fixed textual format.

---

3) Why text becomes \uXXXX

By default, json.dumps() escapes non-ASCII characters:

import json
print(json.dumps({"message": "你好，世界"}))
# {"message": "\u4f60\u597d\uff0c\u4e16\u754c"}

For readable multilingual output:

import json
print(json.dumps({"message": "你好，世界"}, ensure_ascii=False))

When writing files, always pair with UTF-8:

with open("data.json", "w", encoding="utf-8") as f:
    json.dump({"message": "你好，世界"}, f, ensure_ascii=False)

---

4) Streaming GB-scale JSON safely

Loading huge arrays with one json.load() call can blow up memory. Two practical options:

Option A: Prefer JSON Lines (.jsonl)

import json

def read_jsonl(path):
    with open(path, "r", encoding="utf-8") as f:
        for line in f:
            line = line.strip()
            if line:
                yield json.loads(line)

for obj in read_jsonl("events.jsonl"):
    pass  # process incrementally

Option B: Use ijson for large array documents

pip install ijson

import ijson

def iter_huge_array(path):
    with open(path, "rb") as f:
        for item in ijson.items(f, "item"):
            yield item

---

5) json vs orjson vs ujson

• • json (stdlib): best compatibility, zero dependency

• • orjson: strong throughput, great for hot paths

• • ujson: easy drop-in in many projects, but verify edge behavior

orjson quick example:

import orjson
from datetime import datetime

data = {"name": "Alice", "now": datetime.now()}
raw = orjson.dumps(data)  # bytes
obj = orjson.loads(raw)

---

6) Production checklist

1. Use loads/dumps for strings, load/dump for files

1. Standardize datetime policy in one encoder module

1. Default to ensure_ascii=False for multilingual products

1. Prefer JSON Lines in log/event pipelines

1. Benchmark before switching serializers in performance-critical paths

---

Continue with JSON Work tools

• • JSON Validator

• • JSON Beautifier

• • JSON to YAML

All tools run locally in your browser. Your JSON data is not uploaded.

---

References

• • Python json documentation

• • ijson on PyPI

• • orjson on PyPI