Observatory · food and agriculture

Ceres

Farm to market to plate to policy

Country-commodity food-system data and economist-first briefs. Four engines spanning the full value chain: farm (production, yields, inputs), markets (commodity prices, trade flows, stocks), consumption and nutrition (diet composition, food security, stunting), and policy/fiscal (subsidies, tariffs, public expenditure). FAO, AMIS, USDA, IFPRI data. Default port 8008.

Live Dashboard Repository Why it exists
71
Collectors
4
Engines
~195
Countries
8008
Default port
Charter

What this observatory tracks

Farm engine

Crop production volumes, yields per hectare, arable land, fertilizer use, irrigation coverage. The physical supply side of the global food system, structured for cross-country comparison.

Markets engine

Commodity spot and futures prices, trade flows by product, import dependency ratios, stock-to-use ratios. Price transmission from global benchmarks to domestic retail.

Consumption and nutrition engine

Dietary energy supply, protein availability, prevalence of undernourishment, stunting and wasting rates, food price inflation. The demand and welfare side of the value chain.

Policy and fiscal engine

Agricultural subsidies, tariff schedules, public spending on agriculture as share of GDP, food safety net programs. The institutional layer that shapes farm incentives and consumer access.

Engines

4 engines, 71 collectors

Each engine groups collectors by economic domain. Collectors pull from FAO, AMIS, USDA, IFPRI, and other public sources on staggered UTC schedules. All data lands in a single SQLite warehouse with consistent schema.

farm
markets
consumption
policy / fiscal
Stack

Production surface

FastAPI and Starlette middleware: gzip, optional CORS, security headers, X-Request-ID on every response, optional JSON log lines via LOG_JSON. Readiness runs SELECT 1 against SQLite. Optional GET /metrics exposes ceres_http_* counters and latency histograms.

  • Warehouse: WAL SQLite, tuned page cache and mmap, optimize on shutdown.
  • Ingestion: httpx async client, APScheduler for staggered UTC collection, openpyxl for Excel sources.
  • OpenAPI: /docs and /redoc on the running service.
API

Route map (read-only data)

All under /api/data/. Query parameters support filtering, year windows, pagination, and multi-series panels. Admin routes under /api/admin/ require X-Admin-Key when configured.

GET /api/data/sources: filter, sort, paginate registered sources.
GET /api/data/series/{code}: observations for one series; optional country and year range.
GET /api/data/series/{code}/summary: means, extremes, CAGR, latest YoY change.
GET /api/data/series/{code}/persistence: AR(1), half-life, jump ratio for one country.
GET /api/data/series/{code}/features: YoY growth, rolling vol, shock flags.
GET /api/data/series/{code}/comparators: cross-country ranking for the series.
GET /api/data/country/{iso3}: all series for one economy; optional topic or year.
GET /api/data/country/{iso3}/topics: coverage by topic.
GET /api/data/panel: long-format multi-series extract with balancedness diagnostics.
Operations

What operators should know

Seventy-one collectors across four engines. Some sources (FAO bulk, USDA PSD) deliver large payloads. The client spaces requests to respect rate limits. For production, set a real admin secret, decide whether metrics scrape is internal-only, and back up the SQLite file with a quiet window or the SQLite backup API.

The exhaustive manual (schema DDL, every query parameter, troubleshooting, deployment notes) lives in the repo:

ceres/README.md
Run

Start the observatory

From the ceres/ directory:

make setup && make collect-all && make serve

Integration tests against live APIs:

CERES_LIVE_API=1 uv run pytest tests/integration -q

Then open http://127.0.0.1:8008/docs or call /api/data/sources.