For technical teams
A thin Django application acts as the API, the notebook index, and the job orchestrator. Celery workers do the heavy execution; MyST renders to static HTML; object storage distributes it. It’s a stack the IFRC GO team already builds and runs.
Django holds metadata only. Templates, run records and links live in Postgres — never notebook content.
Outputs bypass the app. Static HTML is served straight from object storage, so the app is never in the hot path for reads.
Workers do the heavy lifting. papermill executes; MyST renders; data pulls & processing happen here, once per run.
Control flow
Two paths, same pipeline: a user requests a run, or the system starts one from a new event.
Technology choices
The API, notebook index and orchestrator. Celery manages the job queue; workers scale on k8s. Why: this is exactly the stack the IFRC GO team builds and maintains — well-understood, well-integrated, low long-term maintenance risk.
Executes a notebook with injected parameters, producing a fully-run notebook. Why: the standard tool for parameterized notebook execution — turns a template + a parameter set into a concrete run.
Renders the executed notebook to a clean, self-contained static HTML page. Why: high-quality notebook → web rendering, and it carries interactive widgets into the static export.
The visualization layer: fast maps (lonboard) and composable widgets (anywidget / manywidgets). Why: they render interactively in static HTML with no kernel — sliders, toggles and filters keep working on the published page.
Holds the published HTML + assets, served directly (and cacheable via CDN). Why: cheap, durable, infinitely scalable reads; keeps the app and DB out of the read path.
Montandon is the entry point to correlated hazard, impact and EO data, plus historical records; other sources (INFORM, CEMS, WorldPop, OSM…) are pulled by the worker. Why: a deliberate, heavy reliance on Montandon as the harmonised backbone.
A template is a recipe. The app takes the recipe plus your choices, hands it to a worker that cooks it once, and puts the finished dish on a shelf (object storage) that anyone can pick up. The app just keeps the menu and the labels.
Data model
Light metadata and links — enough to orchestrate runs and power discovery. Never the notebooks themselves.
The template: slug, title, parameter_schema, source repo & version. One template → many runs.
One execution: the parameters used, status, output_url, timestamps, and who/what triggered it. Links to an area, a hazard and (optionally) an event.
A disaster or CEMS/Charter activation — monty_corr_id, hazard type, country, source, time. Ties auto-runs (and UC3 write-back) to the real event.
Honest engineering
The pieces around notebook generation and static publishing are well-trodden. The genuinely experimental part is last-mile visualization.
Raster layers in static HTML. Some analyses need raster / x-y-z tile layers rendered in the exported page. That requires the tiles to be served from somewhere; it’s the area most likely to need path-finding on real use cases. Best de-risked early with actual data.
Execution time & cost. Ingesting multiple large data sources per run can be slow. This is a known Jupyter problem-space with many knobs — caching, resolution, pre-processed inputs, worker sizing.
Once the basic structure exists, natural extensions come cheaply: on-demand runs in new countries / with different parameters, adding arbitrary analysis notebooks, and feeding EOC field observations in as one more signal for UC3 — without changing the method.