Xpublish

Publish Xarray Datasets to the web

A quick example

Serverside: Publish a Xarray Dataset through a rest API

ds.rest.serve(host="0.0.0.0", port=9000)

Client-side: Connect to a published dataset

The published datasets can be accessed from various kinds of client applications, e.g., from within Python using Zarr and fsspec.

import xarray as xr
import zarr
from fsspec.implementations.http import HTTPFileSystem

fs = HTTPFileSystem()
http_map = fs.get_mapper("http://0.0.0.0:9000/zarr/")

# open as a zarr group
zg = zarr.open_consolidated(http_map, mode="r")

# or open as another Xarray Dataset
ds = xr.open_zarr(http_map, consolidated=True)

Or to explore other access methods, open http://0.0.0.0:9000/docs in a browser.

Why?

Xpublish lets you serve/share/publish Xarray Datasets via a web application.

The data and/or metadata in the Xarray Datasets can be exposed in various forms through pluggable REST API endpoints. Efficient, on-demand delivery of large datasets may be enabled with Dask on the server-side.

Xpublish's plugin ecosystem has capabilities including:

• publish on-demand or derived data products
• turning xarray objects into streaming services (e.g. OPeNDAP)

How?

Under the hood, Xpublish is using a web app (FastAPI) that is exposing a REST-like API with builtin and/or user-defined endpoints.

For example, Xpublish provides by default a minimal Zarr compatible REST-like API with the following endpoints:

• zarr/.zmetadata: returns Zarr-formatted metadata keys as json strings.
• zarr/var/0.0.0: returns a variable data chunk as a binary string.

Futher endpoints can be added by installing or writing plugins.