DC Data Baker

Orchestration for writing files to s3 to power our APIs

Install

Python dependencies

This project uses a pip tools style workflow. Use uv or pip-tools. Commands given using uv. See uv equivalence for comparison.

uv venv --prompt . .venv
uv pip sync requirements/dev.txt

Commit hooks

pre-commit install

Javascript dependencies

We're using CDK so you need the cdk tooling.

Probably best to use nvm to manage npm to manage installing libraries.

Then

nvm use --lts
npm install

Managing dependencies

uv equivalence

• uv pip compile is equivalent to pip-compile
• uv pip sync is equivalent to pip-sync
• uv venv is equivalent to python -m venv

Generate requirements/*.txt files

base.txt

uv pip compile --generate-hashes --output-file requirements/base.txt requirements/base.in

dev.txt

uv pip compile --generate-hashes --output-file requirements/dev.txt requirements/base.in requirements/dev.in

Upgrade all dependencies

• Run uv pip compile --generate-hashes --upgrade --output-file requirements/base.txt requirements/base.in

Upgrade a single dependency

• Run uv pip compile --generate-hashes --upgrade-package <pkg-name> --output-file requirements/base.txt requirements/base.in

Adding a dependency

base dependencies

• Add a dependency to requirements/base.in
• Run uv pip compile --generate-hashes --output-file requirements/base.txt requirements/base.in

dev dependencies

• Add a dependency to requirements/dev.in
• Run uv pip compile --generate-hashes --output-file requirements/dev.txt requirements/base.in requirements/dev.in

CDK Setup

AWS_PROFILE=dev-aggregatorapi-dc DC_ENVIRONMENT=development cdk bootstrap

AWS_PROFILE=dev-aggregatorapi-dc DC_ENVIRONMENT=development cdk synth