Deploy CloudFront distributions in front of CopWatch

[copelco]

Deploy CloudFront distribution for app server.

https://app.clickup.com/t/86897a79g

TODO

• Deploy staging CDN
• Remove django-cache-machine
  • Remove cache_response
  • Remove caching package (django-cache-machine), imports, and referenced code
• Temporarily deploy this branch to staging (edit github workflow)
• Update cache primer to (1) query actual site and (2) queue a task per agency to speed it up
• Determine if we need intermediate Let's Encrypt certs for origin hostnames
• Determine how to deploy CDN CloudFormation stacks alongside K8s CloudFormation stack
• Determine how to handle https://nccopwatch.org/resources pages if updated via Admin
• Deploy production CDN

Related PRs (need to be merged)

• Add optional CloudFront distribution for app server aws-web-stacks#35
• Set public_access via amazon.aws.s3_bucket module ansible-role-aws-web-stacks#3

[copelco]

Task linked: CU-86897a79g Cache primed pages routinely are cache misses

[copelco]

Testing notes

Look for x-cache header when requesting pages through CloudFront:

curl --user "forwardjustice:trafficstops" --head https://staging.nccopwatch.org/api/agency/80/stops/                                                          
HTTP/2 200 
content-type: application/json
content-length: 2406
date: Mon, 21 Oct 2024 14:13:13 GMT
allow: GET, HEAD, OPTIONS
x-frame-options: DENY
content-language: en
x-content-type-options: nosniff
x-xss-protection: 1; mode=block
referrer-policy: same-origin
strict-transport-security: max-age=15724800; includeSubDomains
vary: Accept-Encoding
x-cache: Hit from cloudfront
via: 1.1 771410cef18c0261fb48d1f92756f9e2.cloudfront.net (CloudFront)
x-amz-cf-pop: ATL59-P1
x-amz-cf-id: tSl4WzkS7MEz4qRCtuJmX7OIqFq5pWSfwfdD0T1VMbARB13_7cCFcA==
age: 3300

An example query of the origin directly:

curl --user "forwardjustice:trafficstops" --insecure --header "Host: staging.nccopwatch.org" --head https://staging-origin.nccopwatch.org/api/agency/80/stops/
HTTP/2 200 
date: Mon, 21 Oct 2024 15:10:12 GMT
content-type: application/json
content-length: 2406
vary: Accept, Accept-Language
allow: GET, HEAD, OPTIONS
x-frame-options: DENY
content-language: en
x-content-type-options: nosniff
x-xss-protection: 1; mode=block
referrer-policy: same-origin
strict-transport-security: max-age=15724800; includeSubDomains

Remove --head to see the JSON response.

Clear cache

Create invalidation on staging:

aws cloudfront create-invalidation --distribution-id E2OFFI0H5HY2N8 --paths "/*"

[ronardcaktus]

Slow queries with time

15 seconds

> time curl --user "forwardjustice:trafficstops" --insecure --header "Host: staging.nccopwatch.org" --head https://staging-origin.nccopwatch.org/api/agency/-1/search-rate/
HTTP/2 200 
date: Thu, 24 Oct 2024 14:51:19 GMT
content-type: application/json
content-length: 1793
vary: Accept, Accept-Language
allow: GET, HEAD, OPTIONS
expires: Mon, 23 Dec 2024 14:51:19 GMT
cache-control: max-age=5184000
x-frame-options: DENY
content-language: en
x-content-type-options: nosniff
x-xss-protection: 1; mode=block
referrer-policy: same-origin
strict-transport-security: max-age=15724800; includeSubDomains

curl --user "forwardjustice:trafficstops" --insecure --header  --head   0.00s user 0.01s system 0% cpu 15.756 total