Skip to content

Commit

Permalink
Merge pull request #141 from epimorphics/dev
Browse files Browse the repository at this point in the history 
Release Candidate v1.5.4
  • Loading branch information
@bogdanadrianmarc
bogdanadrianmarc authored Sep 24, 2024
2 parents 04d380a + 9867fcc commit ef0babb
Show file tree
Hide file tree
Showing 6 changed files with 34 additions and 267 deletions.
12 changes: 12 additions & 0 deletions CHANGELOG.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,18 @@

## 1.5.3 - 2024-09

- (Jon) Updated the application exceptions controller to instrument the
`ActiveSupport::Notifications` for internal errors
[GH-139](https://github.com/epimorphics/standard-reports-ui/issues/139)
- (Jon) Updated `config/initializers/prometheus.rb` to include the `Middleware
instrumentation` fix for the 0 memory bug by notifying Action Dispatch
subscribers on Prometheus initialise
[GH-139](https://github.com/epimorphics/standard-reports-ui/issues/139)
- (Jon) Updated `config/puma.rb` to include metrics plugin and port information
for the metrics endpoint as environment variable, with default, to enable
running multiple sibling HMLR apps locally if needed without port conflicts
[GH-139](https://github.com/epimorphics/standard-reports-ui/issues/139)
- (Jon) Updated the `lr_common_styles` gem to the latest 1.9.9 patch release.
- (Jon) Moved all mirrored configuration settings from individual environments
into the application configuration to reduce the need to manage multiple
sources of truth
Expand Down
2 changes: 1 addition & 1 deletion Gemfile.lock
View file
Original file line number Diff line number Diff line change
Expand Up @@ -367,7 +367,7 @@ GEM
json
lograge
railties
lr_common_styles (1.9.8)
lr_common_styles (1.9.9)
bootstrap-sass (~> 3.4.0)
font-awesome-rails (~> 4.7.0.1)
govuk_elements_rails (~> 2.0.0)
Expand Down
250 changes: 1 addition & 249 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -14,252 +14,4 @@ Please see the other repositories in the [HM Land Registry Open
Data](https://github.com/epimorphics/hmlr-linked-data/) project for more
details.

## Running the Standard Reports Manager locally

The application connects to the Standard Reports Manager service to submit
requests.

The easiest way to do this is as a local docker container. The image can be
built from [standard reports
manager](https://github.com/epimorphics/standard-reports-manager/) repository.
or pulled from the Amazon Elastic Container Registry
[ECR](https://eu-west-1.console.aws.amazon.com/ecr/repositories/private/018852084843/epimorphics/standard-reports-manager/dev?region=eu-west-1).

### Building and running from [standard reports manager](https://github.com/epimorphics/standard-reports-manager/) repository

To build and a run a new docker image check out the [standard reports
manager](https://github.com/epimorphics/standard-reports-manager/) repository
and run

```sh
make image run
```

### Running an existing [ECR](https://eu-west-1.console.aws.amazon.com/ecr/repositories/private/018852084843/epimorphics/standard-reports-manager/dev?region=eu-west-1) image

Obtaining an ECR image requires:

- AWS IAM credentials to connect to the HMLR AWS account
- the ECR credentials helper installed locally (see
[here](https://github.com/awslabs/amazon-ecr-credential-helper))
- Set the contents of your `~/.docker/config.json` file to be:

```sh
{
"credsStore": "ecr-login"
}
```

This configures the Docker daemon to use the credential helper for all Amazon
ECR registries.

To use a credential helper for a specific ECR registry[^1], create a
`credHelpers` section with the URI of your ECR registry:

```sh
{
[...]
"credHelpers": {
"public.ecr.aws": "ecr-login",
"018852084843.dkr.ecr.eu-west-1.amazonaws.com": "ecr-login"
}
}
```

Once you have a local copy of you required image, it is advisable to run a local
docker bridge network to mirror production and development environments.

Running a client application as a docker image from their respective `Makefile`s
will set this up automatically, but to confirm run

```sh
docker network inspect dnet
```

To create the docker network run

```sh
docker network create dnet
```

### Running as a docker container

Take a copy of the latest Manager development configuation file

```sh
wget -O test/fixtures/conf/app.conf https://raw.githubusercontent.com/epimorphics/standard-reports-manager/master/dev/app.conf
```

```sh
docker run --network dnet -p 8081:8080 --rm --name standard-reports-manager \
-v $(pwd)/test/fixtures/conf/app.conf:/etc/standard-reports/app.conf \
018852084843.dkr.ecr.eu-west-1.amazonaws.com/epimorphics/standard-reports-manager/dev:0.1.1_5ebbea4_00000030
```

the latest image can be found here
[dev](https://github.com/epimorphics/hmlr-ansible-deployment/blob/master/ansible/group_vars/dev/tags.yml)
and
[production](https://github.com/epimorphics/hmlr-ansible-deployment/blob/master/ansible/group_vars/prod/tags.yml).

The full list of versions can be found at [AWS
ECR](https://eu-west-1.console.aws.amazon.com/ecr/repositories/private/018852084843/epimorphics/standard-reports-manager/dev?region=eu-west-1)

Note: port 8080 should be avoided to allow for a reverse proxy to run on this
port.

With this set up, the api service is available on `http://localhost:8081` from
the host or `http://standard-reports-manager:8080` from inside other docker containers.

## Running the app

This application can be run stand-alone as a rails server in `development` mode.
However, when deployed, applications will run behind a reverse proxy.

This enables request to be routed to the appropriate application base on the
request path. In order to simplifiy the proxy configuration we retain the
original path where possible.

For information on how to running a proxy to mimic production and run multple
services together read through the information in our
[simple-web-proxy](https://github.com/epimorphics/simple-web-proxy/edit/main/README.md)
repository.

If running more than one application locally ensure that each is listerning on a
separate port and separate path. In the case of running local docker images, the
required configuration is captured in the `Makefile`.

### Locally

For developing rails applications you can start the server locally using the
following command:

```sh
rails server
```

and visit <localhost:3000> in your browser.

To change to using `production` mode use the `-e` option; or to change to a
different port use the `-p` option.

Note: In `production` mode, `SECRET_KEY_BASE` is also required. It is
insufficient to just set this as the value must be exported. e.g.

```sh
export SECRET_KEY_BASE=$(./bin/rails secret)
```

#### Running Rails as a server with a sub-directory via Makefile

```sh
API_SERVICE_URL=<data-api url> RAILS_ENV=<mode> RAILS_RELATIVE_URL_ROOT=/<path> make server
```

The default for `RAILS_ENV` here is `development`.

### As a docker container

It can be useful to run the compiled Docker image, that will mirror the
production installation, locally yourself. Assuming you have the [Standard
Reports Manager running](#running-the-standard-reports-manager-locally), then
you can run the Docker image for the app itself as follows:

```sh
make image run
```

or, if the image is already built, simply

```sh
make run
```

Docker images run in `production` mode.

To test the running application visit `localhost:<port>/<application path>`.

### Development and Production mode

Applications running in `development` mode default to *not* use a sub-directory
to aid stand-alone development.

Applications running in `production` mode *do* use a sub-directory i.e.
`/app/standard-reports`.

In each case the is achieved by setting `config.relative_url_root` property to
this sub-directory within the file
`config/environments/(development|production).rb`.

If need be, `config.relative_url_root` may by overridden by means of the
`RAILS_RELATIVE_URL_ROOT` environment variable, althought this could also
require rebuilding the assets or docker image.

## Additional Information

### Coding standards

Rubocop should complete with no warnings.

### Tests

Simply:

```sh
API_SERVICE_URL=http://localhost:8081 bundle exec rails test
```

Passing in the `API_SERVICE_URL` is required to ensure the tests run against the
`standard-reports-manager` service running locally.

### Issues

Please add issues to the [shared issues
list](https://github.com/epimorphics/hmlr-linked-data/issues)

### Runtime Configuration environment variables

We use a number of environment variables to determine the runtime behaviour of
the application:

| name | description | default value |
| -------------------------- | ----------------------------------------------------------------------- | -------------------------- |
| `API_SERVICE_URL` | The base URL from which data is accessed, including the HTTP scheme eg. | None |
| | <http://localhost:8081> if running a `standard-reports-manager service` locally | |
| | <http://standard-reports-manager:8080> if running a `standard-reports-manager docker` image locally | |
| `SECRET_KEY_BASE` | See [description](https://api.rubyonrails.org/classes/Rails/Application.html#method-i-secret_key_base). | |
| | For `development` mode a acceptable value is already configured, in production mode this should be set to the output of `rails secret`. | |
| | This is handled automatically when starting a docker container, or the `server` `make` target | |
| `SENTRY_API_KEY` | The DSN for sending reports to the PPD Sentry account | None |

### Deployment

The detailed deployment mapping is described in `deployment.yml`. At the time of
writing, using the new infrastructure, the deployment process is as follows:

- commits to the `dev-infrastructure` branch will deploy the dev server
- commits to the `preprod` branch will deploy the pre-production server
- any commit on the `prod` branch will deploy the production server as a new
release

If the commit is a "new" release, the deployment should be tagged with the same
semantic version number matching the `BREAKING.FEATURE.PATCH` format, e.g.
`v1.2.3`, the same as should be set in the `/app/lib/version.rb`; also, a short
annotation summarising the updates should be included in the tag as well.

Once the production deployment has been completed and verified, please create a
release on the repository using the same semantic version number. Utilise the
`Generate release notes from commit log` option to create specific notes on the
contained changes as well as the ability to diff agains the previous version.

#### `entrypoint.sh` features

- Workaround to removing the PID lock of the Rails process in the event of the
application crashing and not releasing the process.
- Guards to ensure the required environment variables are set accordingly and
trigger the build to fail noisily and log to the system.
- Rails secret creation for `SECRET_KEY_BASE` assignment; see [Runtime
Configuration environment
variables](#runtime-configuration-environment-variables).

[^1]: With Docker 1.13.0 or greater, you can configure Docker to use different
credential helpers for different registries.
For more information about this project visit [the wiki](https://github.com/epimorphics/standard-reports-ui/wiki).
17 changes: 4 additions & 13 deletions app/controllers/application_controller.rb
View file
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,9 @@ def log_request_result
# or attempt to render a generic error page if no specific error page exists
unless Rails.application.config.consider_all_requests_local
rescue_from StandardError do |e|
# Instrument ActiveSupport::Notifications for internal errors:
ActiveSupport::Notifications.instrument('internal_error.application', exception: e)
# Trigger the appropriate error handling method based on the exception
case e.class
when ActionController::RoutingError, ActionView::MissingTemplate
:render404
Expand All @@ -41,10 +44,7 @@ def log_request_result
end

def handle_internal_error(exception)
# Notify subscribers of the internal error event and render the appropriate error page
# or attempt to render a generic error page if no specific error page exists
# unless the exception is a 404, in which case do nothing
instrument_internal_error(exception) unless exception.status == 404
# Render the appropriate error page based on the exception
if exception.instance_of? ArgumentError
render_error(400)
else
Expand Down Expand Up @@ -118,13 +118,4 @@ def detailed_request_log(duration)
end
end
# rubocop:enable Metrics/AbcSize, Metrics/MethodLength

# Notify subscriber(s) of an internal error event with the payload of the
# exception once done
# @param [Exception] exp the exception that caused the error
# @return [ActiveSupport::Notifications::Event] provides an object-oriented
# interface to the event
def instrument_internal_error(exception)
ActiveSupport::Notifications.instrument('internal_error.application', exception: exception)
end
end
4 changes: 4 additions & 0 deletions config/initializers/prometheus.rb
View file
Original file line number Diff line number Diff line change
Expand Up @@ -48,3 +48,7 @@
docstring: 'Histogram of response times for API requests',
buckets: Prometheus::Client::Histogram.exponential_buckets(start: 0.005, count: 16)
)

# Middleware instrumentation
# This fixes the 0 memory bug by notifying Action Dispatch subscribers on Prometheus initialise
ActiveSupport::Notifications.instrument('process_middleware.action_dispatch')
16 changes: 12 additions & 4 deletions config/puma.rb
View file
Original file line number Diff line number Diff line change
Expand Up @@ -10,9 +10,13 @@
min_threads_count = ENV.fetch('RAILS_MIN_THREADS', max_threads_count)
threads min_threads_count, max_threads_count

# Specifies the `port` that Puma will listen on to receive requests; default is 3000.
#
port ENV.fetch('PORT', 3000)
# Specifies the `port` that Puma will listen on to receive requests;
# default is 3000.
port ENV.fetch('PORT', 3000)

# Specifies the `metrics_port` that Puma will listen on to export metrics;
# default is 9393.
metrics_port = ENV.fetch('METRICS_PORT', 9393)

# Specifies the `environment` that Puma will run in.
#
Expand All @@ -36,9 +40,13 @@
#
# preload_app!

# Enable the metrics plugin to export Puma's internal statistics as Prometheus metrics
plugin :metrics
# Bind the metric server to "url". "tcp://" is the only accepted protocol.
metrics_url "tcp://0.0.0.0:#{metrics_port}" if Rails.env.development?

# Allow puma to be restarted by `rails restart` command.
plugin :tmp_restart
plugin :metrics

# Use a custom log formatter to emit Puma log messages in a JSON format
log_formatter do |str|
Expand Down

0 comments on commit ef0babb

Please sign in to comment.