diff --git a/CHANGELOG.md b/CHANGELOG.md index 99fd41d..4d64f25 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,6 +2,18 @@ ## 1.5.3 - 2024-09 +- (Jon) Updated the application exceptions controller to instrument the + `ActiveSupport::Notifications` for internal errors + [GH-139](https://github.com/epimorphics/standard-reports-ui/issues/139) +- (Jon) Updated `config/initializers/prometheus.rb` to include the `Middleware + instrumentation` fix for the 0 memory bug by notifying Action Dispatch + subscribers on Prometheus initialise + [GH-139](https://github.com/epimorphics/standard-reports-ui/issues/139) +- (Jon) Updated `config/puma.rb` to include metrics plugin and port information + for the metrics endpoint as environment variable, with default, to enable + running multiple sibling HMLR apps locally if needed without port conflicts + [GH-139](https://github.com/epimorphics/standard-reports-ui/issues/139) +- (Jon) Updated the `lr_common_styles` gem to the latest 1.9.9 patch release. - (Jon) Moved all mirrored configuration settings from individual environments into the application configuration to reduce the need to manage multiple sources of truth diff --git a/Gemfile.lock b/Gemfile.lock index 70dde98..f412b4d 100644 --- a/Gemfile.lock +++ b/Gemfile.lock @@ -367,7 +367,7 @@ GEM json lograge railties - lr_common_styles (1.9.8) + lr_common_styles (1.9.9) bootstrap-sass (~> 3.4.0) font-awesome-rails (~> 4.7.0.1) govuk_elements_rails (~> 2.0.0) diff --git a/README.md b/README.md index c057e42..08d086e 100644 --- a/README.md +++ b/README.md @@ -14,252 +14,4 @@ Please see the other repositories in the [HM Land Registry Open Data](https://github.com/epimorphics/hmlr-linked-data/) project for more details. -## Running the Standard Reports Manager locally - -The application connects to the Standard Reports Manager service to submit -requests. - -The easiest way to do this is as a local docker container. The image can be -built from [standard reports -manager](https://github.com/epimorphics/standard-reports-manager/) repository. -or pulled from the Amazon Elastic Container Registry -[ECR](https://eu-west-1.console.aws.amazon.com/ecr/repositories/private/018852084843/epimorphics/standard-reports-manager/dev?region=eu-west-1). - -### Building and running from [standard reports manager](https://github.com/epimorphics/standard-reports-manager/) repository - -To build and a run a new docker image check out the [standard reports -manager](https://github.com/epimorphics/standard-reports-manager/) repository -and run - -```sh -make image run -``` - -### Running an existing [ECR](https://eu-west-1.console.aws.amazon.com/ecr/repositories/private/018852084843/epimorphics/standard-reports-manager/dev?region=eu-west-1) image - -Obtaining an ECR image requires: - -- AWS IAM credentials to connect to the HMLR AWS account -- the ECR credentials helper installed locally (see - [here](https://github.com/awslabs/amazon-ecr-credential-helper)) -- Set the contents of your `~/.docker/config.json` file to be: - -```sh -{ - "credsStore": "ecr-login" -} -``` - -This configures the Docker daemon to use the credential helper for all Amazon -ECR registries. - -To use a credential helper for a specific ECR registry[^1], create a -`credHelpers` section with the URI of your ECR registry: - -```sh -{ - [...] - "credHelpers": { - "public.ecr.aws": "ecr-login", - "018852084843.dkr.ecr.eu-west-1.amazonaws.com": "ecr-login" - } -} -``` - -Once you have a local copy of you required image, it is advisable to run a local -docker bridge network to mirror production and development environments. - -Running a client application as a docker image from their respective `Makefile`s -will set this up automatically, but to confirm run - -```sh -docker network inspect dnet -``` - -To create the docker network run - -```sh -docker network create dnet -``` - -### Running as a docker container - -Take a copy of the latest Manager development configuation file - -```sh -wget -O test/fixtures/conf/app.conf https://raw.githubusercontent.com/epimorphics/standard-reports-manager/master/dev/app.conf -``` - -```sh -docker run --network dnet -p 8081:8080 --rm --name standard-reports-manager \ - -v $(pwd)/test/fixtures/conf/app.conf:/etc/standard-reports/app.conf \ - 018852084843.dkr.ecr.eu-west-1.amazonaws.com/epimorphics/standard-reports-manager/dev:0.1.1_5ebbea4_00000030 -``` - -the latest image can be found here -[dev](https://github.com/epimorphics/hmlr-ansible-deployment/blob/master/ansible/group_vars/dev/tags.yml) -and -[production](https://github.com/epimorphics/hmlr-ansible-deployment/blob/master/ansible/group_vars/prod/tags.yml). - -The full list of versions can be found at [AWS -ECR](https://eu-west-1.console.aws.amazon.com/ecr/repositories/private/018852084843/epimorphics/standard-reports-manager/dev?region=eu-west-1) - -Note: port 8080 should be avoided to allow for a reverse proxy to run on this -port. - -With this set up, the api service is available on `http://localhost:8081` from -the host or `http://standard-reports-manager:8080` from inside other docker containers. - -## Running the app - -This application can be run stand-alone as a rails server in `development` mode. -However, when deployed, applications will run behind a reverse proxy. - -This enables request to be routed to the appropriate application base on the -request path. In order to simplifiy the proxy configuration we retain the -original path where possible. - -For information on how to running a proxy to mimic production and run multple -services together read through the information in our -[simple-web-proxy](https://github.com/epimorphics/simple-web-proxy/edit/main/README.md) -repository. - -If running more than one application locally ensure that each is listerning on a -separate port and separate path. In the case of running local docker images, the -required configuration is captured in the `Makefile`. - -### Locally - -For developing rails applications you can start the server locally using the -following command: - -```sh -rails server -``` - -and visit in your browser. - -To change to using `production` mode use the `-e` option; or to change to a -different port use the `-p` option. - -Note: In `production` mode, `SECRET_KEY_BASE` is also required. It is -insufficient to just set this as the value must be exported. e.g. - -```sh -export SECRET_KEY_BASE=$(./bin/rails secret) -``` - -#### Running Rails as a server with a sub-directory via Makefile - -```sh -API_SERVICE_URL= RAILS_ENV= RAILS_RELATIVE_URL_ROOT=/ make server -``` - -The default for `RAILS_ENV` here is `development`. - -### As a docker container - -It can be useful to run the compiled Docker image, that will mirror the -production installation, locally yourself. Assuming you have the [Standard -Reports Manager running](#running-the-standard-reports-manager-locally), then -you can run the Docker image for the app itself as follows: - -```sh -make image run -``` - -or, if the image is already built, simply - -```sh -make run -``` - -Docker images run in `production` mode. - -To test the running application visit `localhost:/`. - -### Development and Production mode - -Applications running in `development` mode default to *not* use a sub-directory -to aid stand-alone development. - -Applications running in `production` mode *do* use a sub-directory i.e. -`/app/standard-reports`. - -In each case the is achieved by setting `config.relative_url_root` property to -this sub-directory within the file -`config/environments/(development|production).rb`. - -If need be, `config.relative_url_root` may by overridden by means of the -`RAILS_RELATIVE_URL_ROOT` environment variable, althought this could also -require rebuilding the assets or docker image. - -## Additional Information - -### Coding standards - -Rubocop should complete with no warnings. - -### Tests - -Simply: - -```sh -API_SERVICE_URL=http://localhost:8081 bundle exec rails test -``` - -Passing in the `API_SERVICE_URL` is required to ensure the tests run against the -`standard-reports-manager` service running locally. - -### Issues - -Please add issues to the [shared issues -list](https://github.com/epimorphics/hmlr-linked-data/issues) - -### Runtime Configuration environment variables - -We use a number of environment variables to determine the runtime behaviour of -the application: - -| name | description | default value | -| -------------------------- | ----------------------------------------------------------------------- | -------------------------- | -| `API_SERVICE_URL` | The base URL from which data is accessed, including the HTTP scheme eg. | None | -| | if running a `standard-reports-manager service` locally | | -| | if running a `standard-reports-manager docker` image locally | | -| `SECRET_KEY_BASE` | See [description](https://api.rubyonrails.org/classes/Rails/Application.html#method-i-secret_key_base). | | -| | For `development` mode a acceptable value is already configured, in production mode this should be set to the output of `rails secret`. | | -| | This is handled automatically when starting a docker container, or the `server` `make` target | | -| `SENTRY_API_KEY` | The DSN for sending reports to the PPD Sentry account | None | - -### Deployment - -The detailed deployment mapping is described in `deployment.yml`. At the time of -writing, using the new infrastructure, the deployment process is as follows: - -- commits to the `dev-infrastructure` branch will deploy the dev server -- commits to the `preprod` branch will deploy the pre-production server -- any commit on the `prod` branch will deploy the production server as a new - release - -If the commit is a "new" release, the deployment should be tagged with the same -semantic version number matching the `BREAKING.FEATURE.PATCH` format, e.g. -`v1.2.3`, the same as should be set in the `/app/lib/version.rb`; also, a short -annotation summarising the updates should be included in the tag as well. - -Once the production deployment has been completed and verified, please create a -release on the repository using the same semantic version number. Utilise the -`Generate release notes from commit log` option to create specific notes on the -contained changes as well as the ability to diff agains the previous version. - -#### `entrypoint.sh` features - -- Workaround to removing the PID lock of the Rails process in the event of the - application crashing and not releasing the process. -- Guards to ensure the required environment variables are set accordingly and - trigger the build to fail noisily and log to the system. -- Rails secret creation for `SECRET_KEY_BASE` assignment; see [Runtime - Configuration environment - variables](#runtime-configuration-environment-variables). - -[^1]: With Docker 1.13.0 or greater, you can configure Docker to use different -credential helpers for different registries. +For more information about this project visit [the wiki](https://github.com/epimorphics/standard-reports-ui/wiki). diff --git a/app/controllers/application_controller.rb b/app/controllers/application_controller.rb index 36f271f..31e9aca 100644 --- a/app/controllers/application_controller.rb +++ b/app/controllers/application_controller.rb @@ -27,6 +27,9 @@ def log_request_result # or attempt to render a generic error page if no specific error page exists unless Rails.application.config.consider_all_requests_local rescue_from StandardError do |e| + # Instrument ActiveSupport::Notifications for internal errors: + ActiveSupport::Notifications.instrument('internal_error.application', exception: e) + # Trigger the appropriate error handling method based on the exception case e.class when ActionController::RoutingError, ActionView::MissingTemplate :render404 @@ -41,10 +44,7 @@ def log_request_result end def handle_internal_error(exception) - # Notify subscribers of the internal error event and render the appropriate error page - # or attempt to render a generic error page if no specific error page exists - # unless the exception is a 404, in which case do nothing - instrument_internal_error(exception) unless exception.status == 404 + # Render the appropriate error page based on the exception if exception.instance_of? ArgumentError render_error(400) else @@ -118,13 +118,4 @@ def detailed_request_log(duration) end end # rubocop:enable Metrics/AbcSize, Metrics/MethodLength - - # Notify subscriber(s) of an internal error event with the payload of the - # exception once done - # @param [Exception] exp the exception that caused the error - # @return [ActiveSupport::Notifications::Event] provides an object-oriented - # interface to the event - def instrument_internal_error(exception) - ActiveSupport::Notifications.instrument('internal_error.application', exception: exception) - end end diff --git a/config/initializers/prometheus.rb b/config/initializers/prometheus.rb index 5691075..648ed3b 100644 --- a/config/initializers/prometheus.rb +++ b/config/initializers/prometheus.rb @@ -48,3 +48,7 @@ docstring: 'Histogram of response times for API requests', buckets: Prometheus::Client::Histogram.exponential_buckets(start: 0.005, count: 16) ) + +# Middleware instrumentation + # This fixes the 0 memory bug by notifying Action Dispatch subscribers on Prometheus initialise + ActiveSupport::Notifications.instrument('process_middleware.action_dispatch') diff --git a/config/puma.rb b/config/puma.rb index 7bdabbf..378775d 100644 --- a/config/puma.rb +++ b/config/puma.rb @@ -10,9 +10,13 @@ min_threads_count = ENV.fetch('RAILS_MIN_THREADS', max_threads_count) threads min_threads_count, max_threads_count -# Specifies the `port` that Puma will listen on to receive requests; default is 3000. -# -port ENV.fetch('PORT', 3000) +# Specifies the `port` that Puma will listen on to receive requests; +# default is 3000. +port ENV.fetch('PORT', 3000) + +# Specifies the `metrics_port` that Puma will listen on to export metrics; +# default is 9393. +metrics_port = ENV.fetch('METRICS_PORT', 9393) # Specifies the `environment` that Puma will run in. # @@ -36,9 +40,13 @@ # # preload_app! +# Enable the metrics plugin to export Puma's internal statistics as Prometheus metrics +plugin :metrics +# Bind the metric server to "url". "tcp://" is the only accepted protocol. +metrics_url "tcp://0.0.0.0:#{metrics_port}" if Rails.env.development? + # Allow puma to be restarted by `rails restart` command. plugin :tmp_restart -plugin :metrics # Use a custom log formatter to emit Puma log messages in a JSON format log_formatter do |str|