From e34da001d78dba172730e61d69355caa14ad8356 Mon Sep 17 00:00:00 2001 From: Jeremy Croteau Date: Fri, 13 Sep 2024 14:48:10 -0400 Subject: [PATCH] feature/APPEALS-34124-43428-29105-28925-33581 - Rails 6.1 upgrade (release) (#22813) (#22817) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * πŸ”§ Assume defaults for `config.action_dispatch.use_cookies_with_metadata` and `config.action_mailer.delivery_job` The following config settings are not backwards compatible: - config.action_dispatch.use_cookies_with_metadata - config.action_mailer.delivery_job Now that Rails 6.0 is stable on production, we can assume their default values going forward. * βœ… Fix flakey spec * πŸ”§ Assume default for `config.action_dispatch.use_authenticated_cookie_encryption` Since we are making other cookie configuration changes in this PR for Rails 6.0, this is an opportune time to migrate this Rails 5.2 cookie setting to its default value as well. * βͺ️ Restore overrides for `config.action_dispatch.use_authenticated_cookie_encryption` and `config.action_dispatch.use_cookies_with_metadata` While testing in PreProd, we discovered that, without these cookie config overrides, re-authentication was broken -- after logging out, a user could not log back in. Since the default settings are still optional going forward, we can restore these overrides and devise a solution to migrate cookies later. For more details, see Jira story APPEALS-54897: https://jira.devops.va.gov/browse/APPEALS-54897 * ✨ Add new utility module for adding DB indexes concurrently Introduces `Caseflow::Migrations::AddIndexConcurrently` as a replacement for `Caseflow::Migration` for migrations on ActiveRecord 6.0 and beyond, since `Caseflow::Migration` is forever coupled to ActiveRecord 5.1 due to its extensive use on legacy migrations and should be deprecated moving forward. * πŸ—‘οΈ Deprecate `Caseflow::Migration` * πŸ”§ Add instructive error message for non-concurrent `add_index` migrations * 🚨 Address linter / codeclimate complaints * ✨ Introduce `SslRedirectExclusionPolicy` To be used in the environment configuration settings for excluding exempt request paths from SSL redirects when `config. force_ssl = true` * ♻️ Replace deprecated controller-level `force_ssl` Replace deprecated controller-level `force_ssl` with equivalent configuration settings in preparation for the Rails 6.1 upgrade. * πŸ”₯ Remove deprecated config setting `config.active_record.sqlite3.represent_boolean_as_integer` This will have no implications for Caseflow, since we are only using the `sqlite3` adapter nominally for the `demo_vacols` database, which is not actually being used in our demo environments (demo environments are deployed as `development` envs). * ⬆️ Update `caseflow-commons` to resolve sub-dependency conflicts Removes unneeded gems `bourbon` and `neat`, which had a sub-dependency conflict on `thor`. * ⬆️ Update rails and other gems as necessary * πŸ› Fix 'uninitialized constant' error when loading app * ⬆️ bin/rails app:update - Apply relevant changes * πŸ”§ Override default for `config.active_record.has_many_inversing` * πŸ”§ Assume default for `config.active_storage.track_variants` We're not currently using ActiveStorage in Caseflow, so it is safe to just assume the default here. * πŸ”§ Override default for `config.active_job.retry_jitter` The default jitter is probably safe, however, I'm not 100% sure that we don't have any jobs that need to be requeued with exact wait times. So we let's override this for now to stay on the safe side. * πŸ”§ Assume default for `config.active_job.skip_after_callbacks_if_terminated` We're not currently usingΒ `throw :abort`Β within anyΒ `before_enqueue`/`before_perform`Β  callbacks on existing Caseflow jobs, so the default should be fine here. For more background, see https://lilyreile.medium.com/rails-6-1-new-framework-defaults-what-they-do-and-how-to-safely-uncomment-them-c546b70f0c5e#4c60 * πŸ”§ Assume default for `config.action_dispatch.cookies_same_site_protection` This setting controls the `SameSite` optional attribute for the `Set-Cookie` header. `SameSite=Lax` means that the cookie is not sent on cross-site requests, such as on requests to load images or frames, but is sent when a user is navigating to the origin site from an external site (for example, when following a link). This is the default behavior if the SameSite attribute is not specified. `Lax` is currently the default assumed by both Chrome and Edge browsers when this attribute is left unspecified, so assuming this value should be sensible. It allows us to have our cake (blocking CSRF attacks) and eat it too (providing a logged-in experience when users navigate to Caseflow across origins). For more background, see - https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Set-Cookie#samesitesamesite-value - https://lilyreile.medium.com/rails-6-1-new-framework-defaults-what-they-do-and-how-to-safely-uncomment-them-c546b70f0c5e#1f15 * πŸ”§ Assume default for `config.action_controller.urlsafe_csrf_tokens` * πŸ”§ Assume default for `ActiveSupport.utc_to_local_returns_utc_offset_times` We're not using `ActiveSupport::TimeZone.utc_to_local` anywhere, so the default is safe to assume here. * πŸ”§ Assume default for `config.action_dispatch.ssl_default_redirect_status` The default is safe to assume. For more background, see https://lilyreile.medium.com/rails-6-1-new-framework-defaults-what-they-do-and-how-to-safely-uncomment-them-c546b70f0c5e#4c3e * πŸ”§ Assume default for `config.active_record.legacy_connection_handling` The default should be safe to assume here, as we do not do any role or shard switching on database connections. For more background, see https://lilyreile.medium.com/rails-6-1-new-framework-defaults-what-they-do-and-how-to-safely-uncomment-them-c546b70f0c5e#8007 * πŸ”§ Assume default for `config.action_view.form_with_generates_remote_forms` We don't use the `form_with` helper anywhere, so this behavior change is inconsequential for us, and we can safely assume the new default. * πŸ”§ Assume default for `config.active_storage.queues.analysis` We do not use ActiveStorage, so the default is safe to assume here. * πŸ”§ Assume default for `config.active_storage.queues.purge` We do not use ActiveStorage, so the default is safe to assume here. * πŸ”§ Assume default for `config.action_mailbox.queues.incineration` We don't use ActionMailbox, so the new default is safe to assume here. * πŸ”§ Assume default for `config.action_mailbox.queues.routing` We do not use ActionMailbox, so the default is safe to assume here. * πŸ”§ Assume default for `config.action_mailer.deliver_later_queue_name` We're not using `ActionMailer::MessageDelivery #deliver_later` anywhere, so the default is safe to assume. * πŸ”§ Assume default for `config.action_view.preload_links_header` This flag can be safely uncommented. Browsers that support Link headers will get a performance boost. Browsers that don’t will ignore them. We override in `development` environments to avoid an edge case leading to an HTTP response header overflow. For more background, see https://lilyreile.medium.com/rails-6-1-new-framework-defaults-what-they-do-and-how-to-safely-uncomment-them-c546b70f0c5e#3679 * πŸ”₯ Remove 'new_framework_defaults_6_1.rb' * πŸ”§ Load defaults for Rails 6.1 * ♻️ Extract constant * ♻️ Migrate to new Rails deprecation config where applicable * ♻️ Push members down now that there is only one subclass * 🩹 Add forgotten disallowed deprecation warning This deprecation warning was addressed by the following PR, but we forgot to add it to the list of disallowed deprecation warnings: https://github.com/department-of-veterans-affairs/caseflow/pull/21614 * πŸ’‘ Update comment Task `rake routes` has been replaced with `rails routes` * βœ… Update test to account for change to `ActionDispatch::Response#content_type`Β  `ActionDispatch::Response#content_type` now returns the full Content-Type header * 🚨 Exclude 'config.ru' from Rubocop cops * 🚚 Move 'db/etl/migrate' to 'db/etl_migrate' * 🚚 Move 'db/etl/schema.rb' to 'db/etl_schema.rb' * ♻️ Arrange 'database.yml' configs by environment Group DB configs by environment in anticipation of reformatting for Rails 6+ multi-DB configuration. * πŸ”§ Reformat 'database.yml' to Rails 6+ multi-DB conventions * πŸ”§ Add etl migration paths to DB config * πŸ”§ Update DB connection names in 'database_cleaner' config * ♻️ Use new database-specific rake tasks After migrating to the Rails 6+ native multi-database configuration, the behavior of some DB management tasks, such as `rake db:migrate` changed such that they now act on ALL databases and not just the primary database. So we must replace the invocations of these tasks with their new, database-specific counterparts. * βž– Remove 'multiverse' gem Now that we have fiully transitioned to Rails-native multi-database support, we are no longer reliant on the 'multiverse' gem and can remove it. * πŸ—ƒοΈ Prohibit execution of vacols DB and non-DB-specific rake tasks After transitioning to Rails-native multi-DB support, the behavior of some DB tasks changed such that they will now act on ALL databases and not just the primary database (ex. `rake db:migrate` will now migrate ALL databases). To avoid accidents, we re-define these tasks here to no-op and output a helpful message to redirect developers toward using their new database-specific counterparts instead. * ♻️ Create new environment for GH workflow 'Make-docs-to-webpage' Instead of performing a bunch of hard-to-maintain `sed` gymnastics to modify the existing 'test' environment, let's create a new 'make_docs' environment (based off of 'test') and configure it appropriately for use by the 'Make-docs-to-webpage' GH workflow. * πŸ’š Remove redundant DB migrations from CI workflow Task `db:schema:load` already loads the checked in schema, so there should be no need to run `db:migrate` afterwards. * πŸ› Fix `spec/mailers/hearing_mailer_spec.rb` - `NoMethodError` Addresses the following error: NoMethodError: undefined method `build_lookup_context' for ActionView::Base:Class * πŸ› Fix `spec/workflows/post_decision_motion_updater_spec.rb` - `FrozenError` Addresses the following error: FrozenError: can't modify frozen Hash: {} * βœ… Add test for `RoSchedulePeriod` * πŸ› Fix `spec/models/schedule_period_spec.rb` - `ActiveRecord::RecordInvalid` Apparently, there were some changes to the inner workings of `ActiveModel::Errors` in Rails 6.1, causing a model to be considered invalid in the case that `errors[:base] == [[]]`. This makes sense, as `[[]]` is not considered "empty". Unfortunately, this was causing `RoSchedulePeriod #validate_spreadsheet` to inadvertently mark the model as invalid upon creation. `HearingSchedule::ValidateRoSpreadsheet #validate` returns an empty array (`[]`) when valid, which gets pushes onto the `RoSchedulePeriod` `errors[:base]` array, resulting in a non-empty array (`[[]]`) and an erroneously invalid disposition. Furthermore, calling `<<` to an `ActiveModel::Errors` message array in order to add an error is a deprecated, so we can take this opportunity to use the new `#add` API to hit two birds with one stone. The change implemented here is not a pure refactoring, however the end-user experience is unchanged in terms of how errors are presented when attempting to upload a spreadsheet with multiple non-conformities. Down the road, we may want to consider moving `HearingSchedule::ValidateRoSpreadsheet` toward using `ActiveModel::Validations` in order to leverage the full `ActiveModel::Errors` API and construct the errors object in the prescribed manner. For more details see - https://api.rubyonrails.org/v6.1.7.7/classes/ActiveModel/Validations.html - https://api.rubyonrails.org/v6.1.7.7/classes/ActiveModel/Errors.html * πŸ› Fix `spec/mailers/hearing_mailer_spec.rb` - `ActionView::Template::Error` * βœ… Fix `spec/models/veteran_spec.rb` * βœ… Fix `spec/sql/ama_cases_sql_spec.rb` Addresses failures such as the below: 0) AMA Cases Tableau data source expected report calculates age and AOD based on person.dob Failure/Error: expect(aod_case["aod_veteran.age"]).to eq("76") expected: "76" got: 0.76e2 * βœ… Fix multiple specs - `Minitest::UnexpectedError` Test helper method `#perform_enqueued_jobs` now wraps exceptions in an `Minitest::UnexpectedError`: https://github.com/rails/rails/blob/914caca2d31bd753f47f9168f2a375921d9e91cc/activejob/lib/active_job/test_helper.rb#L591 So, to assert that a specific exception is raised during execution of the `#perform_enqueued_jobs` block, we must rescue the `Minitest::UnexpectedError` and make the assertion on its error message instead. * βœ… Fix `spec/lib/helpers/association_wrapper_spec.rb` * βœ… Fix `spec/controllers/api/v1/jobs_controller_spec.rb` In Rails 6.1, `ActiveJob #perform_now` was changed to behave as it did once before (at the behest of GitHub), returning the value fo the job instead of true/false. See related GH issue: https://github.com/rails/rails/issues/38040 * πŸ› Fix `spec/controllers/appeals_controller_spec.rb` - `NoMethodError` Addresses error: NoMethodError: undefined method `workflow' for # 0) AppealsController GET appeals when current user is a System Admin when request header does not contain Veteran ID responds with an error Failure/Error: errors: errors.messages[:workflow], NoMethodError: undefined method `workflow' for # # ./app/workflows/case_search_results_base.rb:31:in `search_call' * πŸ› Fix `CaseSearchResultsBase` validations Addresses test failures in `spec/controllers/appeals_controller_spec.rb` similar to the below: AppealsController GET appeals when current user is a System Admin when request header does not contain Veteran ID responds with an error Failure/Error: expect(response_body["errors"][0]["title"]).to eq "Veteran file number missing" expected: "Veteran file number missing" got: nil Using `ActiveModel::Errors` to store error data in an arbitrary format may have been somewhat permissible in the past, but it is an abuse of the object's intended use and is also proving incompatible with the more formalized `ActiveModels::Errors` API in Rails 6.1. In order to preserve the existing response shape of the affected JSON endpoints, we need to move away from the `ActiveModel::Validations` implementation on `CaseSearchResultsBase` (and its descendent classes) to a more bespoke method of performing validations and aggregating errors, since Rails 6.1 `ActiveModel::Errors` is no longer appropriate for our needs here. * βœ… Fix `spec/controllers/application_controller_spec.rb` -- Cache-Control error Addresses the test failure below: ApplicationController no cache headers when toggle set sets Cache-Control etc Failure/Error: expect(response.headers["Cache-Control"]).to eq "no-cache, no-store" expected: "no-cache, no-store" got: "no-store" (compared using ==) # ./spec/controllers/application_controller_spec.rb:59:in `block (4 levels) in ' In Rails 6.1, the `no-store` directive is exclusive of any others that are set on the `Cache-Control` header, which makes sense given the specification https://datatracker.ietf.org/doc/html/rfc7234#section-3 This change was implemented in PR https://github.com/rails/rails/pull/39461 Since it no longer makese sense to set both `no-store` and `no-cache` directives, we will only set `no-store` here, as that is the stronger of the two. * πŸ› Fix multiple specs - `ActiveRecord::EagerLoadPolymorphicError` Addresses multiple test failures caused by the error below: QueueConfig.to_hash title when assigned to an org is formatted as expected Failure/Error: tasks.with_assignees.group("assignees.display_name").count(:all).each_pair.map do |option, count| label = self.class.format_option_label(option, count) self.class.filter_option_hash(option, label) end ActiveRecord::EagerLoadPolymorphicError: Cannot eagerly load the polymorphic association :appeal # ./app/models/queue_column.rb:110:in `assignee_options' * πŸ› Fix `spec/models/task_spec.rb` - `update_all` clears query cache In Rails 6.1.7.7, the method `ActiveRecord::Relation #update_all` will now clear any records cached by the calling relation. This was altering the behavior of `Task #cancel_task_and_child_subtasks` and causing the following test failure: Task#cancel_task_and_child_subtasks cancels all tasks and child subtasks Failure/Error: expect(second_level_tasks[0].versions.count).to eq(initial_versions + 2) expected: 3 got: 2 (compared using ==) # ./spec/models/task_spec.rb:368:in `block (3 levels) in ' To remedy, we will now cache the necessary Task records in an Array, which can be used for generating PaperTrail versions both before and after the `update_all`. * πŸ› Fix `spec/services/hearings/calendar_service_spec.rb` - template rendering error Addresses the following test failure: Hearings::CalendarService.confirmation_calendar_invite returns appropriate iCalendar event Failure/Error: expect(ical_event.description).to eq(expected_description) expected: "You're scheduled for a virtual hearing with a Veterans Law Judge of the Board of Veterans' Appeals.\...to reschedule or cancel your virtual hearing, contact us by email at bvahearingteamhotline@va.gov\n" got: # * πŸ› Fix YAML syntax error caused by whitespace in ENV var Address the following error, found during demo deployment: rake aborted! Cannot load database configuration: YAML syntax error occurred while parsing /caseflow/config/database.yml. Please note that YAML must be consistently indented using spaces. Tabs are not allowed. Error: (): could not find expected ':' while scanning a simple key at line 49 column 5 * ⬆️ Update `caseflow-commons` dependency to latest ref Removes `bourbon` and `neat` dependencies. --- .codeclimate.yml | 3 + .github/workflows/make-docs.yml | 34 +- .github/workflows/workflow.yml | 4 +- .reek.yml | 2 +- .rubocop.yml | 1 + Dangerfile | 6 +- Gemfile | 17 +- Gemfile.lock | 219 ++++++------- MAC_INTEL.md | 4 +- MAC_M1.md | 4 +- Makefile.example | 15 +- Rakefile | 3 +- WINDOWS_11.md | 4 +- app/controllers/api/application_controller.rb | 5 - .../application_base_controller.rb | 5 - app/controllers/application_controller.rb | 2 +- app/controllers/users_controller.rb | 2 +- app/models/etl/record.rb | 2 +- app/models/queue_column.rb | 2 +- app/models/ro_schedule_period.rb | 5 +- app/models/task.rb | 5 +- app/models/vacols/record.rb | 2 +- app/policies/ssl_redirect_exclusion_policy.rb | 27 ++ .../deprecation_warnings/base_handler.rb | 23 -- .../development_handler.rb | 24 -- .../production_handler.rb | 23 +- .../deprecation_warnings/test_handler.rb | 24 -- app/services/hearings/calendar_service.rb | 27 +- app/workflows/case_search_results_base.rb | 48 ++- ..._search_results_for_caseflow_veteran_id.rb | 6 +- .../case_search_results_for_docket_number.rb | 6 +- ..._search_results_for_veteran_file_number.rb | 10 +- app/workflows/post_decision_motion_updater.rb | 8 +- bin/rails | 4 +- bin/rake | 4 +- bin/setup | 8 +- bin/spring | 16 - bin/yarn | 12 +- config.ru | 5 +- config/application.rb | 40 ++- config/boot.rb | 4 +- config/database.yml | 207 ++++++------ config/environment.rb | 2 +- config/environments/development.rb | 30 +- config/environments/make_docs.rb | 152 +++++++++ config/environments/production.rb | 25 +- config/environments/test.rb | 19 +- config/initializers/backtrace_silencers.rb | 7 +- .../initializers/deprecation_warnings.rb | 22 +- .../initializers/filter_parameter_logging.rb | 4 +- config/initializers/permissions_policy.rb | 11 + config/initializers/strong_migrations.rb | 18 ++ config/initializers/warmup_vacols.rb | 2 +- config/puma.rb | 7 +- config/routes.rb | 2 - .../20191111164808_create_etl_appeals.rb | 0 .../20191119204827_create_etl_users.rb | 0 .../20191120184256_create_etl_tasks.rb | 0 .../20191120213930_create_etl_people.rb | 0 ...20191120213941_create_etl_organizations.rb | 0 ...1126164353_remove_users_vacols_not_null.rb | 0 .../20191126203511_add_appeal_timestamps.rb | 0 .../20191209162326_add_appeal_status_sort.rb | 0 .../20191212221851_org_status_change.rb | 0 .../20191212222617_org_status_default.rb | 0 .../20191213194037_add_task_id_index.rb | 0 ...1217193046_create_attorney_case_reviews.rb | 0 .../20191217202614_create_decision_issues.rb | 0 ...8214351_add_more_cached_user_attributes.rb | 0 ...91223213954_add_email_address_to_person.rb | 0 .../20200114201144_add_aod_due_to_age.rb | 0 ...00116215235_add_claimant_dob_to_appeals.rb | 0 ...0121221718_add_appeals_claimant_indexes.rb | 0 .../20200212205344_add_etl_build_tables.rb | 0 .../20200303223155_add_etl_users_index.rb | 0 ...00429152715_create_vha_decision_reviews.rb | 0 .../20200429164923_create_hearings.rb | 0 .../20200430152234_add_hearing_type.rb | 0 ...200501211351_change_hearing_day_id_null.rb | 0 .../20200504133723_change_css_limit.rb | 0 .../20200504204154_remove_null_hearings.rb | 0 ...41923_drop_legacy_hearing_scheduled_for.rb | 0 ...20200517021335_create_etl_remand_reason.rb | 0 ...ha_decision_reviews_to_decision_reviews.rb | 0 ...214159_change_decision_reviews_comments.rb | 0 ...9214329_change_decision_reviews_indices.rb | 0 ...221508_add_decision_issue_benefit_index.rb | 0 .../20200520220924_add_person_ssn.rb | 0 ...xt_and_percent_number_to_decision_issue.rb | 0 ...iority_pushed_cases_to_etl_organization.rb | 0 ...rity_pushed_cases_index_on_organization.rb | 0 ...124530_remove_etl_decision_issues_index.rb | 0 ...cision_issues_rating_issue_ref_id_index.rb | 0 ...0810115309_create_etl_judge_case_review.rb | 0 ...0810115459_create_etl_decision_document.rb | 0 ...0818172716_add_ama_only_to_organization.rb | 0 ...n_null_constraint_on_appeal_docket_type.rb | 0 ..._null_constraint_on_appeal_receipt_date.rb | 0 ..._appeals_from_affinity_to_organizations.rb | 0 ...6_add_scheduled_in_timezone_to_hearings.rb | 0 db/{etl/schema.rb => etl_schema.rb} | 0 docker-bin/env.sh | 3 +- docker-bin/startup.sh | 3 +- lib/caseflow/migration.rb | 21 +- .../migrations/add_index_concurrently.rb | 101 ++++++ lib/tasks/db.rake | 300 ++++++++++++++++++ lib/tasks/local/build.rake | 4 +- lib/tasks/local/destroy.rake | 2 +- lib/tasks/spec.rake | 2 +- scripts/squash.sh | 2 +- .../api/v1/jobs_controller_spec.rb | 2 +- .../application_controller_spec.rb | 2 +- .../decision_reviews_controller_spec.rb | 2 +- spec/factories/ro_schedule_period.rb | 2 +- spec/feature/non_comp/report_page_spec.rb | 9 +- spec/lib/helpers/association_wrapper_spec.rb | 2 +- spec/lib/tasks/db_spec.rb | 295 +++++++++++++++++ spec/models/ro_schedule_period_spec.rb | 22 ++ spec/models/veteran_spec.rb | 103 ++++-- spec/requests/ssl_redirects_spec.rb | 75 +++++ .../development_handler_spec.rb | 46 --- .../production_handler_spec.rb | 6 - .../deprecation_warnings/test_handler_spec.rb | 38 --- spec/sql/ama_cases_sql_spec.rb | 4 +- spec/support/database_cleaner.rb | 40 ++- spec/workflows/ama_appeal_dispatch_spec.rb | 5 +- spec/workflows/legacy_appeal_dispatch_spec.rb | 5 +- 127 files changed, 1599 insertions(+), 670 deletions(-) create mode 100644 app/policies/ssl_redirect_exclusion_policy.rb delete mode 100644 app/services/deprecation_warnings/base_handler.rb delete mode 100644 app/services/deprecation_warnings/development_handler.rb delete mode 100644 app/services/deprecation_warnings/test_handler.rb delete mode 100755 bin/spring create mode 100644 config/environments/make_docs.rb rename app/services/deprecation_warnings/disallowed_deprecations.rb => config/initializers/deprecation_warnings.rb (64%) create mode 100644 config/initializers/permissions_policy.rb rename db/{etl/migrate => etl_migrate}/20191111164808_create_etl_appeals.rb (100%) rename db/{etl/migrate => etl_migrate}/20191119204827_create_etl_users.rb (100%) rename db/{etl/migrate => etl_migrate}/20191120184256_create_etl_tasks.rb (100%) rename db/{etl/migrate => etl_migrate}/20191120213930_create_etl_people.rb (100%) rename db/{etl/migrate => etl_migrate}/20191120213941_create_etl_organizations.rb (100%) rename db/{etl/migrate => etl_migrate}/20191126164353_remove_users_vacols_not_null.rb (100%) rename db/{etl/migrate => etl_migrate}/20191126203511_add_appeal_timestamps.rb (100%) rename db/{etl/migrate => etl_migrate}/20191209162326_add_appeal_status_sort.rb (100%) rename db/{etl/migrate => etl_migrate}/20191212221851_org_status_change.rb (100%) rename db/{etl/migrate => etl_migrate}/20191212222617_org_status_default.rb (100%) rename db/{etl/migrate => etl_migrate}/20191213194037_add_task_id_index.rb (100%) rename db/{etl/migrate => etl_migrate}/20191217193046_create_attorney_case_reviews.rb (100%) rename db/{etl/migrate => etl_migrate}/20191217202614_create_decision_issues.rb (100%) rename db/{etl/migrate => etl_migrate}/20191218214351_add_more_cached_user_attributes.rb (100%) rename db/{etl/migrate => etl_migrate}/20191223213954_add_email_address_to_person.rb (100%) rename db/{etl/migrate => etl_migrate}/20200114201144_add_aod_due_to_age.rb (100%) rename db/{etl/migrate => etl_migrate}/20200116215235_add_claimant_dob_to_appeals.rb (100%) rename db/{etl/migrate => etl_migrate}/20200121221718_add_appeals_claimant_indexes.rb (100%) rename db/{etl/migrate => etl_migrate}/20200212205344_add_etl_build_tables.rb (100%) rename db/{etl/migrate => etl_migrate}/20200303223155_add_etl_users_index.rb (100%) rename db/{etl/migrate => etl_migrate}/20200429152715_create_vha_decision_reviews.rb (100%) rename db/{etl/migrate => etl_migrate}/20200429164923_create_hearings.rb (100%) rename db/{etl/migrate => etl_migrate}/20200430152234_add_hearing_type.rb (100%) rename db/{etl/migrate => etl_migrate}/20200501211351_change_hearing_day_id_null.rb (100%) rename db/{etl/migrate => etl_migrate}/20200504133723_change_css_limit.rb (100%) rename db/{etl/migrate => etl_migrate}/20200504204154_remove_null_hearings.rb (100%) rename db/{etl/migrate => etl_migrate}/20200508141923_drop_legacy_hearing_scheduled_for.rb (100%) rename db/{etl/migrate => etl_migrate}/20200517021335_create_etl_remand_reason.rb (100%) rename db/{etl/migrate => etl_migrate}/20200519213755_change_vha_decision_reviews_to_decision_reviews.rb (100%) rename db/{etl/migrate => etl_migrate}/20200519214159_change_decision_reviews_comments.rb (100%) rename db/{etl/migrate => etl_migrate}/20200519214329_change_decision_reviews_indices.rb (100%) rename db/{etl/migrate => etl_migrate}/20200519221508_add_decision_issue_benefit_index.rb (100%) rename db/{etl/migrate => etl_migrate}/20200520220924_add_person_ssn.rb (100%) rename db/{etl/migrate => etl_migrate}/20200526054515_add_subject_text_and_percent_number_to_decision_issue.rb (100%) rename db/{etl/migrate => etl_migrate}/20200715110034_add_accepts_priority_pushed_cases_to_etl_organization.rb (100%) rename db/{etl/migrate => etl_migrate}/20200715110603_add_accepts_priority_pushed_cases_index_on_organization.rb (100%) rename db/{etl/migrate => etl_migrate}/20210729124530_remove_etl_decision_issues_index.rb (100%) rename db/{etl/migrate => etl_migrate}/20210729130353_add_etl_decision_issues_rating_issue_ref_id_index.rb (100%) rename db/{etl/migrate => etl_migrate}/20210810115309_create_etl_judge_case_review.rb (100%) rename db/{etl/migrate => etl_migrate}/20210810115459_create_etl_decision_document.rb (100%) rename db/{etl/migrate => etl_migrate}/20210818172716_add_ama_only_to_organization.rb (100%) rename db/{etl/migrate => etl_migrate}/20220107125705_remove_non_null_constraint_on_appeal_docket_type.rb (100%) rename db/{etl/migrate => etl_migrate}/20220110143457_remove_non_null_constraint_on_appeal_receipt_date.rb (100%) rename db/{etl/migrate => etl_migrate}/20240129200527_add_exclude_appeals_from_affinity_to_organizations.rb (100%) rename db/{etl/migrate => etl_migrate}/20240617205006_add_scheduled_in_timezone_to_hearings.rb (100%) rename db/{etl/schema.rb => etl_schema.rb} (100%) create mode 100644 lib/caseflow/migrations/add_index_concurrently.rb create mode 100644 lib/tasks/db.rake create mode 100644 spec/lib/tasks/db_spec.rb create mode 100644 spec/requests/ssl_redirects_spec.rb delete mode 100644 spec/services/deprecation_warnings/development_handler_spec.rb delete mode 100644 spec/services/deprecation_warnings/test_handler_spec.rb diff --git a/.codeclimate.yml b/.codeclimate.yml index ff4076a1126..f16d7d860f0 100644 --- a/.codeclimate.yml +++ b/.codeclimate.yml @@ -52,6 +52,7 @@ plugins: mass_threshold: 81 exclude_patterns: - 'db/migrate/*' + - 'db/etl_migrate/*' - 'app/controllers/idt/api/v2/appeals_controller.rb' - 'spec/controllers/idt/api/v2/appeals_controller_spec.rb' - 'spec/controllers/idt/api/appeals_controller_spec.rb' @@ -79,6 +80,7 @@ plugins: enabled: false exclude_patterns: - 'db/migrate/*' + - 'db/etl_migrate/*' - 'app/jobs/dispatch_email_job.rb' - 'app/mailers/dispatch_mailer.rb' rubocop: @@ -115,6 +117,7 @@ plugins: exclude_patterns: - 'db/schema.rb' + - 'db/etl_schema.rb' - 'db/seeds.rb' - 'db/scripts/*' - 'node_modules/**/*' diff --git a/.github/workflows/make-docs.yml b/.github/workflows/make-docs.yml index 3dccd6568ed..7d4516d5621 100644 --- a/.github/workflows/make-docs.yml +++ b/.github/workflows/make-docs.yml @@ -40,32 +40,6 @@ jobs: - name: Checkout code uses: actions/checkout@v4 - # Since installing a client to Oracle DB is a pain and setting up the VACOLS DB is not necessary, let's skip it. - - name: Remove references to VACOLS Oracle DB - run: | - sed '/ruby-oci8/d' -i Gemfile - sed '/activerecord-oracle_enhanced-adapter/d' -i Gemfile - - # Would like to run `bundle lock --update` to update Gemfile.lock without installing gems (to be done in next step) - # but bundler is not installed yet, so manually modify it: - sed '/ruby-oci8/d' -i Gemfile.lock - sed '/^ activerecord-oracle_enhanced-adapter/,/^ [a-z]/{/^ activerecord-oracle_enhanced-adapter/!{/^ [a-z]/!d}}' -i Gemfile.lock - sed '/activerecord-oracle_enhanced-adapter/d' -i Gemfile.lock - sed '/ruby-plsql/d' -i Gemfile.lock - - # Remove VACOLS database configuration - # https://stackoverflow.com/questions/6287755/using-sed-to-delete-all-lines-between-two-matching-patterns - sed '/_vacols:$/,/^$/{/^_vacols:$/!{/^$/!d}}' -i config/database.yml - - # Even though it won't be used in this script, Rails expects a 'test_vacols' configuration due to "#{Rails.env}_vacols" - echo '# Copied from demo_vacols configuration - test_vacols: - adapter: postgresql - pool: 1 - timeout: 5000 - database: test-vacols - ' >> config/database.yml - - name: Setup Ruby and install gems uses: ruby/setup-ruby@v1 with: @@ -73,17 +47,17 @@ jobs: - name: Setup test database env: - RAILS_ENV: test + RAILS_ENV: make_docs POSTGRES_HOST: localhost POSTGRES_PASSWORD: postgres POSTGRES_USER: postgres run: | echo "::group::Set up Caseflow DB" - bin/rails db:create && bin/rails db:schema:load + bin/rails db:create:primary && bin/rails db:schema:load:primary echo "::endgroup::" echo "::group::Set up Caseflow ETL DB" - DB=etl bundle exec rake db:create db:schema:load + bundle exec rake db:create:etl db:schema:load:etl echo "::endgroup::" # Skipping VACOLS since Oracle DB is not set up @@ -95,7 +69,7 @@ jobs: - name: Create DB schema documentation env: - RAILS_ENV: test + RAILS_ENV: make_docs POSTGRES_HOST: localhost POSTGRES_PASSWORD: postgres POSTGRES_USER: postgres diff --git a/.github/workflows/workflow.yml b/.github/workflows/workflow.yml index c33f56c9b6a..f2dec46dfae 100644 --- a/.github/workflows/workflow.yml +++ b/.github/workflows/workflow.yml @@ -193,8 +193,8 @@ jobs: - name: Database setup run: | - ./ci-bin/capture-log "DB=etl bundle exec rake db:create db:schema:load db:migrate" - ./ci-bin/capture-log "bundle exec rake db:create db:schema:load db:migrate" + ./ci-bin/capture-log "bundle exec rake db:create:etl db:schema:load:etl" + ./ci-bin/capture-log "bundle exec rake db:create:primary db:schema:load:primary" ./ci-bin/capture-log "make -f Makefile.example external-db-create" # added line to create external table(s) that are needed for tests diff --git a/.reek.yml b/.reek.yml index 727b53e049b..cd12f9cc576 100644 --- a/.reek.yml +++ b/.reek.yml @@ -291,5 +291,5 @@ directories: # Directories and files below will not be scanned at all exclude_paths: - db/migrate - - db/etl/migrate + - db/etl_migrate - spec diff --git a/.rubocop.yml b/.rubocop.yml index 8773441a13f..68a4a88ff03 100644 --- a/.rubocop.yml +++ b/.rubocop.yml @@ -6,6 +6,7 @@ AllCops: Exclude: - 'bin/**/*' - 'db/**/*' + - 'config.ru' - 'config/**/*' - 'script/**/*' - 'vendor/**/*' diff --git a/Dangerfile b/Dangerfile index 21b9e8bb4c3..1fb8280ddae 100644 --- a/Dangerfile +++ b/Dangerfile @@ -35,19 +35,19 @@ if git.modified_files.grep(/db\/schema.rb/).any? warn("This PR changes the schema. Please use the PR template checklist.") end -if git.modified_files.grep(/db\/etl\/schema.rb/).any? +if git.modified_files.grep(/db\/etl_schema.rb/).any? warn("This PR changes the etl schema. Please use the PR template checklist.") end new_db_migrations = git.modified_files.grep(/db\/migrate\//).any? -new_etl_migrations = git.modified_files.grep(/db\/etl\/migrate\//).any? +new_etl_migrations = git.modified_files.grep(/db\/etl_migrate\//).any? # migration without migrating if new_db_migrations && git.modified_files.grep(/db\/schema.rb/).none? warn("This PR contains db migrations, but the schema.rb is not modified. Did you forget to run 'make migrate'?") end -if new_etl_migrations && git.modified_files.grep(/db\/etl\/schema.rb/).none? +if new_etl_migrations && git.modified_files.grep(/db\/etl_schema.rb/).none? warn("This PR contains etl migrations, but the etl schema.rb is not modified. Did you forget to run 'make migrate'?") end diff --git a/Gemfile b/Gemfile index 17eec027d43..92e60188bc8 100644 --- a/Gemfile +++ b/Gemfile @@ -5,7 +5,7 @@ source ENV["GEM_SERVER_URL"] || "https://rubygems.org" # State machine gem "aasm", "4.11.0" -gem "activerecord-import" +gem "activerecord-import", "1.0.3" gem "acts_as_tree" # amoeba gem for cloning appeals @@ -17,7 +17,7 @@ gem "bgs", git: "https://github.com/department-of-veterans-affairs/ruby-bgs.git" gem "bootsnap", require: false gem "browser" gem "business_time", "~> 0.9.3" -gem "caseflow", git: "https://github.com/department-of-veterans-affairs/caseflow-commons", ref: "716b58caf2116da5fca21c3b3aeea6c9712f3b9d" +gem "caseflow", git: "https://github.com/department-of-veterans-affairs/caseflow-commons", ref: "9bd3635fbd8094d25160669f38d8699e2f1d7a98" gem "connect_mpi", git: "https://github.com/department-of-veterans-affairs/connect-mpi.git", ref: "a3a58c64f85b980a8b5ea6347430dd73a99ea74c" gem "connect_vbms", git: "https://github.com/department-of-veterans-affairs/connect_vbms.git", ref: "9807d9c9f0f3e3494a60b6693dc4f455c1e3e922" gem "console_tree_renderer", git: "https://github.com/department-of-veterans-affairs/console-tree-renderer.git", tag: "v0.1.1" @@ -33,9 +33,6 @@ gem "icalendar" gem "kaminari" gem "logstasher" gem "moment_timezone-rails" -# Rails 6 has native support for multiple dbs, so prefer that over multiverse after upgrade. -# https://github.com/ankane/multiverse#upgrading-to-rails-6 -gem "multiverse" gem "newrelic_rpm" gem "nokogiri", ">= 1.11.0.rc4" gem "paper_trail", "~> 12.0" @@ -56,7 +53,7 @@ gem "pg", platforms: :ruby # Discussion: https://github.com/18F/college-choice/issues/597#issuecomment-139034834 gem "puma", "5.6.4" gem "rack", "~> 2.2.6.2" -gem "rails", "6.0.6.1" +gem "rails", "6.1.7.7" # Used to colorize output for rake tasks gem "rainbow" # React @@ -80,7 +77,7 @@ gem "stringex", require: false gem "strong_migrations" # print trees gem "tty-tree" -gem "tzinfo", "1.2.10" +gem "tzinfo", "~> 2.0" # Use Uglifier as compressor for JavaScript assets gem "uglifier", ">= 1.3.0" gem "validates_email_format_of" @@ -88,14 +85,14 @@ gem "ziptz" group :production, :staging, :ssh_forwarding, :development, :test do # Oracle DB - gem "activerecord-oracle_enhanced-adapter", "~> 6.0.0" + gem "activerecord-oracle_enhanced-adapter", "~> 6.1.0" gem "ruby-oci8", "~> 2.2" end -group :test, :development, :demo do +group :test, :development, :demo, :make_docs do # Security scanners gem "brakeman" - gem "bullet" + gem "bullet", "~> 6.1.0" gem "bundler-audit" # Testing tools gem "capybara" diff --git a/Gemfile.lock b/Gemfile.lock index 2239aa22953..f2db3b52b7f 100644 --- a/Gemfile.lock +++ b/Gemfile.lock @@ -9,16 +9,14 @@ GIT GIT remote: https://github.com/department-of-veterans-affairs/caseflow-commons - revision: 716b58caf2116da5fca21c3b3aeea6c9712f3b9d - ref: 716b58caf2116da5fca21c3b3aeea6c9712f3b9d + revision: 9bd3635fbd8094d25160669f38d8699e2f1d7a98 + ref: 9bd3635fbd8094d25160669f38d8699e2f1d7a98 specs: caseflow (0.4.8) aws-sdk-s3 - bourbon (= 4.2.7) d3-rails jquery-rails momentjs-rails - neat rails (>= 4.2.7.1) redis-namespace redis-rails @@ -100,66 +98,70 @@ GEM remote: https://rubygems.org/ specs: aasm (4.11.0) - actioncable (6.0.6.1) - actionpack (= 6.0.6.1) + actioncable (6.1.7.7) + actionpack (= 6.1.7.7) + activesupport (= 6.1.7.7) nio4r (~> 2.0) websocket-driver (>= 0.6.1) - actionmailbox (6.0.6.1) - actionpack (= 6.0.6.1) - activejob (= 6.0.6.1) - activerecord (= 6.0.6.1) - activestorage (= 6.0.6.1) - activesupport (= 6.0.6.1) + actionmailbox (6.1.7.7) + actionpack (= 6.1.7.7) + activejob (= 6.1.7.7) + activerecord (= 6.1.7.7) + activestorage (= 6.1.7.7) + activesupport (= 6.1.7.7) mail (>= 2.7.1) - actionmailer (6.0.6.1) - actionpack (= 6.0.6.1) - actionview (= 6.0.6.1) - activejob (= 6.0.6.1) + actionmailer (6.1.7.7) + actionpack (= 6.1.7.7) + actionview (= 6.1.7.7) + activejob (= 6.1.7.7) + activesupport (= 6.1.7.7) mail (~> 2.5, >= 2.5.4) rails-dom-testing (~> 2.0) - actionpack (6.0.6.1) - actionview (= 6.0.6.1) - activesupport (= 6.0.6.1) - rack (~> 2.0, >= 2.0.8) + actionpack (6.1.7.7) + actionview (= 6.1.7.7) + activesupport (= 6.1.7.7) + rack (~> 2.0, >= 2.0.9) rack-test (>= 0.6.3) rails-dom-testing (~> 2.0) rails-html-sanitizer (~> 1.0, >= 1.2.0) - actiontext (6.0.6.1) - actionpack (= 6.0.6.1) - activerecord (= 6.0.6.1) - activestorage (= 6.0.6.1) - activesupport (= 6.0.6.1) + actiontext (6.1.7.7) + actionpack (= 6.1.7.7) + activerecord (= 6.1.7.7) + activestorage (= 6.1.7.7) + activesupport (= 6.1.7.7) nokogiri (>= 1.8.5) - actionview (6.0.6.1) - activesupport (= 6.0.6.1) + actionview (6.1.7.7) + activesupport (= 6.1.7.7) builder (~> 3.1) erubi (~> 1.4) rails-dom-testing (~> 2.0) rails-html-sanitizer (~> 1.1, >= 1.2.0) - activejob (6.0.6.1) - activesupport (= 6.0.6.1) + activejob (6.1.7.7) + activesupport (= 6.1.7.7) globalid (>= 0.3.6) - activemodel (6.0.6.1) - activesupport (= 6.0.6.1) - activerecord (6.0.6.1) - activemodel (= 6.0.6.1) - activesupport (= 6.0.6.1) - activerecord-import (1.0.2) + activemodel (6.1.7.7) + activesupport (= 6.1.7.7) + activerecord (6.1.7.7) + activemodel (= 6.1.7.7) + activesupport (= 6.1.7.7) + activerecord-import (1.0.3) activerecord (>= 3.2) - activerecord-oracle_enhanced-adapter (6.0.6) - activerecord (~> 6.0.0) + activerecord-oracle_enhanced-adapter (6.1.6) + activerecord (~> 6.1.0) ruby-plsql (>= 0.6.0) - activestorage (6.0.6.1) - actionpack (= 6.0.6.1) - activejob (= 6.0.6.1) - activerecord (= 6.0.6.1) + activestorage (6.1.7.7) + actionpack (= 6.1.7.7) + activejob (= 6.1.7.7) + activerecord (= 6.1.7.7) + activesupport (= 6.1.7.7) marcel (~> 1.0) - activesupport (6.0.6.1) + mini_mime (>= 1.1.0) + activesupport (6.1.7.7) concurrent-ruby (~> 1.0, >= 1.0.2) - i18n (>= 0.7, < 2) - minitest (~> 5.1) - tzinfo (~> 1.1) - zeitwerk (~> 2.2, >= 2.2.2) + i18n (>= 1.6, < 2) + minitest (>= 5.1) + tzinfo (~> 2.0) + zeitwerk (~> 2.3) acts_as_tree (2.9.0) activerecord (>= 3.0.0) addressable (2.8.0) @@ -1455,24 +1457,22 @@ GEM aws-eventstream (~> 1, >= 1.0.2) backport (1.2.0) base64 (0.2.0) + benchmark (0.3.0) benchmark-ips (2.7.2) bootsnap (1.7.5) msgpack (~> 1.0) - bourbon (4.2.7) - sass (~> 3.4) - thor (~> 0.19) brakeman (4.7.1) browser (5.3.1) builder (3.2.4) - bullet (6.0.1) + bullet (6.1.5) activesupport (>= 3.0.0) uniform_notifier (~> 1.11) bummr (0.5.0) rainbow thor - bundler-audit (0.6.1) + bundler-audit (0.9.1) bundler (>= 1.2.0, < 3) - thor (~> 0.18) + thor (~> 1.0) business_time (0.9.3) activesupport (>= 3.2.0) tzinfo @@ -1531,24 +1531,29 @@ GEM date (3.3.3) ddtrace (0.34.1) msgpack + dead_end (4.0.0) debase (0.2.4.1) debase-ruby_core_source (>= 0.10.2) debase-ruby_core_source (0.10.14) - derailed_benchmarks (1.3.6) + derailed_benchmarks (2.1.2) benchmark-ips (~> 2) + dead_end get_process_mem (~> 0) heapy (~> 0) - memory_profiler (~> 0) + memory_profiler (>= 0, < 2) + mini_histogram (>= 0.3.0) rack (>= 1) - rake (> 10, < 13) - thor (~> 0.19) + rack-test + rake (> 10, < 14) + ruby-statistics (>= 2.1) + thor (>= 0.19, < 2) diff-lcs (1.3) docile (1.1.5) dogstatsd-ruby (4.4.0) - dotenv (2.7.5) - dotenv-rails (2.7.5) - dotenv (= 2.7.5) - railties (>= 3.2, < 6.1) + dotenv (2.8.1) + dotenv-rails (2.8.1) + dotenv (= 2.8.1) + railties (>= 3.2) dry-configurable (0.11.5) concurrent-ruby (~> 1.0) dry-core (~> 0.4, >= 0.4.7) @@ -1580,6 +1585,7 @@ GEM dry-equalizer (~> 0.3) dry-inflector (~> 0.1, >= 0.1.2) dry-logic (~> 1.0, >= 1.0.2) + e2mmap (0.1.0) ecma-re-validator (0.2.1) regexp_parser (~> 1.2) erubi (1.12.0) @@ -1613,8 +1619,8 @@ GEM git (1.13.2) addressable (~> 2.8) rchardet (~> 1.8) - globalid (1.1.0) - activesupport (>= 5.0) + globalid (1.2.1) + activesupport (>= 6.1) govdelivery-tms (2.8.4) activesupport faraday @@ -1702,8 +1708,7 @@ GEM net-imap net-pop net-smtp - marcel (1.0.2) - maruku (0.7.3) + marcel (1.0.4) memory_profiler (0.9.14) meta_request (0.7.2) rack-contrib (>= 1.1, < 3) @@ -1712,6 +1717,7 @@ GEM mime-types (3.3) mime-types-data (~> 3.2015) mime-types-data (3.2019.1009) + mini_histogram (0.3.1) mini_mime (1.1.2) mini_portile2 (2.8.5) minitest (5.19.0) @@ -1720,13 +1726,7 @@ GEM railties (>= 3.1) msgpack (1.4.2) multipart-post (2.1.1) - multiverse (0.2.2) - activerecord (>= 4.2) - activesupport (>= 4.2) - railties (>= 4.2) nap (1.1.0) - neat (4.0.0) - thor (~> 0.19) nenv (0.3.0) net-imap (0.3.7) date @@ -1755,8 +1755,8 @@ GEM activerecord (>= 5.2) request_store (~> 1.1) parallel (1.23.0) - paranoia (2.4.2) - activerecord (>= 4.0, < 6.1) + paranoia (2.6.3) + activerecord (>= 5.1, < 7.2) parser (2.7.2.0) ast (~> 2.4.1) pdf-forms (1.2.0) @@ -1782,20 +1782,20 @@ GEM rack (~> 2.0) rack-test (2.1.0) rack (>= 1.3) - rails (6.0.6.1) - actioncable (= 6.0.6.1) - actionmailbox (= 6.0.6.1) - actionmailer (= 6.0.6.1) - actionpack (= 6.0.6.1) - actiontext (= 6.0.6.1) - actionview (= 6.0.6.1) - activejob (= 6.0.6.1) - activemodel (= 6.0.6.1) - activerecord (= 6.0.6.1) - activestorage (= 6.0.6.1) - activesupport (= 6.0.6.1) - bundler (>= 1.3.0) - railties (= 6.0.6.1) + rails (6.1.7.7) + actioncable (= 6.1.7.7) + actionmailbox (= 6.1.7.7) + actionmailer (= 6.1.7.7) + actionpack (= 6.1.7.7) + actiontext (= 6.1.7.7) + actionview (= 6.1.7.7) + activejob (= 6.1.7.7) + activemodel (= 6.1.7.7) + activerecord (= 6.1.7.7) + activestorage (= 6.1.7.7) + activesupport (= 6.1.7.7) + bundler (>= 1.15.0) + railties (= 6.1.7.7) sprockets-rails (>= 2.0.0) rails-dom-testing (2.1.1) activesupport (>= 5.0.0) @@ -1809,12 +1809,12 @@ GEM rails-html-sanitizer (1.6.0) loofah (~> 2.21) nokogiri (~> 1.14) - railties (6.0.6.1) - actionpack (= 6.0.6.1) - activesupport (= 6.0.6.1) + railties (6.1.7.7) + actionpack (= 6.1.7.7) + activesupport (= 6.1.7.7) method_source - rake (>= 0.8.7) - thor (>= 0.20.3, < 2.0) + rake (>= 12.2) + thor (~> 1.0) rainbow (3.1.1) rake (12.3.3) rb-fsevent (0.11.2) @@ -1918,6 +1918,7 @@ GEM ruby-plsql (0.8.0) ruby-prof (1.4.1) ruby-progressbar (1.13.0) + ruby-statistics (3.0.2) ruby_dep (1.5.0) ruby_parser (3.20.3) sexp_processor (~> 4.16) @@ -1966,18 +1967,20 @@ GEM single_cov (1.3.2) sixarm_ruby_unaccent (1.2.0) socksify (1.7.1) - solargraph (0.38.0) + solargraph (0.40.1) backport (~> 1.1) + benchmark bundler (>= 1.17.2) + e2mmap jaro_winkler (~> 1.5) - maruku (~> 0.7, >= 0.7.3) - nokogiri (~> 1.9, >= 1.9.1) + kramdown (~> 2.3) + kramdown-parser-gfm (~> 1.1) parser (~> 2.3) - reverse_markdown (~> 1.0, >= 1.0.5) - rubocop (~> 0.52) - thor (~> 0.19, >= 0.19.4) + reverse_markdown (>= 1.0.5, < 3) + rubocop (>= 0.52) + thor (~> 1.0) tilt (~> 2.0) - yard (~> 0.9) + yard (~> 0.9, >= 0.9.24) sprockets (3.7.2) concurrent-ruby (~> 1.0) rack (> 1, < 3) @@ -1993,19 +1996,18 @@ GEM terminal-table (1.8.0) unicode-display_width (~> 1.1, >= 1.1.1) test-prof (0.10.1) - thor (0.20.3) - thread_safe (0.3.6) + thor (1.3.1) tilt (2.0.8) timecop (0.9.1) timeout (0.4.0) tty-tree (0.3.0) - tzinfo (1.2.10) - thread_safe (~> 0.1) + tzinfo (2.0.6) + concurrent-ruby (~> 1.0) uglifier (4.1.20) execjs (>= 0.3.0, < 3) unicode-display_width (1.8.0) unicode_utils (1.4.0) - uniform_notifier (1.12.1) + uniform_notifier (1.16.0) uri_template (0.7.0) validates_email_format_of (1.6.3) i18n @@ -2047,8 +2049,8 @@ PLATFORMS DEPENDENCIES aasm (= 4.11.0) - activerecord-import - activerecord-oracle_enhanced-adapter (~> 6.0.0) + activerecord-import (= 1.0.3) + activerecord-oracle_enhanced-adapter (~> 6.1.0) acts_as_tree amoeba anbt-sql-formatter @@ -2057,7 +2059,7 @@ DEPENDENCIES bootsnap brakeman browser - bullet + bullet (~> 6.1.0) bummr bundler-audit business_time (~> 0.9.3) @@ -2093,7 +2095,6 @@ DEPENDENCIES logstasher meta_request moment_timezone-rails - multiverse newrelic_rpm nokogiri (>= 1.11.0.rc4) paper_trail (~> 12.0) @@ -2108,7 +2109,7 @@ DEPENDENCIES pry-byebug (~> 3.9) puma (= 5.6.4) rack (~> 2.2.6.2) - rails (= 6.0.6.1) + rails (= 6.1.7.7) rails-erd rainbow rb-readline @@ -2149,7 +2150,7 @@ DEPENDENCIES test-prof timecop tty-tree - tzinfo (= 1.2.10) + tzinfo (~> 2.0) uglifier (>= 1.3.0) validates_email_format_of webdrivers diff --git a/MAC_INTEL.md b/MAC_INTEL.md index 18fc4ea6a06..ee480bdb68d 100644 --- a/MAC_INTEL.md +++ b/MAC_INTEL.md @@ -96,12 +96,12 @@ 27. Run `make reset` * If issues occur: - 1. Run `bundle exec rake db:create` + 1. Run `bundle exec rake db:create:primary` * If you get connection issues stating no file to be found, run the following: * `rm /opt/homebrew/var/postgres/postmaster.pid` or possibly `rm /usr/local/var/postgres/postmaster.pid` * `brew services restart postgresql` 2. Run `bundle exec rake local:vacols:seed` - 3. Run `bundle exec rake db:schema:load db:seed` + 3. Run `bundle exec rake db:schema:load:primary db:seed` 27. Open a new tab in terminal diff --git a/MAC_M1.md b/MAC_M1.md index 31d15d8eca6..2fbe9244ae4 100644 --- a/MAC_M1.md +++ b/MAC_M1.md @@ -194,7 +194,7 @@ Running Caseflow 5. Start Vacols UTM VM (if not already running) 6. run `make up-m1` to create the docker containers and volumes 7. run `make reset` to (re)create and seed the database; this takes a while (~45 minutes) - 1. if you get a database not found error, run `bundle exec rake db:drop db:create db:schema:load`, and then run `make reset` again + 1. if you get a database not found error, run `bundle exec rake db:drop:primary db:create:primary db:schema:load:primary`, and then run `make reset` again 8. open a second terminal tab/window 9. run `make run-backend-m1` in one tab, and `make run-frontend` in the other 1. the backend should launch on `localhost:3000`. go there in your browser to access Caseflow @@ -566,7 +566,7 @@ The following steps are an alternative to step 7 of the Running Caseflow section * d. `make reset` * i. Resets caseflow and ETL database schemas, seeds databases, and enables feature flags * **If `make reset` returns database not found error: - * a. Run command `bundle exec rake db:drop` + * a. Run command `bundle exec rake db:drop:primary` * b. Download caseflow-db-backup.gz (not able to share this download via policy, ask in the slack channel) * c. Enter terminal, navigate to ~/Downloads * e. Run command diff --git a/Makefile.example b/Makefile.example index 2c1d7565a97..f37b02940ed 100644 --- a/Makefile.example +++ b/Makefile.example @@ -222,19 +222,19 @@ c: ## Start rails console bundle exec rails console etl-migrate: ## Migrate ETL database - DB=etl bundle exec rake db:migrate + bundle exec rake db:migrate:etl etl-test-prepare: - DB=etl bundle exec rake db:test:prepare + bundle exec rake db:test:prepare:etl etl-rollback: ## Rollback ETL database - DB=etl bundle exec rake db:rollback + bundle exec rake db:rollback:etl db-migrate: ## Migrate main Caseflow db - bundle exec rake db:migrate + bundle exec rake db:migrate:primary db-rollback: ## Rollback main Caseflow db - bundle exec rake db:rollback + bundle exec rake db:rollback:primary migrate: external-db-remove etl-migrate etl-test-prepare db-migrate ## Migrate all non-external Rails databases @@ -246,8 +246,8 @@ fresh: ## https://github.com/imsky/git-fresh - updates local master to match or reset: reset-dbs seed-dbs enable-feature-flags ## Resets databases and enable feature flags reset-dbs: ## Resets Caseflow and ETL database schemas - DB=etl bundle exec rake db:drop db:create db:schema:load - bundle exec rake db:drop db:create db:schema:load + bundle exec rake db:drop:etl db:create:etl db:schema:load:etl + bundle exec rake db:drop:primary db:create:primary db:schema:load:primary make consolidated_db_scripts seed-vbms-ext-claim: ## Seed only vbms_ext_claim @@ -260,7 +260,6 @@ seed-dbs: ## Seed all databases bundle exec rake local:vacols:seed bundle exec rake spec:setup_vacols bundle exec rake db:seed - DB=etl bundle exec rake db:seed enable-feature-flags: ## enable all feature flags bundle exec rails runner scripts/enable_features_dev.rb diff --git a/Rakefile b/Rakefile index 838add5d4e9..dac0a4b9bb8 100644 --- a/Rakefile +++ b/Rakefile @@ -10,4 +10,5 @@ Dir[Rails.root.join("lib/tasks/support/**/*.rb")].sort.each { |f| require f } Rails.application.load_tasks -task "db:schema:dump": "strong_migrations:alphabetize_columns" +task "db:schema:dump:primary": "strong_migrations:alphabetize_columns" +task "db:schema:dump:etl": "strong_migrations:alphabetize_columns" diff --git a/WINDOWS_11.md b/WINDOWS_11.md index 2b8139175f4..8d7f388a3d9 100644 --- a/WINDOWS_11.md +++ b/WINDOWS_11.md @@ -134,9 +134,9 @@ Run each line is a separate command, run them one at a time 26. ```cd ~/appeals/caseflow in terminal``` -27. ```bin/rails db:migrate RAILS_ENV=development``` +27. ```bin/rails db:migrate:primary RAILS_ENV=development``` -28. Run ```bundle install``` to install missing gems and then ```bin/rails db:migrate``` ```RAILS_ENV=development``` command if you have too again. +28. Run ```bundle install``` to install missing gems and then ```bin/rails db:migrate:primary``` ```RAILS_ENV=development``` command if you have too again. 29. ```make reset``` (should have split_correlation_tables) diff --git a/app/controllers/api/application_controller.rb b/app/controllers/api/application_controller.rb index ea12981e0fc..adb315f0d93 100644 --- a/app/controllers/api/application_controller.rb +++ b/app/controllers/api/application_controller.rb @@ -5,7 +5,6 @@ class Api::ApplicationController < ActionController::Base include TrackRequestId - force_ssl if: :ssl_enabled? before_action :strict_transport_security before_action :setup_fakes, @@ -51,10 +50,6 @@ def unauthorized render json: { status: "unauthorized" }, status: :unauthorized end - def ssl_enabled? - Rails.env.production? - end - def strict_transport_security response.headers["Strict-Transport-Security"] = "max-age=31536000; includeSubDomains" if request.ssl? end diff --git a/app/controllers/application_base_controller.rb b/app/controllers/application_base_controller.rb index 41f55a7fe3b..f9b1304a804 100644 --- a/app/controllers/application_base_controller.rb +++ b/app/controllers/application_base_controller.rb @@ -8,7 +8,6 @@ class ApplicationBaseController < ActionController::Base include TrackRequestId - force_ssl if: :ssl_enabled? before_action :check_out_of_service before_action :strict_transport_security @@ -35,10 +34,6 @@ def check_out_of_service render "out_of_service", layout: "application" if Rails.cache.read("out_of_service") end - def ssl_enabled? - Rails.env.production? - end - def strict_transport_security response.headers["Strict-Transport-Security"] = "max-age=31536000; includeSubDomains" if request.ssl? end diff --git a/app/controllers/application_controller.rb b/app/controllers/application_controller.rb index c34519195e4..277121cf689 100644 --- a/app/controllers/application_controller.rb +++ b/app/controllers/application_controller.rb @@ -273,7 +273,7 @@ def set_no_cache_headers # https://stackoverflow.com/a/748646 def no_cache # :nocov: - response.headers["Cache-Control"] = "no-cache, no-store" + response.headers["Cache-Control"] = "no-store" response.headers["Pragma"] = "no-cache" response.headers["Expires"] = "Fri, 01 Jan 1990 00:00:00 GMT" # waaaay in the past # :nocov: diff --git a/app/controllers/users_controller.rb b/app/controllers/users_controller.rb index 437047dd68b..63b65843dd6 100644 --- a/app/controllers/users_controller.rb +++ b/app/controllers/users_controller.rb @@ -88,7 +88,7 @@ def filter_by_organization end # Depending on the route and the requested resource, the requested user's id could be sent as :id or :user_id - # ex from rake routes: user GET /users/:id or user_task_pages GET /users/:user_id/task_pages + # ex from rails routes: user GET /users/:id or user_task_pages GET /users/:user_id/task_pages def id @id ||= params[:id] || params[:user_id] end diff --git a/app/models/etl/record.rb b/app/models/etl/record.rb index 79819d27973..a71604ee94c 100644 --- a/app/models/etl/record.rb +++ b/app/models/etl/record.rb @@ -16,7 +16,7 @@ class ETL::Record < ApplicationRecord self.abstract_class = true - establish_connection :"etl_#{Rails.env}" + establish_connection :etl class << self def sync_with_original(original) diff --git a/app/models/queue_column.rb b/app/models/queue_column.rb index b848c345925..750639de49a 100644 --- a/app/models/queue_column.rb +++ b/app/models/queue_column.rb @@ -107,7 +107,7 @@ def task_type_options(tasks) end def assignee_options(tasks) - tasks.with_assignees.group("assignees.display_name").count(:all).each_pair.map do |option, count| + tasks.with_assignees.group("display_name").count(:all).each_pair.map do |option, count| label = self.class.format_option_label(option, count) self.class.filter_option_hash(option, label) end diff --git a/app/models/ro_schedule_period.rb b/app/models/ro_schedule_period.rb index 3966864cb65..7fdbbf7cee0 100644 --- a/app/models/ro_schedule_period.rb +++ b/app/models/ro_schedule_period.rb @@ -17,8 +17,9 @@ class RoSchedulePeriod < SchedulePeriod # Run various validations on the uploaded spreadsheet and record errors def validate_spreadsheet - validate_spreadsheet = HearingSchedule::ValidateRoSpreadsheet.new(spreadsheet, start_date, end_date) - errors[:base] << validate_spreadsheet.validate + spreadsheet_errors = HearingSchedule::ValidateRoSpreadsheet.new(spreadsheet, start_date, end_date).validate + errors.add(:base, spreadsheet_errors) if spreadsheet_errors.any? + errors[:base] end # Create NonAvailibility records for ROs and CO and Allocation records for each RO diff --git a/app/models/task.rb b/app/models/task.rb index f74de948c52..b2244e0f1b0 100644 --- a/app/models/task.rb +++ b/app/models/task.rb @@ -724,15 +724,14 @@ def previous_task def cancel_task_and_child_subtasks # Cancel all descendants at the same time to avoid after_update hooks marking some tasks as completed. # it would be better if we could allow the callbacks to happen sanely - descendant_ids = descendants.pluck(:id) # by avoiding callbacks, we aren't saving PaperTrail versions # Manually save the state before and after. - tasks = Task.open.where(id: descendant_ids) + tasks = Task.open.where(id: descendants).to_a transaction do tasks.each { |task| task.paper_trail.save_with_version } - tasks.update_all( + Task.where(id: tasks).update_all( status: Constants.TASK_STATUSES.cancelled, cancelled_by_id: RequestStore[:current_user]&.id, closed_at: Time.zone.now diff --git a/app/models/vacols/record.rb b/app/models/vacols/record.rb index cf6070a6c70..145c5b0466d 100644 --- a/app/models/vacols/record.rb +++ b/app/models/vacols/record.rb @@ -3,7 +3,7 @@ class VACOLS::Record < ApplicationRecord self.abstract_class = true - establish_connection "#{Rails.env}_vacols".to_sym + establish_connection :vacols ActiveSupport.run_load_hooks(:active_record_vacols, VACOLS::Record) # This method calculates the appropriate date & timezone diff --git a/app/policies/ssl_redirect_exclusion_policy.rb b/app/policies/ssl_redirect_exclusion_policy.rb new file mode 100644 index 00000000000..41f19dba046 --- /dev/null +++ b/app/policies/ssl_redirect_exclusion_policy.rb @@ -0,0 +1,27 @@ +# frozen_string_literal: true + +# @note To be used in the environment configuration settings for excluding exempt request paths from SSL redirects +# when `config.force_ssl = true` +# +# @example config/environments/production.rb +# +# Rails.application.configure do +# config.force_ssl = true +# config.ssl_options = { redirect: { exclude: SslRedirectExclusionPolicy } } +# # etc. +class SslRedirectExclusionPolicy + EXEMPT_PATH_PATTERNS = [ + %r{^/api/docs/v3/}, + %r{^/api/metadata$}, + %r{^/health-check$}, + %r{^/idt/api/v1/}, + %r{^/idt/api/v2/}, + %r{^/pdfjs/} + ].freeze + + # @param [ActionDispatch::Request] request + # @return [TrueClass, FalseClass] true if request path is exempt from an SSL redirect + def self.call(request) + EXEMPT_PATH_PATTERNS.any? { |pattern| pattern =~ request.path } + end +end diff --git a/app/services/deprecation_warnings/base_handler.rb b/app/services/deprecation_warnings/base_handler.rb deleted file mode 100644 index 65b473bfb38..00000000000 --- a/app/services/deprecation_warnings/base_handler.rb +++ /dev/null @@ -1,23 +0,0 @@ -# frozen_string_literal: true - -# @abstract Subclass and override {.call} to implement a custom DeprecationWarnings handler class. -# @note For use with `ActiveSupport::Deprecation.behavior=`. -module DeprecationWarnings - class BaseHandler - class << self - # Subclasses must respond to `.call` to play nice with `ActiveSupport::Deprecation.behavior=`. - # https://github.com/rails/rails/blob/a4581b53aae93a8dd3205abae0630398cbce9204/activesupport/lib/active_support/deprecation/behaviors.rb#L70-L71 - # :reek:LongParameterList - def call(_message, _callstack, _deprecation_horizon, _gem_name) - fail NotImplementedError - end - - # Subclasses must respond to `.arity` to play nice with `ActiveSupport::Deprecation.behavior=`. - # Must return number of arguments accepted by `.call`. - # https://github.com/rails/rails/blob/a4581b53aae93a8dd3205abae0630398cbce9204/activesupport/lib/active_support/deprecation/behaviors.rb#L101 - def arity - method(:call).arity - end - end - end -end diff --git a/app/services/deprecation_warnings/development_handler.rb b/app/services/deprecation_warnings/development_handler.rb deleted file mode 100644 index a61d262e5e6..00000000000 --- a/app/services/deprecation_warnings/development_handler.rb +++ /dev/null @@ -1,24 +0,0 @@ -# frozen_string_literal: true - -require_relative "disallowed_deprecations" - -# @note For use with `ActiveSupport::Deprecation.behavior=`. -module DeprecationWarnings - class DevelopmentHandler < BaseHandler - extend DisallowedDeprecations - - class << self - # :reek:LongParameterList - def call(message, _callstack, _deprecation_horizon, _gem_name) - raise_if_disallowed_deprecation!(message) - emit_warning_to_application_logs(message) - end - - private - - def emit_warning_to_application_logs(message) - Rails.logger.warn(message) - end - end - end -end diff --git a/app/services/deprecation_warnings/production_handler.rb b/app/services/deprecation_warnings/production_handler.rb index fe1c1310d78..7e4f70d0743 100644 --- a/app/services/deprecation_warnings/production_handler.rb +++ b/app/services/deprecation_warnings/production_handler.rb @@ -1,27 +1,32 @@ # frozen_string_literal: true -# @note For use with `ActiveSupport::Deprecation.behavior=`. +# @note For use with `ActiveSupport::Deprecation.behavior=` or `Rails.application.config.active_support.deprecation=` module DeprecationWarnings - class ProductionHandler < BaseHandler + class ProductionHandler APP_NAME = "caseflow" + SLACK_ALERT_TITLE = "Deprecation Warning - #{APP_NAME} (#{ENV['DEPLOY_ENV']})" SLACK_ALERT_CHANNEL = "#appeals-deprecation-alerts" class << self + # Adhere to `.call` signature expected by `ActiveSupport::Deprecation.behavior=`. + # https://github.com/rails/rails/blob/a4581b53aae93a8dd3205abae0630398cbce9204/activesupport/lib/active_support/deprecation/behaviors.rb#L70-L71 # :reek:LongParameterList def call(message, callstack, deprecation_horizon, gem_name) - emit_warning_to_application_logs(message) emit_warning_to_sentry(message, callstack, deprecation_horizon, gem_name) emit_warning_to_slack_alerts_channel(message) rescue StandardError => error Raven.capture_exception(error) end - private - - def emit_warning_to_application_logs(message) - Rails.logger.warn(message) + # Must respond to `.arity` to play nice with `ActiveSupport::Deprecation.behavior=` + # and return number of arguments accepted by `.call`. + # https://github.com/rails/rails/blob/a4581b53aae93a8dd3205abae0630398cbce9204/activesupport/lib/active_support/deprecation/behaviors.rb#L101 + def arity + method(:call).arity end + private + # :reek:LongParameterList def emit_warning_to_sentry(message, callstack, deprecation_horizon, gem_name) # Pre-emptive bugfix for future versions of the `sentry-raven` gem: @@ -44,9 +49,7 @@ def emit_warning_to_sentry(message, callstack, deprecation_horizon, gem_name) end def emit_warning_to_slack_alerts_channel(message) - slack_alert_title = "Deprecation Warning - #{APP_NAME} (#{ENV['DEPLOY_ENV']})" - - SlackService.new.send_notification(message, slack_alert_title, SLACK_ALERT_CHANNEL) + SlackService.new.send_notification(message, SLACK_ALERT_TITLE, SLACK_ALERT_CHANNEL) end end end diff --git a/app/services/deprecation_warnings/test_handler.rb b/app/services/deprecation_warnings/test_handler.rb deleted file mode 100644 index b87945be19c..00000000000 --- a/app/services/deprecation_warnings/test_handler.rb +++ /dev/null @@ -1,24 +0,0 @@ -# frozen_string_literal: true - -require_relative "disallowed_deprecations" - -# @note For use with `ActiveSupport::Deprecation.behavior=`. -module DeprecationWarnings - class TestHandler < BaseHandler - extend DisallowedDeprecations - - class << self - # :reek:LongParameterList - def call(message, _callstack, _deprecation_horizon, _gem_name) - raise_if_disallowed_deprecation!(message) - emit_error_to_stderr(message) - end - - private - - def emit_error_to_stderr(message) - ActiveSupport::Logger.new($stderr).error(message) - end - end - end -end diff --git a/app/services/hearings/calendar_service.rb b/app/services/hearings/calendar_service.rb index caf1936b7ab..8a9be76d602 100644 --- a/app/services/hearings/calendar_service.rb +++ b/app/services/hearings/calendar_service.rb @@ -8,11 +8,6 @@ # emails. class Hearings::CalendarService - class CalendarEventView < ActionView::Base - include Hearings::CalendarTemplateHelper - include Hearings::AppellantNameHelper - end - class << self # Sent when first switching a video hearing to a virtual hearing, # and also when the scheduled time for an existing virtual hearing @@ -100,24 +95,10 @@ def create_calendar_event(hearing) end def render_virtual_hearing_calendar_event_template(email_recipient_info, event_type, locals) - lookup_context = ActionView::Base.build_lookup_context(ActionController::Base.view_paths) - context = CalendarEventView.new(lookup_context) - - # Some *~ magic ~* here. The recipient title is used to determine which template to load: - # - # judge_confirmation_event_description - # representative_confirmation_event_description - # veteran_confirmation_event_description - # - # representative_changed_to_video_event_description - # veteran_changed_to_video_event_description - - template_name = "#{email_recipient_info.title.downcase}_#{event_type}_event_description" - - context.render( - template: "hearing_mailer/calendar_events/#{template_name}", - locals: locals - ) + template_name = + "hearing_mailer/calendar_events/#{email_recipient_info.title.downcase}_#{event_type}_event_description" + + ApplicationController.render(template: template_name, locals: locals) end end end diff --git a/app/workflows/case_search_results_base.rb b/app/workflows/case_search_results_base.rb index ea30ccd739e..f8e4e1a0528 100644 --- a/app/workflows/case_search_results_base.rb +++ b/app/workflows/case_search_results_base.rb @@ -1,12 +1,35 @@ # frozen_string_literal: true class CaseSearchResultsBase - include ActiveModel::Validations + class AbstractMethodError < StandardError; end - validate :vso_employee_has_access + # @note Data Transfer Object (DTO) for encapsulating error data grouped by attribute + class Errors + attr_accessor :messages + + private :messages= + + delegate :none?, to: :messages + + def initialize + @messages = Hash.new([]) + end + + # @param attribute [Symbol] attribute the error is associated with + # @param title [String] summary of the error + # @param detail [String] detailed description of the error + # + # @example + # add(:workflow, title: "Access to Veteran file prohibited", + # detail: "You do not have access to this claims file number") + def add(attribute, title:, detail:) + messages[attribute.to_sym] += [{ title: title, detail: detail }] + end + end def initialize(user:) @user = user + @errors = Errors.new end def call @@ -99,8 +122,25 @@ def search_page_json_appeals(appeals) private + attr_accessor :errors attr_reader :success + def valid? + validate_vso_employee_has_access + validation_hook + errors.none? + end + + # @note (Optional) hook method to be implemented by subclasses for performing subclass-specific validations. + # @example + # def validation_hook + # unless foobar_has_bazbah? + # errors.add(:workflow, title: "Veteran file number missing", + # detail: "HTTP_CASE_SEARCH request header must include Veteran file number") + # end + # end + def validation_hook; end + def access?(file_number) !current_user_is_vso_employee? || bgs.can_access?(file_number) end @@ -109,7 +149,7 @@ def bgs @bgs ||= BGSService.new end - def veterans_exist + def validate_veterans_exist return unless veterans_user_can_access.empty? errors.add(:workflow, not_found_error) @@ -146,7 +186,7 @@ def case_search_results } end - def vso_employee_has_access + def validate_vso_employee_has_access return unless current_user_is_vso_employee? errors.add(:workflow, prohibited_error) if diff --git a/app/workflows/case_search_results_for_caseflow_veteran_id.rb b/app/workflows/case_search_results_for_caseflow_veteran_id.rb index 986b3dfdd4d..3fe2b5604df 100644 --- a/app/workflows/case_search_results_for_caseflow_veteran_id.rb +++ b/app/workflows/case_search_results_for_caseflow_veteran_id.rb @@ -1,8 +1,6 @@ # frozen_string_literal: true class CaseSearchResultsForCaseflowVeteranId < ::CaseSearchResultsBase - validate :veterans_exist - def initialize(caseflow_veteran_ids:, user:) super(user: user) @caseflow_veteran_ids = caseflow_veteran_ids @@ -18,6 +16,10 @@ def veterans attr_reader :caseflow_veteran_ids + def validation_hook + validate_veterans_exist + end + def not_found_error { "title": "Veteran not found", diff --git a/app/workflows/case_search_results_for_docket_number.rb b/app/workflows/case_search_results_for_docket_number.rb index af8ace8a2fb..6c16a50a46d 100644 --- a/app/workflows/case_search_results_for_docket_number.rb +++ b/app/workflows/case_search_results_for_docket_number.rb @@ -1,8 +1,6 @@ # frozen_string_literal: true class CaseSearchResultsForDocketNumber < ::CaseSearchResultsBase - validate :veterans_exist, if: :current_user_is_vso_employee? - def initialize(docket_number:, user:) super(user: user) @docket_number = docket_number.to_s if docket_number @@ -22,6 +20,10 @@ def appeals attr_reader :docket_number + def validation_hook + validate_veterans_exist if current_user_is_vso_employee? + end + def not_found_error { "title": "Docket ID not found", diff --git a/app/workflows/case_search_results_for_veteran_file_number.rb b/app/workflows/case_search_results_for_veteran_file_number.rb index 63cd79983b1..3f0f97fe466 100644 --- a/app/workflows/case_search_results_for_veteran_file_number.rb +++ b/app/workflows/case_search_results_for_veteran_file_number.rb @@ -1,9 +1,6 @@ # frozen_string_literal: true class CaseSearchResultsForVeteranFileNumber < ::CaseSearchResultsBase - validate :file_number_or_ssn_presence - validate :veterans_exist, if: :current_user_is_vso_employee? - def initialize(file_number_or_ssn:, user:) super(user: user) # Ensure we have a string made of solely numeric characters @@ -14,7 +11,12 @@ def initialize(file_number_or_ssn:, user:) attr_reader :file_number_or_ssn - def file_number_or_ssn_presence + def validation_hook + validate_file_number_or_ssn_presence + validate_veterans_exist if current_user_is_vso_employee? + end + + def validate_file_number_or_ssn_presence return if file_number_or_ssn errors.add(:workflow, missing_veteran_file_number_or_ssn_error) diff --git a/app/workflows/post_decision_motion_updater.rb b/app/workflows/post_decision_motion_updater.rb index 70585da819e..45341485500 100644 --- a/app/workflows/post_decision_motion_updater.rb +++ b/app/workflows/post_decision_motion_updater.rb @@ -69,7 +69,7 @@ def create_motion end unless motion.valid? - errors.messages.merge!(motion.errors.messages) + errors.merge!(motion.errors) return end @@ -85,13 +85,13 @@ def handle_denial_or_dismissal abstract_task = create_abstract_task unless abstract_task.valid? - errors.messages.merge!(abstract_task.errors.messages) + errors.merge!(abstract_task.errors) return end new_task = create_denial_or_dismissal_task(abstract_task) unless new_task.valid? - errors.messages.merge!(new_task.errors.messages) + errors.merge!(new_task.errors) return end new_task.save @@ -132,7 +132,7 @@ def create_new_stream_tasks(stream) ) unless attorney_task.valid? - errors.messages.merge!(attorney_task.errors.messages) + errors.merge!(attorney_task.errors) return end diff --git a/bin/rails b/bin/rails index 07396602377..6fb4e4051c4 100755 --- a/bin/rails +++ b/bin/rails @@ -1,4 +1,4 @@ #!/usr/bin/env ruby APP_PATH = File.expand_path('../config/application', __dir__) -require_relative '../config/boot' -require 'rails/commands' +require_relative "../config/boot" +require "rails/commands" diff --git a/bin/rake b/bin/rake index 17240489f64..4fbf10b960e 100755 --- a/bin/rake +++ b/bin/rake @@ -1,4 +1,4 @@ #!/usr/bin/env ruby -require_relative '../config/boot' -require 'rake' +require_relative "../config/boot" +require "rake" Rake.application.run diff --git a/bin/setup b/bin/setup index 5853b5ea875..90700ac4f9a 100755 --- a/bin/setup +++ b/bin/setup @@ -1,5 +1,5 @@ #!/usr/bin/env ruby -require 'fileutils' +require "fileutils" # path to your application root. APP_ROOT = File.expand_path('..', __dir__) @@ -9,8 +9,8 @@ def system!(*args) end FileUtils.chdir APP_ROOT do - # This script is a way to setup or update your development environment automatically. - # This script is idempotent, so that you can run it at anytime and get an expectable outcome. + # This script is a way to set up or update your development environment automatically. + # This script is idempotent, so that you can run it at any time and get an expectable outcome. # Add necessary setup steps to this file. puts '== Installing dependencies ==' @@ -18,7 +18,7 @@ FileUtils.chdir APP_ROOT do system('bundle check') || system!('bundle install') # Install JavaScript dependencies - # system('bin/yarn') + system! 'bin/yarn' # puts "\n== Copying sample files ==" # unless File.exist?('config/database.yml') diff --git a/bin/spring b/bin/spring deleted file mode 100755 index a4759986d15..00000000000 --- a/bin/spring +++ /dev/null @@ -1,16 +0,0 @@ -#!/usr/bin/env ruby -# frozen_string_literal: true - -# This file loads spring without using Bundler, in order to be fast. -# It gets overwritten when you run the `spring binstub` command. - -unless defined?(Spring) - require "rubygems" - require "bundler" - - if (match = Bundler.default_lockfile.read.match(/^GEM$.*?^ (?: )*spring \((.*?)\)$.*?^$/m)) - Gem.paths = { "GEM_PATH" => [Bundler.bundle_path.to_s, *Gem.path].uniq } - gem "spring", match[1] - require "spring/binstub" - end -end diff --git a/bin/yarn b/bin/yarn index 460dd565b4a..9fab2c35079 100755 --- a/bin/yarn +++ b/bin/yarn @@ -1,9 +1,15 @@ #!/usr/bin/env ruby APP_ROOT = File.expand_path('..', __dir__) Dir.chdir(APP_ROOT) do - begin - exec "yarnpkg", *ARGV - rescue Errno::ENOENT + yarn = ENV["PATH"].split(File::PATH_SEPARATOR). + select { |dir| File.expand_path(dir) != __dir__ }. + product(["yarn", "yarn.cmd", "yarn.ps1"]). + map { |dir, file| File.expand_path(file, dir) }. + find { |file| File.executable?(file) } + + if yarn + exec yarn, *ARGV + else $stderr.puts "Yarn executable was not detected in the system." $stderr.puts "Download Yarn at https://yarnpkg.com/en/docs/install" exit 1 diff --git a/config.ru b/config.ru index cb504065454..265d93376ed 100644 --- a/config.ru +++ b/config.ru @@ -1,8 +1,6 @@ -# frozen_string_literal: true - # This file is used by Rack-based servers to start the application. -require ::File.expand_path("../config/environment", __FILE__) +require_relative "config/environment" require "rack" # rubocop:disable all @@ -69,3 +67,4 @@ end # rubocop:enable all run Rails.application +Rails.application.load_server diff --git a/config/application.rb b/config/application.rb index 37e00da3554..fba36ffb18a 100644 --- a/config/application.rb +++ b/config/application.rb @@ -1,6 +1,6 @@ -require_relative 'boot' +require_relative "boot" -require 'rails/all' +require "rails/all" # Require the gems listed in Gemfile, including any gems # you've limited to :test, :development, or :production. @@ -12,13 +12,13 @@ module CaseflowCertification class Application < Rails::Application # Initialize configuration defaults for originally generated Rails version. - config.load_defaults 5.2 + config.load_defaults 6.1 config.autoloader = :classic - # Settings in config/environments/* take precedence over those specified here. - # Application configuration can go into files in config/initializers - # -- all .rb files in that directory are automatically loaded after loading - # the framework and any gems in your application. + # Configuration for the application, engines, and railties goes here. + # + # These settings can be overridden in specific environments using the files + # in config/environments, which are processed later. # ================================================================================================================== # Rails default overrides @@ -84,10 +84,6 @@ class Application < Rails::Application # Default as of 5.2: true config.action_controller.default_protect_from_forgery = false - # Store boolean values in sqlite3 databases as 1 and 0 instead of 't' and 'f' after migrating old data. - # Default as of 5.2: true - config.active_record.sqlite3.represent_boolean_as_integer = false - # ------------------------------------------------------------------------------------------------------------------ # Rails 6.0 default overrides # ------------------------------------------------------------------------------------------------------------------ @@ -108,22 +104,24 @@ class Application < Rails::Application # Default as of 6.0: false config.action_dispatch.return_only_media_type_on_content_type = true - # Use ActionMailer::MailDeliveryJob for sending parameterized and normal mail. - # - # The default delivery jobs (ActionMailer::Parameterized::DeliveryJob, ActionMailer::DeliveryJob), - # will be removed in Rails 6.1. This setting is not backwards compatible with earlier Rails versions. - # If you send mail in the background, job workers need to have a copy of - # MailDeliveryJob to ensure all delivery jobs are processed properly. - # Make sure your entire app is migrated and stable on 6.0 before using this setting. - # Default as of 6.0: "ActionMailer::MailDeliveryJob" - config.action_mailer.delivery_job = nil - # Enable the same cache key to be reused when the object being cached of type # `ActiveRecord::Relation` changes by moving the volatile information (max updated at and count) # of the relation's cache key into the cache version to support recycling cache key. # Default as of 6.0: true config.active_record.collection_cache_versioning = false + # ------------------------------------------------------------------------------------------------------------------ + # Rails 6.1 default overrides + # ------------------------------------------------------------------------------------------------------------------ + + # Support for inversing belongs_to -> has_many Active Record associations. + # Default as of 6.1: true + config.active_record.has_many_inversing = false + + # Apply random variation to the delay when retrying failed jobs. + # Default as of 6.1: 0.15 + config.active_job.retry_jitter = 0 + # ================================================================================================================== # Set Time.zone default to the specified zone and make Active Record auto-convert to this zone. diff --git a/config/boot.rb b/config/boot.rb index 9833741ac57..54cda1cdcc7 100644 --- a/config/boot.rb +++ b/config/boot.rb @@ -1,5 +1,5 @@ ENV['BUNDLE_GEMFILE'] ||= File.expand_path('../Gemfile', __dir__) ENV["NLS_LANG"] = "AMERICAN_AMERICA.UTF8" -require 'bundler/setup' # Set up gems listed in the Gemfile. -require 'bootsnap/setup' # Speed up boot time by caching expensive operations. +require "bundler/setup" # Set up gems listed in the Gemfile. +require "bootsnap/setup" # Speed up boot time by caching expensive operations. diff --git a/config/database.yml b/config/database.yml index 6a2e9dcf41e..88e0cea7747 100644 --- a/config/database.yml +++ b/config/database.yml @@ -18,26 +18,6 @@ default: &default password: <%= ENV['POSTGRES_PASSWORD'] %> <% end %> -demo: - <<: *default - database: caseflow_certification_demo - -ssh_forwarding: - <<: *default - database: caseflow_certification_ssh_forwarding - -development: - <<: *default - database: caseflow_certification_development - -staging: - <<: *default - database: caseflow_certification_staging - -production: - <<: *default - url: <%= ENV["POSTGRES_URL"] %> - # NOTE Rails 6 has native support for multiple databases, # and this configuration should change to nest "primary" and "etl" and "vacols" # under each env name, per @@ -51,83 +31,124 @@ etl_default: &etl_default <% if ENV["ETL_DB_PASSWORD"] || ENV["POSTGRES_PASSWORD"] %> password: <%= ENV["ETL_DB_PASSWORD"] || ENV["POSTGRES_PASSWORD"] %> <% end %> + migrations_paths: db/etl_migrate -etl_development: - <<: *etl_default - database: caseflow_etl_development - -etl_test: - <<: *etl_default - database: caseflow_etl_test<%= ENV["TEST_SUBCATEGORY"] %> - -etl_production: - <<: *etl_default - url: <%= ENV["ETL_DB_URL"] %> - -development_vacols: - adapter: oracle_enhanced - username: VACOLS_DEV - password: VACOLS_DEV - <% if ! ENV['DOCKERIZED'] %> - host: localhost - port: 1521 - <% end %> - database: <%= ENV['DEMO_DB'] || "BVAP" %> - -demo_vacols: - adapter: sqlite3 - pool: 5 - timeout: 5000 - database: db/demo-vacols.sqlite3 - -ssh_forwarding_vacols: - adapter: oracle_enhanced - host: localhost - port: 1526 - username: <%= ENV["VACOLS_USERNAME"] %> - password: <%= ENV["VACOLS_PASSWORD"] %> - database: BVAP - -staging_vacols: - adapter: oracle_enhanced - username: <%= ENV["VACOLS_USERNAME"] %> - password: <%= ENV["VACOLS_PASSWORD"] %> - host: vacols.dev.vaco.va.gov - port: 1526 - database: BVAP.VACO.VA.GOV - -production_vacols: - adapter: oracle_enhanced - username: <%= ENV["VACOLS_USERNAME"] %> - password: <%= ENV["VACOLS_PASSWORD"] %> - host: <%= ENV["VACOLS_HOST"] %> - port: <%= ENV["VACOLS_PORT"] %> - database: <%= ENV["VACOLS_DATABASE"] %> - pool: <%= ENV["DB_CONN_POOL_MAX_SIZE"] || 5 %> +development: + primary: + <<: *default + database: caseflow_certification_development + etl: + <<: *etl_default + database: caseflow_etl_development + vacols: + adapter: oracle_enhanced + username: VACOLS_DEV + password: VACOLS_DEV + <% if ! ENV['DOCKERIZED'] %> + host: localhost + port: 1521 + <% end %> + database: <%= ENV['DEMO_DB'] || "BVAP" %> # Warning: The database defined as "test" will be erased and # re-generated from your development database when you run "rake". # Do not set this db to the same as development or production. test: - <<: *default - username: <%= ENV['POSTGRES_USER'] || `whoami` %> - database: caseflow_certification_test<%= ENV['TEST_SUBCATEGORY'] %> - <% if ENV['POSTGRES_HOST'] %> - host: <%= ENV['POSTGRES_HOST'] %> - <% end %> - <% if ENV['POSTGRES_PASSWORD'] %> - password: <%= ENV['POSTGRES_PASSWORD'] %> - <% end %> + primary: + <<: *default + username: <%= ENV['POSTGRES_USER'] || `whoami` %> + database: caseflow_certification_test<%= ENV['TEST_SUBCATEGORY'] %> + <% if ENV['POSTGRES_HOST'] %> + host: <%= ENV['POSTGRES_HOST'] %> + <% end %> + <% if ENV['POSTGRES_PASSWORD'] %> + password: <%= ENV['POSTGRES_PASSWORD'] %> + <% end %> + etl: + <<: *etl_default + database: caseflow_etl_test<%= ENV["TEST_SUBCATEGORY"] %> + # Increase the receive timeout value due to ORA-12609: TNS: Receive timeout occurred + # Solution: https://www.fatihacar.com/blog/category/databases/oracle/oracle-errors-and-solutions/page/13/ + # Oracle timeout configuration: https://github.com/rsim/oracle-enhanced#timeouts + vacols: + adapter: oracle_enhanced + username: VACOLS_TEST + password: VACOLS_TEST + database: "(DESCRIPTION= + (ADDRESS_LIST=(ADDRESS=(PROTOCOL=tcp)(HOST=<%= ENV['TEST_VACOLS_HOST']%>)(PORT=1521))) + (RECV_TIMEOUT=120)(SEND_TIMEOUT=5) + (CONNECT_DATA=(SID=BVAP)) + )" + +# For use by GH workflow `Make-docs-to-webpage` (See `.github/workflows/make-docs.yml`) +make_docs: + primary: + <<: *default + username: <%= ENV['POSTGRES_USER'] || `whoami` %> + database: caseflow_certification_test<%= ENV['TEST_SUBCATEGORY'] %> + <% if ENV['POSTGRES_HOST'] %> + host: <%= ENV['POSTGRES_HOST'] %> + <% end %> + <% if ENV['POSTGRES_PASSWORD'] %> + password: <%= ENV['POSTGRES_PASSWORD'] %> + <% end %> + etl: + <<: *etl_default + database: caseflow_etl_test<%= ENV["TEST_SUBCATEGORY"] %> + # Even though it won't be used, Rails expects a 'vacols' DB configuration due to + # VACOLS::Record `establish_connection :vacols` + vacols: + adapter: postgresql + pool: 1 + timeout: 5000 + database: test-vacols + +demo: + primary: + <<: *default + database: caseflow_certification_demo + vacols: + adapter: sqlite3 + pool: 5 + timeout: 5000 + database: db/demo-vacols.sqlite3 -# Increase the receive timeout value due to ORA-12609: TNS: Receive timeout occurred -# Solution: https://www.fatihacar.com/blog/category/databases/oracle/oracle-errors-and-solutions/page/13/ -# Oracle timeout configuration: https://github.com/rsim/oracle-enhanced#timeouts -test_vacols: - adapter: oracle_enhanced - username: VACOLS_TEST - password: VACOLS_TEST - database: "(DESCRIPTION= - (ADDRESS_LIST=(ADDRESS=(PROTOCOL=tcp)(HOST=<%= ENV['TEST_VACOLS_HOST']%>)(PORT=1521))) - (RECV_TIMEOUT=120)(SEND_TIMEOUT=5) - (CONNECT_DATA=(SID=BVAP)) - )" +ssh_forwarding: + primary: + <<: *default + database: caseflow_certification_ssh_forwarding + vacols: + adapter: oracle_enhanced + host: localhost + port: 1526 + username: <%= ENV["VACOLS_USERNAME"] %> + password: <%= ENV["VACOLS_PASSWORD"] %> + database: BVAP + +staging: + primary: + <<: *default + database: caseflow_certification_staging + vacols: + adapter: oracle_enhanced + username: <%= ENV["VACOLS_USERNAME"] %> + password: <%= ENV["VACOLS_PASSWORD"] %> + host: vacols.dev.vaco.va.gov + port: 1526 + database: BVAP.VACO.VA.GOV + +production: + primary: + <<: *default + url: <%= ENV["POSTGRES_URL"] %> + etl: + <<: *etl_default + url: <%= ENV["ETL_DB_URL"] %> + vacols: + adapter: oracle_enhanced + username: <%= ENV["VACOLS_USERNAME"] %> + password: <%= ENV["VACOLS_PASSWORD"] %> + host: <%= ENV["VACOLS_HOST"] %> + port: <%= ENV["VACOLS_PORT"] %> + database: <%= ENV["VACOLS_DATABASE"] %> + pool: <%= ENV["DB_CONN_POOL_MAX_SIZE"] || 5 %> diff --git a/config/environment.rb b/config/environment.rb index 426333bb469..cac53157752 100644 --- a/config/environment.rb +++ b/config/environment.rb @@ -1,5 +1,5 @@ # Load the Rails application. -require_relative 'application' +require_relative "application" # Initialize the Rails application. Rails.application.initialize! diff --git a/config/environments/development.rb b/config/environments/development.rb index c56b036c14a..c2c8b84f409 100644 --- a/config/environments/development.rb +++ b/config/environments/development.rb @@ -1,8 +1,16 @@ +require "active_support/core_ext/integer/time" +require_relative "../../config/initializers/deprecation_warnings" + Rails.application.configure do # Settings specified here will take precedence over those in config/application.rb. - # In the development environment your application's code is reloaded on - # every request. This slows down response time but is perfect for development + # When `config.assets.debug == true`, there is an edge case where the length of the Link header could become + # exceptionally long, due to the way concatenated assets are split and included separately, thus exceeding the + # maximum 8192 bytes for HTTP response headers. To preclude this from happening, we override the default here: + config.action_view.preload_links_header = false + + # In the development environment your application's code is reloaded any time + # it changes. This slows down response time but is perfect for development # since you don't have to restart the web server when you make code changes. config.cache_classes = false @@ -45,9 +53,13 @@ end # Print deprecation notices to the Rails logger. - # config.active_support.deprecation = :log - require_relative "../../app/services/deprecation_warnings/development_handler" - ActiveSupport::Deprecation.behavior = DeprecationWarnings::DevelopmentHandler + config.active_support.deprecation = :log + + # Raise exceptions for disallowed deprecations. + config.active_support.disallowed_deprecation = :raise + + # Tell Active Support which deprecation messages to disallow. + config.active_support.disallowed_deprecation_warnings = DeprecationWarnings::DISALLOWED_DEPRECATION_WARNING_REGEXES # Raise an error on page load if there are pending migrations. config.active_record.migration_error = :page_load @@ -64,7 +76,10 @@ config.assets.quiet = true # Raises error for missing translations. - # config.action_view.raise_on_missing_translations = true + # config.i18n.raise_on_missing_translations = true + + # Annotate rendered view with file names. + # config.action_view.annotate_rendered_view_with_filenames = true # Use an evented file watcher to asynchronously detect changes in source code, # routes, locales, etc. This feature depends on the listen gem. @@ -73,6 +88,9 @@ # Bypass DNS rebinding protection for all `demo` sub-domains config.hosts << ".demo.appeals.va.gov" + # Uncomment if you wish to allow Action Cable access from any origin. + # config.action_cable.disable_request_forgery_protection = true + #===================================================================================================================== # Please keep custom config settings below this comment. # This will ensure cleaner diffs when generating config file changes during Rails upgrades. diff --git a/config/environments/make_docs.rb b/config/environments/make_docs.rb new file mode 100644 index 00000000000..fc7a8460dc3 --- /dev/null +++ b/config/environments/make_docs.rb @@ -0,0 +1,152 @@ +require "active_support/core_ext/integer/time" +require "fileutils" + +# The test environment is used exclusively to run your application's +# test suite. You never need to work with it otherwise. Remember that +# your test database is "scratch space" for the test suite and is wiped +# and recreated between test runs. Don't rely on the data there! + +Rails.application.configure do + # Settings specified here will take precedence over those in config/application.rb. + + config.cache_classes = true + + cache_dir = Rails.root.join("tmp", "cache", "test_#{ENV['TEST_SUBCATEGORY']}", $$.to_s) + FileUtils.mkdir_p(cache_dir) unless File.exists?(cache_dir) + config.cache_store = :file_store, cache_dir + + # Do not eager load code on boot. This avoids loading your whole application + # just for the purpose of running a single test. If you are using a tool that + # preloads Rails for running tests, you may have to set it to true. + config.eager_load = false + + # Configure public file server for tests with Cache-Control for performance. + config.public_file_server.enabled = true + config.public_file_server.headers = { + 'Cache-Control' => "public, max-age=#{1.hour.to_i}" + } + + # Show full error reports and disable caching. + config.consider_all_requests_local = false + config.action_controller.perform_caching = false + + # Raise exceptions instead of rendering exception templates. + config.action_dispatch.show_exceptions = true + + # Disable request forgery protection in test environment. + config.action_controller.allow_forgery_protection = false + + # Store uploaded files on the local file system in a temporary directory. + config.active_storage.service = :test + + config.action_mailer.perform_caching = false + + # Tell Action Mailer not to deliver emails to the real world. + # The :test delivery method accumulates sent emails in the + # ActionMailer::Base.deliveries array. + config.action_mailer.delivery_method = :test + + unless ENV['RAILS_ENABLE_TEST_LOG'] + config.logger = Logger.new(nil) + config.log_level = :error + end + + # Raises error for missing translations. + # config.i18n.raise_on_missing_translations = true + + # Annotate rendered view with file names. + # config.action_view.annotate_rendered_view_with_filenames = true + + #===================================================================================================================== + # Please keep custom config settings below this comment. + # This will ensure cleaner diffs when generating config file changes during Rails upgrades. + #===================================================================================================================== + + config.after_initialize do + Bullet.enable = false + Bullet.bullet_logger = true + Bullet.rails_logger = true + Bullet.raise = true + Bullet.unused_eager_loading_enable = false + end + + ENV["VA_DOT_GOV_API_URL"] = "https://sandbox-api.va.gov/" + + # For testing uncertification methods + ENV["TEST_USER_ID"] = "TEST_USER_ID" + ENV["TEST_APPEAL_IDS"] = "123C,456D,678E" + ENV["FULL_GRANT_IDS"] = "VACOLS123,VACOLS234,VACOLS345,VACOLS456" + ENV["PARTIAL_AND_REMAND_IDS"] = "VACOLS321,VACOLS432,VACOLS543,VACOLS654" + + ENV["AWS_ACCESS_KEY_ID"] ||= "dummykeyid" + ENV["AWS_SECRET_ACCESS_KEY"] ||= "dummysecretkey" + + # BatchProcess ENVs + # priority_ep_sync + ENV["BATCH_PROCESS_JOB_DURATION"] ||= "50" # Number of minutes the job will run for + ENV["BATCH_PROCESS_SLEEP_DURATION"] ||= "0" # Number of seconds between loop iterations + ENV["BATCH_PROCESS_BATCH_LIMIT"]||= "100" # Max number of records in a batch + ENV["BATCH_PROCESS_ERROR_DELAY"] ||= "3" # In number of hours + ENV["BATCH_PROCESS_MAX_ERRORS_BEFORE_STUCK"] ||= "3" # When record errors for X time, it's declared stuck + + # RequestIssue paginates_per offset (vbms intake) + ENV["REQUEST_ISSUE_PAGINATION_OFFSET"] ||= "10" + ENV["REQUEST_ISSUE_DEFAULT_UPPER_BOUND_PER_PAGE"] ||= "50" + + config.active_job.queue_adapter = :test + + # Disable SqlTracker from creating tmp/sql_tracker-*.json files -- https://github.com/steventen/sql_tracker/pull/10 + SqlTracker::Config.enabled = false + + # Setup S3 + config.s3_enabled = false + + config.vacols_db_name = "VACOLS_TEST" + + if ENV["TEST_SUBCATEGORY"] + assets_cache_path = Rails.root.join("tmp/cache/assets/#{ENV['TEST_SUBCATEGORY']}") + config.assets.configure do |env| + env.cache = Sprockets::Cache::FileStore.new(assets_cache_path) + end + end + + # Allows rake scripts to be run without querying VACOLS on startup + if ENV["DISABLE_FACTORY_BOT_INITIALIZERS"] + config.factory_bot.definition_file_paths = [] + end + + # VA Notify environment variables + ENV["VA_NOTIFY_API_URL"] ||= "https://staging-api.va.gov/vanotify" + ENV["VA_NOTIFY_API_KEY"] ||= "secret-key" + ENV["VA_NOTIFY_SERVICE_ID"] ||= "fake-service-id" + ENV["VA_NOTIFY_TOKEN_ALG"] ||= "my-secret-algorithm" + ENV["VA_NOTIFY_STATUS_UPDATE_BATCH_LIMIT"] ||= "650" + + # One time Appeal States migration for Legacy & AMA Appeal Batch Sizes + ENV["STATE_MIGRATION_JOB_BATCH_SIZE"] ||= "1000" + + # Quarterly Notifications Batch Sizes + ENV["QUARTERLY_NOTIFICATIONS_JOB_BATCH_SIZE"] ||= "1000" + + # Travel Board Sync Batch Size + ENV["TRAVEL_BOARD_HEARING_SYNC_BATCH_LIMIT"] ||= "250" + + # Time in seconds before the sync lock expires + LOCK_TIMEOUT = ENV["SYNC_LOCK_MAX_DURATION"] ||= "60" + + # Notifications page eFolder link + ENV["CLAIM_EVIDENCE_EFOLDER_BASE_URL"] ||= "https://vefs-claimevidence-ui-uat.stage.bip.va.gov" + + ENV['TEST_VACOLS_HOST'] ||= "localhost" + + # Pacman environment variables + ENV["PACMAN_API_TOKEN_ALG"] ||= "HS512" + ENV["PACMAN_API_URL"] ||= "https://pacman-uat.dev.bip.va.gov" + ENV["PACMAN_API_SAML_TOKEN"] ||= "our-saml-token" + ENV["PACMAN_API_TOKEN_SECRET"] ||= "client-secret" + ENV["PACMAN_API_TOKEN_ISSUER"] ||= "issuer-of-our-token" + ENV["PACMAN_API_SYS_ACCOUNT"] ||= "CSS_ID_OF_OUR_ACCOUNT" + + # Dynatrace variables + ENV["STATSD_ENV"] = "test" +end diff --git a/config/environments/production.rb b/config/environments/production.rb index c1b6847d036..f1d8fe761c6 100644 --- a/config/environments/production.rb +++ b/config/environments/production.rb @@ -1,3 +1,7 @@ +require "active_support/core_ext/integer/time" +require_relative "../../app/services/deprecation_warnings/production_handler" +require_relative "../../config/initializers/deprecation_warnings" + Rails.application.configure do # Settings specified here will take precedence over those in config/application.rb. @@ -29,7 +33,7 @@ config.assets.compile = false # Enable serving of images, stylesheets, and JavaScripts from an asset server. - # config.action_controller.asset_host = 'http://assets.example.com' + # config.asset_host = 'http://assets.example.com' # Specifies the header that your server uses for sending files. # config.action_dispatch.x_sendfile_header = 'X-Sendfile' # for Apache @@ -44,7 +48,10 @@ # config.action_cable.allowed_request_origins = [ 'http://example.com', /http:\/\/example.*/ ] # Force all access to the app over SSL, use Strict-Transport-Security, and use secure cookies. - # config.force_ssl = true + config.force_ssl = true + + require_relative "../../app/policies/ssl_redirect_exclusion_policy" + config.ssl_options = { redirect: { exclude: SslRedirectExclusionPolicy } } # Use the lowest log level to ensure availability of diagnostic information # when problems arise. @@ -77,16 +84,20 @@ # the I18n.default_locale when a translation cannot be found). config.i18n.fallbacks = true - # Send deprecation notices to registered listeners. - # config.active_support.deprecation = :notify - require_relative "../../app/services/deprecation_warnings/production_handler" - ActiveSupport::Deprecation.behavior = DeprecationWarnings::ProductionHandler + # Send deprecation notices to handler. + config.active_support.deprecation = DeprecationWarnings::ProductionHandler + + # Log disallowed deprecations. + config.active_support.disallowed_deprecation = :log + + # Tell Active Support which deprecation messages to disallow. + config.active_support.disallowed_deprecation_warnings = DeprecationWarnings::DISALLOWED_DEPRECATION_WARNING_REGEXES # Use default logging formatter so that PID and timestamp are not suppressed. config.log_formatter = ::Logger::Formatter.new # Use a different logger for distributed setups. - # require 'syslog/logger' + # require "syslog/logger" # config.logger = ActiveSupport::TaggedLogging.new(Syslog::Logger.new 'app-name') if ENV["RAILS_LOG_TO_STDOUT"].present? diff --git a/config/environments/test.rb b/config/environments/test.rb index 83e3a687dd0..49d5e11344f 100644 --- a/config/environments/test.rb +++ b/config/environments/test.rb @@ -1,4 +1,6 @@ +require "active_support/core_ext/integer/time" require "fileutils" +require_relative "../../config/initializers/deprecation_warnings" # The test environment is used exclusively to run your application's # test suite. You never need to work with it otherwise. Remember that @@ -46,19 +48,24 @@ config.action_mailer.delivery_method = :test # Print deprecation notices to the stderr. - # config.active_support.deprecation = :stderr - require_relative "../../app/services/deprecation_warnings/test_handler" - ActiveSupport::Deprecation.behavior = DeprecationWarnings::TestHandler + config.active_support.deprecation = :stderr + + # Raise exceptions for disallowed deprecations. + config.active_support.disallowed_deprecation = :raise + + # Tell Active Support which deprecation messages to disallow. + config.active_support.disallowed_deprecation_warnings = DeprecationWarnings::DISALLOWED_DEPRECATION_WARNING_REGEXES unless ENV['RAILS_ENABLE_TEST_LOG'] config.logger = Logger.new(nil) config.log_level = :error end - config.action_mailer.delivery_method = :test - # Raises error for missing translations. - # config.action_view.raise_on_missing_translations = true + # config.i18n.raise_on_missing_translations = true + + # Annotate rendered view with file names. + # config.action_view.annotate_rendered_view_with_filenames = true #===================================================================================================================== # Please keep custom config settings below this comment. diff --git a/config/initializers/backtrace_silencers.rb b/config/initializers/backtrace_silencers.rb index 59385cdf379..33699c30910 100644 --- a/config/initializers/backtrace_silencers.rb +++ b/config/initializers/backtrace_silencers.rb @@ -1,7 +1,8 @@ # Be sure to restart your server when you modify this file. # You can add backtrace silencers for libraries that you're using but don't wish to see in your backtraces. -# Rails.backtrace_cleaner.add_silencer { |line| line =~ /my_noisy_library/ } +# Rails.backtrace_cleaner.add_silencer { |line| /my_noisy_library/.match?(line) } -# You can also remove all the silencers if you're trying to debug a problem that might stem from framework code. -# Rails.backtrace_cleaner.remove_silencers! +# You can also remove all the silencers if you're trying to debug a problem that might stem from framework code +# by setting BACKTRACE=1 before calling your invocation, like "BACKTRACE=1 ./bin/rails runner 'MyClass.perform'". +Rails.backtrace_cleaner.remove_silencers! if ENV["BACKTRACE"] diff --git a/app/services/deprecation_warnings/disallowed_deprecations.rb b/config/initializers/deprecation_warnings.rb similarity index 64% rename from app/services/deprecation_warnings/disallowed_deprecations.rb rename to config/initializers/deprecation_warnings.rb index df4f14a24b1..4d102c4e702 100644 --- a/app/services/deprecation_warnings/disallowed_deprecations.rb +++ b/config/initializers/deprecation_warnings.rb @@ -1,10 +1,10 @@ # frozen_string_literal: true -# @note Temporary solution for disallowed deprecation warnings. -# To be replaced by ActiveSupport Disallowed Deprecations after upgrading to Rails 6.1: -# https://rubyonrails.org/2020/12/9/Rails-6-1-0-release#disallowed-deprecation-support -module DisallowedDeprecations - class ::DisallowedDeprecationError < StandardError; end +module DeprecationWarnings + # Regular expressions for custom deprecation warnings that we have addressed in the codebase + CUSTOM_DEPRECATION_WARNING_REGEXES = [ + /Caseflow::Migration is deprecated/ + ].freeze # Regular expressions for Rails 6.0 deprecation warnings that we have addressed in the codebase RAILS_6_0_FIXED_DEPRECATION_WARNING_REGEXES = [ @@ -19,19 +19,15 @@ class ::DisallowedDeprecationError < StandardError; end /ActionView::Base instances must implement `compiled_method_container`/, /render file: should be given the absolute path to a file/, /`ActiveRecord::Result#to_hash` has been renamed to `to_a`/, - /Class level methods will no longer inherit scoping/ + /Class level methods will no longer inherit scoping/, + /Controller-level `force_ssl` is deprecated and will be removed from Rails 6\.1/, + /NOT conditions will no longer behave as NOR in Rails 6\.1/ ].freeze # Regular expressions for deprecation warnings that should raise an exception on detection DISALLOWED_DEPRECATION_WARNING_REGEXES = [ + *CUSTOM_DEPRECATION_WARNING_REGEXES, *RAILS_6_0_FIXED_DEPRECATION_WARNING_REGEXES, *RAILS_6_1_FIXED_DEPRECATION_WARNING_REGEXES ].freeze - - # @param message [String] deprecation warning message to be checked against disallow list - def raise_if_disallowed_deprecation!(message) - if DISALLOWED_DEPRECATION_WARNING_REGEXES.any? { |re| re.match?(message) } - fail DisallowedDeprecationError, message - end - end end diff --git a/config/initializers/filter_parameter_logging.rb b/config/initializers/filter_parameter_logging.rb index 4a994e1e7bb..4b34a036689 100644 --- a/config/initializers/filter_parameter_logging.rb +++ b/config/initializers/filter_parameter_logging.rb @@ -1,4 +1,6 @@ # Be sure to restart your server when you modify this file. # Configure sensitive parameters which will be filtered from the log file. -Rails.application.config.filter_parameters += [:password] +Rails.application.config.filter_parameters += [ + :passw, :secret, :token, :_key, :crypt, :salt, :certificate, :otp, :ssn +] diff --git a/config/initializers/permissions_policy.rb b/config/initializers/permissions_policy.rb new file mode 100644 index 00000000000..00f64d71b03 --- /dev/null +++ b/config/initializers/permissions_policy.rb @@ -0,0 +1,11 @@ +# Define an application-wide HTTP permissions policy. For further +# information see https://developers.google.com/web/updates/2018/06/feature-policy +# +# Rails.application.config.permissions_policy do |f| +# f.camera :none +# f.gyroscope :none +# f.microphone :none +# f.usb :none +# f.fullscreen :self +# f.payment :self, "https://secure.example.com" +# end diff --git a/config/initializers/strong_migrations.rb b/config/initializers/strong_migrations.rb index c079aa8007b..c6ebfb8c42d 100644 --- a/config/initializers/strong_migrations.rb +++ b/config/initializers/strong_migrations.rb @@ -1,2 +1,20 @@ StrongMigrations.start_after = 20190111000717 StrongMigrations.auto_analyze = true + +# Customized error message when a new migration uses `add_index` non-concurrently +StrongMigrations.error_messages[:add_index] = <<~TEXT + Adding a non-concurrent index locks the table for writes. + Instead, prefer using `Caseflow::Migrations::AddIndexConcurrently #add_safe_index`: + + class YourMigrationName < ActiveRecord::Migration%{migration_suffix} + include Caseflow::Migrations::AddIndexConcurrently + + def change + add_safe_index # add args here... + end + end + + Nota Bene: Since adding indexes concurrently must occur outside of a transaction, + one should avoid mixing in other DB changes when doing so. It is strongly recommended + to segregate index additions from other DB changes in a separate migration. + TEXT diff --git a/config/initializers/warmup_vacols.rb b/config/initializers/warmup_vacols.rb index 68f5db2f460..053b49551f7 100644 --- a/config/initializers/warmup_vacols.rb +++ b/config/initializers/warmup_vacols.rb @@ -20,7 +20,7 @@ db_config = Rails.application.config.database_configuration[Rails.env] # use specified initial pool size, default to half the maximum size - initial_pool_size = (ENV['DB_CONN_POOL_INITIAL_SIZE'] || db_config['pool'] / 2).to_i + initial_pool_size = (ENV['DB_CONN_POOL_INITIAL_SIZE'] || db_config['primary']['pool'] / 2).to_i Rails.logger.info("creating #{initial_pool_size} initial connections...") unless ApplicationController.dependencies_faked? diff --git a/config/puma.rb b/config/puma.rb index b2c644daa93..b1b0f09e2da 100644 --- a/config/puma.rb +++ b/config/puma.rb @@ -8,9 +8,14 @@ min_threads_count = ENV.fetch("RAILS_MIN_THREADS") { max_threads_count } threads min_threads_count, max_threads_count +# Specifies the `worker_timeout` threshold that Puma will use to wait before +# terminating a worker in development environments. +# +worker_timeout 3600 if ENV.fetch("RAILS_ENV", "development") == "development" + # Specifies the `port` that Puma will listen on to receive requests; default is 3000. # -port ENV.fetch("PORT") { 3000 } +port ENV.fetch("PORT") { 3000 } # Specifies the `environment` that Puma will run in. # diff --git a/config/routes.rb b/config/routes.rb index 9662aaea056..ef67aeffb8e 100644 --- a/config/routes.rb +++ b/config/routes.rb @@ -2,8 +2,6 @@ mount Rswag::Ui::Engine => '/api-docs' mount Rswag::Api::Engine => '/api-docs' # For details on the DSL available within this file, see https://guides.rubyonrails.org/routing.html - # The priority is based upon order of creation: first created -> highest priority. - # See how all your routes lay out with "rake routes". resources :sessions, only: [:new, :update] resources :certifications, path_names: { new: "new/:vacols_id" } do diff --git a/db/etl/migrate/20191111164808_create_etl_appeals.rb b/db/etl_migrate/20191111164808_create_etl_appeals.rb similarity index 100% rename from db/etl/migrate/20191111164808_create_etl_appeals.rb rename to db/etl_migrate/20191111164808_create_etl_appeals.rb diff --git a/db/etl/migrate/20191119204827_create_etl_users.rb b/db/etl_migrate/20191119204827_create_etl_users.rb similarity index 100% rename from db/etl/migrate/20191119204827_create_etl_users.rb rename to db/etl_migrate/20191119204827_create_etl_users.rb diff --git a/db/etl/migrate/20191120184256_create_etl_tasks.rb b/db/etl_migrate/20191120184256_create_etl_tasks.rb similarity index 100% rename from db/etl/migrate/20191120184256_create_etl_tasks.rb rename to db/etl_migrate/20191120184256_create_etl_tasks.rb diff --git a/db/etl/migrate/20191120213930_create_etl_people.rb b/db/etl_migrate/20191120213930_create_etl_people.rb similarity index 100% rename from db/etl/migrate/20191120213930_create_etl_people.rb rename to db/etl_migrate/20191120213930_create_etl_people.rb diff --git a/db/etl/migrate/20191120213941_create_etl_organizations.rb b/db/etl_migrate/20191120213941_create_etl_organizations.rb similarity index 100% rename from db/etl/migrate/20191120213941_create_etl_organizations.rb rename to db/etl_migrate/20191120213941_create_etl_organizations.rb diff --git a/db/etl/migrate/20191126164353_remove_users_vacols_not_null.rb b/db/etl_migrate/20191126164353_remove_users_vacols_not_null.rb similarity index 100% rename from db/etl/migrate/20191126164353_remove_users_vacols_not_null.rb rename to db/etl_migrate/20191126164353_remove_users_vacols_not_null.rb diff --git a/db/etl/migrate/20191126203511_add_appeal_timestamps.rb b/db/etl_migrate/20191126203511_add_appeal_timestamps.rb similarity index 100% rename from db/etl/migrate/20191126203511_add_appeal_timestamps.rb rename to db/etl_migrate/20191126203511_add_appeal_timestamps.rb diff --git a/db/etl/migrate/20191209162326_add_appeal_status_sort.rb b/db/etl_migrate/20191209162326_add_appeal_status_sort.rb similarity index 100% rename from db/etl/migrate/20191209162326_add_appeal_status_sort.rb rename to db/etl_migrate/20191209162326_add_appeal_status_sort.rb diff --git a/db/etl/migrate/20191212221851_org_status_change.rb b/db/etl_migrate/20191212221851_org_status_change.rb similarity index 100% rename from db/etl/migrate/20191212221851_org_status_change.rb rename to db/etl_migrate/20191212221851_org_status_change.rb diff --git a/db/etl/migrate/20191212222617_org_status_default.rb b/db/etl_migrate/20191212222617_org_status_default.rb similarity index 100% rename from db/etl/migrate/20191212222617_org_status_default.rb rename to db/etl_migrate/20191212222617_org_status_default.rb diff --git a/db/etl/migrate/20191213194037_add_task_id_index.rb b/db/etl_migrate/20191213194037_add_task_id_index.rb similarity index 100% rename from db/etl/migrate/20191213194037_add_task_id_index.rb rename to db/etl_migrate/20191213194037_add_task_id_index.rb diff --git a/db/etl/migrate/20191217193046_create_attorney_case_reviews.rb b/db/etl_migrate/20191217193046_create_attorney_case_reviews.rb similarity index 100% rename from db/etl/migrate/20191217193046_create_attorney_case_reviews.rb rename to db/etl_migrate/20191217193046_create_attorney_case_reviews.rb diff --git a/db/etl/migrate/20191217202614_create_decision_issues.rb b/db/etl_migrate/20191217202614_create_decision_issues.rb similarity index 100% rename from db/etl/migrate/20191217202614_create_decision_issues.rb rename to db/etl_migrate/20191217202614_create_decision_issues.rb diff --git a/db/etl/migrate/20191218214351_add_more_cached_user_attributes.rb b/db/etl_migrate/20191218214351_add_more_cached_user_attributes.rb similarity index 100% rename from db/etl/migrate/20191218214351_add_more_cached_user_attributes.rb rename to db/etl_migrate/20191218214351_add_more_cached_user_attributes.rb diff --git a/db/etl/migrate/20191223213954_add_email_address_to_person.rb b/db/etl_migrate/20191223213954_add_email_address_to_person.rb similarity index 100% rename from db/etl/migrate/20191223213954_add_email_address_to_person.rb rename to db/etl_migrate/20191223213954_add_email_address_to_person.rb diff --git a/db/etl/migrate/20200114201144_add_aod_due_to_age.rb b/db/etl_migrate/20200114201144_add_aod_due_to_age.rb similarity index 100% rename from db/etl/migrate/20200114201144_add_aod_due_to_age.rb rename to db/etl_migrate/20200114201144_add_aod_due_to_age.rb diff --git a/db/etl/migrate/20200116215235_add_claimant_dob_to_appeals.rb b/db/etl_migrate/20200116215235_add_claimant_dob_to_appeals.rb similarity index 100% rename from db/etl/migrate/20200116215235_add_claimant_dob_to_appeals.rb rename to db/etl_migrate/20200116215235_add_claimant_dob_to_appeals.rb diff --git a/db/etl/migrate/20200121221718_add_appeals_claimant_indexes.rb b/db/etl_migrate/20200121221718_add_appeals_claimant_indexes.rb similarity index 100% rename from db/etl/migrate/20200121221718_add_appeals_claimant_indexes.rb rename to db/etl_migrate/20200121221718_add_appeals_claimant_indexes.rb diff --git a/db/etl/migrate/20200212205344_add_etl_build_tables.rb b/db/etl_migrate/20200212205344_add_etl_build_tables.rb similarity index 100% rename from db/etl/migrate/20200212205344_add_etl_build_tables.rb rename to db/etl_migrate/20200212205344_add_etl_build_tables.rb diff --git a/db/etl/migrate/20200303223155_add_etl_users_index.rb b/db/etl_migrate/20200303223155_add_etl_users_index.rb similarity index 100% rename from db/etl/migrate/20200303223155_add_etl_users_index.rb rename to db/etl_migrate/20200303223155_add_etl_users_index.rb diff --git a/db/etl/migrate/20200429152715_create_vha_decision_reviews.rb b/db/etl_migrate/20200429152715_create_vha_decision_reviews.rb similarity index 100% rename from db/etl/migrate/20200429152715_create_vha_decision_reviews.rb rename to db/etl_migrate/20200429152715_create_vha_decision_reviews.rb diff --git a/db/etl/migrate/20200429164923_create_hearings.rb b/db/etl_migrate/20200429164923_create_hearings.rb similarity index 100% rename from db/etl/migrate/20200429164923_create_hearings.rb rename to db/etl_migrate/20200429164923_create_hearings.rb diff --git a/db/etl/migrate/20200430152234_add_hearing_type.rb b/db/etl_migrate/20200430152234_add_hearing_type.rb similarity index 100% rename from db/etl/migrate/20200430152234_add_hearing_type.rb rename to db/etl_migrate/20200430152234_add_hearing_type.rb diff --git a/db/etl/migrate/20200501211351_change_hearing_day_id_null.rb b/db/etl_migrate/20200501211351_change_hearing_day_id_null.rb similarity index 100% rename from db/etl/migrate/20200501211351_change_hearing_day_id_null.rb rename to db/etl_migrate/20200501211351_change_hearing_day_id_null.rb diff --git a/db/etl/migrate/20200504133723_change_css_limit.rb b/db/etl_migrate/20200504133723_change_css_limit.rb similarity index 100% rename from db/etl/migrate/20200504133723_change_css_limit.rb rename to db/etl_migrate/20200504133723_change_css_limit.rb diff --git a/db/etl/migrate/20200504204154_remove_null_hearings.rb b/db/etl_migrate/20200504204154_remove_null_hearings.rb similarity index 100% rename from db/etl/migrate/20200504204154_remove_null_hearings.rb rename to db/etl_migrate/20200504204154_remove_null_hearings.rb diff --git a/db/etl/migrate/20200508141923_drop_legacy_hearing_scheduled_for.rb b/db/etl_migrate/20200508141923_drop_legacy_hearing_scheduled_for.rb similarity index 100% rename from db/etl/migrate/20200508141923_drop_legacy_hearing_scheduled_for.rb rename to db/etl_migrate/20200508141923_drop_legacy_hearing_scheduled_for.rb diff --git a/db/etl/migrate/20200517021335_create_etl_remand_reason.rb b/db/etl_migrate/20200517021335_create_etl_remand_reason.rb similarity index 100% rename from db/etl/migrate/20200517021335_create_etl_remand_reason.rb rename to db/etl_migrate/20200517021335_create_etl_remand_reason.rb diff --git a/db/etl/migrate/20200519213755_change_vha_decision_reviews_to_decision_reviews.rb b/db/etl_migrate/20200519213755_change_vha_decision_reviews_to_decision_reviews.rb similarity index 100% rename from db/etl/migrate/20200519213755_change_vha_decision_reviews_to_decision_reviews.rb rename to db/etl_migrate/20200519213755_change_vha_decision_reviews_to_decision_reviews.rb diff --git a/db/etl/migrate/20200519214159_change_decision_reviews_comments.rb b/db/etl_migrate/20200519214159_change_decision_reviews_comments.rb similarity index 100% rename from db/etl/migrate/20200519214159_change_decision_reviews_comments.rb rename to db/etl_migrate/20200519214159_change_decision_reviews_comments.rb diff --git a/db/etl/migrate/20200519214329_change_decision_reviews_indices.rb b/db/etl_migrate/20200519214329_change_decision_reviews_indices.rb similarity index 100% rename from db/etl/migrate/20200519214329_change_decision_reviews_indices.rb rename to db/etl_migrate/20200519214329_change_decision_reviews_indices.rb diff --git a/db/etl/migrate/20200519221508_add_decision_issue_benefit_index.rb b/db/etl_migrate/20200519221508_add_decision_issue_benefit_index.rb similarity index 100% rename from db/etl/migrate/20200519221508_add_decision_issue_benefit_index.rb rename to db/etl_migrate/20200519221508_add_decision_issue_benefit_index.rb diff --git a/db/etl/migrate/20200520220924_add_person_ssn.rb b/db/etl_migrate/20200520220924_add_person_ssn.rb similarity index 100% rename from db/etl/migrate/20200520220924_add_person_ssn.rb rename to db/etl_migrate/20200520220924_add_person_ssn.rb diff --git a/db/etl/migrate/20200526054515_add_subject_text_and_percent_number_to_decision_issue.rb b/db/etl_migrate/20200526054515_add_subject_text_and_percent_number_to_decision_issue.rb similarity index 100% rename from db/etl/migrate/20200526054515_add_subject_text_and_percent_number_to_decision_issue.rb rename to db/etl_migrate/20200526054515_add_subject_text_and_percent_number_to_decision_issue.rb diff --git a/db/etl/migrate/20200715110034_add_accepts_priority_pushed_cases_to_etl_organization.rb b/db/etl_migrate/20200715110034_add_accepts_priority_pushed_cases_to_etl_organization.rb similarity index 100% rename from db/etl/migrate/20200715110034_add_accepts_priority_pushed_cases_to_etl_organization.rb rename to db/etl_migrate/20200715110034_add_accepts_priority_pushed_cases_to_etl_organization.rb diff --git a/db/etl/migrate/20200715110603_add_accepts_priority_pushed_cases_index_on_organization.rb b/db/etl_migrate/20200715110603_add_accepts_priority_pushed_cases_index_on_organization.rb similarity index 100% rename from db/etl/migrate/20200715110603_add_accepts_priority_pushed_cases_index_on_organization.rb rename to db/etl_migrate/20200715110603_add_accepts_priority_pushed_cases_index_on_organization.rb diff --git a/db/etl/migrate/20210729124530_remove_etl_decision_issues_index.rb b/db/etl_migrate/20210729124530_remove_etl_decision_issues_index.rb similarity index 100% rename from db/etl/migrate/20210729124530_remove_etl_decision_issues_index.rb rename to db/etl_migrate/20210729124530_remove_etl_decision_issues_index.rb diff --git a/db/etl/migrate/20210729130353_add_etl_decision_issues_rating_issue_ref_id_index.rb b/db/etl_migrate/20210729130353_add_etl_decision_issues_rating_issue_ref_id_index.rb similarity index 100% rename from db/etl/migrate/20210729130353_add_etl_decision_issues_rating_issue_ref_id_index.rb rename to db/etl_migrate/20210729130353_add_etl_decision_issues_rating_issue_ref_id_index.rb diff --git a/db/etl/migrate/20210810115309_create_etl_judge_case_review.rb b/db/etl_migrate/20210810115309_create_etl_judge_case_review.rb similarity index 100% rename from db/etl/migrate/20210810115309_create_etl_judge_case_review.rb rename to db/etl_migrate/20210810115309_create_etl_judge_case_review.rb diff --git a/db/etl/migrate/20210810115459_create_etl_decision_document.rb b/db/etl_migrate/20210810115459_create_etl_decision_document.rb similarity index 100% rename from db/etl/migrate/20210810115459_create_etl_decision_document.rb rename to db/etl_migrate/20210810115459_create_etl_decision_document.rb diff --git a/db/etl/migrate/20210818172716_add_ama_only_to_organization.rb b/db/etl_migrate/20210818172716_add_ama_only_to_organization.rb similarity index 100% rename from db/etl/migrate/20210818172716_add_ama_only_to_organization.rb rename to db/etl_migrate/20210818172716_add_ama_only_to_organization.rb diff --git a/db/etl/migrate/20220107125705_remove_non_null_constraint_on_appeal_docket_type.rb b/db/etl_migrate/20220107125705_remove_non_null_constraint_on_appeal_docket_type.rb similarity index 100% rename from db/etl/migrate/20220107125705_remove_non_null_constraint_on_appeal_docket_type.rb rename to db/etl_migrate/20220107125705_remove_non_null_constraint_on_appeal_docket_type.rb diff --git a/db/etl/migrate/20220110143457_remove_non_null_constraint_on_appeal_receipt_date.rb b/db/etl_migrate/20220110143457_remove_non_null_constraint_on_appeal_receipt_date.rb similarity index 100% rename from db/etl/migrate/20220110143457_remove_non_null_constraint_on_appeal_receipt_date.rb rename to db/etl_migrate/20220110143457_remove_non_null_constraint_on_appeal_receipt_date.rb diff --git a/db/etl/migrate/20240129200527_add_exclude_appeals_from_affinity_to_organizations.rb b/db/etl_migrate/20240129200527_add_exclude_appeals_from_affinity_to_organizations.rb similarity index 100% rename from db/etl/migrate/20240129200527_add_exclude_appeals_from_affinity_to_organizations.rb rename to db/etl_migrate/20240129200527_add_exclude_appeals_from_affinity_to_organizations.rb diff --git a/db/etl/migrate/20240617205006_add_scheduled_in_timezone_to_hearings.rb b/db/etl_migrate/20240617205006_add_scheduled_in_timezone_to_hearings.rb similarity index 100% rename from db/etl/migrate/20240617205006_add_scheduled_in_timezone_to_hearings.rb rename to db/etl_migrate/20240617205006_add_scheduled_in_timezone_to_hearings.rb diff --git a/db/etl/schema.rb b/db/etl_schema.rb similarity index 100% rename from db/etl/schema.rb rename to db/etl_schema.rb diff --git a/docker-bin/env.sh b/docker-bin/env.sh index 80735dd9ab2..6a1acb1d207 100644 --- a/docker-bin/env.sh +++ b/docker-bin/env.sh @@ -13,8 +13,7 @@ export REDIS_URL_SIDEKIQ=redis://appeals-redis:6379 # envs to make development work in docker without affecting other devs export DOCKERIZED=true export DEMO_PORT=1521 -export DEMO_DB="(DESCRIPTION= - (ADDRESS_LIST=(ADDRESS=(PROTOCOL=tcp)(HOST=VACOLS_DB-development)(PORT=1521)))(RECV_TIMEOUT=120)(SEND_TIMEOUT=5)(CONNECT_DATA=(SID=BVAP)))" +export DEMO_DB="(DESCRIPTION=(ADDRESS_LIST=(ADDRESS=(PROTOCOL=tcp)(HOST=VACOLS_DB-development)(PORT=1521)))(RECV_TIMEOUT=120)(SEND_TIMEOUT=5)(CONNECT_DATA=(SID=BVAP)))" export PATH=/.yarn/bin:/.config/yarn/global/node_modules/.bin:/usr/local/nvm/versions/node/v16.16.0/bin:/usr/local/bundle/bin:/usr/local/bundle/gems/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/opt/oracle/instantclient_12_2: diff --git a/docker-bin/startup.sh b/docker-bin/startup.sh index 97a73eeae4e..5689eecc68d 100644 --- a/docker-bin/startup.sh +++ b/docker-bin/startup.sh @@ -25,7 +25,8 @@ echo "Waiting for Vacols to be ready" rake local:vacols:wait_for_connection echo "Creating DB in PG" -rake db:setup +rake db:create:primary +rake db:schema:load:primary echo "Seeding Facols" rake local:vacols:seed diff --git a/lib/caseflow/migration.rb b/lib/caseflow/migration.rb index 7f8ad1b01db..7435d5de8b6 100644 --- a/lib/caseflow/migration.rb +++ b/lib/caseflow/migration.rb @@ -1,8 +1,25 @@ # frozen_string_literal: true -# Migration with built-in timeout extensions for adding indexes - +# @deprecated Use {Caseflow::Migrations::AddIndexConcurrently} instead, because descendants of this class are forever +# coupled to Active Record 5.1. +# This class should be preserved until all descendant migrations <= LAST_DESCENDANT_MIGRATION_VERSION are pruned. +# +# @note Migration with built-in timeout extensions for adding indexes class Caseflow::Migration < ActiveRecord::Migration[5.1] + # version of last migration that inherits from this class (across 'primary' and 'etl' databases) + LAST_DESCENDANT_MIGRATION_VERSION = 20_240_617_205_006 + + def initialize(*) + super + # Trigger deprecation warning to prevent re-introduction in migrations after LAST_DESCENDANT_MIGRATION_VERSION + if version > LAST_DESCENDANT_MIGRATION_VERSION + ActiveSupport::Deprecation.warn( + "Caseflow::Migration is deprecated and should no longer be used.\n" \ + "If adding an index, see Caseflow::Migrations::AddIndexConcurrently." + ) + end + end + # hardcode this because setting via class method does not work in subclass def disable_ddl_transaction say "disable_ddl_transaction is true" diff --git a/lib/caseflow/migrations/add_index_concurrently.rb b/lib/caseflow/migrations/add_index_concurrently.rb new file mode 100644 index 00000000000..fc3b7ac155e --- /dev/null +++ b/lib/caseflow/migrations/add_index_concurrently.rb @@ -0,0 +1,101 @@ +# frozen_string_literal: true + +# @note Usage: Include this module only in migrations where you need to add an index concurrently. +# Invoke `#add_safe_index` in place of `#add index`. +# +# @note Since this module necessarily disables transactions, you should avoid mixing in other DB changes when +# adding indexes concurrently. Prefer isolating index additions in their own migration. +# +# @see [PostgreSQL: Building Indexes Concurrently](https://www.postgresql.org/docs/14/sql-createindex.html#SQL-CREATEINDEX-CONCURRENTLY) +# @see [thoughtbot: How to Create Postgres Indexes Concurrently in ActiveRecord Migrations](https://thoughtbot.com/blog/how-to-create-postgres-indexes-concurrently-in) +# +# @example +# class YourMigrationName < ActiveRecord::Migration[6.0] +# include Caseflow::Migrations::AddIndexConcurrently +# +# change +# add_safe_index :some_table, :some_column +# end +# end +# +# @example +# # Alternatively, you can add in the requisite incantations yourself, without this module +# +# class YourMigrationName < ActiveRecord::Migration[6.0] +# disable_ddl_transaction! +# +# change +# add_index :some_table, :some_column, algorithm: :concurrently +# rescue StandardError => error +# remove_index :some_table, :some_column +# end +# end +module Caseflow + module Migrations + module AddIndexConcurrently + extend ActiveSupport::Concern + + EXTENDED_STATEMENT_TIMEOUT_DURATION = 30.minutes + + included do + # Disables the automatic transaction wrapping this migration. + # https://github.com/rails/rails/blob/28bb76d3efc39b2ef663dfe2346f7c2621343cd6/activerecord/lib/active_record/migration.rb#L508-L524 + disable_ddl_transaction! + + # @note Use this method in place of `#add_index` to add an index concurrently (i.e. without blocking DB writes). + # - Accommodates long-running index builds by extending the statement_timeout duration + # - Performs a rollback should an error occur during the migration, since transactions are disabled + # + # @note Inspired by {Caseflow::Migration} https://github.com/department-of-veterans-affairs/caseflow/blob/6fc9d26a5ae9417b69d7d1f30cc70bea57a0700d/lib/caseflow/migration.rb#L12-L29 + def add_safe_index(*args) + original_statement_timeout_duration = current_statement_timeout_duration + extend_statement_timeout + add_index_concurrently(args) + rescue StandardError => error + rollback_index(error, args) + raise error # re-raise to abort migration + ensure + restore_original_statement_timeout(original_statement_timeout_duration) + end + + private + + def current_statement_timeout_duration + ActiveSupport::Duration.build( + ActiveRecord::Base.connection.execute("SHOW statement_timeout").first["statement_timeout"].to_i + ) + end + + def extend_statement_timeout + say "Extending statement_timeout to #{EXTENDED_STATEMENT_TIMEOUT_DURATION.inspect}" + ActiveRecord::Base.connection.execute( + ActiveRecord::Base.sanitize_sql(["SET statement_timeout = ?", + EXTENDED_STATEMENT_TIMEOUT_DURATION.in_milliseconds]) + ) + end + + def add_index_concurrently(args) + table, columns, options = *args + options ||= {} + options[:algorithm] ||= :concurrently + add_index(table, columns, options) + end + + def rollback_index(error, args) + say "Caught #{error}, rolling back index" + table, columns, options = *args + options[:column] = columns unless options[:name] + remove_index(table, options) + end + + # :reek:FeatureEnvy + def restore_original_statement_timeout(duration) + say "Restoring statement_timeout to #{duration.inspect}" + ActiveRecord::Base.connection.execute( + ActiveRecord::Base.sanitize_sql(["SET statement_timeout = ?", duration.in_milliseconds]) + ) + end + end + end + end +end diff --git a/lib/tasks/db.rake b/lib/tasks/db.rake new file mode 100644 index 00000000000..e56cb6d3404 --- /dev/null +++ b/lib/tasks/db.rake @@ -0,0 +1,300 @@ +# frozen_string_literal: true + +# Once on Rails 7+, we can add the `database_tasks: false` option to the `database.yml` to permit connection to the +# vacols database without generating database tasks for it: +# https://github.com/rails/rails/blob/984c3ef2775781d47efa9f541ce570daa2434a80/guides/source/active_record_multiple_databases.md?plain=1#L203-L219 +# +# After adopting and sufficiently testing the above setting setting, we can dispense with the workarounds in this file. +if Rails::VERSION::MAJOR >= 7 + ActiveSupport::Deprecation.warn( + "Use the new `database_tasks` DB config option to skip generation of database tasks for the VACOLS DB.\n" \ + "For further details, see https://github.com/rails/rails/blob/984c3ef2775781d47efa9f541ce570daa2434a80/guides/source/active_record_multiple_databases.md?plain=1#L203-L219" + ) +end + +# Explicitly clear any generated database tasks for the vacols DB. +%w[ + db:create:vacols + db:drop:vacols + db:migrate:vacols + db:migrate:status:vacols + db:migrate:up:vacols + db:migrate:down:vacols + db:migrate:redo:vacols + db:rollback:vacols + db:schema:dump:vacols + db:schema:load:vacols + db:structure:dump:vacols + db:structure:load:vacols +].each do |task_name| + Rake::Task[task_name].clear if Rake::Task.task_defined?(task_name) +end + +#----------------------------------------------------------------------------------------------------------------------- + +# The original definition of the `db:seed` task calls `db:abort_if_pending_migrations`, which would attempt to create a +# `schema_migrations` table on ALL configured databases, including the vacols DB: +# https://github.com/rails/rails/blob/ac87f58207cff18880593263be9d83456aa3a2ef/activerecord/lib/active_record/railties/databases.rake#L389-L393 +# +# This would result in the following error, since the Caseflow app does not have permissions to alter the VACOLS schema: +# +# ActiveRecord::StatementInvalid: OCIError: ORA-01031: insufficient privileges +# +# This re-definiton of `db:seed` will instead call the DB-specific `db:abort_if_pending_migrations` as a workaround. + +db_namespace = namespace :db do + Rake::Task["db:seed"].clear if Rake::Task.task_defined?("db:seed") + desc "Loads the seed data from db/seeds.rb" + task seed: :load_config do + db_namespace["abort_if_pending_migrations:primary"].invoke + + # skip for environments where ETL DB does not exist + db_namespace["abort_if_pending_migrations:etl"].invoke unless %w(demo prodtest).include?(ENV["DEPLOY_ENV"]) + + ActiveRecord::Base.establish_connection(ActiveRecord::Tasks::DatabaseTasks.env.to_sym) + + ActiveRecord::Tasks::DatabaseTasks.load_seed + end +end + +#----------------------------------------------------------------------------------------------------------------------- + +# After transitioning to Rails-native multi-DB support, the behavior of some DB tasks changed such that they will now +# act on ALL databases, including the vacols DB, and not just the primary database. +# +# To avoid accidents, we re-define these tasks here to no-op and output a helpful message to redirect developers toward +# using their new database-specific counterparts instead. + +# rubocop:disable Rails/RakeEnvironment, Layout/HeredocIndentation +namespace :db do + Rake::Task["db:create"].clear if Rake::Task.task_defined?("db:create") + desc "[PROHIBITED] Use the appropriate database-specific tasks instead" + task :create do + puts <<~HEREDOC + + db:create is prohibited. + + Prefer using the appropriate database-specific task below: + + db:create:primary # Create primary database for current environment + db:create:etl # Create etl database for current environment + HEREDOC + end + + Rake::Task["db:drop"].clear if Rake::Task.task_defined?("db:drop") + desc "[PROHIBITED] Use the appropriate database-specific tasks instead" + task :drop do + puts <<~HEREDOC + + db:drop is prohibited. + + Prefer using the appropriate database-specific task below: + + db:drop:primary # Drop primary database for current environment + db:drop:etl # Drop etl database for current environment + HEREDOC + end + + Rake::Task["db:migrate"].clear if Rake::Task.task_defined?("db:migrate") + desc "[PROHIBITED] Use the appropriate database-specific tasks instead" + task :migrate do + puts <<~HEREDOC + + db:migrate is prohibited. + + Prefer using the appropriate database-specific task below: + + db:migrate:primary # Migrate primary database for current environment + db:migrate:etl # Migrate etl database for current environment + HEREDOC + end + + namespace :migrate do + Rake::Task["db:migrate:down"].clear if Rake::Task.task_defined?("db:migrate:down") + desc "[PROHIBITED] Use the appropriate database-specific tasks instead" + task :down do + puts <<~HEREDOC + + db:migrate:down is prohibited. + + Prefer using the appropriate database-specific task below: + + db:migrate:down:primary # Runs the "down" for a given migration VERSION on the primary database + db:migrate:down:etl # Runs the "down" for a given migration VERSION on the etl database + HEREDOC + end + + Rake::Task["db:migrate:redo"].clear if Rake::Task.task_defined?("db:migrate:redo") + desc "[PROHIBITED] Use the appropriate database-specific tasks instead" + task :redo do + puts <<~HEREDOC + + db:migrate:redo is prohibited. + + Prefer using the appropriate database-specific task below: + + db:migrate:redo:primary # Rolls back primary database one migration and re-migrates up (options: STEP=x, VERSION=x) + db:migrate:redo:etl # Rolls back etl database one migration and re-migrates up (options: STEP=x, VERSION=x) + HEREDOC + end + + Rake::Task["db:migrate:status"].clear if Rake::Task.task_defined?("db:migrate:status") + desc "[PROHIBITED] Use the appropriate database-specific tasks instead" + task :status do + puts <<~HEREDOC + + db:migrate:status is prohibited. + + Prefer using the appropriate database-specific task below: + + db:migrate:status:primary # Display status of migrations for primary database + db:migrate:status:etl # Display status of migrations for etl database + HEREDOC + end + + Rake::Task["db:migrate:up"].clear if Rake::Task.task_defined?("db:migrate:up") + desc "[PROHIBITED] Use the appropriate database-specific tasks instead" + task :up do + puts <<~HEREDOC + + db:migrate:up is prohibited. + + Prefer using the appropriate database-specific task below: + + db:migrate:up:primary # Runs the "up" for a given migration VERSION on the primary database + db:migrate:up:etl # Runs the "up" for a given migration VERSION on the etl database + HEREDOC + end + end + + Rake::Task["db:reset"].clear if Rake::Task.task_defined?("db:reset") + desc "[PROHIBITED] Use the appropriate database-specific tasks instead" + task :reset do + puts <<~HEREDOC + + db:reset is prohibited. + + Prefer using the appropriate sequence of database-specific tasks below: + + db:drop:primary db:create:primary db:schema:load:primary db:seed # Reset the primary database + db:drop:etl db:create:etl db:schema:load:etl db:seed # Reset the etl database + HEREDOC + end + + Rake::Task["db:rollback"].clear if Rake::Task.task_defined?("db:rollback") + desc "[PROHIBITED] Use the appropriate database-specific tasks instead" + task :rollback do + puts <<~HEREDOC + + db:rollback is prohibited. + + Prefer using the appropriate database-specific task below: + + db:rollback:primary # Rollback primary database for current environment (specify steps w/ STEP=n) + db:rollback:etl # Rollback etl database for current environment (specify steps w/ STEP=n) + HEREDOC + end + + namespace :schema do + Rake::Task["db:schema:dump"].clear if Rake::Task.task_defined?("db:schema:dump") + desc "[PROHIBITED] Use the appropriate database-specific tasks instead" + task :dump do + puts <<~HEREDOC + + db:schema:dump is prohibited. + + Prefer using the appropriate database-specific task below: + + db:schema:dump:primary # Creates a database schema file (either db/schema.rb or db/structure.sql, depending on `config.active_record.schema_format`) for primary database + db:schema:dump:etl # Creates a database schema file (either db/schema.rb or db/structure.sql, depending on `config.active_record.schema_format`) for etl database + HEREDOC + end + + Rake::Task["db:schema:load"].clear if Rake::Task.task_defined?("db:schema:load") + desc "[PROHIBITED] Use the appropriate database-specific tasks instead" + task :load do + puts <<~HEREDOC + + db:schema:load is prohibited. + + Prefer using the appropriate database-specific task below: + + db:schema:load:primary # Loads a database schema file (either db/schema.rb or db/structure.sql, depending on `config.active_record.schema_format`) into the primary database + db:schema:load:etl # Loads a database schema file (either db/schema.rb or db/structure.sql, depending on `config.active_record.schema_format`) into the etl database + HEREDOC + end + end + + Rake::Task["db:setup"].clear if Rake::Task.task_defined?("db:setup") + desc "[PROHIBITED] Use the appropriate database-specific tasks instead" + task :setup do + puts <<~HEREDOC + + db:setup is prohibited. + + Prefer using the appropriate sequence of database-specific tasks below: + + db:create:primary db:schema:load:primary db:seed # Setup the primary database + db:create:etl db:schema:load:etl db:seed # Setup the etl database + HEREDOC + end + + namespace :test do + Rake::Task["db:test:load"].clear if Rake::Task.task_defined?("db:test:load") + desc "[PROHIBITED] Use the appropriate database-specific tasks instead" + task :load do + puts <<~HEREDOC + + db:test:load is prohibited. + + Prefer using the appropriate database-specific task below: + + db:test:load:primary # Recreate the primary test database from the current schema + db:test:load:etl # Recreate the etl test database from the current schema + HEREDOC + end + + Rake::Task["db:test:load_schema"].clear if Rake::Task.task_defined?("db:test:load_schema") + desc "[PROHIBITED] Use the appropriate database-specific tasks instead" + task :load_schema do + puts <<~HEREDOC + + db:test:load_schema is prohibited. + + Prefer using the appropriate database-specific task below: + + db:test:load_schema:primary # Recreate the primary test database from an existent schema file (schema.rb or structure.sql, depending on `config.active_record.schema_format`) + db:test:load_schema:etl # Recreate the etl test database from an existent schema file (schema.rb or structure.sql, depending on `config.active_record.schema_format`) + HEREDOC + end + + Rake::Task["db:test:prepare"].clear if Rake::Task.task_defined?("db:test:prepare") + desc "[PROHIBITED] Use the appropriate database-specific tasks instead" + task :prepare do + puts <<~HEREDOC + + db:test:prepare is prohibited. + + Prefer using the appropriate database-specific task below: + + db:test:prepare:primary # Load the schema for the primary test database + db:test:prepare:etl # Load the schema for the etl test database + HEREDOC + end + + Rake::Task["db:test:purge"].clear if Rake::Task.task_defined?("db:test:purge") + desc "[PROHIBITED] Use the appropriate database-specific tasks instead" + task :purge do + puts <<~HEREDOC + + db:test:purge is prohibited. + + Prefer using the appropriate database-specific task below: + + db:test:purge:primary # Empty the primary test database + db:test:purge:etl # Empty the etl test database + HEREDOC + end + end +end +# rubocop:enable Rails/RakeEnvironment, Layout/HeredocIndentation diff --git a/lib/tasks/local/build.rake b/lib/tasks/local/build.rake index 15b08e42ada..7f202ba75f7 100644 --- a/lib/tasks/local/build.rake +++ b/lib/tasks/local/build.rake @@ -21,7 +21,7 @@ namespace :local do puts "" puts ">>> 04/08 Creating development and test caseflow databases" - system("RAILS_ENV=development bundle exec rake db:create") || abort + system("RAILS_ENV=development bundle exec rake db:create:primary") || abort puts ">>> 05/08 Seeding FACOLS" system("RAILS_ENV=development bundle exec rake local:vacols:seed") || abort @@ -30,7 +30,7 @@ namespace :local do system("RAILS_ENV=test bundle exec rake spec:setup_vacols") || abort puts ">>> 07/08 Loading schema and seeding local caseflow database" - system("RAILS_ENV=development bundle exec rake db:schema:load db:seed") || abort + system("RAILS_ENV=development bundle exec rake db:schema:load:primary db:seed") || abort puts ">>> 08/08 Enabling feature flags" system("bundle exec rails runner scripts/enable_features_dev.rb") || abort diff --git a/lib/tasks/local/destroy.rake b/lib/tasks/local/destroy.rake index e0873e20ffe..541f489e467 100644 --- a/lib/tasks/local/destroy.rake +++ b/lib/tasks/local/destroy.rake @@ -6,7 +6,7 @@ namespace :local do puts ">>> BEGIN local:destroy" puts ">>> 01/02 Dropping development and test caseflow databases" - system("RAILS_ENV=development bundle exec rake db:drop") || abort + system("RAILS_ENV=development bundle exec rake db:drop:primary") || abort puts ">>> 02/02 Tearing down docker volumes" # Note: In some cases, there may be dangling images or volumes that diff --git a/lib/tasks/spec.rake b/lib/tasks/spec.rake index aca6c9082c3..31dfc56f59a 100644 --- a/lib/tasks/spec.rake +++ b/lib/tasks/spec.rake @@ -25,7 +25,7 @@ namespace :spec do task :spec_name do # rubocop:disable Rails/RakeEnvironment envs = "export TEST_SUBCATEGORY=#{spec_name} && export RAILS_ENV=test &&" - ["rake db:create", "rake db:schema:load"].each do |cmd| + ["rake db:create:primary", "rake db:schema:load:primary"].each do |cmd| ShellCommand.run_and_batch_output("#{envs} #{cmd}") end end diff --git a/scripts/squash.sh b/scripts/squash.sh index bb631e46153..751e8463c34 100755 --- a/scripts/squash.sh +++ b/scripts/squash.sh @@ -5,5 +5,5 @@ git ls-files db/migrate/*.rb | sort | tail -1 | \ %x(git rm -f db/migrate/*.rb; mkdir db/migrate; git mv db/schema.rb #{init_schema}; - bundle exec rake db:migrate; + bundle exec rake db:migrate:primary; git add db/schema.rb; git commit -m 'Squashed migrations')" diff --git a/spec/controllers/api/v1/jobs_controller_spec.rb b/spec/controllers/api/v1/jobs_controller_spec.rb index bd632f60f68..036aeb3bd4b 100644 --- a/spec/controllers/api/v1/jobs_controller_spec.rb +++ b/spec/controllers/api/v1/jobs_controller_spec.rb @@ -30,7 +30,7 @@ # needed to reach 90% test coverage it "should successfully run a job" do - expect(HeartbeatTasksJob.perform_now).to eq true + expect(HeartbeatTasksJob.perform_now).to eq 53 end end diff --git a/spec/controllers/application_controller_spec.rb b/spec/controllers/application_controller_spec.rb index 44d392ed3cf..6feb3e44749 100644 --- a/spec/controllers/application_controller_spec.rb +++ b/spec/controllers/application_controller_spec.rb @@ -56,7 +56,7 @@ def index it "sets Cache-Control etc" do get :index - expect(response.headers["Cache-Control"]).to eq "no-cache, no-store" + expect(response.headers["Cache-Control"]).to eq "no-store" end end end diff --git a/spec/controllers/decision_reviews_controller_spec.rb b/spec/controllers/decision_reviews_controller_spec.rb index e16465f0362..4280faa4987 100644 --- a/spec/controllers/decision_reviews_controller_spec.rb +++ b/spec/controllers/decision_reviews_controller_spec.rb @@ -942,7 +942,7 @@ params = { business_line_slug: non_comp_org.url }.merge(generate_report_filters.except("report_type")) get :generate_report, format: :csv, params: params expect(response).to have_http_status(:bad_request) - expect(response.content_type).to eq("application/json") + expect(response.content_type).to eq("application/json; charset=utf-8") json_response = JSON.parse(response.body) expect(json_response["error"]).to eq("param is missing or the value is empty: reportType") end diff --git a/spec/factories/ro_schedule_period.rb b/spec/factories/ro_schedule_period.rb index b3ef91ac530..8746f031fa2 100644 --- a/spec/factories/ro_schedule_period.rb +++ b/spec/factories/ro_schedule_period.rb @@ -1,7 +1,7 @@ # frozen_string_literal: true FactoryBot.define do - before(:create) do + after(:build) do S3Service.store_file(SchedulePeriod::S3_SUB_BUCKET + "/" + "validRoSpreadsheet.xlsx", "spec/support/validRoSpreadsheet.xlsx", :filepath) S3Service.store_file(SchedulePeriod::S3_SUB_BUCKET + "/" + "blankRoSpreadsheet.xlsx", diff --git a/spec/feature/non_comp/report_page_spec.rb b/spec/feature/non_comp/report_page_spec.rb index 6cf2c458097..b714a8d328e 100644 --- a/spec/feature/non_comp/report_page_spec.rb +++ b/spec/feature/non_comp/report_page_spec.rb @@ -331,7 +331,7 @@ def clear_filters end def change_history_csv_file - wait_for(5) + sleep 5 # Copied from Capybara setup download_directory = Rails.root.join("tmp/downloads_#{ENV['TEST_SUBCATEGORY'] || 'all'}") list_of_files = Dir.glob(File.join(download_directory, "*")).select { |f| File.file?(f) } @@ -340,11 +340,4 @@ def change_history_csv_file expect(latest_file).to_not eq(nil) latest_file end - - def wait_for(seconds) - start_time = Time.zone.now - while Time.zone.now - start_time < seconds - # Do nothing, just wait - end - end end diff --git a/spec/lib/helpers/association_wrapper_spec.rb b/spec/lib/helpers/association_wrapper_spec.rb index 5acf5ae0cb1..86e157dc91b 100644 --- a/spec/lib/helpers/association_wrapper_spec.rb +++ b/spec/lib/helpers/association_wrapper_spec.rb @@ -91,7 +91,7 @@ [:attorney_case_reviews, false, false, "task_id", "attorney_case_review_id", nil], [:task_timers, false, false, "task_id", "task_timer_id", nil], # has_one declared in Task - [:cached_appeal, false, true, :appeal_id, "cached_appeal_id", nil] + [:cached_appeal, false, true, "appeal_id", "cached_appeal_id", nil] ] end end diff --git a/spec/lib/tasks/db_spec.rb b/spec/lib/tasks/db_spec.rb new file mode 100644 index 00000000000..fab7ca72e54 --- /dev/null +++ b/spec/lib/tasks/db_spec.rb @@ -0,0 +1,295 @@ +# frozen_string_literal: true + +# rubocop:disable Layout/HeredocIndentation +describe "db" do + include_context "rake" + + describe "create" do + subject { Rake::Task["db:create"].invoke } + + it "no-ops and outputs a helpful message" do + expected_output = <<~HEREDOC + + db:create is prohibited. + + Prefer using the appropriate database-specific task below: + + db:create:primary # Create primary database for current environment + db:create:etl # Create etl database for current environment + HEREDOC + + expect { subject }.to output(expected_output).to_stdout + end + end + + describe "drop" do + subject { Rake::Task["db:drop"].invoke } + + it "no-ops and outputs a helpful message" do + expected_output = <<~HEREDOC + + db:drop is prohibited. + + Prefer using the appropriate database-specific task below: + + db:drop:primary # Drop primary database for current environment + db:drop:etl # Drop etl database for current environment + HEREDOC + + expect { subject }.to output(expected_output).to_stdout + end + end + + describe "migrate" do + subject { Rake::Task["db:migrate"].invoke } + + it "no-ops and outputs a helpful message" do + expected_output = <<~HEREDOC + + db:migrate is prohibited. + + Prefer using the appropriate database-specific task below: + + db:migrate:primary # Migrate primary database for current environment + db:migrate:etl # Migrate etl database for current environment + HEREDOC + + expect { subject }.to output(expected_output).to_stdout + end + end + + describe "migrate:down" do + subject { Rake::Task["db:migrate:down"].invoke } + + it "no-ops and outputs a helpful message" do + expected_output = <<~HEREDOC + + db:migrate:down is prohibited. + + Prefer using the appropriate database-specific task below: + + db:migrate:down:primary # Runs the "down" for a given migration VERSION on the primary database + db:migrate:down:etl # Runs the "down" for a given migration VERSION on the etl database + HEREDOC + + expect { subject }.to output(expected_output).to_stdout + end + end + + describe "migrate:redo" do + subject { Rake::Task["db:migrate:redo"].invoke } + + it "no-ops and outputs a helpful message" do + expected_output = <<~HEREDOC + + db:migrate:redo is prohibited. + + Prefer using the appropriate database-specific task below: + + db:migrate:redo:primary # Rolls back primary database one migration and re-migrates up (options: STEP=x, VERSION=x) + db:migrate:redo:etl # Rolls back etl database one migration and re-migrates up (options: STEP=x, VERSION=x) + HEREDOC + + expect { subject }.to output(expected_output).to_stdout + end + end + + describe "migrate:status" do + subject { Rake::Task["db:migrate:status"].invoke } + + it "no-ops and outputs a helpful message" do + expected_output = <<~HEREDOC + + db:migrate:status is prohibited. + + Prefer using the appropriate database-specific task below: + + db:migrate:status:primary # Display status of migrations for primary database + db:migrate:status:etl # Display status of migrations for etl database + HEREDOC + + expect { subject }.to output(expected_output).to_stdout + end + end + + describe "migrate:up" do + subject { Rake::Task["db:migrate:up"].invoke } + + it "no-ops and outputs a helpful message" do + expected_output = <<~HEREDOC + + db:migrate:up is prohibited. + + Prefer using the appropriate database-specific task below: + + db:migrate:up:primary # Runs the "up" for a given migration VERSION on the primary database + db:migrate:up:etl # Runs the "up" for a given migration VERSION on the etl database + HEREDOC + + expect { subject }.to output(expected_output).to_stdout + end + end + + describe "reset" do + subject { Rake::Task["db:reset"].invoke } + + it "no-ops and outputs a helpful message" do + expected_output = <<~HEREDOC + + db:reset is prohibited. + + Prefer using the appropriate sequence of database-specific tasks below: + + db:drop:primary db:create:primary db:schema:load:primary db:seed # Reset the primary database + db:drop:etl db:create:etl db:schema:load:etl db:seed # Reset the etl database + HEREDOC + + expect { subject }.to output(expected_output).to_stdout + end + end + + describe "rollback" do + subject { Rake::Task["db:rollback"].invoke } + + it "no-ops and outputs a helpful message" do + expected_output = <<~HEREDOC + + db:rollback is prohibited. + + Prefer using the appropriate database-specific task below: + + db:rollback:primary # Rollback primary database for current environment (specify steps w/ STEP=n) + db:rollback:etl # Rollback etl database for current environment (specify steps w/ STEP=n) + HEREDOC + + expect { subject }.to output(expected_output).to_stdout + end + end + + describe "schema:dump" do + subject { Rake::Task["db:schema:dump"].invoke } + + it "no-ops and outputs a helpful message" do + expected_output = <<~HEREDOC + + db:schema:dump is prohibited. + + Prefer using the appropriate database-specific task below: + + db:schema:dump:primary # Creates a database schema file (either db/schema.rb or db/structure.sql, depending on `config.active_record.schema_format`) for primary database + db:schema:dump:etl # Creates a database schema file (either db/schema.rb or db/structure.sql, depending on `config.active_record.schema_format`) for etl database + HEREDOC + + expect { subject }.to output(expected_output).to_stdout + end + end + + describe "schema:load" do + subject { Rake::Task["db:schema:load"].invoke } + + it "no-ops and outputs a helpful message" do + expected_output = <<~HEREDOC + + db:schema:load is prohibited. + + Prefer using the appropriate database-specific task below: + + db:schema:load:primary # Loads a database schema file (either db/schema.rb or db/structure.sql, depending on `config.active_record.schema_format`) into the primary database + db:schema:load:etl # Loads a database schema file (either db/schema.rb or db/structure.sql, depending on `config.active_record.schema_format`) into the etl database + HEREDOC + + expect { subject }.to output(expected_output).to_stdout + end + end + + describe "setup" do + subject { Rake::Task["db:setup"].invoke } + + it "no-ops and outputs a helpful message" do + expected_output = <<~HEREDOC + + db:setup is prohibited. + + Prefer using the appropriate sequence of database-specific tasks below: + + db:create:primary db:schema:load:primary db:seed # Setup the primary database + db:create:etl db:schema:load:etl db:seed # Setup the etl database + HEREDOC + + expect { subject }.to output(expected_output).to_stdout + end + end + + describe "test:load" do + subject { Rake::Task["db:test:load"].invoke } + + it "no-ops and outputs a helpful message" do + expected_output = <<~HEREDOC + + db:test:load is prohibited. + + Prefer using the appropriate database-specific task below: + + db:test:load:primary # Recreate the primary test database from the current schema + db:test:load:etl # Recreate the etl test database from the current schema + HEREDOC + + expect { subject }.to output(expected_output).to_stdout + end + end + + describe "test:load_schema" do + subject { Rake::Task["db:test:load_schema"].invoke } + + it "no-ops and outputs a helpful message" do + expected_output = <<~HEREDOC + + db:test:load_schema is prohibited. + + Prefer using the appropriate database-specific task below: + + db:test:load_schema:primary # Recreate the primary test database from an existent schema file (schema.rb or structure.sql, depending on `config.active_record.schema_format`) + db:test:load_schema:etl # Recreate the etl test database from an existent schema file (schema.rb or structure.sql, depending on `config.active_record.schema_format`) + HEREDOC + + expect { subject }.to output(expected_output).to_stdout + end + end + + describe "test:prepare" do + subject { Rake::Task["db:test:prepare"].invoke } + + it "no-ops and outputs a helpful message" do + expected_output = <<~HEREDOC + + db:test:prepare is prohibited. + + Prefer using the appropriate database-specific task below: + + db:test:prepare:primary # Load the schema for the primary test database + db:test:prepare:etl # Load the schema for the etl test database + HEREDOC + + expect { subject }.to output(expected_output).to_stdout + end + end + + describe "test:purge" do + subject { Rake::Task["db:test:purge"].invoke } + + it "no-ops and outputs a helpful message" do + expected_output = <<~HEREDOC + + db:test:purge is prohibited. + + Prefer using the appropriate database-specific task below: + + db:test:purge:primary # Empty the primary test database + db:test:purge:etl # Empty the etl test database + HEREDOC + + expect { subject }.to output(expected_output).to_stdout + end + end +end +# rubocop:enable Layout/HeredocIndentation diff --git a/spec/models/ro_schedule_period_spec.rb b/spec/models/ro_schedule_period_spec.rb index 774902d3423..78af18d1563 100644 --- a/spec/models/ro_schedule_period_spec.rb +++ b/spec/models/ro_schedule_period_spec.rb @@ -7,6 +7,28 @@ subject { ro_schedule_period.validate_spreadsheet } it { is_expected.to be_truthy } + + context "when spreadsheet is invalid" do + let(:spreadsheet_errors) do + [ + HearingSchedule::ValidateRoSpreadsheet::RoDatesNotUnique.new("RoDatesNotUnique message"), + HearingSchedule::ValidateRoSpreadsheet::RoDatesNotInRange.new("RoDatesNotInRange message"), + HearingSchedule::ValidateRoSpreadsheet::RoDatesNotCorrectFormat.new("RoDatesNotCorrectFormat message"), + HearingSchedule::ValidateRoSpreadsheet::RoTemplateNotFollowed.new("RoTemplateNotFollowed message") + ] + end + + before do + allow_any_instance_of(HearingSchedule::ValidateRoSpreadsheet).to receive(:validate) { spreadsheet_errors } + end + + it "adds errors to model" do + model = build(:ro_schedule_period) + model.valid? + expect(model).to_not be_valid + expect(model.errors[:base]).to match_array([spreadsheet_errors]) + end + end end context "Allocate RO Days Per Given Schedule" do diff --git a/spec/models/veteran_spec.rb b/spec/models/veteran_spec.rb index 7b3a4b5c875..c139c615f7e 100644 --- a/spec/models/veteran_spec.rb +++ b/spec/models/veteran_spec.rb @@ -316,18 +316,24 @@ end end - context "when veteran pay grade is invalid" do - subject { veteran.validate_veteran_pay_grade } - let(:service) do - [{ branch_of_service: "Army", - entered_on_duty_date: "06282002", - released_active_duty_date: "06282003", - pay_grade: "not valid", - char_of_svc_code: "TBD" }] - end + context "pay grade" do + context "when not in BGS pay grade list" do + let(:service) do + [{ branch_of_service: "Army", + entered_on_duty_date: "06282002", + released_active_duty_date: "06282003", + pay_grade: "not valid", + char_of_svc_code: "TBD" }] + end + + it "is invalid within 'bgs' context" do + expect(veteran.valid?(:bgs)).to eq(false) + expect(veteran.errors.messages_for(:pay_grades)).to eq ["invalid_pay_grade"] + end - it "pay grade invalid" do - expect(subject).to eq ["invalid_pay_grade"] + it "is valid outside of 'bgs' context" do + expect(veteran.valid?).to eq(true) + end end end end @@ -554,11 +560,18 @@ end end - context "when a zip code is invalid" do - let(:zip_code) { "1234" } + context "zip code" do + context "when not a valid zip code" do + let(:zip_code) { "1234" } + + it "is invalid within 'bgs' context" do + expect(veteran.valid?(:bgs)).to eq(false) + expect(veteran.errors.messages_for(:zip_code)).to eq ["invalid_zip_code"] + end - it "zip code has invalid characters" do - expect(veteran.validate_zip_code).to eq ["invalid_zip_code"] + it "is valid outside of 'bgs' context" do + expect(veteran.valid?).to eq(true) + end end end @@ -573,21 +586,35 @@ end end - context "given a military address with invalid city characters" do - let(:military_postal_type_code) { "AA" } - let(:city) { "ÐÐÐÐÐ" } - let(:state) { nil } + context "city" do + context "when a military address with invalid city characters" do + let(:military_postal_type_code) { "AA" } + let(:city) { "ÐÐÐÐÐ" } + let(:state) { nil } + + it "is invalid within 'bgs' context" do + expect(veteran.valid?(:bgs)).to eq(false) + expect(veteran.errors.messages_for(:city)).to eq ["invalid_characters"] + end - it "city is considered invalid" do - expect(veteran.validate_city).to eq ["invalid_characters"] + it "is valid outside of 'bgs' context" do + expect(veteran.valid?).to eq(true) + end end end - context "given date of birth is missing leading zeros" do - let(:date_of_birth) { "2/2/1956" } + context "date of birth" do + context "when missing leading zeros" do + let(:date_of_birth) { "2/2/1956" } + + it "is invalid within 'bgs' context" do + expect(veteran.valid?(:bgs)).to eq(false) + expect(veteran.errors.messages_for(:date_of_birth)).to eq ["invalid_date_of_birth"] + end - it "date_of_birth is considered invalid" do - expect(veteran.validate_date_of_birth).to eq ["invalid_date_of_birth"] + it "is valid outside of 'bgs' context" do + expect(veteran.valid?).to eq(true) + end end end @@ -599,22 +626,30 @@ end end - context "#validate_name_suffix" do - subject { veteran.validate_name_suffix } - let(:name_suffix) { "JR." } + context "name suffix" do + context "when containing a special character" do + let(:name_suffix) { "JR." } + + it "is invalid within 'bgs' context" do + expect(veteran.valid?(:bgs)).to eq false + expect(veteran.errors.messages_for(:name_suffix)).to eq(["invalid_character"]) + end - it "name_suffix is considered invalid" do - expect(subject).to eq ["invalid_character"] - expect(veteran.valid?(:bgs)).to eq false + it "is valid outside of 'bgs' context" do + expect(veteran.valid?).to eq(true) + end end - context "name_suffix nil" do + context "when nil" do let(:name_suffix) { nil } - it "name_suffix is considered valid" do - subject + it "is valid within 'bgs' context" do expect(veteran.valid?(:bgs)).to eq true end + + it "is valid outside of 'bgs' context" do + expect(veteran.valid?).to eq(true) + end end end diff --git a/spec/requests/ssl_redirects_spec.rb b/spec/requests/ssl_redirects_spec.rb new file mode 100644 index 00000000000..33c1b3d8412 --- /dev/null +++ b/spec/requests/ssl_redirects_spec.rb @@ -0,0 +1,75 @@ +# frozen_string_literal: true + +describe "SSL Redirects" do + # `app` is what RSpec tests against in request specs, similar to `controller` in controller specs. + # Here, we override it with our modified app. + def app + # Since `CaseflowCertification::Application` is already loaded at this stage, we can't modify the middleware stack, + # so we subclass the application and adjust the relevant SSL config settings. + @app ||= Class.new(Rails.application.class) do + config.force_ssl = true + config.ssl_options = { redirect: { exclude: SslRedirectExclusionPolicy } } + end + end + + before { allow(SslRedirectExclusionPolicy).to receive(:call).and_call_original } + + context "when request is not SSL" do + context "when path matches '/api/docs/v3/'" do + it "is exempt from SSL redirect" do + get "/api/docs/v3/decision_reviews" + expect(SslRedirectExclusionPolicy).to have_received(:call) + expect(response).not_to have_http_status(:redirect) + end + end + + context "when path is '/api/metadata'" do + it "is exempt from SSL redirect" do + get "/api/metadata" + expect(SslRedirectExclusionPolicy).to have_received(:call) + expect(response).not_to have_http_status(:redirect) + end + end + + context "when path is '/health-check'" do + it "is exempt from SSL redirect" do + get "/health-check" + expect(SslRedirectExclusionPolicy).to have_received(:call) + expect(response).not_to have_http_status(:redirect) + end + end + + context "when path matches '/idt/api/v1/'" do + it "is exempt from SSL redirect" do + get "/idt/api/v1/appeals" + expect(SslRedirectExclusionPolicy).to have_received(:call) + expect(response).not_to have_http_status(:redirect) + end + end + + context "when path matches '/idt/api/v2/'" do + it "is exempt from SSL redirect" do + get "/idt/api/v2/appeals" + expect(SslRedirectExclusionPolicy).to have_received(:call) + expect(response).not_to have_http_status(:redirect) + end + end + + context "when path matches '/pdfjs/'" do + it "is exempt from SSL redirect" do + get "/pdfjs/full?file=%2Fcertifications%2F2774535%2Fform9_pdf" + expect(SslRedirectExclusionPolicy).to have_received(:call) + expect(response).not_to have_http_status(:redirect) + end + end + + context "when path is not exempt from SSL redirects" do + it "is redirected with SSL" do + get "/users" + expect(SslRedirectExclusionPolicy).to have_received(:call) + + expect(response).to redirect_to("https://#{request.host}/users") + end + end + end +end diff --git a/spec/services/deprecation_warnings/development_handler_spec.rb b/spec/services/deprecation_warnings/development_handler_spec.rb deleted file mode 100644 index 0848fd5b3ae..00000000000 --- a/spec/services/deprecation_warnings/development_handler_spec.rb +++ /dev/null @@ -1,46 +0,0 @@ -# frozen_string_literal: true - -module DeprecationWarnings - describe DevelopmentHandler do - context ".call" do - subject(:call) do - described_class.call(message, _callstack = [], _deprecation_horizon = "6.0", _gem_name = "Rails") - end - - let(:message) { "dummy deprecation message" } - let(:rails_logger) { Rails.logger } - - before do - allow(Rails).to receive(:logger).and_return(rails_logger) - allow(rails_logger).to receive(:warn) - end - - it "emits a warning to the application logs" do - call - - expect(rails_logger).to have_received(:warn).with(message) - end - - context "when deprecation is allowed" do - let(:message) { "allowed deprecation message" } - - it "does not raise error" do - expect { call }.not_to raise_error - end - end - - context "when deprecation is disallowed" do - let(:message) { "disallowed deprecation message" } - - before do - stub_const("DisallowedDeprecations::DISALLOWED_DEPRECATION_WARNING_REGEXES", - [Regexp.new(Regexp.escape(message))]) - end - - it "raises DisallowedDeprecationError" do - expect { call }.to raise_error(::DisallowedDeprecationError) - end - end - end - end -end diff --git a/spec/services/deprecation_warnings/production_handler_spec.rb b/spec/services/deprecation_warnings/production_handler_spec.rb index e28cf27f0e1..a34c219d584 100644 --- a/spec/services/deprecation_warnings/production_handler_spec.rb +++ b/spec/services/deprecation_warnings/production_handler_spec.rb @@ -25,12 +25,6 @@ module DeprecationWarnings allow(slack_service).to receive(:send_notification) end - it "emits a warning to the application logs" do - call - - expect(rails_logger).to have_received(:warn).with(message) - end - it "emits a warning to Sentry" do call diff --git a/spec/services/deprecation_warnings/test_handler_spec.rb b/spec/services/deprecation_warnings/test_handler_spec.rb deleted file mode 100644 index 0ddf9bec0ad..00000000000 --- a/spec/services/deprecation_warnings/test_handler_spec.rb +++ /dev/null @@ -1,38 +0,0 @@ -# frozen_string_literal: true - -module DeprecationWarnings - describe TestHandler do - context ".call" do - subject(:call) do - described_class.call(message, _callstack = [], _deprecation_horizon = "6.0", _gem_name = "Rails") - end - - let(:message) { "dummy deprecation message" } - - it "logs message to stderr" do - expect { call }.to output("#{message}\n").to_stderr - end - - context "when deprecation is allowed" do - let(:message) { "allowed deprecation message" } - - it "does not raise error" do - expect { call }.not_to raise_error - end - end - - context "when deprecation is disallowed" do - let(:message) { "disallowed deprecation message" } - - before do - stub_const("DisallowedDeprecations::DISALLOWED_DEPRECATION_WARNING_REGEXES", - [Regexp.new(Regexp.escape(message))]) - end - - it "raises DisallowedDeprecationError" do - expect { call }.to raise_error(::DisallowedDeprecationError) - end - end - end - end -end diff --git a/spec/sql/ama_cases_sql_spec.rb b/spec/sql/ama_cases_sql_spec.rb index 6dbafeb8e1d..def32079c57 100644 --- a/spec/sql/ama_cases_sql_spec.rb +++ b/spec/sql/ama_cases_sql_spec.rb @@ -22,10 +22,10 @@ non_aod_case = result.find { |r| r["id"] == not_distributed.id } expect(aod_case["aod_is_advanced_on_docket"]).to eq(true) - expect(aod_case["aod_veteran.age"]).to eq("76") + expect(aod_case["aod_veteran.age"]).to eq(76) expect(non_aod_case["aod_is_advanced_on_docket"]).to eq(false) - expect(non_aod_case["aod_veteran.age"]).to eq("65") + expect(non_aod_case["aod_veteran.age"]).to eq(65) end end end diff --git a/spec/support/database_cleaner.rb b/spec/support/database_cleaner.rb index bc90949aa75..9905b0e7f78 100644 --- a/spec/support/database_cleaner.rb +++ b/spec/support/database_cleaner.rb @@ -5,35 +5,31 @@ RSpec.configure do |config| config.use_transactional_fixtures = false - etl = "etl_#{Rails.env}".to_sym - vacols = "#{Rails.env}_vacols".to_sym - caseflow = Rails.env.to_s.to_sym - vacols_tables_to_preserve = %w[vftypes issref] - # IMPORTANT that in all these hook defs, the "caseflow" connection comes last. + # IMPORTANT that in all these hook defs, the "primary" connection comes last. config.before(:suite) do - DatabaseCleaner[:active_record, { db: etl }].clean_with(:truncation) - DatabaseCleaner[:active_record, { db: vacols }] + DatabaseCleaner[:active_record, { db: :etl }].clean_with(:truncation) + DatabaseCleaner[:active_record, { db: :vacols }] .clean_with(:deletion, except: vacols_tables_to_preserve) - DatabaseCleaner[:active_record, { db: caseflow }].clean_with(:truncation) + DatabaseCleaner[:active_record, { db: :primary }].clean_with(:truncation) end config.before(:each) do |example| # Allows seeded data to persist for use across threads # You will need to manually clean the data once the threads under test close. unless example.metadata[:bypass_cleaner] - DatabaseCleaner[:active_record, { db: vacols }].strategy = :transaction - DatabaseCleaner[:active_record, { db: caseflow }].strategy = :transaction + DatabaseCleaner[:active_record, { db: :vacols }].strategy = :transaction + DatabaseCleaner[:active_record, { db: :primary }].strategy = :transaction end end config.before(:each, db_clean: :truncation) do |example| unless example.metadata[:bypass_cleaner] - DatabaseCleaner[:active_record, { db: vacols }].strategy = + DatabaseCleaner[:active_record, { db: :vacols }].strategy = :deletion, { except: vacols_tables_to_preserve } - DatabaseCleaner[:active_record, { db: caseflow }].strategy = :truncation + DatabaseCleaner[:active_record, { db: :primary }].strategy = :truncation end end @@ -46,42 +42,42 @@ # Driver is probably for an external browser with an app # under test that does *not* share a database connection with the # specs, so use truncation strategy. - DatabaseCleaner[:active_record, { db: vacols }].strategy = + DatabaseCleaner[:active_record, { db: :vacols }].strategy = :deletion, { except: vacols_tables_to_preserve } - DatabaseCleaner[:active_record, { db: caseflow }].strategy = :truncation + DatabaseCleaner[:active_record, { db: :primary }].strategy = :truncation end end config.before(:each) do |example| unless example.metadata[:bypass_cleaner] - DatabaseCleaner[:active_record, { db: vacols }].start - DatabaseCleaner[:active_record, { db: caseflow }].start + DatabaseCleaner[:active_record, { db: :vacols }].start + DatabaseCleaner[:active_record, { db: :primary }].start end end config.append_after(:each) do - DatabaseCleaner[:active_record, { db: vacols }].clean - DatabaseCleaner[:active_record, { db: caseflow }].clean + DatabaseCleaner[:active_record, { db: :vacols }].clean + DatabaseCleaner[:active_record, { db: :primary }].clean clean_application! end # ETL is never used in feature tests and there are only a few, so we tag those with :etl # ETL db uses deletion strategy everywhere because syncing runs in a transaction. config.before(:each, :etl) do - DatabaseCleaner[:active_record, { db: etl }].strategy = :deletion + DatabaseCleaner[:active_record, { db: :etl }].strategy = :deletion end config.before(:each, :etl, db_clean: :truncation) do - DatabaseCleaner[:active_record, { db: etl }].strategy = :truncation + DatabaseCleaner[:active_record, { db: :etl }].strategy = :truncation end config.before(:each, :etl) do Rails.logger.info("DatabaseCleaner.start ETL") - DatabaseCleaner[:active_record, { db: etl }].start + DatabaseCleaner[:active_record, { db: :etl }].start end config.append_after(:each, :etl) do - DatabaseCleaner[:active_record, { db: etl }].clean + DatabaseCleaner[:active_record, { db: :etl }].clean Rails.logger.info("DatabaseCleaner.clean ETL") end end diff --git a/spec/workflows/ama_appeal_dispatch_spec.rb b/spec/workflows/ama_appeal_dispatch_spec.rb index 62a1e336e61..425ab981695 100644 --- a/spec/workflows/ama_appeal_dispatch_spec.rb +++ b/spec/workflows/ama_appeal_dispatch_spec.rb @@ -70,7 +70,10 @@ it "does not call #perform_later on MailRequestJob" do allow(ProcessDecisionDocumentJob).to receive(:perform_later).and_raise(StandardError) expect(MailRequestJob).to_not receive(:perform_later) - expect { subject }.to raise_error(StandardError) + subject + rescue Minitest::UnexpectedError => error # rubocop:disable Lint/SuppressedException + ensure + expect(error.message).to match("StandardError") end end end diff --git a/spec/workflows/legacy_appeal_dispatch_spec.rb b/spec/workflows/legacy_appeal_dispatch_spec.rb index 81c40e35793..a1831d29f04 100644 --- a/spec/workflows/legacy_appeal_dispatch_spec.rb +++ b/spec/workflows/legacy_appeal_dispatch_spec.rb @@ -102,7 +102,10 @@ it "does not call #perform_later on MailRequestJob" do allow(ProcessDecisionDocumentJob).to receive(:perform_later).and_raise(StandardError) expect(MailRequestJob).to_not receive(:perform_later) - expect { subject }.to raise_error(StandardError) + subject + rescue Minitest::UnexpectedError => error # rubocop:disable Lint/SuppressedException + ensure + expect(error.message).to match("StandardError") end end end