Initial 526 form 4142 benefits intake API work

[kylesoskin]

Summary

• *This work is behind a feature toggle (flipper): YES

form526_send_4142_failure_notification - existing flipper, global send emails at all killswitch

disability_526_form4142_polling_records (CentralMail::SubmitForm4142Job::POLLING_FLIPPER_KEY) - new flipper, when enabled, create a polling record with every email job queued

disability_526_form4142_polling_record_failure_email (CentralMail::SubmitForm4142Job::POLLED_FAILURE_EMAIL) - new flipper, when enabled, queue a ZSF failure notification email in the case that a polled result indicates downstream failure to deliver document

• (Summarize the changes that have been made to the platform)

In this PR, a new feature is introduced to utilize the existing FormSubmission and FormSubmissionAttempt models, which already poll benefits intake api, to track the downstream status of form 526 generated form 4142 forms. The feature also includes sending an email, when enabled, to notify the user when a polled failure occurs.

• (If bug, how to reproduce) N/A
• (What is the solution, why is this the solution?)

This is the solution proposed by the VA stakeholders to deal with remaining silent failure situations.

• (Which team do you work for, does your team own the maintenance of this component?)

Benefits and Claims team (DBEX/Carbs team), we own the maintenance of all files edited in this PR, with the exception of app/models/form_submission_attempt.rb and spec/models/form_submission_attempt_spec.rb which are owned by the @department-of-veterans-affairs/platform-va-product-forms team

• (If introducing a flipper, what is the success criteria being targeted?)

100% success. We will turn the flipper on to just create records at first, and then, after verification of the records and polling working, we will then enable the emailing.

Related issue(s)

department-of-veterans-affairs/va.gov-team#94144

Testing done

• New code is covered by unit tests
• Describe what the old behavior was prior to the change

Document was sent to LH and it was not tracked further. Email failure was only sent if we could not get the document to Lighthouse Benefits Intake API. Now, in addition to emailing at that failure point, we also will email the same failure email in the case of Lighthouse receiving the document, and later erroring to get it further downstream.

• Describe the steps required to verify your changes are working as expected. Exclusively stating 'Specs run' is NOT acceptable as appropriate testing

turning on in staging
checking the database records via the rails console
confirming log messages in datadog

• If this work is behind a flipper:
  • Tests need to be written for both the flipper on and flipper off scenarios. Docs.

Done

• What is the testing plan for rolling out the feature?

turning on in staging
testing in staging with flippers on and off
turning on record creation in prod
checking that records and polling are created and operating as expected
checking that datadog logging and metrics are incrementing as expected
only once that is confirmed, then enable the email, via that flipper

What areas of the site does it impact?

(Describe what parts of the site are impacted andifcode touched other areas)

Should only impact backend code touched here.

Acceptance criteria

• I fixed|updated|added unit tests and integration tests for each feature (if applicable).
• No error nor warning in the console.
• Events are being sent to the appropriate logging solution
• Documentation has been updated (link to documentation)
• No sensitive information (i.e. PII/credentials/internal URLs/etc.) is captured in logging, hardcoded, or specs
• Feature/bug has a monitor built into Datadog (if applicable)
• If app impacted requires authentication, did you login to a local build and verify all authenticated routes work as expected
• I added a screenshot of the developed feature

Requested Feedback

This feature makes use of 2 other teams stuff. The simple form teams FormSubmission model(s), and the Burials teams ZSF logging/statsD code. Any feedback from those teams on this use of their stuff is appreciated.

[va-vsp-bot]

Error: A file (or its parent directories) does not have a CODEOWNERS entry. Please update the .github/CODEOWNERS file and add the entry for the Offending file: app/models/form526_ancillary_form4142_status_polling_record.rb

[va-vsp-bot]

Error: A file (or its parent directories) does not have a CODEOWNERS entry. Please update the .github/CODEOWNERS file and add the entry for the Offending file: app/models/form526_ancillary_form4142_status_polling_record.rb

[va-vsp-bot]

Error: A file (or its parent directories) does not have a CODEOWNERS entry. Please update the .github/CODEOWNERS file and add the entry for the Offending file: app/models/form526_ancillary_form4142_status_polling_record.rb

[va-vsp-bot]

Error: A file (or its parent directories) does not have a CODEOWNERS entry. Please update the .github/CODEOWNERS file and add the entry for the Offending file: app/models/form526_ancillary_form4142_status_polling_record.rb

[va-vsp-bot]

Error: A file (or its parent directories) does not have a CODEOWNERS entry. Please update the .github/CODEOWNERS file and add the entry for the Offending file: app/models/form526_ancillary_form4142_status_polling_record.rb

[va-vsp-bot]

Error: A file (or its parent directories) does not have a CODEOWNERS entry. Please update the .github/CODEOWNERS file and add the entry for the Offending file: app/models/form4142_status_polling_record.rb

[va-vsp-bot]

Error: A file (or its parent directories) does not have a CODEOWNERS entry. Please update the .github/CODEOWNERS file and add the entry for the Offending file: app/models/form4142_status_polling_record.rb

[va-vsp-bot]

Error: A file (or its parent directories) does not have a CODEOWNERS entry. Please update the .github/CODEOWNERS file and add the entry for the Offending file: app/models/form4142_status_polling_record.rb

[freeheeling]

I always thought it was presumptive to have this log statement positioned before the upload_doc call.