Skip to content

test(ci): exercise Helm PostgreSQL 15-16-18 upgrade path - #5139

Draft
zdrapela wants to merge 10 commits into
redhat-developer:mainfrom
zdrapela:test/pg15-to-pg18-helm-upgrade
Draft

test(ci): exercise Helm PostgreSQL 15-16-18 upgrade path#5139
zdrapela wants to merge 10 commits into
redhat-developer:mainfrom
zdrapela:test/pg15-to-pg18-helm-upgrade

Conversation

@zdrapela

@zdrapela zdrapela commented Jul 22, 2026 •
edited
Loading

Copy link
Copy Markdown
Member

Summary

  • CI evidence for RHIDP-14594: chart-managed PostgreSQL upgrade using product rhel9/postgresql-* images
  • Flow: PG15 → Playwright → PG16 → Playwright → PG18 → Playwright
  • Unique ARTIFACT_DIR subdirs: showcase-pg15, showcase-pg16, showcase-pg18 (Playwright + pod logs + postgres diagnostics; no overwrite)
  • Two-hop POSTGRESQL_UPGRADE=copy: 15→16→18

Companion Fedora PR: #5141

Verified (prior run, single Playwright at end)

Test plan

  • /test e2e-ocp-helm with Playwright after each major
  • Artifacts under showcase-pg15|pg16|pg18

@zdrapela
test(ci): exercise Helm internal PostgreSQL 15→16→18 upgrade
5af50f2 
Validate the sclorg-supported two-hop path (POSTGRESQL_UPGRADE=copy)
for RHIDP-14594 evidence on e2e-ocp-helm.
@openshift-ci

openshift-ci Bot commented Jul 22, 2026

Copy link
Copy Markdown

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

@zdrapela

zdrapela commented Jul 22, 2026

Copy link
Copy Markdown
Member Author

/test e2e-ocp-helm

@zdrapela zdrapela changed the title test(ci): Helm internal PostgreSQL 15→16→18 upgrade (RHIDP-14594) test(ci): Helm internal PostgreSQL 15 to 18 upgrade (RHIDP-14594) Jul 22, 2026
@zdrapela
fix(ci): wait for Postgres Ready after each upgrade hop
739ee84 
Avoid racing Playwright against a recreating postgresql pod after
removing POSTGRESQL_UPGRADE. Log server_version for upgrade evidence.
@zdrapela zdrapela changed the title test(ci): Helm internal PostgreSQL 15 to 18 upgrade (RHIDP-14594) test(ci): exercise Helm PostgreSQL 15-16-18 upgrade path Jul 22, 2026
@zdrapela

zdrapela commented Jul 22, 2026

Copy link
Copy Markdown
Member Author

/test e2e-ocp-helm

@github-actions

github-actions Bot commented Jul 22, 2026

Copy link
Copy Markdown
Contributor

The container image build workflow finished with status: cancelled.

@github-actions

github-actions Bot commented Jul 22, 2026

Copy link
Copy Markdown
Contributor

Image was built and published successfully. It is available at:

@codecov

codecov Bot commented Jul 22, 2026 •
edited
Loading

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 59.96%. Comparing base (7d73af5) to head (7a93c77).
⚠️ Report is 83 commits behind head on main.

Additional details and impacted files 
@@            Coverage Diff             @@
##             main    #5139      +/-   ##
==========================================
- Coverage   63.69%   59.96%   -3.74%     
==========================================
  Files         123      111      -12     
  Lines        2424     2198     -226     
  Branches      573      545      -28     
==========================================
- Hits         1544     1318     -226     
  Misses        878      878              
  Partials        2        2
Flag Coverage Δ
rhdh 59.96% <ø> (-3.74%) ⬇️
Components Coverage Δ
Backend plugins ∅ <ø> (∅)
Backend app 66.66% <ø> (ø)
Frontend app 58.89% <ø> (ø)
Plugin utils ∅ <ø> (∅)
Dynamic plugins utils ∅ <ø> (∅)

Continue to review full report in Codecov by Harness.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 7d73af5...7a93c77. Read the comment docs.

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@zdrapela
fix(ci): wait for target Postgres image during upgrade hops
0e7c913 
Start from fedora/postgresql-15 (not rhel9 default), require the
expected image before Ready, raise upgrade timeouts, enlarge PVC for
copy mode, and dump pod logs when a hop stalls.
@zdrapela

zdrapela commented Jul 23, 2026

Copy link
Copy Markdown
Member Author

/test e2e-ocp-helm

@github-actions

github-actions Bot commented Jul 23, 2026

Copy link
Copy Markdown
Contributor

Image was built and published successfully. It is available at:

@zdrapela
fix(ci): surface Postgres upgrade diagnostics and sanitize DB listing
1f71602 
Command substitution was swallowing oc describe/logs on hop failure, and
psql warnings were being treated as database names during collation refresh.
@zdrapela

zdrapela commented Jul 23, 2026

Copy link
Copy Markdown
Member Author

/test e2e-ocp-helm

@zdrapela
fix(ci): use rhel9 Postgres images for 15→16→18 upgrade path
e61a9a5 
fedora/postgresql-18 advertises PREV_VERSION=16 but only ships
postgresql-17 binaries, so POSTGRESQL_UPGRADE=copy from 16 never becomes Ready.
@zdrapela

zdrapela commented Jul 23, 2026

Copy link
Copy Markdown
Member Author

Root cause of Hop B failure (from gather-extra logs):

==========  $PGDATA upgrade: 16 -> 18  ==========
===>  Starting old postgresql once again for a clean shutdown...
/usr/lib64/pgsql/postgresql-16/bin/pg_ctl: No such file or directory

quay.io/fedora/postgresql-18 sets POSTGRESQL_PREV_VERSION=16 but only packages /usr/lib64/pgsql/postgresql-17/. Switched the CI path to registry.redhat.io/rhel9/postgresql-{15,16,18} (product chart images), which correctly ship the previous-major binaries.

/test e2e-ocp-helm

@github-actions

github-actions Bot commented Jul 23, 2026

Copy link
Copy Markdown
Contributor

The container image build workflow finished with status: cancelled.

@zdrapela

zdrapela commented Jul 23, 2026

Copy link
Copy Markdown
Member Author

✅ e2e-ocp-helm green on e61a9a512

Evidence (job):

  • rhel9/postgresql-15 → 15.18
  • Hop A POSTGRESQL_UPGRADE=copy → 16.14
  • Hop B POSTGRESQL_UPGRADE=copy → 18.4
  • Showcase Playwright: 36 passed
  • Log: PostgreSQL 15 -> 16 -> 18 Helm upgrade sequence completed

@github-actions

github-actions Bot commented Jul 23, 2026

Copy link
Copy Markdown
Contributor

Image was built and published successfully. It is available at:

@zdrapela
test(ci): run Playwright after each Postgres major during upgrade
7f0721f 
Exercise PG15 → tests → PG16 → tests → PG18 → tests, and always persist
pod logs plus Playwright output under unique ARTIFACT_DIR subdirs per hop.
@zdrapela

zdrapela commented Jul 23, 2026

Copy link
Copy Markdown
Member Author

Updated flow: Playwright after each major (PG15 / PG16 / PG18), with unique ARTIFACT_DIR subdirs (showcase-pg15, showcase-pg16, showcase-pg18) so pod logs and Playwright artifacts are not overwritten.

/test e2e-ocp-helm

@zdrapela zdrapela mentioned this pull request Jul 23, 2026
Closed
5 tasks
@github-actions

github-actions Bot commented Jul 23, 2026

Copy link
Copy Markdown
Contributor

Image was built and published successfully. It is available at:

@zdrapela
fix(e2e): tear down port-forward process groups between Playwright runs
d7c3007 
Multi-phase PG upgrade CI left kubectl redis port-forwards bound on 16379
because stop() only signaled the shell parent. Spawn detached, kill the
process group, use direct kubectl args, and fail fast when OVERALL_RESULT
is already non-zero after a phase.
@zdrapela

zdrapela commented Jul 23, 2026

Copy link
Copy Markdown
Member Author

Fix for multi-phase Playwright: tear down redis port-forward process groups + fail fast when a phase already set OVERALL_RESULT=1.

/test e2e-ocp-helm

@zdrapela

zdrapela commented Jul 23, 2026

Copy link
Copy Markdown
Member Author

/test e2e-ocp-helm

@zdrapela
zdrapela marked this pull request as ready for review July 23, 2026 08:18
@zdrapela

zdrapela commented Jul 23, 2026

Copy link
Copy Markdown
Member Author

/test e2e-ocp-helm

@github-actions

github-actions Bot commented Jul 23, 2026

Copy link
Copy Markdown
Contributor

Image was built and published successfully. It is available at:

@zdrapela
test(ci): wait for RHDH pod replacement and prove PG data survives up…
6e9833e 
…grades

Capture the Backstage pod UID before each major hop, restart if needed, and
wait until the previous UID is gone before Playwright. Seed a unique catalog
Component on PG15 and verify it via API + UI after each upgrade.
@zdrapela

zdrapela commented Jul 23, 2026

Copy link
Copy Markdown
Member Author

/test e2e-ocp-helm

@github-actions

github-actions Bot commented Jul 23, 2026

Copy link
Copy Markdown
Contributor

Image was built and published successfully. It is available at:

@zdrapela
fix(ci): seed pg-upgrade proof from public GitHub catalog URL
d0660ab 
In-cluster http://*.svc targets return 201 for location create but never
ingest entities under Backstage URL reader / SSRF restrictions. Use a
GitHub blob URL at PULL_PULL_SHA instead.
@zdrapela

zdrapela commented Jul 23, 2026

Copy link
Copy Markdown
Member Author

/test e2e-ocp-helm

@github-actions

github-actions Bot commented Jul 23, 2026

Copy link
Copy Markdown
Contributor

Image was built and published successfully. It is available at:

@zdrapela
fix(e2e): assert pg-upgrade proof by catalog title in UI
7a93c77 
The Catalog table renders metadata.title ("PG Upgrade Data Proof"), so
exact-match on metadata.name never appears after search.
@zdrapela

zdrapela commented Jul 23, 2026

Copy link
Copy Markdown
Member Author

/test e2e-ocp-helm

@sonarqubecloud

sonarqubecloud Bot commented Jul 23, 2026

Copy link
Copy Markdown

Quality Gate Passed Quality Gate passed

Issues
1 New issue
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

@github-actions

github-actions Bot commented Jul 23, 2026

Copy link
Copy Markdown
Contributor

Image was built and published successfully. It is available at:

@zdrapela
zdrapela marked this pull request as draft July 24, 2026 07:49
This was referenced Jul 24, 2026
Closed
Closed
@github-actions

github-actions Bot commented Aug 1, 2026

Copy link
Copy Markdown
Contributor

This PR is stale because it has been open 7 days with no activity. Remove stale label or comment or this will be closed in 21 days.

@github-actions github-actions Bot added the Stale label Aug 1, 2026
@zdrapela zdrapela mentioned this pull request Aug 3, 2026
Merged
3 tasks
@github-actions github-actions Bot removed the Stale label Aug 4, 2026
@github-actions

github-actions Bot commented Aug 12, 2026

Copy link
Copy Markdown
Contributor

This PR is stale because it has been open 7 days with no activity. Remove stale label or comment or this will be closed in 21 days.

@github-actions github-actions Bot added Stale and removed Stale labels Aug 12, 2026
@github-actions

github-actions Bot commented Aug 21, 2026

Copy link
Copy Markdown
Contributor

This PR is stale because it has been open 7 days with no activity. Remove stale label or comment or this will be closed in 21 days.

@github-actions github-actions Bot added Stale and removed Stale labels Aug 21, 2026
@github-actions

github-actions Bot commented Aug 30, 2026

Copy link
Copy Markdown
Contributor

This PR is stale because it has been open 7 days with no activity. Remove stale label or comment or this will be closed in 21 days.

@github-actions github-actions Bot added Stale and removed Stale labels Aug 30, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@gustavolira gustavolira Awaiting requested review from gustavolira
@teknaS47 teknaS47 Awaiting requested review from teknaS47

Assignees

No one assigned

Labels

Bug fix do-not-merge/work-in-progress Enhancement Tests

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@zdrapela