feat: add MCP setup script and enhance E2E testing - #55
Open
Ihor-Khomenko wants to merge 8 commits into
Open
Conversation
- Introduced a new Ruby script `setup-mcp.rb` for provisioning users and configurations in the OpenProject MCP. - Updated the Helm chart to include the new setup script in the config map. - Enhanced E2E tests with new configurations and added tests for MCP authentication and LLM evaluations. - Updated `.gitignore` to exclude Python cache files. - Added new dependencies in `package.json` and `package-lock.json` for improved functionality.
- Added support for skipping Nextcloud and Keycloak endpoint checks in the GitHub action. - Updated deployment summary script to conditionally include Nextcloud and Keycloak URLs based on new input parameters. - Introduced a new MCP user, Bob_AI, in the setup script and updated related tests to reflect this change. - Enhanced E2E tests to utilize the new MCP user and improved resource reading and tool selection tests. - Updated Helm chart configurations to support MCP integration and ensure proper provisioning during setup jobs.
- Updated the setup job in Helm charts to include configurable backoffLimit and activeDeadlineSeconds for improved reliability. - Modified E2E workflows to incorporate new input parameters for skipping services like Nextcloud, Keycloak, and XWiki during tests. - Enhanced the deployment summary and diagnostics scripts to conditionally handle skipped services based on the new configurations. - Increased the timeout for the setup job wait script to ensure adequate time for setup completion.
- Added checks in `check-mcp-setup.sh` to validate the use of `update_column` for hashed tokens and prevent direct assignment to `token=`. - Modified `setup-mcp.rb` to use `update_column` for writing the hashed token, ensuring compliance with Doorkeeper's token handling practices. - Enhanced error messaging for better clarity during setup validation.
- Enhanced curl command in diagnostics to retain stderr output for better error reporting. - Updated logic to handle URL validation failures, ensuring previews are kept for debugging. - Improved deployment summary to conditionally display service URLs based on availability. - Refined error messaging for clarity when tests are skipped or not run due to validation issues.
- Introduced a new step in the GitHub action to wait for TLS readiness before proceeding with service checks, enhancing the reliability of the deployment process. - Updated the `preview-check-url.sh` script to increase the default number of attempts for URL checks from 12 to 18, allowing for more robust validation of service availability.
…rkflows - Added support for skipping specific services (Nextcloud, Keycloak, XWiki) during TLS readiness checks in the GitHub action, improving flexibility in deployment scenarios. - Updated the curl command in the `preview-check-url.sh` script to provide detailed HTTP response output for better diagnostics when endpoints are not ready. - Enhanced error messaging to clearly indicate TLS handshake failures and the status of pending endpoints, improving overall reliability and feedback during deployment.
- Enhanced the TLS readiness check in the preview endpoint workflow to capture and report detailed error messages for failed handshakes. - Updated the logging level in multiple workflows from ERROR to INFO for better visibility during execution. - Added diagnostics for the pullpreview-caddy deployment, including logs and status checks, to aid in troubleshooting.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
setup-mcp.rbfor provisioning users and configurations in the OpenProject MCP..gitignoreto exclude Python cache files.package.jsonandpackage-lock.jsonfor improved functionality.Ticket
https://community.openproject.org/wp/TA-26
What are you trying to accomplish?
Delivered an initial MCP test harness so OpenProject’s MCP server can be exercised automatically in the integration QA stack.
What approach did you choose and why?
Helm setup job seeds MCP via charts/opnc-integration/scripts/setup-mcp.rb (Brian QA user, McpConfiguration, OAuth/token fixtures).
Shared Playwright helpers: e2e/utils/mcp-client.ts, e2e/utils/mcp-token.ts, e2e/utils/llm-evaluator.ts.
Playwright E2E (e2e/tests/mcp/, tags @mcp