Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature-16396][storage-plugin] Add Tencent Cloud COS Storage Plugin #16565

Merged
merged 5 commits into from
Sep 27, 2024

Conversation

Mighten
Copy link
Contributor

@Mighten Mighten commented Sep 2, 2024

Purpose of the pull request

This pull request adds a Storage Plugin to the DS Resource Center to support Tencent Cloud COS and aims to close #16396 .

Brief change log

  • Add dolphinscheduler-storage-cos in dolphinscheduler-storage-plugin
  • Add CosRemoteLogHandler in dolphinscheduler-common
  • Add qcloud cos_api to pom.xml files

Verify this pull request

Screenshot-ds

Screenshot-qcloud

Screenshot-tencent-cos-remote-logging

Pull Request Notice

Pull Request Notice

If your pull request contain incompatible change, you should also add it to docs/docs/en/guide/upgrede/incompatible.md

@Mighten Mighten changed the title [feature-16396][Resource Center] Tencent Cloud COS Storage Plugin [feature-16396][plugin] Tencent Cloud COS Storage Plugin Sep 2, 2024
@Mighten Mighten changed the title [feature-16396][plugin] Tencent Cloud COS Storage Plugin [Feature-16396][Storage-plugin] Tencent Cloud COS Storage Plugin Sep 2, 2024
@Mighten Mighten changed the title [Feature-16396][Storage-plugin] Tencent Cloud COS Storage Plugin [Feature-16396][storage-plugin] Tencent Cloud COS Storage Plugin Sep 2, 2024
Copy link
Member

@SbloodyS SbloodyS left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since we don't have any tencent cos support for integration testing. I will -1 on this until we can find a way to test against this in out UT, IT and E2E.

@ruanwenjun ruanwenjun added the discussion discussion label Sep 2, 2024
@Mighten Mighten force-pushed the feature-16396 branch 2 times, most recently from ea644bc to 40e5ff2 Compare September 12, 2024 11:52
@fuchanghai
Copy link
Member

image we should remove bcprov-jdk15on-1.69

@Mighten Mighten force-pushed the feature-16396 branch 4 times, most recently from 8a33b77 to 037a91b Compare September 21, 2024 08:17
@SbloodyS SbloodyS added this to the 3.3.0 milestone Sep 23, 2024
@SbloodyS SbloodyS added the first time contributor First-time contributor label Sep 23, 2024
@SbloodyS SbloodyS changed the title [Feature-16396][storage-plugin] Tencent Cloud COS Storage Plugin [Feature-16396][storage-plugin] Add Tencent Cloud COS Storage Plugin Sep 23, 2024
@Mighten Mighten force-pushed the feature-16396 branch 2 times, most recently from fc7abdd to 2113175 Compare September 23, 2024 05:03
fuchanghai
fuchanghai previously approved these changes Sep 25, 2024
Copy link
Member

@ruanwenjun ruanwenjun left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

* Fixed SonarCloud Quality Readability B (required >= A)
* Added missing license files
* YAML-ized Tencent COS configuration from common.properties
* Added remote logger support with Tencent COS
* Removed redundant dependency: `bcprov-jdk15on-1.69.jar`
* Relocated Tencent COS Constants, and reduced magic values in COS with constants
* Isolated cloud storage configuration in `common.properties` -> `resource-center.yaml`
* Fixed CodeQL: Uncontrolled data used in path expression
* Updated log statements in `CosRemoteLogHandler`
* Updated `CosStorageOperator`
* Added *maven-shade-plugin* in storage-plugin-cos/pom.xml
* Refactored `pom.xml`
Copy link
Member

@SbloodyS SbloodyS left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1, Thanks for your contribution.

@SbloodyS SbloodyS added the feature new feature label Sep 27, 2024
Copy link

sonarcloud bot commented Sep 27, 2024

Quality Gate Failed Quality Gate failed

Failed conditions
0.0% Coverage on New Code (required ≥ 60%)

See analysis details on SonarCloud

@SbloodyS SbloodyS merged commit 9024dca into apache:dev Sep 27, 2024
68 of 70 checks passed
Copy link

boring-cyborg bot commented Sep 27, 2024

Awesome work, congrats on your first merged pull request!

@Mighten Mighten deleted the feature-16396 branch September 27, 2024 09:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Feature][resource center] Can the resource center support Tencent Cloud COS
4 participants