Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature/QueryGroup Add Create API in plugin #14459

Closed
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions CHANGELOG-3.0.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,8 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
- Add task completion count in search backpressure stats API ([#10028](https://github.com/opensearch-project/OpenSearch/pull/10028/))
- Deprecate CamelCase `PathHierarchy` tokenizer name in favor to lowercase `path_hierarchy` ([#10894](https://github.com/opensearch-project/OpenSearch/pull/10894))
- Breaking change: Do not request "search_pipelines" metrics by default in NodesInfoRequest ([#12497](https://github.com/opensearch-project/OpenSearch/pull/12497))
- [QueryGroup] Add QueryGroup CRUD APIs ([#13315](https://github.com/opensearch-project/OpenSearch/pull/13315))
- [QueryGroup] Add QueryGroup Create API ([#14459](https://github.com/opensearch-project/OpenSearch/pull/14459))
Comment on lines +27 to +28
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Merge issue?


### Deprecated

Expand Down
18 changes: 18 additions & 0 deletions plugins/workload-management/build.gradle
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
/*
* SPDX-License-Identifier: Apache-2.0
*
* The OpenSearch Contributors require contributions made to
* this file be licensed under the Apache-2.0 license or a
* compatible open source license.
*
* Modifications Copyright OpenSearch Contributors. See
* GitHub history for details.
*/

opensearchplugin {
description 'OpenSearch Workload Management Plugin.'
classname 'org.opensearch.plugin.wlm.WorkloadManagementPlugin'
}

dependencies {
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
/*
* SPDX-License-Identifier: Apache-2.0
*
* The OpenSearch Contributors require contributions made to
* this file be licensed under the Apache-2.0 license or a
* compatible open source license.
*/

package org.opensearch.plugin.wlm;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should package be plugin.wlm.action? That would be consistent with the structure in core, I guess.


import org.opensearch.action.ActionType;

/**
* Rest action to create QueryGroup
*
ruai0511 marked this conversation as resolved.
Show resolved Hide resolved
* @opensearch.api
*/
public class CreateQueryGroupAction extends ActionType<CreateQueryGroupResponse> {

/**
* An instance of CreateQueryGroupAction
*/
public static final CreateQueryGroupAction INSTANCE = new CreateQueryGroupAction();

Check warning on line 23 in plugins/workload-management/src/main/java/org/opensearch/plugin/wlm/CreateQueryGroupAction.java

View check run for this annotation

Codecov / codecov/patch

plugins/workload-management/src/main/java/org/opensearch/plugin/wlm/CreateQueryGroupAction.java#L23

Added line #L23 was not covered by tests

/**
* Name for CreateQueryGroupAction
*/
public static final String NAME = "cluster:admin/opensearch/query_group/wlm/_create";
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we have wlm prior to query_group in the action path?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that is the right way since wlm is the umbrella under which these query groups come.


/**
* Default constructor
*/
private CreateQueryGroupAction() {
super(NAME, CreateQueryGroupResponse::new);
}

Check warning on line 35 in plugins/workload-management/src/main/java/org/opensearch/plugin/wlm/CreateQueryGroupAction.java

View check run for this annotation

Codecov / codecov/patch

plugins/workload-management/src/main/java/org/opensearch/plugin/wlm/CreateQueryGroupAction.java#L34-L35

Added lines #L34 - L35 were not covered by tests
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,332 @@
/*
* SPDX-License-Identifier: Apache-2.0
*
* The OpenSearch Contributors require contributions made to
* this file be licensed under the Apache-2.0 license or a
* compatible open source license.
*/

package org.opensearch.plugin.wlm;

import org.opensearch.action.ActionRequest;
import org.opensearch.action.ActionRequestValidationException;
import org.opensearch.cluster.metadata.QueryGroup;
import org.opensearch.cluster.metadata.QueryGroup.QueryGroupMode;
import org.opensearch.common.UUIDs;
import org.opensearch.core.common.io.stream.StreamInput;
import org.opensearch.core.common.io.stream.StreamOutput;
import org.opensearch.core.common.io.stream.Writeable;
import org.opensearch.core.xcontent.XContentParser;
import org.joda.time.Instant;

import java.io.IOException;
import java.util.HashMap;
import java.util.Locale;
import java.util.Map;
import java.util.Objects;

import static org.opensearch.cluster.metadata.QueryGroup.ALLOWED_RESOURCES;
import static org.opensearch.cluster.metadata.QueryGroup.MAX_CHARS_ALLOWED_IN_NAME;

/**
* A request for create QueryGroup
* User input schema:
* {
ruai0511 marked this conversation as resolved.
Show resolved Hide resolved
* "jvm": 0.4,
* "mode": "enforced",
* "name": "analytics"
* }
*
* @opensearch.internal
*/
public class CreateQueryGroupRequest extends ActionRequest implements Writeable.Reader<CreateQueryGroupRequest> {
private String name;
private String _id;
private QueryGroupMode mode;
private Map<String, Object> resourceLimits;
private long updatedAtInMillis;

/**
* Default constructor for CreateQueryGroupRequest
*/
public CreateQueryGroupRequest() {}

Check warning on line 52 in plugins/workload-management/src/main/java/org/opensearch/plugin/wlm/CreateQueryGroupRequest.java

View check run for this annotation

Codecov / codecov/patch

plugins/workload-management/src/main/java/org/opensearch/plugin/wlm/CreateQueryGroupRequest.java#L52

Added line #L52 was not covered by tests

/**
* Constructor for CreateQueryGroupRequest
* @param queryGroup - A {@link QueryGroup} object
*/
public CreateQueryGroupRequest(QueryGroup queryGroup) {
this.name = queryGroup.getName();
this._id = queryGroup.get_id();
this.resourceLimits = queryGroup.getResourceLimits();
this.mode = queryGroup.getMode();
this.updatedAtInMillis = queryGroup.getUpdatedAtInMillis();
verifyCreateQueryGroupRequest(name, _id, mode, resourceLimits, updatedAtInMillis);
}

/**
* Constructor for CreateQueryGroupRequest
* @param name - QueryGroup name for CreateQueryGroupRequest
* @param _id - QueryGroup _id for CreateQueryGroupRequest
* @param mode - QueryGroup mode for CreateQueryGroupRequest
* @param resourceLimits - QueryGroup resourceLimits for CreateQueryGroupRequest
* @param updatedAtInMillis - QueryGroup updated time in millis for CreateQueryGroupRequest
*/
public CreateQueryGroupRequest(
String name,
String _id,
QueryGroupMode mode,
Map<String, Object> resourceLimits,
long updatedAtInMillis
) {
this.name = name;
this._id = _id;
this.resourceLimits = resourceLimits;
this.mode = mode;
this.updatedAtInMillis = updatedAtInMillis;
verifyCreateQueryGroupRequest(name, _id, mode, resourceLimits, updatedAtInMillis);
}

Check warning on line 88 in plugins/workload-management/src/main/java/org/opensearch/plugin/wlm/CreateQueryGroupRequest.java

View check run for this annotation

Codecov / codecov/patch

plugins/workload-management/src/main/java/org/opensearch/plugin/wlm/CreateQueryGroupRequest.java#L87-L88

Added lines #L87 - L88 were not covered by tests

/**
* Constructor for CreateQueryGroupRequest
* @param in - A {@link StreamInput} object
*/
public CreateQueryGroupRequest(StreamInput in) throws IOException {
super(in);
name = in.readString();
_id = in.readString();
mode = QueryGroupMode.fromName(in.readString());
resourceLimits = in.readMap();
updatedAtInMillis = in.readLong();
verifyCreateQueryGroupRequest(name, _id, mode, resourceLimits, updatedAtInMillis);
}

/**
* Verification for CreateQueryGroupRequest
* @param name - QueryGroup name for CreateQueryGroupRequest
* @param _id - QueryGroup _id for CreateQueryGroupRequest
* @param mode - QueryGroup mode for CreateQueryGroupRequest
* @param resourceLimits - QueryGroup resourceLimits for CreateQueryGroupRequest
* @param updatedAtInMillis - QueryGroup updated time in millis for CreateQueryGroupRequest
*/
private void verifyCreateQueryGroupRequest(
ruai0511 marked this conversation as resolved.
Show resolved Hide resolved
String name,
String _id,
QueryGroupMode mode,
Map<String, Object> resourceLimits,
long updatedAtInMillis
) {
Objects.requireNonNull(_id, "QueryGroup._id can't be null");
Objects.requireNonNull(name, "QueryGroup.name can't be null");
Objects.requireNonNull(resourceLimits, "QueryGroup.resourceLimits can't be null");
Objects.requireNonNull(mode, "QueryGroup.mode can't be null");
Objects.requireNonNull(updatedAtInMillis, "QueryGroup.updatedAtInMillis can't be null");

validateName(name);
validateMode(mode);
validateResourceLimits(resourceLimits);
validateUpdatedAtInMillis(updatedAtInMillis);
}

/**
* Verification for CreateQueryGroupRequest.name
* @param name - name to be verified
*/
public static void validateName(String name) {
if (name.isEmpty()) {
throw new IllegalArgumentException("QueryGroup.name cannot be empty");
}
if (name.length() > MAX_CHARS_ALLOWED_IN_NAME) {
throw new IllegalArgumentException("QueryGroup.name shouldn't be more than 50 chars long");

Check warning on line 140 in plugins/workload-management/src/main/java/org/opensearch/plugin/wlm/CreateQueryGroupRequest.java

View check run for this annotation

Codecov / codecov/patch

plugins/workload-management/src/main/java/org/opensearch/plugin/wlm/CreateQueryGroupRequest.java#L140

Added line #L140 was not covered by tests
}
if (name.startsWith("-") || name.startsWith("_")) {
throw new IllegalArgumentException("QueryGroup.name cannot start with '_' or '-'.");
}
if (name.matches(".*[ ,:\"*+/\\\\|?#><].*")) {
throw new IllegalArgumentException("QueryGroup.name can't contain spaces, commas, quotes, slashes, :, *, +, |, ?, #, >, or <");
}
}

/**
* Verification for CreateQueryGroupRequest.mode
* @param mode - mode to be verified
*/
public static void validateMode(QueryGroupMode mode) {
if (!mode.getName().equals("monitor")) {
throw new IllegalArgumentException("QueryGroup.mode must be monitor");

Check warning on line 156 in plugins/workload-management/src/main/java/org/opensearch/plugin/wlm/CreateQueryGroupRequest.java

View check run for this annotation

Codecov / codecov/patch

plugins/workload-management/src/main/java/org/opensearch/plugin/wlm/CreateQueryGroupRequest.java#L156

Added line #L156 was not covered by tests
}
}

/**
* Verification for CreateQueryGroupRequest.resourceLimits
* @param resourceLimits - resourceLimits to be verified
*/
public static void validateResourceLimits(Map<String, Object> resourceLimits) {
if (resourceLimits.isEmpty()) {
throw new IllegalArgumentException("QueryGroup.resourceLimits should at least have 1 resource limit.");
}
for (Map.Entry<String, Object> resource : resourceLimits.entrySet()) {
String resourceName = resource.getKey();
Double threshold = (Double) resource.getValue();
Objects.requireNonNull(resourceName, "resourceName can't be null");
Objects.requireNonNull(threshold, "resource value can't be null");

if (threshold < 0 || threshold > 1) {
throw new IllegalArgumentException("Resource value should be between 0 and 1");
}
String str = String.valueOf(threshold);
if (str.contains(".") && str.split("\\.")[1].length() > 2) {
throw new IllegalArgumentException("Resource value should have at most two digits after the decimal point");

Check warning on line 179 in plugins/workload-management/src/main/java/org/opensearch/plugin/wlm/CreateQueryGroupRequest.java

View check run for this annotation

Codecov / codecov/patch

plugins/workload-management/src/main/java/org/opensearch/plugin/wlm/CreateQueryGroupRequest.java#L179

Added line #L179 was not covered by tests
}
if (!ALLOWED_RESOURCES.contains(resourceName.toLowerCase(Locale.ROOT))) {
throw new IllegalArgumentException(
"resource has to be valid, valid resources are: " + ALLOWED_RESOURCES.stream().reduce((x, e) -> x + ", " + e).get()
);
}
}
}

/**
* Verification for CreateQueryGroupRequest.updatedAtInMillis
* @param updatedAt - updated time to be verified
*/
public static void validateUpdatedAtInMillis(long updatedAt) {
long minValidTimestamp = Instant.ofEpochMilli(0L).getMillis();
long currentSeconds = Instant.now().getMillis();
if (minValidTimestamp > updatedAt || updatedAt > currentSeconds) {
throw new IllegalArgumentException("QueryGroup.updatedAtInMillis is not a valid epoch");

Check warning on line 197 in plugins/workload-management/src/main/java/org/opensearch/plugin/wlm/CreateQueryGroupRequest.java

View check run for this annotation

Codecov / codecov/patch

plugins/workload-management/src/main/java/org/opensearch/plugin/wlm/CreateQueryGroupRequest.java#L197

Added line #L197 was not covered by tests
}
}

@Override
public CreateQueryGroupRequest read(StreamInput in) throws IOException {
return new CreateQueryGroupRequest(in);

Check warning on line 203 in plugins/workload-management/src/main/java/org/opensearch/plugin/wlm/CreateQueryGroupRequest.java

View check run for this annotation

Codecov / codecov/patch

plugins/workload-management/src/main/java/org/opensearch/plugin/wlm/CreateQueryGroupRequest.java#L203

Added line #L203 was not covered by tests
}

/**
* Generate a CreateQueryGroupRequest from XContent
* @param parser - A {@link XContentParser} object
*/
public static CreateQueryGroupRequest fromXContent(XContentParser parser) throws IOException {

while (parser.currentToken() != XContentParser.Token.START_OBJECT) {
parser.nextToken();

Check warning on line 213 in plugins/workload-management/src/main/java/org/opensearch/plugin/wlm/CreateQueryGroupRequest.java

View check run for this annotation

Codecov / codecov/patch

plugins/workload-management/src/main/java/org/opensearch/plugin/wlm/CreateQueryGroupRequest.java#L213

Added line #L213 was not covered by tests
}

if (parser.currentToken() != XContentParser.Token.START_OBJECT) {
throw new IllegalArgumentException("expected start object but got a " + parser.currentToken());

Check warning on line 217 in plugins/workload-management/src/main/java/org/opensearch/plugin/wlm/CreateQueryGroupRequest.java

View check run for this annotation

Codecov / codecov/patch

plugins/workload-management/src/main/java/org/opensearch/plugin/wlm/CreateQueryGroupRequest.java#L217

Added line #L217 was not covered by tests
}

XContentParser.Token token;
String fieldName = "";
String name = null;
String _id = UUIDs.randomBase64UUID();
QueryGroupMode mode = null;
long updatedAtInMillis = Instant.now().getMillis();

Check warning on line 225 in plugins/workload-management/src/main/java/org/opensearch/plugin/wlm/CreateQueryGroupRequest.java

View check run for this annotation

Codecov / codecov/patch

plugins/workload-management/src/main/java/org/opensearch/plugin/wlm/CreateQueryGroupRequest.java#L221-L225

Added lines #L221 - L225 were not covered by tests

// Map to hold resources
final Map<String, Object> resourceLimits_ = new HashMap<>();

Check warning on line 228 in plugins/workload-management/src/main/java/org/opensearch/plugin/wlm/CreateQueryGroupRequest.java

View check run for this annotation

Codecov / codecov/patch

plugins/workload-management/src/main/java/org/opensearch/plugin/wlm/CreateQueryGroupRequest.java#L228

Added line #L228 was not covered by tests
while ((token = parser.nextToken()) != null) {
if (token == XContentParser.Token.FIELD_NAME) {
fieldName = parser.currentName();

Check warning on line 231 in plugins/workload-management/src/main/java/org/opensearch/plugin/wlm/CreateQueryGroupRequest.java

View check run for this annotation

Codecov / codecov/patch

plugins/workload-management/src/main/java/org/opensearch/plugin/wlm/CreateQueryGroupRequest.java#L231

Added line #L231 was not covered by tests
} else if (token.isValue()) {
if (fieldName.equals("name")) {
name = parser.text();

Check warning on line 234 in plugins/workload-management/src/main/java/org/opensearch/plugin/wlm/CreateQueryGroupRequest.java

View check run for this annotation

Codecov / codecov/patch

plugins/workload-management/src/main/java/org/opensearch/plugin/wlm/CreateQueryGroupRequest.java#L234

Added line #L234 was not covered by tests
} else if (fieldName.equals("mode")) {
mode = QueryGroupMode.fromName(parser.text());

Check warning on line 236 in plugins/workload-management/src/main/java/org/opensearch/plugin/wlm/CreateQueryGroupRequest.java

View check run for this annotation

Codecov / codecov/patch

plugins/workload-management/src/main/java/org/opensearch/plugin/wlm/CreateQueryGroupRequest.java#L236

Added line #L236 was not covered by tests
} else if (ALLOWED_RESOURCES.contains(fieldName)) {
resourceLimits_.put(fieldName, parser.doubleValue());

Check warning on line 238 in plugins/workload-management/src/main/java/org/opensearch/plugin/wlm/CreateQueryGroupRequest.java

View check run for this annotation

Codecov / codecov/patch

plugins/workload-management/src/main/java/org/opensearch/plugin/wlm/CreateQueryGroupRequest.java#L238

Added line #L238 was not covered by tests
} else {
throw new IllegalArgumentException("unrecognised [field=" + fieldName + " in QueryGroup");

Check warning on line 240 in plugins/workload-management/src/main/java/org/opensearch/plugin/wlm/CreateQueryGroupRequest.java

View check run for this annotation

Codecov / codecov/patch

plugins/workload-management/src/main/java/org/opensearch/plugin/wlm/CreateQueryGroupRequest.java#L240

Added line #L240 was not covered by tests
}
}
}
return new CreateQueryGroupRequest(name, _id, mode, resourceLimits_, updatedAtInMillis);

Check warning on line 244 in plugins/workload-management/src/main/java/org/opensearch/plugin/wlm/CreateQueryGroupRequest.java

View check run for this annotation

Codecov / codecov/patch

plugins/workload-management/src/main/java/org/opensearch/plugin/wlm/CreateQueryGroupRequest.java#L244

Added line #L244 was not covered by tests
}

@Override
public ActionRequestValidationException validate() {
return null;

Check warning on line 249 in plugins/workload-management/src/main/java/org/opensearch/plugin/wlm/CreateQueryGroupRequest.java

View check run for this annotation

Codecov / codecov/patch

plugins/workload-management/src/main/java/org/opensearch/plugin/wlm/CreateQueryGroupRequest.java#L249

Added line #L249 was not covered by tests
}

/**
* name getter
*/
public String getName() {
return name;
}

/**
* name setter
* @param name - name to be set
*/
public void setName(String name) {
this.name = name;
}

Check warning on line 265 in plugins/workload-management/src/main/java/org/opensearch/plugin/wlm/CreateQueryGroupRequest.java

View check run for this annotation

Codecov / codecov/patch

plugins/workload-management/src/main/java/org/opensearch/plugin/wlm/CreateQueryGroupRequest.java#L264-L265

Added lines #L264 - L265 were not covered by tests

/**
* resourceLimits getter
*/
public Map<String, Object> getResourceLimits() {
return resourceLimits;
}

/**
* resourceLimits setter
* @param resourceLimits - resourceLimit to be set
*/
public void setResourceLimits(Map<String, Object> resourceLimits) {
this.resourceLimits = resourceLimits;
}

Check warning on line 280 in plugins/workload-management/src/main/java/org/opensearch/plugin/wlm/CreateQueryGroupRequest.java

View check run for this annotation

Codecov / codecov/patch

plugins/workload-management/src/main/java/org/opensearch/plugin/wlm/CreateQueryGroupRequest.java#L279-L280

Added lines #L279 - L280 were not covered by tests

/**
* mode getter
*/
public QueryGroupMode getMode() {
return mode;
}

/**
* mode setter
* @param mode - mode to be set
*/
public void setMode(QueryGroupMode mode) {
this.mode = mode;
}

Check warning on line 295 in plugins/workload-management/src/main/java/org/opensearch/plugin/wlm/CreateQueryGroupRequest.java

View check run for this annotation

Codecov / codecov/patch

plugins/workload-management/src/main/java/org/opensearch/plugin/wlm/CreateQueryGroupRequest.java#L294-L295

Added lines #L294 - L295 were not covered by tests

@Override
public void writeTo(StreamOutput out) throws IOException {
super.writeTo(out);
QueryGroup.writeToOutputStream(out, name, _id, mode, resourceLimits, updatedAtInMillis);
}

/**
* _id getter
*/
public String get_id() {
return _id;

Check warning on line 307 in plugins/workload-management/src/main/java/org/opensearch/plugin/wlm/CreateQueryGroupRequest.java

View check run for this annotation

Codecov / codecov/patch

plugins/workload-management/src/main/java/org/opensearch/plugin/wlm/CreateQueryGroupRequest.java#L307

Added line #L307 was not covered by tests
}

/**
* UUID setter
* @param _id - _id to be set
*/
public void set_id(String _id) {
this._id = _id;
}

Check warning on line 316 in plugins/workload-management/src/main/java/org/opensearch/plugin/wlm/CreateQueryGroupRequest.java

View check run for this annotation

Codecov / codecov/patch

plugins/workload-management/src/main/java/org/opensearch/plugin/wlm/CreateQueryGroupRequest.java#L315-L316

Added lines #L315 - L316 were not covered by tests

/**
* updatedAtInMillis getter
*/
public long getUpdatedAtInMillis() {
return updatedAtInMillis;
}

/**
* updatedAtInMillis setter
* @param updatedAtInMillis - updatedAtInMillis to be set
*/
public void setUpdatedAtInMillis(long updatedAtInMillis) {
this.updatedAtInMillis = updatedAtInMillis;
}

Check warning on line 331 in plugins/workload-management/src/main/java/org/opensearch/plugin/wlm/CreateQueryGroupRequest.java

View check run for this annotation

Codecov / codecov/patch

plugins/workload-management/src/main/java/org/opensearch/plugin/wlm/CreateQueryGroupRequest.java#L330-L331

Added lines #L330 - L331 were not covered by tests
}
Loading
Loading