-
Notifications
You must be signed in to change notification settings - Fork 4
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Implement queries for cmr and hydrocron #198
Comments
I was able to implement the CMR query which returns a list of granule URs present in CMR for a specified time range. I started to implement the Hydrocron query but ran into some confusion around DynamoDB queries. I would like to query the hydrocron tables in a way that returns all items = hydrocron_table.query(
IndexName="GranuleURIndex",
KeyConditionExpression=(Key("range_start_time").between(start_time, end_time))
) But when I try to do this the query produces an error and complains that I have not included a "key schema element" in the query: botocore.exceptions.ClientError: An error occurred (ValidationException) when calling the Query operation: Query condition missed key schema element From my understanding you cannot query DynamoDB on a date range only and instead would need to perform a scan which is costly. For now I am going to query Hydrocron by I won't be able to easily determine what is in Hydrocron but not in CMR (or overlap between the two). I think we would to set up another global secondary index on the "range_start_time" with a range key of "granuleUR" if we want to facilitate a different query. Although I am not sure that quite fits with the DynamoDB use case. @torimcd - Does this fit with your understanding of DynamoDB queries? Any objections to de-prioritizing determining what is in Hydrocron but not CMR since we were only going to log those results for now? |
We will de-scope determining overlap and what exists in Hydrocron but not CMR for now. Create #206 to track future work. |
The text was updated successfully, but these errors were encountered: