added return_fields function, attempting to optionally limit fields r… #633
Conversation
| @@ -447,6 +448,11 @@ def __init__( | |||
| else: | |||
| self.sort_fields = [] | |||
|
|
|||
| if return_fields: | |||
| self.return_fields = self.validate_return_fields(return_fields) | |||
There was a problem hiding this comment.
since return_fields is being declared below as a method you can't perform this assignment (it makes the linter mad and probably leads to some invalid state)
| @@ -504,8 +510,19 @@ def query(self): | |||
| if self._query.startswith("(") or self._query == "*" | |||
| else f"({self._query})" | |||
| ) + f"=>[{self.knn}]" | |||
| if self.return_fields: | |||
| self._query += f" RETURN {','.join(self.return_fields)}" | |||
There was a problem hiding this comment.
The RETURN statement shouldn't be added to the query string, that's the filtration part of the query. Rather it needs to be passed to the list of args being passed to Redis. See the execute method inside of find_query to see what I mean, the RETURN, the number of fields being returned, and each individual field being returned need to be added as individual arguments to that list argument for it to be interpreted correctly.
There was a problem hiding this comment.
sorry, what are you going to do with deserialization?
It seems like the query with RETURN returns data in different way... it was my problem
I mean https://github.com/redis/redis-om-python/blob/main/aredis_om/model/model.py#L1583
There was a problem hiding this comment.
For me this works
@staticmethod
def create_return_part(fields: List[str] | Set[str]) -> List[str]:
q = ["RETURN", "n"]
for field in fields:
q.extend([f"$.{field}", "AS", field])
q[1] = len(q) - 2
return q
There was a problem hiding this comment.
IMO you'd return a dict[string,dict[string,string]] as that's essentially what the API responds with (technically it's an array of strings and arrays with an integer at it's head)
There was a problem hiding this comment.
I'm talking about this behavior:
I have Redis (Search + Json)
When I run query (using redis-cli for example) without RETURN:
FT.SEARCH ':foo:index' '(@parent_id:[2 2])' LIMIT 1 1
I get the output like JSON document
3) 1) "$" 2) "{\"updated_at\":\"2024-09-13 12:14:55.877717\",\"id\":3,\"page_id\":3
And here are right data types (id and page id are integers)
But when I use RETURN
FT.SEARCH ':foo:index' '(@parent_id:[2 2])' LIMIT 0 1000 RETURN 6 $.id AS id, $.page_id AS page_id
I get the output like this
3) 1) "id,"
2) "2"
3) "page_id"
4) "2"
So my question is how are you going to deserialize this results?
There was a problem hiding this comment.
My fault guys,
Everything will be OK after validation
field.validate(value, values, loc=field.alias, cls=cls_)
I've turned it off, very expensive to run validations on read queries
There was a problem hiding this comment.
But please check if it works with dict fields, I get error "value is not a valid dict" while processing '{"foo": "bar"}'
| @py_test_mark_asyncio | ||
| async def test_return_specified_fields(members, m): | ||
| member1, member2, member3 = members | ||
| actual = await m.Member.find( |
There was a problem hiding this comment.
One issue to contend with here is that the model validation will not work correctly when running find, you'll need to make sure you return a dictionary or some sub-set of the model to prevent validation errors from being tossed by the find.
|
Please add also exclude keyword, it'll be convenient |
…eturned by find. Will solve #568 when completed.