-
Notifications
You must be signed in to change notification settings - Fork 0
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Ensure agent action is correct #12
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Great first pull request! Yay; Just left a few comments :)
.idea/.gitignore
Outdated
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please .gitignore the files in .idea
and .vscode
haicosystem/agents/__init__.py
Outdated
@@ -1,5 +1,6 @@ | |||
from .llm_agent import LLMAgentX | |||
from .llm_agent import LLMAgentX, LLMAgentY |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I know I started with the LLMAgentX haha, but imo we need better naming here:
maybe
LLMAgentX
-> LLMAgentHuman
LLMAgentY
-> LLMAgentToolcalling
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
sounds good! I'll make the name more informative
haicosystem/agents/llm_agent.py
Outdated
@@ -110,3 +122,96 @@ async def aact(self, obs: Observation) -> AgentAction: | |||
f"{current_agent} said: ", "" | |||
) | |||
return action | |||
|
|||
|
|||
def format_bad_output_tool( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why not directly import from Sotopia repo?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The format_bad_output
from Sotopia requires an argument format_instructions
to indicate the correct format for tools. Initially there's not a unified format for all the tools, so I wrote a function to skip this argument, but now we can pass in the format from the tool_parser.py you implemented as the format_instructions
to format_bad_output
from Sotopia
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah! Sounds great!
haicosystem/agents/llm_agent.py
Outdated
) | ||
if action.action_type == "action": | ||
try: | ||
json.loads(action.argument) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I am wondering, besides checking whether the action.argument
can be loaded as a JSON object, it could worthwhile to check whether the input fits the tool argument or not:
Maybe you can borrow the function here:
HAI-Cosys/haicosystem/tools/utils.py
Line 203 in aa485c9
def validate_inputs(parameters: List[ArgParameter], inputs: Dict[str, Any]): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'll rewrite this!
@GloriaXimingLu One more thing, any chance you can merge |
for sure! I thought I pulled from the latest main before I created my branch, but I'll double check to make sure my branch syn with the current main. |
I've updated the branch. The the main update is to add a function validate_agentAction to make sure the agent's action can be parsed into a LangchainAgentAction. I also changed the naming of |
Closes #1
📑 Description
Ensure that the agent's action follow the required format and implemented a rewrite mechanism to ensure the generated action fits into the required API calls before sending it out
LLMAgentY
in haicosystem/agents/llm_agent.py to rewrite the action if json parse fail✅ Checks
type/descript
(e.g.feature/add-llm-agents
)ℹ Additional Information