Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

move examples gateway #992

Open
wants to merge 17 commits into
base: main
Choose a base branch
from
Open

move examples gateway #992

wants to merge 17 commits into from

Conversation

lkk12014402
Copy link
Collaborator

Description

move examples gateway implementation from GenAIComps to GenAIExamples

Copy link

github-actions bot commented Oct 20, 2024

Dependency Review

✅ No vulnerabilities or license issues found.

Scanned Files

def __init__(self, host="0.0.0.0", port=8000):
self.host = host
self.port = port
ServiceOrchestrator.align_inputs = align_inputs
ServiceOrchestrator.align_outputs = align_outputs
ServiceOrchestrator.align_generator = align_generator
self.megaservice = ServiceOrchestrator()

super().__init__(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The graph definition should run before the gateway is started, otherwise the DAG is not recognized correctly by the gateway and the gateway just will be forked to another process without the graph definition. One change could be e.g.

        self.megaservice = ServiceOrchestrator()
        self.add_remote_service()   ## Call here to make sure megaservice know the graph
        super().__init__(
            megaservice=self.megaservice,
            host=self.host,
            port=self.port,
            endpoint="/v1/audioqna",
            input_datatype=AudioChatCompletionRequest,
            output_datatype=ChatCompletionResponse,
        )

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

update code, please review again~ Thanks~

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've reproduced the subsequent CI issues. I guess we need to set MEGA_SERVICE_HOST_IP=0.0.0.0 after this modification.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants