Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Use continuous batching by default #882

Open
wants to merge 58 commits into
base: master
Choose a base branch
from
Open

Use continuous batching by default #882

wants to merge 58 commits into from
+32 −3

Conversation

Wovchena
Copy link
Collaborator

No description provided.

@andrei-kochin andrei-kochin marked this pull request as ready for review September 19, 2024 14:52
andrei-kochin
andrei-kochin reviewed Sep 20, 2024
View reviewed changes
src/cpp/src/llm_pipeline.cpp Outdated Show resolved Hide resolved
src/cpp/src/llm_pipeline.cpp Outdated Show resolved Hide resolved
andrei-kochin
andrei-kochin reviewed Sep 20, 2024
View reviewed changes
src/cpp/src/llm_pipeline.cpp Outdated Show resolved Hide resolved
src/cpp/src/llm_pipeline.cpp Outdated Show resolved Hide resolved
andrei-kochin
andrei-kochin reviewed Sep 20, 2024
View reviewed changes
src/cpp/src/llm_pipeline.cpp Outdated Show resolved Hide resolved
src/cpp/src/llm_pipeline.cpp Outdated Show resolved Hide resolved
@ilya-lavrenov ilya-lavrenov added this to the 2024.5 milestone Sep 20, 2024
andrei-kochin
andrei-kochin reviewed Sep 20, 2024
View reviewed changes
src/cpp/src/llm_pipeline.cpp Outdated Show resolved Hide resolved
src/cpp/src/llm_pipeline.cpp Outdated Show resolved Hide resolved
andrei-kochin
andrei-kochin approved these changes Sep 23, 2024
View reviewed changes
src/cpp/src/llm_pipeline.cpp Outdated Show resolved Hide resolved
src/cpp/src/llm_pipeline.cpp Outdated Show resolved Hide resolved
andrei-kochin
andrei-kochin reviewed Sep 23, 2024
View reviewed changes
src/cpp/src/llm_pipeline.cpp Outdated Show resolved Hide resolved
src/cpp/src/llm_pipeline.cpp Outdated Show resolved Hide resolved
andrei-kochin
andrei-kochin requested changes Sep 25, 2024
View reviewed changes
src/cpp/src/llm_pipeline.cpp Outdated Show resolved Hide resolved
src/cpp/src/llm_pipeline.cpp Outdated Show resolved Hide resolved
ilya-lavrenov
ilya-lavrenov reviewed Oct 14, 2024
View reviewed changes
.github/workflows/causal_lm_cpp.yml Outdated Show resolved Hide resolved
.github/workflows/causal_lm_cpp.yml Outdated Show resolved Hide resolved
ilya-lavrenov
ilya-lavrenov reviewed Oct 14, 2024
View reviewed changes
.github/workflows/causal_lm_cpp.yml Outdated Show resolved Hide resolved
.github/workflows/causal_lm_cpp.yml Outdated Show resolved Hide resolved
ilya-lavrenov added a commit that referenced this pull request Oct 14, 2024
@ilya-lavrenov
Generic fixes for CB integration via LLMPipeline (#961)
3f01a95 
Extracted from
#882
ilya-lavrenov
ilya-lavrenov reviewed Oct 14, 2024
View reviewed changes
.github/workflows/causal_lm_cpp.yml Outdated Show resolved Hide resolved
.github/workflows/causal_lm_cpp.yml Outdated Show resolved Hide resolved
.github/workflows/causal_lm_cpp.yml Outdated Show resolved Hide resolved
@github-actions github-actions bot added category: continuous batching Continuous batching category: LLM LLM pipeline (stateful, static) category: sampling Sampling / Decoding algorithms category: GHA CI based on Github actions category: Python API Python API for GenAI labels Oct 17, 2024
@ilya-lavrenov ilya-lavrenov mentioned this pull request Oct 21, 2024
Merged
ilya-lavrenov
ilya-lavrenov reviewed Oct 21, 2024
View reviewed changes
src/python/py_generate_pipeline.cpp Outdated Show resolved Hide resolved
src/python/py_generate_pipeline.cpp Outdated Show resolved Hide resolved
src/python/py_generate_pipeline.cpp Outdated Show resolved Hide resolved
@github-actions github-actions bot removed the category: Python API Python API for GenAI label Oct 21, 2024
ilya-lavrenov
ilya-lavrenov reviewed Oct 22, 2024
View reviewed changes
src/cpp/src/llm_pipeline.cpp Outdated Show resolved Hide resolved
src/cpp/src/llm_pipeline.cpp Outdated Show resolved Hide resolved
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@andrei-kochin andrei-kochin andrei-kochin requested changes

@ilya-lavrenov ilya-lavrenov ilya-lavrenov left review comments

Requested changes must be addressed to merge this pull request.

Assignees

@ilya-lavrenov ilya-lavrenov

Labels
category: continuous batching Continuous batching category: GHA CI based on Github actions category: LLM LLM pipeline (stateful, static) category: sampling Sampling / Decoding algorithms do_not_merge do_not_review
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@Wovchena @ilya-lavrenov @andrei-kochin @popovaan