Skip to content

Commit

Permalink
Fixing P3L incompatibility with cython. (opendatahub-io#200)
Browse files Browse the repository at this point in the history
* Fixing incompatibility with cython.

* Change type as per reviewer suggestions

Co-authored-by: Gregory Shtrasberg <156009573+gshtras@users.noreply.github.com>

---------

Co-authored-by: Matt Wong <156021403+mawong-amd@users.noreply.github.com>
Co-authored-by: Gregory Shtrasberg <156009573+gshtras@users.noreply.github.com>
  • Loading branch information
3 people authored Sep 20, 2024
1 parent 0e80e85 commit bae9170
Showing 1 changed file with 2 additions and 1 deletion.
3 changes: 2 additions & 1 deletion vllm/model_executor/layers/sampler.py
Original file line number Diff line number Diff line change
Expand Up @@ -1436,7 +1436,8 @@ def _build_sampler_output(
deferred_sample_results_args=deferred_sample_results_args)


def _get_next_prompt_tokens(seq_group: SequenceGroupToSample) -> List[int]:
def _get_next_prompt_tokens(
seq_group: SequenceGroupToSample) -> Tuple[int, ...]:
"""Get a list of next prompt tokens to compute logprob from a
given sequence group.
Expand Down

0 comments on commit bae9170

Please sign in to comment.