Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Vectorized infer_beam_batch for improved performance #697

Merged
merged 1 commit into from
Aug 24, 2024

Conversation

9173860
Copy link
Contributor

@9173860 9173860 commented Aug 21, 2024

This pull request introduces key optimizations and improvements in model_48px aimed at increasing the efficiency of the OCR pipeline:

Vectorized infer_beam_batch:

The infer_beam_batch function in model_48px has been vectorized, significantly improving the speed of OCR inference. The original infer_beam_batch is also kept, so you can easily switch between the two for testing purposes.

Refactored encoders and decoders with forward methods:

Extracted the forward methods for both encoders and decoders. This refactoring enables more straightforward model exporting (e.g., to ONNX), allowing further optimization of inference performance and integration with deployment platforms like Triton Inference Server.

@zyddnys zyddnys merged commit ead6693 into zyddnys:main Aug 24, 2024
0 of 2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants