Max throughout for llama3.2 1B model

[JunhaoWang]

benchmark that targets high througput small llm use case

my use case does not care about latency, but need extremely high throughput at minimum price (cheap t4 gpu or just cpu), i.e. run 1 billion requests every week (avg 100k input tokens, 20k output tokens per request). i didnt find any benchmark targeting this type of usecase, would love some pointers.