Skip to content

OpenPPL/ppl.pmx

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

About OPMX

Open PPL Model Exchange (OPMX) is an open ecosystem that empowers AI developers to choose the right tools as their project evolves. OPMX provides an open source format for AI models. It defines an extensible computation graph model, as well as definitions of built-in operators and standard data types.

Currently OPMX focus on the capabilities and hardware friendliness needed for Large Language Model(LLM) inferencing.

Important Notice

  • Support weight only quant model at llama at 16/07/2024.
  • PMX has changed to OPMX at 25/04/2024.
  • And the domain of operators are also changed to opmx(refers to TOC).
  • You can find the old code at llm_v1

Operator spec

Table of Contents: Link

About add new operator: Link

About update an operator's version: Link

Install OPMX

You can install opmx like:

pip install -e .

Then you can just use OPMX like a python api.

Use OPMX Python API

OPMX provides functional API based on torch.autograd.Function.

Clone the OPMX repo, and import torch_function like this:

import pmx_llm.torch_function as OPMX

And then use it as Pytorch's functional API:

norm, skip_out = OPMX.skip_rms_norm(x, weight, skip_in, -1, eps)

We can use these API in pytorch to custom your own model.

All OPMX function could be exported as custom operators by torch.onnx.export.

Model Zoo

Some opensource model are provided in our model zoo.

Currently models:

About

No description, website, or topics provided.

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages