Skip to content

Commit

Permalink
deploy: 83fb615
Browse files Browse the repository at this point in the history
  • Loading branch information
yiliu30 committed Sep 15, 2023
1 parent 0d0aca5 commit 862e31e
Show file tree
Hide file tree
Showing 521 changed files with 520 additions and 541 deletions.
9 changes: 0 additions & 9 deletions latest/_sources/docs/source/quantization_weight_only.md.txt
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,6 @@ Theoretically, round-to-nearest (RTN) is the most straightforward way to quantiz

There are many excellent works for weight only quantization to improve its accuracy performance, such as AWQ[3], GPTQ[4]. Neural compressor integrates these popular algorithms in time to help customers leverage them and deploy them to their own tasks.


## Supported Framework Model Matrix

| Algorithms/Framework | PyTorch | ONNX Runtime |
Expand All @@ -34,14 +33,6 @@ There are many excellent works for weight only quantization to improve its accur

**Note:** To get the validated accuracy results on popular models, please refer to [PyTorch Models with Torch 2.0.1+cpu in WOQ Mode](./validated_model_list.html/#pytorch-models-with-torch-201cpu-in-woq-mode)

> **RTN:** A quantification method that we can think of very intuitively. It does not require additional datasets and is a very fast quantization method. Generally speaking, RTN will convert the weight into a uniformly distributed integer data type, but some algorithms, such as Qlora, propose a non-uniform NF4 data type and prove its theoretical optimality.

> **GPTQ:** A new one-shot weight quantization method based on approximate second-order information, that is both highly-accurate and highly efficient[4]. The weights of each column are updated based on the fixed-scale pseudo-quantization error and the inverse of the Hessian matrix calculated from the activations. The updated columns sharing the same scale may generate a new max/min value, so the scale needs to be saved for restoration.

> **AWQ:** Proved that protecting only 1% of salient weights can greatly reduce quantization error. the salient weight channels are selected by observing the distribution of activation and weight per channel. The salient weights are also quantized after multiplying a big scale factor before quantization for preserving.

> **TEQ:** A trainable equivalent transformation that preserves the FP32 precision in weight-only quantization. It is inspired by AWQ while providing a new solution to search for the optimal per-channel scaling factor between activations and weights.

## Examples
### **Quantization Capability**:
| Config | Capability |
Expand Down
2 changes: 1 addition & 1 deletion latest/autoapi/block_mask/index.html
Original file line number Diff line number Diff line change
Expand Up @@ -105,7 +105,7 @@
Built with <a href="https://www.sphinx-doc.org/">Sphinx</a> using a
<a href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
provided by <a href="https://readthedocs.org">Read the Docs</a>.
<jinja2.runtime.BlockReference object at 0x7fa9755450c0>
<jinja2.runtime.BlockReference object at 0x7ff61f6c2a40>
<p></p><div><a href='https://www.intel.com/content/www/us/en/privacy/intel-cookie-notice.html' data-cookie-notice='true'>Cookies</a> <a href='https://www.intel.com/content/www/us/en/privacy/intel-privacy-notice.html'>| Privacy</a></div>


Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -144,7 +144,7 @@ <h3>Functions<a class="headerlink" href="#functions" title="Permalink to this he
Built with <a href="https://www.sphinx-doc.org/">Sphinx</a> using a
<a href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
provided by <a href="https://readthedocs.org">Read the Docs</a>.
<jinja2.runtime.BlockReference object at 0x7fa9754d4d90>
<jinja2.runtime.BlockReference object at 0x7ff61f4efca0>
<p></p><div><a href='https://www.intel.com/content/www/us/en/privacy/intel-cookie-notice.html' data-cookie-notice='true'>Cookies</a> <a href='https://www.intel.com/content/www/us/en/privacy/intel-privacy-notice.html'>| Privacy</a></div>


Expand Down
2 changes: 1 addition & 1 deletion latest/autoapi/neural_compressor/adaptor/index.html
Original file line number Diff line number Diff line change
Expand Up @@ -215,7 +215,7 @@ <h2>Package Contents<a class="headerlink" href="#package-contents" title="Permal
Built with <a href="https://www.sphinx-doc.org/">Sphinx</a> using a
<a href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
provided by <a href="https://readthedocs.org">Read the Docs</a>.
<jinja2.runtime.BlockReference object at 0x7fa975669b10>
<jinja2.runtime.BlockReference object at 0x7ff61f4ecc40>
<p></p><div><a href='https://www.intel.com/content/www/us/en/privacy/intel-cookie-notice.html' data-cookie-notice='true'>Cookies</a> <a href='https://www.intel.com/content/www/us/en/privacy/intel-privacy-notice.html'>| Privacy</a></div>


Expand Down
2 changes: 1 addition & 1 deletion latest/autoapi/neural_compressor/adaptor/keras/index.html
Original file line number Diff line number Diff line change
Expand Up @@ -123,7 +123,7 @@ <h3>Classes<a class="headerlink" href="#classes" title="Permalink to this headin
Built with <a href="https://www.sphinx-doc.org/">Sphinx</a> using a
<a href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
provided by <a href="https://readthedocs.org">Read the Docs</a>.
<jinja2.runtime.BlockReference object at 0x7fa975663520>
<jinja2.runtime.BlockReference object at 0x7ff61f6c0700>
<p></p><div><a href='https://www.intel.com/content/www/us/en/privacy/intel-cookie-notice.html' data-cookie-notice='true'>Cookies</a> <a href='https://www.intel.com/content/www/us/en/privacy/intel-privacy-notice.html'>| Privacy</a></div>


Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -104,7 +104,7 @@
Built with <a href="https://www.sphinx-doc.org/">Sphinx</a> using a
<a href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
provided by <a href="https://readthedocs.org">Read the Docs</a>.
<jinja2.runtime.BlockReference object at 0x7fa9756622c0>
<jinja2.runtime.BlockReference object at 0x7ff622d953c0>
<p></p><div><a href='https://www.intel.com/content/www/us/en/privacy/intel-cookie-notice.html' data-cookie-notice='true'>Cookies</a> <a href='https://www.intel.com/content/www/us/en/privacy/intel-privacy-notice.html'>| Privacy</a></div>


Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -104,7 +104,7 @@
Built with <a href="https://www.sphinx-doc.org/">Sphinx</a> using a
<a href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
provided by <a href="https://readthedocs.org">Read the Docs</a>.
<jinja2.runtime.BlockReference object at 0x7fa975662b00>
<jinja2.runtime.BlockReference object at 0x7ff61f45dfc0>
<p></p><div><a href='https://www.intel.com/content/www/us/en/privacy/intel-cookie-notice.html' data-cookie-notice='true'>Cookies</a> <a href='https://www.intel.com/content/www/us/en/privacy/intel-privacy-notice.html'>| Privacy</a></div>


Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -104,7 +104,7 @@
Built with <a href="https://www.sphinx-doc.org/">Sphinx</a> using a
<a href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
provided by <a href="https://readthedocs.org">Read the Docs</a>.
<jinja2.runtime.BlockReference object at 0x7fa975447550>
<jinja2.runtime.BlockReference object at 0x7ff61f45c520>
<p></p><div><a href='https://www.intel.com/content/www/us/en/privacy/intel-cookie-notice.html' data-cookie-notice='true'>Cookies</a> <a href='https://www.intel.com/content/www/us/en/privacy/intel-privacy-notice.html'>| Privacy</a></div>


Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -104,7 +104,7 @@
Built with <a href="https://www.sphinx-doc.org/">Sphinx</a> using a
<a href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
provided by <a href="https://readthedocs.org">Read the Docs</a>.
<jinja2.runtime.BlockReference object at 0x7fa975446440>
<jinja2.runtime.BlockReference object at 0x7ff61f45e590>
<p></p><div><a href='https://www.intel.com/content/www/us/en/privacy/intel-cookie-notice.html' data-cookie-notice='true'>Cookies</a> <a href='https://www.intel.com/content/www/us/en/privacy/intel-privacy-notice.html'>| Privacy</a></div>


Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -104,7 +104,7 @@
Built with <a href="https://www.sphinx-doc.org/">Sphinx</a> using a
<a href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
provided by <a href="https://readthedocs.org">Read the Docs</a>.
<jinja2.runtime.BlockReference object at 0x7fa975660490>
<jinja2.runtime.BlockReference object at 0x7ff61f45caf0>
<p></p><div><a href='https://www.intel.com/content/www/us/en/privacy/intel-cookie-notice.html' data-cookie-notice='true'>Cookies</a> <a href='https://www.intel.com/content/www/us/en/privacy/intel-privacy-notice.html'>| Privacy</a></div>


Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -104,7 +104,7 @@
Built with <a href="https://www.sphinx-doc.org/">Sphinx</a> using a
<a href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
provided by <a href="https://readthedocs.org">Read the Docs</a>.
<jinja2.runtime.BlockReference object at 0x7fa975663400>
<jinja2.runtime.BlockReference object at 0x7ff622d96e90>
<p></p><div><a href='https://www.intel.com/content/www/us/en/privacy/intel-cookie-notice.html' data-cookie-notice='true'>Cookies</a> <a href='https://www.intel.com/content/www/us/en/privacy/intel-privacy-notice.html'>| Privacy</a></div>


Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -104,7 +104,7 @@
Built with <a href="https://www.sphinx-doc.org/">Sphinx</a> using a
<a href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
provided by <a href="https://readthedocs.org">Read the Docs</a>.
<jinja2.runtime.BlockReference object at 0x7fa9756611b0>
<jinja2.runtime.BlockReference object at 0x7ff61f6c08e0>
<p></p><div><a href='https://www.intel.com/content/www/us/en/privacy/intel-cookie-notice.html' data-cookie-notice='true'>Cookies</a> <a href='https://www.intel.com/content/www/us/en/privacy/intel-privacy-notice.html'>| Privacy</a></div>


Expand Down
2 changes: 1 addition & 1 deletion latest/autoapi/neural_compressor/adaptor/mxnet/index.html
Original file line number Diff line number Diff line change
Expand Up @@ -138,7 +138,7 @@ <h3>Classes<a class="headerlink" href="#classes" title="Permalink to this headin
Built with <a href="https://www.sphinx-doc.org/">Sphinx</a> using a
<a href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
provided by <a href="https://readthedocs.org">Read the Docs</a>.
<jinja2.runtime.BlockReference object at 0x7fa9754d43d0>
<jinja2.runtime.BlockReference object at 0x7ff61f45c8e0>
<p></p><div><a href='https://www.intel.com/content/www/us/en/privacy/intel-cookie-notice.html' data-cookie-notice='true'>Cookies</a> <a href='https://www.intel.com/content/www/us/en/privacy/intel-privacy-notice.html'>| Privacy</a></div>


Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -113,7 +113,7 @@ <h2>Submodules<a class="headerlink" href="#submodules" title="Permalink to this
Built with <a href="https://www.sphinx-doc.org/">Sphinx</a> using a
<a href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
provided by <a href="https://readthedocs.org">Read the Docs</a>.
<jinja2.runtime.BlockReference object at 0x7fa975445630>
<jinja2.runtime.BlockReference object at 0x7ff61f45f610>
<p></p><div><a href='https://www.intel.com/content/www/us/en/privacy/intel-cookie-notice.html' data-cookie-notice='true'>Cookies</a> <a href='https://www.intel.com/content/www/us/en/privacy/intel-privacy-notice.html'>| Privacy</a></div>


Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -677,7 +677,7 @@ <h3>Functions<a class="headerlink" href="#functions" title="Permalink to this he
Built with <a href="https://www.sphinx-doc.org/">Sphinx</a> using a
<a href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
provided by <a href="https://readthedocs.org">Read the Docs</a>.
<jinja2.runtime.BlockReference object at 0x7fa975446170>
<jinja2.runtime.BlockReference object at 0x7ff61f23c790>
<p></p><div><a href='https://www.intel.com/content/www/us/en/privacy/intel-cookie-notice.html' data-cookie-notice='true'>Cookies</a> <a href='https://www.intel.com/content/www/us/en/privacy/intel-privacy-notice.html'>| Privacy</a></div>


Expand Down
2 changes: 1 addition & 1 deletion latest/autoapi/neural_compressor/adaptor/onnxrt/index.html
Original file line number Diff line number Diff line change
Expand Up @@ -184,7 +184,7 @@ <h3>Classes<a class="headerlink" href="#classes" title="Permalink to this headin
Built with <a href="https://www.sphinx-doc.org/">Sphinx</a> using a
<a href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
provided by <a href="https://readthedocs.org">Read the Docs</a>.
<jinja2.runtime.BlockReference object at 0x7fa9754473a0>
<jinja2.runtime.BlockReference object at 0x7ff626727250>
<p></p><div><a href='https://www.intel.com/content/www/us/en/privacy/intel-cookie-notice.html' data-cookie-notice='true'>Cookies</a> <a href='https://www.intel.com/content/www/us/en/privacy/intel-privacy-notice.html'>| Privacy</a></div>


Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -124,7 +124,7 @@ <h3>Classes<a class="headerlink" href="#classes" title="Permalink to this headin
Built with <a href="https://www.sphinx-doc.org/">Sphinx</a> using a
<a href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
provided by <a href="https://readthedocs.org">Read the Docs</a>.
<jinja2.runtime.BlockReference object at 0x7fa975669db0>
<jinja2.runtime.BlockReference object at 0x7ff61f3a7d90>
<p></p><div><a href='https://www.intel.com/content/www/us/en/privacy/intel-cookie-notice.html' data-cookie-notice='true'>Cookies</a> <a href='https://www.intel.com/content/www/us/en/privacy/intel-privacy-notice.html'>| Privacy</a></div>


Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -224,7 +224,7 @@ <h3>Functions<a class="headerlink" href="#functions" title="Permalink to this he
Built with <a href="https://www.sphinx-doc.org/">Sphinx</a> using a
<a href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
provided by <a href="https://readthedocs.org">Read the Docs</a>.
<jinja2.runtime.BlockReference object at 0x7fa975444970>
<jinja2.runtime.BlockReference object at 0x7ff61f45fe80>
<p></p><div><a href='https://www.intel.com/content/www/us/en/privacy/intel-cookie-notice.html' data-cookie-notice='true'>Cookies</a> <a href='https://www.intel.com/content/www/us/en/privacy/intel-privacy-notice.html'>| Privacy</a></div>


Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -150,7 +150,7 @@ <h2>Submodules<a class="headerlink" href="#submodules" title="Permalink to this
Built with <a href="https://www.sphinx-doc.org/">Sphinx</a> using a
<a href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
provided by <a href="https://readthedocs.org">Read the Docs</a>.
<jinja2.runtime.BlockReference object at 0x7fa97538c760>
<jinja2.runtime.BlockReference object at 0x7ff622d96b00>
<p></p><div><a href='https://www.intel.com/content/www/us/en/privacy/intel-cookie-notice.html' data-cookie-notice='true'>Cookies</a> <a href='https://www.intel.com/content/www/us/en/privacy/intel-privacy-notice.html'>| Privacy</a></div>


Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -151,7 +151,7 @@ <h3>Classes<a class="headerlink" href="#classes" title="Permalink to this headin
Built with <a href="https://www.sphinx-doc.org/">Sphinx</a> using a
<a href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
provided by <a href="https://readthedocs.org">Read the Docs</a>.
<jinja2.runtime.BlockReference object at 0x7fa9752ff2b0>
<jinja2.runtime.BlockReference object at 0x7ff61f23e380>
<p></p><div><a href='https://www.intel.com/content/www/us/en/privacy/intel-cookie-notice.html' data-cookie-notice='true'>Cookies</a> <a href='https://www.intel.com/content/www/us/en/privacy/intel-privacy-notice.html'>| Privacy</a></div>


Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -133,7 +133,7 @@ <h3>Classes<a class="headerlink" href="#classes" title="Permalink to this headin
Built with <a href="https://www.sphinx-doc.org/">Sphinx</a> using a
<a href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
provided by <a href="https://readthedocs.org">Read the Docs</a>.
<jinja2.runtime.BlockReference object at 0x7fa9752fff40>
<jinja2.runtime.BlockReference object at 0x7ff61f23e230>
<p></p><div><a href='https://www.intel.com/content/www/us/en/privacy/intel-cookie-notice.html' data-cookie-notice='true'>Cookies</a> <a href='https://www.intel.com/content/www/us/en/privacy/intel-privacy-notice.html'>| Privacy</a></div>


Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -133,7 +133,7 @@ <h3>Classes<a class="headerlink" href="#classes" title="Permalink to this headin
Built with <a href="https://www.sphinx-doc.org/">Sphinx</a> using a
<a href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
provided by <a href="https://readthedocs.org">Read the Docs</a>.
<jinja2.runtime.BlockReference object at 0x7fa9752fd6f0>
<jinja2.runtime.BlockReference object at 0x7ff61f23d090>
<p></p><div><a href='https://www.intel.com/content/www/us/en/privacy/intel-cookie-notice.html' data-cookie-notice='true'>Cookies</a> <a href='https://www.intel.com/content/www/us/en/privacy/intel-privacy-notice.html'>| Privacy</a></div>


Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -151,7 +151,7 @@ <h3>Classes<a class="headerlink" href="#classes" title="Permalink to this headin
Built with <a href="https://www.sphinx-doc.org/">Sphinx</a> using a
<a href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
provided by <a href="https://readthedocs.org">Read the Docs</a>.
<jinja2.runtime.BlockReference object at 0x7fa97c924b80>
<jinja2.runtime.BlockReference object at 0x7ff626669bd0>
<p></p><div><a href='https://www.intel.com/content/www/us/en/privacy/intel-cookie-notice.html' data-cookie-notice='true'>Cookies</a> <a href='https://www.intel.com/content/www/us/en/privacy/intel-privacy-notice.html'>| Privacy</a></div>


Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -133,7 +133,7 @@ <h3>Classes<a class="headerlink" href="#classes" title="Permalink to this headin
Built with <a href="https://www.sphinx-doc.org/">Sphinx</a> using a
<a href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
provided by <a href="https://readthedocs.org">Read the Docs</a>.
<jinja2.runtime.BlockReference object at 0x7fa97538f3d0>
<jinja2.runtime.BlockReference object at 0x7ff61f23cbb0>
<p></p><div><a href='https://www.intel.com/content/www/us/en/privacy/intel-cookie-notice.html' data-cookie-notice='true'>Cookies</a> <a href='https://www.intel.com/content/www/us/en/privacy/intel-privacy-notice.html'>| Privacy</a></div>


Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -133,7 +133,7 @@ <h3>Classes<a class="headerlink" href="#classes" title="Permalink to this headin
Built with <a href="https://www.sphinx-doc.org/">Sphinx</a> using a
<a href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
provided by <a href="https://readthedocs.org">Read the Docs</a>.
<jinja2.runtime.BlockReference object at 0x7fa97538d660>
<jinja2.runtime.BlockReference object at 0x7ff61f23ead0>
<p></p><div><a href='https://www.intel.com/content/www/us/en/privacy/intel-cookie-notice.html' data-cookie-notice='true'>Cookies</a> <a href='https://www.intel.com/content/www/us/en/privacy/intel-privacy-notice.html'>| Privacy</a></div>


Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -133,7 +133,7 @@ <h3>Classes<a class="headerlink" href="#classes" title="Permalink to this headin
Built with <a href="https://www.sphinx-doc.org/">Sphinx</a> using a
<a href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
provided by <a href="https://readthedocs.org">Read the Docs</a>.
<jinja2.runtime.BlockReference object at 0x7fa975446620>
<jinja2.runtime.BlockReference object at 0x7ff622d96ef0>
<p></p><div><a href='https://www.intel.com/content/www/us/en/privacy/intel-cookie-notice.html' data-cookie-notice='true'>Cookies</a> <a href='https://www.intel.com/content/www/us/en/privacy/intel-privacy-notice.html'>| Privacy</a></div>


Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -133,7 +133,7 @@ <h3>Classes<a class="headerlink" href="#classes" title="Permalink to this headin
Built with <a href="https://www.sphinx-doc.org/">Sphinx</a> using a
<a href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
provided by <a href="https://readthedocs.org">Read the Docs</a>.
<jinja2.runtime.BlockReference object at 0x7fa975445840>
<jinja2.runtime.BlockReference object at 0x7ff62685bf40>
<p></p><div><a href='https://www.intel.com/content/www/us/en/privacy/intel-cookie-notice.html' data-cookie-notice='true'>Cookies</a> <a href='https://www.intel.com/content/www/us/en/privacy/intel-privacy-notice.html'>| Privacy</a></div>


Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -133,7 +133,7 @@ <h3>Classes<a class="headerlink" href="#classes" title="Permalink to this headin
Built with <a href="https://www.sphinx-doc.org/">Sphinx</a> using a
<a href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
provided by <a href="https://readthedocs.org">Read the Docs</a>.
<jinja2.runtime.BlockReference object at 0x7fa97c7ca440>
<jinja2.runtime.BlockReference object at 0x7ff61f543160>
<p></p><div><a href='https://www.intel.com/content/www/us/en/privacy/intel-cookie-notice.html' data-cookie-notice='true'>Cookies</a> <a href='https://www.intel.com/content/www/us/en/privacy/intel-privacy-notice.html'>| Privacy</a></div>


Expand Down
Loading

0 comments on commit 862e31e

Please sign in to comment.