Skip to content

Commit

Permalink
deploy: 795aeb5
Browse files Browse the repository at this point in the history
  • Loading branch information
WeiweiZhang1 committed Oct 18, 2024
1 parent 9ff97df commit 67b1711
Show file tree
Hide file tree
Showing 562 changed files with 900 additions and 578 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ Functions

.. autoapisummary::

neural_compressor.torch.utils.utility.is_optimum_habana_available
neural_compressor.torch.utils.utility.register_algo
neural_compressor.torch.utils.utility.fetch_module
neural_compressor.torch.utils.utility.set_module
Expand All @@ -26,11 +27,32 @@ Functions
neural_compressor.torch.utils.utility.get_processor_type_from_user_config
neural_compressor.torch.utils.utility.dowload_hf_model
neural_compressor.torch.utils.utility.load_empty_model
neural_compressor.torch.utils.utility.get_module
neural_compressor.torch.utils.utility.get_layer_names_in_block
neural_compressor.torch.utils.utility.to_dtype
neural_compressor.torch.utils.utility.to_device
neural_compressor.torch.utils.utility.get_block_names
neural_compressor.torch.utils.utility.validate_modules
neural_compressor.torch.utils.utility.get_multimodal_block_names
neural_compressor.torch.utils.utility.detect_device
neural_compressor.torch.utils.utility.run_fn_for_vlm_autoround


Module Contents
---------------

.. py:function:: is_optimum_habana_available()
Checks if the Optimum Habana module is available for use with the transformers library.

This function checks two conditions:
1. If the `optimum` package is available using `transformers.utils.import_utils.is_optimum_available`.
2. If the `optimum.habana` module can be found using `importlib.util.find_spec`.

:returns: True if Optimum Habana is available, False otherwise.
:rtype: bool


.. py:function:: register_algo(name)
Decorator function to register algorithms in the algos_mapping dictionary.
Expand Down Expand Up @@ -167,3 +189,110 @@ Module Contents
Load a empty model.


.. py:function:: get_module(module, key)
Get module from model by key name.

:param module: original model
:type module: torch.nn.Module
:param key: module name to be replaced
:type key: str


.. py:function:: get_layer_names_in_block(model, supported_types=[torch.nn.Linear, transformers.modeling_utils.Conv1D], quant_block_list=None)
Retrieves the names of layers within each block of the model.

:returns:

A list of strings, where each string is the name of a layer
within a block of the model.
:rtype: list


.. py:function:: to_dtype(input, dtype=torch.float32)
Moves input data to the specified data type.

Args:
input: The input data to be moved.
dtype: The target data type.

Returns:
The input data on the specified data type.


.. py:function:: to_device(input, device=torch.device('cpu'))
Moves input data to the specified device.

Args:
input: The input data to be moved.
device: The target device.

Returns:
The input data on the specified device.


.. py:function:: get_block_names(model)
Get the block names for transformers-like networks.

Args:
model: The model.

Returns:
block_names: A list whose elements are list of block's layer names


.. py:function:: validate_modules(module_names)
Test a list of modules' validity.

Args:
modules (list of str): List of strings to be validated.

Returns:
bool: True if all modules have equal length or not dependent, otherwise False.


.. py:function:: get_multimodal_block_names(model, quant_vision=False)
Get the multimodal model block names for transformers-like networks.

Args:
model: The model.

Returns:
block_names: A list whose elements are list of block's layer names


.. py:function:: detect_device(device=None)
Detects the device to use for model execution (GPU, HPU, or CPU).

:param device:
- If a string ('cuda', 'cpu', or 'hpu') or torch.device is provided, that device is selected.
- If an integer is provided, it treats it as a GPU device index.
- If None or 'auto', it automatically selects 'cuda' if available, 'hpu' if Habana is available,
or falls back to 'cpu'.
:type device: str, int, torch.device, optional

:returns: The selected device in string format ('cuda:X', 'hpu', or 'cpu').
:rtype: str


.. py:function:: run_fn_for_vlm_autoround(model, dataloader, seqlen=512, nsamples=512)
Runs a model on a provided dataset with automatic device detection for vector-language models.

:param model: The model to run.
:param dataloader: A PyTorch dataloader providing the input data for the model.
:param seqlen: The minimum sequence length of input data to process. Defaults to 512.
:type seqlen: int, optional
:param nsamples: The number of samples to process before stopping. Defaults to 512.
:type nsamples: int, optional

:returns: None


Original file line number Diff line number Diff line change
Expand Up @@ -114,7 +114,7 @@ <h2>Submodules<a class="headerlink" href="#submodules" title="Link to this headi
Built with <a href="https://www.sphinx-doc.org/">Sphinx</a> using a
<a href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
provided by <a href="https://readthedocs.org">Read the Docs</a>.
<jinja2.runtime.BlockReference object at 0x7f06c1e98040>
<jinja2.runtime.BlockReference object at 0x7f95be9e4490>
<p></p><div><a href='https://www.intel.com/content/www/us/en/privacy/intel-cookie-notice.html' data-cookie-notice='true'>Cookies</a> <a href='https://www.intel.com/content/www/us/en/privacy/intel-privacy-notice.html'>| Privacy</a></div>


Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -678,7 +678,7 @@ <h2>Module Contents<a class="headerlink" href="#module-contents" title="Link to
Built with <a href="https://www.sphinx-doc.org/">Sphinx</a> using a
<a href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
provided by <a href="https://readthedocs.org">Read the Docs</a>.
<jinja2.runtime.BlockReference object at 0x7f06c3806bf0>
<jinja2.runtime.BlockReference object at 0x7f95c01cc8b0>
<p></p><div><a href='https://www.intel.com/content/www/us/en/privacy/intel-cookie-notice.html' data-cookie-notice='true'>Cookies</a> <a href='https://www.intel.com/content/www/us/en/privacy/intel-privacy-notice.html'>| Privacy</a></div>


Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -125,7 +125,7 @@ <h2>Module Contents<a class="headerlink" href="#module-contents" title="Link to
Built with <a href="https://www.sphinx-doc.org/">Sphinx</a> using a
<a href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
provided by <a href="https://readthedocs.org">Read the Docs</a>.
<jinja2.runtime.BlockReference object at 0x7f06c37f3ee0>
<jinja2.runtime.BlockReference object at 0x7f95be8d6a10>
<p></p><div><a href='https://www.intel.com/content/www/us/en/privacy/intel-cookie-notice.html' data-cookie-notice='true'>Cookies</a> <a href='https://www.intel.com/content/www/us/en/privacy/intel-privacy-notice.html'>| Privacy</a></div>


Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -225,7 +225,7 @@ <h2>Module Contents<a class="headerlink" href="#module-contents" title="Link to
Built with <a href="https://www.sphinx-doc.org/">Sphinx</a> using a
<a href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
provided by <a href="https://readthedocs.org">Read the Docs</a>.
<jinja2.runtime.BlockReference object at 0x7f06bb67dd80>
<jinja2.runtime.BlockReference object at 0x7f95c38e65f0>
<p></p><div><a href='https://www.intel.com/content/www/us/en/privacy/intel-cookie-notice.html' data-cookie-notice='true'>Cookies</a> <a href='https://www.intel.com/content/www/us/en/privacy/intel-privacy-notice.html'>| Privacy</a></div>


Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -127,7 +127,7 @@ <h2>Submodules<a class="headerlink" href="#submodules" title="Link to this headi
Built with <a href="https://www.sphinx-doc.org/">Sphinx</a> using a
<a href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
provided by <a href="https://readthedocs.org">Read the Docs</a>.
<jinja2.runtime.BlockReference object at 0x7f06bb823760>
<jinja2.runtime.BlockReference object at 0x7f95be8d7f10>
<p></p><div><a href='https://www.intel.com/content/www/us/en/privacy/intel-cookie-notice.html' data-cookie-notice='true'>Cookies</a> <a href='https://www.intel.com/content/www/us/en/privacy/intel-privacy-notice.html'>| Privacy</a></div>


Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -152,7 +152,7 @@ <h2>Module Contents<a class="headerlink" href="#module-contents" title="Link to
Built with <a href="https://www.sphinx-doc.org/">Sphinx</a> using a
<a href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
provided by <a href="https://readthedocs.org">Read the Docs</a>.
<jinja2.runtime.BlockReference object at 0x7f06bb823190>
<jinja2.runtime.BlockReference object at 0x7f95be83a080>
<p></p><div><a href='https://www.intel.com/content/www/us/en/privacy/intel-cookie-notice.html' data-cookie-notice='true'>Cookies</a> <a href='https://www.intel.com/content/www/us/en/privacy/intel-privacy-notice.html'>| Privacy</a></div>


Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -134,7 +134,7 @@ <h2>Module Contents<a class="headerlink" href="#module-contents" title="Link to
Built with <a href="https://www.sphinx-doc.org/">Sphinx</a> using a
<a href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
provided by <a href="https://readthedocs.org">Read the Docs</a>.
<jinja2.runtime.BlockReference object at 0x7f06bb821570>
<jinja2.runtime.BlockReference object at 0x7f95bb9842e0>
<p></p><div><a href='https://www.intel.com/content/www/us/en/privacy/intel-cookie-notice.html' data-cookie-notice='true'>Cookies</a> <a href='https://www.intel.com/content/www/us/en/privacy/intel-privacy-notice.html'>| Privacy</a></div>


Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -134,7 +134,7 @@ <h2>Module Contents<a class="headerlink" href="#module-contents" title="Link to
Built with <a href="https://www.sphinx-doc.org/">Sphinx</a> using a
<a href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
provided by <a href="https://readthedocs.org">Read the Docs</a>.
<jinja2.runtime.BlockReference object at 0x7f06bb821240>
<jinja2.runtime.BlockReference object at 0x7f95bb987220>
<p></p><div><a href='https://www.intel.com/content/www/us/en/privacy/intel-cookie-notice.html' data-cookie-notice='true'>Cookies</a> <a href='https://www.intel.com/content/www/us/en/privacy/intel-privacy-notice.html'>| Privacy</a></div>


Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -152,7 +152,7 @@ <h2>Module Contents<a class="headerlink" href="#module-contents" title="Link to
Built with <a href="https://www.sphinx-doc.org/">Sphinx</a> using a
<a href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
provided by <a href="https://readthedocs.org">Read the Docs</a>.
<jinja2.runtime.BlockReference object at 0x7f06bd7afeb0>
<jinja2.runtime.BlockReference object at 0x7f95bb9857e0>
<p></p><div><a href='https://www.intel.com/content/www/us/en/privacy/intel-cookie-notice.html' data-cookie-notice='true'>Cookies</a> <a href='https://www.intel.com/content/www/us/en/privacy/intel-privacy-notice.html'>| Privacy</a></div>


Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -134,7 +134,7 @@ <h2>Module Contents<a class="headerlink" href="#module-contents" title="Link to
Built with <a href="https://www.sphinx-doc.org/">Sphinx</a> using a
<a href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
provided by <a href="https://readthedocs.org">Read the Docs</a>.
<jinja2.runtime.BlockReference object at 0x7f06bd7afa00>
<jinja2.runtime.BlockReference object at 0x7f95bb9860e0>
<p></p><div><a href='https://www.intel.com/content/www/us/en/privacy/intel-cookie-notice.html' data-cookie-notice='true'>Cookies</a> <a href='https://www.intel.com/content/www/us/en/privacy/intel-privacy-notice.html'>| Privacy</a></div>


Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -134,7 +134,7 @@ <h2>Module Contents<a class="headerlink" href="#module-contents" title="Link to
Built with <a href="https://www.sphinx-doc.org/">Sphinx</a> using a
<a href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
provided by <a href="https://readthedocs.org">Read the Docs</a>.
<jinja2.runtime.BlockReference object at 0x7f06bd7af850>
<jinja2.runtime.BlockReference object at 0x7f95c374c7f0>
<p></p><div><a href='https://www.intel.com/content/www/us/en/privacy/intel-cookie-notice.html' data-cookie-notice='true'>Cookies</a> <a href='https://www.intel.com/content/www/us/en/privacy/intel-privacy-notice.html'>| Privacy</a></div>


Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -134,7 +134,7 @@ <h2>Module Contents<a class="headerlink" href="#module-contents" title="Link to
Built with <a href="https://www.sphinx-doc.org/">Sphinx</a> using a
<a href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
provided by <a href="https://readthedocs.org">Read the Docs</a>.
<jinja2.runtime.BlockReference object at 0x7f06bb822f50>
<jinja2.runtime.BlockReference object at 0x7f95bb987af0>
<p></p><div><a href='https://www.intel.com/content/www/us/en/privacy/intel-cookie-notice.html' data-cookie-notice='true'>Cookies</a> <a href='https://www.intel.com/content/www/us/en/privacy/intel-privacy-notice.html'>| Privacy</a></div>


Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -134,7 +134,7 @@ <h2>Module Contents<a class="headerlink" href="#module-contents" title="Link to
Built with <a href="https://www.sphinx-doc.org/">Sphinx</a> using a
<a href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
provided by <a href="https://readthedocs.org">Read the Docs</a>.
<jinja2.runtime.BlockReference object at 0x7f06bb820490>
<jinja2.runtime.BlockReference object at 0x7f95bb984490>
<p></p><div><a href='https://www.intel.com/content/www/us/en/privacy/intel-cookie-notice.html' data-cookie-notice='true'>Cookies</a> <a href='https://www.intel.com/content/www/us/en/privacy/intel-privacy-notice.html'>| Privacy</a></div>


Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -134,7 +134,7 @@ <h2>Module Contents<a class="headerlink" href="#module-contents" title="Link to
Built with <a href="https://www.sphinx-doc.org/">Sphinx</a> using a
<a href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
provided by <a href="https://readthedocs.org">Read the Docs</a>.
<jinja2.runtime.BlockReference object at 0x7f06bb823040>
<jinja2.runtime.BlockReference object at 0x7f95bb986530>
<p></p><div><a href='https://www.intel.com/content/www/us/en/privacy/intel-cookie-notice.html' data-cookie-notice='true'>Cookies</a> <a href='https://www.intel.com/content/www/us/en/privacy/intel-privacy-notice.html'>| Privacy</a></div>


Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -134,7 +134,7 @@ <h2>Module Contents<a class="headerlink" href="#module-contents" title="Link to
Built with <a href="https://www.sphinx-doc.org/">Sphinx</a> using a
<a href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
provided by <a href="https://readthedocs.org">Read the Docs</a>.
<jinja2.runtime.BlockReference object at 0x7f06bb823490>
<jinja2.runtime.BlockReference object at 0x7f95bb984550>
<p></p><div><a href='https://www.intel.com/content/www/us/en/privacy/intel-cookie-notice.html' data-cookie-notice='true'>Cookies</a> <a href='https://www.intel.com/content/www/us/en/privacy/intel-privacy-notice.html'>| Privacy</a></div>


Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -134,7 +134,7 @@ <h2>Module Contents<a class="headerlink" href="#module-contents" title="Link to
Built with <a href="https://www.sphinx-doc.org/">Sphinx</a> using a
<a href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
provided by <a href="https://readthedocs.org">Read the Docs</a>.
<jinja2.runtime.BlockReference object at 0x7f06bb8230d0>
<jinja2.runtime.BlockReference object at 0x7f95c3954b50>
<p></p><div><a href='https://www.intel.com/content/www/us/en/privacy/intel-cookie-notice.html' data-cookie-notice='true'>Cookies</a> <a href='https://www.intel.com/content/www/us/en/privacy/intel-privacy-notice.html'>| Privacy</a></div>


Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -135,7 +135,7 @@ <h2>Submodules<a class="headerlink" href="#submodules" title="Link to this headi
Built with <a href="https://www.sphinx-doc.org/">Sphinx</a> using a
<a href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
provided by <a href="https://readthedocs.org">Read the Docs</a>.
<jinja2.runtime.BlockReference object at 0x7f06bb67eb90>
<jinja2.runtime.BlockReference object at 0x7f95bb985ff0>
<p></p><div><a href='https://www.intel.com/content/www/us/en/privacy/intel-cookie-notice.html' data-cookie-notice='true'>Cookies</a> <a href='https://www.intel.com/content/www/us/en/privacy/intel-privacy-notice.html'>| Privacy</a></div>


Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -125,7 +125,7 @@ <h2>Module Contents<a class="headerlink" href="#module-contents" title="Link to
Built with <a href="https://www.sphinx-doc.org/">Sphinx</a> using a
<a href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
provided by <a href="https://readthedocs.org">Read the Docs</a>.
<jinja2.runtime.BlockReference object at 0x7f06bb7edcc0>
<jinja2.runtime.BlockReference object at 0x7f95c377b4f0>
<p></p><div><a href='https://www.intel.com/content/www/us/en/privacy/intel-cookie-notice.html' data-cookie-notice='true'>Cookies</a> <a href='https://www.intel.com/content/www/us/en/privacy/intel-privacy-notice.html'>| Privacy</a></div>


Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -143,7 +143,7 @@ <h2>Module Contents<a class="headerlink" href="#module-contents" title="Link to
Built with <a href="https://www.sphinx-doc.org/">Sphinx</a> using a
<a href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
provided by <a href="https://readthedocs.org">Read the Docs</a>.
<jinja2.runtime.BlockReference object at 0x7f06bd7af9a0>
<jinja2.runtime.BlockReference object at 0x7f95bb8ac400>
<p></p><div><a href='https://www.intel.com/content/www/us/en/privacy/intel-cookie-notice.html' data-cookie-notice='true'>Cookies</a> <a href='https://www.intel.com/content/www/us/en/privacy/intel-privacy-notice.html'>| Privacy</a></div>


Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -134,7 +134,7 @@ <h2>Module Contents<a class="headerlink" href="#module-contents" title="Link to
Built with <a href="https://www.sphinx-doc.org/">Sphinx</a> using a
<a href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
provided by <a href="https://readthedocs.org">Read the Docs</a>.
<jinja2.runtime.BlockReference object at 0x7f06bb7edbd0>
<jinja2.runtime.BlockReference object at 0x7f95bb8af040>
<p></p><div><a href='https://www.intel.com/content/www/us/en/privacy/intel-cookie-notice.html' data-cookie-notice='true'>Cookies</a> <a href='https://www.intel.com/content/www/us/en/privacy/intel-privacy-notice.html'>| Privacy</a></div>


Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -134,7 +134,7 @@ <h2>Module Contents<a class="headerlink" href="#module-contents" title="Link to
Built with <a href="https://www.sphinx-doc.org/">Sphinx</a> using a
<a href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
provided by <a href="https://readthedocs.org">Read the Docs</a>.
<jinja2.runtime.BlockReference object at 0x7f06bb7ee800>
<jinja2.runtime.BlockReference object at 0x7f95bbbca290>
<p></p><div><a href='https://www.intel.com/content/www/us/en/privacy/intel-cookie-notice.html' data-cookie-notice='true'>Cookies</a> <a href='https://www.intel.com/content/www/us/en/privacy/intel-privacy-notice.html'>| Privacy</a></div>


Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -159,7 +159,7 @@ <h2>Module Contents<a class="headerlink" href="#module-contents" title="Link to
Built with <a href="https://www.sphinx-doc.org/">Sphinx</a> using a
<a href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
provided by <a href="https://readthedocs.org">Read the Docs</a>.
<jinja2.runtime.BlockReference object at 0x7f06c3489000>
<jinja2.runtime.BlockReference object at 0x7f95bbbc8d30>
<p></p><div><a href='https://www.intel.com/content/www/us/en/privacy/intel-cookie-notice.html' data-cookie-notice='true'>Cookies</a> <a href='https://www.intel.com/content/www/us/en/privacy/intel-privacy-notice.html'>| Privacy</a></div>


Expand Down
Loading

0 comments on commit 67b1711

Please sign in to comment.