Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

QNN Module API(new frontend) Preliminary Support #158

Merged
merged 57 commits into from
Oct 26, 2024
Merged

QNN Module API(new frontend) Preliminary Support #158

merged 57 commits into from
Oct 26, 2024

Commits on Sep 7, 2024

  1. feat : support multithread mergeouput copy.

    @liang1232018
    liang1232018 committed Sep 7, 2024
    Configuration menu
    Copy the full SHA
    e0d936e View commit details
    Browse the repository at this point in the history

  2. update : support FP16 in FFN.

    @liang1232018
    liang1232018 committed Sep 7, 2024
    Configuration menu
    Copy the full SHA
    006772d View commit details
    Browse the repository at this point in the history

Commits on Sep 8, 2024

  1. feat : using FP16 KVCache now.

    @liang1232018
    liang1232018 committed Sep 8, 2024
    Configuration menu
    Copy the full SHA
    171d9a2 View commit details
    Browse the repository at this point in the history

  2. feat : support reference SuperSiLU.

    @liang1232018
    liang1232018 committed Sep 8, 2024
    Configuration menu
    Copy the full SHA
    2588ef0 View commit details
    Browse the repository at this point in the history

  3. feat : implement SuperSiLU with bugs.

    @liang1232018
    liang1232018 committed Sep 8, 2024
    Configuration menu
    Copy the full SHA
    abd6afa View commit details
    Browse the repository at this point in the history

  4. fix : silu execution error.

    @liang1232018
    liang1232018 committed Sep 8, 2024
    Configuration menu
    Copy the full SHA
    339f31b View commit details
    Browse the repository at this point in the history

  5. fix : SuperSiLU bugs in scale.

    @liang1232018
    liang1232018 committed Sep 8, 2024
    Configuration menu
    Copy the full SHA
    f247393 View commit details
    Browse the repository at this point in the history

Commits on Sep 9, 2024

  1. fix : SuperSiLU quant scale bug.

    @liang1232018
    liang1232018 committed Sep 9, 2024
    Configuration menu
    Copy the full SHA
    99a0c4b View commit details
    Browse the repository at this point in the history

  2. feat : optimize shadow execution with NEON and multithread.

    @liang1232018
    liang1232018 committed Sep 9, 2024
    Configuration menu
    Copy the full SHA
    6b850fb View commit details
    Browse the repository at this point in the history

Commits on Sep 10, 2024

  1. fix: fp16 kvcache npu no copy

    @oreomaker
    oreomaker committed Sep 10, 2024
    Configuration menu
    Copy the full SHA
    ffe4717 View commit details
    Browse the repository at this point in the history

Commits on Sep 26, 2024

  1. feat: create some tensor funcs' Layers for qnn exe

    @oreomaker
    oreomaker committed Sep 26, 2024
    Configuration menu
    Copy the full SHA
    5ac41a4 View commit details
    Browse the repository at this point in the history

  2. chore: remove unused duplicate qnn linear

    @oreomaker
    oreomaker committed Sep 26, 2024
    Configuration menu
    Copy the full SHA
    5d7153d View commit details
    Browse the repository at this point in the history

  3. fix: qnn cmake macro position

    @oreomaker
    oreomaker committed Sep 26, 2024
    Configuration menu
    Copy the full SHA
    602c807 View commit details
    Browse the repository at this point in the history

Commits on Sep 27, 2024

  1. refactor: qwen qnn new module api modeling

    @oreomaker
    oreomaker committed Sep 27, 2024
    Configuration menu
    Copy the full SHA
    4686b91 View commit details
    Browse the repository at this point in the history

Commits on Sep 28, 2024

  1. fix: demo qnn weights name 

    feat: tensor.to() simple version
    @oreomaker
    oreomaker committed Sep 28, 2024
    Configuration menu
    Copy the full SHA
    357a4e0 View commit details
    Browse the repository at this point in the history

Commits on Oct 5, 2024

  1. fix: qnn tensor dtype population in module api 

    refactor: add module previous backend
todo: cross backned tensor, qnn activation alloc
    @oreomaker
    oreomaker committed Oct 5, 2024
    Configuration menu
    Copy the full SHA
    887e5f9 View commit details
    Browse the repository at this point in the history

Commits on Oct 6, 2024

  1. refator: qnn transpose and kvcache channel 

    feat: qnn graph build using Module API
    @oreomaker
    oreomaker committed Oct 6, 2024
    Configuration menu
    Copy the full SHA
    6384e4b View commit details
    Browse the repository at this point in the history

Commits on Oct 8, 2024

  1. Merge pull request #3 from liang1232018/develop-xdl-dev 

    Develop xdl dev
    @liang1232018
    liang1232018 authored Oct 8, 2024
    Configuration menu
    Copy the full SHA
    050ffcb View commit details
    Browse the repository at this point in the history

Commits on Oct 9, 2024

  1. feat : support int8 output RMSNorm.

    @liang1232018
    liang1232018 committed Oct 9, 2024
    Configuration menu
    Copy the full SHA
    31d6243 View commit details
    Browse the repository at this point in the history

Commits on Oct 10, 2024

  1. feat : optimize RMSNorm intermediate type.

    @liang1232018
    liang1232018 committed Oct 10, 2024
    Configuration menu
    Copy the full SHA
    6bf38c2 View commit details
    Browse the repository at this point in the history

  2. fix : res input tensor type error.

    @liang1232018
    liang1232018 committed Oct 10, 2024
    Configuration menu
    Copy the full SHA
    674a1b8 View commit details
    Browse the repository at this point in the history

Commits on Oct 11, 2024

  1. fix: qnn module api tensor alloc and dtype set

    @oreomaker
    oreomaker committed Oct 11, 2024
    Configuration menu
    Copy the full SHA
    2b6cb3d View commit details
    Browse the repository at this point in the history

  2. Merge branch 'develop-zh-alpha' of github.com:liang1232018/mllm into … 

    …develop-zh-alpha
    @oreomaker
    oreomaker committed Oct 11, 2024
    Configuration menu
    Copy the full SHA
    ceebd84 View commit details
    Browse the repository at this point in the history

  3. Merge branch 'main' into develop-zh

    @oreomaker
    oreomaker committed Oct 11, 2024
    Configuration menu
    Copy the full SHA
    6063c0a View commit details
    Browse the repository at this point in the history

  4. chore: qnn cmake execcutables

    @oreomaker
    oreomaker committed Oct 11, 2024
    Configuration menu
    Copy the full SHA
    ac16abd View commit details
    Browse the repository at this point in the history

  5. feat : optimize quantization implementation in RMSNorm.

    @liang1232018
    liang1232018 committed Oct 11, 2024
    Configuration menu
    Copy the full SHA
    c771e14 View commit details
    Browse the repository at this point in the history

  6. feat : optimize quantize and supersilu sf to int8 operation.

    @liang1232018
    liang1232018 committed Oct 11, 2024
    Configuration menu
    Copy the full SHA
    111437a View commit details
    Browse the repository at this point in the history

Commits on Oct 13, 2024

  1. feat : optimize SiLU execution without dequantize with uint8 mul.

    @liang1232018
    liang1232018 committed Oct 13, 2024
    Configuration menu
    Copy the full SHA
    f4ff8db View commit details
    Browse the repository at this point in the history

Commits on Oct 14, 2024

  1. feat : optimize SiLU implementation without quantization.

    @liang1232018
    liang1232018 committed Oct 14, 2024
    Configuration menu
    Copy the full SHA
    6f6c12e View commit details
    Browse the repository at this point in the history

Commits on Oct 15, 2024

  1. fix: qnn new front end scale setting 

    refactor: module device backend setup & execute
    @oreomaker
    oreomaker committed Oct 15, 2024
    Configuration menu
    Copy the full SHA
    8c92c65 View commit details
    Browse the repository at this point in the history

  2. refactor: remove unecessary qnn backend multi models and graphs

    @oreomaker
    oreomaker committed Oct 15, 2024
    Configuration menu
    Copy the full SHA
    e1c0997 View commit details
    Browse the repository at this point in the history

  3. feat : optimize RMSNorm implementation.

    @liang1232018
    liang1232018 committed Oct 15, 2024
    Configuration menu
    Copy the full SHA
    382a5f4 View commit details
    Browse the repository at this point in the history

  4. fix : use old accurate RMSNorm execution method.

    @liang1232018
    liang1232018 committed Oct 15, 2024
    Configuration menu
    Copy the full SHA
    21ea6a5 View commit details
    Browse the repository at this point in the history

Commits on Oct 16, 2024

  1. fix : shadow output outlier execution bugs and optimize the input out… 

    …lier execution performance.
    @liang1232018
    liang1232018 committed Oct 16, 2024
    Configuration menu
    Copy the full SHA
    18f0dda View commit details
    Browse the repository at this point in the history

  2. Merge pull request #4 from liang1232018/develop-xdl-accuracy 

    fix : shadow output outlier execution bugs and optimize the input out…
    @liang1232018
    liang1232018 authored Oct 16, 2024
    Configuration menu
    Copy the full SHA
    4657817 View commit details
    Browse the repository at this point in the history

  3. fix : wrong modification of CPUKVCache.

    @liang1232018
    liang1232018 committed Oct 16, 2024
    Configuration menu
    Copy the full SHA
    cf96340 View commit details
    Browse the repository at this point in the history

Commits on Oct 17, 2024

  1. feat: qnn new frontend prefill

    @oreomaker
    oreomaker committed Oct 17, 2024
    Configuration menu
    Copy the full SHA
    ed9abe3 View commit details
    Browse the repository at this point in the history

  2. fix : QNN backend with no openmp.

    @liang1232018
    liang1232018 committed Oct 17, 2024
    Configuration menu
    Copy the full SHA
    9eff6aa View commit details
    Browse the repository at this point in the history

Commits on Oct 18, 2024

  1. feat: qnn new frontend kvcache sharing

    @oreomaker
    oreomaker committed Oct 18, 2024
    Configuration menu
    Copy the full SHA
    7081416 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'develop-xdl' into develop-zh

    @oreomaker
    oreomaker committed Oct 18, 2024
    Configuration menu
    Copy the full SHA
    5b446e5 View commit details
    Browse the repository at this point in the history

  3. Merge branch 'main' into develop-zh

    @yirongjie
    yirongjie authored Oct 18, 2024
    Configuration menu
    Copy the full SHA
    fe36e69 View commit details
    Browse the repository at this point in the history

  4. Merge branch 'main' into develop-zh

    @yirongjie
    yirongjie authored Oct 18, 2024
    Configuration menu
    Copy the full SHA
    d778f72 View commit details
    Browse the repository at this point in the history

  5. fix: bugs in cmake and layer.hpp

    @yirongjie
    yirongjie committed Oct 18, 2024
    Configuration menu
    Copy the full SHA
    5c614c5 View commit details
    Browse the repository at this point in the history

  6. fix: add ign

    @yirongjie
    yirongjie committed Oct 18, 2024
    Configuration menu
    Copy the full SHA
    d7d68f3 View commit details
    Browse the repository at this point in the history

Commits on Oct 19, 2024

  1. Merge branch 'develop-zh' of github.com:liang1232018/mllm into develo… 

    …p-zh
    @oreomaker
    oreomaker committed Oct 19, 2024
    Configuration menu
    Copy the full SHA
    0805d64 View commit details
    Browse the repository at this point in the history

  2. fix: qnn cmake executables

    @oreomaker
    oreomaker committed Oct 19, 2024
    Configuration menu
    Copy the full SHA
    dd4c744 View commit details
    Browse the repository at this point in the history

  3. fix: demo qnn padding tockenize

    @oreomaker
    oreomaker committed Oct 19, 2024
    Configuration menu
    Copy the full SHA
    f351610 View commit details
    Browse the repository at this point in the history

Commits on Oct 21, 2024

  1. Merge branch 'main' into develop-zh

    @yirongjie
    yirongjie authored Oct 21, 2024
    Configuration menu
    Copy the full SHA
    c16091b View commit details
    Browse the repository at this point in the history

  2. fix: cmake

    @yirongjie
    yirongjie committed Oct 21, 2024
    Configuration menu
    Copy the full SHA
    6d79918 View commit details
    Browse the repository at this point in the history

Commits on Oct 24, 2024

  1. Merge branch 'main' into pr/oreomaker/158

    @yirongjie
    yirongjie committed Oct 24, 2024
    Configuration menu
    Copy the full SHA
    d7e9def View commit details
    Browse the repository at this point in the history

  2. Merge branch 'main' into develop-zh

    @UbiquitousLearning
    UbiquitousLearning authored Oct 24, 2024
    Configuration menu
    Copy the full SHA
    6ef60c5 View commit details
    Browse the repository at this point in the history

  3. fix: change module loader to global

    @oreomaker
    oreomaker committed Oct 24, 2024
    Configuration menu
    Copy the full SHA
    b93e504 View commit details
    Browse the repository at this point in the history

Commits on Oct 25, 2024

  1. Merge branch 'develop-zh' of github.com:liang1232018/mllm into develo… 

    …p-zh
    @oreomaker
    oreomaker committed Oct 25, 2024
    Configuration menu
    Copy the full SHA
    6eea3bd View commit details
    Browse the repository at this point in the history

Commits on Oct 26, 2024

  1. Merge branch 'main' into develop-zh

    @UbiquitousLearning
    UbiquitousLearning authored Oct 26, 2024
    Configuration menu
    Copy the full SHA
    2388adb View commit details
    Browse the repository at this point in the history

  2. fix: qnn qwen new frontend modeling

    @oreomaker
    oreomaker committed Oct 26, 2024
    Configuration menu
    Copy the full SHA
    ea065d0 View commit details
    Browse the repository at this point in the history

  3. Merge branch 'develop-zh' of github.com:liang1232018/mllm into develo… 

    …p-zh
    @oreomaker
    oreomaker committed Oct 26, 2024
    Configuration menu
    Copy the full SHA
    c182fbc View commit details
    Browse the repository at this point in the history

  4. fix: cmake

    @yirongjie
    yirongjie committed Oct 26, 2024
    Configuration menu
    Copy the full SHA
    c9f732b View commit details
    Browse the repository at this point in the history