Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Some micro-optimizations for tf2. #94

Open
wants to merge 41 commits into
base: master
Choose a base branch
from
Open

Some micro-optimizations for tf2. #94

wants to merge 41 commits into from

Commits on Dec 7, 2019

  1. Silence some warnings in 1.14 at the expense of making 1.14 the minim… 

    …um version
    @Tilps
    Tilps committed Dec 7, 2019
    Configuration menu
    Copy the full SHA
    6c63629 View commit details
    Browse the repository at this point in the history

  2. Merge remote-tracking branch 'upstream/master' into migrate_1.14

    @Tilps
    Tilps committed Dec 7, 2019
    Configuration menu
    Copy the full SHA
    be869dd View commit details
    Browse the repository at this point in the history

  3. Update requirements.

    @Tilps
    Tilps committed Dec 7, 2019
    Configuration menu
    Copy the full SHA
    a9f7129 View commit details
    Browse the repository at this point in the history

  4. Minimal changes to get training running on 2.0

    @Tilps
    Tilps committed Dec 7, 2019
    Configuration menu
    Copy the full SHA
    7ad15ce View commit details
    Browse the repository at this point in the history

  5. Update requirements.

    @Tilps
    Tilps committed Dec 7, 2019
    Configuration menu
    Copy the full SHA
    d170e20 View commit details
    Browse the repository at this point in the history

  6. Remove uses of get_variable since its initializer don't seem to be ca… 

    …lled anymore...
    @Tilps
    Tilps committed Dec 7, 2019
    Configuration menu
    Copy the full SHA
    3fcbe2b View commit details
    Browse the repository at this point in the history

  7. Fix typo.

    @Tilps
    Tilps committed Dec 7, 2019
    Configuration menu
    Copy the full SHA
    162fa40 View commit details
    Browse the repository at this point in the history

Commits on Dec 8, 2019

  1. Migration in progress some more. 

    This point 'basic' training seems to be functional, maybe.
No test data processing, no grad norm clipping, no swa, no metrics, no saving to pb.gz - lots still to do.
    @Tilps
    Tilps committed Dec 8, 2019
    Configuration menu
    Copy the full SHA
    ff2e256 View commit details
    Browse the repository at this point in the history

  2. Add grad norm clipping back in.

    @Tilps
    Tilps committed Dec 8, 2019
    Configuration menu
    Copy the full SHA
    f202c78 View commit details
    Browse the repository at this point in the history

  3. Add swa support and test result reporting.

    @Tilps
    Tilps committed Dec 8, 2019
    Configuration menu
    Copy the full SHA
    1398d2a View commit details
    Browse the repository at this point in the history

  4. Actually sum the gradients...

    @Tilps
    Tilps committed Dec 8, 2019
    Configuration menu
    Copy the full SHA
    2d97b80 View commit details
    Browse the repository at this point in the history

  5. Fix bad bug with residual blocks, also include the stub for the incom… 

    …plete save code.
    @Tilps
    Tilps committed Dec 8, 2019
    Configuration menu
    Copy the full SHA
    cd3c69d View commit details
    Browse the repository at this point in the history

  6. Constant policy map shouldn't be a variable, or it gets saved.

    @Tilps
    Tilps committed Dec 8, 2019
    Configuration menu
    Copy the full SHA
    d3f95eb View commit details
    Browse the repository at this point in the history

Commits on Dec 9, 2019

  1. Add untested network saving 

    no idea if its output is the right format yet.
    @Tilps
    Tilps committed Dec 9, 2019
    Configuration menu
    Copy the full SHA
    c90a9c4 View commit details
    Browse the repository at this point in the history

  2. Fix permutation which didn't take into account the inserted gammas.

    @Tilps
    Tilps committed Dec 9, 2019
    Configuration menu
    Copy the full SHA
    61158e2 View commit details
    Browse the repository at this point in the history

  3. GlobalAveragePooling2D needs to know the data format.

    @Tilps
    Tilps committed Dec 9, 2019
    Configuration menu
    Copy the full SHA
    8681df6 View commit details
    Browse the repository at this point in the history

  4. Use tf.function on the inner loop of training for massive speed up

    @Tilps
    Tilps committed Dec 9, 2019
    Configuration menu
    Copy the full SHA
    215445d View commit details
    Browse the repository at this point in the history

  5. Extract test summary inner loop to tf.function for a bit more perform… 

    …ance
    @Tilps
    Tilps committed Dec 9, 2019
    Configuration menu
    Copy the full SHA
    bdec903 View commit details
    Browse the repository at this point in the history

Commits on Dec 11, 2019

  1. Basic tensorboard summary data writting and some other cleanup relate… 

    …d to saving.
    @Tilps
    Tilps committed Dec 11, 2019
    Configuration menu
    Copy the full SHA
    0b17356 View commit details
    Browse the repository at this point in the history

  2. Remove some v1 code which I've done a second pass checking conversion.

    @Tilps
    Tilps committed Dec 11, 2019
    Configuration menu
    Copy the full SHA
    43cdaf2 View commit details
    Browse the repository at this point in the history

  3. Add a missing _v2.

    @Tilps
    Tilps committed Dec 11, 2019
    Configuration menu
    Copy the full SHA
    53c6057 View commit details
    Browse the repository at this point in the history

  4. Fix bug in saving swa_weights that I hadn't tested before committing...

    @Tilps
    Tilps committed Dec 11, 2019
    Configuration menu
    Copy the full SHA
    6892b1c View commit details
    Browse the repository at this point in the history

  5. Add mixed precision support to tf2 version.

    @Tilps
    Tilps committed Dec 11, 2019
    Configuration menu
    Copy the full SHA
    c610936 View commit details
    Browse the repository at this point in the history

  6. Renorm also doesn't actually support fused.

    @Tilps
    Tilps committed Dec 11, 2019
    Configuration menu
    Copy the full SHA
    5a2a95b View commit details
    Browse the repository at this point in the history

  7. Re-add accuracy reporting.

    @Tilps
    Tilps committed Dec 11, 2019
    Configuration menu
    Copy the full SHA
    6a37e9f View commit details
    Browse the repository at this point in the history

  8. Some more cleanup of pre-v2 code that is converted or close enough to… 

    … no longer be useful, and a fix for swa summaries being written to the wrong file.
    @Tilps
    Tilps committed Dec 11, 2019
    Configuration menu
    Copy the full SHA
    f2f32a2 View commit details
    Browse the repository at this point in the history

  9. Readd basic update ratios.

    @Tilps
    Tilps committed Dec 11, 2019
    Configuration menu
    Copy the full SHA
    7cbe775 View commit details
    Browse the repository at this point in the history

  10. Small performance optimization to offset the cost of update ratios.

    @Tilps
    Tilps committed Dec 11, 2019
    Configuration menu
    Copy the full SHA
    47208df View commit details
    Browse the repository at this point in the history

  11. Optimize compute update ratio since adding update_ratio_log10 had a n… 

    …oticeable performance impact.
    @Tilps
    Tilps committed Dec 11, 2019
    Configuration menu
    Copy the full SHA
    d22b9b8 View commit details
    Browse the repository at this point in the history

Commits on Dec 12, 2019

  1. Some more cleanup, add weight histograms back.

    @Tilps
    Tilps committed Dec 12, 2019
    Configuration menu
    Copy the full SHA
    ef06365 View commit details
    Browse the repository at this point in the history

  2. Fix net saving for renorm mode.

    @Tilps
    Tilps committed Dec 12, 2019
    Configuration menu
    Copy the full SHA
    753b918 View commit details
    Browse the repository at this point in the history

  3. Add net_to_model and update_steps support. Remove more unsupported co… 

    …de, including upgrade.py which doesn't have an obvious transition.
    @Tilps
    Tilps committed Dec 12, 2019
    Configuration menu
    Copy the full SHA
    72f7cb3 View commit details
    Browse the repository at this point in the history

  4. Some more cleanup.

    @Tilps
    Tilps committed Dec 12, 2019
    Configuration menu
    Copy the full SHA
    9f7defc View commit details
    Browse the repository at this point in the history

  5. More cleanup.

    @Tilps
    Tilps committed Dec 12, 2019
    Configuration menu
    Copy the full SHA
    7823f0d View commit details
    Browse the repository at this point in the history

  6. Small fix needed to make script work in ubuntu.

    @Tilps
    Tilps committed Dec 12, 2019
    Configuration menu
    Copy the full SHA
    5f4d8c4 View commit details
    Browse the repository at this point in the history

  7. explicitly add the training flags to model calls - as apparently needed.

    @Tilps
    Tilps committed Dec 12, 2019
    Configuration menu
    Copy the full SHA
    da94d76 View commit details
    Browse the repository at this point in the history

  8. More cleanup.

    @Tilps
    Tilps committed Dec 12, 2019
    Configuration menu
    Copy the full SHA
    20fe10a View commit details
    Browse the repository at this point in the history

Commits on Dec 13, 2019

  1. Pull iterator advancement out of the tf.function loops, its not suppo… 

    …sed to work there.

Even if it was working just fine... this seems slightly faster.
    @Tilps
    Tilps committed Dec 13, 2019
    Configuration menu
    Copy the full SHA
    3ac88fa View commit details
    Browse the repository at this point in the history

  2. Add experimental dataset loader.

    @Tilps
    Tilps committed Dec 13, 2019
    Configuration menu
    Copy the full SHA
    681ab74 View commit details
    Browse the repository at this point in the history

  3. Some micro-optimizations.

    @Tilps
    Tilps committed Dec 13, 2019
    Configuration menu
    Copy the full SHA
    121545f View commit details
    Browse the repository at this point in the history

  4. Merge branch 'master' into migrate_2.0_part2

    @Tilps
    Tilps committed Dec 13, 2019
    Configuration menu
    Copy the full SHA
    b016a64 View commit details
    Browse the repository at this point in the history