Skip to content

Add profiling and throughput reporting to the benchmarks - #879

Open
alexander-beedie wants to merge 1 commit into
andialbrecht:masterfrom
alexander-beedie:bench/profiling
Open

Add profiling and throughput reporting to the benchmarks#879
alexander-beedie wants to merge 1 commit into
andialbrecht:masterfrom
alexander-beedie:bench/profiling

Conversation

@alexander-beedie

@alexander-beedie alexander-beedie commented Aug 21, 2026

Copy link
Copy Markdown
Contributor

While looking for opportunities to improve performance, I extended the benchmarking script with additional profiling and throughput info/options 👍

Three main additions:

  • A new kB/s stat/column

    Scaling benchmarks only catch super-linear growth; a uniform slowdown would leave every exponent exactly where it was, bit would show up here.

    $ python benchmarks/bench_parse_throughput.py --vector "UNION ALL"
    
    UNION ALL chain:
             n      input         time    ratio     kB/s  status
           100     9885 B      32.2 ms        -      307  ok
           200    20.0 kB      62.3 ms    1.94x      321  ok
           400    40.2 kB     136.2 ms    2.19x      295  ok
           800    80.7 kB     317.4 ms    2.33x      254  ok
      scaling exponent 1.17 (1.0 linear, 2.0 quadratic)  =>  linear
    
    verdict: linear  (1 vector, 0 super-linear)
    
  • --profile

    Runs each vector once (at its largest size) and print the hottest frames by tottime

    $ python benchmarks/bench_parse_throughput.py --vector "UNION ALL" --profile --profile-top 6
    
    UNION ALL chain:  n=800, 80.7 kB
      ncalls  tottime  percall  cumtime  percall filename:lineno(function)
         680760    0.067    0.000    0.067    0.000 {method 'match' of 're.Pattern' objects}
         323171    0.061    0.000    0.125    0.000 sqlparse/utils.py:81(imt)
       47211/11    0.060    0.000    0.160    0.015 sqlparse/engine/grouping.py:478(_group)
          30398    0.056    0.000    0.140    0.000 sqlparse/lexer.py:106(get_tokens)
          60009    0.049    0.000    0.188    0.000 sqlparse/sql.py:235(_token_matching)
            800    0.043    0.000    0.083    0.000 sqlparse/sql.py:310(token_index)
    
  • bench_parse_throughput.py

    Covers a selection of large/realistic SQL not found in the other scripts: CTEs, joins, window functions, UNION ALL chains, multi-statement scripts, comment-heavy text.


Found several nice speedups using the new flags, for which I'll make separate PRs shortly.

@codecov

codecov Bot commented Aug 21, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 97.20%. Comparing base (111b35c) to head (1cd995b).
⚠️ Report is 25 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff             @@
##           master     #879      +/-   ##
==========================================
+ Coverage   97.13%   97.20%   +0.07%     
==========================================
  Files          31       31              
  Lines        3663     3793     +130     
  Branches      328      344      +16     
==========================================
+ Hits         3558     3687     +129     
- Misses         63       65       +2     
+ Partials       42       41       -1     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant