Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update tsfresh.feature_extraction.feature_calculators.skewness to make it consistent with the design principle of not ignoring nan #1051

Closed
wants to merge 3 commits into from

Conversation

yitao-yu
Copy link

@yitao-yu yitao-yu commented Oct 6, 2023

The current implementation of tsfresh.feature_extraction.feature_calculators.skewness utilizes a function from pandas that ignores nan by default, which is not consistent with the performance of other functions and the design principle.

This update provides a fix by adding a skipna=False argument. But since it's discovered by chance, I do suggest a review of other parts of the code.

More

Copy link
Collaborator

@nils-braun nils-braun left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @yitao-yu - you are completely correct!
It would be nice to also check all other calculators. Do you have an idea on how to do this automatically?

@codecov-commenter
Copy link

Codecov Report

All modified lines are covered by tests ✅

Comparison is base (f3a6a7c) 93.43% compared to head (1ab70aa) 93.43%.
Report is 1 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #1051   +/-   ##
=======================================
  Coverage   93.43%   93.43%           
=======================================
  Files          20       20           
  Lines        1888     1888           
  Branches      372      372           
=======================================
  Hits         1764     1764           
  Misses         85       85           
  Partials       39       39           
Files Coverage Δ
tsfresh/feature_extraction/feature_calculators.py 92.46% <100.00%> (ø)

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@yitao-yu
Copy link
Author

yitao-yu commented Oct 17, 2023

Thanks @yitao-yu - you are completely correct! It would be nice to also check all other calculators. Do you have an idea on how to do this automatically?

So after much closer looks, I believe that the package would throw an error(there is a _check_nan function called in the WideTsFrameAdapter class) if you are utilizing tsfresh.feature_extraction.feature_calculators.

For individual features, maybe some sort of testing program can be written to check if they have similar problems. I'm not really sure how to do that.

Or an easier alternative is that I can just write programs to look for features that are calling functions that are implemented under pandas.Series and also has an optional argument skipna.

@nils-braun
Copy link
Collaborator

Sorry for the long pause @yitao-yu - and thanks for looking into it. I think for now, including your change for this function is a good start. Could you fix the code style issue? The we can merge!

@nils-braun nils-braun enabled auto-merge (squash) October 24, 2023 19:58
@nils-braun nils-braun mentioned this pull request Mar 11, 2024
@nils-braun
Copy link
Collaborator

Merged in #1066

@nils-braun nils-braun closed this Mar 11, 2024
auto-merge was automatically disabled March 11, 2024 18:51

Pull request was closed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants