Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add error handling to skip alt data files that don't exist #7

Open
regnans opened this issue Jan 8, 2021 · 2 comments
Open

Add error handling to skip alt data files that don't exist #7

regnans opened this issue Jan 8, 2021 · 2 comments
Labels
bug Something isn't working

Comments

@regnans
Copy link

regnans commented Jan 8, 2021

2021-01-08 12:12:20,944 [INFO]: [Chamber 1 - Blue_Min1.dat] Checking for duplicates in ['TIMESTAMP', 'RECORD']...
2021-01-08 12:12:21,016 [INFO]: [Chamber 1 - Blue_Min1.dat] Found 8216 duplicates
2021-01-08 12:12:21,250 [INFO]: [Chamber 1 - Blue_Min1.dat] Dropped duplicates, kept last.
2021-01-08 12:12:21,250 [INFO]: [Chamber 1 - Blue_Min1.dat] Checking for duplicates in TIMESTAMP...
2021-01-08 12:12:21,281 [INFO]: [Chamber 1 - Blue_Min1.dat] No duplicates found.
2021-01-08 12:12:21,281 [INFO]: [Chamber 1 - Blue_Min1.dat] Checking record order...
2021-01-08 12:12:21,281 [INFO]: [Chamber 1 - Blue_Min1.dat] No out of order timestamps found.
2021-01-08 12:12:21,281 [INFO]: [Chamber 1 - Blue_Min1.dat] Checking for missings rows...
2021-01-08 12:12:21,547 [INFO]: [Chamber 1 - Blue_Min1.dat] Filled 384815 missing rows
2021-01-08 12:12:52,828 [INFO]: [Chamber 1 - Blue_Min1.dat] Loading ../Raw\Chamber 1 - Blue_Min1_alt.dat...
2021-01-08 12:12:52,828 [WARNING]: [Chamber 1 - Blue_Min1_alt.dat] Problem while loading Chamber 1 - Blue_Min1_alt.dat: [Errno 2] No such file or directory: '../Raw\Chamber 1 - Blue_Min1_alt.dat'
2021-01-08 12:12:52,828 [INFO]: [Chamber 1 - Blue_Min1_alt.dat] Checking for duplicates in TIMESTAMP...
2021-01-08 12:12:52,966 [INFO]: [Chamber 1 - Blue_Min1_alt.dat] No duplicates found.
2021-01-08 12:12:52,969 [INFO]: [Chamber 1 - Blue_Min1_alt.dat] Checking record order...
Traceback (most recent call last):
File "T:\Projects\NGEE-Arctic\ZPW\2019\Data\QA\Level0to1.py", line 359, in
d_dev = calcAltDevs(d, fname, **opts["Level 1"], **opts["Input"])
File "T:\Projects\NGEE-Arctic\ZPW\2019\Data\QA\Level0to1.py", line 211, in calcAltDevs
checkOrder(d_alt)
File "T:\Projects\NGEE-Arctic\ZPW\2019\Data\QA\Level0to1.py", line 184, in checkOrder
if df[ts].is_monotonic:
TypeError: 'NoneType' object is not subscriptable[
Level0to1_2021-01-08_121217.log
](url)

@amcmahon01
Copy link
Member

There is a altCheckEnable flag in the config (line 61) that you can set to False if there isn't any alternate data, which will prevent this, though the error handling should be improved to skip trying to process a file that doesn't exist. It does this already for different fields within a file but it appears that logic never made it to the top (file) level. I'm renaming this issue to make it into a "to do".

@amcmahon01 amcmahon01 changed the title script failed when looking for alt data (which doesn't exist). py.log attached. Add error handling to skip alt data files that don't exist Jan 10, 2021
@amcmahon01 amcmahon01 added the bug Something isn't working label Jan 10, 2021
@regnans
Copy link
Author

regnans commented Jan 11, 2021

Thanks! Glad that was an easy fix. Am running it again now. Looks like it is working, at least the alt data issue is fixed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants