Post March version of Neil's files that (I think) incorporate Steve's changes

[eldobbins]

I think what happened in 2024 is that Neil incorporated Steve's changes to his workbooks. There are some other changes too. Jesse stated that the changes were applied to the aggregate tables and not to the raw/monthly/high-resolution data (@jikaczmarski please clarify this statement if I got it wrong.

Assumption: This is the last update we receive from Neil. It should be documented properly and incorporated into this repository to show the value of his work.

Note: CSV were generated from the previous version of the workbooks and corrections were applied to those. So that represents a separate branch of data that must be resolved with this branch.

Note: documenting changes will be difficult. Ideas to try

• checksums to see if the contents of files changed. Won't say what exactly changed, but will identify files that didn't change
• Python scripts to go through tabs and compare data as data frames. Note: this will not work if the format of the data changed. Well, the amount of effort it will require will be gross.
• regenerate the CSV files (using Python scripts) and diff them. Don't copy by hand again. Too tedious

to do

• Find the files. There should be 2 deliveries - the yearly aggregate tables and the 3 big workbooks
• checkout the dev branch and put these files into place in that branch
• Document changes to the files and put them in the CHANGE_LOG file
• When all is documented sufficiently, merge the branch
• Tag this version

[eldobbins]

Addressed by PR #22

[eldobbins]

Never mind these issues in interest of getting this done

• pull out the data dictionary that Neil wrote (tab in one of the files, but which?) If it doesn't take too long, translated that to markdown or text for ease of reading. No need for machine readable version.
• pull out intertie lookup table so that can be highlighted separately
• Document the checksums, what the hell.

[jikaczmarski]

Jesse stated that the changes were applied to the aggregate tables and not to the raw/monthly/high-resolution data (@jikaczmarski please clarify this statement if I got it wrong.

That is a correct statement.

[ianalexmac]

checksums to see if the contents of files changed. Won't say what exactly changed, but will identify files that didn't change

#23 explores using a tarball and hash to check directory contents. See issue for more details.