Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Problem writing the T-file #346

Open
femeunier opened this issue Mar 14, 2022 · 12 comments
Open

Problem writing the T-file #346

femeunier opened this issue Mar 14, 2022 · 12 comments

Comments

@femeunier
Copy link

femeunier commented Mar 14, 2022 •
edited
Loading

Hi all,

I have an issue running a simulation over an African tropical site. Clearly, the issue is related to the writing of the T-file, see error message below. Has anyone successfully produced T-output files recently? Could it be related with the latest version of ED2.2?

Here is my ED2IN file: ED2IN_YGB_tower.txt

Total count in node        1 for grid        1 : POLYGONS=        1 SITES=        1 PATCHES=       26 COHORTS=      565
HDF5-DIAG: Error detected in HDF5 (1.12.1) thread 0:
  #000: H5Dio.c line 291 in H5Dwrite(): can't write data
    major: Dataset
    minor: Write failed
  #001: H5VLcallback.c line 2113 in H5VL_dataset_write(): dataset write failed
    major: Virtual Object Layer
    minor: Write failed
  #002: H5VLcallback.c line 2080 in H5VL__dataset_write(): dataset write failed
    major: Virtual Object Layer
    minor: Write failed
  #003: H5VLnative_dataset.c line 200 in H5VL__native_dataset_write(): could not get a validated dataspace from file_space_id
    major: Invalid arguments to routine
    minor: Bad value
  #004: H5S.c line 266 in H5S_get_validated_dataspace(): selection + offset not within extent
    major: Dataspace
    minor: Out of range
 Variable name:    
 SFCWATER_DEPTH                                                  
 Global dimension:                      1                    26
                 17568                     0                     0
                     0
 Chunk dimension:                       1                    26
                     1                     0                     0
                     0
 Chunk offset:                          0                     0
                 17556                     0                     0
                     0
 Count:                                 1                     1
                     1                     0                     0
                     0
 Stride:                                1                     1
                     1                     0                     0
                     0
::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
 
--------------------------------------------------------------
                     !!! FATAL ERROR !!!                      
--------------------------------------------------------------
    ---> File:        h5_output.F90
    ---> Subroutine:  h5_output
    ---> Reason:      Could not write hyperslab into the dataset
--------------------------------------------------------------
 ED execution halts (see previous error message)...
--------------------------------------------------------------
fatal_error

Could it be related to those lines in ed_state_vars file?

      if (associated(csite%sfcwater_depth)) then
         nvar=nvar+1
           call vtable_edio_r(npts,csite%sfcwater_depth,nvar,igr,init,csite%paglob_id, &
           var_len,var_len_global,max_ptrs,'SFCWATER_DEPTH :33:hist:opti') 
         call metadata_edio(nvar,igr,'No metadata available','[NA]','m') 
      end if

Why is this specific state var also printed in the opti file? Is it really necessary?
@femeunier
Copy link
Author

femeunier commented Mar 14, 2022 •
edited
Loading

I confirm that changing those lines to

      if (associated(csite%sfcwater_depth)) then
         nvar=nvar+1
           call vtable_edio_r(npts,csite%sfcwater_depth,nvar,igr,init,csite%paglob_id, &
           var_len,var_len_global,max_ptrs,'SFCWATER_DEPTH :33:hist') 
         call metadata_edio(nvar,igr,'No metadata available','[NA]','m') 
      end if

solves the issue ... Does it make sense to simply remove that output from the T-file or does anyone need it?

@pmoorcroft
Copy link

pmoorcroft commented Mar 14, 2022 via email

@mpaiao
Copy link
Contributor

mpaiao commented Mar 15, 2022

@pmoorcroft I'm tagging @mrjohnston because I am not sure cc'ing works with GitHub emails.

@mpaiao
Copy link
Contributor

mpaiao commented Mar 15, 2022

@femeunier I am not familiar with -T- files, do they store more than one time per file?
If so, then this explains why sfcwater_depth is causing the crash. It seems this is the only "opti" variable that is not a polygon-level variable, and the number of patches may be changing within the same -T- file. If this is the case, then this is a bug in the code, would you mind creating a quick pull request with your fix?

@pmoorcroft
Copy link

pmoorcroft commented Mar 15, 2022 via email

@mrjohnston
Copy link
Contributor

@femeunier @pmoorcroft @mpaiao I'm very sorry to disappoint, but I haven't been using the T-files & I can't be helpful here 👎 .

...except to say that tagging and cc'ing both work!

@xiangtaoxu
Copy link
Contributor

@mrjohnston uses -I- files (Instantaneous). -T- is for tower flux data comparison?

@pmoorcroft
Copy link

pmoorcroft commented Mar 15, 2022 via email

@femeunier
Copy link
Author

@femeunier I am not familiar with -T- files, do they store more than one time per file? If so, then this explains why sfcwater_depth is causing the crash. It seems this is the only "opti" variable that is not a polygon-level variable, and the number of patches may be changing within the same -T- file. If this is the case, then this is a bug in the code, would you mind creating a quick pull request with your fix?

T files are indeed annual outputs files with fine timescale state variables (to ease comparison with fluxtower data). So including sfcwater_depth is definitely a bug. I will open a pull request to fix it.

@mdietze
Copy link
Contributor

mdietze commented Mar 15, 2022

@femeunier why are you assuming the inclusion of sfcwater_depth in the T files is a bug? I'm pretty sure I put it there deliberately from the start (though "blame" doesn't work for a file this big, so I can't be sure). Snow depth changes very dynamically and without both snow and soil moisture you can't track the fast-time changes in the water cycle. It has also worked successfully for ~15 years -- no idea what's changed that is now causing an error.

@mpaiao
Copy link
Contributor

mpaiao commented Mar 15, 2022

There is a polygon-level surface water (or snow) depth that is updated frequently: FMEAN_SFCW_DEPTH_PY, and from looking at ed_state_vars.F90, this variable is included in the -T- files.

Having patch-level variables in the -T- files may cause the model to crash if the patch dynamics step doesn't happen in January (this is controlled by NL%MONTH_YRSTEP in ED2IN). I added this option 5-6 years ago to compare the model output with a tower site in the Amazon that had selective logging within the tower footprint in September. Observed fluxes responded immediately to the disturbance and I wanted the model to do the same. This option can be useful for sites that burn frequently too.

If we really want patch-level variables in the -T- files, then we should add code to ed_opspec.F90 to ensure that NL%MONTH_YRSTEP = 1 when NL%ITOUTPUT = 3. But in this case, I'd say that the code should write FMEAN_SFCW_DEPTH_PA, which is the frequent average at patch level. SFCWATER_DEPTH is the instantaneous variable, which should be used for history restarts only.

@mdietze
Copy link
Contributor

mdietze commented Mar 15, 2022

Thanks for the clarification @mpaiao! I agree that patch-level snow depth doesn't need to go in the -T- files if polygon-level is already there.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
6 participants
@xiangtaoxu @mdietze @mpaiao @pmoorcroft @femeunier @mrjohnston