SQLalchemy Integrity Error when running msnoise new_jobs #354
Answered
by
zibiwilczynski
zibiwilczynski
asked this question in
Q&A
-
|
Hi, I am facing a problem with sqlalchemy when running I get the error message: Seeing a duplicate entry error, I try to remove the duplicates using the If that helps: I set up a project for a near-surface study using geophones and DAS fiber optics. The data is thus not in seismological format and I had to write the Location and Channel fields myself in my datafiles. An example header looks as follows: I would really appreciate your help on that. Thank you! If needed, the bug report |
Beta Was this translation helpful? Give feedback.
Answered by
zibiwilczynski
Jul 22, 2024
Replies: 1 comment 6 replies
-
|
Hi, which version of MSNoise is this ? Master? The error you get when running new_jobs seems strange, as, in theory, jobs shouldn't be inserted twice, but updated... Your location code is >2 chars (which was forbidden with miniseed2, allowed in miniseed3 norm, which isn't implemented yet in msnoise). Could you check the content of the data_availability table to see if that field is truncated to 2char (leading to, eg. 101 and 102 being both 10, triggering the error. edit: actually the network code is also 3 char, not allowed (and could be trimmed) |
Beta Was this translation helpful? Give feedback.
6 replies
-
|
Hi, The data_availability table did not truncate anything. All the fields are same length as initially. Same for the populated stations table. Initially I had my data in .SEGY format and I converted to .SAC myself adding all the headers myself as well. Do you suggest updating the Location and Network codes to 2 characters? |
Beta Was this translation helpful? Give feedback.
-
|
Hummm, no, could you send me two of those files (two stations)? |
Beta Was this translation helpful? Give feedback.
-
|
Of course, here are the files: For what it's worth: I have numerous files per each day. The filenames represent a time when they were recorded, and files for different channels are stored in different directories but with the same name. Here is the data_availability_snippet.csv. Thank you! |
Beta Was this translation helpful? Give feedback.
-
|
Hi again, I merged the files to 1-day long .SAC files and changed the network and location to 2-char headers. Then the new_job process went through without any errors. |
Beta Was this translation helpful? Give feedback.
Answer selected by
zibiwilczynski
-
|
Hi again @ThomasLecocq, Merging helped and the Cheers, |
Beta Was this translation helpful? Give feedback.
-
|
the clean_duplicates error should be OK with fa3b043 |
Beta Was this translation helpful? Give feedback.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Hi again,
I merged the files to 1-day long .SAC files and changed the network and location to 2-char headers. Then the new_job process went through without any errors.