Prevent inconsistent data in json files. #74
Conversation
Dumping the json content directly into the final output files creates inconsistend data temporarily, which may cause strange effects. Writing to temporary files and atomically renaming them should fix this. Works only on unix-like systems where os.rename can overwrite existing files.
|
@tobox it would be nice if you fix the problems which let the tests to fail. |
|
This only fixes a small aspect of the bigger problem that also caused #44, namely that apparently in many instances, the output of this script currently is served directly by webservers. This PR may guarantee an atomic modification to the JSON files, but not to the RRDs or PNGs. So I somehow have the feeling there should be a more general approach to the bigger problem, rather than fixing several small effects it has. |
|
If somebody was working on fixing the bigger problem, I would agree that this PR should not be merged yet. If nobody is working on it, I guess a workaround for real-world problems should be considered for merging, especially since this minor change is trivial to revert. |
|
I have (unfortunately) seen a complete messed up nodes.json due to some hard reset of a server. I would therefore appreciate such a change. But It would be good when the buffer is actually written to disk before the file is renamed |
|
|
||
| with open(nodelist_fn, 'w') as f: | ||
| with open(nodelist_fn + 'tmp', 'w') as f: |
There was a problem hiding this comment.
There is a '.' missing in front of the tmp'
Dumping the json content directly into the final output files creates
inconsistend data temporarily, which may cause strange
effects. Writing to temporary files and atomically renaming them
should fix this.
Works only on unix-like systems where os.rename can overwrite existing
files.