Export large weight window files through openmc.lib without XML serialization - #4057
Export large weight window files through openmc.lib without XML serialization#4057paulromano wants to merge 11 commits into
Conversation
pshriwise
left a comment
There was a problem hiding this comment.
Looks nice @paulromano! Good idea to address the duplicate implementation issue. There's more than enough to keep track of as it is.
Some design conversation to he bad about which side (Python API or openmc.lib) should initiate the object transfers.
| lib_meshes[mesh.id] = mesh.to_lib_object( | ||
| base_dir=original_dir) | ||
|
|
||
| lib_ww = openmc.lib.WeightWindows(ww.id) |
There was a problem hiding this comment.
Perhaps the openmc.lib.WeightWindows class could have a classmethod that takes in an openmc.WeightWindows object to handle some of the setup going on here.
There was a problem hiding this comment.
Good suggestion! I've added openmc.lib.WeightWindows.from_python(), which limits the logic here to managing the temporary session, deduplicating shared meshes, invoking the conversion, and calling the existing C++ exporter.
|
Thanks @GuySten and @pshriwise for the review! All your comments have been addressed. |
pshriwise
left a comment
There was a problem hiding this comment.
Once last small comment from me, but otherwise I think this looks great!
| ---------- | ||
| mesh : openmc.MeshBase | ||
| Python API mesh to convert. | ||
| uid : int, optional |
There was a problem hiding this comment.
Is this parameter used anywhere in the code currently?
There was a problem hiding this comment.
Good point -- no, this wasn't being used and it seems prudent to preserve the ID of the Python mesh, so I've gone ahead and removed the uid argument.
|
I suggest waiting for #4091. Which touches the same code. |
|
@pshriwise comments have been addressed |
Description
Background
WeightWindowsList.export_to_hdf5()currently creates a temporary model containing the weight windows, writes that model to XML, initializes the OpenMC shared library from the XML, and then calls the existing C++ HDF5 exporter. For weight window files containing hundreds of millions of values, constructing the ASCII representation of the bounds can require multiple GBs of additional memory and eventually raiseMemoryError.#3942 addressed this by implementing a direct HDF5 writer in Python with h5py. #3951 refined that design by moving mesh serialization into methods on each mesh subclass. Both approaches avoid the large XML document, but they introduce a second implementation of the weight window HDF5 format alongside the existing C++ writer. This duplicates format logic across Python and C++, requires the Python implementation to remain synchronized with future format changes, and makes the C API responsible for passing HDF5-specific
hid_tvalues across the language boundary. PR #3951 also introduces a separate cleanup operation to manage objects created for this export path.Approach
This PR avoids XML serialization while continuing to use the existing C++ HDF5 writer.
WeightWindowsList.export_to_hdf5()initializes a minimal temporary OpenMC library session and creates the required meshes and weight windows directly throughopenmc.lib. Once the C++ objects have been populated, it calls the existingopenmc.lib.export_weight_windows()function. The weight window bounds therefore never need to be represented in XML.A new public
MeshBase.to_lib_object()method creates the corresponding runtime mesh in an initialized OpenMC library session. It supports regular, rectilinear, cylindrical, spherical, and unstructured meshes. The necessary mesh names, origins, unstructured-mesh options, length multipliers, and IDs are transferred through APIs using ordinary C-compatible data types. The existingopenmc_add_unstructured_mesh()API is extended to accept all properties needed to reproduce a Python unstructured mesh directly in the library.This design retains the primary benefit of PRs #3942 and #3951: multi-GB weight window data no longer passes through XML. Unlike those approaches, it preserves a single implementation of the weight window HDF5 format. Changes to the format only need to be made in the existing C++ writer, and Python does not need to reproduce C++ serialization behavior with h5py.
Checklist