The model weights are currently initialised by matching regexes against the FQNs.
For each regex, we define a routine specifying how the weights are supposed to be initialised.
This can be prone to errors if FQNs change, as is the case for torch.compile wich introduces a _orig_mod. to FQNs.
Since side-effects like these can easily go unnoticed we should introduce some consistency checks:
- Each FQN must match with exactly one regex.
- If a FQN is not supposed to be initialized via in a initialisation approach (e.g., some norms), the regex should call a "no-op" instead of an init routine.
The model weights are currently initialised by matching regexes against the FQNs.
For each regex, we define a routine specifying how the weights are supposed to be initialised.
This can be prone to errors if FQNs change, as is the case for torch.compile wich introduces a
_orig_mod.to FQNs.Since side-effects like these can easily go unnoticed we should introduce some consistency checks: