Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The SkimDataset structure requires every variable to have a unique name. It also merges OMX variables based on time period, so that e.g. BIKETIME__AM and BIKETIME__PM, which would be 2-d arrays in the OMX file, become just two different parts of a 3-d array called BIKETIME in the SkimDataset. This is problematic when the skims also contain a 2-d array called BIKETIME, as that has no temporal dimension, and it gets loaded into a 2-d array in the SkimDataset, with the same name as the 3-d array, and thus one is overwritten and lost.
A workaround was added via the omx_ignore_patterns setting in #867, which allows the user to not load certain skims, but this relies on the model developer or user to set this up correctly.
This PR includes a skims input check to identify this overwriting condition, and raise an error if it is happening, so that the user can correct the condition via (1) the omx_ignore_patterns setting, (2) revising the skim generation process to not create the overlapping named skims in the file in the first place, or (3) renaming one or both skims if the users actually wants both skims variables in the model. The error message generated includes a link to instructions and discussion of these alternatives.
Enhancements to Skim Dataset Handling:
activitysim/core/skim_dataset.py
,activitysim/core/skim_dict_factory.py
) [1] [2]skim_conflicts
attribute in the skim dictionary factory to track and warn about duplicate skim names and conflicts. (activitysim/core/skim_dict_factory.py
) [1] [2]activitysim/core/skim_dict_factory.py
) [1] [2]Workflow and Dependency Updates:
workflow_dispatch
while maintaining existing conditions for automatic builds. (.github/workflows/branch-docs.yml
)re
anddefaultdict
imports to support new functionality for skim conflict detection and regex-based ignore rules. (activitysim/core/skim_dataset.py
,activitysim/core/skim_dict_factory.py
) [1] [2]Documentation Updates:
docs/users-guide/model_anatomy.rst
) [1] [2]These changes aim to improve the robustness of skim data handling, provide clearer guidance to users, and ensure compatibility with future updates.