Shared: Model generator cleanup. #19311
Open
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…n its own module.
michaelnebel
force-pushed
the
csharp/generatorcleanup
branch
from
cf40d2d to
3809598
Compare
…adjust other queries to the re-factored implementation.
…c version instead.
michaelnebel
force-pushed
the
csharp/generatorcleanup
branch
from
3809598 to
3ef0f89
Compare
There was a problem hiding this comment.
Pull Request Overview
This PR refactors the shared model generator by isolating heuristic queries into their own module, updating query annotations in test files, and removing legacy mixed-model queries and flags. Key changes include:
- Replacing "summary" annotations with "heuristic-summary" in query comments across various test files.
- Updating change note files to reflect the switch from mixed to combined (heuristic/content-based) models.
- Removing deprecated flags from the model generator script.
Reviewed Changes
Copilot reviewed 49 out of 64 changed files in this pull request and generated no comments.
| File | Description |
|---|---|
| java/ql/src/change-notes/2025-04-16-model-generation.md | Updated change note reflecting the query switch for Java models. |
| csharp/ql/test/utils/modelgenerator/dataflow/Summaries.cs | Replaced legacy "summary" tags with "heuristic-summary" where appropriate. |
| csharp/ql/src/change-notes/2025-04-16-model-generation.md | Updated change note reflecting the query switch for C# models. |
Files not reviewed (15)
- csharp/ql/src/utils/modelgenerator/CaptureMixedNeutralModels.ql: Language not supported
- csharp/ql/src/utils/modelgenerator/CaptureMixedSummaryModels.ql: Language not supported
- csharp/ql/src/utils/modelgenerator/CaptureNeutralModels.ql: Language not supported
- csharp/ql/src/utils/modelgenerator/CaptureSinkModels.ql: Language not supported
- csharp/ql/src/utils/modelgenerator/CaptureSourceModels.ql: Language not supported
- csharp/ql/src/utils/modelgenerator/CaptureSummaryModels.ql: Language not supported
- csharp/ql/src/utils/modelgenerator/debug/CaptureSummaryModelsPartialPath.ql: Language not supported
- csharp/ql/src/utils/modelgenerator/debug/CaptureSummaryModelsPath.ql: Language not supported
- csharp/ql/test/utils/modelgenerator/dataflow/CaptureHeuristicSummaryModels.ql: Language not supported
- csharp/ql/test/utils/modelgenerator/dataflow/CaptureNeutralModels.ql: Language not supported
- csharp/ql/test/utils/modelgenerator/dataflow/CaptureSinkModels.ql: Language not supported
- csharp/ql/test/utils/modelgenerator/dataflow/CaptureSourceModels.ql: Language not supported
- java/ql/src/utils/modelgenerator/CaptureMixedNeutralModels.ql: Language not supported
- java/ql/src/utils/modelgenerator/CaptureMixedSummaryModels.ql: Language not supported
- java/ql/src/utils/modelgenerator/CaptureNeutralModels.ql: Language not supported
Comments suppressed due to low confidence (2)
java/ql/src/change-notes/2025-04-16-model-generation.md:4
- [nitpick] There appears to be a potential naming inconsistency: the change note references 'GenerateFlowModel.py', while the PR description uses 'GenerateFlowModels.py'. Consider standardizing the script name for clarity.
* Changes to the MaD model generation infrastructure: Changed the query `java/utils/modelgenerator/summary-models` to use the implementation from `java/utils/modelgenerator/mixed-summary-models`. Removed the now-redundant `java/utils/modelgenerator/mixed-summary-models` query. Similar replacement was made for `java/utils/modelgenerator/neutral-models`. That is, if `GenerateFlowModel.py` is provided with `--with-summaries` combined/mixed models are now generated instead of heuristic models (and similar for `--with-neutrals`).
csharp/ql/src/change-notes/2025-04-16-model-generation.md:4
- [nitpick] There appears to be a potential naming inconsistency: the change note references 'GenerateFlowModel.py', while the PR description uses 'GenerateFlowModels.py'. Consider standardizing the script name for clarity.
* Changes to the MaD model generation infrastructure: Changed the query `cs/utils/modelgenerator/summary-models` to use the implementation from `cs/utils/modelgenerator/mixed-summary-models`. Removed the now-redundant `cs/utils/modelgenerator/mixed-summary-models` query. Similar replacement was made for `cs/utils/modelgenerator/neutral-models`. That is, if `GenerateFlowModel.py` is provided with `--with-summaries` combined/mixed models are now generated instead of heuristic models (and similar for `--with-neutrals`).
geoffw0
reviewed
Apr 17, 2025
| --- | ||
| category: minorAnalysis | ||
| --- | ||
| * Changes to the MaD model generation infrastructure: Changed the query `rust/utils/modelgenerator/summary-models` to use the implementation from `rust/utils/modelgenerator/mixed-summary-models`. Removed the now-redundant `rust/utils/modelgenerator/mixed-summary-models` query. Similar replacement was made for `rust/utils/modelgenerator/neutral-models`. That is, if `GenerateFlowModel.py` is provided with `--with-summaries` combined/mixed models are now generated instead of heuristic models (and similar for `--with-neutrals`). |
There was a problem hiding this comment.
We're not actually doing change notes for Rust yet, because we're still in preview - though the symmetry with other languages is nice and I can't see this one doing any harm.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
In this PR we
*/utils/modelgenerator/summary-modelsto use the implementation from*/utils/modelgenerator/mixed-summary-modelsand then delete the*/utils/modelgenerator/mixed-summary-modelsqueries. Also make the same change to the neutral capturing queries.--with-mixed-*flags from the model generator python script.This effectively means that when genering models with the
GenerateFlowModels.pyscript using the flags--with-summariesand--with-neutralsnow generates combined (or mixed as we used to call them) models based on both the content based and heuristic based data flow model generation.