Fix: create_external_models misclassifies cross-repo models as external#5745
Open
gabepesco wants to merge 1 commit intoSQLMesh:mainfrom
Open
Fix: create_external_models misclassifies cross-repo models as external#5745gabepesco wants to merge 1 commit intoSQLMesh:mainfrom
gabepesco wants to merge 1 commit intoSQLMesh:mainfrom
Conversation
…al (SQLMesh#5326) In a multi-repo setup, `create_external_models` partitions models per-config before passing them to `create_external_models_file`. This caused dependencies that belong to another repo (e.g. `bronze.a` referenced by `silver.c` in repo_2) to appear missing from the current config's model set and be incorrectly classified as external. The fix adds an `all_models` parameter to `create_external_models_file` that represents the full set of loaded models across all repos. Dependency resolution uses this complete set, so cross-repo internal models are never mistaken for external ones. The per-config `models` argument is still used to scope which models' dependencies are written to each repo's `external_models.yaml`. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
gabepesco
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Resolves #5326
Description
In a multi-repo setup,
create_external_modelspartitions models per-config before passing them tocreate_external_models_file. This caused dependencies that belong to another repo (e.g.bronze.areferenced bysilver.cin repo_2) to appear missing from the current config's model set and be incorrectly classified as external.The fix adds an
all_modelsparameter tocreate_external_models_filethat represents the full set of loaded models across all repos. Dependency resolution uses this complete set, so cross-repo internal models are never mistaken for external ones. The per-configmodelsargument is still used to scope which models' dependencies are written to each repo'sexternal_models.yaml.Test Plan
Added
test_multi_repo_create_external_modelsintests/core/integration/test_multi_repo.pyusing the existingexamples/multifixture. The test verifies that aftercreate_external_models(), no "Unable to get schema" warnings are emitted for cross-repo models, and thatplanstill resolves all models as internal (no new snapshots treated as external).Confirmed the test fails without the fix and passes with it. Ran full regression on
tests/core/test_schema_loader.pyandtests/core/integration/test_multi_repo.py, all passing. I also monkey patched it into my own pipeline, and the warning was gone and no other issues occurred.Checklist
make styleand fixed any issuesmake fast-test)git commit -s) per the DCO