Skip to content

rust_analyzer: unpredictable/broken behavior with generated sources #3126

New issue
New issue
Open
#3127
Open
rust_analyzer: unpredictable/broken behavior with generated sources#3126
#3127
@sam-mccall

Description

@sam-mccall
sam-mccall
opened on Dec 20, 2024

At a3778c5, the rust-analyzer generated_srcs_test fails on my machine, complaining about missing source.include_dirs:

log 
$ bazel run test/rust_analyzer:rust_analyzer_test

Building with Aspects...
INFO: Analyzed 15 targets (1 packages loaded, 19 targets configured).
INFO: Found 15 targets...
INFO: Elapsed time: 6.054s, Critical Path: 0.41s
INFO: 25 processes: 1 internal, 24 linux-sandbox.
INFO: Build completed successfully, 25 total actions
Testing 'generated_srcs_test'
Generating rust-project.json...
INFO: Analyzed target @@rules_rust+//tools/rust_analyzer:gen_rust_project (44 packages loaded, 3760 targets configured).
INFO: Found 1 target...
Target @@rules_rust+//tools/rust_analyzer:gen_rust_project up-to-date:
  bazel-bin/external/rules_rust+/tools/rust_analyzer/gen_rust_project
INFO: Elapsed time: 0.307s, Critical Path: 0.01s
INFO: 1 process: 110 action cache hit, 1 internal.
INFO: Build completed successfully, 1 total action
INFO: Running command line: bazel-bin/external/rules_rust+/tools/rust_analyzer/gen_rust_project
Building...
INFO: Analyzed 5 targets (0 packages loaded, 0 targets configured).
INFO: Found 5 targets...
INFO: Elapsed time: 0.344s, Critical Path: 0.21s
INFO: 9 processes: 3 action cache hit, 6 internal, 3 linux-sandbox.
INFO: Build completed successfully, 9 total actions
Testing...
INFO: Analyzed 5 targets (0 packages loaded, 0 targets configured).
FAIL: //:rust_project_json_test (Exit 101) (see /usr/local/google/home/sammccall/.cache/bazel/_bazel_sammccall/3852688809b88d0967f87b3807dc2120/execroot/_main/bazel-out/k8-fastbuild/testlogs/rust_project_json_test/test.log)
INFO: From Testing //:rust_project_json_test:
==================== Test output for //:rust_project_json_test:

running 1 test
test tests::test_generated_srcs ... FAILED

failures:

---- tests::test_generated_srcs stdout ----
{
  "sysroot": "/usr/local/google/home/sammccall/.cache/bazel/_bazel_sammccall/3852688809b88d0967f87b3807dc2120/external/rules_rust++rust+rust_analyzer_1.83.0_tools",
  "sysroot_src": "/usr/local/google/home/sammccall/.cache/bazel/_bazel_sammccall/3852688809b88d0967f87b3807dc2120/external/rules_rust++rust+rust_analyzer_1.83.0_tools/lib/rustlib/src/library",
  "crates": [
    {
      "display_name": "generated_srcs",
      "root_module": "/usr/local/google/home/sammccall/.cache/bazel/_bazel_sammccall/3852688809b88d0967f87b3807dc2120/execroot/_main/bazel-out/k8-fastbuild/bin/lib.rs",
      "edition": "2021",
      "deps": [],
      "is_workspace_member": true,
      "cfg": [
        "test",
        "debug_assertions"
      ],
      "target": "x86_64-unknown-linux-gnu",
      "env": {
        "CARGO_CFG_TARGET_ARCH": "x86_64",
        "CARGO_CFG_TARGET_OS": "linux",
        "CARGO_CRATE_NAME": "generated_srcs",
        "CARGO_MANIFEST_DIR": "/usr/local/google/home/sammccall/.cache/bazel/_bazel_sammccall/3852688809b88d0967f87b3807dc2120/execroot/_main",
        "CARGO_PKG_AUTHORS": "",
        "CARGO_PKG_DESCRIPTION": "",
        "CARGO_PKG_HOMEPAGE": "",
        "CARGO_PKG_NAME": "generated_srcs",
        "CARGO_PKG_VERSION": "0.0.0",
        "CARGO_PKG_VERSION_MAJOR": "0",
        "CARGO_PKG_VERSION_MINOR": "0",
        "CARGO_PKG_VERSION_PATCH": "0",
        "CARGO_PKG_VERSION_PRE": "",
        "REPOSITORY_NAME": "",
        "ZERO_AR_DATE": "1"
      },
      "is_proc_macro": false
    },
    [...]
  ]
}
output_base: /usr/local/google/home/sammccall/.cache/bazel/_bazel_sammccall
thread 'tests::test_generated_srcs' panicked at rust_project_json_test.rs:53:49:
called `Option::unwrap()` on a `None` value
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace


failures:
    tests::test_generated_srcs

test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

================================================================================
INFO: Found 3 targets and 2 test targets...
INFO: Elapsed time: 0.207s, Critical Path: 0.07s
INFO: 3 processes: 4 linux-sandbox.
INFO: Build completed, 1 test FAILED, 3 total actions
//:generated_srcs_test                                                   PASSED in 0.0s
//:rust_project_json_test                                                FAILED in 0.0s
  /usr/local/google/home/sammccall/.cache/bazel/_bazel_sammccall/3852688809b88d0967f87b3807dc2120/execroot/_main/bazel-out/k8-fastbuild/testlogs/rust_project_json_test/test.log

Executed 2 out of 2 tests: 1 test passes and 1 fails locally.

Running the aspect by hand shows there are two crate specs produced, one for the library and one for the test:

  • the library spec has successfully remapped the root_module lib.rs from a generated file to a source file in srcs, and so has a __WORKSPACE__ path with a source.include_dir under __EXEC_ROOT__.
  • the test spec has not performed this remapping (there are no srcs here), and root_module has a __EXEC_ROOT__ path with no source.include_dir.
  • the "consolidate" logic picks one 'at random', and my machine is unlucky
generated_srcs.json 
{
    "aliases": {},
    "cfg": [
        "test",
        "debug_assertions"
    ],
    "crate_id": "ID-bazel-out/k8-fastbuild/bin/test/rust_analyzer/generated_srcs_test/lib.rs",
    "crate_type": "rlib",
    "deps": [],
    "display_name": "generated_srcs",
    "edition": "2021",
    "env": {
        "CARGO_CFG_TARGET_ARCH": "x86_64",
        "CARGO_CFG_TARGET_OS": "linux",
        "CARGO_CRATE_NAME": "generated_srcs",
        "CARGO_MANIFEST_DIR": "${pwd}/test/rust_analyzer/generated_srcs_test",
        "CARGO_PKG_AUTHORS": "",
        "CARGO_PKG_DESCRIPTION": "",
        "CARGO_PKG_HOMEPAGE": "",
        "CARGO_PKG_NAME": "generated_srcs",
        "CARGO_PKG_VERSION": "0.0.0",
        "CARGO_PKG_VERSION_MAJOR": "0",
        "CARGO_PKG_VERSION_MINOR": "0",
        "CARGO_PKG_VERSION_PATCH": "0",
        "CARGO_PKG_VERSION_PRE": "",
        "REPOSITORY_NAME": ""
    },
    "is_workspace_member": true,
    "root_module": "__WORKSPACE__/test/rust_analyzer/generated_srcs_test/lib.rs",
    "source": {
        "exclude_dirs": [],
        "include_dirs": [
            "__EXEC_ROOT__/bazel-out/k8-fastbuild/bin/test/rust_analyzer/generated_srcs_test"
        ]
    },
    "target": "x86_64-unknown-linux-gnu"
}
generated_srcs_test.json 
{
    "aliases": {},
    "cfg": [
        "test",
        "debug_assertions"
    ],
    "crate_id": "ID-bazel-out/k8-fastbuild/bin/test/rust_analyzer/generated_srcs_test/lib.rs",
    "crate_type": "bin",
    "deps": [],
    "display_name": "generated_srcs",
    "edition": "2021",
    "env": {
        "CARGO_CFG_TARGET_ARCH": "x86_64",
        "CARGO_CFG_TARGET_OS": "linux",
        "CARGO_CRATE_NAME": "generated_srcs",
        "CARGO_MANIFEST_DIR": "${pwd}/test/rust_analyzer/generated_srcs_test",
        "CARGO_PKG_AUTHORS": "",
        "CARGO_PKG_DESCRIPTION": "",
        "CARGO_PKG_HOMEPAGE": "",
        "CARGO_PKG_NAME": "generated_srcs",
        "CARGO_PKG_VERSION": "0.0.0",
        "CARGO_PKG_VERSION_MAJOR": "0",
        "CARGO_PKG_VERSION_MINOR": "0",
        "CARGO_PKG_VERSION_PATCH": "0",
        "CARGO_PKG_VERSION_PRE": "",
        "REPOSITORY_NAME": "",
        "ZERO_AR_DATE": "1"
    },
    "is_workspace_member": false,
    "root_module": "__EXEC_ROOT__/bazel-out/k8-fastbuild/bin/test/rust_analyzer/generated_srcs_test/lib.rs",
    "source": {
        "exclude_dirs": [],
        "include_dirs": []
    },
    "target": "x86_64-unknown-linux-gnu"
}

Digging deeper into this, it seems both are broken, and it's not clear that making the test pass is a real fix.

  • the reason we have generated sources: rustc requires a crate's sources to be in the same dir. If some are generated and others are not, rust_library will generate extra symlinks so all files are in the generated tree.
  • the reason we try to restore the original path: if rust-project.json describes a crate rooted at blaze-out/gen/foo.rs, but you open workspace/foo.rs in your editor, rust-analyzer doesn't know it's the same file.
  • but restoring the original path doesn't actually work, because rust-analyzer also requires the crate's sources to be in the same dir. When we open workspace/foo.rs and see pub mod generated, rust-analyzer looks for workspace/generated.rs, and can't find it, because it's actually blaze-out/gen/generated.rs.

So if we don't remap the path, we get a rust-project.json that "works" if you open all the files through their generated locations, but rust-analyzer doesn't know which crates your regular source files belong to.
If we do remap the path, we get a rust-project.json where rust-analyzer knows which crates your files are part of, but can't actually understand them because generated files are missing.

(The existing logic tries to point include_dirs to the generated directory when there might be sources there, but I don't think this actually achieves anything: this isn't a C-style #include path, rather each module is resolved relative to the current one.)

I'm not really sure what to do about this. If we have to choose, there are arguments for either behavior:

  • if you're not directly editing the crate with generated sources, but rather something that depends on it, using the generated sources is way better: you get correct semantic analysis.
  • if you are, then probably partly-broken results are better than none at all

Maybe there's a clean solution that someone smarter than me can find...

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions