Coverage instrumentation loses information in macro-generated code

[narpfel]

I tried this code (using thiserror v1.0.58):

use thiserror::Error;

#[derive(Error, Debug)]
pub enum E {
    #[error("{}", if *.0 { "a" } else { "b" })]
    X(bool)
}

#[cfg(test)]
mod tests {
    use super::E;

    #[test]
    fn test() {
        assert_eq!(E::X(true).to_string(), "a");
    }
}

I expected to see this happen: When running the test with cargo llvm-cov --text, the else arm should be marked as uncovered while the then arm should be shown as covered.

Instead, this happened: The code in the #[error] attribute does not have any coverage data. When run with cargo llvm-cov --text, this is the result:

$ cargo llvm-cov --text
[...]
    1|       |use thiserror::Error;
    2|       |
    3|      1|#[derive(Error, Debug)]
    4|       |pub enum E {
    5|       |    #[error("{}", if *.0 { "a" } else { "b" })]
    6|       |    X(bool)
    7|       |}
    8|       |
    9|       |#[cfg(test)]
   10|       |mod tests {
   11|       |    use super::E;
   12|       |
   13|       |    #[test]
   14|      1|    fn test() {
   15|      1|        assert_eq!(E::X(true).to_string(), "a");
   16|      1|    }
   17|       |}

Note that the proc-macro generated code is shown as covering line 3.

If the code is wrapped in a function (either a normal function or a lambda), the coverage information is correct:

use thiserror::Error;

#[derive(Error, Debug)]
pub enum E {
    #[error("{}", (|| if *.0 { "a" } else { "b" })())]
    X(bool)
}

#[cfg(test)]
mod tests {
    use super::E;

    #[test]
    fn test() {
        assert_eq!(E::X(true).to_string(), "a");
    }
}

will lead to

cargo llvm-cov --text
[...]
    1|       |use thiserror::Error;
    2|       |
    3|      1|#[derive(Error, Debug)]
    4|       |pub enum E {
    5|      1|    #[error("{}", (|| if *.0 { "a" } else { "b" })())]
                                                          ^0
    6|       |    X(bool)
    7|       |}
    8|       |
    9|       |#[cfg(test)]
   10|       |mod tests {
   11|       |    use super::E;
   12|       |
   13|       |    #[test]
   14|      1|    fn test() {
   15|      1|        assert_eq!(E::X(true).to_string(), "a");
   16|      1|    }
   17|       |}

Note that now "b" is shown as uncovered.

I initially encountered this while writing a proc-macro similar to thiserror that generates error messages based on enum definitions. To work around this issue, I wrapped the code that I wanted to have coverage information in IIFEs inside the proc-macro.

Meta

rustc --version --verbose:

$ rustc --version --verbose
rustc 1.79.0-nightly (9d79cd5f7 2024-04-05)
binary: rustc
commit-hash: 9d79cd5f79e75bd0d2083260271307ce9acd9081
commit-date: 2024-04-05
host: x86_64-unknown-linux-gnu
release: 1.79.0-nightly
LLVM version: 18.1.2
$ cargo llvm-cov --version
cargo-llvm-cov 0.6.9
$ grep 'name = "thiserror"' Cargo.lock -A1
name = "thiserror"
version = "1.0.58"

[narpfel]

This also happens with declarative macros:

macro_rules! m {
    ($f:ident @ $b:ident @ $e:expr) => {
        fn $f($b: bool) -> &'static str {
            $e
        }
    }
}

m! {
    f @ b @ if b { "a" } else { "b" }
}

m! {
    g @ b @ (|| if b { "a" } else { "b" })()
}


fn main() {
    f(true);
    g(true);
}

$ cargo llvm-cov run --text
[...]
    1|       |macro_rules! m {
    2|       |    ($f:ident @ $b:ident @ $e:expr) => {
    3|      1|        fn $f($b: bool) -> &'static str {
    4|      1|            $e
    5|      1|        }
    6|       |    }
    7|       |}
    8|       |
    9|       |m! {
   10|       |    f @ b @ if b { "a" } else { "b" }
   11|       |}
   12|       |
   13|       |m! {
   14|      1|    g @ b @ (|| if b { "a" } else { "b" })()
                                                  ^0
   15|       |}
   16|       |
   17|       |
   18|      1|fn main() {
   19|      1|    f(true);
   20|      1|    g(true);
   21|      1|}

Reproduction without `cargo-llvm-cov`

$ rustc -C instrument-coverage main.rs
$ ./main
$ ~/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/bin/llvm-profdata merge -sparse *.profraw -o main.profdata
$ ~/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/bin/llvm-cov show -format=text -instr-profile=main.profdata -object main -show-line-counts-or-regions
    1|       |macro_rules! m {
    2|       |    ($f:ident @ $b:ident @ $e:expr) => {
    3|      1|        fn $f($b: bool) -> &'static str {
    4|      1|            $e
    5|      1|        }
  ------------------
  | _RNvCsk2J25hMqgkX_4main1f:
  |    3|      1|        fn $f($b: bool) -> &'static str {
  |    4|       |            $e
  |    5|      1|        }
  ------------------
  | _RNvCsk2J25hMqgkX_4main1g:
  |    3|      1|        fn $f($b: bool) -> &'static str {
  |    4|      1|            $e
  |    5|      1|        }
  ------------------
    6|       |    }
    7|       |}
    8|       |
    9|       |m! {
   10|       |    f @ b @ if b { "a" } else { "b" }
   11|       |}
   12|       |
   13|       |m! {
   14|      1|    g @ b @ (|| if b { "a" } else { "b" })()
                                                  ^0
   15|       |}
   16|       |
   17|       |
   18|      1|fn main() {
   19|      1|    f(true);
   20|      1|    g(true);
   21|      1|}

[xd009642]

Another example:

use tracing::instrument;

#[instrument]
fn instrumented_function() {
    println!("Hello");
}

#[cfg(test)]
mod tests {
    use super::*;

    #[test]
    fn is_it_hit() {
        instrumented_function();
    }
}

One thing interesting, is with #[instrument] commented out there's one identified code region which covers the entirety of the function. With it uncommented there are two code regions both are on line 3 (so #[instrument]). One is columns 1-2 and the last is columns 13-14 (which is essentially the end of the attribute). Now this doesn't happen for #[tokio::main] which is just fairly simple case of a macro creating an object and doing a chain of methods on it and passing the source in.

[crazymykl]

The #[derive(Parser)] macro from the clap-derive crate also hits this bug. Short, self-contained example here: crazymykl/clap-coverage-ssce.