[NFC][analyzer] Document configuration options

[NagyDonat]

This commit documents the process of specifying values for the analyzer options and checker options implemented in the static analyzer, and adds a script which includes the documentation of the analyzer options (which was previously only available through a command-line flag) in the RST-based web documentation.

[llvmbot]

@llvm/pr-subscribers-clang-static-analyzer-1

@llvm/pr-subscribers-clang

Author: Donát Nagy (NagyDonat)

Changes

This commit documents the process of specifying values for the analyzer options and checker options implemented in the static analyzer, and adds a script which includes the documentation of the analyzer options (which was previously only available through a command-line flag) in the RST-based web documentation.

---

Full diff: https://github.com/llvm/llvm-project/pull/135169.diff

6 Files Affected:

• (modified) clang/docs/CMakeLists.txt (+26)
• (modified) clang/docs/analyzer/user-docs.rst (+1)
• (modified) clang/docs/analyzer/user-docs/CommandLineUsage.rst (+2)
• (added) clang/docs/analyzer/user-docs/Options.rst.in (+102)
• (added) clang/docs/tools/generate_analyzer_options_docs.py (+242)
• (modified) clang/include/clang/StaticAnalyzer/Core/AnalyzerOptions.def (+3)

diff --git a/clang/docs/CMakeLists.txt b/clang/docs/CMakeLists.txt
index 4fecc007f5995..9dfcc692ff87d 100644
--- a/clang/docs/CMakeLists.txt
+++ b/clang/docs/CMakeLists.txt
@@ -143,6 +143,32 @@ if (LLVM_ENABLE_SPHINX)
     gen_rst_file_from_td(DiagnosticsReference.rst -gen-diag-docs ../include/clang/Basic/Diagnostic.td "${docs_targets}")
     gen_rst_file_from_td(ClangCommandLineReference.rst -gen-opt-docs ../include/clang/Driver/ClangOptionDocs.td "${docs_targets}")
 
+    # Another generated file from a different source
+    set(docs_tools_dir ${CMAKE_CURRENT_SOURCE_DIR}/tools)
+    set(aopts_rst_rel_path analyzer/user-docs/Options.rst)
+    set(aopts_rst "${CMAKE_CURRENT_BINARY_DIR}/${aopts_rst_rel_path}")
+    set(analyzeroptions_def "${CMAKE_CURRENT_SOURCE_DIR}/../include/clang/StaticAnalyzer/Core/AnalyzerOptions.def")
+    set(aopts_rst_in "${CMAKE_CURRENT_SOURCE_DIR}/${aopts_rst_rel_path}.in")
+    set(generate_aopts_docs generate_analyzer_options_docs.py)
+    add_custom_command(
+      OUTPUT ${aopts_rst}
+      COMMAND ${Python3_EXECUTABLE} ${generate_aopts_docs} -i ${analyzeroptions_def} -t ${aopts_rst_in} -o ${aopts_rst}
+      WORKING_DIRECTORY ${docs_tools_dir}
+      VERBATIM
+      COMMENT "Generating ${aopts_rst}"
+      DEPENDS ${docs_tools_dir}/${generate_aopts_docs}
+              ${aopts_rst_in}
+              copy-clang-rst-docs
+      )
+    add_custom_target(generate-analyzer-options-rst DEPENDS ${aopts_rst})
+    foreach(target ${docs_targets})
+      add_dependencies(${target} generate-analyzer-options-rst)
+    endforeach()
+
+    # Technically this is redundant because generate-analyzer-options-rst
+    # depends on the copy operation (because it wants to drop a generated file
+    # into a subdirectory of the copied tree), but I'm leaving it here for the
+    # sake of clarity.
     foreach(target ${docs_targets})
       add_dependencies(${target} copy-clang-rst-docs)
     endforeach()
diff --git a/clang/docs/analyzer/user-docs.rst b/clang/docs/analyzer/user-docs.rst
index e265f033a2c54..67c1dfaa40965 100644
--- a/clang/docs/analyzer/user-docs.rst
+++ b/clang/docs/analyzer/user-docs.rst
@@ -8,6 +8,7 @@ Contents:
 
    user-docs/Installation
    user-docs/CommandLineUsage
+   user-docs/Options
    user-docs/UsingWithXCode
    user-docs/FilingBugs
    user-docs/CrossTranslationUnit
diff --git a/clang/docs/analyzer/user-docs/CommandLineUsage.rst b/clang/docs/analyzer/user-docs/CommandLineUsage.rst
index 59f8187f374a9..0252de80b788f 100644
--- a/clang/docs/analyzer/user-docs/CommandLineUsage.rst
+++ b/clang/docs/analyzer/user-docs/CommandLineUsage.rst
@@ -194,6 +194,8 @@ When compiling your application to run on the simulator, it is important that **
 
 If you aren't certain which compiler Xcode uses to build your project, try just running ``xcodebuild`` (without **scan-build**). You should see the full path to the compiler that Xcode is using, and use that as an argument to ``--use-cc``.
 
+.. _command-line-usage-CodeChecker:
+
 CodeChecker
 -----------
 
diff --git a/clang/docs/analyzer/user-docs/Options.rst.in b/clang/docs/analyzer/user-docs/Options.rst.in
new file mode 100644
index 0000000000000..eced3597ed567
--- /dev/null
+++ b/clang/docs/analyzer/user-docs/Options.rst.in
@@ -0,0 +1,102 @@
+========================
+Configuring the Analyzer
+========================
+
+The clang static analyzer supports two kinds of options:
+
+1. Global **analyzer options** influence the behavior of the analyzer engine.
+   They are documented on this page, in the section :ref:`List of analyzer
+   options<list-of-analyzer-options>`.
+2. The **checker options** belong to individual checkers (e.g.
+   ``core.BitwiseShift:Pedantic`` and ``unix.Stream:Pedantic`` are completely
+   separate options) and customize the behavior of that particular checker.
+   These are documented within the documentation of each individual checker at
+   :doc:`../checkers`.
+
+Assigning values to options
+===========================
+
+With the compiler frontend
+--------------------------
+
+All options can be configured by using the ``-analyzer-config`` flag of ``clang
+-cc1`` (the so-called *compiler frontend* part of clang). The values of the
+options are specified with the syntax ``-analyzer-config
+OPT=VAL,OPT2=VAL2,...`` which supports specifying multiple options, but
+separate flags like ``-analyzer-config OPT=VAL -analyzer-config OPT2=VAL2`` are
+also accepted (with equivalent behavior). Analyzer options and checker options
+can be freely intermixed here because it's easy to recognize that checker
+option names are always prefixed with ``some.groups.NameOfChecker:``.
+
+With the clang driver
+---------------------
+
+In a conventional workflow ``clang -cc1`` (which is a low-level internal
+interface) is invoked indirectly by the clang *driver* (i.e. plain ``clang``
+without the ``-cc1`` flag), which acts as an "even more frontend" wrapper layer
+around the ``clang -cc1`` *compiler frontend*. In this situation **each**
+command line argument intended for the *compiler frontend* must be prefixed
+with ``-Xclang``.
+
+For example the following command analyzes ``foo.c`` in :ref:`shallow mode
+<analyzer-option-mode>` with :ref:`loop unrolling
+<analyzer-option-unroll-loops>`:
+
+::
+
+  clang --analyze -Xclang -analyzer-config -Xclang mode=shallow,unroll-loops=true foo.c
+
+When this is executed, the *driver* will compose and execute the following
+``clang -cc1`` command (which can be inspected by passing the ``-v`` flag to
+the *driver*):
+
+::
+
+  clang -cc1 -analyze [...] -analyzer-config mode=shallow,unroll-loops=true foo.c
+
+Here ``[...]`` stands for dozens of low-level flags which ensure that ``clang
+-cc1`` does the right thing (e.g. ``-fcolor-diagnostics`` when it's suitable;
+``-analyzer-checker`` flags to enable a sane default set of checkers). Also
+note the distinction that the ``clang`` *driver* requires ``--analyze`` (double
+dashes) while the ``clang -cc1`` *compiler frontend* requires ``-analyze``
+(single dash).
+
+With CodeChecker
+----------------
+
+If the analysis is performed through :ref:`CodeChecker
+<command-line-usage-CodeChecker>` (which e.g. supports the analysis of a whole
+project instead of a single file) then it will act as another indirection
+layer. CodeChecker provides separate command-line flags called
+``--analyzer-config`` (for analyzer options) and ``--checker-config`` (for
+checker options):
+
+::
+
+  CodeChecker analyze -o outdir --checker-config clangsa:unix.Stream:Pedantic=true  \
+          --analyzer-config clangsa:mode=shallow clangsa:unroll-loops=true          \
+          -- compile_commands.json
+
+These CodeChecker flags may be followed by multiple ``OPT=VAL`` pairs as
+separate arguments (and this is why the example needs to use ``--`` before
+``compile_commands.json``). The option names are all prefixed with ``clangsa:``
+to ensure that they are passed to the clang static analyzer (and not other
+analyzer tools that are also supported by CodeChecker).
+
+.. _list-of-analyzer-options:
+
+List of analyzer options
+========================
+
+.. warning::
+   These options are primarily intended for development purposes. Changing
+   their values may drastically alter the behavior of the analyzer, and may
+   even result in instabilities or crashes!
+
+..
+   The contents of this section are automatically generated by the script
+   clang/docs/tools/generate_analyzer_options_docs.py from the header file
+   AnalyzerOptions.def to ensure that the RST/web documentation is synchronized
+   with the command line help options.
+
+.. OPTIONS_LIST_PLACEHOLDER
diff --git a/clang/docs/tools/generate_analyzer_options_docs.py b/clang/docs/tools/generate_analyzer_options_docs.py
new file mode 100644
index 0000000000000..5dfc571deb9a0
--- /dev/null
+++ b/clang/docs/tools/generate_analyzer_options_docs.py
@@ -0,0 +1,242 @@
+#!/usr/bin/env python3
+# A tool to automatically generate documentation for the config options of the
+# clang static analyzer by reading `AnalyzerOptions.def`.
+
+import argparse
+from collections import namedtuple
+from enum import Enum, auto
+import re
+import sys
+import textwrap
+
+
+# The following code implements a trivial parser for the narrow subset of C++
+# which is used in AnalyzerOptions.def. This supports the following features:
+# - ignores preprocessor directives, even if they are continued with \ at EOL
+# - ignores comments: both /* ... */ and // ...
+# - parses string literals (even if they contain \" escapes)
+# - concatenates adjacent string literals
+# - parses numbers even if they contain ' as a thousands separator
+# - recognizes MACRO(arg1, arg2, ..., argN) calls
+
+
+class TT(Enum):
+    "Token type enum."
+    number = auto()
+    ident = auto()
+    string = auto()
+    punct = auto()
+
+
+TOKENS = [
+    (re.compile(r"-?[0-9']+"), TT.number),
+    (re.compile(r"\w+"), TT.ident),
+    (re.compile(r'"([^\\"]|\\.)*"'), TT.string),
+    (re.compile(r"[(),]"), TT.punct),
+    (re.compile(r"/\*((?!\*/).)*\*/", re.S), None),  # C-style comment
+    (re.compile(r"//.*\n"), None),  # C++ style oneline comment
+    (re.compile(r"#.*(\\\n.*)*(?<!\\)\n"), None),  # preprocessor directive
+    (re.compile(r"\s+"), None),  # whitespace
+]
+
+Token = namedtuple("Token", "kind code")
+
+
+def report_unexpected(s, pos):
+    lines = (s[:pos] + "X").split("\n")
+    lineno, col = (len(lines), len(lines[-1]))
+    print(
+        "unexpected character %r in AnalyzerOptions.def at line %d column %d"
+        % (s[pos], lineno, col),
+        file=sys.stderr,
+    )
+
+
+def tokenize(s):
+    result = []
+    pos = 0
+    while pos < len(s):
+        for regex, kind in TOKENS:
+            if m := regex.match(s, pos):
+                if kind is not None:
+                    result.append(Token(kind, m.group(0)))
+                pos = m.end()
+                break
+        else:
+            report_unexpected(s, pos)
+            pos += 1
+    return result
+
+
+def join_strings(tokens):
+    result = []
+    for tok in tokens:
+        if tok.kind == TT.string and result and result[-1].kind == TT.string:
+            # If this token is a string, and the previous non-ignored token is
+            # also a string, then merge them into a single token. We need to
+            # discard the closing " of the previous string and the opening " of
+            # this string.
+            prev = result.pop()
+            result.append(Token(TT.string, prev.code[:-1] + tok.code[1:]))
+        else:
+            result.append(tok)
+    return result
+
+
+MacroCall = namedtuple("MacroCall", "name args")
+
+
+class State(Enum):
+    "States of the state machine used for parsing the macro calls."
+    init = auto()
+    after_ident = auto()
+    before_arg = auto()
+    after_arg = auto()
+
+
+def get_calls(tokens, macro_names):
+    state = State.init
+    result = []
+    current = None
+    for tok in tokens:
+        if state == State.init and tok.kind == TT.ident and tok.code in macro_names:
+            current = MacroCall(tok.code, [])
+            state = State.after_ident
+        elif state == State.after_ident and tok == Token(TT.punct, "("):
+            state = State.before_arg
+        elif state == State.before_arg:
+            if current is not None:
+                current.args.append(tok)
+                state = State.after_arg
+        elif state == State.after_arg and tok.kind == TT.punct:
+            if tok.code == ")":
+                result.append(current)
+                current = None
+                state = State.init
+            elif tok.code == ",":
+                state = State.before_arg
+        else:
+            current = None
+            state = State.init
+    return result
+
+
+# The information will be extracted from calls to these two macros:
+# #define ANALYZER_OPTION(TYPE, NAME, CMDFLAG, DESC, DEFAULT_VAL)
+# #define ANALYZER_OPTION_DEPENDS_ON_USER_MODE(TYPE, NAME, CMDFLAG, DESC,
+#                                              SHALLOW_VAL, DEEP_VAL)
+
+MACRO_NAMES_ARGCOUNTS = {
+    "ANALYZER_OPTION": 5,
+    "ANALYZER_OPTION_DEPENDS_ON_USER_MODE": 6,
+}
+
+
+def string_value(tok):
+    if tok.kind != TT.string:
+        raise ValueError(f"expected a string token, got {tok.kind.name}")
+    text = tok.code[1:-1]  # Remove quotes
+    text = re.sub(r"\\(.)", r"\1", text)  # Resolve backslash escapes
+    return text
+
+
+def cmdflag_to_rst_title(cmdflag_tok):
+    text = string_value(cmdflag_tok)
+    underline = "-" * len(text)
+    ref = f".. _analyzer-option-{text}:"
+
+    return f"{ref}\n\n{text}\n{underline}\n\n"
+
+
+def desc_to_rst_paragraphs(tok):
+    desc = string_value(tok)
+
+    # Escape a star that would act as inline emphasis within RST.
+    desc = desc.replace("ctu-max-nodes-*", r"ctu-max-nodes-\*")
+
+    # Many descriptions end with "Value: <list of accepted values>", which is
+    # OK for a terse command line printout, but should be prettified for web
+    # documentation.
+    # Moreover, the option ctu-invocation-list shows some example file content
+    # which is formatted as a preformatted block.
+    paragraphs = [desc]
+    extra = ""
+    if m := re.search(r"(^|\s)Value:", desc):
+        paragraphs = [desc[: m.start()], "Accepted values:" + desc[m.end() :]]
+    elif m := re.search(r"\s*Example file.content:", desc):
+        paragraphs = [desc[: m.start()]]
+        extra = "Example file content::\n\n  " + desc[m.end() :] + "\n\n"
+
+    wrapped = [textwrap.fill(p, width=80) for p in paragraphs if p.strip()]
+
+    return "\n\n".join(wrapped + [""]) + extra
+
+
+def default_to_rst(tok):
+    if tok.kind == TT.string:
+        if tok.code == '""':
+            return "(empty string)"
+        return tok.code
+    if tok.kind == TT.ident:
+        return tok.code
+    if tok.kind == TT.number:
+        return tok.code.replace("'", "")
+    raise ValueError(f"unexpected token as default value: {tok.kind.name}")
+
+
+def defaults_to_rst_paragraph(defaults):
+    strs = [default_to_rst(d) for d in defaults]
+
+    if len(strs) == 1:
+        return f"Default value: {strs[0]}\n\n"
+    if len(strs) == 2:
+        return (
+            f"Default value: {strs[0]} (in shallow mode) / {strs[1]} (in deep mode)\n\n"
+        )
+    raise ValueError("unexpected count of default values: %d" % len(defaults))
+
+
+def macro_call_to_rst_paragraphs(macro_call):
+    if len(macro_call.args) != MACRO_NAMES_ARGCOUNTS[macro_call.name]:
+        return ""
+
+    try:
+        _, _, cmdflag, desc, *defaults = macro_call.args
+
+        return (
+            cmdflag_to_rst_title(cmdflag)
+            + desc_to_rst_paragraphs(desc)
+            + defaults_to_rst_paragraph(defaults)
+        )
+    except ValueError as ve:
+        print(ve.args[0], file=sys.stderr)
+        return ""
+
+
+def get_option_list(input_file):
+    with open(input_file, encoding="utf-8") as f:
+        contents = f.read()
+    tokens = join_strings(tokenize(contents))
+    macro_calls = get_calls(tokens, MACRO_NAMES_ARGCOUNTS)
+
+    result = ""
+    for mc in macro_calls:
+        result += macro_call_to_rst_paragraphs(mc)
+    return result
+
+
+p = argparse.ArgumentParser()
+p.add_argument("-i", "--input", help="path to AnalyzerOptions.def")
+p.add_argument("-t", "--template", help="path of template file")
+p.add_argument("-o", "--output", help="path of output file")
+opts = p.parse_args()
+
+with open(opts.template, encoding="utf-8") as f:
+    doc_template = f.read()
+
+PLACEHOLDER = ".. OPTIONS_LIST_PLACEHOLDER\n"
+
+rst_output = doc_template.replace(PLACEHOLDER, get_option_list(opts.input))
+
+with open(opts.output, "w", newline="", encoding="utf-8") as f:
+    f.write(rst_output)
diff --git a/clang/include/clang/StaticAnalyzer/Core/AnalyzerOptions.def b/clang/include/clang/StaticAnalyzer/Core/AnalyzerOptions.def
index f9f22a9ced650..8326f5309035e 100644
--- a/clang/include/clang/StaticAnalyzer/Core/AnalyzerOptions.def
+++ b/clang/include/clang/StaticAnalyzer/Core/AnalyzerOptions.def
@@ -7,6 +7,9 @@
 //===----------------------------------------------------------------------===//
 //
 //  This file defines the analyzer options avaible with -analyzer-config.
+//  Note that clang/docs/tools/generate_analyzer_options_docs.py relies on the
+//  structure of this file, so if this file is refactored, then make sure to
+//  update that script as well.
 //
 //===----------------------------------------------------------------------===//
 

[NagyDonat]

The HTML page produced by this commit is available at: https://html-preview.github.io/?url=https://gist.githubusercontent.com/NagyDonat/8d8ca3ac2292ceb8044a4bd1b54f3a14/raw/75f794c8e7f4eafce2507f6c196cd74062bc2461/Options.html

(I slightly edited the locally built HTML to ensure that it loads CSS/JS from clang.llvm.org/docs instead of local files.)

[steakhal]

Thank you for proposing this. It looks really good already.
The code looks nice and dense, and the output also looks wonderful.
I had some design questions and a few remarks on the testing and error handling.

Good job.

[steakhal]

We could inline this given it doesn't shorten much.

[NagyDonat]

Fair point, I'll inline the script name.

[steakhal]

BTW isn't the -i flag meant for interactive uses?

[NagyDonat]

This is unrelated to the interpreter flag: the Python interpreter is parametrized as

python3 <arguments for the interpreter> script.py <arguments for the script>

and I decided to implement an -i flag for specifying the input file of the script. I could change this (probably to --input) if you think that it's misleading.

[steakhal]

We should put a disclaimer here that clang does not intend to preserve backward compatibility or announce breaking changes with -cc1 flags.
Use them with that in mind.

[steakhal]

BTW I believe that one could pass analyzer options to the analyzer action using the -Xanalyzer although I rarely ever see that.

[NagyDonat]

OK, I'll add a disclaimer and perhaps document -Xanalyzer if that works as well.

[steakhal]

Generally, anything that is not the default is practically unsupported unfortunately. It's a messed up situation, but let's admit that. There are only a few exceptional configs, such as Z3 cross checking, but generally, this is the case.

[steakhal]

Maybe we should say that crash reports are welcomed, and depending on the severity we may decide to fix them.

[steakhal]

I wonder if we can have a test for this? At least some canary test that the script works.

[NagyDonat]

Good idea. I'll probably drop a simple lit test into clang/test/Analysis which validates that the script can parse everything in AnalyzerOptions.def. I don't think that more testing is worth the additional effort -- this is a low priority plumbing script after all.

[steakhal]

I don't write "official" python scripts, but I've seen a pattern having a check for "main" and what not elsewhere - except in this script. Do you know what conventions to follow? Have you checked similar in-tree python scripts?

[NagyDonat]

I copied this structure from another (somewhat analogous) script in the repo, but I didn't count the number of scripts that check / don't check __main__.

[steakhal]

So we have some error handling. I wonder if we should prepare for (thus detect) some malformed .def files for common mistakes to see how that use-case would look like.
I think devs should have a clear message what did they mess up in a .def file.
We could demonstrate the proper detection in a test file.

[NagyDonat]

Yep, I implemented some "fail gracefully" printouts for the situations where the script runs into an unexpected situation, but I wouldn't say that these are supported features of the script. As this code is only intended for one very specific non-user-facing role, I think even removing the error handling is more reasonable than writing tests for it.

[NagyDonat]

@steakhal I implemented most of your review suggestions (the only missing thing is that I'll set up a test to validate that this script can parse AnalyzerOptions.def).
EDIT: I also added tests in 180afc0 , so I think I answered everything.

Please check the wording of the disclaimers added by 461d3db because I'm not too confident about them. (In particular, you suggested that I should add a disclaimer about the malleable state of clang -cc1 flags, but there is no more stable alternative for specifying analyzer flags and -analyzer-config is unchanged since 2017, so I also added an "anti-disclaimer" sentence. There is probably a better solution than this...)