Skip to content
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.

Commit 91dd967

Browse files
committedMar 20, 2025·
[lex.pptoken] Separate out placeholder keywords
The _preprocessing-token_ term is defined as one of another set of grammar productions, with the three placeholder tokens standing out as specific single tokens. Moving the placeholder tokens into their own production simplifies both the _preprocessing-token_ and _keyword_ grammar productions, plus it guarantees a consistency for the set of placeholder between both places. IT also makes it easier for [cpp] to directly cross-reference where needed. Pedantically, the term placeholder token itself is problematic as tokens are a phase 7 concern, where they become keywords. Within the earlier phases where these placeholders are used the correct term is a preprocessing token, so strictly these should be placeholder preprocessing tokens, which is a mouthful. So renaming the term to placeholder keyword, which is hopefully less of a concern as there are no keywords in phase 3--6 to confuse with phase 7 keywords. Finally, clean up the informative list in text of the various kinds of preprocessing tokens, rendering as a list in the source (one kind per line) and adding a cross-reference to each such item.
1 parent 0bd8d94 commit 91dd967

File tree

2 files changed

+40
-26
lines changed

2 files changed

+40
-26
lines changed
 

‎source/lex.tex

+38-24
Original file line numberDiff line numberDiff line change
@@ -498,12 +498,10 @@
498498
\indextext{token!preprocessing|(}%
499499
\begin{bnf}
500500
\nontermdef{preprocessing-token}\br
501+
keyword-placeholder\br
501502
header-name\br
502-
import-keyword\br
503-
module-keyword\br
504-
export-keyword\br
505-
identifier\br
506503
pp-number\br
504+
identifier\br
507505
character-literal\br
508506
user-defined-character-literal\br
509507
string-literal\br
@@ -518,12 +516,15 @@
518516
In this document,
519517
glyphs are used to identify
520518
elements of the basic character set\iref{lex.charset}.
521-
The categories of preprocessing token are: header names,
522-
placeholder tokens produced by preprocessing \tcode{import} and \tcode{module} directives
523-
(\grammarterm{import-keyword}, \grammarterm{module-keyword}, and \grammarterm{export-keyword}),
524-
identifiers, preprocessing numbers, character literals (including user-defined character
525-
literals), string literals (including user-defined string literals), preprocessing
526-
operators and punctuators, and single non-whitespace characters that do not lexically
519+
The categories of preprocessing token are:
520+
keyword plaeholders\iref{lex.placeholder},
521+
header names\iref{lex.header},
522+
preprocessing numbers\iref{lex.ppnumber},
523+
identifiers\iref{lex.name},
524+
character literals\iref{lex.ccon} including user-defined character literals,
525+
string literals\iref{lex.string} including user-defined string literals,
526+
preprocessing operators and punctuators\iref{lex.operators}, and
527+
single non-whitespace characters that do not lexically
527528
match the other preprocessing token categories.
528529
If a \unicode{0027}{apostrophe} or a \unicode{0022}{quotation mark} character
529530
matches the last category, the program is ill-formed.
@@ -551,17 +552,6 @@
551552
shall have the lexical form of a keyword, an identifier, a literal,
552553
or an operator or punctuator.
553554

554-
\pnum
555-
The \grammarterm{import-keyword} is produced
556-
by processing an \keyword{import} directive\iref{cpp.import},
557-
the \grammarterm{module-keyword} is produced
558-
by preprocessing a \keyword{module} directive\iref{cpp.module}, and
559-
the \grammarterm{export-keyword} is produced
560-
by preprocessing either of the previous two directives.
561-
\begin{note}
562-
None has any observable spelling.
563-
\end{note}
564-
565555
\pnum
566556
If the input stream has been parsed into preprocessing tokens up to a
567557
given character:
@@ -631,6 +621,32 @@
631621
\end{example}
632622
\indextext{token!preprocessing|)}
633623

624+
\rSec1[lex.placeholder]{Keyword placeholders}
625+
\indextext{placeholder!keyword|(}%
626+
627+
\begin{bnf}
628+
\nontermdef{keyword-placeholder}\br
629+
export-keyword\br
630+
import-keyword\br
631+
module-keyword
632+
\end{bnf}
633+
634+
\pnum
635+
Keyword placeholders are produced by preprocessing \keyword{import}
636+
and \keyword{module} directives.
637+
The \grammarterm{import-keyword} is produced
638+
by preprocessing an \keyword{import} directive\iref{cpp.import},
639+
the \grammarterm{module-keyword} is produced
640+
by preprocessing a \keyword{module} directive\iref{cpp.module}, and
641+
the \grammarterm{export-keyword} is produced
642+
by preprocessing either of the previous two directives.
643+
644+
\pnum
645+
\begin{note}
646+
Keyword placeholders do not have any observable spelling.
647+
\end{note}
648+
\indextext{placeholder!keyword|(}%
649+
634650
\rSec1[lex.header]{Header names}
635651

636652
\indextext{header!name|(}%
@@ -964,9 +980,7 @@
964980
\begin{bnf}
965981
\nontermdef{keyword}\br
966982
\textnormal{any identifier listed in \tref{lex.key}}\br
967-
\grammarterm{import-keyword}\br
968-
\grammarterm{module-keyword}\br
969-
\grammarterm{export-keyword}
983+
\grammarterm{keyword-placeholder}
970984
\end{bnf}
971985

972986
\pnum

‎source/preprocessor.tex

+2-2
Original file line numberDiff line numberDiff line change
@@ -1233,7 +1233,7 @@
12331233
\pnum
12341234
The \tcode{module} and \tcode{export} (if it exists) preprocessing tokens
12351235
are replaced by the \grammarterm{module-keyword} and
1236-
\grammarterm{export-keyword} preprocessing tokens respectively.
1236+
\grammarterm{export-keyword} keyword placeholders respectively\iref{lex.placeholder}.
12371237
\begin{note}
12381238
This makes the line no longer a directive
12391239
so it is not removed at the end of phase 4.
@@ -1291,7 +1291,7 @@
12911291
In all three forms of \grammarterm{pp-import},
12921292
the \tcode{import} and \tcode{export} (if it exists) preprocessing tokens
12931293
are replaced by the \grammarterm{import-keyword} and
1294-
\grammarterm{export-keyword} preprocessing tokens respectively.
1294+
\grammarterm{export-keyword} keyword placeholders respectively\iref{lex.placeholder}.
12951295
\begin{note}
12961296
This makes the line no longer a directive
12971297
so it is not removed at the end of phase 4.

0 commit comments

Comments
 (0)
Please sign in to comment.