Skip to content

Commit ed6f6ae

Browse files
committed
[lex.pptoken] Separate out placeholder keywords
The _preprocessing-token_ term is defined as one of another set of grammar productions, with the three placeholder tokens standing out as specific single tokens. Moving the placeholder tokens into their own production simplifies both the _preprocessing-token_ and _keyword_ grammar productions, plus it guarantees a consistency for the set of placeholder between both places. IT also makes it easier for [cpp] to directly cross-reference where needed. Pedantically, the term placeholder token itself is problematic as tokens are a phase 7 concern, where they become keywords. Within the earlier phases where these placeholders are used the correct term is a preprocessing token, so strictly these should be placeholder preprocessing tokens, which is a mouthful. So renaming the term to placeholder keyword, which is hopefully less of a concern as there are no keywords in phase 3--6 to confuse with phase 7 keywords. Finally, clean up the informative list in text of the various kinds of preprocessing tokens, rendering as a list in the source (one kind per line) and adding a cross-reference to each such item.
1 parent 13ba96c commit ed6f6ae

File tree

2 files changed

+40
-26
lines changed

2 files changed

+40
-26
lines changed

source/lex.tex

+38-24
Original file line numberDiff line numberDiff line change
@@ -495,12 +495,10 @@
495495
\indextext{token!preprocessing|(}%
496496
\begin{bnf}
497497
\nontermdef{preprocessing-token}\br
498+
placeholder-keyword\br
498499
header-name\br
499-
import-keyword\br
500-
module-keyword\br
501-
export-keyword\br
502-
identifier\br
503500
pp-number\br
501+
identifier\br
504502
character-literal\br
505503
user-defined-character-literal\br
506504
string-literal\br
@@ -515,12 +513,15 @@
515513
In this document,
516514
glyphs are used to identify
517515
elements of the basic character set\iref{lex.charset}.
518-
The categories of preprocessing token are: header names,
519-
placeholder tokens produced by preprocessing \tcode{import} and \tcode{module} directives
520-
(\grammarterm{import-keyword}, \grammarterm{module-keyword}, and \grammarterm{export-keyword}),
521-
identifiers, preprocessing numbers, character literals (including user-defined character
522-
literals), string literals (including user-defined string literals), preprocessing
523-
operators and punctuators, and single non-whitespace characters that do not lexically
516+
The categories of preprocessing token are:
517+
placeholder keywords\iref{lex.placeholder},
518+
header names\iref{lex.header},
519+
preprocessing numbers\iref{lex.ppnumber},
520+
identifiers\iref{lex.name},
521+
character literals\iref{lex.ccon} including user-defined character literals,
522+
string literals\iref{lex.string} including user-defined string literals,
523+
preprocessing operators and punctuators\iref{lex.operators}, and
524+
single non-whitespace characters that do not lexically
524525
match the other preprocessing token categories.
525526
If a \unicode{0027}{apostrophe} or a \unicode{0022}{quotation mark} character
526527
matches the last category, the program is ill-formed.
@@ -548,17 +549,6 @@
548549
shall have the lexical form of a keyword, an identifier, a literal,
549550
or an operator or punctuator.
550551

551-
\pnum
552-
The \grammarterm{import-keyword} is produced
553-
by processing an \keyword{import} directive\iref{cpp.import},
554-
the \grammarterm{module-keyword} is produced
555-
by preprocessing a \keyword{module} directive\iref{cpp.module}, and
556-
the \grammarterm{export-keyword} is produced
557-
by preprocessing either of the previous two directives.
558-
\begin{note}
559-
None has any observable spelling.
560-
\end{note}
561-
562552
\pnum
563553
If the input stream has been parsed into preprocessing tokens up to a
564554
given character:
@@ -628,6 +618,32 @@
628618
\end{example}
629619
\indextext{token!preprocessing|)}
630620

621+
\rSec1[lex.placeholder]{Placeholder keywords}
622+
\indextext{keyword!placeholder|(}%
623+
624+
\begin{bnf}
625+
\nontermdef{placeholder-keyword}\br
626+
import-keyword\br
627+
module-keyword\br
628+
export-keyword
629+
\end{bnf}
630+
631+
\pnum
632+
Placeholder keywords are produced by preprocessing \keyword{import}
633+
and \keyword{module} directives.
634+
The \grammarterm{import-keyword} is produced
635+
by preprocessing an \keyword{import} directive\iref{cpp.import},
636+
the \grammarterm{module-keyword} is produced
637+
by preprocessing a \keyword{module} directive\iref{cpp.module}, and
638+
the \grammarterm{export-keyword} is produced
639+
by preprocessing either of the previous two directives.
640+
641+
\pnum
642+
\begin{note}
643+
Placeholder keywords do not have any observable spelling.
644+
\end{note}
645+
\indextext{keyword!placeholder|)}%
646+
631647
\rSec1[lex.header]{Header names}
632648

633649
\indextext{header!name|(}%
@@ -960,9 +976,7 @@
960976
\begin{bnf}
961977
\nontermdef{keyword}\br
962978
\textnormal{any identifier listed in \tref{lex.key}}\br
963-
\grammarterm{import-keyword}\br
964-
\grammarterm{module-keyword}\br
965-
\grammarterm{export-keyword}
979+
\grammarterm{placeholder-keyword}
966980
\end{bnf}
967981

968982
\pnum

source/preprocessor.tex

+2-2
Original file line numberDiff line numberDiff line change
@@ -796,7 +796,7 @@
796796
\pnum
797797
The \tcode{module} and \tcode{export} (if it exists) preprocessing tokens
798798
are replaced by the \grammarterm{module-keyword} and
799-
\grammarterm{export-keyword} preprocessing tokens respectively.
799+
\grammarterm{export-keyword} placeholder keywords respectively\iref{lex.placeholder}.
800800
\begin{note}
801801
This makes the line no longer a directive
802802
so it is not removed at the end of phase 4.
@@ -854,7 +854,7 @@
854854
In all three forms of \grammarterm{pp-import},
855855
the \tcode{import} and \tcode{export} (if it exists) preprocessing tokens
856856
are replaced by the \grammarterm{import-keyword} and
857-
\grammarterm{export-keyword} preprocessing tokens respectively.
857+
\grammarterm{export-keyword} placeholder keywords respectively\iref{lex.placeholder}.
858858
\begin{note}
859859
This makes the line no longer a directive
860860
so it is not removed at the end of phase 4.

0 commit comments

Comments
 (0)