| Document #: | P3394R4 [Latest] [Status] |
| Date: | 2025-06-19 |
| Project: | Programming Language C++ |
| Audience: |
CWG, LWG |
| Reply-to: |
Wyatt Childers <[email protected]> Dan Katz <[email protected]> Barry Revzin <[email protected]> Daveed Vandevoorde <[email protected]> |
Since [P3394R3]: wording. Removed ability to
put attributes and annotations into the same
attribute-specifier.
Since [P3394R2]: wording, including rebasing
off of [P2996R13]. Also removed
annotate to streamline the process,
it can always be added later.
Since [P3394R1]:
is_annotation,
annotations_of,
annotations_of_with_type (a binary
function that returns a vector<info>),
and annotate.Since [P3394R0]:
Ever since writing [P1240R0] (Scalable Reflection in C++), but more so since [P2996R0] (Reflection for C++26), we have been requested to add a capability to annotate declarations in a way that reflection can observe. For example, Jeremy Ong presented compelling arguments in a post to the SG7 reflector. Corentin Jabot also noticed the need while P1240 was evolving and wrote [P1887R0] (Typesafe Reflection on attributes), which proposes syntax not entirely unlike what we present here.
In early versions of P2996 (and P1240 before that), a workaround was to encode properties in the template arguments of alias template specializations:
template <typename T, auto... Annotations> using Noted = T; struct C { Noted<int, 1> a; Noted<int*, some, thing> b; };
It was expected that something like type_of(^^C::a)
would produce a reflection of Noted<int, 1>
and that can be taken apart with metafunctions like
template_arguments_of — which both
preserves the type as desired (a is
still an
int) and
allows reflection queries to get at the desired annotations (the
1,
some, and
thing in this case).
There are problems with this approach, unfortunately:
In this paper, we propose simple mechanisms that more directly support the ability to annotate C++ constructs.
We’ll start with a few motivating examples for the feature. We’ll describe the details of the feature in the subsequent section.
These examples are inspired from libraries in other programming languages that provide some mechanism to annotate declarations.
Rust’s clap library provides a way to add annotations to declarations to help drive how the parser is declared. We can now do the same:
struct Args { [[=clap::Help("Name of the person to greet")]] [[=clap::Short, =clap::Long]] std::string name; [[=clap::Help("Number of times to greet")]] [[=clap::Short, =clap::Long]] int count = 1; }; int main(int argc, char** argv) { Args args = clap::parse<Args>(argc, argv); for (int i = 0; i < args.count; ++i) { std::cout << "Hello " << args.name << '\n'; } }
Here, we provide three types
(Short,
Long, and
Help) which help define how these
member variables are intended to be used on the command-line. This is
implemented on top of Lyra.
When run:
$ demo -h USAGE: demo [-?|-h|--help] [-n|--name <name>] [-c|--count <count>] Display usage information. OPTIONS, ARGUMENTS: -?, -h, --help -n, --name <name> Name of the person to greet -c, --count <count> Number of times to greet $ demo -n wg21 --count 3 Hello wg21 Hello wg21 Hello wg21
While Short and
Long can take explicit values, by
default they use the first letter and whole name of the member that they
annotate.
The core of the implementation is that parse<Args>
loops over all the non-static data members of
Args, then finds all the
clap-related annotations and invokes
them:
template <typename Args> auto parse(int argc, char** argv) -> Args { Args args; auto cli = lyra::cli(); // ... template for (constexpr info M : nonstatic_data_members_of(^^Args)) { auto id = std::string id(identifier_of(mem)); auto opt = lyra::opt(args.[:M:], id); template for (constexpr info A : annotations_of(M)) { if constexpr (parent_of(type_of(A)) == ^^clap) { // for those annotions that are in the clap namespace // invoke them on our option extract<[:type_of(A):]>(A).apply_annotation(opt, id); } } cli.add_argument(opt); } // ... };
So, for instance, Short would be
implemented like this:
namespace clap { struct ShortArg { // optional isn't structural yet but let's pretend optional<char> value; constexpr auto operator()(char c) const -> ShortArg { return {.value=c}; }; auto apply_annotation(lyra::opt& opt, std::string_view id) const -> void { char first = value.value_or(id[0]); opt[std::string("-") + first]; } }; inline constexpr auto Short = ShortArg(); }
Overall, a fairly concise implementation for an extremely user-friendly approach to command-line argument parsing.
The pytest framework comes with a decorator to parametrize test functions. We can now do the same:
namespace N { [[=parametrize({ Tuple{1, 1, 2}, Tuple{1, 2, 3} })]] void test_sum(int x, int y, int z) { std::println("Called test_sum(x={}, y={}, z={})", x, y, z); } struct Fixture { Fixture() { std::println("setup fixture"); } ~Fixture() { std::println("teardown fixture"); } [[=parametrize({Tuple{1}, Tuple{2}})]] void test_one(int x) { std::println("test one({})", x); } void test_two() { std::println("test two"); } }; } int main() { invoke_all<^^N>(); }
When run, this prints:
Called test_sum(x=1, y=1, z=2) Called test_sum(x=1, y=2, z=3) setup fixture test one(1) teardown fixture setup fixture test one(2) teardown fixture setup fixture test two teardown fixture
Here, parametrize returns a value
that is some specialization of
Parametrize (which is basically an
array of tuples, except that
std::tuple
isn’t structural so the implementation rolls its own).
The rest of the implementation looks for all the free functions named
test_* or
nonstatic member functions of class types that start with
test_* and
invokes them once, or with each parameter, depending on the presence of
the annotation. That looks like this:
consteval auto parametrization_of(std::meta::info M) -> std::meta::info { for (auto a : annotations_of(M)) { auto t = type_of(a); if (has_template_arguments(t) and template_of(t) == ^^Parametrize) { return a; } } return std::meta::info(); } template <std::meta::info M, class F> void invoke_single_test(F f) { constexpr auto A = parametrization_of(M); if constexpr (A != std::meta::info()) { // if we are parametrized, pull out that value // and for each tuple, invoke the function // this is basically calling std::apply on an array of tuples constexpr auto Params = extract<[:type_of(A):]>(A); for (auto P : Params) { P.apply(f); } } else { f(); } } template <std::meta::info Namespace> void invoke_all() { template for (constexpr std::meta::info M : members_of(Namespace)) { if constexpr (is_function(M) and identifier_of(M).starts_with("test_")) { invoke_single_test<M>([:M:]); } else if constexpr (is_type(M)) { template for (constexpr std::meta::info F : nonstatic_member_functions_of(M)) { if constexpr (identifier_of(F).starts_with("test_")) { invoke_single_test<F>([&](auto... args){ typename [:M:] fixture; fixture.[:F:](args...); }); } } } } }
Rust’s serde library is a framework for serialization and deserialization. It is easy enough with reflection to do member-wise serialization. But how do you opt into that? An annotation provides a cheap mechanism of doing just that (built on top of Boost.Json):
struct [[=serde::derive]] Point { int x, y; };
Allowing:
// prints {"x":1,"y":2} std::cout << boost::json::value_from(Point{.x=1, .y=2});
But opting in is just the first thing you might want to do with
serialization. You might also, for instance, want to change how fields
are serialized. serde provides a lot
of attributes to do so. The easiest to look at is
rename, which uses the provided
string instead of the name of the non-static data member:
struct [[=serde::derive]] Person { [[=serde::rename("first name")]] std::string first; [[=serde::rename("last name")]] std::string last; };
Which leads to:
// prints {"first name":"Peter","last name":"Dimov"} std::cout << boost::json::value_from(Person{.first="Peter", .last="Dimov"});
The implementation for these pieces is fairly straightforward. We
provide an opt-in for the value conversion function when the serde::derive
annotation is present. In that case, we walk all the non-static data
members and write them into the boost::json::value
output. If a serde::Rename
annotation is present, we use that instead of the data member’s
name:
namespace serde { inline constexpr struct{} derive{}; struct rename { char const* field; }; } namespace boost::json { template <class T> requires (has_annotation(^^T, serde::derive)) void tag_invoke(value_from_tag const&, value& v, T const& t) { auto& obj = v.emplace_object(); template for (constexpr auto M : nonstatic_data_members_of(^^T)) { constexpr auto field = []{ std::optional<std::meta::info> res; for (std::meta::info r : annotations_of_with_type(M, ^^serde::rename)) { if (res and *res != r) { throw "oops"; } res = r; } return res .transform([](std::meta::info r){ return std::string_view(extract<serde::rename>(r).field); }) .value_or(identifier_of(M)); }(); obj[field] = boost::json::value_from(t.[:M:]); } } }
You can imagine extending this out to support a wide variety of other
serialization-specific attributes that shouldn’t otherwise affect the
C++ usage of the type. For instance, a more complex approach
additionally supports the
skip_serializing_if annotation while
first collecting all serde
annotations into a struct.
The core idea is that an annotation is a compile-time value that can be associated with a construct to which attributes can appertain. Annotation and attributes are somewhat related ideas, and we therefore propose a syntax for the former that builds on the existing syntax for the latter.
At its simplest:
struct C { [[=1]] int a; };
Syntactically, an annotation is an attribute of the form
= expr where
expr is a constant-expression
(which syntactically excludes, e.g., comma-expression)
to which the glvalue-to-prvalue conversion has been applied if the
expression wasn’t a prvalue to start with.
Currently, we require that an annotation has structural type because
we’re going to return annotations through std::meta::info,
and currently all reflection values must be structural.
Attributes are very close in spirit to annotations. So it made sense to piggy-back on the attribute syntax to add annotations. Existing attributes are designed with fairly open grammar and they can be ignored by implementations, which makes it difficult to connect them to user code. Given a declarations like:
[[nodiscard, gnu::always_inline]] [[deprecated("don't use me")]] void f();
What could reflecting on f
return? Because attributes are ignorable, an implementation might simply
ignore them. Additionally, there is no particular value associated with
any of these attributes that would be sensible to return. We’re limited
to returning either a sequence of strings. Or, with P3294, token sequences.
But it turns out to be quite helpful to preserve the actual values without requiring libraries to do additional parsing work. Thus, we need to distinguish annotations (whose values we need to preserve and return back to the user) from attributes (whose values we do not). Thus, we looked for a sigil introducing a general expression.
Originally, the plus sign
(+) was
considered (as in P1887), but it is not ideal because a prefix
+ has a
meaning for some expressions and not for others, and that would not
carry over to the attribute notation. A prefix
= was found
to be reasonably meaningful in the sense that the annotation “equals”
the value on the right, while also being syntactically unambiguous. We
also discussed using the reflection operator
(^^) as an
introducer (which is attractive because the annotation ultimately comes
back to the programmer as a reflection value), but that raised questions
about an annotation itself being a reflection value (which is not
entirely improbable).
As such, this paper proposes annotations as distinct from attributes,
introduced with a prefix
=.
We propose the following set of library functions to work with annotations:
namespace std::meta { consteval bool is_annotation(info); consteval vector<info> annotations_of(info item); // (1) consteval vector<info> annotations_of_with_type(info item, info type); // (2) }
is_annotation checks whether a
particular reflection represents an annotation.
We provide two functions to retrieve all the annotations of a particular item:
annotations_of(item)
returns all the annotations on
item.annotations_of_item_with_type(item, type)
returns all the annotations a on
item such that type_of(a) == type.In earlier revisions, these were overloads (both spelled
annotations_of), but starting in R2
the second function was renamed to add clarity on what the second
parameter actually means. There were previously additional functions
proposed for ergonomic reasons, but those were removed during the Hagenberg
meeting.
Annotations can be repeated:
[[=42, =42]] int x; static_assert(annotations_of(^^x).size() == 2);
Annotations spread over multiple declarations of the same entity accumulate:
[[=42]] int f(); [[=24]] int f(); static_assert(annotations_of(^^f).size() == 2);
Annotations follow appertainance rules like attributes, but shall not appear in the attribute-specifier-seq of a type-specifier-seq or an empty-declaration:
struct [[=0]] S {}; // Okay: Appertains to S. [[=42]] int f(); // Okay: Appertains to f. int f[[=0]] (); // Ditto. int [[=24]] f(); // Error: Cannot appertain to int. [[=123]]; // Error: No applicable construct.
Earlier revisions of the paper allowed annotations and attributes to
appear within the same
attribute-specifier:
[[nodiscard, =42]] int f();
However, in Sofia, it was pointed out that for parsing purposes,
allowing intermixing of annotations and attributes is challenging
because it makes it harder to determine if = T < x, y
means that y is the a new attribute
or part of a template argument list. Since there was no motivation to
allow intermixing them and clear benefits to the separation, we’re
mandating that a given
attribute-specifier is
either all attributes or all annotations. This also simplifies the
grammar and the wording. The above, if desired, can be written as:
[[nodiscard]] [[=42]] int f();
The core language feature and the basic query functions have been
implemented in the EDG front end and in Bloomberg’s P2996 Clang fork
(with option -freflection-latest),
both available on Compiler Explorer.
As evidenced in the motivating examples earlier, there is a lot of value in this proposal even in this simple form. However, there is more to consider when it comes to annotations.
This proposal right now lets us unconditionally add an annotation to a type:
struct [[=X]] Always;
But it does not let us conditionally add an annotation to a type:
template <class T> struct /* X only for some T */ Sometimes;
Or to really generalize annotations. For instance, in the clap
example earlier, our example showed usage with
clap::Short
and
clap::Long.
What if somebody wants to compose these into their own annotation that
attaches both
clap::Short
and
clap::Long
to a declaration?
More broadly, there is clear value in having an annotation be able to be invoked by the declaration itself. Doing so allows the two uses above easily enough. An interesting question, though, is whether this callback (syntax to be determined) is invoked at the beginning of the declaration or at the end of the declaration. For annotations on classes, this would be before the class is complete or after the class is complete. Before completeness allows the class to observe the annotation during instantiation. After completeness allows the annotation callback to observe properties of the type. In some sense, Herb Sutter’s [P0707R4] (Metaclasses: Generative C++) was adding annotations on classes, invoked on class completeness, that allow mutation of the class.
One concrete, simpler example. We can, with this proposal as-is,
create a Debug annotation that a
user can add to their type and a specialization of std::formatter for
all types that have a Debug
annotation as follows:
template <auto V> struct Derive { }; template <auto V> inline constexpr Derive<V> derive; inline constexpr struct{} Debug; template <class T> requires (has_annotation(^^T, derive<Debug>)) struct std::formatter<T> { // ... }; struct [[=derive<Debug>]] Point { int x; int y; }; int main() { auto p = Point{.x=1, .y=2}; // prints p=Point{.x=1, .y=2} std::println("p={}", p); }
This works, but it’s not really the ideal way of doing it.
This could still run into potential issues with ambiguous specialization
of std::formatter.
Better would be to allow the Debug
annotation to, at the point of completion of
Point, inject an explicit
specialization of std::formatter.
This would rely both on the ability for the annotation to be called back
and language support for such injection (see [P3294R2] (Code
Injection with Token Sequences)).
There are still open questions as to how to handle such callbacks.
Does an annotation that gets called back merit different syntax from an
annotation that doesn’t? Can it mutate the entity that it is attached
to? How do we name the potential callbacks? Should the callback be
registered implicitly (e.g., if C of type
X with member X::annotate_declaration(...)
appears, that member is automatically a callback invoked when an entity
is first declared with an annotation of type
X) or explicitly (e.g., calling
annotated_declaration_callback(^^X, X_handler)
would cause X_handler(...) to
be invoked when an entity is first declared with an annotation of type
X).
The wording is relative to [P2996R13].
Change 6.8.2 [basic.fundamental] to add “annotation” to the list of reflection kinds:
x A value of type
std::meta::infois called a reflection. There exists a unique null reflection; every other reflection is a representation of
- (x.1) a value of scalar type ([temp.param]),
- (x.2) an object with static storage duration ([basic.stc]),
- (x.3) a variable ([basic.pre]),
- (x.4) a structured binding ([dcl.struct.bind]),
- (x.5) a function ([dcl.fct]),
- (x.6) an enumerator ([dcl.enum]),
- (x.7) an annotation ([dcl.attr.grammar]),
- (x.8) a type alias ([dcl.typedef]),
- (x.9) a type ([basic.types]),
- (x.10) a class member ([class.mem]),
- (x.11) an unnamed bit-field ([class.bit]),
- (x.12) a primary class template ([temp.pre]),
- (x.13) a function template ([temp.pre]),
- (x.14) a primary variable template ([temp.pre]),
- (x.15) an alias template ([temp.alias]),
- (x.16) a concept ([temp.concept]),
- (x.17) a namespace alias ([namespace.alias]),
- (x.18) a namespace ([basic.namespace.general]),
- (x.19) a direct base class relationship ([class.derived.general]), or
- (x.20) a data member description ([class.mem.general]).
Update annotation equality in 7.6.10 [expr.eq]/5+:
5+ If both operands are of type
std::meta::info, they compare equal if both operands
- (5+.1) are null reflection values,
- (5+.2) represent values that are template-argument-equivalent ([temp.type]),
- (5+.3) represent the same object,
- (5+.4) represent the same entity,
- (5+.5) represent the same annotation ([dcl.attr.annotation]),
- (5+.6) represent the same direct base class relationship, or
- (5+.7) represent equal data member descriptions ([class.mem.general]),
and they compare unequal otherwise.
Extend the grammar in 9.13.1 [dcl.attr.grammar]:
1 Attributes and annotations specify additional information for various source constructs such as types, variables, names, blocks, or translation units.
attribute-specifier-seq: attribute-specifier-seqopt attribute-specifier attribute-specifier: [ [ attribute-using-prefixopt attribute-list ] ] + [ [ annotation-list ] ] alignment-specifier alignment-specifier: alignas ( type-id ...opt ) alignas ( constant-expression ...opt ) attribute-using-prefix: using attribute-namespace : attribute-list: attributeopt attribute-list , attributeopt attribute ... attribute-list , attribute ... + annotation-list: + annotation ...opt + annotation-list , annotation ...opt attribute: attribute-token attribute-argument-clauseopt + annotation: + = constant-expression
Clarify the restriction on no attributes in 9.13.1 [dcl.attr.grammar]/4:
4 In an
attribute-list, an ellipsis may appear only following anattributeif thatattribute’s specification permits it. Anattributefollowed by an ellipsis is a pack expansion. Anattribute-specifierthat contains anattribute-listwith noattributes has no effect. The order in which theattribute-tokensappear in anattribute-listis not significant. […]* An
annotationfollowed by an ellipsis is a pack expansion ([temp.variadic]).
Add a new subclause under 9.13 [dcl.attr] called [dcl.attr.annotation] “Annotations”:
1 An annotation may be applied to any declaration of a type, type alias, variable, function, namespace, enumerator,
base-specifier, or non-static data member.2 Let
Ebe the expressionstd::meta::reflect_constant(constant-expression).Eshall be a constant expression; the result ofEis the underlying constant of the annotation.3 Each
annotationproduces a unique annotation.4 Substituting into an
annotationis not in the immediate context.[ Example 1:[[=1]] void f(); [[=2, =3, =2]] void g(); void g [[=4, =2]] ();
fhas one annotation andghas five annotations. These can be queried with metafunctions such asstd::meta::annotations_of([meta.reflect.annotation]). — end example ][ Example 2:— end example ]template <class T> [[=T::type()]] void f(T t); void f(int); void g() { f(0); // OK f('0'); // error, substituting into the annotation results in an invalid expression }
Change the pack expansion rule in 13.7.4 [temp.variadic]/5.9:
5 […] Pack expansions can occur in the following contexts:
- (5.9) In an
attribute-list([dcl.attr.grammar]); the pattern is anattribute. In anannotation-list; the pattern is anannotation.
Add to the <meta>
synopsis:
namespace std::meta { // ... consteval bool is_bit_field(info r); consteval bool is_enumerator(info r); + consteval bool is_annotation(info r); // .. // [meta.reflection.annotation], annotation reflection + consteval vector<info> annotations_of(info item); + consteval vector<info> annotations_of_with_type(info item, info type); }
Add is_annotation to
[meta.reflection.queries]:
consteval bool is_enumerator(info r); + consteval bool is_annotation(info r);1 Returns:
trueifrrepresents an enumerator or annotation, respectively. Otherwise,false.
Update the meanings of
has-type,
type_of and
value_of in
[meta.reflection.queries]:
consteval bool has-type(info r); // exposition only1 Returns:
trueifrrepresents a value, annotation, object, variable, function whose type does not contain an undeduced placeholder type and that is not a constructor or destructor, enumerator, non-static data member, unnamed bit-field, direct base class relationship, or data member description. Otherwise,false.consteval info type_of(info r);2 Constant When:
has-type(r)istrue.3 Returns:
- (3.1) If
rrepresents a value, object, variable, function, non-static data member, or unnamed bit-field, then the type of what is represented byr.
- (3.*) Otherwise, if
rrepresents an annotation, thentype_of(constant_of(r)).
- (3.2) Otherwise, if
rrepresents an enumeratorNof an enumerationE, then:- (3.3) Otherwise, if
rrepresents a direct base class relationship, then a reflection of the type of the direct base class.- (3.4) Otherwise, for a data member description (
T,N,A,W,NUA) ([class.mem.general]), a reflection of the typeT.consteval info object_of(info r);4 Constant When: […]
5 Returns: […]
consteval info constant_of(info r);6 Let
Rbe a constant expression of typeinfosuch thatR == ristrue. Ifrrepresents an annotation, then letCbe its underlying constant.7 Constant When: Either
rrepresents an annotation or[: R :]is a validsplice-expression([expr.prim.splice]).8 Effects: Equivalent to:
+ if constexpr (is_annotation(R)) { + return C; + } else { return reflect_constant([: R :]); + }
Add the new section [meta.reflection.annotation]:
consteval vector<info> annotations_of(info item);1 Constant When:
itemrepresents a type, type alias, variable, function, namespace, enumerator, direct base class relationship, or non-static data member.2 Let
Ebe
- (2.1) the corresponding
base-specifierifitemrepresents a direct base class relationship,- (2.2) otherwise, the entity represented by
item.3 Returns: A
vectorcontaining all of the reflectionsRrepresenting each annotation applying to each declaration ofEthat precedes either some point in the evaluation context ([expr.const]) or a point immediately following theclass-specifierof the outermost class for which such a point is in a complete-class context.For any two reflections
R1andR2in the returnedvector, if the annotation represented byR1precedes the annotation represented byR2, thenR1appears beforeR2. IfR1andR2represent annotations from the same translation unitT, any element in the returnedvectorbetweenR1andR2represents an annotation fromT. [ Note 1: The order in which two annotations appear is otherwise unspecified. — end note ][ Example 1:— end example ][[=1]] void f(); [[=2, =3]] void g(); void g [[=4]] (); static_assert(annotations_of(^^f).size() == 1); static_assert(annotations_of(^^g).size() == 3); static_assert([: constant_of(annotations_of(^^g)[0]) :] == 2); static_assert(extract<int>(annotations_of(^^g)[1]) == 3); static_assert(extract<int>(annotations_of(^^g)[2]) == 4); struct Option { bool value; }; struct C { [[=Option{true}]] int a; [[=Option{false}]] int b; }; static_assert(extract<Option>(annotations_of(^^C::a)[0]).value); static_assert(!extract<Option>(annotations_of(^^C::b)[0]).value); template <class T> struct [[=42]] D { }; constexpr std::meta::info a1 = annotations_of(^^D<int>)[0]; constexpr std::meta::info a2 = annotations_of(^^D<char>)[0]; static_assert(a1 != a2); static_assert(constant_of(a1) == constant_of(a2)); [[=1]] int x, y; static_assert(annotations_of(^^x)[0] == annotations_of(^^y)[0]);consteval vector<info> annotations_of_with_type(info item, info type);4 Constant When:
- (4.1)
annotations_of(item)is a constant subexpression and- (4.2)
dealias(type)represents a type that is complete from some point in the evaluation context.5 Returns: A
vectorcontaining each elementeofannotations_of(item)whereremove_const(type_of(e)) == remove_const(type)istrue, preserving their order.