More named universal character escapes

Contents

1. Introduction

1.1. History

[P2071R2] introduced named-universal-character escapes into C++23, which makes it possible to use "escape sequences" like \N{NO-BREAK SPACE} within string or character literals. These provide much-needed clarity as compared to \u00A0. Some code points additionally or exclusively have aliases. For example, DELETE (control alias) and DEL (abbreviation alias) correspond to U+007F within the Unicode standard. There is no name for U+007F that is not categorized as an alias.

SG16 voted unanimously to support aliases within a named-universal-character at Prague 2020 when discussing R0:

Match name aliases?

SF	F	N	A	SA
8	2	0	0	0

EWG reaffirmed that decision at the same meeting:

This [named universal character escapes] should further support aliases

SF	F	N	A	SA
18	2	1	0	0

However, some categories of aliases are disallowed in C++, as explained in [P2071R2] §8.2 Name sources:

Unicode aliases provide another critical service. As mentioned above, once assigned, names are immutable. Corrections are only offered by providing an alias. Aliases, accoring to the NamedAliases tables in the Unicode Character Database, come in five varieties:

• correction Aliases for cases where an incorrect assigned name was published. For example, U+FE18 has an assigned name of PRESENTATION FORM FOR VERTICAL RIGHT WHITE LENTICULAR BRAKCET and a correction alias of PRESENTATION FORM FOR VERTICAL RIGHT WHITE LENTICULAR BRACKET (note the typo correction).
• control Aliases for various control characters. For example, NULL for U+0000.
• alternate Aliases for widely used alternate names. For example, BYTE ORDER MARK for U+FEFF.
• figment Aliases for names that were documented, but never accepted in a standard. For example, HIGH OCTET PRESET for U+0081.
• abbreviation Aliases for common abbreviations. For example, NBSP for U+00A0.

The intent is to use the aliases classified as correction, control, and alternate as recognized names.

While the paper does not make it obvious why figment and abbreviation are excluded, the underlying reason is that the C++ standard referenced ISO/IEC 10646 at the time, where figment aliases are not included whatsoever, and where only a subset of the abbreviation aliases in the Unicode standard is included. These issues were discussed at the 2021-11-03 SG16 meeting for R1; see [P2512R0]. R2 was then plenary-approved in 2022 with no support for abbreviation and figment aliases.

Following [P2736R2], the C++ standard references the Unicode standard instead of ISO/IEC 10646, and such a restriction is no longer motivated.

1.2. Inconsistency with other languages

Many design choices of [P2071R2] are ultimately motivated by [P2071R2] §8.5 Existing practice. For example, the \N{...} syntax in C++ is identical to Python and Perl. While C++ shares a syntax, it does not permit the same categories of aliases:

Although this made historical sense, it now feels like an arbitrary restriction.

2. Motivation

2.1. Abbreviations

While the usefulness of some abbreviations is debatable, some of them significantly shorten commonly used code points.

All that is to say that some abbreviations are well-motivated. Furthermore, allowing abbreviations would establish consistency with Python and Perl.

2.2. Figments

There are currently only three aliases classified as figment:

• U+0080 PADDING CHARACTER
• U+0081 HIGH OCTET PRESET
• U+0099 SINGLE GRAPHIC CHARACTER INTRODUCER

While there also exist PAD, HOP, and SGC abbreviations for these characters, these are rather obscure and a user may prefer to use the figment names for additional clarity. Therefore, they should also be supported by C++. This would also establish consistency with Python and Perl.

3. Proposed change

The abbreviation and figment categories should also be permitted within a named-universal-character.

4. Impact on implementations

Permitting abbreviations and figments is essentially trivial. [UnicodeNameAliases] contains a list of all aliases, with 354 abbreviations and 3 figments. This is a drop in the ocean compared to the existing set of names.

Furthermore, the same guarantees of uniqueness (will never conflict with other names) and immutability (will never change) are provided for figment and abbreviation as for the control, alternate, and correction categories. This is generally the case for any names listed in [UnicodeNameAliases]. See [UnicodeAliasStability]. Therefore, upwards compatibility is not threatened.

5. Wording

The following change is relative to [N5008].

Change [lex.universal.char] paragraph 3 as follows:

Change a feature-test macro in [tab:cpp.predefined.ft] as follows:

6. References