Allowing signed char and unsigned char in random number generation

Contents

1. Introduction

While it may seem perfectly reasonable to generate random bytes in the form of unsigned char, [rand.req.genl] requires that where IntType appears as a template parameter name (such as in uniform_int_distribution), providing unsigned char as an argument causes template instantiation to have undefined behavior. This is functionally equivalent to making the program ill-formed, no diagnostic required.

In 2013, [LWG2326] uniform_int_distribution<unsigned char> should be permitted stated that it's just silly that we have a random number library with no natural way to generate random bytes. As silly as it may be, the issue was closed as NAD (not a defect) because it was perceived as a feature request rather than the resolution of a defect. Some discussion of a writing a paper took place, but that paper never manifested.

It is unclear why the restriction exists in the first place; I was unable to find any discussion in published papers. The restriction dates back at least to [N1932], published 2006. One may suspect that it has to do with unsigned char and other single-byte types being subject to integer promotion (to int), but paradoxically, unsigned short and short are supported despite undergoing promotion.

In any case, signed char and unsigned char should be supported.

2. Motivation

The ability to generate random bytes or octets is extremely valuable for fuzz testing. For example, when testing an implementation of an LZ77 or LZ78 compression algorithm, a user would typically

1. generate a random sequence of bytes,
2. compress that sequence using their algorithm,
3. decompress the result, and
4. compare whether the decompressed bytes are the same as the original bytes.

It is easy to imagine testing methodology that involves random bytes for any kind of codec, network protocol, compiler, syntax highlighter, etc.

3. Design

3.1. Which character types to support

The current restriction which forbids unsigned char is bad, but it's not obvious how much the restriction should be relaxed. There are a few plausible options:

• Permit all standard integers. That is, the current restriction, plus signed char and unsigned char.
• Permit all standard integers and char.
• Permit all integral types.

Permitting all integral types imposes an unreasonable implementation burden. One of the biggest issues is linear_congruential_engine<__int128> assuming the implementation provides an extended 128-bit integer type, which recent version of GCC and Clang do. libstdc++ permits __int128 in most of <random>, but rejects linear_­congruential_engine<__int128> using:

See https://github.com/gcc-mirror/gcc/blob/0635bfb53a145cc005a15657635abf8ea9e6e9ba/libstdc%2B%2B-v3/include/bits/random.h#L521-L522. The difficulty lies in the fact that linear_congruential_engine uses modular arithmetic, which requires N×2-bit division for N-bit integer types. For long long, the existence of __int128 makes this easy, but for __int128, 256-bit division would be required.

Permitting only char in addition to standard integer types seems unmotivated, although there is precedent for this in <charconv>. If char is supported, why not character types in general? If characters types in general are supported, why not integral types and unscoped enumerations? The design becomes somewhat arbitrary.

Overall, permitting only standard integers is an obvious and minimal fix.

3.2. Whether to diagnose unsupported types

It may also be desirable to diagnose the use of unsupported types in <random>, rather than treating it as IFNDR. However, this would raise warnings in substantial amounts of existing code without providing much benefit to users. For instance, GCC has supported all integral types in <random> for years now, and numerous uses of uniform_int_distribution<char> and other disallowed types can be found.

Therefore, it may be best to simply perpetuate the status quo and leave diagnostics up to implementations. This effectively means that [rand.req.gen] provides a minimal list of supported types, which the implementation optionally extends.

4. Implementation experience

At the time of writing, the MSVC STL has a static_assert which prevents instantiation of std::uniform_int_distri­bution<unsigned char>; see:
https://github.com/microsoft/STL/blob/9b021cf66875d2fbf4627aad8db594595aebb148/stl/inc/random#L43

libc++ supports signed char and unsigned char as an extension; see:
https://github.com/llvm/llvm-project/blob/9d075d271ff15ec153ada3413d294540206a15ed/libcxx/include/__random/is_valid.h#L49.

libstdc++ supports any integral type as an extension; see:
https://github.com/gcc-mirror/gcc/blob/0635bfb53a145cc005a15657635abf8ea9e6e9ba/libstdc%2B%2B-v3/include/bits/uniform_int_dist.h#L90-L91.

5. Wording

The changes are relative to [N5032].

Add a feature-test macro to [version.syn] as follows:

Change [rand.req.general] paragraph 1 as follows:

6. References