Doc No: ISO/IEC JTC1/SC22/WG21/SD-4 Date: 2017-12-06 Project: JTC1.22.32 Reply To: Herb Sutter Convener, SC22/WG21 Microsoft Corp. 1 Microsoft Way Redmond WA USA 98052 Email: firstname.lastname@example.org Tel: +1-425-707-6533
WG21 Practices and Procedures: The "How We Work" Cheat Sheet
- Purpose and audience
- Code of conduct
- Proposals and papers
- Subgroup responsibility and polls
- Closing plenary session responsibility and polls
- Technical specifications
- Ballots and comments
- How to report a potential issue or concern, at any level of severity
- Escalating a serious "cannot live with" objection, or significant new information
- Other topics
Purpose and audience
This document contains important information that applies to all active WG21 participants, meaning accredited national body experts and especially those who attend our face-to-face meetings. Although this document is also useful to new participants to get up to speed, the material herein covers important procedures and situations that arise regularly in our meetings, email lists, formal ballots, and other participation, and everyone who participates in WG21 is expected to be familiar with this information.
This document summarizes some of WG21's current practices and procedures, in addition to the requirements of the ISO/IEC Directives and JTC 1 Supplement available at the ISO Directives and Policies page.
Note: This document does not repeat the basic material already covered in the following:
- Meetings and participation: How WG21 meetings and teleconferences work; how to get involved and participate at face-to-face meetings; and what the committee email lists (aka "reflectors") are and who can join them.
- How to submit a proposal: How to get initial feedback to explore an idea and craft a proposal; whether an implementation is required; what ISO intellectual property rules to keep in mind; and how to present a proposal at a face-to-face meeting.
- The life of an ISO proposal: From 'cool idea' to 'international standard'
- SD-3: Study group organizational information.
- Meeting wikis: Contains meeting-specific wikis that contain all the meeting's agendas, detailed schedules, and discussion notes, as well as a meeting-nonspecific wiki containing timeless information such as how to join the mailing lists.
Code of conduct
WG21 participants acknowledge the responsibility and value of participating in developing International Standards (ISes) and Technical Specifications (TSes). To ensure a productive and professional environment, WG21 participants follow the ISO Code of Conduct (Code) in our face-to-face meetings, teleconferences, and mailing list discussions.
The Code includes, in part:
- We will identify and escalate disputes in a timely manner to ensure rapid resolution. We will uphold the agreed dispute resolution processes.
- We will act in good faith and with due care and diligence.
- We will avoid collusive or anticompetitive behaviour.
- We will promote a culture of fair and ethical behaviour.
- We are committed to respecting others and the professional culture of international standardization within ISO. In meetings we are committed to:
- Conducting ourselves in a professional manner
- Respecting others and their opinions
- Accepting group decisions
- Ensuring that the views of all (including those whose first language is not that of the meeting) are heard and understood
If you think someone has violated the Code of Conduct at a WG21 meeting or during a mailing list discussion, or have any other concern or question about it, please contact the WG21 Convener (currently Herb Sutter, email@example.com). If the question is related to a specific subgroup, you can also contact that subgroup's chair, who will then also inform the Convener as appropriate.
Note: The committee is made up of many world-class experts who can disagree, can have impassioned debates, and can express criticism of technical alternatives and of group progress or positions. This is not in itself unprofessional or disrespectful when it remains focused on technical arguments, and participants are expected to be able to engage in vigorous and frank technical debate without being quick to take offense; ISO rules are designed to ensure that participants can always speak freely and make their positions clear. However, it is unhelpful to offer criticism without providing a basis or rationale as well as specific suggestions for improvement, and it is always unprofessional and a violation of the Code for committee discussion to overstep into shouting, off-color remarks, or personal disrespect including ad hominem by calling into question another person's character or motives.
Consensus is defined as follows in the ISO/IEC Guide 2:2004, as quoted in the ISO/IEC Directives:
consensus: General agreement, characterized by the absence of sustained opposition to substantial issues by any important part of the concerned interests and by a process that involves seeking to take into account the views of all parties concerned and to reconcile any conflicting arguments.
NOTE Consensus need not imply unanimity.
Importantly, note what consensus is not:
Consensus is not a tyranny of the majority. The majority cannot just say "we have enough votes" and refuse to listen to the concerns of the minority. The ISO Code of Conduct requires the majority to 'ensure that the views of all are heard and understood.'
Consensus is not a tyranny of the minority. The minority, once they have had opportunity to be heard, cannot just say "we still object so you can't proceed" and refuse to accept the consensus of the majority to pursue a different direction. The ISO Code of Conduct requires the minority to 'accept group decisions.'
Consensus does not preclude judicious escalation ("throwing an exception"). In rare cases, where the minority still believe that a serious error of judgment is being made, appropriate escalation can be appropriate in accord with the Escalation procedures covered later in this document; note that the key words are "rare" (not regular), "still" (it was raised before), and "serious." The ISO Code of Conduct encourages 'escalating disputes in a timely manner' with an 'agreed dispute resolution processes.' Escalation can become inappropriate in two ways: (a) when a serious objection is not raised "in a timely manner," such as being withheld until plenary session or formal ballot time rather than being raised consistently throughout the process starting as early as possible; or (b) when a participant or national body regularly uses the escalation process to express a pattern of strong disagreement on topic after topic, which erodes their credibility and is not the purpose of the escalation resolution process (like exception handling, escalation handling is for hard errors, and is not designed for expressing less serious conditions or for what should be ordinary control flow).
Proposals and papers
If a proposal doesn't have a paper, it doesn't exist. Any significant proposal requires a paper, including every proposal that is a design addition or change.
Note: A paper is not needed for minor issues that are about improving standardese wording to implement the decided design. Those do not affect the design itself, and can be put on issues lists only.
High-quality proposal papers. A proposal paper is much more than an "idea" that is written down. A good proposal paper should:
Be example-based: Demonstrate motivating examples of how the code we have to write today is problematic and needs improvement. Show specific examples of how the proposed feature is intended to be used, including how those motivating examples would look if we had the new proposed feature.
Be principle-based: Articulate the design principles for the proposed solution. Show how the proposed solution fits with the rest of the language’s principles and design philosophy (note: explicitly not with the language’s quirks). Ideally include citations of principles/philosophy articulated in The Design and Evolution of C++ (D&E).
Show design alternatives considered along with concrete examples showing why they were not pursued. The author should demonstrate that they have considered the problem reasonably thoroughly, and are not just running with the first idea that occurred to them.
These three points are the main things that make the difference between "someone's cool idea" (which may be interesting but is procedurally unuseful) and "a real proposal" (that can be seriously evaluated and progressed).
Note: For additional details about writing proposals, see also:
- How to submit a proposal: How to get initial feedback to explore an idea and craft a proposal; whether an implementation is required; what ISO intellectual property rules to keep in mind; and how to present a proposal at a face-to-face meeting.
- The life of an ISO proposal: From 'cool idea' to 'international standard'
On-time papers. All papers to be considered at a meeting must be received before that meeting's pre-meeting mailing deadline. The agenda for each meeting includes all papers in mailings since the previous meeting (from the post-meeting mailing of the previous meeting to the pre-meeting mailing of this meeting, inclusive), as well as possibly selected earlier papers that were not sufficiently addressed at a previous meeting in the judgment of the subgroup chair. Requiring papers to be received on time ensures that national body experts have sufficient time to consider the proposals in advance and arrive at the meeting prepared to participate in a productive discussion.
Late papers. A proposal that does not have an on-time paper published since the previous meeting is not actionable and will not be considered during official meeting time; the author is encouraged to submit a revision as an on-time paper for a future meeting. (Note: Followup papers to an on-time paper, such as late or in-meeting rebuttal/elaboration/update papers to an on-time paper, can of course be considered; they are part of considering the on-time paper.)
Corollary: If a topic does not have at least one on-time paper, the whole topic is not on the meeting agenda. Sticking to the published agenda, including the papers published in the pre-meeting mailing, is essential to ensure all stakeholders have full opportunity to be present so that we have orderly progress. Per ISO rules, if a WG meeting were to take action on a topic that was not mentioned on the agenda (including in the topics of on-time papers), a national body could formally escalate an objection in SC 22 and JTC 1 to the entire WG meeting on the grounds that the national body had insufficient notice so as to be able to send any (or the right) participants to participate in that technical discussion topic.
Prepared presenters. A proposal that does not have one of its authors, or another qualified and prepared presenter, to present it is not actionable and will not be considered during official meeting time. There are several reasons we insist a real proposer be present:
The purpose of a presentation is for a person who is an expert in the paper's domain (including in constraints as well as design alternatives considered) to educate a group of general C++ experts. This requires that the presenter be qualified to answer detailed questions about design constraints in the problem domain, design alternatives considered and how they worked out, and so forth at a level that the group as a whole does not already know.
A major outcome of the subgroup discussion is to give feedback and direction to the author regarding how to improve their proposal to create an updated revision for the next meeting. The presenter must be someone who is invested in the proposal and ensure feedback will be used, which means the presenter will personally produce a new revision of the paper based on feedback or else will communicate feedback to the author to do so. Otherwise, the group's time to generate the feedback is meaningless.
Note: In the past, sometimes the subgroup chair has tried to help find a presenter. This is not useful and is actively discouraged, because in practice it results in a presenter who is at best semi-familiar with the topic and who is not personally invested in working on the paper.
Delay vs. bird in the hand. We cannot act on ideas without papers, and we do not significantly delay progress on concrete proposals in order to wait for alternative proposals we might get in the future. If we have an active proposal that is making progress, and an objection is raised that a different competing approach would be better but that other approach does not yet have a paper, the EWG or LEWG subgroup chair may elect to delay the progress of the active proposal by one meeting to give the objector(s) opportunity to bring an on-time proposal paper. Otherwise, if a competing alternative does not have a paper, it does not exist and will not block progress of a proposal that we do have before us.
Note: At early discussion phases, subgroup chairs have discretion to allow limited exploration of alternatives to an active proposal to decide whether a second paper is warranted, or to discuss broader prerequisite topics that are exposed by a paper under discussion.
Using the 1-2 year TS lag effectively. If a proposal has consensus to be in a TS, but does not yet have consensus to be in the IS, that means its progress is delayed by approximately 1-2 years; that is the time it takes to publish the TS, and then wait for usually one year before the TS is proposed to be merged into the IS with any changes based on the TS experience. That window is a limited "grace period" wherein any participants who prefer a different approach than the one in that proposal should be actively working on developing their own alternative proposals. Do not wait, thinking that two years is a long time; it isn't. If by the time the TS is ready to be considered for merging into the IS we do not have alternative proposals actively progressing, the default action to is to move forward with the proposal we have, we will not wait indefinitely for something better.
Subgroup responsibility and polls
Subgroup polls. In subgroup polls, each person in the room can cast one vote. The subgroup chair may take any polls they choose; however, typically the most frequent polls are "5-way" polls where the choices are Strongly Favor, Weakly Favor, Neutral, Weakly Against, and Strongly Against. Notes:
Not everyone in the room needs to vote (even Neutral). Not everyone is interested in every proposal, and participants who are not familiar with the material of a particular poll (did not read the paper and/or were not closely paying attention to the discussion) typically do not vote on that poll.
Anyone voting Against, especially Strongly Against, may be asked for their reasons by the chair. They should be prepared to articulate their rationale, including either what change(s) would address their concern and change their position, or else that they are opposed to pursuing the proposal in any form and no change would change their position.
Study groups. Study groups (SGs) own "incubating" to produce an initial proposal that can then enter the usual design group processing. An SG is formed by the Convener at the recommendation of a design subgroup chair when there appears to be broad interest in a topic, there is not yet a unified initial proposal to bring to the design subgroup (typically because proposals are immature, incomplete, and/or competing), and there is a strong candidate to chair the SG to ensure it operates productively.
Note: For additional information, see SD-3: Study group organizational information.
Design subgroups. The design subgroups (EWG and LEWG) own selecting and recommending feature designs and ship vehicles (TS or IS). Notes:
Discussion should be constructive, help achieve a broad understanding of the proposal within the design subgroup, and to result in polls that give the proposal author meaningful and actionable feedback on whether and how to improve the proposal. Both positive and negative comments are welcome, but should always be constructive and respectful.
If anyone has a concern that a particular new proposal should not take up extended subgroup time (e.g., is not well thought out, is in a direction that previously did not get interest or consensus, or is otherwise unlikely to progress in its current form), they should share their concern privately and in advance with the subgroup chair. The subgroup chair may choose (based on such requests or just on their own judgment) to expedite the proposal's initial handling to make sure a representative overview is heard but then determine whether it has enough subgroup interest before spending extended time exploring the details; for example, the chair might time-box initial discussion and then poll whether the subgroup is interested in discussing the proposal's details further.
The proposal author is expected to participate in the design subgroup to present the proposal, answer questions, participate in design discussion, and implement any changes directed by the group, typically for subsequent revisions of the proposal to be considered at the next meeting. In rare cases, if the proposal author chooses not to implement a change directed by the group, the author should be prepared to defend that decision and bring new information to the next meeting as part of the paper revision; proposals that do not implement design subgroup direction harm their own chances of gaining consensus within the design group and/or in plenary.
When a proposal's design achieves a general consensus of the design subgroup, as determined by the subgroup chair, for its design and ship vehicle, it is forwarded to the wording subgroups. The proposal author is expected to be available in the wording subgroup to make wording changes.
Note: EWG and LEWG currently delegate to SG1 (Concurrency and Parallelism) the responsibility to recommend a design and target ship vehicle for features in its domain.
Wording subgroups. The wording subgroups (CWG and LWG) own the recommendation of the "standardese" wording that implements the intended design. Notes:
If the wording subgroup has design questions or suggestions, those questions are referred back to the design group.
The proposal author is expected to participate in the wording subgroup to answer questions and implement wording changes directed by the group.
When a proposal's wording achieves a general consensus of the wording subgroup, as determined by the subgroup chair, that its wording correctly expresses the intended design, it is forwarded to the closing plenary (whole-WG21) session.
Closing plenary session responsibility and polls
Plenary polls. In plenary polls, each person present can cast a vote if they are the voting representative of a U.S. organization (one vote per INCITS company) or are representing a non-U.S. national body that is a P-member of SC22 (as listed in the ISO LiveLink global directory, one vote per person of a non-U.S. P-member). The Convener may take any polls they choose; however, typically the most frequent polls are "3-way" polls where the choices are Favor, Against, and Neutral. Notes:
The default is to ask if there is objection to unanimous consent. (Unanimous consent means that all participant positions are either Favor or Neutral, and that none are Against). Most polls pass by unanimous consent.
If there is an objection to unanimous consent, the Convener takes a 3-way poll as described above. If there is significant opposition, the Convener will usually ask whether the Against votes are personal positions or national body positions, typically by asking the national body panel chairs who are present, or by taking a two-stage U.S. poll (one vote per INCITS company to determine the U.S. position, followed by one vote per national body to determine overall national consensus).
Closing plenary session. The WG21 closing plenary session owns approval of adding a proposal and its wording to a working draft (TS or IS). The plenary session's responsibility is to decide whether there is consensus, as determined by the Convener, to adopt each proposed change that is recommended by the design and wording subgroups. The plenary session does not actively change the design or entertain extended design discussion, which always happens in the design subgroups. However, the plenary session participants may ask questions like: "does this handle case X?" and "did you consider alternative Y?" For such questions, the meeting chair will ask the appropriate subgroup chair (who may invite comment from the proposal's author) to answer whether that item was considered and to summarize the result of the discussion and the reason for the decision of the subgroup.
Plenary session polls also include recommendations to ship a document for a formal ballot or for publication. If such a poll gets consensus, the document is considered to represent the consensus of WG21 for its current stage of development.
A TS is a beta branch. In ISO, a TS is for a feature that does not yet have consensus for an IS. A TS is a separately published "beta branch" that can ship separately to gain experience -- either with an experimental part of the design, or to get enough overall experience so that participants are comfortable that the overall design works as well as hoped. We usually do not put work into a TS unless WG21 likes the general direction the feature is exploring, and would entertain merging it into the IS in some form if it turns out to work out successfully; features we know we do not intend to put in the IS are not work shipping in beta form either.
Note: The only exception so far is the Transactional Memory TS. In that case, WG21 was asked, and agreed, to take a specification produced by an industry group that included several ISO C++ participants, give it additional feedback, and republish a refined version as an ISO TS without an expectation of merging it into the IS because the material was known to be experimental.
Specific feedback requests. The TS may be just to get more general experience and "bake time." However, if there are any known specific questions on which feedback is desired, those should be mentioned in the TS.
Breaking changes. A major reason to put a feature in a TS is to permit taking further breaking changes before it is standardized. Once a feature is in an IS, it is essentially cast in stone and much more difficult to update with breaking changes. There is never an expectation that a TS will be applied to the IS without change, because enabling subsequent changes is part of the point of releasing a TS first; although it is possible that a TS might be merged into the IS without change if we happened to feel everything was perfect the first time, in practice WG21 has never adopted a TS into the IS without significant change.
Ballots and comments
National bodies and participation. All ISO JTC 1 / SC 22 "P" (Participating) national bodies can send accredited experts to participate in person at WG21 meetings and can vote on ballot documents. National bodies are expected to send experts to participate in person in WG21 if they want to affect the direction of the standard.
Note: Typically that means regularly sending representatives to attend WG21 face-to-face meetings. In the rare case that an NBs is an SC22 P-member but is unable to send representatives to attend WG21 meetings (typically due to severe economic hardship whereby they can participate by remote ballot review but not in person due to travel costs, or due to force majeure such as unexpected changes in governmental travel restrictions), they are still strongly encouraged to participate in WG21 email lists and raise concerns there instead of waiting for ballot comments.
Ballot purpose and structure. An ISO comment ballot is a request for national bodies (NBs) to provide and specific feedback on the material that is in the balloted document. It has two components:
- Vote for general approval/disapproval: Essentially, "do you want to see this document progress (possibly with changes), yes or no?" A Disapproval (No) vote should be used only to express that the national body does not want the document (TS or IS) to progress at all in any form -- to block further progress on all parts of the work item.
Note: In the past, ISO used to have ballot resolution meetings (BRMs) where resolving key comments by a national body could turn a No into a Yes, and national bodies would report whether they changed their vote. We no longer have that procedure or any formal mechanism to change a vote after the ballot. Therefore, No should mean "No, we don't want this TS or IS at all," because sufficient No votes have that effect -- most ballots fail if more than 1/3 of NBs vote No, so voting No only to try to give the NB's comments a higher priority is ineffective (we consider all NB feedback at all times, not just on ballots, and regardless of the Yes/No vote on a ballot) and counterproductive (could actually cause the whole project to fail).
- Comments for specific feedback: Regardless of the approval/disapproval vote, most ballots can contain comments. All editorial and technical comments on the content of that document are in scope. A comment is expected to contain a proposed resolution with sufficient detail to be actionable; if more detail is required than fits easily in the comment form, referencing or attaching a paper is encouraged.
Inappropriate ballot comments. Two specific kinds of comments are therefore not appropriate:
A ballot comment that requests adding an additional feature that is not already in the document is out of scope. New feature proposals are encouraged, but the correct way to bring them to WG21 is by presenting a proposal paper and working to build consensus to add the feature.
A ballot comment that requests a change that was already considered and decided otherwise at a WG21 meeting, and comes from a national body that was present at the meeting and had an opportunity to have their objections be heard and considered, is out of harmony with the ISO Code of Conduct's commitment to 'accept group decisions.' Once the WG has consensus to send a document for ballot, to repeat as an NB comment an objection that previously failed to carry the day is actually be making, not a new technical objection, but an objection to the consensus of the WG.
There are two exceptions:
- If sufficient new information came to light since the last discussion that it is reasonable to believe enough people could change their opinions to affect the previous consensus. That information, of course, requires an on-time paper for the ballot resolution meeting, and ideally that paper would be available early and referenced directly in the ballot comment.
- If the national body really strongly objects to a specific item so much that they are willing to vote No on progressing the whole TS or IS that it's in, and have informed the committee all along of the strength of their feeling (see "Escalation..." below). If WG21 were to ballot a document over those objections, they would know that a No With Comments could happen; but the No vote, and negative comments, would be regarding a previously discussed and decided matter, they should never arise for the first time as a surprise No vote on a ballot.
Ballot comment resolution improves consensus. Any technical changes made during ballot comment resolution have one goal: to improve consensus. In the case of a successful final approval ballot, where after ballot comment resolution the document will be sent directly to ISO for publication without another ballot, any design change made between the ballot and publication will be expected to have near-unanimous consent in subgroups and in plenary. This is because we need to have strong confidence that the design change is not reducing the existing consensus, and we do not have another ballot to find out whether the change would affect the opinion of any NBs who are not represented at the meeting where ballot comment resolution is being done.
How to report a potential issue or concern, at any level of severity
If you notice a potential problem, please follow these steps:
- Keep the mindset that it's a "potential" problem. Very often, even experienced participants find that a potential problem they spot turns out to be a misunderstanding or to have a well-considered rationale. Keeping the view that it's a "potential" problem greatly improves the tone and productiveness of all discussions.
- Describe the problem, and ask whether it was considered.
- If it was considered, and you think you have disagreement with the decision, explain how and why, and based on what (implementation/deployment/user experience are useful points).
- If it wasn't considered, again explain how and why there's a problem, and based on what.
- Discuss whether it can be handled as an issue or whether a paper is needed.
- File the issue or write the paper, or otherwise make sure such a follow-up happens.
- Consider beforehand how severe the problem is. Is the house on fire, or is it just a minor gaffe/inconvenience? To whom? Based on what?
- Listen to what other people say, and consider their viewpoints on the matter. Even if people disagree with your viewpoint, remember to read/listen what they say, and consider why.
In general, these steps should happen:
- As early as possible, preferably during the discussion in a meeting, even more preferably before a meeting on a reflector. This is in accord with the ISO Code of Conduct's "timely manner" expectation.
- If you didn't notice it then, before the straw poll.
- If you didn't notice it then, after the meeting on a reflector.
Escalating a serious "cannot live with" objection, or significant new information
If you feel that a subgroup decision is making a serious mistake so severe that you feel you cannot live with the result, and/or you learn of significant new information not considered in the subgroup discussion that could reasonably change the view of the other participants:
If it is a subgroup decision made during the week and that will be brought forward to plenary that week, you should proactively raise it during the week:< >
On the committee email lists: Ideally the issue should be raised on the relevant subgroup's email list immediately following the subgroup discussion to which the objection appertains, but in no case any later than the deadline of 5pm the evening before the closing plenary session (and posting much sooner than that is strongly encouraged, ideally right after the subgroup decision they feel is wrong, is strongly encouraged).
Within your own national body: This is so that the national body chair has opportunity to caucus with their members and determine whether the position may be a national position (normally the default is no, not a national position, unless it was discussed before or during the meeting to determine that it is a national position).That way, when we come to plenary, everyone has had opportunity to be informed of the serious concern with time to consider it, and when the Convener asks your national body's chair whether the objection is a personal or national position, they will be equipped to answer. Objections that are not so escalated, but are raised or re-raised in plenary, should not be given weight in plenary. (In the rare case of new information that is found 'just the night before' plenary, it has still passed the cutoff for plenary participants to be able to give it informed and prepared consideration in time for plenary, and it is a race condition that we linearize by saying it has failed to be raised before the cutoff, and should be handled per the next bullet including with a followup paper.)
If it is new information found after the deadline (including between meetings), they should raise it:< >
On the committee email lists.
As a national body ballot comment. If a participant does not raise the concern within their national body, and succeed in convincing enough people in their caucus to make it a national body position (by whatever internal rules that particular national body panel uses to determine such), it won't become a national position.In both cases, it should be raised with a discussion paper containing detailed information (at least a draft paper if a final one is not yet available).
Combination of rules. WG21 meetings are concurrent ISO and INCITS meetings, and given that ISO and INCITS have overlapping and sometimes mutually contradictory rules, WG21 operates under a combination of the rules, which we review with ISO from time to time when there is a question. For example, ISO does not allow observers to attend meetings, whereas INCITS does, and WG21 does allow observers.
Protected materials. The following materials are not publicly available and are always password-protected:
- Documents containing the final text of a TS/IS, on which ISO asserts copyright.
- Meeting records of discussion, meeting wikis, and non-SG committee email lists (aka reflectors), which often include personal positions and discussion.