💾 Archived View for gemini.bortzmeyer.org › rfc-mirror › rfc7735.txt captured on 2023-01-29 at 04:57:05.
⬅️ Previous capture (2021-11-30)
-=-=-=-=-=-=-
Internet Engineering Task Force (IETF) R. Sparks
Request for Comments: 7735 Oracle
Category: Informational T. Kivinen
ISSN: 2070-1721 INSIDE Secure
January 2016
Tracking Reviews of Documents
Abstract
Several review teams ensure specific types of review are performed on
Internet-Drafts as they progress towards becoming RFCs. The tools
used by these teams to assign and track reviews would benefit from
tighter integration to the Datatracker. This document discusses
requirements for improving those tools without disrupting current
work flows.
Status of This Memo
This document is not an Internet Standards Track specification; it is
published for informational purposes.
This document is a product of the Internet Engineering Task Force
(IETF). It represents the consensus of the IETF community. It has
received public review and has been approved for publication by the
Internet Engineering Steering Group (IESG). Not all documents
approved by the IESG are a candidate for any level of Internet
Standard; see Section 2 of RFC 5741.
Information about the current status of this document, any errata,
and how to provide feedback on it may be obtained at
http://www.rfc-editor.org/info/rfc7735.
Copyright Notice
Copyright (c) 2016 IETF Trust and the persons identified as the
document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents
(http://trustee.ietf.org/license-info) in effect on the date of
publication of this document. Please review these documents
carefully, as they describe your rights and restrictions with respect
to this document. Code Components extracted from this document must
include Simplified BSD License text as described in Section 4.e of
the Trust Legal Provisions and are provided without warranty as
described in the Simplified BSD License.
Sparks & Kivinen Informational [Page 1]
RFC 7735 Review Tracking January 2016
Table of Contents
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2
2. Overview of Current Workflows . . . . . . . . . . . . . . . . 3
3. Requirements . . . . . . . . . . . . . . . . . . . . . . . . 5
3.1. Secretariat Focused . . . . . . . . . . . . . . . . . . . 5
3.2. Review-Team Secretary Focused . . . . . . . . . . . . . . 5
3.3. Reviewer Focused . . . . . . . . . . . . . . . . . . . . 8
3.4. Review Requester and Consumer Focused . . . . . . . . . . 10
3.5. Statistics Focused . . . . . . . . . . . . . . . . . . . 11
4. Security Considerations . . . . . . . . . . . . . . . . . . . 12
5. Informative References . . . . . . . . . . . . . . . . . . . 12
Appendix A. A Starting Point for Django Models Supporting the
Review Tool . . . . . . . . . . . . . . . . . . . . 14
Appendix B. Suggested Features Deferred for Future Work . . . . 15
Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . . 16
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 16
1. Introduction
As Internet-Drafts are processed, reviews are requested from several
review teams. For example, the General Area Review Team (Gen-ART)
and the Security Directorate (SecDir) perform reviews of documents
that are in IETF Last Call. Gen-ART always performs a follow-up
review when the document is scheduled for an IESG Telechat. SecDir
usually performs a follow-up review, but the SecDir secretary may
choose not to request that follow-up if any issues identified at Last
Call are addressed and there are otherwise no major changes to the
document. These teams also perform earlier reviews of documents on
demand. There are several other teams that perform similar services,
often focusing on specific areas of expertise.
The secretaries of these teams manage a pool of volunteer reviewers.
Documents are assigned to reviewers, taking various factors into
account. For instance, a reviewer will not be assigned a document
for which he is an author or shepherd. Reviewers are given a
deadline, usually driven by the end of Last Call or an IESG Telechat
date. The reviewer sends each completed review to the team's mailing
list and to any other lists that are relevant for the document being
reviewed. Often, a thread ensues on one or more of those lists to
resolve any issues found in the review.
The secretaries and reviewers from several teams are using a tool
developed and maintained by Tero Kivinen. Much of its design
predates the modern Datatracker. The application currently keeps its
own data store and learns about documents needing review by
inspecting Datatracker and tools.ietf.org pages. Most of those pages
are easy to parse, but the Last Call pages, in particular, require
Sparks & Kivinen Informational [Page 2]
RFC 7735 Review Tracking January 2016
some effort. Tighter integration with the Datatracker would simplify
the logic used to identify documents that are ready for review, make
it simpler for the Datatracker to associate reviews with documents,
and allow users to reuse their Datatracker credentials. It would
also make it easier to detect other potential review-triggering
events, such as a document entering Working Group (WG) Last Call or
when an RFC's standard level is being changed without revising the
RFC. Tero currently believes this integration is best achieved by a
new implementation of the tool. This document captures requirements
for that reimplementation, with a focus on the workflows that the new
implementation must take care not to disrupt. It also discusses new
features, including changes suggested for the existing tool at its
issue tracker [art-trac].
For more information about the various review teams, see the
following references.
+-------------------------------+---------------------+
| General Area Review Team | [Gen-ART] [RFC6385] |
| Security Directorate | [SecDir] |
| Applications Area Directorate | [AppsDir] |
| Operations Area Directorate | [OPS-dir] |
| Routing Area Directorate | [RTG-dir] |
| MIB Doctors | [MIBdoctors] |
| YANG Doctors | [YANGdoctors] |
+-------------------------------+---------------------+
2. Overview of Current Workflows
This section gives a high-level overview of how the review team
secretaries and reviewers use the existing tool. It is not intended
to be comprehensive documentation of how review teams operate.
Please see the references for those details.
For many teams, the team's secretary periodically (typically once a
week) checks the tool for documents it has identified as ready for
review. The tool compiles a list from Last Call announcements and
IESG Telechat agendas. The secretary creates a set of assignments
from this list and enters them into the reviewer pool, choosing the
reviewers in roughly a round-robin order. That order can be
perturbed by several factors. Reviewers have different levels of
availability. Some are willing to review multiple documents a month.
Others may only be willing to review a document every other month.
The assignment process takes exceptional conditions such as reviewer
vacations into account. Furthermore, secretaries are careful not to
assign a document to a reviewer that is an author, shepherd,
responsible WG chair, or has some other already existing association
with the document. The preference is to get a reviewer with a fresh
Sparks & Kivinen Informational [Page 3]
RFC 7735 Review Tracking January 2016
perspective. The secretary may discover reasons to change
assignments while going through the list of documents. In order to
not cause a reviewer to make a false start on a review, the
secretaries complete the full list of assignments before sending
notifications to anyone. This assignment process can take several
minutes and it is possible for new Last Calls to be issued while the
secretary is making assignments. The secretary typically checks to
see if new documents are ready for review just before issuing the
assignments and updates the assignments if necessary.
Some teams operate in more of a review-on-demand model. The Routing
Area Directorate (RTG-dir), for example, primarily initiates reviews
at the request of a Routing AD. They may also start an early review
at the request of a WG chair. In either case, the reviewers are
chosen manually from the pool of available reviewers driven by
context rather than using a round-robin ordering.
The issued assignments are either sent to the review team's email
list or are emailed directly to the assigned reviewer. The
assignments are reflected in the tool. For those teams handling
different types of reviews (Last Call vs. Telechat, for example), the
secretary typically processes the documents for each type of review
separately, and potentially with different assignment criteria. In
Gen-ART, for example, the Last Call reviewer for a document will
almost always get the follow-up Telechat review assignment.
Similarly, SecDir assigns any re-reviews of a document to the same
reviewer. Other teams may choose to assign a different reviewer.
Reviewers discover their assignments through email or by looking at
their queue in the tool. The secretaries for some teams (such as the
OPS-dir and RTG-dir) insulate their team members from using the tool
directly. These reviewers only work through the review team's email
list or through direct email. On teams that have the reviewers use
the tool directly, most reviewers only check the tool when they see
they have an assignment via the team's email list. A reviewer has
the opportunity to reject the assignment for any reason. While the
tool provides a way to reject assignments, reviewers typically use
email to coordinate rejections with the team secretary. The
secretary will find another volunteer for any rejected assignments.
The reviewer can indicate that the assignment is accepted in the tool
before starting the review, but this feature is rarely used.
The reviewer sends a completed review to the team's email list or
secretary, as well as any other lists relevant to the review, and
usually the draft's primary email alias. For instance, many Last
Call reviews are also sent to the IETF general list. The teams
typically have a template format for the review. Those templates
usually start with a summary of the conclusion of the review.
Sparks & Kivinen Informational [Page 4]
RFC 7735 Review Tracking January 2016
Typical summaries are "ready for publication" or "on the right track
but has open issues". The reviewer (or in the case of teams that
insulate their reviewers, the secretary) uses the tool to indicate
that the review is complete, provides the summary, and has an
opportunity to provide a link to the review in the archives. Note,
however, that having to wait for the document to appear in the
archive to know the link to paste into the tool is a significant
enough impedance that this link is often not provided by the
reviewer. The SecDir secretary manually collects these links from
the team's email list and adds them to the tool.
Occasionally, a document is revised between when a review assignment
is made and when the reviewer starts the review. Different teams can
have different policies about whether the reviewer should review the
assigned version or the current version.
3. Requirements
3.1. Secretariat Focused
o The Secretariat must be able to configure secretaries and
reviewers for review teams (by managing Role records).
o The Secretariat must be able to perform any secretary action on
behalf of a review team secretary (and thus, must be able to
perform any reviewer action on behalf of the reviewer).
3.2. Review-Team Secretary Focused
o A secretary must be able to see what documents are ready for a
review of a given type (such as a Last Call review).
o A secretary must be able to assign reviews for documents that may
not have been automatically identified as ready for a review of a
given type. (In addition to being the primary assignment method
for teams that only initiate reviews on demand, this allows the
secretary to work around errors and handle special cases,
including early review requests.)
o A secretary must be able to work on and issue a set of assignments
as an atomic unit. No assignment should be issued until the
secretary declares the set of assignments complete.
o The tool must support teams that have multiple secretaries. The
tool should warn secretaries that are simultaneously working on
assignments and protect against conflicting assignments being
made.
Sparks & Kivinen Informational [Page 5]
RFC 7735 Review Tracking January 2016
o It must be easy for the secretary to discover that more documents
have become ready for review while working on an assignment set.
o The tool should make preparing the assignment email to the team's
email list easy. For instance, the tool could prepare the
message, give the secretary an opportunity to edit it, and handle
sending it to the team's email list.
o It must be possible for a secretary to indicate that the review
team will not provide a review for a document (or a given version
of a document). This indication should be taken into account when
presenting the documents that are ready for a review of a given
type. This will also make it possible to show on a document's
page that no review is expected from this team.
o A secretary must be able to easily see who the next available
reviewers are, in order.
o A secretary must be able to edit a reviewer's availability, both
in terms of frequency, not-available-until-date, and skip-next-
n-assignments. (See the description of these settings in
Section 3.3.)
o The tool should make it easy for the secretary to see any team
members that have requested to review a given document when it
becomes available for review.
o The tool should make it easy for the secretary to identify that a
reviewer is already involved with a document. The current tool
allows the secretary to provide a regular expression to match
against the document name. If the expression matches, the
document is not available for assignment to this reviewer. For
example, Tero will not be assigned documents matching '^draft-
(kivinen|ietf-tcpinc)-.*