Draft print and archival format for Institute documents — submissions
The 18 submissions received, published in full with declared interests and secretariat responses.
§2Submissions and responses
18 submissions were received. Each is published in full below with its declared interest, the secretariat response and the disposition. The Institute publishes submissions it did not accept in the same form as those it did.
A declared interest is published without being tested
This is a submission on print and archival format for Institute documents.
A respondent declares their own interest and the declaration is published as given. Nothing states what the Institute does where a declaration appears incomplete on its face.
The respondent proposes that the position be stated, whatever it is.
The secretariat accepts this submission.
The policy now states that declarations are published as made, that the Institute does not investigate them, and that where a declaration appears incomplete the respondent is asked to complete it before publication. That the Institute does not verify is now on the face of every submission page.
There is no route to challenge a disposition
Having read the draft of print and archival format for Institute documents, the respondent puts one point to the secretariat.
The respondent states that a submission not accepted receives a reason and nothing further, and that a respondent who considers the reason mistaken has nowhere to go.
The respondent proposes a review of disposition by a person who did not make it.
The secretariat accepts this submission. A disposition that cannot be challenged is a decision without a check.
A disposition may now be referred for review to a member of the assessment committee who did not author the response, once per submission, with the referral and its outcome published alongside the original disposition whether or not it changes.
A declaration is made on appointment and never refreshed
Having read the draft under consultation, which concerns print and archival format for Institute documents, the respondent submits as follows.
Interests change. A declaration made three years ago and published unchanged conveys currency it does not have, and the reader has no way to tell the difference between an unchanged interest and an unrefreshed one.
The respondent proposes annual re-declaration with the date of last declaration published.
The respondent notes that submission 002 has already been made and confines this submission to a matter not covered by it.
The secretariat accepts annual re-declaration and declines to require declaration on every document.
Interests are re-declared annually and the date of last declaration is published against each entry. Declaration against each document was considered and would produce a volume of unchanged text in which a change would be harder to see.
Assessment committee meetings are held in private
The respondent submits on print and archival format for Institute documents. Governance documents are read by very few people and relied on by everyone who reads anything else the Institute publishes.
Comparable bodies admit observers to at least part of a meeting. The document does not say whether the Institute has considered doing so.
The respondent proposes that part of each meeting be open, or that the reason for holding meetings in private be stated.
The secretariat notes this submission and states the position rather than changing it.
Meetings are held in private and the reason is now stated: the committee assesses draft ratings that will change before publication, and an observed deliberation on a draft rating tends to produce a published deliberation rather than a candid one. The minutes and the dissenting positions are published.
Anonymous submissions should be accepted where the respondent risks employment consequences
The respondent has read print and archival format for Institute documents and submits on a matter of presentation.
The respondent states that people in a position to identify a defect in a manufacturer's documentation are frequently employed by that manufacturer, and that requiring a signature excludes exactly those submissions.
The respondent proposes an anonymous route with verification held by the secretariat and not published.
The secretariat does not accept this submission, and records that the exclusion it identifies is real.
The Institute publishes no anonymous submission, because a submission whose author is not published cannot be weighed against a declared interest and the declaration is the mechanism the whole surface rests on. Material bearing on a factual defect may instead be submitted to the evidence log, where it is assessed on the document and the submitter is named only with consent. The submission remains published in full.
A four-week window is too short for a respondent without institutional support
The respondent’s comment on print and archival format for Institute documents is made from the standpoint of a person who might be affected by a decision taken under it.
An organisation with staff can answer a consultation in four weeks. An individual reading in their own time, particularly one who needs the document explained before they can respond to it, generally cannot.
The respondent proposes a minimum of eight weeks for any consultation on a document of general application.
The secretariat accepts the principle and adopts a shorter minimum than proposed.
The minimum window is now six weeks for a document of general application and four for a document-specific draft. A respondent who asks for more time before a window closes is given it, and that is now stated in the policy.
Absence of evidence is presented in a form a reader will take as negative evidence
The respondent submits on print and archival format for Institute documents. The point would apply equally to any document in the series.
Where the Institute has identified no study, the draft states that no evidence was found. In several places that sentence sits immediately after a paragraph describing an adverse outcome, and the juxtaposition invites the reading that the compound was studied and found wanting.
The respondent proposes a standing formulation, used identically wherever the situation arises, distinguishing an outcome that was studied and not demonstrated from an outcome that has not been studied at all.
The secretariat accepts this submission. The two states are different, they support different decisions, and the draft rendered them in language a reader could not reliably separate.
A standing formulation has been adopted and is applied wherever an evidence gap is recorded, distinguishing an outcome assessed and not demonstrated from an outcome not assessed. The formulation is identical at every occurrence so that it can be recognised at a glance.
No term limit applies to membership of the assessment committee
This submission addresses print and archival format for Institute documents from the standpoint of an outside body that would have to rely on it.
Members are appointed and the document does not say for how long. A body whose membership does not turn over converges on a settled view and stops noticing that it has one.
The respondent proposes a fixed term, renewable once, with the appointment date published against each member.
The respondent has read submission 003 with interest and adds one observation the secretariat may find useful.
The secretariat accepts this submission.
Members are appointed for a four-year term renewable once. Appointment and expiry dates are published in the register of members.
Disagreement within the assessment committee is not visible
This submission concerns print and archival format for Institute documents and a convention used across the Institute’s output.
The respondent states that ratified documents present a single position and that a reader cannot tell whether a rating was unanimous or contested.
The respondent proposes that a dissenting view be recorded where a member asks for it to be.
This submission is made by a respondent with a different professional interest in the outcome from the one behind submission 006.
The secretariat accepts this submission. A document that conceals a division presents more certainty about its own judgement than it holds.
A ratification record is now published with every document stating whether the assessment was unanimous, and a member may record a dissenting statement which is published with the document rather than in minutes.
Funding should not be accepted from any party the Institute assesses
The draft of print and archival format for Institute documents was read for what it obliges the Institute to do rather than for what it says the Institute intends.
The respondent states that disclosure does not neutralise an incentive, and that accepting funding from an assessed party creates one whether or not it is acted on.
The respondent proposes an outright prohibition on funding from any assessed supplier or manufacturer.
The secretariat accepts this submission in part. Funding from an assessed party is not accepted for assessment work. A general prohibition covering all activity is not adopted, because it would extend to unrestricted contributions the Institute can accept without any assessment consequence.
The policy now prohibits funding from an assessed party for any activity bearing on that party's assessment, requires that any other relationship be published on the funding page and on every page it could bear on, and records the sums involved rather than the fact of a relationship alone.
A party criticised in an assessment should have a right of reply
This submission concerns the draft of print and archival format for Institute documents. The respondent has served on committees operating comparable regimes and the observation draws on that.
The respondent states that a supplier or sponsor whose documentation is assessed adversely has no route to respond other than a comment period that may not be open.
The respondent proposes a standing right of reply published alongside the assessment.
The secretariat accepts this submission in part. A route to submit a correction of fact is opened at any time and is not dependent on a consultation window. A general right of reply published alongside the assessment is not adopted, because it would give an assessed party the last word on the Institute's own document.
A standing correction route is now published, corrections of fact are assessed and applied at any time with the change recorded in the amendment log, and a submission of substance received outside a window is held and published in the next comment period on that document.
Quantitative claims are reproduced without the method that produced them
The respondent read print and archival format for Institute documents in draft and has confined this submission to one clause.
Several figures in the draft are quoted from sources that determined them by different methods. A figure obtained by one determination and a figure obtained by another are not comparable, and the draft places them in the same sentence without distinguishing them.
The respondent, an analytical chemist, proposes that every quantitative claim carry the method that produced it at the point of use rather than in the reference.
The secretariat accepts this submission. Placing two figures side by side is an implicit claim that they are the same kind of quantity, and in the cases identified they were not.
Every quantitative claim now carries the determination that produced it at the point of use, and figures obtained by non-comparable methods are no longer presented in the same row or sentence.
The position of a contributor who withdraws consent is not stated
The respondent submits on print and archival format for Institute documents.
The respondent states that consent to be named is obtained at submission and that the policy is silent on what happens if it is later withdrawn, which leaves both the Institute and the contributor without a rule.
The respondent proposes that the position be stated in advance and be the same for everyone.
The secretariat accepts this submission. A rule made at the point of the first withdrawal is a rule made about a particular person.
The policy now states that a contributor may withdraw from the register at any time and that the register entry is removed, while a signed submission already published remains published, because the consultation record is a record of what was said and by whom. The position is stated at the point consent is given.
The materiality threshold for a declarable holding is set too high
The respondent submits on print and archival format for Institute documents, on a matter that is not specific to this draft but is visible in it.
The respondent states that the threshold in the draft excludes holdings a reader would regard as relevant, and that the cost of declaring a small holding is negligible against the cost of a reader discovering an undeclared one.
The respondent proposes that all holdings be declared regardless of size.
This point is adjacent to the one made in submission 008 and the respondent puts it in a form the secretariat can act on.
The secretariat accepts this submission in part. The threshold is lowered. Declaration of every holding without threshold is not adopted, because an undifferentiated list obscures the holdings that matter.
The materiality threshold is lowered and is now expressed as a proportion of the declarer's holdings as well as an absolute figure, so that a holding material to the person is declarable whether or not it is large in absolute terms.
A reader who acted on a document is not told when it is corrected
This is a submission on print and archival format for Institute documents, from a respondent representing readers rather than practitioners.
Corrections are published in a log. Nobody who read the original page is notified, and the corrected page carries the correction in a form that a reader arriving fresh will read past.
The respondent proposes that a corrected document carry a visible notice at the point of the correction for a stated period, in addition to the log entry.
The secretariat accepts this submission. A correction filed where only the diligent will find it is a correction the reader who needs it will miss.
A corrected passage now carries an inline notice for twelve months from the date of correction, naming the log entry, in addition to the entry itself.
References should carry a persistent identifier for every cited source
The respondent has read print and archival format for Institute documents in draft and makes one submission.
Several references in the draft carry a journal, a year and a volume but no persistent identifier. The respondent states that retrieval of such a reference is materially slower and that identifiers should be supplied throughout.
The respondent asks in the alternative that where an identifier exists but is not carried, the omission be explained rather than left as a gap the reader must interpret.
The secretariat accepts the second limb of this submission and declines the first. Identifiers are supplied wherever the Institute holds one. Where the Institute does not hold an identifier it will not supply one, because a reconstructed identifier that resolves to the wrong record is a worse defect than an absent one.
Every reference without a persistent identifier now carries an explicit statement that the identifier is not held by the Institute, so that its absence is a recorded fact rather than an apparent oversight.
The policy does not say who may make a submission
The respondent read print and archival format for Institute documents in draft. The point applies to it and to the series generally.
The document describes the form of a submission and not who is eligible to make one. A reader without a professional affiliation cannot tell whether the process is open to them.
The respondent proposes an explicit statement that anyone may submit, with the declared-interest requirement applying equally to all.
The respondent supports submission 005 so far as it goes and adds the matter set out here.
The secretariat accepts this submission. The process is open and the document did not say so.
The policy now states that any person may make a submission, that no affiliation is required, and that the declaration requirement applies identically to every respondent.
The policy does not say what happens to a document the Institute is asked to remove
The respondent notes that print and archival format for Institute documents governs situations that will arise rarely and matter greatly when they do, and submits accordingly.
The respondent states that assessments of named parties will attract removal requests, and that a policy silent on them will be applied inconsistently under pressure.
The respondent proposes that the grounds on which a document may be withdrawn be stated in advance and be exhaustive.
The respondent read submission 016 after drafting this one and has not altered it, the two points being distinct.
The secretariat accepts this submission. A policy written after the first request will be written under the conditions least suited to writing it.
The grounds for withdrawal are now stated exhaustively, being a factual error that cannot be corrected in place and a legal obligation, a withdrawn document is replaced by a notice recording the fact and the ground rather than removed, and every withdrawal is recorded in the corrections log.
References cited on this page
References are numbered in order of first citation in this document. Each superscript in the text links to its entry below.
- International Organization for Standardization. ISO/IEC 17025:2017 General Requirements for the Competence of Testing and Calibration Laboratories. ISO/IEC Standard 2017;3rd edition. identifier not held by the Institute
Identifiers are reproduced only where the Institute holds them. Where a digital object identifier or PubMed identifier is not shown, the Institute has recorded the journal and year and has not constructed an identifier.