Draft standard: freeze–thaw and in-use stability assessment… — submissions
The 15 submissions received, published in full with declared interests and secretariat responses.
§2Submissions and responses
15 submissions were received. Each is published in full below with its declared interest, the secretariat response and the disposition. The Institute publishes submissions it did not accept in the same form as those it did.
References should carry a persistent identifier for every cited source
Several references in the draft carry a journal, a year and a volume but no persistent identifier. The respondent, who works in a library setting, states that retrieval of such a reference is materially slower and that identifiers should be supplied throughout.
The respondent asks in the alternative that where an identifier exists but is not carried, the omission be explained rather than left as a gap the reader must interpret.
The secretariat accepts the second limb of this submission and declines the first. Identifiers are supplied wherever the Institute holds one. Where the Institute does not hold an identifier it will not supply one, because a reconstructed identifier that resolves to the wrong record is a worse defect than an absent one.
Every reference without a persistent identifier now carries an explicit statement that the identifier is not held by the Institute, so that its absence is a recorded fact rather than an apparent oversight.
Point estimates are given without an interval
Several estimates in the draft appear as single figures. The respondent states that a point estimate without an interval invites a precision the underlying data do not support, and that the effect is worst where the estimate is drawn from a small contributing set.
The respondent proposes that no point estimate appear anywhere in the document set without its interval, including in summary tables and in the abstract.
The secretariat accepts this submission in part. Intervals are added wherever the source reports one. The proposal is declined for figures the source published without an interval, because the Institute will not compute an interval a source did not report.
Every estimate now carries its interval where the source reported one, and where it did not, the estimate is annotated as reported without an interval rather than left to appear as a precise figure.
The scope does not say which materials the standard excludes
The scope states what the standard applies to. The respondent states that in practice the difficult question is whether a borderline material is in scope, and that the draft leaves it to be inferred.
The respondent proposes an explicit exclusion list.
The secretariat accepts this submission. A scope stated only positively is a scope that will be stretched.
The scope now states both what the standard applies to and what it does not, with the excluded classes named and, where another standard covers them, cross-referenced.
Two factual descriptions of the sponsor's programme are inaccurate
The submission is made on behalf of the marketing-authorisation holder and is confined to two matters of fact. The draft describes a trial as terminated where the sponsor closed it at a pre-specified interim analysis, and gives a dose in a unit that does not match the approved labelling.
Supporting documentation, comprising the published trial report and the current summary of product characteristics, accompanied the submission. No view is expressed on the certainty ratings, which the sponsor considers a matter for the assessment committee.
The secretariat accepts this submission. Both points are matters of fact, both were checkable against documents the Institute holds, and both were wrong in the draft.
The trial status now reads as completed at a pre-specified interim analysis, with the analysis identified, and the dose is stated in the unit used in the approved labelling. The submission is identified as an industry submission on its face in accordance with the conflicts policy.
The limitations section states what the method cannot do but not what a reader will wrongly conclude
The limitations section is accurate and abstract. The respondent states that the misreadings this standard should prevent are specific and predictable, and that naming them would be more use than a general statement of scope.
The respondent proposes that the limitations section list the specific inferences a conformant result does not support.
The secretariat accepts this submission. A limitation stated abstractly is a limitation a reader will not apply to the case in front of them.
The limitations section now lists the specific conclusions a conformant determination does not support, each stated as a sentence a reader could otherwise have written, and the list is placed before the references rather than in an annex.
A superseded version should remain reachable from the version that replaced it
The respondent states that the draft supersedes an earlier document and that a reader who cited the earlier version has no way to reach it from the new one, which makes it impossible to see what changed.
The respondent asks that every version carry a link both to what it supersedes and to what supersedes it.
The secretariat notes this submission. The corrections and versioning policy already requires bidirectional version links and every superseded document is retained at its own address.
No amendment arises. The requirement is stated in the corrections and versioning policy and the amendment log of this document links to the version it replaced. The respondent is correct that the link was absent from the draft page furnished for consultation, which was a defect of the consultation copy and not of the policy.
The document should state what a reader ought to do
The draft assesses evidence and stops. The respondent, a practising clinician, states that a reader arriving at the document with a decision to make is left to convert an assessment into an action without help, and proposes that each document close with a recommendation.
The respondent argues that other evidence bodies issue recommendations and that declining to do so transfers the difficult part of the work to the reader.
The secretariat does not accept this submission, and records that the point is a reasonable one rather than a misunderstanding.
The Institute assesses evidence and does not issue recommendations, because a recommendation embeds values and a resource context that the Institute does not hold and cannot state. That constitutional limit is published on the methodology page and is not varied by consultation. The submission remains published in full.
The document is unreadable without specialist training
The respondent, a trustee of a patient organisation, states that the draft is written for a reader who already understands certainty grading, and that the people most affected by the subject matter will not reach the assessment at all.
The respondent proposes a plain-language summary at the head of every document, written to the same standard of accuracy as the document itself and not as a promotional abstract.
The secretariat accepts this submission in part. A plain-language summary is added. The proposal that it replace the technical abstract is declined, because the abstract is the part of the document other assessors read and cite.
Every document now opens with a plain-language summary of not more than 150 words, placed above the technical abstract and carrying the same certainty language, so that the two cannot diverge.
The standard assumes a reference standard of known assigned content
The respondent states that for many compounds in scope no reference standard with an assigned content exists, and that the standard as drafted has nothing to say to a laboratory in that position.
The respondent proposes that the standard state what can be determined without one and how such a determination should be labelled.
The secretariat accepts this submission. A standard applicable only where a reference standard exists is silent in the cases most in need of it.
The standard now states which determinations remain available without a reference standard of assigned content, requires that such determinations be labelled as relative rather than absolute, and requires the assumption used in place of an assigned content to be stated.
Absence of evidence is presented in a form a reader will take as negative evidence
Where the Institute has identified no study, the draft states that no evidence was found. In several places that sentence sits immediately after a paragraph describing an adverse outcome, and the juxtaposition invites the reading that the compound was studied and found wanting.
The respondent proposes a standing formulation, used identically wherever the situation arises, distinguishing an outcome that was studied and not demonstrated from an outcome that has not been studied at all.
The secretariat accepts this submission. The two states are different, they support different decisions, and the draft rendered them in language a reader could not reliably separate.
A standing formulation has been adopted and is applied wherever an evidence gap is recorded, distinguishing an outcome assessed and not demonstrated from an outcome not assessed. The formulation is identical at every occurrence so that it can be recognised at a glance.
Terms are used in senses that differ from established pharmacopoeial usage
The respondent, employed by a national competent authority and writing in a personal capacity, identifies three terms used in the draft in senses that differ from their established pharmacopoeial meanings.
The respondent proposes that established usage be followed or that the departure be stated explicitly at first use.
The secretariat accepts this submission. A term used in a private sense inside a document that will be read alongside compendial texts is a source of avoidable error.
The three terms now follow established pharmacopoeial usage, the glossary records the definition and its source, and where the Institute uses a term in a narrower sense the departure is stated at first use in the standard.
Rounding conventions are not stated and change the reported result
The respondent states that the draft reports figures to varying precision and does not specify when rounding occurs in the calculation chain, which can move a result across an acceptance limit.
The respondent proposes that rounding occur only at the reporting step and that the convention be stated.
The secretariat accepts this submission. Intermediate rounding is a silent source of disagreement between laboratories analysing the same material.
The standard now requires that no rounding be applied before the reporting step, states the reporting precision for each quantity, and states the rounding convention used at that step.
The matrix interferes with the determination and no control is required
The respondent states that the matrices in scope can suppress or enhance the response, and that without an inhibition and enhancement control a result cannot be attributed to the analyte.
Data showing recovery outside acceptable limits in an uncontrolled matrix accompanied the submission.
The secretariat accepts this submission. A determination on an interfering matrix without a validated control establishes nothing, and the draft permitted one.
An inhibition and enhancement control is now required for every determination under this standard, with stated recovery limits, and a determination reported without it is recorded as not conformant rather than as a low result.
The search date is not on the face of the document
The draft carries a publication date and a review date but not the date on which the evidence was last searched. Those are three different dates and only the third tells a reader how current the assessment is. A document published in one quarter may rest on a search run two quarters earlier, and nothing on the page allows that gap to be measured.
The respondent proposes that the search date be printed adjacent to every certainty rating rather than in the methods section, on the ground that a reader who acts on a rating is unlikely to have read the methods section first.
The secretariat accepts this submission. The distinction between publication, review and search dates is real and the draft did not make it visible where it mattered.
The search date is now printed adjacent to every certainty rating and is carried in the document metadata, so that the interval between the search and the reading is visible without reference to the methods section.
Acceptance criteria are expressed in terms specific to one instrument family
The respondent states that two criteria are expressed in units reported by one manufacturer's software and not by others, which makes conformance a function of the instrument rather than of the determination.
The respondent proposes that criteria be expressed in instrument-independent quantities.
The secretariat accepts this submission in part. Both criteria are restated in instrument-independent terms. The conversion from the manufacturer-specific quantity is given rather than removed, because laboratories will continue to read it from their software.
The acceptance criteria are now expressed in instrument-independent quantities, with the conversion from the common manufacturer-specific reporting units given in a note so that a laboratory can check conformance without recalculating by hand.
References cited on this page
References are numbered in order of first citation in this document. Each superscript in the text links to its entry below.
- International Organization for Standardization. ISO/IEC 17025:2017 General Requirements for the Competence of Testing and Calibration Laboratories. ISO/IEC Standard 2017;3rd edition. identifier not held by the Institute
Identifiers are reproduced only where the Institute holds them. Where a digital object identifier or PubMed identifier is not shown, the Institute has recorded the journal and year and has not constructed an identifier.