Draft funding and sponsorship policy — submissions
The 10 submissions received, published in full with declared interests and secretariat responses.
§2Submissions and responses
10 submissions were received. Each is published in full below with its declared interest, the secretariat response and the disposition. The Institute publishes submissions it did not accept in the same form as those it did.
The terms on which the documents may be reused are not stated
The respondent, a librarian, states that the site carries no statement of reuse terms and that an assessment which cannot be quoted with confidence will be paraphrased instead, which is worse for accuracy.
The respondent proposes an explicit reuse statement.
The secretariat accepts this submission. An unstated position on reuse is a position that will be guessed at.
The reuse terms are now stated explicitly on the legal page and in the document footer, permitting quotation and reproduction with attribution and the document version, and requiring that any reproduction carry the certainty rating that accompanied the quoted statement.
Declared interests should appear on the document rather than on a separate page
The draft links to a central conflicts register. The respondent argues that a reader assessing whether to rely on a particular document should not have to leave it to find out who assessed it and what they declared.
The respondent proposes that the interests of every named contributor to a document be printed on that document.
The secretariat notes this submission and records that the draft already provides for it, which the respondent could reasonably have missed because the provision sits in an appendix.
Every document carries the declared interests of its named contributors in its front matter, and the central register exists so that a reader can see a person across all documents rather than one at a time. No amendment arises; the provision has been moved from the appendix into the body of the methodology document so that it is findable.
The same concept is given three different names in one document
The draft refers to the same quantity as a response rate, a responder proportion and a categorical outcome in different sections. The respondent, who works in health-technology assessment, states that a reader cannot tell whether the three refer to one thing or to three.
The respondent proposes that the glossary term be used at every occurrence and that the glossary entry be linked at first use in each section rather than only at first use in the document.
The secretariat accepts this submission. The variation was stylistic and its cost to the reader exceeds any benefit.
A single term is now used throughout for each defined concept, and the glossary entry is linked at first use within each section rather than once per document.
A sortable table implies a comparison the underlying data do not support
The draft presents a sortable table whose columns are drawn from sources of differing quality. The respondent states that sorting on such a column produces an ordering that looks like a ranking and is not one.
The respondent proposes that sorting be disabled on any column whose values are not commensurable.
The secretariat notes this submission and records that the point is correct in principle.
No amendment arises here because every sortable table in the document set already carries a standing statement above it that the ordering is not a ranking and that the values in each column are commensurable only where the column header says so. The proposal to disable sorting was considered and not adopted, because a reader who cannot sort a table generally sorts it elsewhere and without the statement.
The document should state what a reader ought to do
The draft assesses evidence and stops. The respondent, a practising clinician, states that a reader arriving at the document with a decision to make is left to convert an assessment into an action without help, and proposes that each document close with a recommendation.
The respondent argues that other evidence bodies issue recommendations and that declining to do so transfers the difficult part of the work to the reader.
The secretariat does not accept this submission, and records that the point is a reasonable one rather than a misunderstanding.
The Institute assesses evidence and does not issue recommendations, because a recommendation embeds values and a resource context that the Institute does not hold and cannot state. That constitutional limit is published on the methodology page and is not varied by consultation. The submission remains published in full.
A party criticised in an assessment should have a right of reply
The respondent states that a supplier or sponsor whose documentation is assessed adversely has no route to respond other than a comment period that may not be open.
The respondent proposes a standing right of reply published alongside the assessment.
The secretariat accepts this submission in part. A route to submit a correction of fact is opened at any time and is not dependent on a consultation window. A general right of reply published alongside the assessment is not adopted, because it would give an assessed party the last word on the Institute's own document.
A standing correction route is now published, corrections of fact are assessed and applied at any time with the change recorded in the amendment log, and a submission of substance received outside a window is held and published in the next comment period on that document.
The secretariat answers submissions and is not itself named
The respondent states that responses are attributed to the secretariat as a body, and that a reader cannot tell who wrote a response or whether that person had a relevant interest.
The respondent proposes that responses be signed individually.
The secretariat accepts this submission in part. Responses are attributed to the named members of the secretariat responsible for the document rather than to individuals per response, and the interests of those members are published.
Every comment period now names the secretariat members responsible for it, with their declared interests, in the consultation front matter, so that the responses can be read against the interests of the people who wrote them.
Recusal should follow automatically from any declared financial interest
The respondent states that case-by-case recusal decisions require the body making them to judge its own members, and that an automatic rule removes the judgement and with it the appearance of partiality.
The respondent proposes automatic recusal on any declared financial interest in a named company.
The secretariat does not accept this submission. An automatic rule of that breadth would recuse most of the people competent to assess the evidence, and the Institute would be left with assessors chosen for their distance from the field.
Recusal remains determined against the specific document, with the decision, its reasons and the person recused published in the document front matter. The submission is published in full because it identifies a genuine tension the Institute has resolved in one direction rather than removed.
The document is unreadable without specialist training
The respondent, a trustee of a patient organisation, states that the draft is written for a reader who already understands certainty grading, and that the people most affected by the subject matter will not reach the assessment at all.
The respondent proposes a plain-language summary at the head of every document, written to the same standard of accuracy as the document itself and not as a promotional abstract.
The secretariat accepts this submission in part. A plain-language summary is added. The proposal that it replace the technical abstract is declined, because the abstract is the part of the document other assessors read and cite.
Every document now opens with a plain-language summary of not more than 150 words, placed above the technical abstract and carrying the same certainty language, so that the two cannot diverge.
The document should not describe uses outside the approved indication
The submission is made on behalf of a marketing-authorisation holder. It states that the draft describes uses of the compound that fall outside the approved indication, that such uses are not supported by the sponsor, and that describing them may be read as legitimising them.
The sponsor asks that the sections concerned be removed, or in the alternative that they carry a prominent statement that the sponsor does not support such use.
The secretariat does not accept this submission. The Institute records what is supplied and what is used, because a reader encountering a compound outside a regulated route is the reader most in need of an assessment of the evidence for it.
The sections remain. Every such section already states the regulatory status of the use described and states that the evidence for it is assessed separately from the evidence for the approved indication. The submission is published in full and identified as an industry submission.
References cited on this page
References are numbered in order of first citation in this document. Each superscript in the text links to its entry below.
- International Organization for Standardization. ISO/IEC 17025:2017 General Requirements for the Competence of Testing and Calibration Laboratories. ISO/IEC Standard 2017;3rd edition. identifier not held by the Institute
Identifiers are reproduced only where the Institute holds them. Where a digital object identifier or PubMed identifier is not shown, the Institute has recorded the journal and year and has not constructed an identifier.