Draft synthesis: Reconstitution practice and delivered dose — submissions
The 9 submissions received, published in full with declared interests and secretariat responses.
§2Submissions and responses
9 submissions were received. Each is published in full below with its declared interest, the secretariat response and the disposition. The Institute publishes submissions it did not accept in the same form as those it did.
The sampling frame of each contributing survey is not reported
The respondent has read the draft synthesis on What is the effect of reconstitution practice on the dose actually delivered from a lyophilised research peptide? and makes one submission about the analytical evidence it pools.
The included surveys each obtained material from a set of sources, and the draft reports what was found without reporting how the sources were chosen.
The respondent states that a survey that bought from the most visible sellers and a survey that bought at random from a directory answer different questions, and that pooling them without recording the difference produces a figure that describes neither.
The respondent proposes that the sampling frame, the number of sources approached and the number that supplied be tabulated for every contributing survey.
The secretariat accepts this submission. How material was obtained determines what a survey can support, and the draft treated the surveys as interchangeable.
Every contributing survey now carries a sampling row stating the frame, the number of sources approached and the number supplying, and the synthesis reports separately for surveys whose frames are not comparable.
Results obtained by different methods are pooled without recording the method
The respondent’s comment on the draft review of What is the effect of reconstitution practice on the dose actually delivered from a lyophilised research peptide? arises from comparing the pooled figures against the reports they were taken from.
The contributing results were obtained by more than one determination, and the draft reports a single distribution.
The respondent states that the determinations do not measure the same quantity, that the difference between them is systematic rather than random, and that a pooled distribution across methods has no referent.
The respondent proposes that every result carry the method that produced it and that pooling occur only within a method.
The secretariat accepts this submission. A figure whose method is not recorded cannot be compared with another figure, and the draft pooled across the difference.
Every contributing result now carries its determination, pooling occurs only within a determination, and where a survey did not report the method the result is listed as method not stated and excluded from the pooled estimates.
Risk-of-bias judgements are reported as an overall rating without the domains
The respondent submits on the draft review of What is the effect of reconstitution practice on the dose actually delivered from a lyophilised research peptide?. This submission concerns how the contributing results were obtained rather than what they show.
The draft reports a single overall risk-of-bias judgement per study. The respondent states that the overall judgement conceals which domain drove it, and that a reader assessing whether the concern applies to their question needs the domain.
The respondent proposes a domain-level table with the supporting quotation for each judgement.
The respondent notes submission 001 above and does not repeat the ground it covers.
The secretariat accepts this submission. The overall judgement is a summary of the domains and publishing only the summary makes it uncheckable.
Risk of bias is now reported at domain level for every included study, with the text on which each judgement was based quoted and referenced, and the overall judgement derived from the domains rather than asserted alongside them.
Multiple vials from one lot are counted as independent observations
This submission addresses the draft synthesis on What is the effect of reconstitution practice on the dose actually delivered from a lyophilised research peptide? from the standpoint of a reader who will act on the summary figure.
Several contributing surveys report results for more than one vial drawn from a single lot, and the draft counts each vial as a separate observation.
The respondent states that vials from one lot share a fill and a manufacturing history, so the effective number of independent observations is the number of lots and not the number of vials, and that the interval reported is narrower than the data support.
The respondent proposes that the unit of analysis be the lot, with within-lot variation reported separately.
The secretariat accepts the unit-of-analysis point and declines to discard the within-lot data, which are informative about fill uniformity.
The unit of analysis is now the lot throughout, the number of lots is reported beside the number of vials, and within-lot variation is reported as a separate quantity rather than folded into the pooled interval.
What the contributing methods could not have detected is not stated
This submission concerns the draft synthesis on What is the effect of reconstitution practice on the dose actually delivered from a lyophilised research peptide?. The respondent works with determinations of this kind and the observation arises from that.
The draft reports what the contributing surveys found and does not report what their methods were incapable of finding.
The respondent states that a reverse-phase method is blind to aggregation, that an ultraviolet detector does not see a species without a chromophore, and that a reader takes an unqualified negative as an absence.
The respondent proposes that every reported absence carry the detection scope of the method that produced it.
The secretariat accepts the disclosure and declines to reclassify the findings, which were correctly reported for the methods used.
Every reported absence now carries the detection scope of the method behind it, and the review states in its abstract that an absence in this evidence base is an absence within a stated scope rather than an absence in the material.
The review question is drawn too broadly to be answerable
The respondent read the draft review of What is the effect of reconstitution practice on the dose actually delivered from a lyophilised research peptide? and has confined this submission to a single matter of measurement.
The respondent states that the population as registered spans groups in which the intervention would be expected to behave differently, and that a single estimate across them is uninformative.
The respondent proposes that the question be split and the review re-registered.
Submission 001 raises an adjacent matter. The respondent regards the two as separable and addresses only this one.
The secretariat does not accept this submission. The breadth was registered before screening, the pre-specified subgroups address the variation the respondent identifies, and re-registering after the results are known would convert a pre-specified analysis into a post hoc one.
The question stands as registered. The point is recorded in the limitations, and the assessment committee has noted it for the protocol of the next version of this review, which will be registered before the search is run. The submission remains published in full.
The review protocol is described but its registration record is not linked
The draft review of What is the effect of reconstitution practice on the dose actually delivered from a lyophilised research peptide? was read against the certificates and reports it draws on.
The respondent asks for the prospective registration record of the review protocol so that the registered outcomes can be compared with the reported ones.
The respondent states that without it the claim of prospective registration cannot be checked.
The secretariat accepts this submission in part. The protocol is published in full on the Institute site with its version history, which permits the comparison the respondent asks for. Where an external registration identifier is not held by the Institute, the review says so rather than supplying one.
Each synthesis now links its own protocol with the version in force at screening, and states explicitly where an external registration identifier is not held, so that the absence is a recorded fact.
Declared interests should appear on the document rather than on a separate page
The respondent notes that the review question — What is the effect of reconstitution practice on the dose actually delivered from a lyophilised research peptide? — is answerable only if the contributing determinations measured the same quantity, and submits with that in view.
The draft links to a central conflicts register. The respondent argues that a reader assessing whether to rely on a particular document should not have to leave it to find out who assessed it and what they declared.
The respondent proposes that the interests of every named contributor to a document be printed on that document.
The secretariat notes this submission and records that the draft already provides for it, which the respondent could reasonably have missed because the provision sits in an appendix.
Every document carries the declared interests of its named contributors in its front matter, and the central register exists so that a reader can see a person across all documents rather than one at a time. No amendment arises; the provision has been moved from the appendix into the body of the methodology document so that it is findable.
The search was limited by publication date and the limit is not justified
Having read the draft synthesis on What is the effect of reconstitution practice on the dose actually delivered from a lyophilised research peptide?, the respondent puts one point about the underlying measurements to the committee.
The search strategy carries a start date some years before the review. No reason is given. If the limit reflects the date of first synthesis of the compound it is defensible and should be said; if it reflects convenience it is a source of bias.
The respondent proposes that every date limit carry its justification in the search-strategy section.
The respondent has read submission 005 and puts a further matter to the secretariat.
The secretariat accepts the requirement to justify and declines to remove limits that are justified.
Every date limit in the series now carries its justification in the search-strategy section. Limits that could not be justified on re-examination have been removed and the affected searches re-run.
References cited on this page
References are numbered in order of first citation in this document. Each superscript in the text links to its entry below.
- International Organization for Standardization. ISO/IEC 17025:2017 General Requirements for the Competence of Testing and Calibration Laboratories. ISO/IEC Standard 2017;3rd edition. identifier not held by the Institute
Identifiers are reproduced only where the Institute holds them. Where a digital object identifier or PubMed identifier is not shown, the Institute has recorded the journal and year and has not constructed an identifier.