Registered Reports are a publication format in which the research question and the quality of methodology are peer-reviewed before the data are collected and analysed. Registered Reports are intended to strengthen the methodology and transparency of research papers. They are a research and publishing tool to support the reporting of research that has been conducted fully prospectively, i.e. research for which it is possible to publish the background, research question, planned methodological steps, and analysis plans before the data collection and analyses commence. Registered Reports are offered by Quality of Life Research as an additional article format extending our range of research and applications published in our journal.
Submissions of Registered Reports need to adhere to a two-stage review process which is outlined in the following. Quality of Life Research considers submissions of Registered Reports in the journal's usual remit: they can be quantitative and qualitative studies of any design, which investigate quality of life aspects of treatment, care and rehabilitation; utilize patient-reported outcomes; and/or focus on clinical and policy applications of (health-related) quality of life research. The format is open to systematic reviews and reports based on clinical trials.
In principle this format favours quantitative and/or experimental research, but the two key considerations when assessing a Registered Report for publication at Quality of Life Research are whether (i) the theoretical background is convincing and leads to the proposed research question/hypotheses; (ii) the methods and results are prospectively described to indicate the exact course of action that will be taken (which in some instances may take the form of decision strategies that have to be implemented instead of defining a single course of action), while considering appropriate standards of transparency and reproducibility.
A backbone of this publishing format is that the data for which the registration is undertaken are not available at the time of writing, i.e. no data collection has started yet. Therefore, this publishing format is closed to secondary data analysis projects, unless a very strong case can be made that the data were not available at the time of conception of the study.
If a research project does not immediately seem to fit into this description and since it is a new format with our journal, we encourage pre-submission inquiries for Registered Report projects with [j.r.boehnke(at)dundee.ac.uk]
The two-stage review process at Quality of Life Research
Submissions of Registered Reports need to adhere to a two-stage review process. During the first stage, the authors submit a paper with fully written background, methods and analysis sections before data collection has commenced. This version is peer-reviewed and an editorial decision is rendered with a fast turn-around time. If the submission passes initial review, the paper is accepted for publication, regardless of the outcome, as long as the methods and analysis in the finally submitted study (peer-review at the second stage of the process) conform to the initially approved submission and that the conclusions are justified given the data.
- Stage 1 submission: Authors submit manuscripts comprising the full background section (which should clearly lead to the research question and methods applied in this manuscript), and the methods section (including exact proposed analyses, decision criteria for the selection of methods, and criteria to interpret the results in light of the research question/ hypotheses). For quantitative papers, the submission of syntax at least for the explanation of the exact statistical procedures testing the main hypotheses (e.g., as an electronic supplement) is recommended.
- Stage 1 review: Reviewers evaluate study proposals before data are collected, assessing the soundness of the research question, feasibility of the methodology, and the quality and reproducibility (e.g., adherence to appropriate reporting guidelines, clarity of procedures, submitted syntax for the analyses etc.) of the proposed procedures and analyses.
- It is important for authors to note that reviewers may suggest changes that require changes of the study design and/or re-submissions to ethics.
- Quality of Life Research will review Registered Reports in parallel to submitted ethics applications, but an in-principle acceptance can only be given after all approvals are in place (see details below).
- In principle acceptance: Manuscripts that pass peer review will be issued an 'in principle acceptance', indicating that the article will be published pending successful completion of the study.
- At this stage Quality of Life Research will render an "Accept, but incomplete" decision and authors will be asked to register their approved protocol on a recognised repository (e.g., such as the Open Science Framework; https://osf.io/) either publicly or under private embargo until submission of the Stage 2 manuscript.
- Once this has been completed and the link to the registered manuscript has been added to the report, the report will be fully accepted at Stage 1.
- Stage 2 submission: Following study completion, authors submit their finalized manuscript for re-review, now including Results and Discussion sections.
- The manuscript needs to contain a link to the approved and registered Stage 1 protocol (on a recognised repository, see above). The cover letter should state the page number in the manuscript that lists the URL.
- Stage 2 review: Reviewers appraise whether the authors’ adhered to the registered procedures and analyses; and that any conclusions and implications for future research, policy, and practice, are justified given the data.
Information for Authors
Essential feature of the Registered Reports format is that a significant part of the manuscript is assessed prior to data collection. The submission and review process is consequently divided into two stages.
Stage 1: Initial manuscript submission and review
Initial submissions should include the Stage 1 manuscript (requirements detailed below) and a brief cover letter. The description of the requirements for the Stage 1 manuscript below uses mainly a quantitative empirical paper as a template. This should not stop other types of submissions: as long as the applied research methods are strictly prospective and oriented along best-practice guidelines for conduct and reporting of the particular type of research we will in principle consider the submission for the Registered Reports format. The guidelines below should be used to understand which type of detail is required and adapted/ translated into other types of research.
Authors are welcome to first submit pre-submission enquiries to the Editorial office of Quality of Life Research at firstname.lastname@example.org for advice on the likely suitability of a study as a Registered Report. The designated lead editor for registered reports is Jan R. Boehnke.
A brief scientific case for consideration should be made:
Authors are encouraged to highlight the novelty of their research question and how the proposed study will contribute to the advancement of the field. The authors should also include a statement that the collection of data has not started, yet, and include an envisaged timeline counting from the point of in-principle acceptance.
Authors should include an estimated timeline for completing the study if the initial submission is accepted in principle. The authors should indicate the time the project will take to be conducted and when the submission of the full paper is planned. The general expectation is that the full paper (stage 2) will be submitted six months after the data collection has finished.
Authors should include a statement confirming that all necessary support (e.g. funding, facilities) and approvals (e.g. ethics) are in place for the proposed research. Note that manuscripts will be generally considered only for studies that are able to commence in the next three months; however, authors with alternative plans are encouraged to contact the editorial team for advice.
This section should be written as a standard background section to a manuscript and clearly provide the theoretical background for and derivation of the research questions for this manuscript.Please note that following in principle acceptance, the Background section should not be altered during the Stage 2 process.
The general rule for writing the Methods and Results sections for a Registered Report is that they should be written as for usual manuscript submission (e.g. including use of and adherence to best-practice reporting guidelines), only that exact numbers of people recruited, descriptive and inferential statistics etc are missing. The methods section should include:
- Full description of proposed sample characteristics, incl. inclusion/exclusion criteria for participants, how these are assessed etc. (in principle supporting the flow diagram following relevant guidelines for the submission category such as CONSORT, STROBE, PRISMA).
- Full description of data inclusion and exclusion (e.g. outlier extraction). Procedures for objectively defining exclusion criteria due to technical errors or for any other reasons must be specified, including details of how and under what conditions data would be replaced (see missing data below).
- A description of all study procedures in sufficient detail to allow another researcher to repeat the methodology exactly, without requiring further information. These procedures must be adhered to exactly in the subsequent experiments or any Stage 2 manuscript can be rejected.
- If interventions (experimental, treatment, service change) are part of the study, these need to be described in sufficient detail, ideally based on appropriate reporting guidelines (e.g., TIDieR).
- A detailed description of the planned quantitative or qualitative analysis steps. For quantitative data this includes all pre-processing steps, and a precise description of all planned analyses, including appropriate correction for multiple comparisons. See the table below for examples of typical specific methodological requirements for different types of studies. Moreover, any covariates or regressors must be stated. Where analysis decisions are contingent on the outcome of prior analyses, these contingencies must be specified and adhered to. Only pre-planned analyses can be reported in the main Results section of Stage 2 submissions. However, unplanned exploratory analyses will be admissible in a separate section of the Results (see below)
- Quantitative studies must include a justification of the planned sample size that is relevant to the particular study design (e.g., power analysis, precision analysis or Bayesian equivalents) and the description should be detailed enough to allow the replication of this calculation. Estimated effect sizes should be justified with reference to the existing literature. Since publication bias overinflates published estimates of effect size, a power analysis must be based on the lowest available or meaningful estimate of the effect size.
- In the case of highly uncertain effect sizes, a variable sample size and interim data analysis is permissible but with inspection points stated in advance, appropriate error correction for ‘peeking’ employed, and a final stopping rule for data collection outlined (a priori stopping boundaries should be clearly defined and referenced.
- In the case of a flexible adaptive designs (e.g. re-estimation of sample size) or other SMART trial or study designs all elements and decision points need to be defined and referenced.
- The approach to handle missing data should be described. It should detail if and when the imputed analyses or the observed data analyses are used as a the primary source for decision making; if and when imputations or other approaches for dealing with missing data are employed; and which type of approach for dealing with missing data is used and if a sensitivity analysis is planned, using different approaches.
- In case of an experimental study/ clinical trial, the main analysis should be reported as 'intention-to-treat'.
- Methods involving Bayesian hypothesis testing are encouraged. For studies involving analyses with Bayes factors, the predictions of the theory must be specified so that a Bayes factor can be calculated. Authors should indicate what distribution will be used to represent the predictions of the theory and how its parameters will be specified. For example, whether a uniform up to some specified maximum, or a normal/half-normal to represent a likely effect size, or a JZS/Cauchy with a specified scaling constant is used? For inference by Bayes factors, authors must be able to guarantee data collection until the Bayes factor is at least 6 times in favour of the experimental hypothesis over the null hypothesis (or vice versa). Authors with resource limitations are permitted to specify a maximum feasible sample size at which data collection must cease regardless of the Bayes factor; however to be eligible for advance acceptance this number must be sufficiently large that inconclusive results at this sample size would nevertheless be an important message for the field.
- Full descriptions must be provided of any outcome-neutral criteria that must be met for successful testing of the stated hypotheses. Such quality checks might include the absence of floor or ceiling effects in data distributions, positive controls, or other quality checks that are orthogonal to the experimental hypotheses.
- Since registered reports for qualitative studies are at the moment rare, no particular requirements are stated for the Methods section beyond the following: (i) the sample size and sample collection methods should be defined in advance and if sampling is based on theoretical saturation and/or purposive sample characteristics, the strategy should be described to achieve this and the stopping criterion employed; (ii) the analytic framework, its theoretical foundation and how it will be applied should be described; (iii) the procedures to elicit the data, pre-processing (e.g., transcription) and the steps for performing the analysis should be described (incl. number of raters, number of rounds of review, inclusion of external stakeholders etc); (iv) and any other procedures that will be used when moving from the original data to an interpretation.
- Ethics. All Registered Reports need to comply with the current ethical standards required by Quality of Life Research. Quality of Life Research will review Registered Reports in parallel to submitted ethics applications (please confirm submission in cover letter), but an in-principle acceptance can only be given after all approvals are in place. Please include all relevant ethics information (e.g., approving institutional review board, submission number, consent procedures) in the Methods section.Any description of prospective methods or analysis plans should be written in future tense.
- A results section at the Stage 1 submission optional, but in line with common standards of protocol and statistical analysis plan writing, we encourage authors to also submit the prospectively planned parts of the results section. These are typically the descriptive tables for the sample obtained and the primary outcome analysis. These can usually be written without the data having been obtained and the relevant tables be prepared as empty shells.
- Pilot Data: Optional. Can be included to establish proof of concept, effect size estimations, or feasibility of proposed methods. Any pilot experiments will be published with the final version of the manuscript and will be clearly distinguished from data obtained for the registered experiment(s).
- A Discussion section at the Stage 1 submission is optional. We nevertheless encourage authors to provide the first paragraph of the Discussion section that reiterates the goal and offers an interpretation of the main findings.
Following Stage 1 peer review manuscripts will either be rejected, given the opportunity to revise or offered an in-principle acceptance. An in-principle acceptance indicates that the article will be published pending completion of the approved methods and analytic procedures, in addition to a defensible interpretation of the results.
Authors are reminded that any deviation from the stated procedures and change of the wording of the accepted registered report (apart from providing the gathered empirical data), regardless of how minor it may seem to the authors, could lead to rejection of the manuscript at Stage 2. In cases where the registered report is altered after in-principle acceptance due to unforeseen circumstances (e.g. change of equipment or technical error), the authors must consult the Editors immediately for advice, and prior to the completion of data collection. Minor changes to the protocol may be permitted according to editorial discretion. In such cases, in-principle acceptance would be preserved and the deviation reported in the Stage 2 submission. If the authors wish to alter the experimental procedures more substantially following in-principle acceptance but still wish to publish their article as a Registered Report, the manuscript must be withdrawn and resubmitted as a new Stage 1 submission.
Note that registered analyses must be undertaken, but additional unregistered analyses can also be included in a final manuscript (see below).
Stage 2: Full manuscript submission and review
Authors will be asked to complete their study and resubmit their manuscripts within the agreed timeline following IPA. Failure to do so may result in a Stage 1 manuscript being considered withdrawn.
A Stage 2 submission should include the following additions:
Confirmation in that all non-pilot data was collected after the IPA and analysed in the manner previously described during stage 1. Any unforeseen changes in methods and analyses must be clearly noted.
Please note that none of the text submitted at Stage 1 should be altered from the approved Stage 1 submission, and that the stated hypothesis cannot be amended or expanded.
Results & Discussion
The outcome of all registered analyses must be reported in the manuscript, except in rare instances where a registered and approved analysis s subsequently shown to be logically flawed or unfounded. In such cases, the authors, reviewers, and editor must agree that a collective error of judgment was made and that the analysis was inappropriate. In such cases the analysis would still be mentioned in the Methods but omitted with justification from the Results.
It is reasonable that authors may wish to include additional analyses that were not included in the registered submission. For instance, a new analytic approach might become available between IPA and Stage 2 review, or a particularly interesting and unexpected finding may emerge. Such analyses are admissible but must be clearly justified in the text, appropriately caveated, and reported in a separate section of the Results titled “Exploratory analyses”. Authors should be careful not to base their conclusions on these exploratory analyses.
Authors reporting null hypothesis significance tests are required to report exact p-values and effect sizes for all inferential analyses.
The Discussion section should be written as for any empirical paper. The discussion should put the study and its result into the context of the wider literature, incl. potential developments and milestone papers since the registration (as the registered Background section needs to remain unchanged). The Discussion should include a section discussing strengths and limitations of the conducted study.
Please see our submission guidelines for details of the additional declarations that must be included, as with standard article types.
Manuscripts can be withdrawn at the authors’ discretion following in-principle acceptance. However, the journal will publish the manuscript’s proposed title, author names, the abstract from the approved Stage 1 submission and a brief reason for the failure to complete the study.
Information for Reviewers
The review process for Registered Reports is divided into two stages. At Stage 1, reviewers are asked to assess study proposals before data are collected. At Stage 2, reviewers consider the full study, including results and interpretation.
Stage 1: Initial manuscript submission and review
Stage 1 manuscripts will include only a Background and Methods (including proposed analyses) sections, optionally a results section that shows all the proposed reporting for the main analyses (but without numbers or with pilot data), and an optional first discussion paragraph. In considering papers at Stage 1, we ask reviewers to assess:
- The importance of the research question(s)and its potential implication for future research, policy or practice in the field of study.
- The novelty and need for the study vis-à-vis existing literature and the arguments presented by the authors in the Background section.
- The logic, rationale, and plausibility of the proposed hypotheses.
- The soundness/scientific quality and feasibility of the planned methodology and analysis.
- Whether the clarity and degree of methodological detail is sufficient to enable replication the proposed experimental procedures and analysis.
- Whether the authors have pre-specified sufficient outcome-neutral tests for ensuring that the results obtained are able to test the stated hypotheses, including positive controls and quality checks, or thematic analyses and data saturation methods in the case of qualitative studies
Following Stage 1 peer review, manuscripts will be accepted, offered the opportunity to revise, or rejected outright. Manuscripts that pass peer review will be issued an in-principle acceptance, indicating that the article will be published pending successful completion of the study according to the registered methods and analytic procedures, as well as a defensible and evidence-based interpretation of the results.
Stage 2: Full manuscript submission and review
Following completion of the study, authors will complete the manuscript, including Results and Discussion sections. These Stage 2 manuscripts will more closely resemble a regular article format. The manuscript will then be returned to the reviewers, who will be asked to appraise:
- Whether the data are able to test the authors’ proposed hypotheses by satisfying the approved outcome-neutral conditions (such as quality checks, positive controls).
- Whether the background, rationale and stated hypotheses are the same as the approved Stage 1 submission (required).
- Whether the authors adhered precisely to the registered methods and procedures.Whether any unregistered post hoc analyses added by the authors are justified, methodologically sound, and informative.
- Whether the authors’ conclusions are justified given the data and the original registration paper.
Reviewers at Stage 2 may suggest that authors report additional post hoc tests on their data; however, authors are not obliged to do so unless such tests are necessary to satisfy one or more of the Stage 2 review criteria. Please note that editorial assessments will not be based on the perceived importance or conclusiveness of the results, and neither should reviewers re-assess the novelty of the research question at Stage 2.