19  Checklists

Adapted by UCD-SeRG team from original by Jade Benjamin-Chung

19.1 Pre-analysis plan checklist

  • Brief background on the study (a condensed version of the introduction section of the paper)
  • Hypotheses / objectives
  • Study design
  • Description of data
  • Definition of outcomes
  • Definition of interventions / exposures
  • Definition of covariates
  • Statistical power calculation
  • Statistical model description
  • Covariate selection / screening
  • Standard error estimation method
  • Missing data analysis
  • Assessment of effect modification / subgroup analyses
  • Sensitivity analyses
  • Negative control analyses

19.2 Code checklist

  • Does the script run without errors?
  • Is code self-contained within repo and/or associated Box folder?
  • Is all commented out code / remarks removed?
  • Does the header accurately describe the process completed in the script?
  • Is the script pushed to its github repository?
  • Does the code adhere to the coding style guide?
  • Are all warnings ignorable? Should any warnings be intentionally suppressed or addressed?

19.3 Manuscript checklist

This is adapted in part from How to tackle the reproducibility crisis in ten steps (Baker 2019).

  • Have you completed the relevant reporting checklist, if applicable? (See EQUATOR Network (“EQUATOR Network: Enhancing the QUAlity and Transparency of Health Research,” n.d.) for a collection of checklists)
  • Are the study results within the manuscript replicable (i.e., if you rerun the code in the study’s repository, the tables and figures will be exactly replicated?)
  • Is a target journal selected?
  • Is the title declarative, in other words, does it state the object/findings rather than suggest them?
  • Is the word count of the manuscript close to the target journal’s allowance?
  • Does the manuscript adhere to the formatting guide of the target journal?
  • Does the manuscript use a consistent voice (passive or active – usually active is preferred … pun intended)?
  • Is each figure and table (including supplementary material) referenced in the main text?
  • Is there a caption for each figure and table (including supplementary material)?
  • Are tables/figures and supplementary material numbered in accordance with their appearance in the main text?
  • Does the text use past tense if it is reporting research findings or future tense if it is a study protocol?
  • Does the text avoid subjective wording (e.g., “interesting”, “dramatic”)?
  • Does the text use minimal abbreviations, and are all abbreviations defined at first use?
  • Does the text avoid directionless words? (e.g., instead of writing, ‘Precipitation influences disease risk’, write, ‘Precipitation was associated with increased disease risk’).
  • Does the text avoid making causal claims that are not supported by the study design? Be careful about the words “effect”, “increase”, and “decrease”, which are often interpreted as causal.
  • Does the text avoid describing results with the word “significant”, which can easily be confused with statistical significance? (see references on this topic here)
  • Have you drafted author contributions? Do they follow the CRediT: Contributor Roles Taxonomy (“CRediT: Contributor Roles Taxonomy,” n.d.) for author contributions?

19.4 Figure checklist

  • Are the x-axis and y-axis labeled?
  • If the figure includes panels, is each panel labeled?
  • Are there sufficient numerical / text labels and breaks on the x-axis and y-axis?
  • Is the font size appropriate (i.e., large enough to read, not so large that it distracts from the data presented in the figure?)
  • Are the colors used colorblind friendly? See a colorblind-friendly palette here, a neat palette generator with colorblind options here, and an article on why this matters: The misuse of colour in science communication (Crameri, Shephard, and Heron 2020)
  • Are colors/shapes/line types defined in a legend?
  • Are the legends and other labels easy to understand with minimal abbreviations?
  • If there is overplotting, is transparency used to show overlapping data?
  • Are 95% confidence intervals or other measures of precision shown, if applicable?