Project 2: Should we standardise reporting in agricultural studies, and if so, how?
Supervisors: Professor William Sutherland and Dr Amelia Hood
Standardised reporting involves standardising how scientific studies are described, for example by creating a list of key variables that scientists must report. This practice is not currently adopted in the fields of conservation or ecology, but it is increasingly common in medicine. Standardised reporting reduces ambiguity and increases the completeness and comparability of studies. This is important for study methods to be interpreted correctly, particularly in the context of evidence synthesis, which is crucial for converting research to decision-making. Medical research has pioneered the practice of standardising reporting, potentially because it involves complex systems and the cost of mishaps is large. These features are also present in conservation research, and we propose piloting the use of standardised reporting in conservation with agriculture as a case study.
We have conducted a preliminary analysis of the standard of reporting in cassava agriculture and found that it is low. We looked at 45 key variables that were co-developed with a group of agricultural researchers and found that many important variables were underreported: e.g. 25% of studies did not report the plot size and 90% did not report the distance between plots. These are examples of information that should be reported, but is missing. However, bad reporting is not the only cause of missing information. For example, 75% of studies did not report herbicide use and 85% did not report irrigation practices. These are examples of information that is not reported because the researchers did not think that it was relevant, or because they only reported practices that were done rather than those that weren’t (i.e. they tend to only report herbicide use if it is applied). These are not examples of bad reporting, but this does result in information being left out that may later be useful for reinterpretation. Furthermore, listing practices that weren’t done would likely be little additional work for the researcher and it may increase the impact of their work as it would make it more useable.
We propose developing a standardised checklist that is used when reporting agricultural studies. Checklists are known to be effective safety measures in aviation and medicine, where they serve as memory aids, standardise performance across users, and promote communication. We searched agricultural journals and found that Ecology Letters was the only relevant journal that uses a checklist for reporting. This project will produce a more comprehensive checklist that will be promoted to a range of journals. To address the trade-off between thorough reporting and over-burdening scientists and practitioners, we propose co-developing this list with a large and representative sample of the agricultural community. Co-development can increasing checklist uptake. We imagine that the checklist would be added to the supplementary information in a standard format so that the written method would not be lengthened, but these details will be determined throughout the co-development process. An additional benefit of using standard tables would be that the information in them could easily be automatically extracted from publications, which would greatly increase the efficiency of mass data extraction, such as for evidence synthesis.
Key References
1. https://www.equator-network.org/.
2. Society, E. ESR paper on structured reporting in radiology. Insights Imaging 9, 1–7 (2018).
3. Chaparro, A., Keebler, J. R., Lazzara, E. H. & Diamond, A. Checklists: A Review of Their Origins, Benefits, and Current Uses as a Cognitive Aid in Medicine. Ergon. Des. 27, 21–26 (2019).
4. Ecology Letters. Checklist for reporting experimental details in manuscripts for Ecology Letters.
The Role of the Intern
The primary role of the intern is to review the standard of reporting in agricultural literature. The intern will search the literature to get a representative sample of agricultural meta-analyses, and then collate the data to quantify the standard of reporting in the studies within them. They will conduct statistical analyses on these data too (e.g. does the standard of reporting improve through time or vary regionally).
They will use these results to develop a network of stakeholders (academics, practitioners, editors of agricultural journals) interested in co-developing and/or promoting standardised reporting in agricultural research. We have already started this process and have identified national and international collaborators spanning academic and industry research institutes, and will widen this network once we have further data. The co-development of the standardised list will be beyond the lifetime of the project, but the student will be welcome to stay involved in any capacity once their internship has finished.
Student Profile
Essential
- Demonstrated interest in conservation
- Experience analysing datasets
- Comfortable with data wrangling and plotting in R
- Experience searching and critically appraising scientific literature
- Good team player
Desirable
- Demonstrated interest in agriculture
- Experience analysing large datasets
- Experience conducting linear models in R
- Experience applying science in practice (e.g. working on conservation projects or in the policy)
Expected Outputs
A review of the standard of reporting in agricultural literature, to later be used for publication. A network of stakeholders interested in co-developing and/or promoting standardised reporting in agricultural research.
