Literature reviews are an integral part of the process and advice of scientific research. Whilst systematic reviews have become regarded every bit the highest standard of prove synthesis, many literature reviews fall curt of these standards and may stop upward presenting biased or incorrect conclusions. In this mail service, Neal Haddaway highlights 8 common problems with literature review methods, provides examples for each and provides practical solutions for ways to mitigate them.

Researchers regularly review the literature – it's an integral part of day-to-24-hour interval enquiry: finding relevant research, reading and digesting the main findings, summarising across papers, and making conclusions near the prove base of operations every bit a whole. Still, there is a key difference betwixt brief, narrative approaches to summarising a selection of studies and attempting to reliably and comprehensively summarise an evidence base of operations to support decision-making in policy and practice.

Then-called 'evidence-informed decision-making' (EIDM) relies on rigorous systematic approaches to synthesising the evidence. Systematic review has become the highest standard of evidence synthesis and is well established in the pipeline from research to practice in the field of wellness. Systematic reviews must include a suite of specifically designed methods for the carry and reporting of all synthesis activities (planning, searching, screening, appraising, extracting information, qualitative/quantitative/mixed methods synthesis, writing; due east.thou. encounter the Cochrane Handbook). The method has been widely adapted into other fields, including environment (the Collaboration for Ecology Evidence) and social policy (the Campbell Collaboration).

Despite the growing involvement in systematic reviews, traditional approaches to reviewing the literature keep to persist in contemporary publications beyond disciplines. These reviews, some of which are incorrectly referred to every bit 'systematic' reviews, may exist susceptible to bias and equally a consequence, may end upwards providing incorrect conclusions. This is of detail concern when reviews accost key policy- and exercise- relevant questions, such as the ongoing COVID-19 pandemic or climate alter.

These limitations with traditional literature review approaches could exist improved relatively hands with a few key procedures; some of them not prohibitively costly in terms of skill, time or resources.

In our recent paper in Nature Ecology and Evolution, we highlight eight common problems with traditional literature review methods, provide examples for each from the field of environmental direction and ecology, and provide applied solutions for ways to mitigate them.

Trouble Solution
Lack of relevance – express stakeholder appointment can produce a review that is of limited applied use to conclusion-makers Stakeholders tin can be identified, mapped and contacted for feedback and inclusion without the need for extensive budgets – bank check out best-practise guidance
Mission creep – reviews that don't publish their methods in an a priori protocol tin endure from shifting goals and inclusion criteria Carefully design and publish an a priori protocol that outlines planned methods for searching, screening, data extraction, critical appraisement and synthesis in item. Make utilise of existing organisations to support you (eastward.g. the Collaboration for Environmental Bear witness).
A lack of transparency/replicability in the review methods may mean that the review cannot be replicated – a central tenet of the scientific method! Be explicit, and brand use of high-quality guidance and standards for review deport (east.g. CEE Guidance) and reporting (PRISMA or ROSES)
Choice bias (where included studies are non representative of the testify base) and a lack of comprehensiveness (an inappropriate search method) tin can hateful that reviews end up with the incorrect testify for the question at mitt Carefully design a search strategy with an info specialist; trial the search strategy (against a criterion listing); use multiple bibliographic databases/languages/sources of grey literature; publish search methods in an a priori protocol for peer-review
The exclusion of grey literature and failure to test for evidence of publication bias can result in wrong or misleading conclusions Include attempts to detect greyness literature, including both 'file-drawer' (unpublished bookish) enquiry and organisational reports. Test for possible show of publication bias.
Traditional reviews often lack advisable critical appraisal of included report validity, treating all bear witness as equally valid – we know some research is more valid and we need to account for this in the synthesis. Advisedly programme and trial a critical appraisal tool before starting the procedure in total, learning from existing robust critical appraisal tools.
Inappropriate synthesis (e.g. using vote-counting and inappropriate statistics) tin can negate all of the preceding systematic effort. Vote-counting (tallying studies based on their statistical significance) ignores report validity and magnitude of issue sizes. Select the synthesis method carefully based on the data analysed. Vote-counting should never be used instead of meta-analysis. Formal methods for narrative synthesis should be used to summarise and describe the evidence base of operations.

At that place is a lack of awareness and appreciation of the methods needed to ensure systematic reviews are as free from bias and as reliable as possible: demonstrated by recent, flawed, high-profile reviews. We call on review authors to conduct more rigorous reviews, on editors and peer-reviewers to gate-keep more than strictly, and the community of methodologists to better support the broader inquiry community. But by working together can we build and maintain a strong organization of rigorous, show-informed controlling in conservation and environmental direction.

Note: This article gives the views of the authors, and not the position of the LSE Touch Blog, nor of the London School of Economic science. Please review our comments policy if you accept any concerns on posting a annotate below

Image credit: Jaeyoung Geoffrey Kang via unsplash

Print Friendly, PDF & Email