A critique: The good and bad of a review
- DOI : 10.5667/tang.2015.0008
- Author: McMullen Debbie, McClean Rhett, Pak Sok Cheon
- Publish: TANG Volume 5, Issue3, p16.1~16.3, 31 Aug 2015
Evidence based medicine involves using both the individual clinician’s expertise and the current best available external clinical evidence from systematic research in deciding on the appropriate care for individual patients. The current approach to evidence based practice in healthcare adds a third component which is patient values. Evidence based practice is thus a triad, in which the practitioner’s expertise, research evidence and the patient’s values are all given consideration. The balance to be struck between them depends on the individual case. The literature indicates that complementary medicine practitioners are moving away from traditional knowledge and towards the use of evidence based practice in their clinical discussions. In the context of the daily practice of complementary medicine practitioners and their continuing development of their knowledge base of evidence based practice, this short review discusses the good and bad of a review journal article.
critique , review , methodology , bias , evidence
It is crucial that all health professionals uphold current best practice standards. In evidence based practice, this is founded in quality research. Time poor practitioners often opt to use reviews to achieve this. Therefore, the ability to objectively critique a review journal article to determine its strengths and weaknesses and its value to clinical practice is essential. Not all reviews are created equal; hence, it is important to recognize the advantages and disadvantages of a review. This short review looks at some of these features based on a published review article by Yuan (2013) entitled ‘Cancer prevention by green tea: Evidence from epidemiologic studies’.
Yuan’s article published in 2013 is a narrative review with a critical evaluation which describes and discusses the evidence from a compilation of previously published papers relating to the efficacy of green tea and/or green tea polyphenols and extracts for the prevention of human cancers (Green et al., 2006). This type of review uses qualitative rather than quantitative methods and although there are limitations in qualitative studies they can help to produce hypotheses and pin-point variables for future quantitative studies (Creswell, 2013; Ho et al., 2008). Details identifying data bases and search strategy, inclusion/exclusion criteria and data extraction methods are not usually included in this type of review. The author has not included these strategies and methods, hence, the reader cannot be certain of the author’s motives or biases when selecting articles or their validity (Collins and Fauser, 2005; Green et al., 2006). This review shows some degree of selection bias as many of the results are taken from previously published studies by or involving the author, nevertheless, this has been balanced with a review of other clinical trials and observational studies. Yuan’s review is relatively unsystematic despite the use of studies ranking high on the evidence hierarchy (Green et al., 2006; MeInyk and Fineout-Overholt, 2011). An unsystematic method with subjective selection of articles leaves a review open to a greater potential for selection bias and the possibility of the omission of valuable studies that could result in a different conclusion, hence, a non-objective review. Further, the use of good methodology is vital to a good quality review such as data extraction and analysis identifying how the findings were achieved; producing valid results and allowing the reader to judge the merit of the paper (Abalos et al., 2001; Green et al., 2006; MeInyk and Fineout-Overholt, 2011; Rother, 2007). In contrast to Yuan’s review, a similar review by Hou et al. (2013) regarding green tea and gastric cancer, provided full details on searching and methodology; identifying the three search engines and keywords used, inclusion/exclusion criteria and data extraction methods, also including a ‘flow diagram of identification of relevant studies’ which made it easy to identify their method. Thereby, making their review more systematic and less prone to bias gives the reader more confidence in the quality and validity of the review (Ho et al., 2008).
Green et al. (2006) point out that the title and abstract are important components used by data base indexing. Marshall (2005) notes that including keywords and capturing the main topic are essential for computer literature searches. By omitting keywords, this review may miss out on being included in search results. Therefore, the title, abstract and keywords are crucial aspects involved to ensure an article appears in the search results. A good title allows the reader to clearly interpret the topic of the study and should catch the attention of the reader to enable them to determine the relativity of the topic and/or interest to either inspire them to read or reject it (Caldwell et al., 2005; Marshall, 2005). In Yuan’s paper the title accurately reflects the focus of the paper, green tea and cancer prevention, together with the words ‘evidence from epidemiologic studies’ indicating other publications are summarized in this review, therefore allowing the reader to interpret the content of the study and enable data bases to include the review in search results; an unclear or confusing title can mislead the reader (Caldwell et al., 2011). According to Green et al. (2006), an abstract should present a brief summary of the main components including the aim and/or objective of the study, methodology and main findings. Marshall (2005) concurs by adding that readers use an abstract to judge its relevance to their particular needs. In Yuan’s review the abstract is brief, summarizing the content. However, there is no clear or specific statement of aims and/or objectives presented in this review which is a critical factor to focus the study and clearly identify what the research is attempting to achieve (Marshall, 2005). Boswell and Cannon (2014) note that in qualitative research a broad question rather than an hypothesis is usual. The closest to, but far from, a statement of aim is found in the introduction ‘the results of green tea consumption on the protection/risk of various cancer sites in humans assessed here are’. After reading Yuan’s abstract the reader has to assume the aims but does get a general idea of the direction of the review. This contrasts with the Hou et al. (2013) study which makes a very clear statement about the aim of their study with ‘The aim of this systematic and up-to-date review was to critically evaluate all epidemiological studies published so far to report an association between green tea consumption and GC risk’. Yuan does, however, provide a foundation for the review, noting previous research in animal models consistently showed positive results in prevention in contrast to the mixed results in human studies so far. According to Caldwell et al. (2011), the research should be placed in context by presenting a rationale for the research within current knowledge of the focus topic. Hence, the title and abstract should correctly represent the paper. The abstract identifies the cancers included in the review as oral/digestive tract cancers (oral, esophageal, gastric, colorectal), liver, lung, prostate and breast cancers and a brief overview of the study types used has been identified here, for some cancers. A very brief summary of findings for each cancer is included in the abstract with the main findings of the review reported as well as recommendations for further research. Therefore, given the information included in the abstract, it basically conforms to what is expected in an abstract of a narrative review except for the important statement of aims and/or objectives and omission of methodology not usually included in a narrative review (Caldwell et al., 2011; Green et al., 2006).
The quality of a paper is a reflection of the relevance, recency and design of studies used including whether primary or secondary sources are used (Ho et al., 2008; Marshall, 2005). As the author has a considerable amount of previously published research on this topic, it would be remiss to exclude them, however, it is a matter of perspective in considering if they are primary or secondary studies, although technically they are secondary. The literature included should be up-to-date and comprehensive (Caldwell et al., 2011). Given the paucity of research in complementary medicine (Ernst et al., 2004), this review’s inclusion of research mostly from the past ten years is reasonable. Generally, study design is the main measure of validity of the findings. Within this context, the hierarchy of evidence is the most widely used grading system placing studies in order of scientific rigor (Ho et al., 2008). For example, expert opinion is relegated to the bottom of the hierarchy because it is has the most risk of bias, therefore, more prone to threats of internal validity and therefore providing the weakest evidence. Systematic review, meta-analysis and randomized controlled trials (RCTs) are considered least prone to bias, have the most sound design providing the strongest evidence, accordingly, are top of the hierarchy (Hess, 2004; MeInyk and Fineout-Overholt, 2011). Yuan’s review includes mostly quality evidence, according to the evidence hierarchy, and includes large cohort studies, randomized, placebo-controlled trials including several phase 2 clinical trials, several meta analyses and systematic reviews, and a couple of case studies. Case studies can provide information not always reported in other studies such as clinical trials and can add to a practitioner’s knowledge (MeInyk and Fineout-Overholt, 2011).
Yuan’s paper is categorized into the specific cancers and their relevant studies. Within the categories each study reviewed has a brief description, results, discussion and recommendations with an overall analysis, main findings and recommendation for each cancer. A review should not just describe the studies included, but make a critical appraisal (Marshall, 2005). Yuan has given a critical examination, such as evidenced by noting in esophageal cancer that the inconsistencies in results across studies may be due to confounding effects, particularly in Asian populations where there are high rates of this cancer as well as high consumption of hot green tea. Further, noting the association between green tea consumption and a greater likelihood of smoking and alcohol. To clarify any association between green tea and esophageal cancer, Yuan recommends studies controlling these factors. Study descriptions are, however, inconsistent. For example, the study sample populations are included for some Chinese and Japanese populations, and not for others. It is important to clarify the sample population in order for the reader to decide if the research is relevant to their agenda (Boswell and Cannon, 2014). Further, data such as relative risk and confidence intervals appropriate for study types such as RCTs and cohort studies were not reported for all (MeInyk and Fineout-Overholt, 2011). Therefore, the reader is unable to make fair comparisons between studies. A summary table of the data would improve the review by ensuring all relevant data is included promoting clarity, making it easy and fast for the reader to assimilate the data without having to refer to the article as was presented in the Hou et al. (2013).
Boswell and Cannon (2014), and Caldwell et al. (2011) concur that conclusions must be based on the results of the included studies and should not make exaggerated claims. Further, that author recommendations for future study often arise from the conclusions (Marshall, 2005). The evidence provided in this review supports Yuan’s conclusions confirming green tea as a preventative in cancer in humans is inconclusive, in contrast to strong positive animal studies. However, he does surmise from the reviewed articles possible reasons for inconsistency and recommends further RCTs to provide more conclusive evidence on the effects of green tea on carcinogenesis and prevention in defined populations and particular cancers.
Health professionals need to recognize that the synthesis of information in a narrative review is the author’s interpretation and that another review will not necessarily return the same results (Webb and Roe, 2007). In evidence based practice, the best available evidence with the least likelihood of bias should be used in guiding clinical decisions (MeInyk and Fineout-Overholt, 2011). From a complementary medicine practitioner perspective, as a multidisciplinary profession, evidence comes from various research approaches and strengths of evidence (Caldwell et al., 2005). Due to the interpretive character of narrative reviews, the broader perspective of the topic rather than a clinical focus and lack of systematic approach, Yuan’s review should not be considered strong evidence on which to base clinical decisions. However, it is useful for keeping up-to-date with the research topic and to prompt further investigation/reading (Green et al., 2006; Webb and Roe, 2007).