Using Qualitative Evidence in Decision Making for Health and Social Interventions: An Approach to Assess Confidence in Findings from Qualitative Evidence Syntheses (GRADE-CERQual)

1. Lewin S, Bosch-Capblanch X, Oliver S, Akl EA, Vist GE, et al. (2012) Guidance for Evidence-Informed Policies about Health Systems: Assessing How Much Confidence to Place in the Research Evidence. PLoS Med 9: e1001187. doi: 10.1371/journal.pmed.1001187 22448147

2. Oxman AD, Lavis JN, Lewin S, Fretheim A (2009) SUPPORT Tools for evidence-informed health Policymaking (STP) 1: What is evidence-informed policymaking? Health Res Policy Syst 7 Suppl 1: S1. doi: 10.1186/1478-4505-7-S1-S1 20018099

3. Gulmezoglu AM, Chandler J, Shepperd S, Pantoja T (2013) Reviews of qualitative evidence: a new milestone for Cochrane. Cochrane Database Syst Rev 11: ED000073. doi: org/10.1002/14651858.ED000073 24524152

4. Noyes J, Gough D, Lewin S, Mayhew A, Michie S, et al. (2013) A research and development agenda for systematic reviews that ask complex questions about complex interventions. Journal of Clinical Epidemiology 66: 1262–1270. doi: 10.1016/j.jclinepi.2013.07.003 23953084

5. Petticrew M, Rehfuess E, Noyes J, Higgins JPT, Mayhew A, et al. (2013) Synthesizing evidence on complex interventions: how meta-analytical, qualitative, and mixed-method approaches can contribute. Journal of Clinical Epidemiology 66: 1230–1243. doi: 10.1016/j.jclinepi.2013.06.005 23953082

6. Petticrew M, Roberts H (2006) Systematic Reviews in the Social Sciences: A Practical Guide. Oxford, UK: Wiley-Blackwell.

7. Gough D, Thomas J, Oliver S (2012) Clarifying differences between review designs and methods. Syst Rev 1: 28. doi: 10.1186/2046-4053-1-28 22681772

8. Thomas J, Sutcliffe K, Harden A, Oakley A, Oliver S, et al. (2003) Children and healthy eating: a systematic review of barriers and facilitators. London: EPPI-Centre, Social Science Research Unit, Institute of Education, University of London.

9. Glenton C, Colvin CJ, Carlsen B, Swartz A, Lewin S, et al. (2013) Barriers and facilitators to the implementation of lay health worker programmes to improve access to maternal and child health: qualitative evidence synthesis. Cochrane Database Syst Rev 10: CD010414. doi: 10.1002/14651858.CD010414.pub2 24101553

10. Guyatt G, Oxman AD, Akl EA, Kunz R, Vist G, et al. (2011) GRADE guidelines: 1. Introduction-GRADE evidence profiles and summary of findings tables. J Clin Epidemiol 64: 383–394. doi: 10.1016/j.jclinepi.2010.04.026 21195583

11. Goldsmith MR, Bankhead CR, Austoker J (2007) Synthesising quantitative and qualitative research in evidence-based patient information. J Epidemiol Community Health 61: 262–70. 17325406

12. Munn Z, Porritt K, Lockwood C, Aromataris E, Pearson A (2014) Establishing confidence in the output of qualitative research synthesis: the ConQual approach. BMC Medical Research Methodology 14: 108. doi: 10.1186/1471-2288-14-108 25927294

13. Treweek S, Oxman AD, Alderson P, Bossuyt PM, Brandt L, et al. (2013) Developing and Evaluating Communication Strategies to Support Informed Decisions and Practice Based on Evidence (DECIDE): Protocol and Preliminary Results. Implement Sci 9; 8: 6. doi: 10.1186/1748-5908-8-6 23302501

14. WHO (2012) Optimizing health worker roles to improve access to key maternal and newborn health interventions through task shifting. Geneva: World Health Organization.

15. Guyatt GH, Oxman AD, Vist GE, Kunz R, Falck-Ytter Y, et al. (2008) GRADE: an emerging consensus on rating quality of evidence and strength of recommendations. BMJ 336: 924–926. doi: 10.1136/bmj.39489.470347.AD 18436948

16. Bohren MA, Hunter EC, Munthe-Kaas HM, Souza JP, Vogel JP, et al. (2014) Facilitators and barriers to facility-based delivery in low- and middle-income countries: a qualitative evidence synthesis. Reprod Health 11: 71. doi: 10.1186/1742-4755-11-71 25238684

17. Colvin CJ, de Heer J, Winterton L, Mellenkamp M, Glenton C, et al. (2013) A systematic review of qualitative evidence on barriers and facilitators to the implementation of task-shifting in midwifery services. Midwifery 29: 1211–1221. doi: 10.1016/j.midw.2013.05.001 23769757

18. Munthe-Kaas HM, Hammerstrøm KT, Kurtze N, Nordlund KR (2013) Effekt av og erfaringer med kontinuitetsfremmende tiltak i barnevernsinstitusjoner. Oslo: Norwegian Knowledge Centre for the Health Services. Available: http://www.kunnskapssenteret.no/publikasjoner/effekt-av-og-erfaringer-med-kontinuitetsfremmende-tiltak-i-barnevernsinstitusjoner

19. Rashidian A, Shakibazadeh E, Karimi- Shahanjarini A, Glenton C, Noyes J, et al. (2013) Barriers and facilitators to the implementation of doctor-nurse substitution strategies in primary care: qualitative evidence synthesis (Protocol). Cochrane Database of Systematic Reviews 2: CD010412.

20. Lewin S, Glenton C, Munthe-Kaas H, al. e (2013) Assessing how much certainty to place in findings from qualitative evidence syntheses: the CerQual approach. Oral presentation, 20th Cochrane Colloquium, Quebec 2013. 20th Cochrane Colloquium. Quebec.

21. Petticrew M, Egan M, Thomson H, Hamilton V, Kunkler R, et al. (2008) Publication bias in qualitative research: what becomes of qualitative research presented at conferences? J Epidemiol Community Health 62: 552–554. doi: 10.1136/jech.2006.059394 18477755

22. Barbour RS (2001) Checklists for improving rigour in qualitative research: a case of the tail wagging the dog? BMJ 322: 1115–1117. 11337448

23. Barbour RS, Barbour M (2003) Evaluating and synthesizing qualitative research: the need to develop a distinctive approach. J Eval Clin Pract 9: 179–186. 12787181

24. Dixon-Woods M, Fitzpatrick R, Roberts K (2001) Including qualitative research in systematic reviews: opportunities and problems. J Eval Clin Pract 7: 125–133. 11489038

25. CASP (2011) Qualitative Appraisal Checklist for Qualitative Research. Critical Appraisal Skills Programme. www.casp-uk.net/#!casp-tools-checklists/c18f8

26. Government Chief Social Researcher's Office (2003) Quality in Qualitative Evaluation: A framework for assessing research evidence. United Kingdom: Cabinet Office.

27. Walsh D, Downe S (2006) Appraising the quality of qualitative research. Midwifery 22: 108–119. 16243416

28. Booth A, Carroll C, Ilott I, Low LL, Cooper K (2013) Desperately seeking dissonance: identifying the disconfirming case in qualitative evidence synthesis. Qual Health Res 23: 126–141. doi: 10.1177/1049732312466295 23166156

29. Ponterotto JG (2006) Brief note on the origins, evolution, and meaning of the qualitative research concept “Thick description”. The Qualitative Report 11: 538–549.

30. Suri H (2011) Purposeful sampling in qualitative research synthesis. Qualitative Research Journal 11: 63–75.

31. Bowen GA (2008) Naturalistic inquiry and the saturation concept: a research note. Qualitative research 8: 16.

32. Francis J, Johnston M, Robertson C, Glidewel L, Entwistle V, et al. (2010) What is an adequate sample size? Operationalising data saturation for theory-based interview studies. Psychology & Health 25: 1229–1245.

33. Guest G, Bunce A, Johnson L (2006) How many interviews are enough? An experiment with data saturation and variability. Field methods 18: 59–82.

34. O'Reilly M, Parker N (2013) 'Unsatisfactory Saturation': a critical exploration of the notion of saturated sample sizes in qualitative research. Qualitative Research 13: 190–197.

35. Guyatt G, Oxman AD, Sultan S, Brozek J, Glasziou P, et al. (2013) GRADE guidelines: 11. Making an overall rating of confidence in effect estimates for a single outcome and for all outcomes. J Clin Epidemiol 66: 151–157. doi: 10.1016/j.jclinepi.2012.01.006 22542023

36. Bohren MA, Vogel JP, Hunter EC, Lutsiv O, Makh SK, et al. (2015) The Mistreatment of Women during Childbirth in Health Facilities Globally: A Mixed-Methods Systematic Review. PLoS Med 12: e1001847; discussion e1001847. doi: 10.1371/journal.pmed.1001847 26126110

37. Dixon-Woods M, Shaw RL, Agarwal S, Smith JA (2004) The problem of appraising qualitative research. Qual Saf Health Care 13: 223–225. 15175495

38. Dixon-Woods M, Sutton A, Shaw R, Miller T, Smith J, et al. (2007) Appraising qualitative research for inclusion in systematic reviews: a quantitative and qualitative comparison of three methods. J Health Serv Res Policy 12: 42–47. 17244397

39. Murphy E, Dingwall R, Greatbatch D, Parker S, Watson P (1998) Qualitative research methods in health technology assessment: a review of the literature. Health Technol Assess 2: iii–ix, 1–274. 9919458

40. Tong A, Sainsbury P, Craig J (2007) Consolidated criteria for reporting qualitative research (COREQ): a 32-item checklist for interviews and focus groups. Int J Qual Health Care 19: 349–357. 17872937

41. Patton MQ (1999) Enhancing the quality and credibility of qualitative analysis. Health Services Research 35: 1189–1208.

42. Sandelowski M (2001) Real qualitative researchers do not count: The use of numbers in qualitative research. Research in Nursing & Health 24: 230–240.

43. Berkowitz S (1997) Analyzing qualitative data. In: Frechtling J, Sharp L, editors. User-friendly handbook for mixed method evaluations. Arlington, VA: Division of Research, Evaluation and Communication, National Science Foundation.

44. Booth A, Papaioannou D, Sutton A (2012) Systematic Approaches to a Successful Literature Review. London, UK: Sage Publications.

45. Popay J, Roberts H, Sowden A (2006) Guidance on the Conduct of Narrative Synthesis in Systematic Reviews. A Product from the ESRC Methods Programme. Lancaster: Institute of Health Research.

46. Barnett-Page E, Thomas J (2009) Methods for the synthesis of qualitative research: a critical review. BMC Med Res Methodol 9: 59. doi: 10.1186/1471-2288-9-59 19671152