Mixed methods grant applications in the health sciences: An analysis of reviewer comments

English version

Autoři: Timothy C. Guetterman ^aff001; Rae V. Sakakibara ^aff002; Vicki L. Plano Clark ^aff003; Mark Luborsky ^aff004; Sarah M. Murray ^aff005; Felipe González Castro ^aff006; John W. Creswell ^aff002; Charles Deutsch ^aff007; Joseph J. Gallo ^aff005
Působiště autorů: Graduate School, Creighton University, Omaha, Nebraska, United States of America ^aff001; Department of Family Medicine, University of Michigan, Ann Arbor, Michigan, United States of America ^aff002; School of Education, University of Cincinnati, Cincinnati, Ohio, United States of America ^aff003; Institute of Gerontology, Wayne State University, Detroit, Michigan, United States of America ^aff004; Department of Mental Health, Johns Hopkins Bloomberg School of Public Health, Baltimore, Maryland, United States of America ^aff005; College of Nursing and Health Innovation, Arizona State University, Phoenix, Arizona, United States of America ^aff006; Harvard Catalyst, Harvard University, Boston, Massachusetts, United States of America ^aff007
Vyšlo v časopise: PLoS ONE 14(11)
Kategorie: Research Article
prolekare.web.journal.doi_sk: https://doi.org/10.1371/journal.pone.0225308

Souhrn

Our aim was to understand how reviewers appraise mixed methods research by analyzing reviewer comments for grant applications submitted primarily to the National Institutes of Health. We requested scholars and consultants in the Mixed Methods Research Training Program (MMRTP) for the Health Sciences to send us summary statements from their mixed methods grant applications and obtained 40 summary statements of funded (40%) and unfunded (60%) mixed methods grant applications. We conducted a document analysis using a coding rubric based on the NIH Best Practices for Mixed Methods Research in the Health Sciences and allowed inductive codes to emerge. Reviewers favorably appraised mixed methods applications demonstrating coherence among aims and research design elements, detailed methods, plans for mixed methods integration, and the use of theoretical models. Reviewers identified weaknesses in mixed methods applications that lacked methodological details or rationales, had a high participant burden, and failed to delineate investigator roles. Successful mixed methods applications convey assumptions behind the methods chosen to accomplish specific aims and clearly detail the procedures to be taken. Investigators planning to use mixed methods should remember that reviewers are looking for both points of view.

Klíčová slova:

Data management – Medicine and health sciences – Research design – Qualitative studies – Careers in research – Research grants – Social research – Health services research

Zdroje

1. Creswell JW, Plano Clark VL. Designing and conducting mixed methods research. 3rd ed. Washington, D.C.: Sage Publications; 2017.

2. Creswell JW, Plano Clark VL, Gutmann ML, Hanson WE. Advanced mixed methods research designs. In: Tashakkori A, Teddlie C, editors. Handbook of mixed methods in social and behavioral research. Thousand Oaks: Sage; 2003. p. 209–40.

3. O'Cathain A. A practical guide to using qualitative research with randomized controlled trials. Oxford, UK: Oxford University Press; 2018.

4. Dziak JJ, Nahum-Shani I, Collins LM. Multilevel factorial experiments for developing behavioral interventions: Power, sample size, and resource considerations. Psychol Methods. 2012;17(2):153–75. doi: 10.1037/a0026972 22309956

5. Kravitz RL, Duan N, Braslow J. Evidence-based medicine, heterogeneity of treatment effects, and the trouble with averages. Milbank Q. 2004;82(4):661–87. doi: 10.1111/j.0887-378X.2004.00327.x 15595946.

6. Kawachi I, Berkman LF. Neighborhoods and health. Oxford: Oxford University Press; 2003.

7. Curry L, Shield R, Wetle T, editors. Improving aging and public health research: Qualitative and mixed methods. Washington, D.C.: American Public Health Association; 2006.

8. Plano Clark VL. The adoption and practice of mixed methods: U.S. trends in federally funded health-related research. Qual Inq. 2010;16:428–40.

9. Coyle CE, Schulman-Green D, Feder S, Toraman S, Prust ML, Plano Clark VL, et al. Federal funding for mixed methods research in the health sciences in the United States: Recent trends. J Mix Methods Res. 2018;12(3):1–20.

10. Wisdom JP, Cavaleri MA, Onwuegbuzie AJ, Green CA. Methodological reporting in qualitative, quantitative, and mixed methods health services research articles. Health Serv Res. 2012;47(2):721–45. doi: 10.1111/j.1475-6773.2011.01344.x 22092040; PubMed Central PMCID: PMC3419885.

11. Creswell JW, Klassen AC, Plano Clark VL, Clegg Smith K, Meisser HF. Best practices for mixed methods research in the health sciences. Washington, D.C.: Commissioned by the Office of Behavioral and Social Sciences Research (OBSSR), 2011.

12. NIH Office of Behavioral and Social Sciences. Best practices for mixed methods research in the health sciences. 2nd ed. Bethesda, MD: National Institutes of Health; 2018.

13. Dahlberg B, Wittink M, Gallo JJ. Funding and publishing integrated studies: Writing effective mixed methods manuscripts and grant proposals. In: Tashakkori A, Teddlie C, editors. Handbook of mixed methods in social & behavioral research. 2nd ed. London: Sage Publications; 2010. p. 775–802.

14. Wisdom JP, Fetters MD. Funding for mixed methods research: Sources and strategies. In: Hesse-Biber SN, Johnson RB, editors. The Oxford handbook of multimethod and mixed methods research inquiry. New York, New York: Oxford University Press; 2015. p. 314–32.

15. Curry L, Nunez-Smith M. Mixed methods in health sciences research: A practical primer. Thousand Oaks, California: Sage Publishers; 2014.

16. O'Cathain A. Writing a proposal. A practical guide to using qualitative research with randomized controlled trials: Oxford University Press; 2018. p. 79–87.

17. Guetterman T, Creswell JW, Deutsch C, Gallo JJ. Skills development and academic productivity of scholars in the NIH Mixed Methods Research Training Program for the Health Sciences (invited publication). Int J Mult Res Approaches. 2018;10(1):1–17.

18. Guetterman T, Creswell JW, Deutsch C, Gallo JJ. Process evaluation of a retreat for scholars in the first cohort: The NIH Mixed Methods Research Training Program for the Health Sciences. J Mix Methods Res. 2019;13:52–68. doi: 10.1177/1558689816674564 30631250

19. Guetterman T, Creswell JW, Wittink MN, Barg FK, Castro F, Dahlberg B, et al. Development of a self-rated mixed methods skills assessment: The NIH Mixed Methods Research Training Program for the Health Sciences. J Contin Educ Health Prof. 2017;37(2):76–82. doi: 10.1097/CEH.0000000000000152 28562495

20. Braun V, Clarke V. Using thematic analysis in psychology. Qual Res Psychol. 2006;3(2):77–101.

21. Cheek J. It depends: Possible impacts of moving the field of mixed methods research toward best practice guidelines. In: Hesse-Biber SN, Johnson RB, editors. The Oxford handbook of multimethod and mixed methods research inquiry. New York, New York: Oxford University Press; 2015. p. 314–32.

22. Luborsky M, Sankar A. Cultural forces in the acceptance of qualitative research: Advancing mixed method research. In: Curry L, Shield R, Wetle T, editors. Improving aging and public health research: Qualitative and mixed methods. Washington, D.C.: American Public Health Association; 2006.

23. Polit DF, Beck CT. Generalization in quantitative and qualitative research: Myths and strategies. Int J Nurs Stud. 2010;47(11):1451–8. doi: 10.1016/j.ijnurstu.2010.06.004 20598692.

24. Morse JM. Qualitative generalizability. Qual Health Res. 1999;9(1):5–6.

25. Chmiel JA, Chmiel M. Generalization in and from qualitative analysis. In: Flick U, editor. The SAGE handbook of qualitative data analysis. London: Sage Publications; 2013.

26. Myers M. Qualitative research and the generalizability question: Standing firm with Proteus. Qual Rep. 2000;4.

27. Luborsky M, Rubinstein R. Sampling in qualitative research: Rationales, issues, and methods. Res Aging. 1995;17(1):89–113. doi: 10.1177/0164027595171005 22058580

28. Teddlie C, Yu F. Mixed methods sampling: A typology with examples. J Mix Methods Res. 2007;1:77–100.

29. Collins KM, Onwuegbuzie AJ, Jiao QG. A mixed methods investigation of mixed methods sampling designs in social and health science research. J Mix Methods Res. 2007;1:267–94.

30. Kraemer HC, Mintz J, Noda A, Tinklenberg J, Yesavage JA. Caution regarding the use of pilot studies to guide power calculations for study proposals. Arch Gen Psychiatry. 2006;63(5):484–9. doi: 10.1001/archpsyc.63.5.484 16651505.

31. Drabble SJ, O'Cathain A, Thomas KJ, Rudolph A, Hewison J. Describing qualitative research undertaken with randomised controlled trials in grant proposals: A documentary analysis. BMC Med Res Methodol. 2014;14:24. doi: 10.1186/1471-2288-14-24 24533771; PubMed Central PMCID: PMC3937073.

32. Creswell JW, Poth CN. Qualitative inquiry and research design: Choosing among five approaches. 4th ed. Thousand Oaks, CA: Sage; 2018.

33. Sandelowski M, Barroso J. Writing the proposal for a qualitative research methodology project. Qual Health Res. 2003;13(6):781–820. doi: 10.1177/1049732303013006003 12891715.

34. Castro FG, Kellison JG, Boyd SJ, Kopak A. A methodology for conducting integrative mixed methods research and data analyses. J Mix Methods Res. 2010;4(4):342–60. doi: 10.1177/1558689810382916 22167325; PubMed Central PMCID: PMC3235529.

35. Hesse-Biber S. Feminist approaches to triangulation: Uncovering subjugated knowledge and fostering social change in mixed methods research. J Mix Methods Res. 2012;6:137–46.

36. Fetters MD, Curry LA, Creswell JW. Achieving integration in mixed methods designs-principles and practices. Health Serv Res. 2013;48(6 Pt 2):2134–56. doi: 10.1111/1475-6773.12117 24279835; PubMed Central PMCID: PMC4097839.

37. Fetters MD, Molina-Azorin JF. The Journal of Mixed Methods Research starts a new decade: The mixed methods research integration trilogy and its dimensions. J Mix Methods Res. 2017;11:291–307.

38. Patient-Centered Outcomes Research Institute. PCORI Methodology Standards. February 2019 [cited 2019 July 9]. Available from: https://www.pcori.org/research-results/about-our-research/research-methodology/pcori-methodology-standards-MixedMethodsResearch.

39. Curry LA, O'Cathain A, Plano Clark VL, Aroni R, Fetters M, Berg D. The role of group dynamics in mixed methods health sciences research teams. J Mix Methods Res. 2012;6:5–20.

40. O'Cathain A, Murphy E, Nicholl J. Multidisciplinary, interdisciplinary, or dysfunctional? Team working in mixed-methods research. Qual Health Res. 2008;18(11):1574–85. doi: 10.1177/1049732308325535 18849518.

41. Bowers B, Cohen LW, Elliot AE, Grabowski DC, Fishman NW, Sharkey SS, et al. Creating and supporting a mixed methods health services research team. Health Serv Res. 2013;48(6 Pt 2):2157–80. doi: 10.1111/1475-6773.12118 24138774; PubMed Central PMCID: PMC3870895.

42. Shaw EK, Ohman-Strickland PA, Piasecki A, Hudson SV, Ferrante JM, McDaniel RR Jr., et al. Effects of facilitated team meetings and learning collaboratives on colorectal cancer screening rates in primary care practices: A cluster randomized trial. Ann Fam Med. 2013;11(3):220–8, S1-8. doi: 10.1370/afm.1505 23690321; PubMed Central PMCID: PMC3659138.

43. Joo JH, Hwang S, Abu H, Gallo JJ. An innovative model of depression care delivery: Peer mentors in collaboration with a mental health professional to relieve depression in older adults. Am J Geriatr Psychiatry. 2016;24(5):407–16. doi: 10.1016/j.jagp.2016.02.002 27066731; PubMed Central PMCID: PMC5116434.

44. Barg FK, Huss-Ashmore R, Wittink MN, Murray GF, Bogner HR, Gallo JJ. A mixed methods approach to understand loneliness and depression in older adults. J Gerontol B Psychol Sci Soc Sci. 2006;61(6):S329–39. doi: 10.1093/geronb/61.6.s329 17114313

45. Gallo JJ, Abshire M, Hwang SY, Nolan MT. Advance directives, medical conditions, and preferences for end-of-life care in a 12-year follow-up study. J Pain Symptom Manage. 2019;57(3):556–65. doi: 10.1016/j.jpainsymman.2018.12.328 30576712

Mixed methods grant applications in the health sciences: An analysis of reviewer comments

Souhrn

Klíčová slova:

Zdroje

PLOS One

Získaná hemofilie - Povědomí o nemoci a její diagnostika

Eozinofilní granulomatóza s polyangiitidou