The Patient-Centered Medical Home (PCMH) is an innovative, rapidly expanding model of healthcare delivery that represents a significant effort at primary care transformation and reform. [1,2] Patient-Centered Medical Homes, endorsed by the primary care professional organizations in 2007, were developed to provide costeffective, comprehensive and coordinated care that is team-based and patient-centered and includes the provision of preventive, behavioral health, and acute and chronic care. [2-5] Goals of thePatient-Centered Medical Home model include improving health outcomes, enhancing access and quality of care, and reducing healthcare costs over time. [6-8].

Although PCMH implementation is underway in a variety of settings across the U.S, [9-11] adopting the PCMH model can be challenging even in motivated and efficient practices. [10] A major challenge is that given the diverse political, organizational, philosophical, cultural, community, financial, patient and practice factors within primary care settings, implementation presents unique challenges and appears to require a variety of approaches. [11-13].

A number of demonstration projects have tested the efficacy and effectiveness of the PCMH model. [9,14] Projects have implemented various interventions that target specific components required to transform practices into Patient-Centered Medical Homes including processes of care, components of practice delivery, practice functioning and culture, and technical components. [9,15,16] Additionally, the PCMH model has been implemented in a variety of other environments and settings including Medicare, numerous state Medicaid agencies, and commercial insurers. [12,14,17] Lessons learned from the demonstration PCMH projects provide the background for currently funded PCMH programs in addressing health care delivery and/or lowering health care costs. [18-21].

Various instruments have been developed to evaluate PCMH transformation in research or demonstration projects [22,23] with some of the instruments measuring the extent to which the practices meet accreditation or recognition requirements. [24] We aimed to go further, and used qualitative as well as quantitative measures to assess how the practices would transform and how stakeholders experienced the transformation process. To do this, we selected, modified or created surveys and qualitative interview questions that addressed the practice as a whole.

In this article, we present the study design of the Brown Primary Care Transformation Initiative, a 5-year Health Resources and Services Administration (HRSA) funded study, which was developed to help facilitate Patient-Centered Medical Home and other practice transformation initiatives in the state of Rhode Island. For this study, we defined the context of practice transformation somewhat more broadly than is typically conceived. Our objectives were to: 1) train medical students, family medicine residents, and faculty/community family physicians in practice transformation; 2) convene a series of three Think Tanks with PCMH experts focused on important theoretical and practical problems associated with PCMH; 3) transform teaching practices in which students, residents, and faculty/ community primary care physicians train and work; 4) develop a methodology for measuring transformation and evaluating curriculum change; and 5) disseminate results (Figure 1).

Figure 1: Overview of the design of the Brown Primary Care Transformation Initiative

The institutional review board at Memorial Hospital of Rhode Island approved the study.

Program Design and Key Components

Training activities included engaging medical students, family medicine (FM) residents, and primary care clinicians in practice transformation which involved: participating in a PCMH module during the third year FM clerkship for medical students; participating in a PCMH Block Rotation during each of three years in residency for FM residents that focused on management of patients from a population health and PCMH perspective; and taking part in faculty meetings, grand rounds, team meetings, FM Department-led community PCMH activities, and Family Care Center (FCC) practicewide PCMH meetings by FM faculty and community physicians. Additionally, immersion activities include medical students’ completion of a third year FM Clerkship continuity experience at a family practice site; involvement of FM residents in the development of the FCC as a PCMH and practicing in other PCMH locations; and active engagement in practice transformation at clinical practices by faculty and community physicians.

Think Tanks: Three Think Tanks were convened by the Brown Family Medicine department between 2011 and 2013. Multidisciplinary local, regional, national and international leaders in PCMH were invited to attend. The purpose of the first Think Tank was to identify domains and data collection tools to be included in the development of a comprehensive and feasibly implementable evaluation methodology. These tools were developed to help identify how transformation within the practice environment occurred for the purpose of improving the health of patients, increase satisfaction in the practice, and contain costs. Based on the results of our PCMH evaluation Think Tank as well as a review of the global PCMH literature and experience with our own Brown Primary Care Transformation Initiative, we developed a mixed-methods measurement set for use at baseline, for process evaluation, and for the post-transformation facilitation intervention. The second Think Tank was convened to identify elements necessary for successful transformation and to determine how the core elements of PCMH transformation should be prioritized. During the second Think Tank, participants were asked to consider and illustrate elements necessary for transformation and development of practices that would truly embody the spirit, structure, services, and outcomes of a successful PCMH. The third Think Tank was convened to explore issues regarding adolescent health in the Patient-Centered Medical Home. This Think Tank group developed a conceptual model that emphasized access to care and information, integrative systems, empowerment, engagement, promotion and prevention, all within the context of the community and policy environment and with a focus on equity. At the culmination of the third Think Tank, recommended next steps included developing a statewide collaboration to promote adolescent access to health care and innovation and bring health care to the adolescents.

Study settings and transformation of teaching practices

PCMH transformation of eight practices has been undertaken and activities included transformation facilitation to promote establishment of PCMH models and to conduct baseline and followup assessments of the practices. The eight practices include the Family Care Center (FCC) and the Internal Medicine Clinic at Memorial Hospital of Rhode Island (MHRI) and six other family medicine primary care teaching sites in Rhode Island. To be eligible, the practice had to be a primary care teaching site, had to have an electronic medical record (or be in the process of obtaining one), be able to identify a physician champion, and be motivated to engage in the process. The eight sites were also involved in the structured curriculum to train medical students, residents and primary care clinicians. Additionally, the Department of Family Medicine has been working collaboratively with other practice transformation initiative programs and stakeholders throughout Rhode Island.

Development of a methodology for evaluation of curriculum change

To evaluate the PCMH block rotation curriculum in the family medicine residency, qualitative in-depth interviews were conducted with core faculty and family medicine residents each lasting approximately 30 minutes. Process evaluation interviews with residents are currently being implemented and outcome interviews with residents and faculty have been planned. Topics include the residents’ and faculty members’ attitudes toward and experience with learning and preparedness in teaching and implementing the core elements of PCMH practice transformation. In addition, information is obtained from the family medicine residents regarding their perception and/or understanding of PCMH and how the rotation and clinical work contribute to their understanding and knowledge of the PCMH model of care.

Development of a methodology for measuring transformation

Activities have been developed to facilitate transformation, to promote the establishment of PCMH models and to assess the process by conducting baseline, process evaluation, and follow-up evaluations of the practices. A mixed-methods approach has been developed for evaluation of the transformation process that includes: qualitative measures (participant observation, patient and staff observation pathways, and in-depth interviews with patients, providers, and staff) and quantitative measures (quantitative surveys for the whole practice as well as for providers, staff and patients). All quantitative and qualitative patient instruments were available in English and Spanish.

Formal practice reports were developed that synthesized results in an understandable, nonjudgmental format. Additional measures included the National Committee for Quality Assurance-NCQA measures, measures from the Patient Centered Medical Home Assessment-PCMH-A) [24,25] and process evaluation information on transformation progression. Although we were interested in collecting recommended utilization data and clinical benchmarks, given that some practices experienced difficulties in providing data, practice information has depended on each practice’s capabilities of providing the needed data. Site visits were also conducted at each of the eight recruited primary care practices to further our understanding of the variety of settings in which PCMH transformation was being introduced.

Qualitative measures

Interviews were conducted with providers and family medicine residents from the FCC that focused on the attitudes and knowledge regarding their understanding of implementing and/or teach the principles of PCMH. Interviews were also conducted with administrators, providers and staff members who were involved in the transformation process. The interviews focused on initial plans for becoming a PCMH, attitudes and knowledge regarding PCMH transformation, and perceived barriers and facilitators to change. All champions and all, or selected other, providers and staff (depending on the size of the practice) were interviewed at each site with each interview lasting approximately 30 minutes. Following the initial meeting, a kick-off group meeting was held with all practice providers and staff followed by data collection. Written reports were presented in person to the practice PCMH champion and team to discuss areas of challenge and improvement towards PCMH transformation and to develop transformation goals and plans specific to each practice. Meetings were held regularly between our facilitator and the practice team and champions. In addition, periodic learning collaborative sessions on specific PCMH transformation topics, open to all sites, were held. During these collaborative sessions, information was provided regarding various PCMH topics followed by staff sharing experiences across sites.

Individual interviews with providers and staff were conducted to elicit information in their own words about how they conceptualized their job roles, office work flow, communication and team work processes, working relationships, patient engagement, and their vision for practice transformation. The interviews were audio-recorded, and interviewers wrote a summary of each interview. Participant observation techniques were conducted to allow the researchers to triangulate interview data with direct observation of how the practice functioned. Three forms of participant observation were used for the baseline assessment: 1) passive observation of the office environment; 2) patient pathways where an evaluator accompanied patients during the medical visit from registration through checkout noting such factors as wait times, chronology and content of the visit, communication, tone and mood of providers, staff and patients; and 3) staff pathways where the evaluator shadowed individual staff members to observe and identify areas of effectiveness and challenge as they conducted their normal work tasks.

As a form of process evaluation, the project facilitation and evaluation staff participated in focus groups twice yearly to discuss their experiences across recruited practice sites. Detailed minutes were kept for the full staff meetings that convened 2-4 times each month to discuss progress at each practice. In addition, staff maintained a log of monthly written reflections about their experiences working with the sites, noting facilitators and barriers they have encountered in their work with the practices, and in the practices’ efforts toward PCMH transformation.

Quantitative measures

We included quantitative measurement tools that we considered to be feasible and practical to implement, and validated when available, and that addressed the practice as a whole. Information was obtained on provider and staff demographics, job satisfaction, levels of burnout, [26] and clinician support for patient activation (CS-PAM). [27] We also collected surveys that assessed patient activation (PAM), [28] communication (Interpersonal Process of Care Survey: Short Form (IPC-18) [29] and patient satisfaction (using the HRSA Patient Satisfaction Survey) [30]. These data were collected at baseline using a stratified sampling approach. Surveys were also collected at one and a half years post-PCMH transformation initiation. During the follow-up assessments, optional and flexible evaluations have been obtained at individual practices. Thus this comprehensive, mixed-methods measurement set that was compiled as a result of the PCMH Evaluation Think Tank, literature review, and measures currently being used, is designed to evaluate the PCMH transformation intervention process in the eight primary care practices with which we are working (Figure 2).

Figure 2: Brown Primary Care Transformation Initiative Timeline of Evaluation

Patient, provider, and staff survey sampling

Patient sample sizes ranged between 30 to 115 surveys at each site with the number of surveys completed at each site based on the size of the site’s patient population. In addition, all members completed specific providers and staff surveys.

Qualitative data analysis

For the evaluation of the provider and family medicine resident interviews, data were analyzed using immersion/crystallization. [31] This involved having the study team members review transcripts independently and then meet periodically as a group to discuss data, interpretation, and application of findings to help improve the PCMH curriculum and progress with transformation of the residency training clinical site.

Analysis of the qualitative data for the eight primary care practices included: 1) listening to the interview recordings; 2) extraction of data relevant to understanding the practice culture and factors that may impact the transformation process; 3) team group discussion of the data; 4) creation of reports for each practice about the description of the practice and the identification of goals, barriers and facilitators to practice change; and 5) compilation of lessons learned regarding significant practice changes in each practice and across practices. Deidentified quotes from the patient, provider and staff interviews were included in the report text to illustrate key points and to support an interpretation of the critical factors for progressing with practice transformation.

Quantitative data analyses

Descriptive analyses being performed included frequency tables for each categorical variable and minimum, maximum, range, median, mean, and standard deviation for each continuous variable to summarize the data as well as detect outliers, data entry errors, and missing values. Descriptive statistics for items from surveys included medians and frequencies and responses were analyzed using the Kolmogorov-Smirnov test. Trends in changes in patient outcomes from baseline to the 2- and 3-year follow-ups are in the process of being examined following completion of data collection. Since patient surveys were not completed by the same patients at baseline and follow-up, repeated measures analyses could not be performed and we used the practice as the unit of analysis. Although not yet completed, for the analysis of outcomes, patient data will be aggregated from all practice sites. In order to evaluate quality improvement efforts, multiple regression models will be performed, adjusting for significant covariates (p=0.05). To identify independent associations between patient characteristics and high patient satisfaction, we will use logistic regression analysis to model highest patient satisfaction quartile (vs. lower) as a function of patient sociodemographic characteristics (including age, race/ethnicity, gender) and practice size. All analyses will be performed using SPSS, version 21.

Resident and provider curriculum evaluation interviews were undertaken at the onset of the study, prior to implementation of the PCMH rotation for residents and enhanced involvement of faculty in PCMH implementation at the residency’s continuity practice, during which twelve graduating family medicine residents and seventeen core family medicine residency faculty participated in interviews. Results suggested that most residents and faculty in our family medicine residency program were at that early stage receptive to the concepts of PCMH, and the majority were excited about the potential of PCMH and had positive associations with PCMH. However, residents’ understanding of what PCMH entails was most often narrowly confined and vaguely expressed, and faculty requested more formal curricular goals and objectives to help guide their teaching of PCMH. During the follow-ups, one class of thirteen residents from one class will be interviewed at the end of each of their 3 training years until graduation. Residents will also be surveyed following the PCMH block rotation.

Baseline characteristics of the eight practices undergoing transformation are presented in Table 1. [Insert Table 1. Baseline Characteristics of the Primary Care Practices Undergoing Transformation]. The eight primary care practices in our PCMH transformation and evaluation case study vary in size, composition of providers, services offered, and community demographics. Practices include a two-physician micro-practice, a one-physician/one medical assistant practice, a multi-physician/multiple staff private practice, two community health

Table 1: Baseline Characteristics of the Primary Care Practices Undergoing Transformation

centers, a university student health primary care practice, a family medicine residency continuity practice, and an internal medicine residency continuity practice. Six of the sites are located in an urban setting; one is suburban, and one is located in a rural setting. The number of providers also varies by practice size. The number of providers varies from 1 in the smallest practice to 50 providers in the largest residency continuity site. Fifty percent of practices have a case manager, 50% have open access scheduling, and 100% have an EMR. The patient activation survey results are similar across practices while the clinician activation measure is greatest for the smallest practices.

Time plan

The development of the program began in August, 2010 and the study will be completed in June, 2015. Rolling provider recruitment began in May, 2011 and was completed in September, 2013 and mixed-method assessments began in June, 2011 and will be completed in February, 2015.

We have described the processes of the Brown Primary Care Transformation Initiative in developing and delivering a multi-faceted didactic curriculum utilizing PCMH principles to train medical students and residents in practice transformation, facilitation of PCMH transformation of eight primary care practices, and identifying and developing a comprehensive, feasibly administrated mixedmethods measurement set based on existing literature and the PCMH Evaluation Think Tanks convened by our group. We are in the process of utilizing the mixed-methods evaluation tools developed in Years 1 and 2 to assess the facilitation of PCMH transformation of the eight primary care practices and to evaluate the training of the residents and faculty in the concepts of primary care practice transformation as well as their knowledge, opinions and preparedness to implement and/or teach the principles of the PCMH.

To date, PCMH evaluation assessments have varied widely with each PCMH measure subject to considerable variability in the definition and operationalization of the components. [32] Thus the evaluation of PCMH progression among primary care practices as well as the relation of PCMH progress with quality outcomes and cost containment remains challenging. [32] For this study, we are focusing on measures that may help us to uncover how and why transformation occurs in the cultural contexts underlying diverse forms of primary care practice.

Several limitations should be mentioned. First, patients who completed the surveys may be patients who have frequent clinic visits. Second, only individuals who spoke English or Spanish completed surveys and interviews, thus presenting a potential bias although English and Spanish are the primary languages spoken at the recruited practices. Finally, providers and practice staff may have been reluctant to be completely forthcoming with their opinions about their work environment. To address this issue, we have taken measures to best ensure privacy during the patient and provider and staff interviews and survey completion, and maintaining confidentiality and anonymity of all data during analysis.

In summary, our study offers a small but diverse group of practices from which we can assess how transformation is taking place within Rhode Island primary care practices utilizing our mixed-methods evaluation measurement set at baseline and the one and a half year follow-up time point, and to explore how useful the mixed quantitative and qualitative evaluation methods are for understanding transformation in practice culture and patient outcomes. Additionally, our approach provides a unique perspective in addressing training of medical students, family medicine residents, and faculty in PCMH transformation. Our experience may thus provide us with valuable insight into implementing PCMH transformation in primary care practice and teaching environments.

The authors would like to acknowledge the assistance of the research assistants for their help with data collection and analysis and help with the patient pathways and observer evaluations. Also, we are grateful for the participation of the primary care practice leadership, providers, staff, and patients in personal interviews and surveys

This project was supported by a grant from the U.S. Department of Health and Human Services, Health Resources and Services Administration, HRSA grant #1D54HP20675-01-00 as part of its program to study transformation of primary care.

1. Rittenhouse DR, Shortell SM (2009) The patient-centered medical home: will it stand the test of health reform? JAMA 301: 2038-2040.
2. Rittenhouse DR, Casalino LP, Shortell SM, McClellan SR, Gillies RR, et al. (2011) Small and medium-size physician practices use few patient-centered medical home processes. Health Aff (Millwood) 30: 1575-1584.
3. American College of Physicians. What is the Patient-Centered Medical Home?
4. Dickens M, Green J, Kohrt A, Pearson H (1992) American Academy of Pediatrics Ad Hoc Task Force on Definition of the Medical Home: The medical home. Pediatrics 90: 774.
5. Lebrun-Harris LA, Shi L, Zhu J, Burke MT, Sripipatana A, et al (2013) Effects of patient-centered medical home attributes on patients’ perceptions of quality in federally supported health centers. Ann Fam Med 11: 508-16.
6. Starfield B, Shi L (2004) The medical home, access to care, and insurance: a review of evidence. Pediatrics 113: 1493-1498.
7. Franks P, Fiscella K (1998) Primary care physicians and specialists as personal physicians. Health care expenditures and mortality experience. J FamPract 47: 105-109.
8. Beal A, Doty M, Hernandez S, Shea KK, Davis K (2007) Closing the divide: How medical homes promote equity in health care-results from the commonwealth fund 2006 health care quality survey. Washington DC: The Commonwealth Fund.
9. Patient-Centered Primary Care Collaborative. Proof in practice: a compilation of patient-centered medical home pilot and demonstration projects.
10. Nutting PA, Miller WL, Crabtree BF, Jaen CR, Stewart EE, et al. (2009) Initial lessons from the first national demonstration project on practice transformation to a patient-centered medical home. Ann Fam Med 7: 254–260.
11. Alexander JA, Bae D (2012) Does the patient-centred medical home work? A critical synthesis of research on patient-centred medical homes and patient-related outcomes. Health Serv Manage Res 25: 51-59.
12. Bitton A, Martin C, Landon BE (2010) A nationwide survey of patient centered medical home demonstration projects. J Gen Intern Med 25: 584-592.
13. Bitton A, Schwartz GR, Stewart EE, Henderson DE, Keohane CA, et al. (2012) Off the hamster wheel? Qualitative evaluation of a payment-linked patient-centered medical home (PCMH) pilot. Milbank Q 90: 484-515.
14. Crabtree BF, Chase SM, Wise CG, Schiff GD, Schmidt LA, et al. (2011) Evaluation of patient centered medical home practice transformation initiatives. Med Care 49: 10-16.
15. Carrier E, Gourevitch MN, Shah NR (2009) Medical homes: challenges in translating theory into practice. Med Care 47: 714-722.
16. Fields D, Leshen E, Patel K (2010) Analysis & commentary. Driving quality gains and cost savings through adoption of medical homes. Health Aff (Millwood) 29: 819-826.
17. Centers for Medicare and Medicaid Services. Multi-payer Advanced Primary Care Practice (MAPCP) Demonstration fact sheet.
18. Jaén CR, Ferrer RL, Miller WL, Palmer RF, Wood R, et al. (2010) Patient outcomes at 26 months in the patient-centered medical home National Demonstration Project. Ann Fam Med 8 Suppl 1: S57-67.
19. Reid RJ, Fishman PA, Yu O, Ross TR, Tufano JT, et al. (2009) Patient-centered medical home demonstration: a prospective, quasi-experimental, before and after evaluation. Am J Manag Care 15: e71-87.
20. Reid RJ, Coleman K, Johnson EA, Fishman PA, Hsu C, et al. (2010) The Group Health medical home at year two: cost savings, higher patient satisfaction, and less burnout for providers. Health Aff (Millwood) 29: 835-843.
21. Paulus RA, Davis K, Steele GD (2008) Continuous innovation in health care: implications of the Geisinger experience. Health Aff (Millwood) 27: 1235-1245.
22. Birnberg JM, Drum ML, Huang ES, Casalino LP, Lewis SE, et al. (2011) Development of a safety net medical home scale for clinics. J Gen Intern Med 26: 1418-1425.
23. Jaen CR, Crabtree RF, Palmer RF, Ferrer RL, Nutting PA et al. (2010) Methods for evaluating practice change toward a Patient-Centered Medical Home. Ann Fam Med 8: S9-20.
24. National Committee for Quality Assurance (2011) Standard for the Patient-Centered Medical Home (PCMH).
25. Safety Net Medical Home Initiative (2013) Patient-Centered Medical Home Assessment.
26. Maslach C, Jackson SE, Leiter MP (2013) Maslach Burnout Inventory.
27. Hibbard JH, Collins PA, Mahoney E, Baker LH (2010) The development and testing of a measure assessing clinician beliefs about patient self-management. Health Expect 13: 65-72.
28. Hibbard JH, Stockard J, Mahoney ER, Tusler M (2004) Development of the Patient Activation Measure (PAM): conceptualizing and measuring activation in patients and consumers. Health Serv Res 39: 1005-1026.
29. Stewart AL, Nápoles-Springer AL, Gregorich SE, Santoyo-Olsson J (2007) Interpersonal processes of care survey: Patient-reported measures for diverse groups. Health Services Research 42: 1235-1256.
30. HRSA Health Resources and Services Administration Primary Care: The Health Center Program (2014) Patient Satisfaction Survey.
31. Borkan J (1999) "Immersion/Crystallization." In BF Crabtree and WL Miller Doing Qualitative Research Thousand Oaks, CA: Sage Publications: 179-194.
32. Alexander JA, Paustian M, Wise CG, Green LA, Fetters MD, et al. (2013) Assessment and measurement of patient-centered medical home implementation: the BCBSM experience. Ann Fam Med 11 Suppl 1: S74-81.