Want to create or adapt books like this? Learn more about how Pressbooks supports open publishing practices.
Multiple Case Studies
Nadia Alqahtani and Pengtong Qu
The case study approach is popular across disciplines in education, anthropology, sociology, psychology, medicine, law, and political science (Creswell, 2013). It is both a research method and a strategy (Creswell, 2013; Yin, 2017). In this type of research design, a case can be an individual, an event, or an entity, as determined by the research questions. There are two variants of the case study: the single-case study and the multiple-case study. The former design can be used to study and understand an unusual case, a critical case, a longitudinal case, or a revelatory case. On the other hand, a multiple-case study includes two or more cases or replications across the cases to investigate the same phenomena (Lewis-Beck, Bryman & Liao, 2003; Yin, 2017). …a multiple-case study includes two or more cases or replications across the cases to investigate the same phenomena
The difference between the single- and multiple-case study is the research design; however, they are within the same methodological framework (Yin, 2017). Multiple cases are selected so that “individual case studies either (a) predict similar results (a literal replication) or (b) predict contrasting results but for anticipatable reasons (a theoretical replication)” (p. 55). When the purpose of the study is to compare and replicate the findings, the multiple-case study produces more compelling evidence so that the study is considered more robust than the single-case study (Yin, 2017).
To write a multiple-case study, a summary of individual cases should be reported, and researchers need to draw cross-case conclusions and form a cross-case report (Yin, 2017). With evidence from multiple cases, researchers may have generalizable findings and develop theories (Lewis-Beck, Bryman & Liao, 2003).
Creswell, J. W. (2013). Qualitative inquiry and research design: Choosing among five approaches (3rd ed.). Los Angeles, CA: Sage.
Lewis-Beck, M., Bryman, A. E., & Liao, T. F. (2003). The Sage encyclopedia of social science research methods . Los Angeles, CA: Sage.
Yin, R. K. (2017). Case study research and applications: Design and methods . Los Angeles, CA: Sage.
Key Research Books and Articles on Multiple Case Study Methodology
Yin discusses how to decide if a case study should be used in research. Novice researchers can learn about research design, data collection, and data analysis of different types of case studies, as well as writing a case study report.
Chapter 2 introduces four major types of research design in case studies: holistic single-case design, embedded single-case design, holistic multiple-case design, and embedded multiple-case design. Novice researchers will learn about the definitions and characteristics of different designs. This chapter also teaches researchers how to examine and discuss the reliability and validity of the designs.
Creswell, J. W., & Poth, C. N. (2017). Qualitative inquiry and research design: Choosing among five approaches . Los Angeles, CA: Sage.
This book compares five different qualitative research designs: narrative research, phenomenology, grounded theory, ethnography, and case study. It compares the characteristics, data collection, data analysis and representation, validity, and writing-up procedures among five inquiry approaches using texts with tables. For each approach, the author introduced the definition, features, types, and procedures and contextualized these components in a study, which was conducted through the same method. Each chapter ends with a list of relevant readings of each inquiry approach.
This book invites readers to compare these five qualitative methods and see the value of each approach. Readers can consider which approach would serve for their research contexts and questions, as well as how to design their research and conduct the data analysis based on their choice of research method.
Günes, E., & Bahçivan, E. (2016). A multiple case study of preservice science teachers’ TPACK: Embedded in a comprehensive belief system. International Journal of Environmental and Science Education, 11 (15), 8040-8054.
In this article, the researchers showed the importance of using technological opportunities in improving the education process and how they enhanced the students’ learning in science education. The study examined the connection between “Technological Pedagogical Content Knowledge” (TPACK) and belief system in a science teaching context. The researchers used the multiple-case study to explore the effect of TPACK on the preservice science teachers’ (PST) beliefs on their TPACK level. The participants were three teachers with the low, medium, and high level of TPACK confidence. Content analysis was utilized to analyze the data, which were collected by individual semi-structured interviews with the participants about their lesson plans. The study first discussed each case, then compared features and relations across cases. The researchers found that there was a positive relationship between PST’s TPACK confidence and TPACK level; when PST had higher TPACK confidence, the participant had a higher competent TPACK level and vice versa.
Recent Dissertations Using Multiple Case Study Methodology
Milholland, E. S. (2015). A multiple case study of instructors utilizing Classroom Response Systems (CRS) to achieve pedagogical goals . Retrieved from ProQuest Dissertations & Theses Global. (Order Number 3706380)
The researcher of this study critiques the use of Classroom Responses Systems by five instructors who employed this program five years ago in their classrooms. The researcher conducted the multiple-case study methodology and categorized themes. He interviewed each instructor with questions about their initial pedagogical goals, the changes in pedagogy during teaching, and the teaching techniques individuals used while practicing the CRS. The researcher used the multiple-case study with five instructors. He found that all instructors changed their goals during employing CRS; they decided to reduce the time of lecturing and to spend more time engaging students in interactive activities. This study also demonstrated that CRS was useful for the instructors to achieve multiple learning goals; all the instructors provided examples of the positive aspect of implementing CRS in their classrooms.
Li, C. L. (2010). The emergence of fairy tale literacy: A multiple case study on promoting critical literacy of children through a juxtaposed reading of classic fairy tales and their contemporary disruptive variants . Retrieved from ProQuest Dissertations & Theses Global. (Order Number 3572104)
To explore how children’s development of critical literacy can be impacted by their reactions to fairy tales, the author conducted a multiple-case study with 4 cases, in which each child was a unit of analysis. Two Chinese immigrant children (a boy and a girl) and two American children (a boy and a girl) at the second or third grade were recruited in the study. The data were collected through interviews, discussions on fairy tales, and drawing pictures. The analysis was conducted within both individual cases and cross cases. Across four cases, the researcher found that the young children’s’ knowledge of traditional fairy tales was built upon mass-media based adaptations. The children believed that the representations on mass-media were the original stories, even though fairy tales are included in the elementary school curriculum. The author also found that introducing classic versions of fairy tales increased children’s knowledge in the genre’s origin, which would benefit their understanding of the genre. She argued that introducing fairy tales can be the first step to promote children’s development of critical literacy.
Asher, K. C. (2014). Mediating occupational socialization and occupational individuation in teacher education: A multiple case study of five elementary pre-service student teachers . Retrieved from ProQuest Dissertations & Theses Global. (Order Number 3671989)
This study portrayed five pre-service teachers’ teaching experience in their student teaching phase and explored how pre-service teachers mediate their occupational socialization with occupational individuation. The study used the multiple-case study design and recruited five pre-service teachers from a Midwestern university as five cases. Qualitative data were collected through interviews, classroom observations, and field notes. The author implemented the case study analysis and found five strategies that the participants used to mediate occupational socialization with occupational individuation. These strategies were: 1) hindering from practicing their beliefs, 2) mimicking the styles of supervising teachers, 3) teaching in the ways in alignment with school’s existing practice, 4) enacting their own ideas, and 5) integrating and balancing occupational socialization and occupational individuation. The study also provided recommendations and implications to policymakers and educators in teacher education so that pre-service teachers can be better supported.
Multiple Case Studies Copyright © 2019 by Nadia Alqahtani and Pengtong Qu is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License , except where otherwise noted.
Share This Book
Not logged in
Multiple case study, page actions.
- View source
A multiple case study is a research method that uses multiple research sites to gain a more comprehensive understanding of a particular phenomenon. It examines multiple cases in order to analyze patterns and the relationships between variables. This type of research method is used in management to gain a deeper understanding of a particular problem or issue. It is a systematic approach to gathering and analyzing data from multiple different sources such as individuals, organizations, or communities. The multiple case study approach allows researchers to gain greater insight into complex problems by considering a variety of perspectives, contexts, and sources of information .
- 1 Example of multiple case study
- 2 When to use multiple case study
- 3 Types of multiple case study
- 4 Steps of multiple case study
- 5 Limitations of multiple case study
- 6 Other approaches related to multiple case study
- 7 References
Example of multiple case study
- A multiple case study example could be a study of different companies in the same industry in order to analyze the differences in their strategies and performance. For instance, a researcher may examine three companies in the automotive industry and determine what strategies have been successful and which have not. They may then compare the results of these three companies in order to determine which strategies are most effective.
- Another example of a multiple case study could be an examination of how different countries have responded to the COVID-19 pandemic. In this case, the researcher could look at different strategies adopted by countries worldwide and analyze the results of those strategies. They could then compare the results in order to determine which strategies have been most successful in mitigating the spread of the virus.
- A third example could be a study of different schools and how they have adapted to the online learning environment . The researcher could look at the successes and failures of different schools in order to determine which strategies are most effective in transitioning to remote learning. They could then use these findings to suggest changes and improvements to the schools’ policies and procedures .
When to use multiple case study
A multiple case study approach is a useful tool for researchers looking to gain a deeper understanding of complex issues. This method can be used in a variety of contexts, such as studying organizational management , social phenomena, or public health interventions. It can provide a more comprehensive understanding of the problem by considering a variety of perspectives, contexts, and sources of information. Examples of when multiple case studies can be used include:
- Examining the effectiveness of a particular policy or program in multiple contexts.
- Exploring the dynamics of organizational change across different settings.
- Investigating the impact of a cultural or social phenomenon on different communities.
- Analyzing the differences in responses to a public health intervention between populations.
- Understanding the dynamics of an issue in order to inform the development of new policies or practices.
Types of multiple case study
- Exploratory multiple case study: An exploratory multiple case study is used to explore a research problem in greater detail. It is used when the research question or problem is not well-defined, or when the researcher is uncertain about the best approach to study the problem. This type of multiple case study is often used to generate new ideas and to identify potential research topics.
- Explanatory Multiple Case Study: An explanatory multiple case study is used to explain a research problem in detail. It is used when the researcher is looking to explain the cause of an event or phenomenon. This type of multiple case study is used to identify patterns and relationships between variables, and to identify potential explanations for the phenomenon being studied.
- Descriptive Multiple Case Study: A descriptive multiple case study is used to describe a research problem in detail. It is used when the researcher wants to provide a comprehensive overview of a particular topic or phenomenon. This type of multiple case study is useful for providing a detailed description of a particular event or phenomenon and its context.
- Comparative Multiple Case Study: A comparative multiple case study is used to compare two or more research sites. It is used when the researcher wants to compare and contrast a phenomenon across multiple sites. This type of multiple case study is useful for examining similarities and differences between different research sites.
- Embedded Multiple Case Study: An embedded multiple case study is used to embed a single case study within a larger research project . It is used when the researcher wants to incorporate a single case study within a larger research project. This type of multiple case study is useful for exploring the complexities of a particular research problem, and for providing an in-depth understanding of a particular phenomenon.
Steps of multiple case study
A multiple case study is a research method that uses multiple research sites to gain a more comprehensive understanding of a particular phenomenon. The multiple case study approach allows researchers to gain greater insight into complex problems by considering a variety of perspectives, contexts, and sources of information. The following steps are necessary for conducting a successful multiple case study:
- Selecting the research sites : The first step in a multiple case study is to select the research sites. This requires careful consideration of factors such as the size and scope of the problem, the availability of data and resources, and the accessibility of the research sites.
- Gathering data : After selecting the research sites, the next step is to gather data. This can be done through interviews, surveys, focus groups, and other data collection methods .
- Analyzing the data : Once the data has been gathered, it must be analyzed in order to identify patterns and relationships between variables. This requires careful analysis of the data and may involve using statistical methods such as regression and factor analysis.
- Drawing conclusions : After the data has been analyzed, the next step is to draw conclusions. This involves synthesizing the data and making sense of it in order to answer the research question.
- Reporting the results : The final step is to report the results of the multiple case study. This can be done through a written report, a presentation, or a multimedia format.
Limitations of multiple case study
Multiple case studies have some limitations that should be taken into consideration when using this method. These limitations include:
- The multiple case study approach can be time consuming and resource intensive, as researchers must collect and analyze data from multiple different sources.
- It can be difficult to identify patterns and relationships between variables when studying multiple cases.
- The data collected from multiple cases may be difficult to generalize to a larger population.
- The multiple case study approach is limited to studying phenomena in limited contexts, and does not provide a holistic picture of a phenomenon.
- It can be difficult to control for all variables in a multiple case study, which can lead to inaccurate results.
Other approaches related to multiple case study
A multiple case study is a research method that uses multiple research sites to gain a more comprehensive understanding of a particular phenomenon. Other approaches related to multiple case studies include:
- Qualitative research : Qualitative research is an empirical research approach which focuses on understanding the perspectives, experiences, and beliefs of people in their contexts. It typically involves interviews, observations, and other forms of data collection.
- Grounded Theory : Grounded theory is an inductive research method that examines how social processes are created, maintained, and changed. It involves the systematic collection and analysis of data to generate new theory.
- Action Research : Action research is a type of research that involves the active participation of stakeholders in the research process . It focuses on identifying and resolving practical problems in an organization or community.
- Gustafsson, J. (2017). Single case studies vs. multiple case studies: A comparative study .
- Methods and techniques
- Recent changes
- Random page
- Page information
Table of Contents
- Special pages
User page tools
- What links here
- Related changes
- Printable version
- Permanent link
- This page was last edited on 18 November 2023, at 01:04.
- Content is available under CC BY-SA Attribution-ShareAlike 4.0 International unless otherwise noted.
- About CEOpedia | Management online
- Open access
- Published: 27 June 2011
The case study approach
- Sarah Crowe 1 ,
- Kathrin Cresswell 2 ,
- Ann Robertson 2 ,
- Guro Huby 3 ,
- Anthony Avery 1 &
- Aziz Sheikh 2
BMC Medical Research Methodology volume 11 , Article number: 100 ( 2011 ) Cite this article
The case study approach allows in-depth, multi-faceted explorations of complex issues in their real-life settings. The value of the case study approach is well recognised in the fields of business, law and policy, but somewhat less so in health services research. Based on our experiences of conducting several health-related case studies, we reflect on the different types of case study design, the specific research questions this approach can help answer, the data sources that tend to be used, and the particular advantages and disadvantages of employing this methodological approach. The paper concludes with key pointers to aid those designing and appraising proposals for conducting case study research, and a checklist to help readers assess the quality of case study reports.
Peer Review reports
The case study approach is particularly useful to employ when there is a need to obtain an in-depth appreciation of an issue, event or phenomenon of interest, in its natural real-life context. Our aim in writing this piece is to provide insights into when to consider employing this approach and an overview of key methodological considerations in relation to the design, planning, analysis, interpretation and reporting of case studies.
The illustrative 'grand round', 'case report' and 'case series' have a long tradition in clinical practice and research. Presenting detailed critiques, typically of one or more patients, aims to provide insights into aspects of the clinical case and, in doing so, illustrate broader lessons that may be learnt. In research, the conceptually-related case study approach can be used, for example, to describe in detail a patient's episode of care, explore professional attitudes to and experiences of a new policy initiative or service development or more generally to 'investigate contemporary phenomena within its real-life context' [ 1 ]. Based on our experiences of conducting a range of case studies, we reflect on when to consider using this approach, discuss the key steps involved and illustrate, with examples, some of the practical challenges of attaining an in-depth understanding of a 'case' as an integrated whole. In keeping with previously published work, we acknowledge the importance of theory to underpin the design, selection, conduct and interpretation of case studies[ 2 ]. In so doing, we make passing reference to the different epistemological approaches used in case study research by key theoreticians and methodologists in this field of enquiry.
This paper is structured around the following main questions: What is a case study? What are case studies used for? How are case studies conducted? What are the potential pitfalls and how can these be avoided? We draw in particular on four of our own recently published examples of case studies (see Tables 1 , 2 , 3 and 4 ) and those of others to illustrate our discussion[ 3 – 7 ].
What is a case study?
A case study is a research approach that is used to generate an in-depth, multi-faceted understanding of a complex issue in its real-life context. It is an established research design that is used extensively in a wide variety of disciplines, particularly in the social sciences. A case study can be defined in a variety of ways (Table 5 ), the central tenet being the need to explore an event or phenomenon in depth and in its natural context. It is for this reason sometimes referred to as a "naturalistic" design; this is in contrast to an "experimental" design (such as a randomised controlled trial) in which the investigator seeks to exert control over and manipulate the variable(s) of interest.
Stake's work has been particularly influential in defining the case study approach to scientific enquiry. He has helpfully characterised three main types of case study: intrinsic , instrumental and collective [ 8 ]. An intrinsic case study is typically undertaken to learn about a unique phenomenon. The researcher should define the uniqueness of the phenomenon, which distinguishes it from all others. In contrast, the instrumental case study uses a particular case (some of which may be better than others) to gain a broader appreciation of an issue or phenomenon. The collective case study involves studying multiple cases simultaneously or sequentially in an attempt to generate a still broader appreciation of a particular issue.
These are however not necessarily mutually exclusive categories. In the first of our examples (Table 1 ), we undertook an intrinsic case study to investigate the issue of recruitment of minority ethnic people into the specific context of asthma research studies, but it developed into a instrumental case study through seeking to understand the issue of recruitment of these marginalised populations more generally, generating a number of the findings that are potentially transferable to other disease contexts[ 3 ]. In contrast, the other three examples (see Tables 2 , 3 and 4 ) employed collective case study designs to study the introduction of workforce reconfiguration in primary care, the implementation of electronic health records into hospitals, and to understand the ways in which healthcare students learn about patient safety considerations[ 4 – 6 ]. Although our study focusing on the introduction of General Practitioners with Specialist Interests (Table 2 ) was explicitly collective in design (four contrasting primary care organisations were studied), is was also instrumental in that this particular professional group was studied as an exemplar of the more general phenomenon of workforce redesign[ 4 ].
What are case studies used for?
According to Yin, case studies can be used to explain, describe or explore events or phenomena in the everyday contexts in which they occur[ 1 ]. These can, for example, help to understand and explain causal links and pathways resulting from a new policy initiative or service development (see Tables 2 and 3 , for example)[ 1 ]. In contrast to experimental designs, which seek to test a specific hypothesis through deliberately manipulating the environment (like, for example, in a randomised controlled trial giving a new drug to randomly selected individuals and then comparing outcomes with controls),[ 9 ] the case study approach lends itself well to capturing information on more explanatory ' how ', 'what' and ' why ' questions, such as ' how is the intervention being implemented and received on the ground?'. The case study approach can offer additional insights into what gaps exist in its delivery or why one implementation strategy might be chosen over another. This in turn can help develop or refine theory, as shown in our study of the teaching of patient safety in undergraduate curricula (Table 4 )[ 6 , 10 ]. Key questions to consider when selecting the most appropriate study design are whether it is desirable or indeed possible to undertake a formal experimental investigation in which individuals and/or organisations are allocated to an intervention or control arm? Or whether the wish is to obtain a more naturalistic understanding of an issue? The former is ideally studied using a controlled experimental design, whereas the latter is more appropriately studied using a case study design.
Case studies may be approached in different ways depending on the epistemological standpoint of the researcher, that is, whether they take a critical (questioning one's own and others' assumptions), interpretivist (trying to understand individual and shared social meanings) or positivist approach (orientating towards the criteria of natural sciences, such as focusing on generalisability considerations) (Table 6 ). Whilst such a schema can be conceptually helpful, it may be appropriate to draw on more than one approach in any case study, particularly in the context of conducting health services research. Doolin has, for example, noted that in the context of undertaking interpretative case studies, researchers can usefully draw on a critical, reflective perspective which seeks to take into account the wider social and political environment that has shaped the case[ 11 ].
How are case studies conducted?
Here, we focus on the main stages of research activity when planning and undertaking a case study; the crucial stages are: defining the case; selecting the case(s); collecting and analysing the data; interpreting data; and reporting the findings.
Defining the case
Carefully formulated research question(s), informed by the existing literature and a prior appreciation of the theoretical issues and setting(s), are all important in appropriately and succinctly defining the case[ 8 , 12 ]. Crucially, each case should have a pre-defined boundary which clarifies the nature and time period covered by the case study (i.e. its scope, beginning and end), the relevant social group, organisation or geographical area of interest to the investigator, the types of evidence to be collected, and the priorities for data collection and analysis (see Table 7 )[ 1 ]. A theory driven approach to defining the case may help generate knowledge that is potentially transferable to a range of clinical contexts and behaviours; using theory is also likely to result in a more informed appreciation of, for example, how and why interventions have succeeded or failed[ 13 ].
For example, in our evaluation of the introduction of electronic health records in English hospitals (Table 3 ), we defined our cases as the NHS Trusts that were receiving the new technology[ 5 ]. Our focus was on how the technology was being implemented. However, if the primary research interest had been on the social and organisational dimensions of implementation, we might have defined our case differently as a grouping of healthcare professionals (e.g. doctors and/or nurses). The precise beginning and end of the case may however prove difficult to define. Pursuing this same example, when does the process of implementation and adoption of an electronic health record system really begin or end? Such judgements will inevitably be influenced by a range of factors, including the research question, theory of interest, the scope and richness of the gathered data and the resources available to the research team.
Selecting the case(s)
The decision on how to select the case(s) to study is a very important one that merits some reflection. In an intrinsic case study, the case is selected on its own merits[ 8 ]. The case is selected not because it is representative of other cases, but because of its uniqueness, which is of genuine interest to the researchers. This was, for example, the case in our study of the recruitment of minority ethnic participants into asthma research (Table 1 ) as our earlier work had demonstrated the marginalisation of minority ethnic people with asthma, despite evidence of disproportionate asthma morbidity[ 14 , 15 ]. In another example of an intrinsic case study, Hellstrom et al.[ 16 ] studied an elderly married couple living with dementia to explore how dementia had impacted on their understanding of home, their everyday life and their relationships.
For an instrumental case study, selecting a "typical" case can work well[ 8 ]. In contrast to the intrinsic case study, the particular case which is chosen is of less importance than selecting a case that allows the researcher to investigate an issue or phenomenon. For example, in order to gain an understanding of doctors' responses to health policy initiatives, Som undertook an instrumental case study interviewing clinicians who had a range of responsibilities for clinical governance in one NHS acute hospital trust[ 17 ]. Sampling a "deviant" or "atypical" case may however prove even more informative, potentially enabling the researcher to identify causal processes, generate hypotheses and develop theory.
In collective or multiple case studies, a number of cases are carefully selected. This offers the advantage of allowing comparisons to be made across several cases and/or replication. Choosing a "typical" case may enable the findings to be generalised to theory (i.e. analytical generalisation) or to test theory by replicating the findings in a second or even a third case (i.e. replication logic)[ 1 ]. Yin suggests two or three literal replications (i.e. predicting similar results) if the theory is straightforward and five or more if the theory is more subtle. However, critics might argue that selecting 'cases' in this way is insufficiently reflexive and ill-suited to the complexities of contemporary healthcare organisations.
The selected case study site(s) should allow the research team access to the group of individuals, the organisation, the processes or whatever else constitutes the chosen unit of analysis for the study. Access is therefore a central consideration; the researcher needs to come to know the case study site(s) well and to work cooperatively with them. Selected cases need to be not only interesting but also hospitable to the inquiry [ 8 ] if they are to be informative and answer the research question(s). Case study sites may also be pre-selected for the researcher, with decisions being influenced by key stakeholders. For example, our selection of case study sites in the evaluation of the implementation and adoption of electronic health record systems (see Table 3 ) was heavily influenced by NHS Connecting for Health, the government agency that was responsible for overseeing the National Programme for Information Technology (NPfIT)[ 5 ]. This prominent stakeholder had already selected the NHS sites (through a competitive bidding process) to be early adopters of the electronic health record systems and had negotiated contracts that detailed the deployment timelines.
It is also important to consider in advance the likely burden and risks associated with participation for those who (or the site(s) which) comprise the case study. Of particular importance is the obligation for the researcher to think through the ethical implications of the study (e.g. the risk of inadvertently breaching anonymity or confidentiality) and to ensure that potential participants/participating sites are provided with sufficient information to make an informed choice about joining the study. The outcome of providing this information might be that the emotive burden associated with participation, or the organisational disruption associated with supporting the fieldwork, is considered so high that the individuals or sites decide against participation.
In our example of evaluating implementations of electronic health record systems, given the restricted number of early adopter sites available to us, we sought purposively to select a diverse range of implementation cases among those that were available[ 5 ]. We chose a mixture of teaching, non-teaching and Foundation Trust hospitals, and examples of each of the three electronic health record systems procured centrally by the NPfIT. At one recruited site, it quickly became apparent that access was problematic because of competing demands on that organisation. Recognising the importance of full access and co-operative working for generating rich data, the research team decided not to pursue work at that site and instead to focus on other recruited sites.
Collecting the data
In order to develop a thorough understanding of the case, the case study approach usually involves the collection of multiple sources of evidence, using a range of quantitative (e.g. questionnaires, audits and analysis of routinely collected healthcare data) and more commonly qualitative techniques (e.g. interviews, focus groups and observations). The use of multiple sources of data (data triangulation) has been advocated as a way of increasing the internal validity of a study (i.e. the extent to which the method is appropriate to answer the research question)[ 8 , 18 – 21 ]. An underlying assumption is that data collected in different ways should lead to similar conclusions, and approaching the same issue from different angles can help develop a holistic picture of the phenomenon (Table 2 )[ 4 ].
Brazier and colleagues used a mixed-methods case study approach to investigate the impact of a cancer care programme[ 22 ]. Here, quantitative measures were collected with questionnaires before, and five months after, the start of the intervention which did not yield any statistically significant results. Qualitative interviews with patients however helped provide an insight into potentially beneficial process-related aspects of the programme, such as greater, perceived patient involvement in care. The authors reported how this case study approach provided a number of contextual factors likely to influence the effectiveness of the intervention and which were not likely to have been obtained from quantitative methods alone.
In collective or multiple case studies, data collection needs to be flexible enough to allow a detailed description of each individual case to be developed (e.g. the nature of different cancer care programmes), before considering the emerging similarities and differences in cross-case comparisons (e.g. to explore why one programme is more effective than another). It is important that data sources from different cases are, where possible, broadly comparable for this purpose even though they may vary in nature and depth.
Analysing, interpreting and reporting case studies
Making sense and offering a coherent interpretation of the typically disparate sources of data (whether qualitative alone or together with quantitative) is far from straightforward. Repeated reviewing and sorting of the voluminous and detail-rich data are integral to the process of analysis. In collective case studies, it is helpful to analyse data relating to the individual component cases first, before making comparisons across cases. Attention needs to be paid to variations within each case and, where relevant, the relationship between different causes, effects and outcomes[ 23 ]. Data will need to be organised and coded to allow the key issues, both derived from the literature and emerging from the dataset, to be easily retrieved at a later stage. An initial coding frame can help capture these issues and can be applied systematically to the whole dataset with the aid of a qualitative data analysis software package.
The Framework approach is a practical approach, comprising of five stages (familiarisation; identifying a thematic framework; indexing; charting; mapping and interpretation) , to managing and analysing large datasets particularly if time is limited, as was the case in our study of recruitment of South Asians into asthma research (Table 1 )[ 3 , 24 ]. Theoretical frameworks may also play an important role in integrating different sources of data and examining emerging themes. For example, we drew on a socio-technical framework to help explain the connections between different elements - technology; people; and the organisational settings within which they worked - in our study of the introduction of electronic health record systems (Table 3 )[ 5 ]. Our study of patient safety in undergraduate curricula drew on an evaluation-based approach to design and analysis, which emphasised the importance of the academic, organisational and practice contexts through which students learn (Table 4 )[ 6 ].
Case study findings can have implications both for theory development and theory testing. They may establish, strengthen or weaken historical explanations of a case and, in certain circumstances, allow theoretical (as opposed to statistical) generalisation beyond the particular cases studied[ 12 ]. These theoretical lenses should not, however, constitute a strait-jacket and the cases should not be "forced to fit" the particular theoretical framework that is being employed.
When reporting findings, it is important to provide the reader with enough contextual information to understand the processes that were followed and how the conclusions were reached. In a collective case study, researchers may choose to present the findings from individual cases separately before amalgamating across cases. Care must be taken to ensure the anonymity of both case sites and individual participants (if agreed in advance) by allocating appropriate codes or withholding descriptors. In the example given in Table 3 , we decided against providing detailed information on the NHS sites and individual participants in order to avoid the risk of inadvertent disclosure of identities[ 5 , 25 ].
What are the potential pitfalls and how can these be avoided?
The case study approach is, as with all research, not without its limitations. When investigating the formal and informal ways undergraduate students learn about patient safety (Table 4 ), for example, we rapidly accumulated a large quantity of data. The volume of data, together with the time restrictions in place, impacted on the depth of analysis that was possible within the available resources. This highlights a more general point of the importance of avoiding the temptation to collect as much data as possible; adequate time also needs to be set aside for data analysis and interpretation of what are often highly complex datasets.
Case study research has sometimes been criticised for lacking scientific rigour and providing little basis for generalisation (i.e. producing findings that may be transferable to other settings)[ 1 ]. There are several ways to address these concerns, including: the use of theoretical sampling (i.e. drawing on a particular conceptual framework); respondent validation (i.e. participants checking emerging findings and the researcher's interpretation, and providing an opinion as to whether they feel these are accurate); and transparency throughout the research process (see Table 8 )[ 8 , 18 – 21 , 23 , 26 ]. Transparency can be achieved by describing in detail the steps involved in case selection, data collection, the reasons for the particular methods chosen, and the researcher's background and level of involvement (i.e. being explicit about how the researcher has influenced data collection and interpretation). Seeking potential, alternative explanations, and being explicit about how interpretations and conclusions were reached, help readers to judge the trustworthiness of the case study report. Stake provides a critique checklist for a case study report (Table 9 )[ 8 ].
The case study approach allows, amongst other things, critical events, interventions, policy developments and programme-based service reforms to be studied in detail in a real-life context. It should therefore be considered when an experimental design is either inappropriate to answer the research questions posed or impossible to undertake. Considering the frequency with which implementations of innovations are now taking place in healthcare settings and how well the case study approach lends itself to in-depth, complex health service research, we believe this approach should be more widely considered by researchers. Though inherently challenging, the research case study can, if carefully conceptualised and thoughtfully undertaken and reported, yield powerful insights into many important aspects of health and healthcare delivery.
Yin RK: Case study research, design and method. 2009, London: Sage Publications Ltd., 4
Keen J, Packwood T: Qualitative research; case study evaluation. BMJ. 1995, 311: 444-446.
Article CAS PubMed PubMed Central Google Scholar
Sheikh A, Halani L, Bhopal R, Netuveli G, Partridge M, Car J, et al: Facilitating the Recruitment of Minority Ethnic People into Research: Qualitative Case Study of South Asians and Asthma. PLoS Med. 2009, 6 (10): 1-11.
Article Google Scholar
Pinnock H, Huby G, Powell A, Kielmann T, Price D, Williams S, et al: The process of planning, development and implementation of a General Practitioner with a Special Interest service in Primary Care Organisations in England and Wales: a comparative prospective case study. Report for the National Co-ordinating Centre for NHS Service Delivery and Organisation R&D (NCCSDO). 2008, [ http://www.sdo.nihr.ac.uk/files/project/99-final-report.pdf ]
Robertson A, Cresswell K, Takian A, Petrakaki D, Crowe S, Cornford T, et al: Prospective evaluation of the implementation and adoption of NHS Connecting for Health's national electronic health record in secondary care in England: interim findings. BMJ. 2010, 41: c4564-
Pearson P, Steven A, Howe A, Sheikh A, Ashcroft D, Smith P, the Patient Safety Education Study Group: Learning about patient safety: organisational context and culture in the education of healthcare professionals. J Health Serv Res Policy. 2010, 15: 4-10. 10.1258/jhsrp.2009.009052.
Article PubMed Google Scholar
van Harten WH, Casparie TF, Fisscher OA: The evaluation of the introduction of a quality management system: a process-oriented case study in a large rehabilitation hospital. Health Policy. 2002, 60 (1): 17-37. 10.1016/S0168-8510(01)00187-7.
Stake RE: The art of case study research. 1995, London: Sage Publications Ltd.
Sheikh A, Smeeth L, Ashcroft R: Randomised controlled trials in primary care: scope and application. Br J Gen Pract. 2002, 52 (482): 746-51.
PubMed PubMed Central Google Scholar
King G, Keohane R, Verba S: Designing Social Inquiry. 1996, Princeton: Princeton University Press
Doolin B: Information technology as disciplinary technology: being critical in interpretative research on information systems. Journal of Information Technology. 1998, 13: 301-311. 10.1057/jit.1998.8.
George AL, Bennett A: Case studies and theory development in the social sciences. 2005, Cambridge, MA: MIT Press
Eccles M, the Improved Clinical Effectiveness through Behavioural Research Group (ICEBeRG): Designing theoretically-informed implementation interventions. Implementation Science. 2006, 1: 1-8. 10.1186/1748-5908-1-1.
Article PubMed Central Google Scholar
Netuveli G, Hurwitz B, Levy M, Fletcher M, Barnes G, Durham SR, Sheikh A: Ethnic variations in UK asthma frequency, morbidity, and health-service use: a systematic review and meta-analysis. Lancet. 2005, 365 (9456): 312-7.
Sheikh A, Panesar SS, Lasserson T, Netuveli G: Recruitment of ethnic minorities to asthma studies. Thorax. 2004, 59 (7): 634-
CAS PubMed PubMed Central Google Scholar
Hellström I, Nolan M, Lundh U: 'We do things together': A case study of 'couplehood' in dementia. Dementia. 2005, 4: 7-22. 10.1177/1471301205049188.
Som CV: Nothing seems to have changed, nothing seems to be changing and perhaps nothing will change in the NHS: doctors' response to clinical governance. International Journal of Public Sector Management. 2005, 18: 463-477. 10.1108/09513550510608903.
Lincoln Y, Guba E: Naturalistic inquiry. 1985, Newbury Park: Sage Publications
Barbour RS: Checklists for improving rigour in qualitative research: a case of the tail wagging the dog?. BMJ. 2001, 322: 1115-1117. 10.1136/bmj.322.7294.1115.
Mays N, Pope C: Qualitative research in health care: Assessing quality in qualitative research. BMJ. 2000, 320: 50-52. 10.1136/bmj.320.7226.50.
Mason J: Qualitative researching. 2002, London: Sage
Brazier A, Cooke K, Moravan V: Using Mixed Methods for Evaluating an Integrative Approach to Cancer Care: A Case Study. Integr Cancer Ther. 2008, 7: 5-17. 10.1177/1534735407313395.
Miles MB, Huberman M: Qualitative data analysis: an expanded sourcebook. 1994, CA: Sage Publications Inc., 2
Pope C, Ziebland S, Mays N: Analysing qualitative data. Qualitative research in health care. BMJ. 2000, 320: 114-116. 10.1136/bmj.320.7227.114.
Cresswell KM, Worth A, Sheikh A: Actor-Network Theory and its role in understanding the implementation of information technology developments in healthcare. BMC Med Inform Decis Mak. 2010, 10 (1): 67-10.1186/1472-6947-10-67.
Article PubMed PubMed Central Google Scholar
Malterud K: Qualitative research: standards, challenges, and guidelines. Lancet. 2001, 358: 483-488. 10.1016/S0140-6736(01)05627-6.
Article CAS PubMed Google Scholar
Yin R: Case study research: design and methods. 1994, Thousand Oaks, CA: Sage Publishing, 2
Yin R: Enhancing the quality of case studies in health services research. Health Serv Res. 1999, 34: 1209-1224.
Green J, Thorogood N: Qualitative methods for health research. 2009, Los Angeles: Sage, 2
Howcroft D, Trauth E: Handbook of Critical Information Systems Research, Theory and Application. 2005, Cheltenham, UK: Northampton, MA, USA: Edward Elgar
Book Google Scholar
Blakie N: Approaches to Social Enquiry. 1993, Cambridge: Polity Press
Doolin B: Power and resistance in the implementation of a medical management information system. Info Systems J. 2004, 14: 343-362. 10.1111/j.1365-2575.2004.00176.x.
Bloomfield BP, Best A: Management consultants: systems development, power and the translation of problems. Sociological Review. 1992, 40: 533-560.
Shanks G, Parr A: Positivist, single case study research in information systems: A critical analysis. Proceedings of the European Conference on Information Systems. 2003, Naples
The pre-publication history for this paper can be accessed here: http://www.biomedcentral.com/1471-2288/11/100/prepub
We are grateful to the participants and colleagues who contributed to the individual case studies that we have drawn on. This work received no direct funding, but it has been informed by projects funded by Asthma UK, the NHS Service Delivery Organisation, NHS Connecting for Health Evaluation Programme, and Patient Safety Research Portfolio. We would also like to thank the expert reviewers for their insightful and constructive feedback. Our thanks are also due to Dr. Allison Worth who commented on an earlier draft of this manuscript.
Authors and affiliations.
Division of Primary Care, The University of Nottingham, Nottingham, UK
Sarah Crowe & Anthony Avery
Centre for Population Health Sciences, The University of Edinburgh, Edinburgh, UK
Kathrin Cresswell, Ann Robertson & Aziz Sheikh
School of Health in Social Science, The University of Edinburgh, Edinburgh, UK
You can also search for this author in PubMed Google Scholar
Correspondence to Sarah Crowe .
The authors declare that they have no competing interests.
AS conceived this article. SC, KC and AR wrote this paper with GH, AA and AS all commenting on various drafts. SC and AS are guarantors.
Rights and permissions
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License ( http://creativecommons.org/licenses/by/2.0 ), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
Reprints and Permissions
About this article
Cite this article.
Crowe, S., Cresswell, K., Robertson, A. et al. The case study approach. BMC Med Res Methodol 11 , 100 (2011). https://doi.org/10.1186/1471-2288-11-100
Received : 29 November 2010
Accepted : 27 June 2011
Published : 27 June 2011
DOI : https://doi.org/10.1186/1471-2288-11-100
Share this article
Anyone you share the following link with will be able to read this content:
Sorry, a shareable link is not currently available for this article.
Provided by the Springer Nature SharedIt content-sharing initiative
- Case Study Approach
- Electronic Health Record System
- Case Study Design
- Case Study Site
- Case Study Report
BMC Medical Research Methodology
- Open supplemental data
- Reference Manager
- Simple TEXT file
People also looked at
Original research article, a multiple case study to understand how students experience science and engineering practices.
- 1 Science Office, Omaha Public Schools, Omaha, NE, United States
- 2 Department of Educational Psychology, University of Nebraska-Lincoln, Lincoln, NE, United States
- 3 Department of Biology, University of Nebraska at Omaha, Omaha, NE, United States
- 4 STEM TRAIL Center, University of Nebraska at Omaha, Omaha, NE, United States
The Next Generation Science Standards (NGSS), amid recent shifts in science curriculum, call for students to learn science through the practices of scientists and engineers (science and engineering practices, or SEPs). SEPs, related to inquiry learning, are ways students learn science content by doing science. Students have varied experiences learning science and engineering practices, including exposure in the classroom, from media, and in science fairs. Using a qualitative, multiple case study design, we analyzed public school educators’ and middle and high school students’ (ages 12–18) interview transcripts about learning through the science and engineering practices. Findings demonstrate that students learn different aspects of science and engineering practices during both in school and out-of-school science learning. Several transcending themes emerged from our interview data leading to recommendations for educators. Specific science and engineering practices might be better leveraged to introduce students to scientific research, students saw themselves as scientists leading to development of science identity while learning through SEPs, the relevancy of their work drove student learning, and resiliency was important during many of their learning experiences.
The Next Generation Science Standards (NGSS) are being implemented across educational institutions in the United States ( National Research Council [NRC], 2012 ). Educators are continually challenged by pedagogical aspects of students “doing science” through the science and engineering practices (SEPs) ( Duschl and Bybee, 2014 ; Table 1 ). While one of the strongest predictors for the pursuit of a career in STEM fields is the career of a student’s parents ( Miller et al., 2018 ), many learners may not be exposed to the practices of scientists and/or engineers if they do not have a parent in a STEM field. In some instances, pop-culture media might be an initial or only opportunity for youth to see the process of “doing science” in action. As more and more states adopt NGSS style standards, those requirements warrant learning to take place through these experiences in our kindergarten through high school classrooms. Therefore, educators are tasked with quickly adapting to the question: how else might our young learners learn about doing science?
Table 1. The science and engineering practices (adapted from Figures 1 to 8, Bybee, 2011 ).
Updating science processes in the classroom: Adoption of Next Generation Science Standards
Recently, scientific inquiry shows a conflated relationship, and is oftentimes confused, with science and engineering practices. This conflation can be seen in publications from the American Association for the Advancement of Science [AAAS] (1993) , as well as in work by Chinn and Malhotra (2002) who identify scientific inquiry as the practices of scientists. Crawford (2014) importantly adds that the methods of science differ among disciplines, but that inquiry might be more inclusive in its reliance on complex data analysis, and that it builds on scientific literature and principles. Bybee (2011) notes that changing from an inquiry model of science education to the more rigorous use of science and engineering practices is central to science education, which is further advocated for in the National Research Committee’s publication A Framework for K-12 Science Education: Practices, Crosscutting Concepts, Core Ideas “…students cannot fully understand scientific and engineering ideas without engaging in the practices of inquiry and the discourses by which such ideas are developed and refined” ( National Research Council [NRC], 2012 , p. 218).
The Framework emphasized the term “practices” to clarify and reshape what inquiry in science education looked like, with a focus on modeling and argumentation from evidence ( Crawford, 2014 ). Students should be immersed in the practices to learn science concepts and to learn how to do the practices of scientists and engineers ( Bybee and Van Scotter, 2006 ). The specific principles of the SEPs are captured in Table 1 (Bybee, 2011 ). The SEPs are expansive, and ever evolving given the nature of science and engineering, but the school day is still restricted to the same hours. Consequently, educators, administrators, and parents are seeking additional opportunities for youth to gain such skills in and out of the classroom.
To date, forty-four states have adopted NGSS or NGSS-like standards ( National Science Teachers Assoication [NSTA], 2020 ), which include standards that are based on the three-dimensions of learning science: disciplinary core ideas, crosscutting concepts, and science and engineering practices. Guiding principles established by the National Research Council for SEPs include: student exposure to all eight practices during each predetermined grade band; practices are expectations for what students should experience while learning science concepts; all eight of the practices overlap with each other and build off of one another; aspects of each practice (not the entirety of the practice) are included in performance expectations; practices are based on progressing complexities; and success using these language-intensive SEPs relies on discourse in science ( National Research Council [NRC], 2012 ). Considering the NRC’s guidelines of frequent student exposure to all the SEPs, it is important to understand the opportunities learners are afforded to engage with SEPs.
Teachers using science and engineering practices in the classroom
As noted, many states have adopted NGSS and NGSS-like standards, yet there are concerns about how three-dimensional learning, specifically SEPs, is enacted in schools. Teachers are likely purposeful in how they engage students in SEPs while delivering instruction based on NGSS and other three-dimensional standards. However, there are incongruencies between the perception of the teacher and the perception of classroom observers regarding how well SEP’s are being executed, and concerns about overall frequency of use. Malkawi and Rababah (2018) studied the frequency that Jordanian teachers of twelfth grade science asked learners to engage in each of the SEP’s. For this study teachers self-reported having used each SEP during instruction, although some were used more sparingly than others. Findings from this study indicated a need for more frequent student exposure to SEPs, particularly amid concerns of the Jordanian education system falling behind compared to science achievement measures of other countries. In a similar study, Kawasaki (2015) used ethnographic classroom notes, open-ended teacher interviews, and open-ended questionnaires from a situated perspective for seven teachers of middle and high school science in the United States. Teachers described carrying out more content instruction through SEPs than was deemed to have actually taken place in the classroom. Additionally, teachers conflated science and engineering practices with hands-on experiences such as traditional verification labs/experiments in the classroom. Even though teachers utilize SEPs via a variety of frameworks, and despite the inconsistencies in reporting the frequency of use and need for more SEPs in everyday science lessons, other opportunities exist for many youth to engage in SEPs.
Teachers using science and engineering practices in out-of-school-time science learning
Recent research has shown science fairs as effective ways for students to engage the SEPs ( Koomen et al., 2018 ; DeLisi et al., 2021 ). Since Bybee (2011) states, “Science and engineering practices should be thought of as both learning outcomes and instructional strategies” (p. 39), we aimed to investigate some of the strategies that may further support students on their pathway to learning SEPs. Additionally, since 2014, states have been adopting the NGSS and while some states and districts have worked to be proactive with adoption, the preparation for teachers to adapt to yet another set of standards is less embedded. 1 We sought to understand SEPs through the eyes of educators and of youth participating in research experiences within science.
Therefore, we aim to determine:
• What are students’ understandings about when and how they are engaging in science and engineering practices, both during science class and out-of-school time?
• How might students be learning content through the use of science and engineering practices?
Study context and theoretical framework
The data set utilized for this study was initially collected for a study on how middle and high school students (grades 6–12) experience science research for science fair in the United States ( Andersson et al., 2021 ). Students from our sample might be considered “elite,” or at least very science-driven, as many of them performed at high levels both in science class and during their science fair experiences. During the analysis of that primary research question, we noticed an emergent pattern of SEPs, which led us back to the data set, to then use the SEPs as a lens through which to view the data set. This allowed us to analyze data in implicit ways (students were not specifically asked about SEPs) SEPs show up for students in these experiences, and how teachers and science fair sponsors view students’ research experiences.
Theoretical framework: Conceptual change theory
Inquiry approaches to science learning can be viewed through a combination of theories of learning: community of practice; sociocultural theory, and conceptual change theory ( Crawford, 2014 ). Conceptual change theory notes that students have prior conceptual knowledge that is influenced by educational materials like curriculum, which can lead to alterations in conceptions ( Strike and Posner, 1982 ). The sociocultural theory lens views student engagement with one another while using inquiry approaches in classroom contexts ( Vygotsky, 1978 ). Finally, the community of practice framework acknowledges the dynamic interactions of students and teachers within and among overlapping communities such as social, school, and scientific ( Lave and Wenger, 1991 ). For this study, the focus will be placed on the concept of community of practice, as we investigate if and where student interactions with science and engineering practices are taking place.
Core research question.
The central goal of this study is to understand how students experience science and engineering practices.
The research questions for this study included:
1) How do students experience science and engineering practices both in school and in out-of-school time science learning?
2) How do educators (i.e., educators sponsoring the youth’s participation in a science fair) believe their students experience science and engineering practices in school and in out-of-school time learning?
Qualitative research has been identified to have five traditional approaches: narrative, phenomenology, grounded theory, ethnography, or case study ( Creswell and Poth, 2018 ). The approach of multiple-case study was selected for this work because it provides an in-depth understanding of the situation and contextual meaning for those involved ( Saldaña, 2016 ; Yin, 2018 ). This study follows both youth (i.e., students with experiences in grades 6–12) and science educators (formal, classroom teachers) as they respond to questions about understanding in scientific processes and overlaying these responses to be in alignment (and to what degree) with the NGSS SEPs, see Figure 1 . The researchers viewed the student participants as one case and the science educators as a second case.
Figure 1. Procedures for methodology [adapted from Ivankova et al. (2006) ].
The researchers employed criterion-based sampling ( Creswell and Poth, 2018 ) as student participants all had experienced multiple years of science fair and phenomena-based, three-dimensional science instruction aligned to the Next Generation Science Standards. Further, all science fair sponsors (educators) also had multiple years of experience with leading students through science fairs, as well as their normal science teaching duties. The researchers gained access to participants through a regional science fair, with Institutional Review Board (IRB, University of Nebraska Lincoln) approval. Sponsors/science instructors were not necessarily responsible for the student participants who were interviewed for this study.
Participants were selected based upon their shared experiences of having at least two recent entrances (within three years) into an annual metropolitan area science and engineering fair that took place while they were in grades 6–12. Multiple entrances and recent engagement in science fair helped to ensure that study participants were close to their experiences and would have ample experiences from which to draw. Students with multiple experiences have an expanded worldview of conducting science research for science fair compared to participants who may have had a singular experience. Sampling methods included convenience sampling through partnership with a local science fair for the sampling pool, and maximum variation of the sample by selecting participants who varied across demographic markers. The science and engineering fair IRB approved our request for research participants and worked with us to contact participants. We obtained parental and participant informed consent per institutional IRB guidelines. Our final pool of student participants included: students who identified as male (2), and female (6); high school freshman (1), juniors (3), and college freshman (2) and sophomores (2); African American (4), Caucasian (2), Latinx (1), and Indian (1). Additionally, seven of the eight interviewees had participated at the state science fair level, five of whom presented at the American Junior Academy of Sciences in connection with AAAS. The participant pool also included five educators who had previously sponsored students to enter a science fair, or “sponsors.” More is included in the limitations section; however, this study is limited to the experiences of students who have been successful in science fair as an out of school time activity.
Data collection and analysis
Students and sponsors were interviewed about their science research experiences both in and out of school, but interview questions did not explicitly ask about science and engineering practices. We chose this data collection method to gain an implicit understanding of student experiences. Additionally, even though data was collected in a state that was in its third year of adoption of NGSS-like science standards, that include a focus on SEPs, it is unclear whether all participants have explicitly experienced these in the classroom, i.e., there are many challenges inherent to science-standards-implementation leading to dissimilarities in student experiences. So, our research aims are more exploratory in seeking to understand the contexts and conditions under which students may have had opportunities to use SEPs.
Data was collected through semi-structured interviews, taking place via in-person meetings, telephone, and/or through Zoom. Thirteen total participants were each interviewed twice (eight students, five sponsors) for a total of 26 interviews. Each participant was interviewed by one of two researchers. First interviews focused on in-class science learning and second interviews focused on OST science learning and a comparison of both types, see Supplementary Appendix 1 for the Interview Guide. All recruitment, collection, and analysis were guided by local university IRB (# 20190619326EP-COLLA).
Interview audio was recorded, with permission from the participant, and then transcribed using automated transcription services such as Temi.com . Data was immediately de-identified and pseudonyms added, which are used throughout the results presented in this paper. Interview transcripts were checked for accuracy and then cleaned by the research team in MAXQDA.
The researchers developed a codebook based on an understanding of the SEPs ( Schwarz et al., 2017 ) and used the Lexile search function of MAXQDA to identify initial segments of transcriptions aligned to each of the SEPs, see Table 2 . Each transcript was then separately coded for each of the eight SEPs by two of the researchers, and then cross-checked across the research team for accuracy and for agreement. Discrepancies were discussed until an agreement was reached. For the final round of data analysis, and for congruency and to determine inter-rater reliability, three researchers completed side-by-side review with active discussions of codes and emerging themes through greater than 87% of interview transcripts. All SEPs were found across our study, being present in either or both student and sponsor coded transcriptions, Table 3 provides descriptive information about the numbers of coded segments.
Table 2. Science and engineering practices Lexile search terms.
Table 3. Science and engineering practices number of coded segments.
The resulting data surrounding emergent themes utilizing the SEPs as a lens included subgroupings (1) SEP-specific emergent themes (i.e., by individual SEPs), see Table 4 , and (2) Transcendent themes across SEPs, see Figure 2 .
Table 4. Themes and supporting evidence by SEP.
Figure 2. Science and engineering practice (SEP) specific and transcending themes.
A key difference between in-class science learning vs. OST science learning, such as scientific research for science fair, is the depth of engagement in science as identified by the inquiry continuum ( Biggers and Forbes, 2012 ). Classroom science learning, as prescribed by NGSS, likely gives students guided, more structured inquiry opportunities with learning focused on one or a few SEPs, without following through a complete cycle of scientific investigation. OST learning, as typified in research for science fair, includes engagement in an entire scientific research investigation, which may be more open inquiry than are classroom experiences. In other words, we may expect more thorough SEP experiences during OST science fair.
Science and engineering practice-specific emergent themes
All SEPs were represented in our entire dataset, albeit in varying amounts, see Table 3 , with some SEPs (math and computational thinking, engaging in argument from evidence) being more thinly supported while others yielded very rich findings (asking questions and defining problems, planning, and carrying out investigations). Each SEP was not, however, found in each participant’s interview data, nor were all SEPs represented in both OST and in-school science learning. This led to some important nuances in the data and areas of future research focus, which will be further discussed. Sample quotes which support each theme are found in Table 4 .
Asking questions and defining problems
Students had access to this SEP primarily through OST learning, but there was overlap with in-school experiences for some. Students and sponsors agree on the iterative nature of this SEP in practice, as the process of asking questions is more circular than it is linear. Relevancy was a theme across this SEP, as students who were able to develop their own questions were more engaged in the learning. In fact, some participants who developed research questions noted the broad implications of their research, how it fit into the research landscape, that is they “saw themselves among scientists.”
Sponsors noted that the current science curriculum can allow for students to explore open-ended questions. However, even though educators have license to provide learning opportunities for students to ask questions and define problems during in-school learning, students may not be exposed to this SEP early enough to develop student skills in asking investigable questions. Additionally, classroom contexts such as 30 students in a classroom for one 45-min segment create difficulties, as several sponsors identified. In this study, OST learning environments seem best positioned to allow students to access this SEP.
Due to the impact of this SEP on the learners interviewed for this study, we recommend that educators consider “Asking Questions and Defining Problems” as an access point for students to experience science learning through science research, both in-school and during OST. One method noted in our data suggests that more equitable outcomes might be achieved if students can discuss background content and work on question development in small groups, while focusing on what is relevant to the students and meaningful to the community.
Developing and using models
Students including Aliah, Sarah, and Damian, and sponsors such as Alissa, discussed the importance of using models toward learning gains. For example, Damian discussed the importance of developing models of scientific concepts to interpret phenomena, otherwise “you either get lost or you kinda just get bored with it” in your efforts to learn.
An important nuance to our interview data regarding Developing and Using Models is that the codes hailed almost entirely from in-class science learning. Our data suggest that students are most explicitly understanding this SEP when they are guided through aspects of it. There are implications here for the role of and need for mentors, whether they are classroom teachers or experts sponsoring students with their science-research endeavors.
Planning and carrying out investigations
Planning and carrying out investigations was implicated in both OST and in-school science learning. In the classroom, students largely noted the excitement for and effectiveness of learning when able to carry out investigations rather than learning from “reading from a textbook.” For one student participant, Sarah, learning how to plan and carry out experiments was key to how they developed confidence in doing science, which led to the student “seeing themselves among scientists.” Another student, Valarie, realized the iterative nature of this SEP, while “discovering new tools which changed the plan for data collection.” Relevance and autonomy were again important while students carried out their own investigations.
Mentoring was one of the key themes from our data analysis. Coded segments from several sponsor interviews identified the importance of mentoring with planning and carrying out investigations . From a mentor’s perspective, students benefit from guidance in understanding constraints of any given research, including variables and research methods. Student participants benefited from “dreaming big” about their ideal experiment, and then being supported in their exploration by sponsors, science coaches, and other mentors.
Findings from this SEP also brought out a negative case – Jared, a sponsor, saw school curriculum as the “dumbing down” of the process of science. OST requires students to develop this on their own, which is where many learners get “left behind.” For some student participants, the in-class science learning, while providing opportunity for student choice in how to set up an experiment, was mostly confirmatory science research – “what has already been established by other scientists and experts.” Perhaps some of the greatest potential for student gains in being able to effectively learn by doing science come across in access to this SEP in environments which are rich in mentoring, support, and opportunity for authentic investigation.
Analyzing and interpreting data
The majority of codes for Analyzing and Interpreting Data came through in sponsor interviews, and mostly during OST science learning. Sponsors noted the importance of mentoring, while students analyze and interpret data, which aligns with research findings from Chen et al. (2011) . This can include students being guided through challenges associated with having null data. Further, sponsors noted that students get opportunities to collect and analyze data in both contexts (OST and in-school), but it is important for students to be able to analyze data they have collected, which is an access point for students to gain their science identity. Therefore, student ability to collect and analyze their own data might be a major contributor toward students “seeing themselves among scientists.”
Sarah discussed the interrelatedness of this SEP and Planning and Conducting Experiments , based on a lesson in Physics class, while another student noted the need for precision, and the time burden of data analysis. Although these findings are lightly represented in our data, they may nonetheless be important and require further investigation.
Using mathematics and computational thinking
Students likely have opportunities to use math and computational thinking when they’re conducting scientific research for OST science research but also during in-class learning. Yet, our findings yielded minor insights into this SEP. Both findings from sponsors showed students gained access to “doing science” and researching their interests came through math and computational thinking. For example, a student who enjoyed playing basketball became more interested in research when the sponsor and student started keeping track of shooting percentages from various points on a basketball court. To understand the implications of this SEP more fully, further research should explicitly investigate this SEP in both learning contexts, especially the potential of this SEP to be a gateway to doing science.
Constructing explanations and design solutions
Student participants experienced this SEP, both in-school and during OST science learning, according to sponsors and to student participants. Despite having relatively few coded segments compared to other SEPs such as Planning and Carrying Out Investigations, key findings came through in the interactive nature of constructing explanations such as adjusting research methods and data collection. Additionally, students expressed satisfaction in their ability to construct explanations from their data, as well as gaining confidence in explanations they were able to develop. Student feelings of success here may also play into them “seeing themselves among scientists.”
Engaging in argument from evidence
Our interview data yielded few coded segments for Engaging in Argument from Evidence, but once again this provided us with key insights. One sponsor, Doug, noted that students struggle to argue from their evidence when they gain inconclusive results, yet this may be critical for students to experience so they may more fully understand scientific research. Two student participants who experienced this SEP during OST learning, Marcus in an engineering project, and Andrea in a honeybee ecology project, discussed using not only the results of the experiment, but also the experimental design to effectively deliver their argument. While these two students found success with engaging in argument from evidence based on their experiences in science fair, there was a clear lack of responses from all students during science in school. This key finding is important as engaging in argument from evidence is central to the purpose of science ( Bybee, 2011 ; Chen et al., 2011 ; National Research Council [NRC], 2012 ). A key implication here is for science educators to focus on this science practice more explicitly during instruction. Science education leaders should also look for opportunities to support teachers with appropriate curricula and professional learning experiences that include opportunities to practice engaging in argument from evidence.
Obtaining, evaluating, and communicating evidence
Nearly all our research participants experienced this SEP both in-school and during OST learning. The key theme was the potential for students to build confidence in their ability to effectively communicate science, both during in-class presentations of their work (OST or in-school), and, for example, presenting their research at science fairs. There was another key interplay between in-school and OST learning here, in that several students spoke to various methods of obtaining information they needed for class projects or for OST research. Students used sources such as internet searches, classroom lessons, reading texts and research articles, seeking professionals, and technology and science departments across the school to inform their coursework and their OST research. In other words, knowledge students gained from one context was not solely used in that context.
Transcending themes (themes found across science and engineering practices)
From the perspective of educational researchers who are also educators, one of the most rewarding aspects of this research was the emergence of themes that transcended SEPs. These themes not only support prescriptive uses of students doing scientific research, but also demonstrate the potential for powerful learning outcomes, both in and out of the classroom.
Introduction into scientific research
Findings suggest that some SEPs might be leveraged to introduce students to the processes of scientific investigation. Some of our participants did not embark on doing scientific research, rather they were drawn in by their exposure to one of the SEPs, typically during OST science learning. For example, a sponsor, Leslie, used a student’s interest in mathematics to get the student started on a science research project, through the mathematics and computational thinking SEP. Jared, also a sponsor, reported the benefits of jumping into learning sequences with more “guided inquiry,” using the SEP collecting and analyzing data for students to do their own collection and analysis, further giving the students an entry point into the SEPs and consequently into scientific research.
Our findings highlighted mathematics and computational thinking , analyzing and interpreting data , and especially asking questions and defining problems as entry points for the participants of this study. However, we postulate that any SEP which aligns with the interests and skills of a student might be used to introduce that student to scientific research. For example, a student who may enjoy being on the stage might be drawn in by obtaining, evaluating, and communicating information , or even engaging in argument from evidence . Sponsors we interviewed demonstrated skill in their ability to identify the strengths and interests of their students and then orient specific SEPs toward those strengths and interests, thereby opening doors for students which may not otherwise have been opened.
Finding yourself among scientists
Research participants experienced important breakthroughs during their OST science learning. These were points where students felt accomplished in the product of their learning, especially during the SEPs using mathematics and computational thinking, planning, and carrying out investigations , and analyzing and interpreting data. Students felt like they were “doing real science,” realizing they had completed processes which professional scientists and engineers do. Further, some students reported that they felt comfortable asking questions and discussing research with scientists. We believe this theme interplays with student science identity ( Starr et al., 2020 ). This finding warrants further investigation to understand this relationship more fully.
The research participants, both the students and mentors, reported learning about the role of failure in science and engineering. Both reported moving from a place of avoiding failure at all costs to embracing failure within their time of doing science and employing the science and engineering practices. These findings align with what is known about resilience and conceptual change, specifically related to anomalous data ( Chinn and Brewer, 1993 ). This transcending theme is interconnected with other transcending themes, considering that students reported gaining confidence in their abilities which led to students finding themselves among scientists.
Relevancy of the work
Opportunities for students to explore problems that need to be solved, which are relevant to themselves, meaningful to the community are important hooks to gaining student involvement. This theme coincides with “introduction into scientific research” in that when students are able to ask their own questions, they typically ask questions which are meaningful to themselves. Relevancy of the work is what initially draws many students toward scientific investigation, so this is a catalyst for students to be exposed to any and hopefully all the SEPs.
While some sponsors, and likely many other educators, note that opportunities for students to investigate personally relevant topics are often confounded by the rigidity of standards and curricula, we feel it is still critical to highlight our findings here. When learning was relevant to students, they were more engaged in the learning activities. The OST science research experiences of our participants seemed to afford such opportunities, and in several cases student learning from their OST science activities enhanced their classroom learning.
This study adds to the research on how students experience science and engineering practices. Specifically, we examined how students reported experiencing science and engineering practices during in-school science learning and OST science learning, i.e., science fair. The researchers also utilized interview data from educators who sponsored students engaged in science fairs. We realized that students often initially lack confidence in completing science research, and that this can be abated by having an educator, or other mentor, interact with the students as they progress through the practices. Increasing the students’ confidence leads to students seeing themselves among scientists. From the mentor’s data we also learned there may be a personal cost to students if their inquiry questions are not legitimized. So, a student’s questions, no matter how far-fetched, are a doorway to the student’s thinking. Further, if a mentor shuts down the question, doors to student thinking and doing science may be effectively closed, discouraging their participation in science.
The students also reported the importance of mentors. Although we had one student that reported seeking success to prove her science research abilities to her teacher/mentor, all the other students reported that the sponsors had a massive role for the students. Specifically, students found that what they learned while engaging in science and engineering practices translated to other learning experiences. The students who had already progressed into college reported their confidence to lead small groups with projects in college. They also reported feeling more prepared to enter the workforce even if they were not focused on science as a career. For example, they were able to find use in the lessons learned in talking with adults and presenting findings to judges.
Embedding science and engineering practices into student activities inside and outside the traditional classroom
Science and engineering practices, as part of the Next Generation Science Standards, have a history embedded in inquiry approaches to science learning. Research indicating the benefits of this approach to science learning is abundant ( McGee-Brown et al., 2003 ; National Research Council [NRC], 2012 ; Marshall and Alston, 2014 ; Osborne, 2014 ). Additionally, and of essential importance are the implications that inquiry learning and learning through the use of SEPs can benefit the diverse learners that are present in our classrooms ( Rahm and Moore, 2016 ). In other words, students should have access to this type of learning, in various settings.
Learners might engage with SEPs in their science classroom, or through more informal science-learning experiences such as OST. In the classroom, educators (pre-service and in-service) might not have the pedagogical or content background necessary to implement the necessary shifts, that is, students learning content through the use of SEP’s. Research has found that professional development can help provide more of these learning opportunities in the classroom for our science learners ( Banilower et al., 2018 ). Curricular projects are available that can be scaled to district needs, including teacher preparation. Outside of the classroom, students may have access to SEP’s through activities such as science club and preparation for science competitions, which have been shown to provide excellent opportunities for those who can participate. An additional challenge to both in-school and OST science learning is understanding the epistemic and conceptual aspects of science and engineering practices, including the understanding of the nature of science. Finally, assessment systems might be a critical factor in the amount of resources that districts, schools, and teachers put into the vision of learning science content through the use of science and engineering practices.
There remains a gap in our understanding of how the science and engineering practices are being implemented in the classroom and potential inequities in this. There are also gaps in the student perspective of exposure to SEPs through informal, unstructured learning opportunities. Research on how students experience this most recent attempt at inquiry learning could yield great benefits toward many levels of education: lesson, curriculum, standards, and equity within science education.
Are some students more aligned to understanding and interpreting science and engineering practices, regardless of when or how presented?
During data analysis, it became clear that some students were more effective than others at articulating their scientific research experiences in the classroom or OST, be that visiting a laboratory of an independent scientist, or by working directly with their teacher. Consequently, we investigated any alignment by type of experience. For example, if a student articulated very specific and explicit acknowledgment and understanding of SEPs, did they cluster by a particular type of experience that could be modeled more in the future? Unfortunately, these data did not cluster to be by shared types of activities and were instead randomly dispersed.
Classroom science learning experiences as described by our participants are just a snapshot of their entire learning experience, which may include learning progressions through all SEPs, and science disciplines as prescribed by NGSS. Our data do not yield an entire understanding of this learning, but just an indication of how some higher-level learners have experienced science and engineering practices in the classroom. Additionally, although student participants in this study hailed from various racial, ethnic, and gender identities, our sample was homogenous in that our participants typically excelled at school and sought out, or were sought out by educators for, involvement in OST science learning. Because we intentionally included students with multiple experiences in science fairs, our data set is limited to students who likely had “resource-rich” backgrounds ( Bencze and Bowen, 2009 ). Finally, every research method has its limitations, as is the case with our qualitative approach here. While we gained deep insight toward our research questions, the findings will be difficult to generalize to broader/different populations than what we investigated here. However, we contend that our findings provide a rich description that contributes to our understanding of how middle and high school students may experience SEPs.
Data availability statement
The original contributions presented in this study are included in the article/ Supplementary material , further inquiries can be directed to the corresponding author.
The studies involving human participants were reviewed and approved by the University of Nebraska-Lincoln Institutional Review Board. Written informed consent to participate in this study was provided by the participants’ legal guardian/next of kin.
CS and JA designed the study and collected the data. CS, JA, and CC analyzed the data and contributed to the manuscript. All authors contributed to the article and approved the submitted version.
This work was supported by the grant from NSF NoyceSCIENCE #1659058.
We thank the study participants for the time they gave us in interviews and member-checks. The researchers also acknowledge Hayley Jurek for assisting with the graphics created. Finally, we also acknowledge Eric Buhs and Nealy Grandgenette for their assistance with the initial data collection and for their continued support.
Conflict of interest
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.
The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/feduc.2022.960346/full#supplementary-material
- ^ https://ngss.nsta.org/Default.aspx
American Association for the Advancement of Science [AAAS] (1993). Project 2061 Atlas of Scientific Literacy. Washington, DC: AAAS.
Andersson, J., Schaben, C., Buhs, E., and Grandgenett, N. (2021). Phenomenology of secondary students’ experiences in out-of-school time science research. Sci. Educ. 28, 30–39.
Banilower, E. R., Smith, P. S., Malzahn, K. A., Plumley, C. L., Gordon, E. M., and Hayes, M. L. (2018). Report of the 2018 NSSME+. Chapel Hill, NC: Horizon Research Inc.
Bencze, J. L., and Bowen, G. M. (2009). A national science fair: exhibiting support for the knowledge economy. Int. J. Sci. Educ. 31, 2459–2483. doi: 10.1080/09500690802398127
CrossRef Full Text | Google Scholar
Biggers, M., and Forbes, C. T. (2012). Balancing teacher and student roles in elementary classrooms: preservice elementary teachers’ learning about the inquiry continuum. Int. J. Sci. Educ. 34, 2205–2229. doi: 10.1080/09500693.2012.694146
Bybee, R. W. (2011). Scientific and engineering practices in K-12 classrooms: understanding “a framework for K-12 science education”. Sci. Teach. 78, 4–40. doi: 10.1186/s41235-020-00222-9
PubMed Abstract | CrossRef Full Text | Google Scholar
Bybee, R. W., and Van Scotter, P. (2006). Reinventing the science curriculum. Educ. Leadership 64, 43–47.
Chen, J. J., Lin, H. S., Hsu, Y. S., and Lee, H. (2011). Data and claim: the refinement of science fair work through argumentation. Int. J. Sci. Educ. Part B 1, 147–164.
Chinn, C., and Malhotra, B. (2002). Epistemologically authentic inquiry in schools: a theoretical framework for evaluating inquiry tasks. Sci. Educ. 86, 175–218.
Chinn, C. A., and Brewer, W. F. (1993). The role of anomalous data in knowledge acquisition: a theoretical framework and implications for science education. Rev. Educ. Res. 63, 1–49. doi: 10.3102/00346543063001001
Crawford, B. (2014). “Scientific Practices and Inquiry in the Science Classroom,” in Handbook of Research on Science Education , eds N. G. Lederman and S. K. Abell (New York, NY: Routledge).
Creswell, J., and Poth, C. (2018). Qualitative Inquiry and Research Design: Choosing Among Five Approaches , 4th Edn. Thousand Oaks, CA: Sage.
DeLisi, J., Kook, J. F., Levy, A. J., Fields, E., and Winfiled, L. (2021). An examination of the features of science fairs that support students’ understanding of science and engineering practices. J. Res. Sci. Teach. 58, 491–519. doi: 10.1002/tea.21669
Duschl, R. A., and Bybee, R. W. (2014). Planning and carrying out investigations: an entry to learning and to teacher professional development around NGSS science and engineering practices. Int. J. STEM Educ. 1, 1–9. doi: 10.1186/s40594-014-0012-6
Ivankova, N. V., Creswell, J. W., and Stick, S. L. (2006). Using mixed-methods sequential explanatory design: from theory to practice. Field Methods 18, 3–20. doi: 10.1177/1525822X05282260
Kawasaki, J. N. (2015). Examining Teachers’ Goals and Classroom Instruction Around the Science and Engineering Practices in the Next Generation Science Standards. Los Angeles, CA: UCLA.
Koomen, M. H., Rodriquez, E., Hoffman, A., Petersen, C., and Oberhauser, K. (2018). Authentic science with citizen science and student-driven science fair projects. Sci. Educ. 102, 593–644. doi: 10.1002/sce.21335
Lave, J., and Wenger, E. (1991). Situated Learning: Legitimate Peripheral Participation. New York, NY: Cambridge University Press.
Malkawi, A. R., and Rababah, E. Q. (2018). Jordanian twelfth-grade science teachers’ self-reported usage of science and engineering practices in the next generation science standards. Int. J. Sci. Educ. 40, 961–976.
Marshall, J. C., and Alston, D. M. (2014). Effective, sustained inquiry-based instruction promotes higher science proficiency among all groups: a 5-Year analysis. J. Sci. Teach. Educ. 25, 807–821.
McGee-Brown, M., Martin, C., Monsaas, J., and Stombler, M. (2003). “What scientists do: science olympiad enhancing science inquiry through student collaboration, problem solving, and creativity,” in Proceedings of the Annual National Science Teachers Association meeting , (Philadelphia, PA).
Miller, K., Sonnert, G., and Sadler, P. (2018). The influence of students’ participation in STEM competitions on their interest in STEM careers. Int. J. Sci. Educ. Part B: Commun. Public Engagement 8, 95–114. doi: 10.1371/journal.pone.0108497
National Research Council [NRC] (2012). A Framework for K-12 Science Education: Practices, Crosscutting Concepts, and Core Ideas. Uttar Pradesh: The National Academy of the Sciences.
National Science Teachers Assoication [NSTA] (2020). Compare NGSS to existing state standards2. NGSS@NSTA. Available online at: https://ngss.nsta.org/compare-the-standards.aspx (accessed November 20, 2022).
Osborne, J. (2014). “Scientific practices and inquiry in the science classroom,” in Handbook of Research on Science Education , eds N. G. Lederman and S. K. Abell (New York, NY: Routledge).
Rahm, J., and Moore, J. C. (2016). A case study of long-term engagement and identity-in-practice: insights into the STEM pathways of four underrepresented youths. J. Res. Sci. Teach. 53, 768–801.
Saldaña, J. (2016). The Coding Manual for Qualitative Researchers. Los Angeles, CA: SAGE Publications.
Schwarz, C., Passmore, C., and Reiser, B. J. (eds) (2017). Helping Students Make Sense of the World Using Next Generation Science and Engineering Practices. Arlington, VA: NSTA Press.
Starr, C. R., Hunter, L., Dunkin, R., Honig, S., Palomino, R., and Leaper, C. (2020). Engaging in science practices in classrooms predicts increases in undergraduates’ STEM motivation, identity, and achievement: a short-term longitudinal study. J. Res. Sci. Teach. 57, 1093–1118.
Strike, K., and Posner, G. (1982). Conceptual change and science teaching. Eur. J. Sci. Educ. 4, 231–240. doi: 10.1080/0140528820040302
Vygotsky, L. (1978). Mind in Society: The Development of Higher Mental Processes. Cambridge, MA: Harvard University.
Yin, S. R. (2018). Case Study Research and Applications: Design and Method , 6th Edn. Thousand Oaks, CA: Sage.
Keywords : out-of-school time, NGSS, science fair, science and engineering practices (SEPs), multiple case study, science fair mentor
Citation: Schaben C, Andersson J and Cutucache C (2022) A multiple case study to understand how students experience science and engineering practices. Front. Educ. 7:960346. doi: 10.3389/feduc.2022.960346
Received: 02 June 2022; Accepted: 11 November 2022; Published: 01 December 2022.
Copyright © 2022 Schaben, Andersson and Cutucache. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY) . The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Chris Schaben, [email protected]