case study clinical method

• - Google Chrome

Intended for healthcare professionals

• My email alerts
• BMA member login
• Username * Password * Forgot your log in details? Need to activate BMA Member Log In Log in via OpenAthens Log in via your institution

Search form

• Advanced search
• Search responses
• Search blogs
• Writing a case report...

Writing a case report in 10 steps

• Related content
• Peer review
• Victoria Stokes , foundation year 2 doctor, trauma and orthopaedics, Basildon Hospital ,
• Caroline Fertleman , paediatrics consultant, The Whittington Hospital NHS Trust
• victoria.stokes1{at}nhs.net

Victoria Stokes and Caroline Fertleman explain how to turn an interesting case or unusual presentation into an educational report

It is common practice in medicine that when we come across an interesting case with an unusual presentation or a surprise twist, we must tell the rest of the medical world. This is how we continue our lifelong learning and aid faster diagnosis and treatment for patients.

It usually falls to the junior to write up the case, so here are a few simple tips to get you started.

First steps

Begin by sitting down with your medical team to discuss the interesting aspects of the case and the learning points to highlight. Ideally, a registrar or middle grade will mentor you and give you guidance. Another junior doctor or medical student may also be keen to be involved. Allocate jobs to split the workload, set a deadline and work timeframe, and discuss the order in which the authors will be listed. All listed authors should contribute substantially, with the person doing most of the work put first and the guarantor (usually the most senior team member) at the end.

Getting consent

Gain permission and written consent to write up the case from the patient or parents, if your patient is a child, and keep a copy because you will need it later for submission to journals.

Information gathering

Gather all the information from the medical notes and the hospital’s electronic systems, including copies of blood results and imaging, as medical notes often disappear when the patient is discharged and are notoriously difficult to find again. Remember to anonymise the data according to your local hospital policy.

Write up the case emphasising the interesting points of the presentation, investigations leading to diagnosis, and management of the disease/pathology. Get input on the case from all members of the team, highlighting their involvement. Also include the prognosis of the patient, if known, as the reader will want to know the outcome.

Coming up with a title

Discuss a title with your supervisor and other members of the team, as this provides the focus for your article. The title should be concise and interesting but should also enable people to find it in medical literature search engines. Also think about how you will present your case study—for example, a poster presentation or scientific paper—and consider potential journals or conferences, as you may need to write in a particular style or format.

Background research

Research the disease/pathology that is the focus of your article and write a background paragraph or two, highlighting the relevance of your case report in relation to this. If you are struggling, seek the opinion of a specialist who may know of relevant articles or texts. Another good resource is your hospital library, where staff are often more than happy to help with literature searches.

How your case is different

Move on to explore how the case presented differently to the admitting team. Alternatively, if your report is focused on management, explore the difficulties the team came across and alternative options for treatment.

Finish by explaining why your case report adds to the medical literature and highlight any learning points.

Writing an abstract

The abstract should be no longer than 100-200 words and should highlight all your key points concisely. This can be harder than writing the full article and needs special care as it will be used to judge whether your case is accepted for presentation or publication.

Discuss with your supervisor or team about options for presenting or publishing your case report. At the very least, you should present your article locally within a departmental or team meeting or at a hospital grand round. Well done!

Competing interests: We have read and understood BMJ’s policy on declaration of interests and declare that we have no competing interests.

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, generate accurate citations for free.

• Knowledge Base

Methodology

• What Is a Case Study? | Definition, Examples & Methods

What Is a Case Study? | Definition, Examples & Methods

Published on May 8, 2019 by Shona McCombes . Revised on November 20, 2023.

A case study is a detailed study of a specific subject, such as a person, group, place, event, organization, or phenomenon. Case studies are commonly used in social, educational, clinical, and business research.

A case study research design usually involves qualitative methods , but quantitative methods are sometimes also used. Case studies are good for describing , comparing, evaluating and understanding different aspects of a research problem .

Table of contents

When to do a case study, step 1: select a case, step 2: build a theoretical framework, step 3: collect your data, step 4: describe and analyze the case, other interesting articles.

A case study is an appropriate research design when you want to gain concrete, contextual, in-depth knowledge about a specific real-world subject. It allows you to explore the key characteristics, meanings, and implications of the case.

Case studies are often a good choice in a thesis or dissertation . They keep your project focused and manageable when you don’t have the time or resources to do large-scale research.

You might use just one complex case study where you explore a single subject in depth, or conduct multiple case studies to compare and illuminate different aspects of your research problem.

Receive feedback on language, structure, and formatting

Professional editors proofread and edit your paper by focusing on:

• Academic style
• Vague sentences
• Style consistency

See an example

Once you have developed your problem statement and research questions , you should be ready to choose the specific case that you want to focus on. A good case study should have the potential to:

• Provide new or unexpected insights into the subject
• Challenge or complicate existing assumptions and theories
• Propose practical courses of action to resolve a problem
• Open up new directions for future research

TipIf your research is more practical in nature and aims to simultaneously investigate an issue as you solve it, consider conducting action research instead.

Unlike quantitative or experimental research , a strong case study does not require a random or representative sample. In fact, case studies often deliberately focus on unusual, neglected, or outlying cases which may shed new light on the research problem.

Example of an outlying case studyIn the 1960s the town of Roseto, Pennsylvania was discovered to have extremely low rates of heart disease compared to the US average. It became an important case study for understanding previously neglected causes of heart disease.

However, you can also choose a more common or representative case to exemplify a particular category, experience or phenomenon.

Example of a representative case studyIn the 1920s, two sociologists used Muncie, Indiana as a case study of a typical American city that supposedly exemplified the changing culture of the US at the time.

While case studies focus more on concrete details than general theories, they should usually have some connection with theory in the field. This way the case study is not just an isolated description, but is integrated into existing knowledge about the topic. It might aim to:

• Exemplify a theory by showing how it explains the case under investigation
• Expand on a theory by uncovering new concepts and ideas that need to be incorporated
• Challenge a theory by exploring an outlier case that doesn’t fit with established assumptions

To ensure that your analysis of the case has a solid academic grounding, you should conduct a literature review of sources related to the topic and develop a theoretical framework . This means identifying key concepts and theories to guide your analysis and interpretation.

There are many different research methods you can use to collect data on your subject. Case studies tend to focus on qualitative data using methods such as interviews , observations , and analysis of primary and secondary sources (e.g., newspaper articles, photographs, official records). Sometimes a case study will also collect quantitative data.

Example of a mixed methods case studyFor a case study of a wind farm development in a rural area, you could collect quantitative data on employment rates and business revenue, collect qualitative data on local people’s perceptions and experiences, and analyze local and national media coverage of the development.

The aim is to gain as thorough an understanding as possible of the case and its context.

Prevent plagiarism. Run a free check.

In writing up the case study, you need to bring together all the relevant aspects to give as complete a picture as possible of the subject.

How you report your findings depends on the type of research you are doing. Some case studies are structured like a standard scientific paper or thesis , with separate sections or chapters for the methods , results and discussion .

Others are written in a more narrative style, aiming to explore the case from various angles and analyze its meanings and implications (for example, by using textual analysis or discourse analysis ).

In all cases, though, make sure to give contextual details about the case, connect it back to the literature and theory, and discuss how it fits into wider patterns or debates.

If you want to know more about statistics , methodology , or research bias , make sure to check out some of our other articles with explanations and examples.

• Normal distribution
• Degrees of freedom
• Null hypothesis
• Discourse analysis
• Control groups
• Mixed methods research
• Non-probability sampling
• Quantitative research
• Ecological validity

Research bias

• Rosenthal effect
• Implicit bias
• Cognitive bias
• Selection bias
• Negativity bias
• Status quo bias

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the “Cite this Scribbr article” button to automatically add the citation to our free Citation Generator.

McCombes, S. (2023, November 20). What Is a Case Study? | Definition, Examples & Methods. Scribbr. Retrieved August 5, 2024, from https://www.scribbr.com/methodology/case-study/

Is this article helpful?

Shona McCombes

Other students also liked, primary vs. secondary sources | difference & examples, what is a theoretical framework | guide to organizing, what is action research | definition & examples, "i thought ai proofreading was useless but..".

I've been using Scribbr for years now and I know it's a service that won't disappoint. It does a good job spotting mistakes”

Case Study Research Method in Psychology

Saul McLeod, PhD

Editor-in-Chief for Simply Psychology

BSc (Hons) Psychology, MRes, PhD, University of Manchester

Saul McLeod, PhD., is a qualified psychology teacher with over 18 years of experience in further and higher education. He has been published in peer-reviewed journals, including the Journal of Clinical Psychology.

Learn about our Editorial Process

Olivia Guy-Evans, MSc

Associate Editor for Simply Psychology

BSc (Hons) Psychology, MSc Psychology of Education

Olivia Guy-Evans is a writer and associate editor for Simply Psychology. She has previously worked in healthcare and educational sectors.

On This Page:

Case studies are in-depth investigations of a person, group, event, or community. Typically, data is gathered from various sources using several methods (e.g., observations & interviews).

The case study research method originated in clinical medicine (the case history, i.e., the patient’s personal history). In psychology, case studies are often confined to the study of a particular individual.

The information is mainly biographical and relates to events in the individual’s past (i.e., retrospective), as well as to significant events that are currently occurring in his or her everyday life.

The case study is not a research method, but researchers select methods of data collection and analysis that will generate material suitable for case studies.

Freud (1909a, 1909b) conducted very detailed investigations into the private lives of his patients in an attempt to both understand and help them overcome their illnesses.

This makes it clear that the case study is a method that should only be used by a psychologist, therapist, or psychiatrist, i.e., someone with a professional qualification.

There is an ethical issue of competence. Only someone qualified to diagnose and treat a person can conduct a formal case study relating to atypical (i.e., abnormal) behavior or atypical development.

 Famous Case Studies

• Anna O – One of the most famous case studies, documenting psychoanalyst Josef Breuer’s treatment of “Anna O” (real name Bertha Pappenheim) for hysteria in the late 1800s using early psychoanalytic theory.
• Little Hans – A child psychoanalysis case study published by Sigmund Freud in 1909 analyzing his five-year-old patient Herbert Graf’s house phobia as related to the Oedipus complex.
• Bruce/Brenda – Gender identity case of the boy (Bruce) whose botched circumcision led psychologist John Money to advise gender reassignment and raise him as a girl (Brenda) in the 1960s.
• Genie Wiley – Linguistics/psychological development case of the victim of extreme isolation abuse who was studied in 1970s California for effects of early language deprivation on acquiring speech later in life.
• Phineas Gage – One of the most famous neuropsychology case studies analyzes personality changes in railroad worker Phineas Gage after an 1848 brain injury involving a tamping iron piercing his skull.

Clinical Case Studies

• Studying the effectiveness of psychotherapy approaches with an individual patient
• Assessing and treating mental illnesses like depression, anxiety disorders, PTSD
• Neuropsychological cases investigating brain injuries or disorders

Child Psychology Case Studies

• Studying psychological development from birth through adolescence
• Cases of learning disabilities, autism spectrum disorders, ADHD
• Effects of trauma, abuse, deprivation on development

Types of Case Studies

• Explanatory case studies : Used to explore causation in order to find underlying principles. Helpful for doing qualitative analysis to explain presumed causal links.
• Exploratory case studies : Used to explore situations where an intervention being evaluated has no clear set of outcomes. It helps define questions and hypotheses for future research.
• Descriptive case studies : Describe an intervention or phenomenon and the real-life context in which it occurred. It is helpful for illustrating certain topics within an evaluation.
• Multiple-case studies : Used to explore differences between cases and replicate findings across cases. Helpful for comparing and contrasting specific cases.
• Intrinsic : Used to gain a better understanding of a particular case. Helpful for capturing the complexity of a single case.
• Collective : Used to explore a general phenomenon using multiple case studies. Helpful for jointly studying a group of cases in order to inquire into the phenomenon.

Where Do You Find Data for a Case Study?

There are several places to find data for a case study. The key is to gather data from multiple sources to get a complete picture of the case and corroborate facts or findings through triangulation of evidence. Most of this information is likely qualitative (i.e., verbal description rather than measurement), but the psychologist might also collect numerical data.

1. Primary sources

• Interviews – Interviewing key people related to the case to get their perspectives and insights. The interview is an extremely effective procedure for obtaining information about an individual, and it may be used to collect comments from the person’s friends, parents, employer, workmates, and others who have a good knowledge of the person, as well as to obtain facts from the person him or herself.
• Observations – Observing behaviors, interactions, processes, etc., related to the case as they unfold in real-time.
• Documents & Records – Reviewing private documents, diaries, public records, correspondence, meeting minutes, etc., relevant to the case.

2. Secondary sources

• News/Media – News coverage of events related to the case study.
• Academic articles – Journal articles, dissertations etc. that discuss the case.
• Government reports – Official data and records related to the case context.
• Books/films – Books, documentaries or films discussing the case.

3. Archival records

Searching historical archives, museum collections and databases to find relevant documents, visual/audio records related to the case history and context.

Public archives like newspapers, organizational records, photographic collections could all include potentially relevant pieces of information to shed light on attitudes, cultural perspectives, common practices and historical contexts related to psychology.

4. Organizational records

Organizational records offer the advantage of often having large datasets collected over time that can reveal or confirm psychological insights.

Of course, privacy and ethical concerns regarding confidential data must be navigated carefully.

However, with proper protocols, organizational records can provide invaluable context and empirical depth to qualitative case studies exploring the intersection of psychology and organizations.

• Organizational/industrial psychology research : Organizational records like employee surveys, turnover/retention data, policies, incident reports etc. may provide insight into topics like job satisfaction, workplace culture and dynamics, leadership issues, employee behaviors etc.
• Clinical psychology : Therapists/hospitals may grant access to anonymized medical records to study aspects like assessments, diagnoses, treatment plans etc. This could shed light on clinical practices.
• School psychology : Studies could utilize anonymized student records like test scores, grades, disciplinary issues, and counseling referrals to study child development, learning barriers, effectiveness of support programs, and more.

How do I Write a Case Study in Psychology?

Follow specified case study guidelines provided by a journal or your psychology tutor. General components of clinical case studies include: background, symptoms, assessments, diagnosis, treatment, and outcomes. Interpreting the information means the researcher decides what to include or leave out. A good case study should always clarify which information is the factual description and which is an inference or the researcher’s opinion.

1. Introduction

• Provide background on the case context and why it is of interest, presenting background information like demographics, relevant history, and presenting problem.
• Compare briefly to similar published cases if applicable. Clearly state the focus/importance of the case.

2. Case Presentation

• Describe the presenting problem in detail, including symptoms, duration,and impact on daily life.
• Include client demographics like age and gender, information about social relationships, and mental health history.
• Describe all physical, emotional, and/or sensory symptoms reported by the client.
• Use patient quotes to describe the initial complaint verbatim. Follow with full-sentence summaries of relevant history details gathered, including key components that led to a working diagnosis.
• Summarize clinical exam results, namely orthopedic/neurological tests, imaging, lab tests, etc. Note actual results rather than subjective conclusions. Provide images if clearly reproducible/anonymized.
• Clearly state the working diagnosis or clinical impression before transitioning to management.

3. Management and Outcome

• Indicate the total duration of care and number of treatments given over what timeframe. Use specific names/descriptions for any therapies/interventions applied.
• Present the results of the intervention,including any quantitative or qualitative data collected.
• For outcomes, utilize visual analog scales for pain, medication usage logs, etc., if possible. Include patient self-reports of improvement/worsening of symptoms. Note the reason for discharge/end of care.

4. Discussion

• Analyze the case, exploring contributing factors, limitations of the study, and connections to existing research.
• Analyze the effectiveness of the intervention,considering factors like participant adherence, limitations of the study, and potential alternative explanations for the results.
• Identify any questions raised in the case analysis and relate insights to established theories and current research if applicable. Avoid definitive claims about physiological explanations.
• Offer clinical implications, and suggest future research directions.

5. Additional Items

• Thank specific assistants for writing support only. No patient acknowledgments.
• References should directly support any key claims or quotes included.
• Use tables/figures/images only if substantially informative. Include permissions and legends/explanatory notes.
• Provides detailed (rich qualitative) information.
• Provides insight for further research.
• Permitting investigation of otherwise impractical (or unethical) situations.

Case studies allow a researcher to investigate a topic in far more detail than might be possible if they were trying to deal with a large number of research participants (nomothetic approach) with the aim of ‘averaging’.

Because of their in-depth, multi-sided approach, case studies often shed light on aspects of human thinking and behavior that would be unethical or impractical to study in other ways.

Research that only looks into the measurable aspects of human behavior is not likely to give us insights into the subjective dimension of experience, which is important to psychoanalytic and humanistic psychologists.

Case studies are often used in exploratory research. They can help us generate new ideas (that might be tested by other methods). They are an important way of illustrating theories and can help show how different aspects of a person’s life are related to each other.

The method is, therefore, important for psychologists who adopt a holistic point of view (i.e., humanistic psychologists ).

Limitations

• Lacking scientific rigor and providing little basis for generalization of results to the wider population.
• Researchers’ own subjective feelings may influence the case study (researcher bias).
• Difficult to replicate.
• Time-consuming and expensive.
• The volume of data, together with the time restrictions in place, impacted the depth of analysis that was possible within the available resources.

Because a case study deals with only one person/event/group, we can never be sure if the case study investigated is representative of the wider body of “similar” instances. This means the conclusions drawn from a particular case may not be transferable to other settings.

Because case studies are based on the analysis of qualitative (i.e., descriptive) data , a lot depends on the psychologist’s interpretation of the information she has acquired.

This means that there is a lot of scope for Anna O , and it could be that the subjective opinions of the psychologist intrude in the assessment of what the data means.

For example, Freud has been criticized for producing case studies in which the information was sometimes distorted to fit particular behavioral theories (e.g., Little Hans ).

This is also true of Money’s interpretation of the Bruce/Brenda case study (Diamond, 1997) when he ignored evidence that went against his theory.

Breuer, J., & Freud, S. (1895).  Studies on hysteria . Standard Edition 2: London.

Curtiss, S. (1981). Genie: The case of a modern wild child .

Diamond, M., & Sigmundson, K. (1997). Sex Reassignment at Birth: Long-term Review and Clinical Implications. Archives of Pediatrics & Adolescent Medicine , 151(3), 298-304

Freud, S. (1909a). Analysis of a phobia of a five year old boy. In The Pelican Freud Library (1977), Vol 8, Case Histories 1, pages 169-306

Freud, S. (1909b). Bemerkungen über einen Fall von Zwangsneurose (Der “Rattenmann”). Jb. psychoanal. psychopathol. Forsch ., I, p. 357-421; GW, VII, p. 379-463; Notes upon a case of obsessional neurosis, SE , 10: 151-318.

Harlow J. M. (1848). Passage of an iron rod through the head.  Boston Medical and Surgical Journal, 39 , 389–393.

Harlow, J. M. (1868).  Recovery from the Passage of an Iron Bar through the Head .  Publications of the Massachusetts Medical Society. 2  (3), 327-347.

Money, J., & Ehrhardt, A. A. (1972).  Man & Woman, Boy & Girl : The Differentiation and Dimorphism of Gender Identity from Conception to Maturity. Baltimore, Maryland: Johns Hopkins University Press.

Money, J., & Tucker, P. (1975). Sexual signatures: On being a man or a woman.

Further Information

• Case Study Approach
• Case Study Method
• Enhancing the Quality of Case Studies in Health Services Research
• “We do things together” A case study of “couplehood” in dementia
• Using mixed methods for evaluating an integrative approach to cancer care: a case study

• Privacy Policy

Home » Case Study – Methods, Examples and Guide

Case Study – Methods, Examples and Guide

Table of Contents

A case study is a research method that involves an in-depth examination and analysis of a particular phenomenon or case, such as an individual, organization, community, event, or situation.

It is a qualitative research approach that aims to provide a detailed and comprehensive understanding of the case being studied. Case studies typically involve multiple sources of data, including interviews, observations, documents, and artifacts, which are analyzed using various techniques, such as content analysis, thematic analysis, and grounded theory. The findings of a case study are often used to develop theories, inform policy or practice, or generate new research questions.

Types of Case Study

Types and Methods of Case Study are as follows:

Single-Case Study

A single-case study is an in-depth analysis of a single case. This type of case study is useful when the researcher wants to understand a specific phenomenon in detail.

For Example , A researcher might conduct a single-case study on a particular individual to understand their experiences with a particular health condition or a specific organization to explore their management practices. The researcher collects data from multiple sources, such as interviews, observations, and documents, and uses various techniques to analyze the data, such as content analysis or thematic analysis. The findings of a single-case study are often used to generate new research questions, develop theories, or inform policy or practice.

Multiple-Case Study

A multiple-case study involves the analysis of several cases that are similar in nature. This type of case study is useful when the researcher wants to identify similarities and differences between the cases.

For Example, a researcher might conduct a multiple-case study on several companies to explore the factors that contribute to their success or failure. The researcher collects data from each case, compares and contrasts the findings, and uses various techniques to analyze the data, such as comparative analysis or pattern-matching. The findings of a multiple-case study can be used to develop theories, inform policy or practice, or generate new research questions.

Exploratory Case Study

An exploratory case study is used to explore a new or understudied phenomenon. This type of case study is useful when the researcher wants to generate hypotheses or theories about the phenomenon.

For Example, a researcher might conduct an exploratory case study on a new technology to understand its potential impact on society. The researcher collects data from multiple sources, such as interviews, observations, and documents, and uses various techniques to analyze the data, such as grounded theory or content analysis. The findings of an exploratory case study can be used to generate new research questions, develop theories, or inform policy or practice.

Descriptive Case Study

A descriptive case study is used to describe a particular phenomenon in detail. This type of case study is useful when the researcher wants to provide a comprehensive account of the phenomenon.

For Example, a researcher might conduct a descriptive case study on a particular community to understand its social and economic characteristics. The researcher collects data from multiple sources, such as interviews, observations, and documents, and uses various techniques to analyze the data, such as content analysis or thematic analysis. The findings of a descriptive case study can be used to inform policy or practice or generate new research questions.

Instrumental Case Study

An instrumental case study is used to understand a particular phenomenon that is instrumental in achieving a particular goal. This type of case study is useful when the researcher wants to understand the role of the phenomenon in achieving the goal.

For Example, a researcher might conduct an instrumental case study on a particular policy to understand its impact on achieving a particular goal, such as reducing poverty. The researcher collects data from multiple sources, such as interviews, observations, and documents, and uses various techniques to analyze the data, such as content analysis or thematic analysis. The findings of an instrumental case study can be used to inform policy or practice or generate new research questions.

Case Study Data Collection Methods

Here are some common data collection methods for case studies:

Interviews involve asking questions to individuals who have knowledge or experience relevant to the case study. Interviews can be structured (where the same questions are asked to all participants) or unstructured (where the interviewer follows up on the responses with further questions). Interviews can be conducted in person, over the phone, or through video conferencing.

Observations

Observations involve watching and recording the behavior and activities of individuals or groups relevant to the case study. Observations can be participant (where the researcher actively participates in the activities) or non-participant (where the researcher observes from a distance). Observations can be recorded using notes, audio or video recordings, or photographs.

Documents can be used as a source of information for case studies. Documents can include reports, memos, emails, letters, and other written materials related to the case study. Documents can be collected from the case study participants or from public sources.

Surveys involve asking a set of questions to a sample of individuals relevant to the case study. Surveys can be administered in person, over the phone, through mail or email, or online. Surveys can be used to gather information on attitudes, opinions, or behaviors related to the case study.

Artifacts are physical objects relevant to the case study. Artifacts can include tools, equipment, products, or other objects that provide insights into the case study phenomenon.

How to conduct Case Study Research

Conducting a case study research involves several steps that need to be followed to ensure the quality and rigor of the study. Here are the steps to conduct case study research:

• Define the research questions: The first step in conducting a case study research is to define the research questions. The research questions should be specific, measurable, and relevant to the case study phenomenon under investigation.
• Select the case: The next step is to select the case or cases to be studied. The case should be relevant to the research questions and should provide rich and diverse data that can be used to answer the research questions.
• Collect data: Data can be collected using various methods, such as interviews, observations, documents, surveys, and artifacts. The data collection method should be selected based on the research questions and the nature of the case study phenomenon.
• Analyze the data: The data collected from the case study should be analyzed using various techniques, such as content analysis, thematic analysis, or grounded theory. The analysis should be guided by the research questions and should aim to provide insights and conclusions relevant to the research questions.
• Draw conclusions: The conclusions drawn from the case study should be based on the data analysis and should be relevant to the research questions. The conclusions should be supported by evidence and should be clearly stated.
• Validate the findings: The findings of the case study should be validated by reviewing the data and the analysis with participants or other experts in the field. This helps to ensure the validity and reliability of the findings.
• Write the report: The final step is to write the report of the case study research. The report should provide a clear description of the case study phenomenon, the research questions, the data collection methods, the data analysis, the findings, and the conclusions. The report should be written in a clear and concise manner and should follow the guidelines for academic writing.

Examples of Case Study

Here are some examples of case study research:

• The Hawthorne Studies : Conducted between 1924 and 1932, the Hawthorne Studies were a series of case studies conducted by Elton Mayo and his colleagues to examine the impact of work environment on employee productivity. The studies were conducted at the Hawthorne Works plant of the Western Electric Company in Chicago and included interviews, observations, and experiments.
• The Stanford Prison Experiment: Conducted in 1971, the Stanford Prison Experiment was a case study conducted by Philip Zimbardo to examine the psychological effects of power and authority. The study involved simulating a prison environment and assigning participants to the role of guards or prisoners. The study was controversial due to the ethical issues it raised.
• The Challenger Disaster: The Challenger Disaster was a case study conducted to examine the causes of the Space Shuttle Challenger explosion in 1986. The study included interviews, observations, and analysis of data to identify the technical, organizational, and cultural factors that contributed to the disaster.
• The Enron Scandal: The Enron Scandal was a case study conducted to examine the causes of the Enron Corporation’s bankruptcy in 2001. The study included interviews, analysis of financial data, and review of documents to identify the accounting practices, corporate culture, and ethical issues that led to the company’s downfall.
• The Fukushima Nuclear Disaster : The Fukushima Nuclear Disaster was a case study conducted to examine the causes of the nuclear accident that occurred at the Fukushima Daiichi Nuclear Power Plant in Japan in 2011. The study included interviews, analysis of data, and review of documents to identify the technical, organizational, and cultural factors that contributed to the disaster.

Application of Case Study

Case studies have a wide range of applications across various fields and industries. Here are some examples:

Business and Management

Case studies are widely used in business and management to examine real-life situations and develop problem-solving skills. Case studies can help students and professionals to develop a deep understanding of business concepts, theories, and best practices.

Case studies are used in healthcare to examine patient care, treatment options, and outcomes. Case studies can help healthcare professionals to develop critical thinking skills, diagnose complex medical conditions, and develop effective treatment plans.

Case studies are used in education to examine teaching and learning practices. Case studies can help educators to develop effective teaching strategies, evaluate student progress, and identify areas for improvement.

Social Sciences

Case studies are widely used in social sciences to examine human behavior, social phenomena, and cultural practices. Case studies can help researchers to develop theories, test hypotheses, and gain insights into complex social issues.

Law and Ethics

Case studies are used in law and ethics to examine legal and ethical dilemmas. Case studies can help lawyers, policymakers, and ethical professionals to develop critical thinking skills, analyze complex cases, and make informed decisions.

Purpose of Case Study

The purpose of a case study is to provide a detailed analysis of a specific phenomenon, issue, or problem in its real-life context. A case study is a qualitative research method that involves the in-depth exploration and analysis of a particular case, which can be an individual, group, organization, event, or community.

The primary purpose of a case study is to generate a comprehensive and nuanced understanding of the case, including its history, context, and dynamics. Case studies can help researchers to identify and examine the underlying factors, processes, and mechanisms that contribute to the case and its outcomes. This can help to develop a more accurate and detailed understanding of the case, which can inform future research, practice, or policy.

Case studies can also serve other purposes, including:

• Illustrating a theory or concept: Case studies can be used to illustrate and explain theoretical concepts and frameworks, providing concrete examples of how they can be applied in real-life situations.
• Developing hypotheses: Case studies can help to generate hypotheses about the causal relationships between different factors and outcomes, which can be tested through further research.
• Providing insight into complex issues: Case studies can provide insights into complex and multifaceted issues, which may be difficult to understand through other research methods.
• Informing practice or policy: Case studies can be used to inform practice or policy by identifying best practices, lessons learned, or areas for improvement.

Advantages of Case Study Research

There are several advantages of case study research, including:

• In-depth exploration: Case study research allows for a detailed exploration and analysis of a specific phenomenon, issue, or problem in its real-life context. This can provide a comprehensive understanding of the case and its dynamics, which may not be possible through other research methods.
• Rich data: Case study research can generate rich and detailed data, including qualitative data such as interviews, observations, and documents. This can provide a nuanced understanding of the case and its complexity.
• Holistic perspective: Case study research allows for a holistic perspective of the case, taking into account the various factors, processes, and mechanisms that contribute to the case and its outcomes. This can help to develop a more accurate and comprehensive understanding of the case.
• Theory development: Case study research can help to develop and refine theories and concepts by providing empirical evidence and concrete examples of how they can be applied in real-life situations.
• Practical application: Case study research can inform practice or policy by identifying best practices, lessons learned, or areas for improvement.
• Contextualization: Case study research takes into account the specific context in which the case is situated, which can help to understand how the case is influenced by the social, cultural, and historical factors of its environment.

Limitations of Case Study Research

There are several limitations of case study research, including:

• Limited generalizability : Case studies are typically focused on a single case or a small number of cases, which limits the generalizability of the findings. The unique characteristics of the case may not be applicable to other contexts or populations, which may limit the external validity of the research.
• Biased sampling: Case studies may rely on purposive or convenience sampling, which can introduce bias into the sample selection process. This may limit the representativeness of the sample and the generalizability of the findings.
• Subjectivity: Case studies rely on the interpretation of the researcher, which can introduce subjectivity into the analysis. The researcher’s own biases, assumptions, and perspectives may influence the findings, which may limit the objectivity of the research.
• Limited control: Case studies are typically conducted in naturalistic settings, which limits the control that the researcher has over the environment and the variables being studied. This may limit the ability to establish causal relationships between variables.
• Time-consuming: Case studies can be time-consuming to conduct, as they typically involve a detailed exploration and analysis of a specific case. This may limit the feasibility of conducting multiple case studies or conducting case studies in a timely manner.
• Resource-intensive: Case studies may require significant resources, including time, funding, and expertise. This may limit the ability of researchers to conduct case studies in resource-constrained settings.

About the author

Muhammad Hassan

Researcher, Academic Writer, Web developer

You may also like

Survey Research – Types, Methods, Examples

Experimental Design – Types, Methods, Guide

Qualitative Research Methods

Quantitative Research – Methods, Types and...

Questionnaire – Definition, Types, and Examples

Mixed Methods Research – Types & Analysis

• Bipolar Disorder
• Therapy Center
• When To See a Therapist
• Types of Therapy
• Best Online Therapy
• Best Couples Therapy
• Best Family Therapy
• Managing Stress
• Sleep and Dreaming
• Understanding Emotions
• Self-Improvement
• Healthy Relationships
• Student Resources
• Personality Types
• Sweepstakes
• Guided Meditations
• Verywell Mind Insights
• 2024 Verywell Mind 25
• Mental Health in the Classroom
• Editorial Process
• Meet Our Review Board
• Crisis Support

What Is a Case Study?

Weighing the pros and cons of this method of research

Verywell / Colleen Tighe

• Pros and Cons

What Types of Case Studies Are Out There?

Where do you find data for a case study, how do i write a psychology case study.

A case study is an in-depth study of one person, group, or event. In a case study, nearly every aspect of the subject's life and history is analyzed to seek patterns and causes of behavior. Case studies can be used in many different fields, including psychology, medicine, education, anthropology, political science, and social work.

The point of a case study is to learn as much as possible about an individual or group so that the information can be generalized to many others. Unfortunately, case studies tend to be highly subjective, and it is sometimes difficult to generalize results to a larger population.

While case studies focus on a single individual or group, they follow a format similar to other types of psychology writing. If you are writing a case study, we got you—here are some rules of APA format to reference.  

At a Glance

A case study, or an in-depth study of a person, group, or event, can be a useful research tool when used wisely. In many cases, case studies are best used in situations where it would be difficult or impossible for you to conduct an experiment. They are helpful for looking at unique situations and allow researchers to gather a lot of˜ information about a specific individual or group of people. However, it's important to be cautious of any bias we draw from them as they are highly subjective.

What Are the Benefits and Limitations of Case Studies?

A case study can have its strengths and weaknesses. Researchers must consider these pros and cons before deciding if this type of study is appropriate for their needs.

One of the greatest advantages of a case study is that it allows researchers to investigate things that are often difficult or impossible to replicate in a lab. Some other benefits of a case study:

• Allows researchers to capture information on the 'how,' 'what,' and 'why,' of something that's implemented
• Gives researchers the chance to collect information on why one strategy might be chosen over another
• Permits researchers to develop hypotheses that can be explored in experimental research

On the other hand, a case study can have some drawbacks:

• It cannot necessarily be generalized to the larger population
• Cannot demonstrate cause and effect
• It may not be scientifically rigorous
• It can lead to bias

Researchers may choose to perform a case study if they want to explore a unique or recently discovered phenomenon. Through their insights, researchers develop additional ideas and study questions that might be explored in future studies.

It's important to remember that the insights from case studies cannot be used to determine cause-and-effect relationships between variables. However, case studies may be used to develop hypotheses that can then be addressed in experimental research.

Case Study Examples

There have been a number of notable case studies in the history of psychology. Much of  Freud's work and theories were developed through individual case studies. Some great examples of case studies in psychology include:

• Anna O : Anna O. was a pseudonym of a woman named Bertha Pappenheim, a patient of a physician named Josef Breuer. While she was never a patient of Freud's, Freud and Breuer discussed her case extensively. The woman was experiencing symptoms of a condition that was then known as hysteria and found that talking about her problems helped relieve her symptoms. Her case played an important part in the development of talk therapy as an approach to mental health treatment.
• Phineas Gage : Phineas Gage was a railroad employee who experienced a terrible accident in which an explosion sent a metal rod through his skull, damaging important portions of his brain. Gage recovered from his accident but was left with serious changes in both personality and behavior.
• Genie : Genie was a young girl subjected to horrific abuse and isolation. The case study of Genie allowed researchers to study whether language learning was possible, even after missing critical periods for language development. Her case also served as an example of how scientific research may interfere with treatment and lead to further abuse of vulnerable individuals.

Such cases demonstrate how case research can be used to study things that researchers could not replicate in experimental settings. In Genie's case, her horrific abuse denied her the opportunity to learn a language at critical points in her development.

This is clearly not something researchers could ethically replicate, but conducting a case study on Genie allowed researchers to study phenomena that are otherwise impossible to reproduce.

There are a few different types of case studies that psychologists and other researchers might use:

• Collective case studies : These involve studying a group of individuals. Researchers might study a group of people in a certain setting or look at an entire community. For example, psychologists might explore how access to resources in a community has affected the collective mental well-being of those who live there.
• Descriptive case studies : These involve starting with a descriptive theory. The subjects are then observed, and the information gathered is compared to the pre-existing theory.
• Explanatory case studies : These   are often used to do causal investigations. In other words, researchers are interested in looking at factors that may have caused certain things to occur.
• Exploratory case studies : These are sometimes used as a prelude to further, more in-depth research. This allows researchers to gather more information before developing their research questions and hypotheses .
• Instrumental case studies : These occur when the individual or group allows researchers to understand more than what is initially obvious to observers.
• Intrinsic case studies : This type of case study is when the researcher has a personal interest in the case. Jean Piaget's observations of his own children are good examples of how an intrinsic case study can contribute to the development of a psychological theory.

The three main case study types often used are intrinsic, instrumental, and collective. Intrinsic case studies are useful for learning about unique cases. Instrumental case studies help look at an individual to learn more about a broader issue. A collective case study can be useful for looking at several cases simultaneously.

The type of case study that psychology researchers use depends on the unique characteristics of the situation and the case itself.

There are a number of different sources and methods that researchers can use to gather information about an individual or group. Six major sources that have been identified by researchers are:

• Archival records : Census records, survey records, and name lists are examples of archival records.
• Direct observation : This strategy involves observing the subject, often in a natural setting . While an individual observer is sometimes used, it is more common to utilize a group of observers.
• Documents : Letters, newspaper articles, administrative records, etc., are the types of documents often used as sources.
• Interviews : Interviews are one of the most important methods for gathering information in case studies. An interview can involve structured survey questions or more open-ended questions.
• Participant observation : When the researcher serves as a participant in events and observes the actions and outcomes, it is called participant observation.
• Physical artifacts : Tools, objects, instruments, and other artifacts are often observed during a direct observation of the subject.

If you have been directed to write a case study for a psychology course, be sure to check with your instructor for any specific guidelines you need to follow. If you are writing your case study for a professional publication, check with the publisher for their specific guidelines for submitting a case study.

Here is a general outline of what should be included in a case study.

Section 1: A Case History

This section will have the following structure and content:

Background information : The first section of your paper will present your client's background. Include factors such as age, gender, work, health status, family mental health history, family and social relationships, drug and alcohol history, life difficulties, goals, and coping skills and weaknesses.

Description of the presenting problem : In the next section of your case study, you will describe the problem or symptoms that the client presented with.

Describe any physical, emotional, or sensory symptoms reported by the client. Thoughts, feelings, and perceptions related to the symptoms should also be noted. Any screening or diagnostic assessments that are used should also be described in detail and all scores reported.

Your diagnosis : Provide your diagnosis and give the appropriate Diagnostic and Statistical Manual code. Explain how you reached your diagnosis, how the client's symptoms fit the diagnostic criteria for the disorder(s), or any possible difficulties in reaching a diagnosis.

Section 2: Treatment Plan

This portion of the paper will address the chosen treatment for the condition. This might also include the theoretical basis for the chosen treatment or any other evidence that might exist to support why this approach was chosen.

• Cognitive behavioral approach : Explain how a cognitive behavioral therapist would approach treatment. Offer background information on cognitive behavioral therapy and describe the treatment sessions, client response, and outcome of this type of treatment. Make note of any difficulties or successes encountered by your client during treatment.
• Humanistic approach : Describe a humanistic approach that could be used to treat your client, such as client-centered therapy . Provide information on the type of treatment you chose, the client's reaction to the treatment, and the end result of this approach. Explain why the treatment was successful or unsuccessful.
• Psychoanalytic approach : Describe how a psychoanalytic therapist would view the client's problem. Provide some background on the psychoanalytic approach and cite relevant references. Explain how psychoanalytic therapy would be used to treat the client, how the client would respond to therapy, and the effectiveness of this treatment approach.
• Pharmacological approach : If treatment primarily involves the use of medications, explain which medications were used and why. Provide background on the effectiveness of these medications and how monotherapy may compare with an approach that combines medications with therapy or other treatments.

This section of a case study should also include information about the treatment goals, process, and outcomes.

When you are writing a case study, you should also include a section where you discuss the case study itself, including the strengths and limitiations of the study. You should note how the findings of your case study might support previous research. 

In your discussion section, you should also describe some of the implications of your case study. What ideas or findings might require further exploration? How might researchers go about exploring some of these questions in additional studies?

Need More Tips?

Here are a few additional pointers to keep in mind when formatting your case study:

• Never refer to the subject of your case study as "the client." Instead, use their name or a pseudonym.
• Read examples of case studies to gain an idea about the style and format.
• Remember to use APA format when citing references .

Crowe S, Cresswell K, Robertson A, Huby G, Avery A, Sheikh A. The case study approach .  BMC Med Res Methodol . 2011;11:100.

Crowe S, Cresswell K, Robertson A, Huby G, Avery A, Sheikh A. The case study approach . BMC Med Res Methodol . 2011 Jun 27;11:100. doi:10.1186/1471-2288-11-100

Gagnon, Yves-Chantal.  The Case Study as Research Method: A Practical Handbook . Canada, Chicago Review Press Incorporated DBA Independent Pub Group, 2010.

Yin, Robert K. Case Study Research and Applications: Design and Methods . United States, SAGE Publications, 2017.

By Kendra Cherry, MSEd Kendra Cherry, MS, is a psychosocial rehabilitation specialist, psychology educator, and author of the "Everything Psychology Book."

• Policy & Compliance
• Clinical Trials

NIH Definition of Clinical Trial Case Studies

The case studies provided below are designed to help you identify whether your study would be considered by NIH to be a clinical trial. Expect the case studies and related guidance to evolve over the upcoming year. For continuity and ease of reference, case studies will retain their original numbering and will not be renumbered if cases are revised or removed.

The simplified case studies apply the following four questions to determine whether NIH would consider the research study to be a clinical trial:

• Does the study involve human participants?
• Are the participants prospectively assigned to an intervention?
• Is the study designed to evaluate the effect of the intervention on the participants?
• Is the effect being evaluated a health-related biomedical or behavioral outcome?

If the answer to all four questions is “yes,” then the clinical study would be considered a clinical trial according to the NIH definition.

See this page for more information about the NIH definition of a clinical trial.

General Case Studies

Institute or center specific case studies.

The study involves the recruitment of research participants who are randomized to receive one of two approved drugs. It is designed to compare the effects of the drugs on the blood level of a protein.

• Does the study involve human participants? Yes, the study involves human participants.
• Are the participants prospectively assigned to an intervention? Yes, the participants are prospectively assigned to receive an intervention, one of two drugs.
• Is the study designed to evaluate the effect of the intervention on the participants? Yes, the study is designed to evaluate the effect of the drugs on the level of the protein in the participants’ blood.
• Is the effect being evaluated a health-related biomedical or behavioral outcome? Yes, the effect being evaluated, the level of a protein, is a health-related biomedical outcome.

The study involves the recruitment of research participants with condition Y to receive a drug that has been approved for another indication. It is designed to measure the drug’s effects on the level of a biomarker associated with the severity of condition Y.

• Are the participants prospectively assigned to an intervention? Yes, the participants are prospectively assigned to receive an intervention, the approved drug.
• Is the study designed to evaluate the effect of the intervention on the participants? Yes, the study is designed to evaluate the drug’s effect on the level of the biomarker.
• Is the effect being evaluated a health-related biomedical or behavioral outcome? Yes, the effect being evaluated, the level of a biomarker, is a health-related biomedical outcome.

The study involves the recruitment of research participants with condition X to receive investigational compound A. It is designed to assess the pharmacokinetic properties of compound A.

• Are the participants prospectively assigned to an intervention? Yes, the participants are prospectively assigned to receive an intervention, compound A.
• Is the study designed to evaluate the effect of the intervention on the participants? Yes, the study is designed to evaluate how the body interacts with compound A
• Is the effect being evaluated a health-related biomedical or behavioral outcome? Yes, the effect being evaluated, pharmacokinetic properties, is a health-related biomedical outcome.

The study involves the recruitment of research participants with disease X to receive an investigational drug. It is designed to assess safety and determine the maximum tolerated dose of the drug.

• Are the participants prospectively assigned to an intervention? Yes, the participants are prospectively assigned to receive an intervention, the investigational drug.
• Is the study designed to evaluate the effect of the intervention on the participants? Yes, the study is designed to assess safety and determine the maximum tolerated dose of the investigational drug.
• Is the effect being evaluated a health-related biomedical or behavioral outcome? Yes, the effect being evaluated, safety and maximum tolerated dose, is a health-related biomedical outcome.

The study involves the recruitment of research participants with disease X to receive a chronic disease management program. It is designed to assess usability and to determine the maximum tolerated dose of the chronic disease program (e.g., how many in-person and telemedicine visits with adequate adherence).

• Are the participants prospectively assigned to an intervention? Yes, the participants are prospectively assigned to receive an intervention, the chronic disease management program.
• Is the study designed to evaluate the effect of the intervention on the participants? Yes, the study is designed to determine the maximum tolerated dose of the program to obtain adequate adherence.
• Is the effect being evaluated a health-related biomedical or behavioral outcome? Yes, the effect being evaluated, tolerable intensity and adequate adherence of the intervention, is a health-related outcome.

The study involves the recruitment of research participants with disease X to receive either an investigational drug or a placebo. It is designed to evaluate the efficacy of the investigational drug to relieve disease symptoms.

• Are the participants prospectively assigned to an intervention? Yes, the participants are prospectively assigned to receive an intervention, the investigational drug or placebo.
• Is the study designed to evaluate the effect of the intervention on the participants? Yes, the study is designed to evaluate the effect of the investigational drug on the participants’ symptoms.
• Is the effect being evaluated a health-related biomedical or behavioral outcome? Yes, the effect being evaluated, relief of symptoms, is a health-related outcome.

The study involves the recruitment of research participants with disease X to receive an investigational drug. It is designed to assess whether there is a change in disease progression compared to baseline. There is no concurrent control used in this study.

• Is the study designed to evaluate the effect of the intervention on the participants? Yes, the study is designed to evaluate the effect of the investigational drug on the subject’s disease progression.
• Is the effect being evaluated a health-related biomedical or behavioral outcome? Yes, the effect being evaluated, disease progression, is a health-related outcome.

The study involves the recruitment of research participants with disease X to test an investigational in vitro diagnostic device (IVD). It is designed to evaluate the ability of the device to measure the level of an antibody in blood.

• Are the participants prospectively assigned to an intervention? No, in this context the IVD would not be considered an intervention. The IVD is being used to test its ability to measure antibody levels, but not to test its effects on any health-related biomedical or behavioral outcomes. 

The study involves the recruitment of research participants with disease X to be evaluated with an investigational in vitro diagnostic device (IVD). The study is designed to evaluate how knowledge of certain antibody levels impacts clinical management of disease.

• Are the participants prospectively assigned to an intervention? Yes, the participants are prospectively assigned to an intervention, measurement of an antibody level, with the idea that knowledge of that antibody level might affect clinical management.
• Is the study designed to evaluate the effect of the intervention on the participants? Yes, the study is designed to evaluate how knowledge of the level of an antibody might inform treatment.
• Is the effect being evaluated a health-related biomedical or behavioral outcome? Yes, the effect being measured, how blood antibody levels inform treatment, is a health-related outcome.

The study involves the recruitment of healthy volunteers who will be randomized to different durations of sleep deprivation (including no sleep deprivation as a control) and who will have stress hormone levels measured. It is designed to determine whether the levels of stress hormones in blood rise in response to different durations of sleep deprivation.

• Does the study involve human participants? Yes, the healthy volunteers are human participants.
• Are the participants prospectively assigned to an intervention? Yes, the participants are prospectively assigned to an intervention, different durations of sleep deprivation followed by a blood draw.
• Is the study designed to evaluate the effect of the intervention on the participants? Yes, the study is designed to measure the effect of different durations of sleep deprivation on stress hormone levels.
• Is the effect being evaluated a health-related biomedical or behavioral outcome? Yes, the effect being evaluated, stress hormone levels, is a health-related biomedical outcome.

The study involves the analysis of de-identified, stored blood samples and de-identified medical records of patients with disease X who were treated with an approved drug. The study is designed to evaluate the level of a protein in the blood of patients that is associated with therapeutic effects of the drug.

• Does the study involve human participants? No, the study does not involve human participants because only de-identified samples and information are used.

The study involves the analysis of identifiable, stored blood samples and identified medical records of patients with disease X who were treated with an approved drug. The study is designed to evaluate the level of a protein in the blood of patients that is associated with therapeutic effects of the drug.

• Does the study involve human participants? Yes, patients are human participants because the blood and information are identifiable.
• Are the participants prospectively assigned to an intervention? No, secondary research with biospecimens or health information is not a clinical trial.

The study involves the recruitment of a healthy volunteers whose blood is drawn for genomic analysis. It is designed to identify the prevalence of a genetic mutation in the cohort and evaluate potential association between the presence of the mutation and the risk of developing a genetic disorder.

• Are the participants prospectively assigned to an intervention? No, sample collection (blood draw) is not an intervention in this context.

Physicians report that some patients being treated with drug A for disease X are also experiencing some improvement in a second condition, condition Y. The study involves the recruitment of research participants who have disease X and condition Y and are being treated with drug A. The participants are surveyed to ascertain whether they are experiencing an improvement in condition Y.

• Are the participants prospectively assigned to an intervention? No, participants are not prospectively assigned to receive an intervention as they are receiving drugs as part of their clinical care. The surveys are being used for measurement, not to modify a biomedical or behavioral outcome.

The study involves the recruitment of patients with disease X who are receiving one of three standard therapies as part of their clinical care. It is designed to assess the relative effectiveness of the three therapies by monitoring survival rates using medical records over a few years.

• Are the participants prospectively assigned to an intervention? No, there is no intervention. The therapies are prescribed as part of clinical care; they are not prospectively assigned for the purpose of the study. The study is observational.

The study involves the recruitment of research participants with disease X vs. healthy controls and comparing these participants on a range of health processes and outcomes including genomics, biospecimens, self-report measures, etc. to explore differences that may be relevant to the development of disease X.

• Are the participants prospectively assigned to an intervention? No, the measures needed to assess the outcomes are not interventions in this context, as the study is not intended to determine whether the measures modify a health-related biomedical or behavioral outcome.

The study involves the recruitment of healthy volunteers for a respiratory challenge study; participants are randomized to receive different combinations of allergens. The study evaluates the severity and mechanism of the immune response to different combinations of allergens introduced via inhalation.

• Does the study involve human participants? Yes, healthy volunteers are human participants.
• Are the participants prospectively assigned to an intervention? Yes, healthy volunteers are prospectively assigned to randomly selected combinations of allergens.
• Is the study designed to evaluate the effect of the intervention on the participants? Yes, the study is evaluating the effects of different combinations of allergens on the immune response in healthy individuals.
• Is the effect being evaluated a health-related biomedical or behavioral outcome? Yes, the study evaluates the severity and mechanism of the immune reaction to allergens, which are health-related biomedical outcomes.

The study involves the recruitment of research participants with Alzheimer’s disease (AD) to evaluate the effects of an investigational drug on memory, and retention and recall of information.

• Are the participants prospectively assigned to an intervention? Yes, participants are prospectively assigned to receive the investigational drug.
• Is the study designed to evaluate the effect of the intervention on the participants? Yes, the study is evaluating the effects of the drug on participants’ memory.
• Is the effect being evaluated a health-related biomedical or behavioral outcome? Yes, the study evaluates memory, and retention and recall of information in the context of AD.

The study involves the recruitment of individuals to receive a new behavioral intervention for sedentary behavior. It is designed to measure the effect of the intervention on hypothesized differential mediators of behavior change.

• Are the participants prospectively assigned to an intervention? Yes, participants are prospectively assigned to receive a behavioral intervention.
• Is the study designed to evaluate the effect of the intervention on the participants? Yes, the study is evaluating the effects of the intervetion on mediators of behavior change.
• Is the effect being evaluated a health-related biomedical or behavioral outcome? Yes, the effect being evaluated, mediators of behavior change, are behavioral outcomes relevant to health.

The study involves the recruitment of patients with disease X to be evaluated with a new visual acuity task. It is designed to evaluate the ability of the new task to measure visual acuity as compared with the gold standard Snellen Test

• Are the participants prospectively assigned to an intervention? Yes, the participants are prospectively assigned to an intervention, the new visual acuity test.
• Is the study designed to evaluate the effect of the intervention on the participants? No, the study is designed to evaluate the ability of the new visual acuity test to measure visual acuity as compared to the gold standard Snellen Test, but not to modify visual acuity.

The study involves the recruitment of research participants with CHF who were hospitalized before or after implementation of the Medicare incentives to reduce re-hospitalizations. Morbidity, mortality, and quality of life of these participants are evaluated to compare the effects of these Medicare incentives on these outcomes.

• Are the participants prospectively assigned to an intervention? No, the intervention (incentives to reduce re-hospitalization) were assigned by Medicare, not by the research study.

The study involves the recruitment of healthcare providers to assess the extent to which being provided with genomic sequence information about their patients informs their treatment of those patients towards improved outcomes.

• Does the study involve human participants? Yes, both the physicians and the patients are human participants.
• Are the participants prospectively assigned to an intervention? Yes, physicians are prospectively assigned to receive genomic sequence information, which is the intervention.
• Is the study designed to evaluate the effect of the intervention on the participants? Yes, the study is designed to evaluate the effect of intervening with physicians, on the treatment they provide to their patients.
• Is the effect being evaluated a health-related, biomedical, or behavioral outcome? Yes, the effect being evaluated, the extent to which providing specific information to physicians informs the treatment of patients, is a health-related outcome.

The study involves the recruitment of research participants with a behavioral condition to receive either an investigational behavioral intervention or a behavioral intervention in clinical use. It is designed to evaluate the effectiveness of the investigational intervention compared to the intervention in clinical use in reducing the severity of the obsessive compulsive disorder.

• Are the participants prospectively assigned to an intervention? Yes, the participants are prospectively assigned to an intervention, either the investigational intervention or an intervention in clinical use.
• Is the study designed to evaluate the effect of the intervention on the participants? Yes, the study is designed to evaluate whether the investigational intervention is as effective as the standard intervention, at changing behavior.
• Is the effect being evaluated a health-related, biomedical, or behavioral outcome? Yes, the effect being evaluated, the interventions’ effectiveness in reducing the severity of the condition, is a health-related behavioral outcome.

The study involves the recruitment of physicians who will be randomly assigned to use a new app or an existing app, which cues directed interviewing techniques. The study is designed to determine whether the new app is better than the existing app at assisting physicians in identifying families in need of social service support. The number of community service referrals will be measured.

• Does the study involve human participants? Yes, both the physicians and the families are human participants.
• Are the participants prospectively assigned to an intervention? Yes, physicians are prospectively assigned to use one of two apps, which are the interventions.
• Is the study designed to evaluate the effect of the intervention on the participants? Yes, the study is designed to evaluate the effect of intervening with physicians, on social service support referral for families.
• Is the effect being evaluated a health-related, biomedical, or behavioral outcome? Yes, the effect being evaluated, the number of referrals, is a health-related outcome.

The study involves the recruitment of parents to participate in focus groups to discuss topics related to parental self-efficacy and positive parenting behaviors. It is designed to gather information needed to develop an intervention to promote parental self-efficacy and positive parenting behaviors.

• Does the study involve human participants? Yes, the parents are human participants.
• Are the participants prospectively assigned to an intervention? No, a focus group is not an intervention.

The study involves the recruitment of healthy volunteers to test a new behavioral intervention. It is designed to evaluate the effect of a meditation intervention on adherence to exercise regimens and quality of life to inform the design of a subsequent, fully-powered trial.

• Does the study involve human participants? Yes, study participants are human participants.
• Are the participants prospectively assigned to an intervention? Yes, the participants are prospectively assigned to a behavioral intervention.
• Is the study designed to evaluate the effect of the intervention on the participants? Yes, the study is designed to evaluate the effect of the intervention on adherence, and quality of life.
• Is the effect being evaluated a health-related biomedical or behavioral outcome? Yes, adherence and quality of life are health-related outcomes.

A study will test the feasibility a mobile phone app designed to increase physical activity. A group of sedentary individuals will use the app for a week while their interactions with the app are monitored. The number of interactions with the app will be measured, as well as any software issues. Participants will also complete a survey indicating their satisfaction with and willingness to use the app, as well as any feedback for improvement. The app’s effect on physical activity, weight, or cardiovascular fitness will not be evaluated.

• Does the study involve human participants? Yes, sedentary individuals will be enrolled.
• Are the participants prospectively assigned to an intervention? The participants will interact with the app for a week.
• Is the study designed to evaluate the effect of the intervention on the participants? No. While the participants’ interactions are monitored (steps or heart rate may be recorded in this process), the study is NOT measuring the effect of using the app ON the participant. The study is only measuring the usability and acceptability of the app, and testing for bugs in the software. The effect on physical activity is NOT being measured.
• Is the effect being evaluated a health-related biomedical or behavioral outcome? N/A

The study involves the recruitment of healthy family members of patients hospitalized for disease X to test two CPR training strategies. Participants will receive one of two training strategies. The outcome is improved CPR skills retention.

• Does the study involve human participants? Yes, family members of patients are human participants.
• Are the participants prospectively assigned to an intervention? Yes, the participants are prospectively assigned to one of two CPR educational strategies.
• Is the study designed to evaluate the effect of the intervention on the participants? Yes, the study is designed to evaluate the effect of educational strategies on CPR skills.
• Is the effect being evaluated a health-related biomedical or behavioral outcome? Yes, retention of CPR skills is a health-related behavioral outcome.

The study involves the recruitment of research participants in three different communities (clusters) to test three CPR training strategies. The rate of out-of- hospital cardiac arrest survival will be compared.

• Are the participants prospectively assigned to an intervention? Yes, the participants are prospectively assigned to receive one of three types of CPR training, which is the intervention.
• Is the study designed to evaluate the effect of the intervention on the participants? Yes, the study is designed to evaluate the effect of different CPR training strategies on patient survival rates post cardiac arrest.
• Is the effect being evaluated a health-related biomedical or behavioral outcome? Yes, out-of-hospital cardiac arrest survival is a health-related outcome.

A study involves the recruitment of school children to evaluate two different tools for monitoring food intake. Food consumption behavior will be measured by asking children to activate a pocket camera during meals and to use a diary to record consumed food. The accuracy of the two food monitoring methods in measuring energy intake will be assessed.

• Does the study involve human participants? Yes, children are human participants.
• Are the participants prospectively assigned to an intervention? No, in this context the monitoring methods would not be considered an intervention. The study is designed to test the accuracy of two monitoring methods, but not to test the effect on any health-related biomedical or behavioral outcomes. 

A study involves the recruitment of school children to evaluate two different tools for monitoring food intake. Food consumption behavior will be measured by asking children to activate a pocket camera during meals and to use a diary to record consumed food. Changes to eating behavior will be assessed.

• Are the participants prospectively assigned to an intervention? Yes, the participants are prospectively assigned to two food monitoring methods.
• Is the study designed to evaluate the effect of the intervention on the participants? Yes, the study is designed to determine whether using the monitoring methods changes eating behavior.
• Is the effect being evaluated a health-related biomedical or behavioral outcome? Yes, eating behavior is a health-related outcome.

A study involves the recruitment of children at two schools to monitor eating behavior. Children’s food choices will be monitored using a remote food photography method. Food consumption and the accuracy of food monitoring methods will be assessed.

• Does the study involve human participants? Yes, the children participating in this study are human participants.
• Are the participants prospectively assigned to an intervention? No, not in this context. The study involves observing and measuring eating behavior, but not modifying it. This is an observational study.

A study involves the recruitment of children at two schools to evaluate their preferences for graphics and colors used in healthy food advertisements. Children will be presented with multiple health advertisements and their preferences for graphics and colors will be assessed.

• Are the participants prospectively assigned to an intervention? Yes, the participants are prospectively assigned to see different advertisements.
• Is the study designed to evaluate the effect of the intervention on the participants? Yes, the study is designed to evaluate the advertisements.
• Is the effect being evaluated a health-related biomedical or behavioral outcome? No, preferences are not health-related biomedical or behavioral outcomes.

The study involves ambulatory patients who have new-onset stable angina and who are recruited from community practices. They are randomized to undergo CT angiography or an exercise stress test of the doctor’s choice. To keep the trial pragmatic, the investigators do not prescribe a protocol for how physicians should respond to test results. The study is designed to determine whether the initial test (CT angiography or stress test) affects long-term rates of premature death, stroke, or myocardial infarctions.

• Are the participants prospectively assigned to an intervention? Yes, the participants are randomized to undergo CT angiography or an exercise stress test.
• Is the study designed to evaluate the effect of the intervention on the participants? Yes, the study is designed to determine whether the initial test done affects long-term rates of certain clinical events.
• Is the effect being evaluated a health-related biomedical or behavioral outcome? Yes, premature death, stroke, and myocardial infarction are health-related biomedical outcomes.

The study involves patients who present with stable angina to community practices. As part of their routine care some of their physicians refer them for CT angiography, while others refer them for exercise stress tests. The study is designed to see whether or not there's an association between the type of test that is chosen and long-term risk of death, stroke, or myocardial infarction.

• Are the participants prospectively assigned to an intervention? No, the intervention is not prospectively assigned by the investigators. Rather, the intervention, in this case diagnostic study, occurs as part of routine clinical care.

The investigators conduct a longitudinal study of patients with schizophrenia. Their physicians, as part of their standard clinical care, prescribe antipsychotic medication. The investigators conduct an imaging session before starting treatment; they repeat imaging 4-6 weeks later.

• Does the study involve human participants? Yes.
• Are the participants prospectively assigned to an intervention? No, not in this context.  Antipsychotic medications are given as part of clinical care, not as part of a prospective, approved research protocol.  

The investigators conduct a longitudinal study of patients with schizophrenia. Their physicians, as part of their standard clinical care, prescribe antipsychotic medication. As part of the research protocol, all participants will be prescribed the same dose of the antipsychotic medication. The investigators conduct an imaging session before starting treatment; they repeat imaging 4-6 weeks later.

• Are the participants prospectively assigned to an intervention? Yes, although participants are all receiving antipsychotic medication as part of their standard medical care, the dose of the antipsychotic medication is determined by the research protocol, rather than individual clinical need.
• Is the study designed to evaluate the effect of the intervention on the participants?  Yes, the study is designed to evaluate the effect of a dose of antipsychotic medication on brain function.
• Is the effect being evaluated a health-related biomedical or behavioral outcome ? Yes, brain function measured by imaging is a health-related outcome.

The study involves recruitment of healthy volunteers who will wear a thermal compression device around their legs. This pilot study is designed to examine preliminary performance and safety of a thermal compression device worn during surgery. Investigators will measure core temperature, comfort, and presence of skin injury in 15-minute intervals.

• Are the participants prospectively assigned to an intervention? Yes, participants are assigned to wear a thermal compression device.
• Is the study designed to evaluate the effect of the intervention on the participants?  Yes, the study is designed to evaluate the effect of the thermal compression device on participant core temperature, comfort, and presence of skin injury.
• Is the effect being evaluated a health-related biomedical or behavioral outcome ? Yes, participant core temperature, comfort, and presence of skin injury are health-related biomedical outcomes.

The study involves collection of data on hospitalizations for various acute illnesses among people who live close to a border between two states that have recently implemented different laws related to public health (e.g. smoking regulations, soda taxes). The investigators want to take advantage of this “natural experiment” to assess the health impact of the laws.

• Does the study involve human participants?  Yes, the study involves human participants.
• Are the participants prospectively assigned to an intervention?  No, the interventions were assigned by state laws and state of residence, not by the research study.

The study involves recruitment of healthy volunteers to engage in working memory tasks while undergoing transcranial magnetic stimulation (TMS) to induce competing local neuronal activity. The study is measuring task performance to investigate the neural underpinnings of working memory storage and processing.

• Are the participants prospectively assigned to an intervention? Yes, healthy volunteers are prospectively assigned to receive TMS stimulation protocols during a working memory task.
• Is the study designed to evaluate the effect of the intervention on the participants? Yes, the study is evaluating the effects of local TMS stimulation on working memory performance and oscillatory brain activity in healthy individuals.
• Is the effect being evaluated a health-related biomedical or behavioral outcome? Yes, the study evaluates working memory processes, which are health-related biomedical outcomes.

The study involves recruitment of healthy volunteers to engage in a social valuation task while dopamine tone in the brain is manipulated using tolcapone, an FDA-approved medication. The study aims to understand the role of dopamine in social decision-making and to search for neural correlates of this valuation using fMRI.

• Are the participants prospectively assigned to an intervention? Yes, healthy volunteers are prospectively assigned to receive tolcapone during a social valuation task.
• Is the study designed to evaluate the effect of the intervention on the participants? Yes, the study is evaluating the effects of modulating dopamine tone on social decision-making. Although this study uses an FDA-approved drug to modulate dopamine tone, the goal of this intervention is to understand the role of dopamine in a fundamental phenomenon (social valuation), and not to study the mechanism of action of the drug or its clinical effects.

The career development candidate proposes to independently lead a study to test a new drug A on patients with disease X. Patients will be randomized to a test and control group, with the test group receiving one dose of drug A per week for 12 months and controls receiving placebo. To assess presence, number, and type of any polyps, a colonoscopy will be performed. To assess biomarkers of precancerous lesions, colon mucosal biopsies will be collected. Complete blood count will be measured, and plasma will be stored for potential biomarker evaluation.

• Are the participants prospectively assigned to an intervention? Yes, the participants are prospectively assigned to receive an intervention, drug A or placebo.
• Is the study designed to evaluate the effect of the intervention on the participants? Yes, the study is designed to evaluate the effect of drug A and placebo on the presence and type of polyps.
• Is the effect being evaluated a health-related biomedical or behavioral outcome? Yes, the effect being evaluated, the presence and type of polyps, is a health-related biomedical outcome.

Ancillary Study to Case Study #42a: Some types of drug A being evaluated in Case Study #42a have been reported to impact renal function. An internal medicine fellow performs an ancillary study where stored plasma from Case Study #42a will be evaluated for multiple biomarkers of renal function.

• Does the study involve human participants? Yes, patients are human participants because the plasma and information are identifiable.
• Are the participants prospectively assigned to an intervention? No, because the assignment of participants to an intervention occurs as part of an existing, separately funded clinical trial. This proposal would be considered an ancillary study that is not an independent clinical trial.

Ancillary Study to Case Study #42a: An internal medicine fellow designs an independent ancillary trial where a subset of patients from the parent trial in Case Study #42a will also receive drug B, based on the assumption that a two-drug combination will work significantly better than a single drug at both improving renal function and reducing polyps. The test subjects will be evaluated for renal function via plasma clearance rates at 6 and 12 months after initiation of drugs A and B.

• Are the participants prospectively assigned to an intervention? Yes, the participants are prospectively assigned to receive an intervention, drugs A and B.
• Is the study designed to evaluate the effect of the intervention on the participants? Yes, the study is designed to evaluate the effect of drugs A and B on renal function.
• Is the effect being evaluated a health-related biomedical or behavioral outcome? Yes, the effect being evaluated, renal function, is a health-related biomedical outcome.

A group of healthy young adults will perform a Go/No-Go task while undergoing fMRI scans. The purpose of the study is to characterize the pattern of neural activation in the frontal cortex during response inhibition, and the ability of the participant to correctly withhold a response on no-go

• Does the study involve human participants? Yes, healthy young adults will be enrolled in this study.
• Are the participants prospectively assigned to an intervention? Yes, the participants will be prospectively assigned to perform a Go/No-Go task, which involves different levels of inhibitory control.
• Is the study designed to evaluate the effect of the intervention on the participants? Yes, the study is designed to evaluate the effect of the Go/No-Go task on neural activation in the frontal cortex. The study will measure inhibitory control and the neural systems being engaged. In this study, the Go/No-Go task is the independent variable, and behavioral performance and the associated fMRI activations are the dependent variables.
• Is the effect being evaluated a health-related biomedical or behavioral outcome? Yes, the neural correlates of inhibitory control and behavioral performance are health-related biomedical outcomes.

A group of adolescents will participate in a longitudinal study examining changes in executive function over the course of a normal school year. Color naming performance on the standard version of the Stroop test will be obtained. All measures will be compared at multiple time points during the school year to examine changes in executive function. The purpose is to observe changes in executive function and to observe if differences exist in the Stroop effect over the course of the school year for these adolescents.

• Does the study involve human participants? Yes, adolescents will be enrolled in this study.
• Are the participants prospectively assigned to an intervention? No, there is no intervention in this study and no independent variable manipulated. The adolescents are not prospectively assigned to an intervention, but instead the investigator will examine variables of interest (including the Stroop test) over time. The Stroop effect is used as a measurement of point-in-time data.
• Is the study designed to evaluate the effect of the intervention on the participants? No, there is no intervention. Performance on the Stroop test is a well-established measure of executive function and the test is not providing an independent variable of interest here. It is not being used to manipulate the participants or their environment. The purpose is simply to obtain a measure of executive function in adolescents over the course of the school year.
• Is the effect being evaluated a health-related biomedical or behavioral outcome? N/A. No effect of an intervention is being evaluated.

A group of participants with social anxiety will perform an experimentally manipulated Stroop test. In this variant of the Stroop test, the stimuli presented are varied to include emotional and neutral facial expressions presented in different colors. Participants are instructed to name the colors of the faces presented, with the expectation that they will be slower to name the color of the emotional face than the neutral face. The purpose of the study is to examine the degree to which participants with social anxiety will be slower to process emotional faces than neutral faces.

• Does the study involve human participants? Yes, participants with social anxiety will be enrolled in this study.
• Are the participants prospectively assigned to an intervention? Yes, the participants will be prospectively assigned to perform a modified Stroop test using different colored emotional/neutral faces to explore emotional processing in people with social anxiety. Note that the independent variable is the presentation of emotional vs neutral faces.
• Is the study designed to evaluate the effect of the intervention on the participants? Yes, the study is designed to measure the effect of emotional valence (i.e. emotional faces) on participant response time to name the color. The purpose is to determine whether the response time to emotional faces is exaggerated for people with social anxiety as compared to neutral faces. Note that the response time to name the colors is the dependent variable in this study.
• Is the effect being evaluated a health-related biomedical or behavioral outcome? Yes, the processing of emotional information is a health-related biomedical outcome.

The study involves healthy volunteers and compares temporal SNR obtained with a new fMRI pulse sequence with that from another sequence.

• Are the participants prospectively assigned to an intervention? No, in this context the different pulse sequences would not be considered an intervention. The pulse sequences are not being used to modify any biomedical or behavioral outcome; rather the investigator is comparing performance characteristics of the two pulse sequences.

The study is designed to demonstrate that a new imaging technology (e.g. MRI, PET, ultrasound technologies, or image processing algorithm) is equivalent to, or has better sensitivity/specificity than a standard of care imaging technology. Aim one will use the new imaging technology and the gold standard in ten healthy volunteers. Aim Two will use the new imaging technology and the gold standard before and after a standard care procedure in ten patients. In both aims the performance of the new technology will be compared to the gold standard. No clinical care decisions will be made based on the use of the device in this study.

• Does the study involve human participants? YES. Aim one will study ten healthy volunteers, and aim two will study ten patient volunteers.
• Are the participants prospectively assigned to an intervention? Yes, participants will be prospectively assigned to be evaluated with a new imaging technology and the gold standard technology.
• Is the study designed to evaluate the effect of the intervention on the participants? No, the study is not measuring the effect of the technologies ON the human subjects. The study is determining if the new technology is equivalent or better than the gold standard technology. No effect on the participant is being measured.

An investigator proposes to add secondary outcomes to an already funded clinical trial of a nutritional intervention. The trial is supported by other funding, but the investigator is interested in obtaining NIH funding for studying oral health outcomes. Participants in the existing trial would be assessed for oral health outcomes at baseline and at additional time points during a multi-week dietary intervention. The oral health outcomes would include measures of gingivitis and responses to oral health related quality of life questionnaires. Oral fluids would be collected for analysis of inflammatory markers and microbiome components.

• Are the participants prospectively assigned to an intervention? No, because the assignment of participants to an intervention (and the administration of the intervention) occur as part of an existing, separately funded clinical trial. This proposal would be considered an ancillary study that leverages an already existing clinical trial.

The goal of the project is to use functional neuroimaging to distinguish patients with temporomandibular disorders (TMD) who experience TMD pain through centralized pain processes from those with TMD related to peripheral pain. Pain processing in a study cohort of TMD patients and healthy controls will be measured through functional magnetic resonance neuroimaging (fMRI) following transient stimulation of pain pathways through multimodal automated quantitative sensory testing (MAST QST). TMD patients will receive study questionnaires to better correlate the extent to which TMD pain centralization influences TMD prognosis and response to standard of care peripherally targeted treatment (prescribed by physicians, independently of the study).

• Are the participants prospectively assigned to an intervention? No, not in this context. The transient stimulation of pain pathways and the fMRI are being performed to measure and describe brain activity, but not to modify it.

An investigator proposes to perform a study of induced gingivitis in healthy humans, to study microbial colonization and inflammation under conditions of health and disease. During a 3-week gingivitis induction period, each study participant will use a stent to cover the teeth in one quadrant during teeth brushing. A contralateral uncovered quadrant will be exposed to the individual's usual oral hygiene procedures, to serve as a control. Standard clinical assessments for gingivitis will be made and biospecimens will be collected at the point of maximal induced gingivitis, and again after normal oral hygiene is resumed. Biospecimens will be assessed for microbial composition and levels of inflammation-associated chemokines.

• Are the participants prospectively assigned to an intervention? Yes, the participants are prospectively assigned to an intervention, abstaining from normal oral hygiene for a portion of the mouth, to induce gingivitis.
• Is the study designed to evaluate the effect of the intervention on the participants? Yes, the study is designed to evaluate the effect of the induced gingivitis on microbial composition and levels of inflammatory chemokines in oral samples.
• Is the effect being evaluated a health-related biomedical or behavioral outcome? Yes, the microbial composition and chemokine levels in oral samples are health-related biomedical outcomes.

The study will enroll older adults with hearing loss, comparing the effectiveness of enhanced hearing health care (HHC) to usual HHC. In addition to routine hearing-aid consultation and fitting, participants randomized to enhanced HCC will be provided patient-centered information and education about a full range of hearing assistive technologies and services. Study outcomes include the utilization of technology or services, quality of life, communication abilities, and cognitive function.

• Does the study involve human participants? Yes, the study enrolls older adults with hearing loss.
• Are the participants prospectively assigned to an intervention? Yes, participants are randomized to receive enhanced HCC or usual HCC interventions.
• Is the study designed to evaluate the effect of the intervention on the participants? Yes, the study will evaluate enhanced HCC’s effectiveness in modifying participant behavior and biomedical outcomes.
• Is the effect being evaluated a health-related biomedical or behavioral outcome? Yes, rate of technology/service utilization is a behavioral outcome and quality of life, communications, and cognition are biomedical outcomes that may be impacted by the interventions.

The study involves the recruitment of obese individuals who will undergo a muscle biopsy before and after either exercise training or diet-induced weight loss. Sarcolemmal 1,2-disaturated DAG and C18:0 ceramide species and mitochondrial function will be measured. Levels will be correlated with insulin sensitivity.

• Are the participants prospectively assigned to an intervention? Yes, the participants are assigned to either exercise training or a diet.
• Is the study designed to evaluate the effect of the intervention on the participants? Yes, the study is designed to compare the effects of the interventions on muscle metabolism.
• Is the effect being evaluated a health-related biomedical or behavioral outcome? Yes, muscle metabolism/signaling is a health-related outcome.

The study involves the recruitment of participants with type 2 diabetes who will undergo a muscle biopsy before and after a fast to measure acetylation on lysine 23 of the mitochondrial solute carrier adenine nucleotide translocase 1 (ANT1). Levels will be related to rates of fat oxidation.

• Are the participants prospectively assigned to an intervention? Yes, the participants are assigned to undergo a fast.
• Is the study designed to evaluate the effect of the intervention on the participants? Yes, the study is designed to compare the effects of the fast on molecular parameters of metabolism.
• Is the effect being evaluated a health-related biomedical or behavioral outcome? Yes, metabolism is a health-related outcome.

Insulin-resistant and insulin-sensitive nondiabetic adults who have a parent with type 2 diabetes will be followed over time to understand the role of mitochondrial dysfunction in the development of diabetes. Oral glucose tolerance tests will be performed annually to measure insulin sensitivity and glycemic status. Participants will also undergo a brief bout of exercise, and mitochondrial ATP synthesis rates will be measured by assessing the rate of recovery of phosphocreatine in the leg muscle, using 31P magnetic resonance spectroscopy.

• Are the participants prospectively assigned to an intervention? No, the participants are not assigned to an intervention; the OGTT and 31P MRS are measures.

Participants with chronic kidney disease will be recruited to receive one of two drug agents. After 6 weeks of therapy, subjects will undergo vascular function testing and have measures of oxidative stress evaluated in their plasma and urine. Results of the function testing and the oxidative stress biomarkers will be related to drug treatment.

• Are the participants prospectively assigned to an intervention? Yes, the participants are assigned to receive two different drugs.
• Is the study designed to evaluate the effect of the intervention on the participants? Yes, the study is designed to compare the effects of the drugs on vascular function.
• Is the effect being evaluated a health-related biomedical or behavioral outcome? Yes, vascular function is a health-related outcome.

Participants with Autosomal Dominant Polycystic Kidney Disease will be recruited to receive an oral curcumin therapy or placebo and the participants will undergo vascular function testing, renal imaging to assess kidney size, and assessment of oxidative stress biomarkers in urine and plasma after an ascorbic acid challenge. Changes in these outcomes will be related to oral therapy.

• Are the participants prospectively assigned to an intervention? Yes, the participants are assigned to receive medication or placebo.
• Is the study designed to evaluate the effect of the intervention on the participants? Yes, the study is designed to compare the effects of the drugs on vascular function and kidney size.
• Is the effect being evaluated a health-related biomedical or behavioral outcome? Yes, vascular function and kidney size are health-related outcomes.

Kidney transplant recipients will be recruited to undergo an experimental imaging procedure at several timepoints up to 4 months post-transplantation. Output from the images will be related to pathological assessments of the transplant as well as clinical measures of renal function.

• Are the participants prospectively assigned to an intervention? No, the participants are not assigned to receive an intervention. They undergo transplantation as part of their routine clinical care. The imaging procedure is a measure and not an intervention.

The study proposes the development of a novel probe to assess clearance of a nutritional metabolite in a given disease state. The probe is a GMP grade, deuterated, intravenously administered tracer and clearance is assessed by mass spectrometry analysis of serial blood draws. Participants will either receive a micronutrient supplement or will receive no supplementation. The clearance rate of the probe will be compared in the two groups, to understand the performance of the probe.

• Are the participants prospectively assigned to an intervention? Yes, the participants are assigned to receive either a micronutrient supplement or nothing.
• Is the study designed to evaluate the effect of the intervention on the participants? No, the intervention is being used to assess the performance of the probe and is not looking at an effect on the participant.

In order to assess the contribution of ingested glycolate to oxalate production, healthy participants will be recruited to a study involving the consumption of a controlled diet for three days, followed by an infusion of 13C2-glycolate. Blood and urine will be collected during the subsequent 24 hours to assess the amount of labeled glycolate in plasma and urine oxalate.

• Are the participants prospectively assigned to an intervention? Yes, the participants are assigned to receive a controlled diet for three days.
• Is the study designed to evaluate the effect of the intervention on the participants? No, the intervention (controlled diet) is being used to minimize exogenous dietary sources of oxalate in the participants prior to the labeled tracer infusion. The study will not be evaluating the effect of the diet on the participants.

This page last updated on: April 28, 2021

• Bookmark & Share
• E-mail Updates
• Privacy Notice
• Accessibility
• National Institutes of Health (NIH), 9000 Rockville Pike, Bethesda, Maryland 20892
• NIH... Turning Discovery Into Health

Clinical Case Study

Ai generator.

In 1970, the world first got acquainted with Genie. It was also the little girl’s first time to see a world beyond the potty chair where she was often bound to. Barely a contact outside for most of her life, she was a ripe case for studying the effects of extreme isolation in young children. Clinical case studies shed light on rare and specific circumstances, like Genie’s ordeal, that help us understand the bigger picture. Largely qualitative research , these case studies are an attempt to understand a subject and the case, usually in relation to a general concept.

8+ Clinical Case Study Templates and Examples

Clinical case studies can focus on a person, group, or community. In contrast to case reports , these studies don’t end in reporting about the diagnosis, treatment, and follow-up of patients. Case studies abide by the research methodology and design to understand an experience. During a case study analysis, both subjective and objective accounts of the events are deemed valid data. By focusing on a pixel of the picture, you can learn something that you would have otherwise overlooked. We have prepared the following case study templates that you can use in your research. For your reference, we added examples of scenarios where clinical case studies are being used.

1. Case Study Analysis Template

• Google Docs

Size: A4 & US Letter Sizes

Case studies are a common method of research in medical and psychological sciences. They are vivid narratives about undocumented cases that strike researchers as irregular and interesting. Their highly descriptive content are valuable information to the respective scientific community. They also open new avenues of inquiry and offer an in-depth treatment of a topic that empirical research cannot give. Its comprehensive nature helps make case study a popular research option, even if it falls short of evidence-based data. Thinking of using this research method? Get started with this template!

2. Clinical Case Study Template

Size: 49 KB

Since the studies contain detailed accounts, you have to format all the information into categories. The defined structure of the article makes all the information easy to absorb. A case study generally contains the following sections: abstract, introduction, patient information, review of related literature, methodology , findings, then the conclusion. The comprehensive nature of this research method might deter novice researchers, while veteran medical writers might just need a reminder. In either case, this sample outline is for you!

3. Clinical Case Study Sample

Size: 939 KB

This research method is usable in answering different inquiries. It is notable that case studies are heavy on the qualitative data. Researchers can obtain relevant data from interviews, questionnaires , personal and patients’ observations, journals, clinical reports, and existing literature. However, as seen in this attached example, quantitative data can also be collected as the researchers deem fit. Because the goal is not to derive data that can represent a population, researchers can use a smaller sample size. Study how to make both numbers and descriptions work to your advantage in preparing your clinical case study with this example!

4. Medical Case Study Example

Size: 563 KB

In a physician’s life, he or she is bound to come across a case that medical school and textbooks did not warn him or her. Clinical case studies are a form of communication about novel findings or observations in practice. Sort of like a medical buzz, the studies contain information like unreported health complications, adverse response to treatment, or new remedial methods. These case studies can also branch into new research directions. This case study illustrates how misdiagnosis can be harmful to the patient. Because some diseases can have overlapping symptoms, it can be hard to identify which is which. The case study alerted the medical community that a seemingly mundane skin condition can point to something more serious.

5. Psychological Case Study Example

Size: 425 KB

In the field of psychology, clinic psychologists and therapists can report about their interactions with the patient. Some of these cases can stand out as rare and unusual. Others may also serve as a useful reference. Practitioners can obtain information through semi-structured interviews wherein the patient talks with a mental health professional. After the sessions, the practitioner can interpret his or her findings into diagnosis and recommend a treatment plan . Psychology is not entirely removed from medicine. The specialist can incorporate the medical history of the patient in his or her interpretation. This sample case study shows medicine and psychology can work together in the prevention of stress-induced asthma attacks.

6. Sample Clinical Case Study

Size: 85 KB

The descriptive take of clinical case studies on a situation presents an exhaustive analysis that is not available in empirical research. However, the qualitative nature of these studies is a double-edged sword. The combination of subjective and objective analysis makes the content susceptible to personal biases. Because the case is unique to an individual or a group, researchers cannot replicate the result. The replicability of findings is a hallmark of reliable research. Therefore, clinical case studies have a low-reliability measure. The attached case study is an example of the use of descriptive analysis in the diagnosis and treatment of a patient with depression and adjustment disorder with mixed anxiety.

7. Medical Case Study Guide

Size: 266 KB

Another point raised against clinical case studies is the issue of memory distortion. The human memory is not a machine that can record and retrieve information at command. It is fallible, and it will make mistakes. The patients can emphasize a few parts of their history and overlook otherwise important pieces of the puzzle. Reliance on memory recall when writing the study can also fail the researchers. The sample clinical case study added here shows how a patient’s recollection of events in her life can be used in the presentation of the case. If the patient failed to recall important details, the researchers might have a different interpretations of the case.

8. Student Medical Case Study

Despite criticisms regarding susceptibility to biases and low-reliability measure, clinical case studies have been an indispensable tool for learning. Studies have reported a significant improvement in the academic performance of students after the integration of case studies into the learning ecosystem. Case studies are situation-based narratives about a textbook principle. Application-motivated learning is effective because the theoretical framework isn’t removed from the real-world experience. This case study is an example of those that are used in the classroom. The students are presented with a problem and series of follow up questions that will help them understand and address the issues exhibited.

9. Clinical Case Study Article

Size: 171 KB

Unlike empirical investigations, the goal is not to come up with results that can represent a population. Case studies focus on understanding an unusual plight through subjective and objective analysis. Understandably, such situation might not hold for most people. They are also the method of choice for understanding circumstances that cannot be reproduced in controlled testing environments, like Genie’s case earlier or the case discussed in the attached case study sample. Therapy for anorexia nervosa and obsessive personality disorder is hard to come by using quantitative research. Replicating such conditions will constitute a criminal offense. What case studies lack in the universality of the results, they make up for the richness of the insights obtained. It acknowledges that the human experience will always have a degree of subjectivity. This defense of clinical case studies makes them significant in their own right.

Text prompt

• Instructive
• Professional

10 Examples of Public speaking

20 Examples of Gas lighting

• Open access
• Published: 08 August 2024

Pitfalls of single-study external validation illustrated with a model predicting functional outcome after aneurysmal subarachnoid hemorrhage

• Jordi de Winkel 1 , 2 ,
• Carolien C. H. M. Maas 2 ,
• Bob Roozenbeek 1 ,
• David van Klaveren 2 &
• Hester F. Lingsma 2  

BMC Medical Research Methodology volume  24 , Article number:  176 ( 2024 ) Cite this article

Metrics details

Prediction models are often externally validated with data from a single study or cohort. However, the interpretation of performance estimates obtained with single-study external validation is not as straightforward as assumed. We aimed to illustrate this by conducting a large number of external validations of a prediction model for functional outcome in subarachnoid hemorrhage (SAH) patients.

We used data from the Subarachnoid Hemorrhage International Trialists (SAHIT) data repository ( n  = 11,931, 14 studies) to refit the SAHIT model for predicting a dichotomous functional outcome (favorable versus unfavorable), with the (extended) Glasgow Outcome Scale or modified Rankin Scale score, at a minimum of three months after discharge. We performed leave-one-cluster-out cross-validation to mimic the process of multiple single-study external validations. Each study represented one cluster. In each of these validations, we assessed discrimination with Harrell’s c-statistic and calibration with calibration plots, the intercepts, and the slopes. We used random effects meta-analysis to obtain the (reference) mean performance estimates and between-study heterogeneity (I 2 -statistic). The influence of case-mix variation on discriminative performance was assessed with the model-based c -statistic and we fitted a “membership model” to obtain a gross estimate of transportability.

Across 14 single-study external validations, model performance was highly variable. The mean c-statistic was 0.74 (95%CI 0.70–0.78, range 0.52–0.84, I 2  = 0.92), the mean intercept was -0.06 (95%CI -0.37–0.24, range -1.40–0.75, I 2  = 0.97), and the mean slope was 0.96 (95%CI 0.78–1.13, range 0.53–1.31, I 2  = 0.90). The decrease in discriminative performance was attributable to case-mix variation, between-study heterogeneity, or a combination of both. Incidentally, we observed poor generalizability or transportability of the model.

Conclusions

We demonstrate two potential pitfalls in the interpretation of model performance with single-study external validation. With single-study external validation. (1) model performance is highly variable and depends on the choice of validation data and (2) no insight is provided into generalizability or transportability of the model that is needed to guide local implementation. As such, a single single-study external validation can easily be misinterpreted and lead to a false appreciation of the clinical prediction model. Cross-validation is better equipped to address these pitfalls.

Peer Review reports

Clinical prediction models are used to predict the probability of a disease or an outcome conditional on a set of patient characteristics. As clinical prediction models are meant to facilitate clinical decision-making, it is paramount that the prognostic estimates are valid and precise. Unreliable risk estimates could give rise to faulty decision-making and thus patient harm [ 1 ]. For example, in vascular neurology we use the PHASES score (Population, Hypertension, Age, Size, Earlier Subarachnoid Hemorrhage, Site). The PHASES score is a clinical prediction model used to predict the 5-year rupture risk in patients with an unruptured intracranial aneurysm [ 2 ]. The risk of rupture predicted by the PHASES score is balanced with the risk of intervention to determine optimal management [ 3 ]. Misclassification could result in withholding preventive aneurysm treatment in high-risk patients or unnecessary treatment-related harm in low-risk patients. To investigate accuracy and precision of the prognostic estimates, validation is performed.

Two types of validation are distinguished: internal validation and external validation. Internal validation assesses the robustness of the model and the degree of overfitting (i.e., modeling random noise within the development data). External validation is used to investigate model performance in independent data that was not involved in model development. Model performance is expressed in terms of discrimination – the ability to distinguish patients likely to experience the outcome of interest from those who are not – and calibration – the agreement between the predicted and observed risk.

External validation is usually conducted in a study with data from a single center of a certain period, from a certain geographical area, and consisting of patients selected based on specific criteria. This method is called “single-study external validation”. The model performance obtained through this type of validation is often thought to represent model performance that can be expected in the population. However, this interpretation is not as straightforward as assumed. It is highly debatable whether (a single) single-study external validation provides true insight into the validity and accuracy of the risk predictions and there are several pitfalls to consider when interpreting its results [ 4 ]. There is an alternative, hybrid, internal–external cross-validation approach available for clustered data that might better address these pitfalls. Clustered data consists of multiple sources differing in multiple dimensions (e.g., geographical areas, studies, periods, etc.). In this approach, each cluster is left out once to validate the model in to obtain cluster-specific performance estimates.

Because the number of model development and validation studies is rising exponentially accurate critical appraisal of such studies is becoming increasingly important. Here, we conducted a large number of single-study validations of a prediction model for functional outcome in aneurysmal subarachnoid hemorrhage (aSAH) patients to illustrate potential pitfalls leading to misinterpretation of model performance.

Study population and model development

We used data from The Subarachnoid Hemorrhage International Trialists (SAHIT) data repository, an individual participant meta-analysis registry (Supplementary Material 1). The SAHIT data repository consisted of eleven randomized controlled trials (RCTs) [ 5 , 6 , 7 , 8 , 9 , 10 , 11 , 12 , 13 , 14 , 15 , 16 ] and ten prospective observational hospital registries ( n = 13,046) [ 17 , 18 , 19 , 20 , 21 ]. Patient enrolment took place in four continents over 30 years (1983–2012). From the SAHIT data repository, we selected studies that determined functional outcome with the (extended) Glasgow Outcome Scale ((e)GOS) or modified Rankin Scale (mRS) with a minimum follow-up of 3 months ( n  = 11,931). We excluded nine data sources (Supplementary Material 2).

The GOS was dichotomized into favorable and unfavorable (defined as a GOS 4–5 versus GOS 1–3, or eGOS 4–8 versus 1–3, respectively). The GOS ranges from one to five, with five being no symptoms and one death. The eGOS is a more detailed nine-level version of the GOS. If the GOS scores were not available we used the mRS or eGOS scores. The mRS is a 7-level scale ranging from zero, no symptoms to six, death. We dichotomized the mRS into 0–3, favorable, and 4–6, unfavorable. All functional outcome scales are commonly used in stroke research [ 22 , 23 ]. Conversion algorithms were described in Supplementary Material 3. Missing data were handled by using multiple imputation by chained equations. We assumed data were missing at random [ 24 ].

We refitted the previously published SAHIT (neuroimaging) model using identical data and predictors and similar modeling strategies as in the original development study [ 25 ]. The purpose of this study was not to develop a new or better model. In the SAHIT model, age, aneurysm location, aneurysm size, World Federation of Neurological Surgeons (WFNS) grade on admission, premorbid hypertension, and CT Fisher grade were used as predictors of functional outcome [ 26 , 27 ]. We dichotomized the WFNS grade into I-III and IV-V and categorized aneurysm location into anterior cerebral artery, anterior communicating artery, internal cerebral artery, middle cerebral artery, posterior communicating artery, and posterior circulation.

Model performance

We assessed discrimination with Harrell’s c -statistic and the model-based c -statistic [ 28 ]. We compared Harrell’s c -statistics at single-study and cross-validation validation with the internally validated optimism-corrected Harrell’s c -statistic obtained via bootstrapping (benchmark). Next, to quantify to which extent the case-mix variation influences the discriminative performance at validation we evaluated the difference between Harrell’s c -statistic and the model-based c -statistic. This difference is attributable to case-mix variation. We assessed calibration graphically with calibration plots and numerically with the intercept and the slope. To facilitate understanding of the study, we have included a detailed explanation of all performance measures that are discussed in this study (Supplementary Material 4).

We performed two types of validation to assess model performance: leave-one-cluster-out internal–external cross-validation (henceforth cross-validation) and single-study external validation. With cross-validation, each cluster (representing one study or registry) is alternatingly left out of model development to assess model performance [ 29 ]. The split in the data is non-random because existing heterogeneity between clusters is utilized (by study, geographical area, period). To mimic the process of multiple single-study external validations, we assessed model performance, obtained via cross-validation, individually in each cluster as if they were predefined external validation clusters. We assessed the performance metrics with leave-one-cluster-out cross-validation as with single-study external validation.

Next, we pooled the performance with random effects meta-analysis to obtain mean performance estimates that serve as a reference value for overall external model performance [ 30 ]. We used the I 2 -statistic to describe the degree of variability in model performance that is attributable to between-cluster heterogeneity. All statistical analyses were performed with R software (version 3.6.3, R Foundation for Statistical Computing) using the rms (version 6.2.0) [ 31 ], mice (version 3.13.0) [ 32 ], Hmisc (version 4.5.0) [ 33 ], metamisc (version 0.2.5) [ 34 ], CalibrationCurves (version 0.1.2) [ 35 ], metafor (version 3.4.0) [ 36 ], and PredictionTools (version 0.1.0) [ 37 ] packages. We adhered to the Transparent Reporting of a multivariable prediction model for Individual Prognosis or Diagnosis checklist (TRIPOD) statement (Supplementary Material 5) [ 38 ].

Transportability

We assessed the relatedness of the derivation and validation clusters by fitting a logistic regression “membership model” [ 29 ]. This model predicts a patient being part of the development (0) or validation (1) cluster and includes all predictors and the outcome. A high c -statistic of the membership model (easy discrimination) means that the development and validation clusters are not related and a low c -statistic (hard discrimination) vice versa. By jointly assessing Harrell’s c -statistic and the c -statistic of the membership model we obtain a gross estimate of the generalizability and transportability of the model. Generalizability is the reproducibility in a similar population and transportability is the reproducibility in a different but related population. Transportability is important because many models will be applied to populations that differ from the original study population. To illustrate, observing a high membership model c -statistic and a high Harrell’s c -statistic means that even though there was a lot of heterogeneity between clusters the model still discriminated well.

Baseline characteristics

In the overall SAHIT data repository, the mean age was 53 years (SD 13, Table  1 ), 80% of patients presented in a favorable clinical condition (WFNS grade I-III, n  = 8869), and 38% had premorbid hypertension ( n  = 2884). In 5% of patients, no hemorrhage was detected on the CT scan (Fisher grade 1, n = 493), 21% had a Fisher grade 2 (n = 2050), 39% had a Fisher grade 3 ( n  = 3762), and 35% had a Fisher grade 4 ( n  = 3440). Most ruptured aneurysms were smaller than 13 mm (84%, n  = 7314) and 88% had a ruptured intracranial aneurysm located in the anterior circulation ( n  = 9081). Eighty percent of patients had a favorable functional outcome at a minimum of 3-month follow-up (range 60–92%, n  = 8537).

Cluster-specific baseline characteristics

We observed large heterogeneity in baseline characteristics between individual clusters (between-study heterogeneity). The proportion of patients presenting in unfavorable clinical condition (WFNS IV-V) ranged from 0% (IHAST) to 78% (HHU) and the proportion of aneurysms larger than 13 mm from 6% (MAPS) to 26% (Tirilazad). In the Tirilazad cluster and ISAT clusters, no posterior communicating artery aneurysms (PCOM) were included, in the Tirilazad cluster no anterior communicating artery aneurysms (ACOM), and in EPO/Statin no anterior cerebral artery aneurysms. Conversely, in the Conscious-I, EPO/Statin, I-HAST, MAPS, MASH1/2, and Utrecht clusters the majority (> 50%) of aneurysms were located at the ACOM and PCOM sites. Both the mean age and the proportion of hypertension were similar across clusters. Favorable functional outcome ranged from 60% (Chicago) to 92% (MAPS).

Discrimination

The predictor effects were described in Supplementary Material 6. The internally validated, optimism-corrected Harrell’s c -statistic was 0.77 (95% CI 0.76–0.79, Table  2 ). We observed a large variability in discriminative performance when evaluating the c -statistics at single-study external validation. Model discrimination ranged from (very) poor, comparable to a coin flip, in the IMASH cluster (0.52, 95% CI 0.45–0.59), to moderate in the Leeds cluster (0.66, 95% CI 0.52–0.79), and the ISAT clusters (0.68, 95% CI 0.65–0.71), to excellent in the Conscious-1 (0.78, 95% CI 0.72–0.83), the D-SAT (0.80, 95% CI 0.75–0.85), and the SHOP clusters (0.84, 95% CI 0.82–0.86). The pooled mean c -statistic with cross-validation was 0.75 (95% CI 0.70–0.78). The I 2 -statistic was 0.92, indicating that the proportion of the total variability in the c -statistic was to a large extent explained by between-study heterogeneity.

We observed a substantial decrease in discriminative performance in 6 clusters compared to the optimism-corrected c -statistic (benchmark) and the pooled mean c-statistic (reference for overall external performance). Specifically, in the Leeds, IMASH, and Chicago clusters, this decline can be attributed to case-mix variation (Harrell’s 0.66, 0.52, and 0.76 versus model-based 0.76, 0.78, and 0.73, respectively). In the HHU and IHAST clusters, the drop is due to miscalibration (Harrell’s 0.73 and 0.71 versus model-based 0.71 and 0.68, respectively) and in the ISAT cluster it is explained by a combination of case-mix variation and miscalibration (Harrell's 0.68 and model-based 0.70).

Calibration

Again, we observed large variability in the calibration with single-study external validation across clusters. Calibration ranged from poor in the HHU cluster with an intercept of -1.40 (95% CI -2.08–-0.71, Fig.  1 A-N) and slope of 1.17 (0.16–2.17) to excellent in the CONSCIOUS-I, IHAST, ISAT, MASH1/2, and D-SAT clusters. The pooled mean intercept was -0.06 (95% CI -0.37–0.24) and the pooled mean slope was 0.96 (95% CI 0.78–1.17) with cross-validation. For both the intercept (I 2  = 0.97) and the slope (I 2  = 0.90), the I 2 -statistic indicated that the degree of total variability was largely explained by between-study heterogeneity.

A-M Internal-External Cross-Validation Calibration Plots. Abbreviations: C (ROC) =  c -statistic or receiving operating curve

In most clusters the membership models’ c -statistics were moderately high (IHAST, IMASH, ISAT, MAPS, DSAT, SHOP, and Utrecht, all between 0.70–0.80, Table  3 ) or high (CONSCIOUS-I, Chicago, EPO/Statin, HHU, and Tirilazad, all above 0.80). Despite this, the discriminative performance remained satisfactory indicating good transportability of the model. In other words: even in distinctly different study populations the model discriminated well between high-risk and low-risk patients. There was one exception to this generalization. In the Leeds cluster the derivation and validation samples were similar (membership model c -statistic 0.69), but the discriminative performance was unsatisfactory indicating poor generalizability of the model (Supplementary Material 7).

We used the SAHIT data repository, a large individual participant meta-analysis dataset, to study external validity in a large number of single-study cohorts to illustrate the potential pitfalls in interpreting model performance, of single-study external validation. Although single-study external validation is preferred over no at all, our analysis clearly illustrates two pitfalls and how this may lead to a false appreciation of the model.

(1) We observed that model performance was highly variable between cohorts. This means that the appreciation of the model is highly dependent on the choice of the validation data. For example, validating the model in the IMASH (0.52), Leeds (0.66), or the ISAT cluster (0.68) would indicate poor to moderate discrimination. Contrarily, the reference mean c -statistic (0.74) and most cluster-specific c -statistics suggest otherwise. There is no formal threshold for the c -statistic for implementation in clinical practice. Conversely, the model performance in the SHOP cluster (0.84) was probably more optimistic than can be expected in the population. A similar problem is observed when examining the calibration of the model at external validation. In some clusters such as HHU, I-MASH, Leeds or SHOP calibration was (very) poor, but in others it was excellent. Several factors may explain the large variation in performance.

First, case-mix variation is known to influence discriminative performance [ 39 , 40 ]. The more patients within a cluster are alike (homogeneous) the more difficult it is to discriminate between high and low-risk individuals. It is known that ISAT excluded 90% of initially screened patients leaving patients with similar characteristics such as favorable prognosis and aneurysms predominantly located at specific sites [ 41 ]. As such, in ISAT we observed a slightly higher model-based c -statistic than Harrell’s c -statistic. Validating the model in the ISAT cohort alone may lead to a false conclusion about the model’s discriminative performance.

Second, between-study heterogeneity can also affect model performance. Case-mix variation refers to the differences between subjects within a population, while between-study heterogeneity refers to differences between study populations. Slight changes in the definition or the measurement of predictors and the outcome can change the size and direction of predictor effects [ 42 ]. Additionally, predictors may affect treatment decisions downstream (confounding by indication) and subsequently affect patient outcomes. Because of this, such slight changes can lead to severe miscalibration and poor discrimination of a model.

With cross-validation, between-study heterogeneity can be used to benefit interpretation. The SAHIT registry consists of data from randomized trials that have had stringently selected (thus homogeneous) study populations, that together, form a very heterogeneous overall study population. Differences in predictor and outcome definition or measurement and the context of the study period and geographical origin of the SAHIT registry can be used to explain variability in model performance. The I 2 -statistics obtained with random effects meta-analysis confirmed that the proportion of total variability in model performance was largely explained by between-study heterogeneity.

(2) Without understanding the heterogeneity between the derivation and validation sample and the population we cannot assess the generalizability (i.e., reproducibility in a similar population) and the transportability (i.e., reproducibility in a different but related population) of the model [ 43 , 44 ]. For example, the intended subpopulation – in which the model is to be applied – may have distinct differences from the validation sample (e.g., in the healthcare setting, measurement of biomarkers or imaging findings, and treatment algorithms). Also, the validation may have been conducted in an independent, but equally selected sample not representative of the population. In both cases, we do not obtain a valid insight into the expected model performance.

Due to overall improvements in diagnostic capabilities, treatment algorithms, and patients’ outcomes the validity of all models will eventually expire. Thus, even if a model performs well it will most likely not be globally and eternally applicable [ 45 ]. Because of this, most models will need local and continuous updating. Some clinical prediction models will require updating of the intercept and/or the slope, while others may need a full re-estimation of the parameters. Geographical, temporal, or methodological clustering can be utilized to assess model performance across multiple dimensions and inform updating efforts to fit the intended context. In addition, the membership model can be used to obtain a gross estimate of the relatedness of two samples.

Strengths and Limitations

This study is strengthened by the use of a large individual participant meta-analysis dataset to assess model performance across multiple dimensions. Furthermore, because of the clustered nature of the SAHIT data repository, we were able to investigate between-cluster heterogeneity, generalizability, and transportability.

Our study is limited by the fact that in some clusters predictors were missing completely. We performed multiple imputation of the entire SAHIT data repository instead of the individual clusters. This may mean that between-cluster heterogeneity could be diluted and that model performance may be overestimated for individual clusters with completely missing predictor variables. The largest proportion of missingness for the full cohort was with premorbid hypertension (36%) and will not have a large effect on the overall conclusions of this study.

Another limitation of our study was the use of multiple outcome scales and varying time points of assessment. We chose to use the GOS at 3 months as much as possible, but not all studies assessed outcome equally. Patients can improve or worsen from 3 months up to 12 months, but we hypothesize that even though on an ordinal scale these changes will be substantial on a dichotomized scale this might be limited.

Recommendations

Insight into case-mix variation, between-cluster heterogeneity, generalizability, and transportability is required to decide if a clinical prediction model performs well enough for to be considered for implementation. Implementation without these insights could lead to patient harm due to inaccurate medical decision-making and possibly incentivize the development of new clinical prediction models instead of validating already existing ones, thereby contributing to research waste. Even for a relatively rare disease an abundance of models predicting functional outcome in aSAH patients already exist [ 46 , 47 , 48 , 49 , 50 ]. Most of these models contain more or less the same set of predictors. The rising availability of clustered data from large international collaborations will open up possibilities for leave-one-cluster-out cross-validation and should be utilized [ 51 ].

We advocate using cross-validation instead of single-study external validation, but there are also disadvantages to this approach. First, cross-validation requires a large clustered dataset with sufficient patients per cluster that is usually obtained through international collaborations and may not always be available. Second, an important feature of external validation is an independent evaluation of a clinical prediction model. Usually, leave-one-cluster-out internal–external cross-validation will be conducted directly after model development and thus not performed independently. To aid transparency, regression formulas, code, and data should be made publicly available.

If not available, a reasonable alternative strategy is conducting multiple (smaller) single-study external validations each exploring another dimension. We stress that a single single-study external validation cannot be interpreted as decisive proof of a well or poor model performance and that local and continuous validation is usually required. As a minimum, we advise evaluating selection criteria, recruitment dates, geographical location, and study design of the development and the validation data to obtain a gross estimate of between-cluster heterogeneity.

Two potential pitfalls of single-study external validation have to be considered when interpreting such a validation study. (1) Model performance with single-study external validation can depend heavily on the choice of validation data and can thus lead to a false appreciation of a clinical prediction model. (2) To accurately appreciate generalizability and transportability it is necessary to investigate heterogeneity between the derivation and validation data and the representativeness to the intended population. Thus, a single validation is not equipped to draw definitive conclusions about model performance. As an alternative leave-one-cluster-out internal–external cross-validation enables inspecting model performance across multiple settings with varying temporal, geographical, and methodological dimensions and can inform more reliably about expected performance and whether local revision is required.

Availability of data and materials

We are not at liberty to make the data supporting this publication available. Patient-level data is available upon reasonable request to the SAHIT data repository. More detailed information about the SAHIT data repository including an ethical statement is available elsewhere [ 52 , 53 ].

Upon publication, Rcode will be made publicly available via: https://github.com/WinkelJordi/crossvalidation .

Abbreviations

Aneurysmal subarachnoid hemorrhage

Computed tomography

Glasgow Outcome Scale

Modified Rankin Scale

Randomized controlled trial

Subarachnoid Haemorrhage International Trialists’

World Federation of Neurological Surgeons grade

Ramspek CL, Jager KJ, Dekker FW, Zoccali C, van Diepen M. External validation of prognostic models: what, why, how, when and where? Clin Kidney J. 2021;14(1):49–58. https://doi.org/10.1093/ckj/sfaa188 .

Article   PubMed   Google Scholar  

Greving JP, Wermer MJ, Brown RD Jr, et al. Development of the PHASES score for prediction of risk of rupture of intracranial aneurysms: a pooled analysis of six prospective cohort studies. Lancet Neurol. 2014;13(1):59–66. https://doi.org/10.1016/S1474-4422(13)70263-1 .

Algra AM, Greving JP, de Winkel J, et al. Development of the SAFETEA scores for predicting risks of complications of preventive endovascular or microneurosurgical intracranial aneurysm occlusion. Neurology. 2022. https://doi.org/10.1212/WNL.0000000000200978 .

Van Calster B, Steyerberg EW, Wynants L, van Smeden M. There is no such thing as a validated prediction model. BMC Med. 2023;21(1):70. https://doi.org/10.1186/s12916-023-02779-w .

Article   PubMed   PubMed Central   Google Scholar  

Etminan N, Beseoglu K, Eicker SO, Turowski B, Steiger HJ, Hanggi D. Prospective, randomized, open-label phase II trial on concomitant intraventricular fibrinolysis and low-frequency rotation after severe subarachnoid hemorrhage. Stroke. 2013;44(8):2162–8. https://doi.org/10.1161/STROKEAHA.113.001790 .

Article   CAS   PubMed   Google Scholar  

Jang YG, Ilodigwe D, Macdonald RL. Metaanalysis of tirilazad mesylate in patients with aneurysmal subarachnoid hemorrhage. Neurocrit Care. 2009;10(1):141–7. https://doi.org/10.1007/s12028-008-9147-y .

Macdonald RL, Kassell NF, Mayer S, et al. Clazosentan to overcome neurological ischemia and infarction occurring after subarachnoid hemorrhage (CONSCIOUS-1): randomized, double-blind, placebo-controlled phase 2 dose-finding trial. Stroke. 2008;39(11):3015–21. https://doi.org/10.1161/STROKEAHA.108.519942 .

McDougall CG, Johnston SC, Gholkar A, et al. Bioactive versus bare platinum coils in the treatment of intracranial aneurysms: the MAPS (Matrix and Platinum Science) trial. AJNR Am J Neuroradiol. 2014;35(5):935–42. https://doi.org/10.3174/ajnr.A3857 .

Molyneux A, Kerr R, International Subarachnoid Aneurysm Trial Collaborative G, et al. International Subarachnoid Aneurysm Trial (ISAT) of neurosurgical clipping versus endovascular coiling in 2143 patients with ruptured intracranial aneurysms: a randomized trial. J Stroke Cerebrovasc Dis. 2002;11(6):304–14. https://doi.org/10.1053/jscd.2002.130390 .

Pickard JD, Murray GD, Illingworth R, et al. Effect of oral nimodipine on cerebral infarction and outcome after subarachnoid haemorrhage: British aneurysm nimodipine trial. BMJ. 1989;298(6674):636–42. https://doi.org/10.1136/bmj.298.6674.636 .

Article   CAS   PubMed   PubMed Central   Google Scholar  

Suarez JI, Martin RH, Calvillo E, et al. The Albumin in Subarachnoid Hemorrhage (ALISAH) multicenter pilot clinical trial: safety and neurologic outcomes. Stroke. 2012;43(3):683–90. https://doi.org/10.1161/STROKEAHA.111.633958 .

Todd MM, Hindman BJ, Clarke WR, Torner JC. Intraoperative Hypothermia for Aneurysm Surgery Trial I. Mild intraoperative hypothermia during surgery for intracranial aneurysm. N Engl J Med. 2005;352(2):135–45. https://doi.org/10.1056/NEJMoa040975 .

Tseng MY, Hutchinson PJ, Richards HK, et al. Acute systemic erythropoietin therapy to reduce delayed ischemic deficits following aneurysmal subarachnoid hemorrhage: a Phase II randomized, double-blind, placebo-controlled trial. Clinical article J Neurosurg. 2009;111(1):171–80. https://doi.org/10.3171/2009.3.JNS081332 .

van den Bergh WM, Algra A, van Kooten F, et al. Magnesium sulfate in aneurysmal subarachnoid hemorrhage: a randomized controlled trial. Stroke. 2005;36(5):1011–5. https://doi.org/10.1161/01.STR.0000160801.96998.57 .

Wong GK, Poon WS, Chan MT, et al. Intravenous magnesium sulphate for aneurysmal subarachnoid hemorrhage (IMASH): a randomized, double-blinded, placebo-controlled, multicenter phase III trial. Stroke. 2010;41(5):921–6. https://doi.org/10.1161/STROKEAHA.109.571125 .

Tseng MY, Hutchinson PJ, Turner CL, et al. Biological effects of acute pravastatin treatment in patients after aneurysmal subarachnoid hemorrhage: a double-blind, placebo-controlled trial. J Neurosurg. 2007;107(6):1092–100. https://doi.org/10.3171/JNS-07/12/1092 .

Johnston SC, Dowd CF, Higashida RT, et al. Predictors of rehemorrhage after treatment of ruptured intracranial aneurysms: the Cerebral Aneurysm Rerupture After Treatment (CARAT) study. Stroke. 2008;39(1):120–5. https://doi.org/10.1161/STROKEAHA.107.495747 .

Schatlo B, Fung C, Fathi AR, et al. Introducing a nationwide registry: the Swiss study on aneurysmal subarachnoid haemorrhage (Swiss SOS). Acta Neurochir (Wien). 2012;154(12):2173–8; discussion 2178.  https://doi.org/10.1007/s00701-012-1500-4 . 

Helbok R, Kurtz P, Vibbert M, et al. Early neurological deterioration after subarachnoid haemorrhage: risk factors and impact on outcome. J Neurol Neurosurg Psychiatry. 2013;84(3):266–70. https://doi.org/10.1136/jnnp-2012-302804 .

Smith ML, Abrahams JM, Chandela S, Smith MJ, Hurst RW, Le Roux PD. Subarachnoid hemorrhage on computed tomography scanning and the development of cerebral vasospasm: the Fisher grade revisited. Surg Neurol. 2005;63(3):229–34. discussion 234–5.  https://doi.org/10.1016/j.surneu.2004.06.017 .

Reilly C, Amidei C, Tolentino J, Jahromi BS, Macdonald RL. Clot volume and clearance rate as independent predictors of vasospasm after aneurysmal subarachnoid hemorrhage. J Neurosurg. 2004;101(2):255–61. https://doi.org/10.3171/jns.2004.101.2.0255 .

Jennett B, Bond M. Assessment of outcome after severe brain damage. Lancet. 1975;1(7905):480–4. https://doi.org/10.1016/s0140-6736(75)92830-5 .

van Swieten JC, Koudstaal PJ, Visser MC, Schouten HJ, van Gijn J. Interobserver agreement for the assessment of handicap in stroke patients. Stroke. 1988;19(5):604–7. https://doi.org/10.1161/01.str.19.5.604 .

D.B. R. Multiple Imputation for Nonresponse in Surveys. John Wiley & Sons, New York (1987); 1987.

Jaja BNR, Saposnik G, Lingsma HF, et al. Development and validation of outcome prediction models for aneurysmal subarachnoid haemorrhage: the SAHIT multinational cohort study. BMJ. 2018;360:j5745. https://doi.org/10.1136/bmj.j5745 .

Report of World Federation of Neurological Surgeons Committee on a Universal Subarachnoid Hemorrhage Grading Scale. J Neurosurg. 1988;68(6):985–6. https://doi.org/10.3171/jns.1988.68.6.0985 .

Fisher CM, Kistler JP, Davis JM. Relation of cerebral vasospasm to subarachnoid hemorrhage visualized by computerized tomographic scanning. Neurosurgery. 1980;6(1):1–9. https://doi.org/10.1227/00006123-198001000-00001 .

van Klaveren D, Gonen M, Steyerberg EW, Vergouwe Y. A new concordance measure for risk prediction models in external validation settings. Stat Med. 2016;35(23):4136–52. https://doi.org/10.1002/sim.6997 .

Debray TP, Vergouwe Y, Koffijberg H, Nieboer D, Steyerberg EW, Moons KG. A new framework to enhance the interpretation of external validation studies of clinical prediction models. J Clin Epidemiol. 2015;68(3):279–89. https://doi.org/10.1016/j.jclinepi.2014.06.018 .

van Klaveren D, Steyerberg EW, Perel P, Vergouwe Y. Assessing discriminative ability of risk models in clustered data. BMC Med Res Methodol. 2014;14:5. https://doi.org/10.1186/1471-2288-14-5 .

Harrell Jr. FE. rms: Regression Modeling Strategies. R Package version 6.3-0. 2022. Available from: https://cran.rproject.org/web/packages/rms/index.html .

van Buuren S, Groothuis-Oudshoorn K. mice: Multivariate Imputation by Chained Equations in R. J Stat Softw. 2011;45(3):1–67.

Article   Google Scholar  

Harrel Jr. FE. Hmisc: Harrell Miscellaneous. R packages version 4.7-0. 2022. Available from: https://cran.rproject.org/web/packages/Hmisc/index.html .

Debray T, de Jong V. metamisc: Meta-Analysis of Diagnosis and Prognosis Research Studies. R package version 0.2.4. 2021. Available from: https://cran.r-project.org/web/packages/metamisc/index.html .

De Cock B, Nieboer D, Van Calster B, Steyerberg EW, Vergouwe Y. CalibrationCurves: Calibration performance. R package version 0.1.2. 2016.

Viechtbauer W. Conducting meta-analyses in R with the metafor package. J Stat Softw. 2010;36(3):1–48.

Maas C, Kent DM, Hughes MC, Dekker R, Lingsma HF, van Klaveren D. Performance metrics for models designed to predict treatment effect. BMC Med Res Methodol. 2023;23(1):165. https://doi.org/10.1186/s12874-023-01974-w .

Collins GS, Reitsma JB, Altman DG, Moons KG. Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD): the TRIPOD Statement. Br J Surg. 2015;102(3):148–58. https://doi.org/10.1002/bjs.9736 .

Austin PC, Steyerberg EW. Interpreting the concordance statistic of a logistic regression model: relation to the variance and odds ratio of a continuous explanatory variable. BMC Med Res Methodol. 2012;12:82. https://doi.org/10.1186/1471-2288-12-82 .

Vergouwe Y, Moons KG, Steyerberg EW. External validity of risk models: Use of benchmark values to disentangle a case-mix effect from incorrect coefficients. Am J Epidemiol. 2010;172(8):971–80. https://doi.org/10.1093/aje/kwq223 .

Molyneux AJ, Kerr RS, Yu LM, et al. International subarachnoid aneurysm trial (ISAT) of neurosurgical clipping versus endovascular coiling in 2143 patients with ruptured intracranial aneurysms: a randomised comparison of effects on survival, dependency, seizures, rebleeding, subgroups, and aneurysm occlusion. Lancet. 2005;366(9488):809–17. https://doi.org/10.1016/S0140-6736(05)67214-5 .

Luijken K, Groenwold RHH, Van Calster B, Steyerberg EW, van Smeden M. Impact of predictor measurement heterogeneity across settings on the performance of prediction models: A measurement error perspective. Stat Med. 2019;38(18):3444–59. https://doi.org/10.1002/sim.8183 .

Steyerberg EW, Vergouwe Y. Towards better clinical prediction models: seven steps for development and an ABCD for validation. Eur Heart J. 2014;35(29):1925–31. https://doi.org/10.1093/eurheartj/ehu207 .

Justice AC, Covinsky KE, Berlin JA. Assessing the generalizability of prognostic information. Ann Intern Med. 1999;130(6):515–24. https://doi.org/10.7326/0003-4819-130-6-199903160-00016 .

Steyerberg EW, Nieboer D, Debray TPA, van Houwelingen HC. Assessment of heterogeneity in an individual participant data meta-analysis of prediction models: an overview and illustration. Stat Med. 2019;38(22):4290–309. https://doi.org/10.1002/sim.8296 .

Dijkland SA, Roozenbeek B, Brouwer PA, et al. Prediction of 60-day case fatality after aneurysmal subarachnoid hemorrhage: external validation of a prediction model. Crit Care Med. 2016;44(8):1523–9. https://doi.org/10.1097/CCM.0000000000001709 .

Risselada R, Lingsma HF, Bauer-Mehren A, et al. Prediction of 60 day case-fatality after aneurysmal subarachnoid haemorrhage: results from the International Subarachnoid Aneurysm Trial (ISAT). Eur J Epidemiol. 2010;25(4):261–6. https://doi.org/10.1007/s10654-010-9432-x .

van Donkelaar CE, Bakker NA, Birks J, et al. Prediction of outcome after aneurysmal subarachnoid hemorrhage. Stroke. 2019;50(4):837–44. https://doi.org/10.1161/STROKEAHA.118.023902 .

Witsch J, Frey HP, Patel S, et al. Prognostication of long-term outcomes after subarachnoid hemorrhage: the FRESH score. Ann Neurol. Jul2016;80(1):46–58. https://doi.org/10.1002/ana.24675 .

Witsch J, Kuohn L, Hebert R, et al. Early prognostication of 1-year outcome after subarachnoid hemorrhage: the FRESH score validation. J Stroke Cerebrovasc Dis. 2019;28(10):104280. https://doi.org/10.1016/j.jstrokecerebrovasdis.2019.06.038 .

Debray TP, Moons KG, Ahmed I, Koffijberg H, Riley RD. A framework for developing, implementing, and evaluating clinical prediction models in an individual participant data meta-analysis. Stat Med. 2013;32(18):3158–80. https://doi.org/10.1002/sim.5732 .

Jaja BN, Attalla D, Macdonald RL, et al. The Subarachnoid Hemorrhage International Trialists (SAHIT) Repository: advancing clinical research in subarachnoid hemorrhage. Neurocrit Care. 2014;21(3):551–9. https://doi.org/10.1007/s12028-014-9990-y .

Macdonald RL, Cusimano MD, Etminan N, et al. Subarachnoid Hemorrhage International Trialists data repository (SAHIT). World Neurosurg. 2013;79(3–4):418–22. https://doi.org/10.1016/j.wneu.2013.01.006 .

Download references

Acknowledgements

The authors wish to thank the SAHIT Collaboration for supplying the data necessary to conduct this study.

This research received no specific grant from any funding agency in the public, commercial or not-for-profit sectors.

Author information

Authors and affiliations.

Department of Neurology, Erasmus MC University Medical Center Rotterdam, 40 Doctor Molewaterplein, P.O. Box 2040, Rotterdam, Zuid-Holland, 3015 GD, The Netherlands

Jordi de Winkel & Bob Roozenbeek

Department of Public Health, Erasmus MC University Medical Center Rotterdam, Rotterdam, Zuid-Holland, The Netherlands

Jordi de Winkel, Carolien C. H. M. Maas, David van Klaveren & Hester F. Lingsma

You can also search for this author in PubMed   Google Scholar

Contributions

Study Conceptualization (J.d.W., C.C.H.M, D.v.K., B.R., H.F.L.); Literature review (J.d.W.); Formal analysis (J.d.W., C.C.H.M.); Visualization (J.d.W.); Writing original draft (J.d.W.); Critical review of the manuscript (J.d.W., C.C.H.M, D.v.K., B.R., H.F.L.); Supervision (H.F.L.).

Corresponding author

Correspondence to Jordi de Winkel .

Ethics declarations

Ethics approval and consent to participate.

The institutions’ medical ethical research committee (MERC) of the Erasmus MC University Medical Center Rotterdam approved the study protocol under the exemption category and waived the need for written informed consent (MEC-2020–0810). Participant consent was not obtained and the study involved de-identified data.

Consent for publication

Not applicable.

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Supplementary material 1., rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

de Winkel, J., Maas, C.C.H.M., Roozenbeek, B. et al. Pitfalls of single-study external validation illustrated with a model predicting functional outcome after aneurysmal subarachnoid hemorrhage. BMC Med Res Methodol 24 , 176 (2024). https://doi.org/10.1186/s12874-024-02280-9

Download citation

Received : 15 August 2023

Accepted : 09 July 2024

Published : 08 August 2024

DOI : https://doi.org/10.1186/s12874-024-02280-9

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Intracranial Aneurysm
• Logistic Regression
• Validation Study
• Subarachnoid Hemorrhage

BMC Medical Research Methodology

ISSN: 1471-2288

• General enquiries: [email protected]

Information

• Author Services

Initiatives

You are accessing a machine-readable page. In order to be human-readable, please install an RSS reader.

All articles published by MDPI are made immediately available worldwide under an open access license. No special permission is required to reuse all or part of the article published by MDPI, including figures and tables. For articles published under an open access Creative Common CC BY license, any part of the article may be reused without permission provided that the original article is clearly cited. For more information, please refer to https://www.mdpi.com/openaccess .

Feature papers represent the most advanced research with significant potential for high impact in the field. A Feature Paper should be a substantial original Article that involves several techniques or approaches, provides an outlook for future research directions and describes possible research applications.

Feature papers are submitted upon individual invitation or recommendation by the scientific editors and must receive positive feedback from the reviewers.

Editor’s Choice articles are based on recommendations by the scientific editors of MDPI journals from around the world. Editors select a small number of articles recently published in the journal that they believe will be particularly interesting to readers, or important in the respective research area. The aim is to provide a snapshot of some of the most exciting work published in the various research areas of the journal.

Original Submission Date Received: .

• Active Journals
• Find a Journal
• Proceedings Series
• For Authors
• For Reviewers
• For Editors
• For Librarians
• For Publishers
• For Societies
• For Conference Organizers
• Open Access Policy
• Institutional Open Access Program
• Special Issues Guidelines
• Editorial Process
• Research and Publication Ethics
• Article Processing Charges
• Testimonials
• Preprints.org
• SciProfiles
• Encyclopedia

Article Menu

• Subscribe SciFeed
• Author Biographies
• Google Scholar
• on Google Scholar
• Table of Contents

Find support for a specific problem in the support section of our website.

Please let us know what you think of our products and services.

Visit our dedicated information section to learn more about MDPI.

JSmol Viewer

Tumor-like lesions in the craniovertebral junction: a case series, systematic review, and meta-analysis.

Simple Summary

1. introduction, 2. materials and methods, 2.1. personal case series, 2.2. literature search strategy, 2.3. eligibility criteria, 2.4. screening and data extraction, 2.5. data analysis, 3.1. personal case series ( table 1 ).

3.2. Search Strategy

3.3. patient baseline characteristics for tumor-like lesions, 3.4. the meta-analysis of neurenteric cyst: prognostic factors for recurrence, 3.5. the meta-analysis of the neurenteric cyst according to the extent of resection, 3.6. the meta-analysis of neurenteric cyst according to different variables, 4. discussion, 4.1. tumor types and demographics, 4.2. clinical implications, 4.3. surgical outcomes and recurrence, 4.4. complications and management, 4.5. meta-analysis of the neurenteric cyst located at the cvj, 4.6. future directions, 5. conclusions, supplementary materials, author contributions, institutional review board statement, informed consent statement, data availability statement, conflicts of interest.

• Vhora, R.; Vhora, S.; Katkar, A.; Ghate, P. A case report of craniovertebral junction intradural extramedullary neurenteric cyst. Med. J. Dr. D. Y. Patil. Univ. 2014 , 7 , 373. [ Google Scholar ] [ CrossRef ]
• Menezes, A.H.; Traynelis, V.C. Spinal neurenteric cysts in the magnetic resonance imaging era. Neurosurgery 2006 , 58 , 97–105, discussion 97–105. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Aebi, M.; Abumi, K.; Barbieri, A.; Behari, S.; Benzel, E.C.; Bolognese, P.; Burute, N.; Cacciola, F.; Carpentier, A.; Carrau, R.L.; et al. The Craniovertebral Junction: Diagnosis, Pathology, Surgical Techniques ; Goel, A., Cacciola, F., Eds.; Georg Thieme KG: Stuttgart, Germany, 2011. [ Google Scholar ] [ CrossRef ]
• Park, S.J.; Hong, J.H.; Han, M.S.; Moon, B.J.; Koo, J.Y.; Lee, J.K. Neurenteric cyst of the craniovertebral junction treated to reduce recurrence using different strategies: Two case reports and a literature review. Medicine 2023 , 102 , e33844. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Menendez, R.H.; D’Osvaldo, D.H.; Vilarino, A.; Amante, M.F.; Dillon, H.S. Neurenteric Cyst of the Ventral Craniocervical Junction: Case Report and Review of the Literature. World Neurosurg. 2019 , 125 , 257–260. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Shetty, S.R.; Panigrahi, M.; Rao, S. Neurenteric cyst at the craniovertebral junction: A report of two cases. Asian J. Neurosurg. 2013 , 8 , 188–191. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Kim, W.Y.; Lim, J.; Cho, K.G. Anterior Craniocervical Junctional Neurenteric Cyst. Brain Tumor Res. Treat. 2021 , 9 , 106–110. [ Google Scholar ] [ CrossRef ]
• Liu, J.K.; Couldwell, W.T. Far-lateral transcondylar approach: Surgical technique and its application in neurenteric cysts of the cervicomedullary junction. Report of two cases. Neurosurg. Focus. 2005 , 19 , E9. [ Google Scholar ] [ CrossRef ]
• Haque, M.; Rahman, A.; Ahmed, N.; Alam, S. Huge Ventral Cervicomedullary Neurenteric Cyst: A Rare Entity with Good Surgical Outcome and Appraisal. Asian J. Neurosurg. 2020 , 15 , 1016–1019. [ Google Scholar ] [ CrossRef ]
• Menezes, A.H. Craniovertebral junction neoplasms in the pediatric population. Childs Nerv. Syst. 2008 , 24 , 1173–1186. [ Google Scholar ] [ CrossRef ]
• Menezes, A.H.; Dlouhy, B.J. Neurenteric cysts at foramen magnum in children: Presentation, imaging characteristics, and surgical management-case series and literature review. Childs Nerv. Syst. 2020 , 36 , 1379–1384. [ Google Scholar ] [ CrossRef ]
• Gupta, S.K.; Sharma, B.S.; Khosla, V.K.; Mathuria, S.N.; Pathak, A.; Tewari, M.K. Far lateral approach for foramen magnum lesions. Neurol. Med. Chir. 2000 , 40 , 48–52, discussion 52–44. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Crockard, H.A.; Sen, C.N. The transoral approach for the management of intradural lesions at the craniovertebral junction: Review of 7 cases. Neurosurgery 1991 , 28 , 88–97, discussion 97–88. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Kak, V.K.; Gupta, R.K.; Sharma, B.S.; Banerjee, A.K. Craniospinal enterogenous cyst: MR findings. J. Comput. Assist. Tomogr. 1990 , 14 , 470–472. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Moher, D.; Liberati, A.; Tetzlaff, J.; Altman, D.G.; Group, P. Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement. PLoS Med. 2009 , 6 , e1000097. [ Google Scholar ] [ CrossRef ]
• Palmisciano, P.; Ferini, G.; Watanabe, G.; Conching, A.; Ogasawara, C.; Scalia, G.; Bin-Alamer, O.; Haider, A.S.; Passanisi, M.; Maugeri, R.; et al. Surgical Management of Craniovertebral Junction Schwannomas: A Systematic Review. Curr. Oncol. 2022 , 29 , 4842–4855. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Maiorano, E.; Spena, G.; Sovardi, F.; Dehgani-Mobaraki, P.; Pagella, F.; Montalbetti, A.; Peppucci, E.; Grasso, C.; Zoia, C. Extremely Rare Pathologies of the Craniovertebral Junction Region: A Case Series and Review of the Literature. Surgeries 2023 , 4 , 420–433. [ Google Scholar ] [ CrossRef ]
• Caldarelli, M.; Massimi, L.; Kondageski, C.; Di Rocco, C. Intracranial midline dermoid and epidermoid cysts in children. J. Neurosurg. 2004 , 100 , 473–480. [ Google Scholar ] [ CrossRef ]
• Benzagmout, M.; Agharbi, S.; Chakour, K.; Chaoui, M.E. Dermoid cyst of the posterior fossa. Neurosciences 2011 , 16 , 153–155. [ Google Scholar ]
• Zarineh, A.; Leon, M.E.; Saad, R.S.; Silverman, J.F. Multiple neuroenteric cysts at cerebello-pontine angle and foramen magnum: A case report and review of the literature. Neuropathology 2009 , 29 , 318–322. [ Google Scholar ] [ CrossRef ]
• Tripathi, M.; Savardekar, A.; Chhabra, R.; Radotra, B.D.; Gupta, S.K. Unusual imaging finding of a “fluid-fluid” level in a neurenteric cyst at anterior margin of the cervico-medullary junction. Br. J. Neurosurg. 2015 , 29 , 432–434. [ Google Scholar ] [ CrossRef ]
• Bejjani, G.K.; Wright, D.C.; Schessel, D.; Sekhar, L.N. Endodermal cysts of the posterior fossa. Report of three cases and review of the literature. J. Neurosurg. 1998 , 89 , 326–335. [ Google Scholar ] [ CrossRef ]
• Budke, M.; Luglietto, D.; Vitaliano Muzii, F. Surgical anatomy of a neurenteric cyst anterior to the brainstem. Interdisciplinary Neurosurgery 2020 , 21 , 100754. [ Google Scholar ] [ CrossRef ]
• Sharma, R.; Chandramouli, T.C.; Rao, R.M. Ventral foramen magnum neurenteric cyst presenting as acute rapidly progressive quadriparesis and respiratory compromise: A case report and review of literature. Neurol. India 2013 , 61 , 187–189. [ Google Scholar ] [ CrossRef ]
• Fujii, Y.; Nagaishi, M.; Nakae, R.; Takigawa, T.; Tanaka, Y.; Suzuki, K. Intracranial endodermal cyst presenting with nonobstructive hydrocephalus: A case report. Medicine 2019 , 98 , e14322. [ Google Scholar ] [ CrossRef ]
• Kemp, O.A.G.; Deepak, S.; Salem, O.; Arzoglou, V. The oldest presenting neurenteric cyst of the spinal cord. Br. J. Neurosurg. 2023 , 37 , 856–859. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Nelson, S.M.; Mathis, D.A.; Hobbs, J.K.; Timpone, V.M. Intracranial neurenteric cyst mimicking an ependymoma: Imaging features, pathologic correlation and review of literature. Clin. Imaging 2017 , 44 , 117–120. [ Google Scholar ] [ CrossRef ]
• Tsutsumi, S.; Izumi, H.; Nonaka, S.; Okura, H.; Suzuki, T.; Ito, M.; Yasumoto, Y.; Ishii, H. Spinal endodermal cyst undergoing malignant transformation and marked elevation of serum carbohydrate 19-9 level. Br. J. Neurosurg. 2023 , 37 , 347–349. [ Google Scholar ] [ CrossRef ]
• Al-Ahmed, I.H.; Boughamoura, M.; Dirks, P.; Kulkarni, A.V.; Rutka, J.T.; Drake, J.M. Neurosurgical management of neurenteric cysts in children. J. Neurosurg. Pediatr. 2013 , 11 , 511–517. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• de Oliveira, R.S.; Cinalli, G.; Roujeau, T.; Sainte-Rose, C.; Pierre-Kahn, A.; Zerah, M. Neurenteric cysts in children: 16 consecutive cases and review of the literature. J. Neurosurg. 2005 , 103 , 512–523. [ Google Scholar ] [ CrossRef ]
• Sahara, Y.; Nagasaka, T.; Takayasu, M.; Takagi, T.; Hata, N.; Yoshida, J. Recurrence of a neurenteric cyst with malignant transformation in the foramen magnum after total resection. Case report. J. Neurosurg. 2001 , 95 , 341–345. [ Google Scholar ] [ CrossRef ]
• Kimura, H.; Nagatomi, A.; Ochi, M.; Kurisu, K. Intracranial neurenteric cyst with recurrence and extensive craniospinal dissemination. Acta Neurochir. 2006 , 148 , 347–352, discussion 352. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Yang, Y.; Fang, J.; Li, D.A.; Wang, L.; Ji, N.; Zhang, J. Recurrent intracranial neurenteric cyst with malignant transformation: A case report and literature review. Oncol. Lett. 2016 , 11 , 3395–3402. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Malcolm, G.P.; Symon, L.; Kendall, B.; Pires, M. Intracranial neurenteric cysts. Report of two cases. J. Neurosurg. 1991 , 75 , 115–120. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Prasad, G.L.; Borkar, S.A.; Subbarao, K.C.; Sharma, M.C.; Mahapatra, A.K. Neurenteric cyst of the ventral cervicomedullary region. J. Pediatr. Neurosci. 2012 , 7 , 188–190. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Fuse, T.; Yamada, K.; Kamiya, K.; Inagaki, H. Neurenteric cyst at the craniovertebral junction: Report of two cases. Surg. Neurol. 1998 , 50 , 431–436. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Koksel, T.; Revesz, T.; Crockard, H.A. Craniospinal neurenteric cyst. Br. J. Neurosurg. 1990 , 4 , 425–428. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Tucker, A.; Miyake, H.; Tsuji, M.; Ukita, T.; Ito, S.; Matsuda, N.; Ohmura, T. Neurenteric cyst of the lower clivus. Neurosurgery 2010 , 66 , E224–E225. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Menezes, A.H.; Ryken, T.C. Craniocervical intradural neurenteric cysts. Pediatr. Neurosurg. 1995 , 22 , 88–95. [ Google Scholar ] [ CrossRef ]
• Chen, F.; Marx, S.; Zhang, C.; Cao, J.; Yu, Y.; Chen, D. Intramedullary bronchogenic cyst in the foramen magnum region accompanied with syringomyelia: A case report and literature review. Medicine 2019 , 98 , e14353. [ Google Scholar ] [ CrossRef ]
• Deeb, Z.L.; Maroon, J.C. Diagnosis of craniocervical dermoid by computed tomography and magnetic resonance imaging: A case report. J. Comput. Tomogr. 1985 , 9 , 305–310. [ Google Scholar ] [ CrossRef ]
• Lazareff, J.A.; Hoil Parra, J.A. Intradural neurenteric cyst at the craniovertebral junction. Childs Nerv. Syst. 1995 , 11 , 536–538. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Weis, J.; Reul, J.; Mayfrank, L.; Ramaekers, V.; Thron, A. Duplication of a vertebral artery associated with epidermoid cyst of the posterior fossa. Eur. Radiol. 1997 , 7 , 412–414. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Diekmann-Guiroy, B.; Huang, P.S. Klippel-Feil syndrome in association with a craniocervical dermoid cyst presenting as aseptic meningitis in an adult: Case report. Neurosurgery 1989 , 25 , 652–655. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Park, J.G.; Babu, R.; Kranz, P.G.; McLendon, R.E.; Adamson, C. Intraaxial dermoid cyst of the medulla. J. Neurosurg. 2013 , 119 , 442–445. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Zhang, Y.K.; Geng, S.M.; Liu, P.N.; Lv, G. Association of Craniovertebral Junction Anomalies, Klippel-Feil Syndrome, Ruptured Dermoid Cyst and Mirror Movement in One Patient: A Unique Case and Literature Review. Turk. Neurosurg. 2016 , 26 , 153–165. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Andrabi, Y.; Nejat, F.; Khashab, M.E.; Ashrafi, M.R. Mirror movement associated with neural tube defects. Neuropsychiatr. Dis. Treat. 2008 , 4 , 1273–1276. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Brooks, B.S.; Duvall, E.R.; el Gammal, T.; Garcia, J.H.; Gupta, K.L.; Kapila, A. Neuroimaging features of neurenteric cysts: Analysis of nine cases and review of the literature. AJNR Am. J. Neuroradiol. 1993 , 14 , 735–746. [ Google Scholar ]
• Chandra, P.S.; Gupta, A.; Mishra, N.K.; Mehta, V.S. Association of craniovertebral and upper cervical anomalies with dermoid and epidermoid cysts: Report of four cases. Neurosurgery 2005 , 56 , E1155, discussion E1155. [ Google Scholar ]
• Matsushima, K.; Kohno, M.; Izawa, H.; Tanaka, Y. Partial Transcondylar Approach for Ventral Foramen Magnum Neurenteric Cyst: 2-Dimensional Operative Video. Oper. Neurosurg. 2019 , 16 , E81. [ Google Scholar ] [ CrossRef ]
• Lieb, G.; Krauss, J.; Collmann, H.; Schrod, L.; Sorensen, N. Recurrent bacterial meningitis. Eur. J. Pediatr. 1996 , 155 , 26–30. [ Google Scholar ] [ CrossRef ]
• Macdonald, R.L.; Schwartz, M.L.; Lewis, A.J. Neurenteric cyst located dorsal to the cervical spine: Case report. Neurosurgery 1991 , 28 , 583–587, discussion 587–588. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Muzumdar, D.; Goel, A. Posterior cranial fossa dermoid in association with craniovertebral and cervical spinal anomaly: Report of two cases. Pediatr. Neurosurg. 2001 , 35 , 158–161. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Aksoy, F.G.; Aksoy, O.G.; Gomori, J.M. Klippel-Feil syndrome in association with posterior fossa suboccipital dermoid cyst. Eur. Radiol. 2001 , 11 , 142–144. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• McLaughlin, N.; Weil, A.G.; Demers, J.; Shedid, D. Klippel-Feil syndrome associated with a craniocervico-thoracic dermoid cyst. Surg. Neurol. Int. 2013 , 4 , S61–S66. [ Google Scholar ] [ CrossRef ]
• Schelper, R.L.; Kagan-Hallet, K.S.; Huntington, H.W. Brainstem subarachnoid respiratory epithelial cysts: Report of two cases and review of the literature. Hum. Pathol. 1986 , 17 , 417–422. [ Google Scholar ] [ CrossRef ]
• Kozak, J.; Bizik, I.; Surkala, J.; Steno, J.; Steno, A. Neurenteric cysts, incidence and surgical treatment. Bratisl. Lek. Listy 2019 , 120 , 680–685. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Iglesias, J.; Carrasco; De la Fuente, P.; Galbarriatu, L.; Paternain, C.; Ruiz de Gopegui, E.; Zaldumbide, L.; Pomposo, I. Neurenteric CYST of the craniocervical junction. Case report. Interdiscip. Neurosurg. 2021 , 23 , 100852. [ Google Scholar ] [ CrossRef ]
• Abdulqader, S.B.; Elahi, B.N.; Alshoumer, A.; Alzhrani, G. Neurenteric cyst of the dorsal craniocervical junction in an adult: A case report and operative video. Surg. Neurol. Int. 2022 , 13 , 291. [ Google Scholar ] [ CrossRef ]
• Kawamura, Y.; Oketani, H.; Mizoguchi, M.; Hata, N.; Suzuki, S.O.; Iihara, K. A Dorsally Located Endodermal Cyst in the Foramen Magnum Mimicking an Arachnoid Cyst: A Case Report. Pediatr. Neurosurg. 2020 , 55 , 197–202. [ Google Scholar ] [ CrossRef ]
• Wang, X.; Song, G.; Chen, G.; Guo, H.; Li, M.; Liang, J.; Bao, Y. Single-Center Clinical Characteristics and Treatment Experience of Foramen Magnum Neurenteric Cyst: Report of 6 Cases and Brief Review of the Literature. World Neurosurg. 2018 , 112 , e608–e616. [ Google Scholar ] [ CrossRef ]
• Yagnik, K.J.; Vakharia, K.; Vaubel, R.A.; Vizcaino, M.A.; Benson, J.C.; Daniels, D.J.; Link, M.J.; Van Gompel, J.J. Surgical Experience and Management of Intracranial Neurenteric Cysts: Single-Center Experience and Review of the Literature. J. Neurol. Surg. B Skull Base 2023 , 84 , 272–280. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Prasad, G.L.; Sharma, B.S.; Mahapatra, A.K. Ventral foramen magnum neurenteric cysts: A case series and review of literature. Neurosurg. Rev. 2016 , 39 , 535–544. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Chen, J.; Lai, R.; Li, Z.; Gao, J.; Li, Y.; Wang, T.; Li, Y. Case Report Series and Review of Rare Intradural Extramedullary Neoplasms-Bronchiogenic Cysts. Medicine 2015 , 94 , e2039. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Lai, P.M.R.; Zaazoue, M.A.; Francois, R.; Dupervil, D.G.; Berkowitz, A.L.; Alexandrescu, S.; Proctor, M.R. Neurenteric cyst at the dorsal craniocervical junction in a child: Case report. J. Clin. Neurosci. 2018 , 48 , 86–89. [ Google Scholar ] [ CrossRef ]
• Pop, M.M.; Bouros, D.; Klimko, A.; Florian, I.A.; Florian, I.S. Intracranial epidermoid cysts: Benign entities with malignant behavior: Experience with 36 cases. Sci. Rep. 2023 , 13 , 6474. [ Google Scholar ] [ CrossRef ]
• Raghunath, A.; Devi, B.I.; Bhat, D.I.; Somanna, S. Unusual complications of a benign tumour—Our experience with midline posterior fossa epidermoids. Br. J. Neurosurg. 2013 , 27 , 69–73. [ Google Scholar ] [ CrossRef ]
• Esfahani, D.R.; Burokas, L.; Brown, H.G.; Hahn, Y.S.; Nikas, D. Management of an unusual, recurrent neurenteric cyst in an infant: Case report and review of the literature. Childs Nerv. Syst. 2017 , 33 , 1603–1607. [ Google Scholar ] [ CrossRef ]
• Preece, M.T.; Osborn, A.G.; Chin, S.S.; Smirniotopoulos, J.G. Intracranial neurenteric cysts: Imaging and pathology spectrum. AJNR Am. J. Neuroradiol. 2006 , 27 , 1211–1216. [ Google Scholar ] [ PubMed ]
• Ozalp, H.; Hamzaoglu, V.; Karatas, D.; Dagtekin, A.; Yildiz, M.; Avci, E. Rare Cause of Acute Tetraplegia and Respiratory Arrest: Cervicomedullary Neuroenteric Cyst in a Child: Case Report. NMC Case Rep. J. 2019 , 6 , 1–4. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Wang, L.; Zhang, J.; Wu, Z.; Jia, G.; Zhang, L.; Hao, S.; Geng, S. Diagnosis and management of adult intracranial neurenteric cysts. Neurosurgery 2011 , 68 , 44–52, discussion 52. [ Google Scholar ] [ CrossRef ]
• Pettersson, S.D.; Ali, S.; Burmaka, P.; Fercho, J.; Szmuda, T.; Abuhaimed, A.; Alotaibi, Y.; Sloniewski, P.; Krakowiak, M. Predictors for complete surgical resection of posterior fossa neurenteric cysts: A case report and meta-analysis. Surg. Neurol. Int. 2021 , 12 , 530. [ Google Scholar ] [ CrossRef ] [ PubMed ]

Click here to enlarge figure

Share and Cite

Pop, M.M.; Bouros, D.; Klimko, A.; Pop, L.A.; Topal, P.; Topal, A.; Florian, I.S. Tumor-Like Lesions in the Craniovertebral Junction: A Case Series, Systematic Review, and Meta-Analysis. Cancers 2024 , 16 , 2788. https://doi.org/10.3390/cancers16162788

Pop MM, Bouros D, Klimko A, Pop LA, Topal P, Topal A, Florian IS. Tumor-Like Lesions in the Craniovertebral Junction: A Case Series, Systematic Review, and Meta-Analysis. Cancers . 2024; 16(16):2788. https://doi.org/10.3390/cancers16162788

Pop, Maria Mihaela, Dragos Bouros, Artsiom Klimko, Laura Ancuta Pop, Paula Topal, Anil Topal, and Ioan Stefan Florian. 2024. "Tumor-Like Lesions in the Craniovertebral Junction: A Case Series, Systematic Review, and Meta-Analysis" Cancers 16, no. 16: 2788. https://doi.org/10.3390/cancers16162788

Article Metrics

Article access statistics, supplementary material.

ZIP-Document (ZIP, 436 KiB)

Further Information

Mdpi initiatives, follow mdpi.

Subscribe to receive issue release notifications and newsletters from MDPI journals

• Advanced search

Advanced Search

First contact physiotherapy: an evaluation of clinical effectiveness and costs

• Find this author on Google Scholar
• Find this author on PubMed
• Search for this author on this site
• ORCID record for Nicola E Walsh
• ORCID record for Serena Halls
• ORCID record for Rachel Thomas
• ORCID record for Alice Berry
• ORCID record for Cathy Liddiard
• ORCID record for Margaret E Cupples
• ORCID record for Heather Gage
• ORCID record for Daniel Jackson
• ORCID record for Fiona Cramp
• ORCID record for Hannah Stott
• ORCID record for Paula Kersten
• ORCID record for Justin Jagosh
• ORCID record for Peter Williams
• Figures & Data

Background First contact physiotherapy practitioners (FCPPs) are embedded within general practice, providing expert assessment, diagnosis, and management plans for patients with musculoskeletal disorders (MSKDs), without the prior need for GP consultation.

Aim To determine the clinical effectiveness and costs of FCPP models compared with GP-led models of care.

Design and setting Multiple site case-study design of general practices in the UK.

Method General practice sites were recruited representing the following three models: 1) GP-led care; 2) FCPPs who could not prescribe or inject (FCPPs-standard [St]); and 3) FCPPs who could prescribe and/or inject (FCPPs-additional qualifications [AQ]). Patient participants from each site completed outcome data at baseline, 3 months, and 6 months. The primary outcome was the SF-36 Physical Component Summary (PCS) score. Healthcare usage was collected for 6 months.

Results In total, 426 adults were recruited from 46 practices across the UK. Non-inferiority analysis showed no significant difference in physical function (SF-36 PCS) across all three arms at 6 months ( P = 0.667). At 3 months, a significant difference in numbers improving was seen between arms: 54.7% ( n = 47) GP consultees, 72.4% ( n = 71) FCPP-St, and 66.4% ( n = 101) FCPP-AQ ( P = 0.037). No safety issues were identified. Following initial consultation, a greater proportion of patients received medication (including opioids) in the GP-led arm (44.7%, n = 42), compared with FCPP-St (18.4%, n = 21) and FCPP-AQ (24.7%, n = 40) ( P <0.001). NHS costs (initial consultation and over 6-month follow-up) were significantly higher in the GP-led model (median £105.5 per patient) versus FCPP-St (£41.0 per patient) and FCPP-AQ (£44.0 per patient) ( P <0.001).

Conclusion FCPP-led models of care provide safe, clinically effective patient management, with cost-benefits and reduced opioid use in this cohort.

• general practice
• physiotherapy
• delivery of health care
• musculoskeletal diseases
• Introduction

General practice is experiencing unprecedented demand for appointments at a time when the number of fully qualified GPs is falling, part-time working is increasing, and average patient caseload is rising. 1 The Additional Roles Reimbursement Scheme was introduced in 2019 with the intention of growing the capacity of the primary care workforce. 2 First contact physiotherapy practitioners (FCPPs) were one of five professional roles initially identified for expedited implementation, 2 in recognition of the growing demands musculoskeletal disorders (MSKDs) place on general practice, which account for up to 30% of consultations. 3 FCPPs have an extended appointment time (normally 20 minutes) to assess, diagnose, and determine the most appropriate interventions and manage onward referral for patients without the prior need for GP consultation. 4 Some FCPPs also have the capability to provide injection therapy, and following legislation change in 2013, licensed physiotherapists can independently prescribe, including, since 2015, some controlled drugs. 5 By 2024, all adults in England consulting with a suspected MSKD should be offered a consultation with a FCPP within their local practice. 6

Since its inception, local service evaluations indicate that FCPPs reduce the need for GP consultation, referral to secondary care services, and prescribed medications, while improving patient and staff satisfaction. 7 The only large-scale evaluation of FCPP was conducted as part of an NHS England national pilot of the initiative and reported against pre-determined criteria including the following: re-consultation rates with the GP; improvements in patient symptoms at 3 months; provision of self-management and/or exercise advice for the condition; and impact on ability to work. 8 Pre-determined criteria were largely successfully met, apart from limited information on presenteeism and the ability to work. While this evaluation provided important data on the potential of FCPP, there was no insight regarding longer-term clinical outcomes, use of healthcare resources, or differences in outcomes compared with traditional GP-led models of care.

The current study aimed to determine the impact of FCPP on clinical outcomes and healthcare resource use for 6 months post-consultation compared with GP-led models of care.

How this fits in

Setting and practice recruitment

General practices across the UK were invited to participate either via expressions of interest in response to a previous survey regarding FCPP provision, 9 or through advertisement via Clinical Research Networks. The aim was to recruit across all four nations, from a range of urban and rural areas, and differing levels of deprivation; deprivation index was based on practice report and confirmed by nationally available data. 10 – 13

Description of services

General practice study sites were categorised into the following three study arms, according to their existing service provision:

no FCPP service: MSKD management with GP-led consultation (‘GP’);

standard FCPP with no additional competencies for prescribing and/or injecting (‘FCPP-St’); and

FCPP with additional qualifications to prescribe and/or inject (‘FCPP-AQ’).

Participant recruitment

Patients who attended appointments for MSKDs in the study sites were given recruitment materials by the clinician or an allocated practice staff member. They were invited to contact the study team for further information, or to express their willingness to participate. Volunteers were screened for eligibility.

The inclusion criteria were as follows: 1) patients consulting with a suspected MSKD episode, defined as any acute or chronic disorder related to the spinal or peripheral musculoskeletal (MSK) system; 2) patients not consulted for the same problem in preceding 3 months; and 3) patients aged ≥18 years. The exclusion criteria were as follows: 1) receiving palliative care; and 2) non-English speaking and unwilling to provide informed consent and communicate through an interpreter.

Eligible participants provided written, informed consent. Recruitment started in December 2019, slowed in January 2020, owing to the emerging COVID-19 pandemic, and paused in March 2020. Recruitment re-started under COVID-19 restrictions in July 2020 and ended in April 2022. Final assessments were completed in October 2022.

Data collection

Information on age, gender, reason for consultation, MSK risk (using STarT MSK), education, and employment were collected by telephone at baseline (post-consultation). Participants were also asked about their consultation experience and any safety concerns (to be reported elsewhere). There were no notable differences across groups.

Questionnaires regarding Patient Reported Outcome Measures (PROMs) were posted to participants following initial consultation (baseline) and at 3 months and 6 months post-consultation. The questionnaires were self-completed and returned by post. The primary outcome measure was the change from baseline to 6 months in the SF-36 Physical Component Summary (PCS) score. 14 Secondary clinical outcomes were SF-36 Mental Component Summary score; Musculoskeletal Health Questionnaire (MSK-HQ, total and physical); perceived safety of health care, using the healthcare experience in general practice survey, short form (Patient Reported Experiences and Outcomes of Safety in Primary Care; PREOS-PC Q5), on a 10-point scale: completely unsafe (0) to completely safe (10); and Roland–Morris Disability Questionnaire (for patients with low back pain). EQ-5D-5L, a generic measure of health-related quality of life, was gathered for use in the economic evaluation. 15

Sample size

The total participants required per arm was 181 across 39 sites. This was based on a non-inferiority margin of 2 units in SF-36 PCS scale, 14 a minimal clinically important difference of 4 points 16 and standard deviation (SD) 6.5, 17 a one-sided P = 0.05 non-inferior hypothesis test, with 80% power, a design effect of 1.09 for a cluster size of 14 and an intraclass correlation coefficient (ICC) of 0.0075, 18 and 20% attrition. COVID-19 impacted recruitment, so figures were revisited. Actual attrition rates were used (5%) and number of sites were increased ( n = 46), which required a total sample size of n = 462 ( n = 154 per arm).

Data analyses

The primary outcome was the change in SF-36 PCS score from baseline to 6 months compared between arms, using a one-way analysis of variance; in case of difference, a post-hoc unpaired t -test was performed. Further comparisons were undertaken in the context of stepwise linear regression modelling, incorporating demographic and clinical data, including baseline SF-36 PCS score. Outcomes from baseline to 3 months are also reported.

Economic analysis

The base case economic analysis adopted an NHS and social care perspective. Information on service use related to the MSK condition was gathered retrospectively by telephone interview at 3 months and 6 months, using a tailored version of the Client Service Receipt Inventory (CSRI). 19 This included: NHS and private healthcare services (primary, community, accident and emergency [A&E], outpatient referrals, and inpatient stays) and social care. Unit costs 20 , 21 were applied to service use and summed (months 1–6) at the participant level, including the cost of the index consultation (see Supplementary Information S1). Group costs were inspected and compared. Owing to the skewed nature of the total costs data, stepwise logistic regression was used to model the presence or absence of additional costs over and above the cost of the initial presentation, with service model as a dummy variable and baseline demographic and clinical factors as covariates. A societal perspective was included through consideration of self-reported days off work and inability to perform usual activities, and the private perspective through out-of-pocket expenditures.

Analyses were carried out using IBM SPSS Statistics (version 27). Database access can be requested via: http://researchdata.uwe.ac.uk/703 .

A total of 426 participants were recruited from 46 general practices across the UK, with a range of deprivation indices and rural or urban locations. Of the 426 participants, there were 110 (25.8%) from GP-led care, 124 (29.1%) from FCPP-St, and 192 (45.1%) from FCPP-AQ. A total of 46 GP practices were involved: 13 GP-led care practices (with 1, 2, 2, 5, 6, 6, 7, 10, 11, 14, 14, 15, and 17 participants), 15 FCPP-St practices (with 1, 3, 3, 3, 4, 4, 5, 7, 7, 9, 9, 14, 15, 17, and 23 participants), and 18 FCPP-AQ practices (with 1, 1, 4, 4, 6, 8, 8, 9, 11, 12, 14, 15, 15, 16, 16, 16, 17, and 19 participants). The study completion rates in each arm for PROMs and CSRIs, along with attrition patterns, can be seen in Supplementary Table S1.

Mean age was 63 years (SD 13.2); 34.1% ( n = 145) were male and 97.8% ( n = 408) reported White ethnicity. There were no statistically significant differences in individual baseline demographics between arms. There was some discrepancy in practice-level deprivation across arms, with a higher representation of low deprived practices in the FCPP-St arm ( Table 1 ). Data were returned at all three time points by 377 (88.5%) participants, including 320 (75.1%) who provided completed PROM and CSRI data. Details of attrition from the study are given in Supplementary Table S1.

• View inline

Baseline demographics: summary statistics with comparison of the three service models

Clinical data revealed no statistically significant differences between arms at baseline, except for the EQ-5D-5L (visual analogue scale [VAS]; better state of health reported in FCPP-St model) and for MSK-HQ total (a more desirable MSK status was indicated in FCPP-St model). Participants reported a range of peripheral and spinal diagnoses (up to two pain sites); given the previously reported high incidence of low back pain in primary care, 18 a 24.9% ( n = 106/426) prevalence was noted ( Table 2 ).

Baseline clinical summary for each of the three service models

Outcomes analysis

The primary outcome variable was the change in SF-36 PCS score from baseline to 6 months; in an unadjusted analysis, no statistically significant difference was found between arms ( Table 3 ). This was confirmed under linear regression, with a final model ( R 2 = 0.138, n = 332) predicting change = 15.074–0.333x (SF-36 PCS score at baseline) + 2.377 (if university educated) + 2.402 (if in full-time employment). Service model along with age at baseline, gender (male: yes/no), ethnic origin (White: yes/no), whether MSKD area at baseline included back (yes/no), whether MSKD area at baseline included knee or leg or hip or foot or ankle (yes/ no), and whether the presented MSK condition had affected employment or ability to perform usual activities (yes/ no) were not significant (see Supplementary Table S2).

Primary and secondary outcome changes from baseline to 3 months and from baseline to 6 months (positive changes indicate improvement)

However, when each of these change outcomes was simplified from the change in continuous score to an improved or worsened/stayed the same scenario, a statistically significant difference between arms was seen in two instances. At 3 months, the FCPP-St and FCPP-AQ service models delivered a statistically significant greater improvement rate for the primary outcome variable SF-36 PCS score compared with the GP-led service model ( P = 0.037). At 6 months, the FCPP-St and FCPP-AQ service models delivered a statistically significant greater improvement rate for the secondary outcome MSK-HQ physical compared with the GP-led service model ( P = 0.016; Table 3 ). No other statistically significant differences in outcomes were found between arms. No safety issues were identified.

Healthcare utilisation and costs

The initial consultation was assumed to be face-to-face with a GP, FCPP-St, or FCPP-AQ. CSRI data were available for 370/426 (86.9%) of participants at 3 months, 348 (81.7%) at 6 months (see Supplementary Table S1). Health service use after the initial consultation was low in all arms, most being within general practice; few participants reported hospital use. Key health service usage (GP and physiotherapist) and prescribing outcomes are shown in Table 4 . In the 3 months following initial consultation, a greater proportion of patients received medication (including opioids) in the GP-led arm (44.7%; n = 42) compared with FCPP-St (18.4%; n = 21) and FCPP-AQ (24.7%; n = 40) (χ 2 P <0.001) . A full breakdown of NHS service use, including medication prescribing, at 3 months and 6 months, is shown in Supplementary Tables S3 and S4. There was scattered use of the private sector while use of over-the-counter medications was commonplace (see Supplementary Tables S5 and S6).

Key self-reported NHS service usages associated with the presenting musculoskeletal condition, not including initial presentation, at 3 months and 6 months

Group mean total costs (health services, excluding medications) over 6-month follow-up for the three service models are shown in Table 5 . Comparisons were performed both excluding and including inpatient (planned MSK surgery) events, and assuming the FCPP-St and FCPP-AQ were both working at salary level band 7; a sensitivity analysis was performed with the FCPP-AQ costed at the higher band 8a. In each comparison, there is a statistically significant difference in costs between the three models ( P <0.001) with the GP model the more costly (median £105.5 per patient versus £41.0 for FCPP-St and £44.0 for FCPP-AQ in the band 7 calculation), and no statistically significant difference between the FCPP-St and FCPP-AQ. In the band 8a comparison, the FCPP-AQ was significantly more costly than the FCPP-St. Regarding days lost through inability to work or perform usual activities, the FCPP-St model showed greater reductions in days lost compared with GP-led care and FCPP-AQ, but there was no statistically significant difference between GP-led care and FCPP-AQ ( Table 6 ). Only eight participants had absences covered by sick notes in the first 3 months and three during the second period (two of which were new).

Total costs (£) summary statistics, months 0–6

Changes in days lost (unable to work or perform usual activities), with comparisons of the three service models

Backwards stepwise logistic regression to model the presence or absence of additional health service costs in months 0–6 over and above the initial presentation (excluding inpatient), with re-running of the final model to include additional participants for whom data were missing only for non-significant predictors, led to the model in Supplementary Table S2 (with Nagelkerke R 2 = 0.072, n = 334). The model demonstrates a significantly (2.181 times) higher likelihood of incurring additional costs after the initial consultation with a GP-led service model compared with a FCPP-St or FCPP-AQ service model. Higher scores in baseline SF-36 PCS score are also significantly associated with a lower likelihood of incurring additional cost (adjusted odds ratio of 0.966 implies that a participant with a baseline SF-36 PCS score, which is 10 points higher than another participant, is 0.966 10 = 0.708 times less likely to incur additional cost). No other predictors were statistically significant.

The analysis demonstrated that neither FCPP model was inferior in relation to clinical outcome at 6-month post-consultation compared with the GP-led model, but both were significantly less costly; P <0.001. There were no significant differences in quality-of-life changes (based on EQ-5D-5L) between the models at 3 months or 6 months, so given the cost differentials, no formal cost-effectiveness analysis was undertaken ( Tables 3 and 5 ).

Analysis demonstrated no statistically significant difference in clinical outcomes between different service models after 6 months. However, the GP-led model of care was approximately 2.5 times costlier than the FCPP-St and FCPP-AQ models. Furthermore, at 3 months, a greater proportion of patients who consulted with FCPPs had improved, compared with those who had consulted with GPs, and time off work or unable to perform usual activities was reduced in the FCPP-St consultees.

Strengths and limitations

To the authors’ knowledge, this is the first study that has compared GP-and FCPP-led models of care for MSKDs and included data from all four UK nations. It provides a robust overview of the service innovation to support decision making, and a qualitative analysis, which was conducted concurrently, will allow further interpretation of findings.

Recruitment was severely hampered by the COVID-19 pandemic, yet this study still provides the most extensive dataset of FCPPs to date. There was uneven recruitment across study arms and sites because the drive for FCPP recruitment, resulting from the Additional Roles Reimbursement Scheme, made the identification of GP-led sites challenging; and recruitment within some individual sites was lower than anticipated. At site level, there was some variation in deprivation across arms: the FCPP-St consisted of relatively more practices with lower levels of deprivation compared with the other arms, which may explain the higher levels of quality of life (EQ-5D-5L [VAS] and MSK-HQ) reported at baseline within this arm. However, while these differences were of statistical significance, neither was of clinical significance, based on previously reported levels of minimum clinical important difference 23 , 24 and, importantly, there was no difference in the primary outcome measure at baseline across arms. All sites that expressed an interest in participation were recruited, so this variation did not result from selective recruitment. Furthermore, at the level of individual participants, no significant differences were found between groups regarding levels of education or employment.

The sample was almost exclusively White and not representative of practice cohorts despite efforts for diverse recruitment at practice and patient level. Only 12/46 (26.1%) sites returned requested data regarding numbers invited to participate in the study, so how representative the study sample is of those eligible is unable to be reported. Much of the recruitment was undertaken under COVID-19 restrictions, which disproportionately impacted people of ethnic minority heritage, which may have influenced decision to participate, although in consultation with recruitment sites, it was identified that fewer people from ethnic minority communities consult FCPP staff. There was potential recruitment bias as not all eligible participants consented to join the study.

Comparison with existing literature

To the authors’ knowledge, this is the first study to show a comparison between GP and FCPP clinical outcomes and resource use, confirming the proposed benefits of the new model of care. While at 6 months there were no differences in patient improvement across the models studied, at 3 months a significantly greater proportion of patients who consulted with FCPPs had improved compared with GP consultees, with positive impact on ability to work or perform usual activities in FCPP-St ( P = 0.005). Previous work highlighted GP propensity for pharmacological management rather than guideline-based self-management and rehabilitation strategies, which may account for these differences; 25 – 27 indeed, a greater proportion of patients under GP-led care were prescribed medication, including opioid derivatives. The authors are unable to identify any factors in the study design that would account for this finding and believe this is a result of clinical decision making. Other work has shown that FCPPs with a licence to prescribe are still reluctant to use this intervention, instead choosing to use their capability to deprescribe where possible and intervene with non-pharmacological measures. 28

From an onward resource use perspective, data showed minimal reliance on other services within each model and therefore relatively low costs. For services that were used, there was a greater number of referrals onto outpatient physiotherapy by GPs, as would be expected; other work has suggested GP overuse of magnetic resonance imaging (MRI), but this was not found. 29 These data were obtained through self-report so may have been subject to recall bias. It is noted, however, that other studies report the similarities in self-report versus medical record review, and in some cases note greater accuracy with patient recall. 30

A previous evaluation in England reported that GP workload was positively impacted by FCPPs. It found most patients did not consult their GP with the same problem within 3 months of seeing the FCPP. 8 This concurs with the present study’s findings that only 23/276 (8.3%) of patients consulted the GP for the same problem having seen the FCPP, whereas many more (30.9%) initial GP consultees re-consulted the GP for the same problem within the study period ( Table 4 ).

A predominant aim of introducing FCPPs is to make better use of resources in general practice. The present study shows clear cost benefits to implementing FCPP models compared with GP-led care given the extent of MSKD consultations in primary care. 3

Implications for research and practice

This research supports continued implementation of FCPP in general practice as a safe, clinically effective, and cost-beneficial approach to managing people with MSKDs. Given FCPPs’ low reliance on prescription medications, it may also assist in reducing opioid prescriptions in primary care. Further research is required to understand why there appears to be disproportionate consultations from people of ethnic minority heritage to ensure appropriate access for all.

• Acknowledgments

The FRONTIER team would like to thank all participants for their time and valuable contribution to the study. The authors would also like to thank Gemma Artz, Pete Young, Jude Hancock, Alison Diaper, the Study Steering Committee, and the research team at Bristol, North Somerset and South Gloucestershire Integrated Care Board for their expertise and support.

This study was funded by the National Institute for Health and Care Research (NIHR) Health and Social Care Delivery Research Programme (reference: 16/116/03). The views expressed are those of the authors and not necessarily those of the NIHR or the Department of Health and Social Care.

Ethical approval

Granted on 18 June 2019 (Integrated Research Application System ID: 261530; Research Ethics Committee reference: 19/NI/0108). Health Research Authority approval was granted on 25 June 2019.

University of the West of England Database Repository Database access can be requested via: http://researchdata.uwe.ac.uk/703 .

Freely submitted; externally peer reviewed.

Competing interests

The authors have declared no competing interests.

Discuss this article:

bjgp.org/letters

• Received October 25, 2023.
• Revision requested November 28, 2023.
• Accepted January 22, 2024.
• © The Authors

This article is Open Access: CC BY 4.0 licence ( http://creativecommons.org/licences/by/4.0/ ).

• British Medical Association
• NHS England, NHS Improvement
• NHS England
• Chartered Society of Physiotherapy
• Home Office
• Jordan KP ,
• 10. ↵ Ministry of Housing, Communities & Local Government . The English Indices of Deprivation 2019 Frequently asked questions (FAQs) , https://assets.publishing.service.gov.uk/media/5dfb3d7ce5274a3432700cf3/IoD2019_FAQ_v4.pdf (accessed 29 Jul 2024).
• Scottish Government
• Northern Ireland Statistics and Research Agency
• Kosinski M ,
• Bjorner JB ,
• EuroQol Research Foundation
• Aeschlimann A ,
• Ogollah RO ,
• Salisbury C ,
• Montgomery AA ,
• Hollinghurst S ,
• Personal Social Service Research Unit (PSSRU)
• Devlin NJ ,
• Reneman MF ,
• Speijer BLGN ,
• Lancaster G ,
• Hayward R ,
• Wallis JA ,
• Ackerman IN ,
• Brusco NK ,
• Smithson J ,
• Parkunan A ,
• Wallace E ,
• Moriarty F ,
• McGarrigle C ,

Online First

Thank you for recommending British Journal of General Practice.

NOTE: We only request your email address so that the person to whom you are recommending the page knows that you wanted them to see it, and that it is not junk mail. We do not capture any email address.

Citation Manager Formats

• EndNote (tagged)
• EndNote 8 (xml)
• RefWorks Tagged
• Ref Manager

• Tweet Widget
• Facebook Like
• Google Plus One

Jump to section

More in this toc section.

• Media depictions of primary care teleconsultation safety: a thematic analysis of UK newspapers
• Unpacking complexity in addressing the contribution of trauma to women’s ill health: a qualitative study of perspectives from general practice
• Trends in full-time working in general practice: a repeated cross-sectional study

Related Articles

Cited by....

British Journal of General Practice

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• Int J Qual Stud Health Well-being

Methodology or method? A critical review of qualitative case study reports

Despite on-going debate about credibility, and reported limitations in comparison to other approaches, case study is an increasingly popular approach among qualitative researchers. We critically analysed the methodological descriptions of published case studies. Three high-impact qualitative methods journals were searched to locate case studies published in the past 5 years; 34 were selected for analysis. Articles were categorized as health and health services ( n= 12), social sciences and anthropology ( n= 7), or methods ( n= 15) case studies. The articles were reviewed using an adapted version of established criteria to determine whether adequate methodological justification was present, and if study aims, methods, and reported findings were consistent with a qualitative case study approach. Findings were grouped into five themes outlining key methodological issues: case study methodology or method, case of something particular and case selection, contextually bound case study, researcher and case interactions and triangulation, and study design inconsistent with methodology reported. Improved reporting of case studies by qualitative researchers will advance the methodology for the benefit of researchers and practitioners.

Case study research is an increasingly popular approach among qualitative researchers (Thomas, 2011 ). Several prominent authors have contributed to methodological developments, which has increased the popularity of case study approaches across disciplines (Creswell, 2013b ; Denzin & Lincoln, 2011b ; Merriam, 2009 ; Ragin & Becker, 1992 ; Stake, 1995 ; Yin, 2009 ). Current qualitative case study approaches are shaped by paradigm, study design, and selection of methods, and, as a result, case studies in the published literature vary. Differences between published case studies can make it difficult for researchers to define and understand case study as a methodology.

Experienced qualitative researchers have identified case study research as a stand-alone qualitative approach (Denzin & Lincoln, 2011b ). Case study research has a level of flexibility that is not readily offered by other qualitative approaches such as grounded theory or phenomenology. Case studies are designed to suit the case and research question and published case studies demonstrate wide diversity in study design. There are two popular case study approaches in qualitative research. The first, proposed by Stake ( 1995 ) and Merriam ( 2009 ), is situated in a social constructivist paradigm, whereas the second, by Yin ( 2012 ), Flyvbjerg ( 2011 ), and Eisenhardt ( 1989 ), approaches case study from a post-positivist viewpoint. Scholarship from both schools of inquiry has contributed to the popularity of case study and development of theoretical frameworks and principles that characterize the methodology.

The diversity of case studies reported in the published literature, and on-going debates about credibility and the use of case study in qualitative research practice, suggests that differences in perspectives on case study methodology may prevent researchers from developing a mutual understanding of practice and rigour. In addition, discussion about case study limitations has led some authors to query whether case study is indeed a methodology (Luck, Jackson, & Usher, 2006 ; Meyer, 2001 ; Thomas, 2010 ; Tight, 2010 ). Methodological discussion of qualitative case study research is timely, and a review is required to analyse and understand how this methodology is applied in the qualitative research literature. The aims of this study were to review methodological descriptions of published qualitative case studies, to review how the case study methodological approach was applied, and to identify issues that need to be addressed by researchers, editors, and reviewers. An outline of the current definitions of case study and an overview of the issues proposed in the qualitative methodological literature are provided to set the scene for the review.

Definitions of qualitative case study research

Case study research is an investigation and analysis of a single or collective case, intended to capture the complexity of the object of study (Stake, 1995 ). Qualitative case study research, as described by Stake ( 1995 ), draws together “naturalistic, holistic, ethnographic, phenomenological, and biographic research methods” in a bricoleur design, or in his words, “a palette of methods” (Stake, 1995 , pp. xi–xii). Case study methodology maintains deep connections to core values and intentions and is “particularistic, descriptive and heuristic” (Merriam, 2009 , p. 46).

As a study design, case study is defined by interest in individual cases rather than the methods of inquiry used. The selection of methods is informed by researcher and case intuition and makes use of naturally occurring sources of knowledge, such as people or observations of interactions that occur in the physical space (Stake, 1998 ). Thomas ( 2011 ) suggested that “analytical eclecticism” is a defining factor (p. 512). Multiple data collection and analysis methods are adopted to further develop and understand the case, shaped by context and emergent data (Stake, 1995 ). This qualitative approach “explores a real-life, contemporary bounded system (a case ) or multiple bounded systems (cases) over time, through detailed, in-depth data collection involving multiple sources of information … and reports a case description and case themes ” (Creswell, 2013b , p. 97). Case study research has been defined by the unit of analysis, the process of study, and the outcome or end product, all essentially the case (Merriam, 2009 ).

The case is an object to be studied for an identified reason that is peculiar or particular. Classification of the case and case selection procedures informs development of the study design and clarifies the research question. Stake ( 1995 ) proposed three types of cases and study design frameworks. These include the intrinsic case, the instrumental case, and the collective instrumental case. The intrinsic case is used to understand the particulars of a single case, rather than what it represents. An instrumental case study provides insight on an issue or is used to refine theory. The case is selected to advance understanding of the object of interest. A collective refers to an instrumental case which is studied as multiple, nested cases, observed in unison, parallel, or sequential order. More than one case can be simultaneously studied; however, each case study is a concentrated, single inquiry, studied holistically in its own entirety (Stake, 1995 , 1998 ).

Researchers who use case study are urged to seek out what is common and what is particular about the case. This involves careful and in-depth consideration of the nature of the case, historical background, physical setting, and other institutional and political contextual factors (Stake, 1998 ). An interpretive or social constructivist approach to qualitative case study research supports a transactional method of inquiry, where the researcher has a personal interaction with the case. The case is developed in a relationship between the researcher and informants, and presented to engage the reader, inviting them to join in this interaction and in case discovery (Stake, 1995 ). A postpositivist approach to case study involves developing a clear case study protocol with careful consideration of validity and potential bias, which might involve an exploratory or pilot phase, and ensures that all elements of the case are measured and adequately described (Yin, 2009 , 2012 ).

Current methodological issues in qualitative case study research

The future of qualitative research will be influenced and constructed by the way research is conducted, and by what is reviewed and published in academic journals (Morse, 2011 ). If case study research is to further develop as a principal qualitative methodological approach, and make a valued contribution to the field of qualitative inquiry, issues related to methodological credibility must be considered. Researchers are required to demonstrate rigour through adequate descriptions of methodological foundations. Case studies published without sufficient detail for the reader to understand the study design, and without rationale for key methodological decisions, may lead to research being interpreted as lacking in quality or credibility (Hallberg, 2013 ; Morse, 2011 ).

There is a level of artistic license that is embraced by qualitative researchers and distinguishes practice, which nurtures creativity, innovation, and reflexivity (Denzin & Lincoln, 2011b ; Morse, 2009 ). Qualitative research is “inherently multimethod” (Denzin & Lincoln, 2011a , p. 5); however, with this creative freedom, it is important for researchers to provide adequate description for methodological justification (Meyer, 2001 ). This includes paradigm and theoretical perspectives that have influenced study design. Without adequate description, study design might not be understood by the reader, and can appear to be dishonest or inaccurate. Reviewers and readers might be confused by the inconsistent or inappropriate terms used to describe case study research approach and methods, and be distracted from important study findings (Sandelowski, 2000 ). This issue extends beyond case study research, and others have noted inconsistencies in reporting of methodology and method by qualitative researchers. Sandelowski ( 2000 , 2010 ) argued for accurate identification of qualitative description as a research approach. She recommended that the selected methodology should be harmonious with the study design, and be reflected in methods and analysis techniques. Similarly, Webb and Kevern ( 2000 ) uncovered inconsistencies in qualitative nursing research with focus group methods, recommending that methodological procedures must cite seminal authors and be applied with respect to the selected theoretical framework. Incorrect labelling using case study might stem from the flexibility in case study design and non-directional character relative to other approaches (Rosenberg & Yates, 2007 ). Methodological integrity is required in design of qualitative studies, including case study, to ensure study rigour and to enhance credibility of the field (Morse, 2011 ).

Case study has been unnecessarily devalued by comparisons with statistical methods (Eisenhardt, 1989 ; Flyvbjerg, 2006 , 2011 ; Jensen & Rodgers, 2001 ; Piekkari, Welch, & Paavilainen, 2009 ; Tight, 2010 ; Yin, 1999 ). It is reputed to be the “the weak sibling” in comparison to other, more rigorous, approaches (Yin, 2009 , p. xiii). Case study is not an inherently comparative approach to research. The objective is not statistical research, and the aim is not to produce outcomes that are generalizable to all populations (Thomas, 2011 ). Comparisons between case study and statistical research do little to advance this qualitative approach, and fail to recognize its inherent value, which can be better understood from the interpretive or social constructionist viewpoint of other authors (Merriam, 2009 ; Stake, 1995 ). Building on discussions relating to “fuzzy” (Bassey, 2001 ), or naturalistic generalizations (Stake, 1978 ), or transference of concepts and theories (Ayres, Kavanaugh, & Knafl, 2003 ; Morse et al., 2011 ) would have more relevance.

Case study research has been used as a catch-all design to justify or add weight to fundamental qualitative descriptive studies that do not fit with other traditional frameworks (Merriam, 2009 ). A case study has been a “convenient label for our research—when we ‘can't think of anything ‘better”—in an attempt to give it [qualitative methodology] some added respectability” (Tight, 2010 , p. 337). Qualitative case study research is a pliable approach (Merriam, 2009 ; Meyer, 2001 ; Stake, 1995 ), and has been likened to a “curious methodological limbo” (Gerring, 2004 , p. 341) or “paradigmatic bridge” (Luck et al., 2006 , p. 104), that is on the borderline between postpositivist and constructionist interpretations. This has resulted in inconsistency in application, which indicates that flexibility comes with limitations (Meyer, 2001 ), and the open nature of case study research might be off-putting to novice researchers (Thomas, 2011 ). The development of a well-(in)formed theoretical framework to guide a case study should improve consistency, rigour, and trust in studies published in qualitative research journals (Meyer, 2001 ).

Assessment of rigour

The purpose of this study was to analyse the methodological descriptions of case studies published in qualitative methods journals. To do this we needed to develop a suitable framework, which used existing, established criteria for appraising qualitative case study research rigour (Creswell, 2013b ; Merriam, 2009 ; Stake, 1995 ). A number of qualitative authors have developed concepts and criteria that are used to determine whether a study is rigorous (Denzin & Lincoln, 2011b ; Lincoln, 1995 ; Sandelowski & Barroso, 2002 ). The criteria proposed by Stake ( 1995 ) provide a framework for readers and reviewers to make judgements regarding case study quality, and identify key characteristics essential for good methodological rigour. Although each of the factors listed in Stake's criteria could enhance the quality of a qualitative research report, in Table I we present an adapted criteria used in this study, which integrates more recent work by Merriam ( 2009 ) and Creswell ( 2013b ). Stake's ( 1995 ) original criteria were separated into two categories. The first list of general criteria is “relevant for all qualitative research.” The second list, “high relevance to qualitative case study research,” was the criteria that we decided had higher relevance to case study research. This second list was the main criteria used to assess the methodological descriptions of the case studies reviewed. The complete table has been preserved so that the reader can determine how the original criteria were adapted.

Framework for assessing quality in qualitative case study research.

Adapted from Stake ( 1995 , p. 131).

Study design

The critical review method described by Grant and Booth ( 2009 ) was used, which is appropriate for the assessment of research quality, and is used for literature analysis to inform research and practice. This type of review goes beyond the mapping and description of scoping or rapid reviews, to include “analysis and conceptual innovation” (Grant & Booth, 2009 , p. 93). A critical review is used to develop existing, or produce new, hypotheses or models. This is different to systematic reviews that answer clinical questions. It is used to evaluate existing research and competing ideas, to provide a “launch pad” for conceptual development and “subsequent testing” (Grant & Booth, 2009 , p. 93).

Qualitative methods journals were located by a search of the 2011 ISI Journal Citation Reports in Social Science, via the database Web of Knowledge (see m.webofknowledge.com). No “qualitative research methods” category existed in the citation reports; therefore, a search of all categories was performed using the term “qualitative.” In Table II , we present the qualitative methods journals located, ranked by impact factor. The highest ranked journals were selected for searching. We acknowledge that the impact factor ranking system might not be the best measure of journal quality (Cheek, Garnham, & Quan, 2006 ); however, this was the most appropriate and accessible method available.

International Journal of Qualitative Studies on Health and Well-being.

Search strategy

In March 2013, searches of the journals, Qualitative Health Research , Qualitative Research , and Qualitative Inquiry were completed to retrieve studies with “case study” in the abstract field. The search was limited to the past 5 years (1 January 2008 to 1 March 2013). The objective was to locate published qualitative case studies suitable for assessment using the adapted criterion. Viewpoints, commentaries, and other article types were excluded from review. Title and abstracts of the 45 retrieved articles were read by the first author, who identified 34 empirical case studies for review. All authors reviewed the 34 studies to confirm selection and categorization. In Table III , we present the 34 case studies grouped by journal, and categorized by research topic, including health sciences, social sciences and anthropology, and methods research. There was a discrepancy in categorization of one article on pedagogy and a new teaching method published in Qualitative Inquiry (Jorrín-Abellán, Rubia-Avi, Anguita-Martínez, Gómez-Sánchez, & Martínez-Mones, 2008 ). Consensus was to allocate to the methods category.

Outcomes of search of qualitative methods journals.

In Table III , the number of studies located, and final numbers selected for review have been reported. Qualitative Health Research published the most empirical case studies ( n= 16). In the health category, there were 12 case studies of health conditions, health services, and health policy issues, all published in Qualitative Health Research . Seven case studies were categorized as social sciences and anthropology research, which combined case study with biography and ethnography methodologies. All three journals published case studies on methods research to illustrate a data collection or analysis technique, methodological procedure, or related issue.

The methodological descriptions of 34 case studies were critically reviewed using the adapted criteria. All articles reviewed contained a description of study methods; however, the length, amount of detail, and position of the description in the article varied. Few studies provided an accurate description and rationale for using a qualitative case study approach. In the 34 case studies reviewed, three described a theoretical framework informed by Stake ( 1995 ), two by Yin ( 2009 ), and three provided a mixed framework informed by various authors, which might have included both Yin and Stake. Few studies described their case study design, or included a rationale that explained why they excluded or added further procedures, and whether this was to enhance the study design, or to better suit the research question. In 26 of the studies no reference was provided to principal case study authors. From reviewing the description of methods, few authors provided a description or justification of case study methodology that demonstrated how their study was informed by the methodological literature that exists on this approach.

The methodological descriptions of each study were reviewed using the adapted criteria, and the following issues were identified: case study methodology or method; case of something particular and case selection; contextually bound case study; researcher and case interactions and triangulation; and, study design inconsistent with methodology. An outline of how the issues were developed from the critical review is provided, followed by a discussion of how these relate to the current methodological literature.

Case study methodology or method

A third of the case studies reviewed appeared to use a case report method, not case study methodology as described by principal authors (Creswell, 2013b ; Merriam, 2009 ; Stake, 1995 ; Yin, 2009 ). Case studies were identified as a case report because of missing methodological detail and by review of the study aims and purpose. These reports presented data for small samples of no more than three people, places or phenomenon. Four studies, or “case reports” were single cases selected retrospectively from larger studies (Bronken, Kirkevold, Martinsen, & Kvigne, 2012 ; Coltart & Henwood, 2012 ; Hooghe, Neimeyer, & Rober, 2012 ; Roscigno et al., 2012 ). Case reports were not a case of something, instead were a case demonstration or an example presented in a report. These reports presented outcomes, and reported on how the case could be generalized. Descriptions focussed on the phenomena, rather than the case itself, and did not appear to study the case in its entirety.

Case reports had minimal in-text references to case study methodology, and were informed by other qualitative traditions or secondary sources (Adamson & Holloway, 2012 ; Buzzanell & D'Enbeau, 2009 ; Nagar-Ron & Motzafi-Haller, 2011 ). This does not suggest that case study methodology cannot be multimethod, however, methodology should be consistent in design, be clearly described (Meyer, 2001 ; Stake, 1995 ), and maintain focus on the case (Creswell, 2013b ).

To demonstrate how case reports were identified, three examples are provided. The first, Yeh ( 2013 ) described their study as, “the examination of the emergence of vegetarianism in Victorian England serves as a case study to reveal the relationships between boundaries and entities” (p. 306). The findings were a historical case report, which resulted from an ethnographic study of vegetarianism. Cunsolo Willox, Harper, Edge, ‘My Word’: Storytelling and Digital Media Lab, and Rigolet Inuit Community Government (2013) used “a case study that illustrates the usage of digital storytelling within an Inuit community” (p. 130). This case study reported how digital storytelling can be used with indigenous communities as a participatory method to illuminate the benefits of this method for other studies. This “case study was conducted in the Inuit community” but did not include the Inuit community in case analysis (Cunsolo Willox et al., 2013 , p. 130). Bronken et al. ( 2012 ) provided a single case report to demonstrate issues observed in a larger clinical study of aphasia and stroke, without adequate case description or analysis.

Case study of something particular and case selection

Case selection is a precursor to case analysis, which needs to be presented as a convincing argument (Merriam, 2009 ). Descriptions of the case were often not adequate to ascertain why the case was selected, or whether it was a particular exemplar or outlier (Thomas, 2011 ). In a number of case studies in the health and social science categories, it was not explicit whether the case was of something particular, or peculiar to their discipline or field (Adamson & Holloway, 2012 ; Bronken et al., 2012 ; Colón-Emeric et al., 2010 ; Jackson, Botelho, Welch, Joseph, & Tennstedt, 2012 ; Mawn et al., 2010 ; Snyder-Young, 2011 ). There were exceptions in the methods category ( Table III ), where cases were selected by researchers to report on a new or innovative method. The cases emerged through heuristic study, and were reported to be particular, relative to the existing methods literature (Ajodhia-Andrews & Berman, 2009 ; Buckley & Waring, 2013 ; Cunsolo Willox et al., 2013 ; De Haene, Grietens, & Verschueren, 2010 ; Gratton & O'Donnell, 2011 ; Sumsion, 2013 ; Wimpenny & Savin-Baden, 2012 ).

Case selection processes were sometimes insufficient to understand why the case was selected from the global population of cases, or what study of this case would contribute to knowledge as compared with other possible cases (Adamson & Holloway, 2012 ; Bronken et al., 2012 ; Colón-Emeric et al., 2010 ; Jackson et al., 2012 ; Mawn et al., 2010 ). In two studies, local cases were selected (Barone, 2010 ; Fourie & Theron, 2012 ) because the researcher was familiar with and had access to the case. Possible limitations of a convenience sample were not acknowledged. Purposeful sampling was used to recruit participants within the case of one study, but not of the case itself (Gallagher et al., 2013 ). Random sampling was completed for case selection in two studies (Colón-Emeric et al., 2010 ; Jackson et al., 2012 ), which has limited meaning in interpretive qualitative research.

To demonstrate how researchers provided a good justification for the selection of case study approaches, four examples are provided. The first, cases of residential care homes, were selected because of reported occurrences of mistreatment, which included residents being locked in rooms at night (Rytterström, Unosson, & Arman, 2013 ). Roscigno et al. ( 2012 ) selected cases of parents who were admitted for early hospitalization in neonatal intensive care with a threatened preterm delivery before 26 weeks. Hooghe et al. ( 2012 ) used random sampling to select 20 couples that had experienced the death of a child; however, the case study was of one couple and a particular metaphor described only by them. The final example, Coltart and Henwood ( 2012 ), provided a detailed account of how they selected two cases from a sample of 46 fathers based on personal characteristics and beliefs. They described how the analysis of the two cases would contribute to their larger study on first time fathers and parenting.

Contextually bound case study

The limits or boundaries of the case are a defining factor of case study methodology (Merriam, 2009 ; Ragin & Becker, 1992 ; Stake, 1995 ; Yin, 2009 ). Adequate contextual description is required to understand the setting or context in which the case is revealed. In the health category, case studies were used to illustrate a clinical phenomenon or issue such as compliance and health behaviour (Colón-Emeric et al., 2010 ; D'Enbeau, Buzzanell, & Duckworth, 2010 ; Gallagher et al., 2013 ; Hooghe et al., 2012 ; Jackson et al., 2012 ; Roscigno et al., 2012 ). In these case studies, contextual boundaries, such as physical and institutional descriptions, were not sufficient to understand the case as a holistic system, for example, the general practitioner (GP) clinic in Gallagher et al. ( 2013 ), or the nursing home in Colón-Emeric et al. ( 2010 ). Similarly, in the social science and methods categories, attention was paid to some components of the case context, but not others, missing important information required to understand the case as a holistic system (Alexander, Moreira, & Kumar, 2012 ; Buzzanell & D'Enbeau, 2009 ; Nairn & Panelli, 2009 ; Wimpenny & Savin-Baden, 2012 ).

In two studies, vicarious experience or vignettes (Nairn & Panelli, 2009 ) and images (Jorrín-Abellán et al., 2008 ) were effective to support description of context, and might have been a useful addition for other case studies. Missing contextual boundaries suggests that the case might not be adequately defined. Additional information, such as the physical, institutional, political, and community context, would improve understanding of the case (Stake, 1998 ). In Boxes 1 and 2 , we present brief synopses of two studies that were reviewed, which demonstrated a well bounded case. In Box 1 , Ledderer ( 2011 ) used a qualitative case study design informed by Stake's tradition. In Box 2 , Gillard, Witt, and Watts ( 2011 ) were informed by Yin's tradition. By providing a brief outline of the case studies in Boxes 1 and 2 , we demonstrate how effective case boundaries can be constructed and reported, which may be of particular interest to prospective case study researchers.

Article synopsis of case study research using Stake's tradition

Ledderer ( 2011 ) used a qualitative case study research design, informed by modern ethnography. The study is bounded to 10 general practice clinics in Denmark, who had received federal funding to implement preventative care services based on a Motivational Interviewing intervention. The researcher question focussed on “why is it so difficult to create change in medical practice?” (Ledderer, 2011 , p. 27). The study context was adequately described, providing detail on the general practitioner (GP) clinics and relevant political and economic influences. Methodological decisions are described in first person narrative, providing insight on researcher perspectives and interaction with the case. Forty-four interviews were conducted, which focussed on how GPs conducted consultations, and the form, nature and content, rather than asking their opinion or experience (Ledderer, 2011 , p. 30). The duration and intensity of researcher immersion in the case enhanced depth of description and trustworthiness of study findings. Analysis was consistent with Stake's tradition, and the researcher provided examples of inquiry techniques used to challenge assumptions about emerging themes. Several other seminal qualitative works were cited. The themes and typology constructed are rich in narrative data and storytelling by clinic staff, demonstrating individual clinic experiences as well as shared meanings and understandings about changing from a biomedical to psychological approach to preventative health intervention. Conclusions make note of social and cultural meanings and lessons learned, which might not have been uncovered using a different methodology.

Article synopsis of case study research using Yin's tradition

Gillard et al. ( 2011 ) study of camps for adolescents living with HIV/AIDs provided a good example of Yin's interpretive case study approach. The context of the case is bounded by the three summer camps of which the researchers had prior professional involvement. A case study protocol was developed that used multiple methods to gather information at three data collection points coinciding with three youth camps (Teen Forum, Discover Camp, and Camp Strong). Gillard and colleagues followed Yin's ( 2009 ) principles, using a consistent data protocol that enhanced cross-case analysis. Data described the young people, the camp physical environment, camp schedule, objectives and outcomes, and the staff of three youth camps. The findings provided a detailed description of the context, with less detail of individual participants, including insight into researcher's interpretations and methodological decisions throughout the data collection and analysis process. Findings provided the reader with a sense of “being there,” and are discovered through constant comparison of the case with the research issues; the case is the unit of analysis. There is evidence of researcher immersion in the case, and Gillard reports spending significant time in the field in a naturalistic and integrated youth mentor role.

This case study is not intended to have a significant impact on broader health policy, although does have implications for health professionals working with adolescents. Study conclusions will inform future camps for young people with chronic disease, and practitioners are able to compare similarities between this case and their own practice (for knowledge translation). No limitations of this article were reported. Limitations related to publication of this case study were that it was 20 pages long and used three tables to provide sufficient description of the camp and program components, and relationships with the research issue.

Researcher and case interactions and triangulation

Researcher and case interactions and transactions are a defining feature of case study methodology (Stake, 1995 ). Narrative stories, vignettes, and thick description are used to provoke vicarious experience and a sense of being there with the researcher in their interaction with the case. Few of the case studies reviewed provided details of the researcher's relationship with the case, researcher–case interactions, and how these influenced the development of the case study (Buzzanell & D'Enbeau, 2009 ; D'Enbeau et al., 2010 ; Gallagher et al., 2013 ; Gillard et al., 2011 ; Ledderer, 2011 ; Nagar-Ron & Motzafi-Haller, 2011 ). The role and position of the researcher needed to be self-examined and understood by readers, to understand how this influenced interactions with participants, and to determine what triangulation is needed (Merriam, 2009 ; Stake, 1995 ).

Gillard et al. ( 2011 ) provided a good example of triangulation, comparing data sources in a table (p. 1513). Triangulation of sources was used to reveal as much depth as possible in the study by Nagar-Ron and Motzafi-Haller ( 2011 ), while also enhancing confirmation validity. There were several case studies that would have benefited from improved range and use of data sources, and descriptions of researcher–case interactions (Ajodhia-Andrews & Berman, 2009 ; Bronken et al., 2012 ; Fincham, Scourfield, & Langer, 2008 ; Fourie & Theron, 2012 ; Hooghe et al., 2012 ; Snyder-Young, 2011 ; Yeh, 2013 ).

Study design inconsistent with methodology

Good, rigorous case studies require a strong methodological justification (Meyer, 2001 ) and a logical and coherent argument that defines paradigm, methodological position, and selection of study methods (Denzin & Lincoln, 2011b ). Methodological justification was insufficient in several of the studies reviewed (Barone, 2010 ; Bronken et al., 2012 ; Hooghe et al., 2012 ; Mawn et al., 2010 ; Roscigno et al., 2012 ; Yeh, 2013 ). This was judged by the absence, or inadequate or inconsistent reference to case study methodology in-text.

In six studies, the methodological justification provided did not relate to case study. There were common issues identified. Secondary sources were used as primary methodological references indicating that study design might not have been theoretically sound (Colón-Emeric et al., 2010 ; Coltart & Henwood, 2012 ; Roscigno et al., 2012 ; Snyder-Young, 2011 ). Authors and sources cited in methodological descriptions were inconsistent with the actual study design and practices used (Fourie & Theron, 2012 ; Hooghe et al., 2012 ; Jorrín-Abellán et al., 2008 ; Mawn et al., 2010 ; Rytterström et al., 2013 ; Wimpenny & Savin-Baden, 2012 ). This occurred when researchers cited Stake or Yin, or both (Mawn et al., 2010 ; Rytterström et al., 2013 ), although did not follow their paradigmatic or methodological approach. In 26 studies there were no citations for a case study methodological approach.

The findings of this study have highlighted a number of issues for researchers. A considerable number of case studies reviewed were missing key elements that define qualitative case study methodology and the tradition cited. A significant number of studies did not provide a clear methodological description or justification relevant to case study. Case studies in health and social sciences did not provide sufficient information for the reader to understand case selection, and why this case was chosen above others. The context of the cases were not described in adequate detail to understand all relevant elements of the case context, which indicated that cases may have not been contextually bounded. There were inconsistencies between reported methodology, study design, and paradigmatic approach in case studies reviewed, which made it difficult to understand the study methodology and theoretical foundations. These issues have implications for methodological integrity and honesty when reporting study design, which are values of the qualitative research tradition and are ethical requirements (Wager & Kleinert, 2010a ). Poorly described methodological descriptions may lead the reader to misinterpret or discredit study findings, which limits the impact of the study, and, as a collective, hinders advancements in the broader qualitative research field.

The issues highlighted in our review build on current debates in the case study literature, and queries about the value of this methodology. Case study research can be situated within different paradigms or designed with an array of methods. In order to maintain the creativity and flexibility that is valued in this methodology, clearer descriptions of paradigm and theoretical position and methods should be provided so that study findings are not undervalued or discredited. Case study research is an interdisciplinary practice, which means that clear methodological descriptions might be more important for this approach than other methodologies that are predominantly driven by fewer disciplines (Creswell, 2013b ).

Authors frequently omit elements of methodologies and include others to strengthen study design, and we do not propose a rigid or purist ideology in this paper. On the contrary, we encourage new ideas about using case study, together with adequate reporting, which will advance the value and practice of case study. The implications of unclear methodological descriptions in the studies reviewed were that study design appeared to be inconsistent with reported methodology, and key elements required for making judgements of rigour were missing. It was not clear whether the deviations from methodological tradition were made by researchers to strengthen the study design, or because of misinterpretations. Morse ( 2011 ) recommended that innovations and deviations from practice are best made by experienced researchers, and that a novice might be unaware of the issues involved with making these changes. To perpetuate the tradition of case study research, applications in the published literature should have consistencies with traditional methodological constructions, and deviations should be described with a rationale that is inherent in study conduct and findings. Providing methodological descriptions that demonstrate a strong theoretical foundation and coherent study design will add credibility to the study, while ensuring the intrinsic meaning of case study is maintained.

The value of this review is that it contributes to discussion of whether case study is a methodology or method. We propose possible reasons why researchers might make this misinterpretation. Researchers may interchange the terms methods and methodology, and conduct research without adequate attention to epistemology and historical tradition (Carter & Little, 2007 ; Sandelowski, 2010 ). If the rich meaning that naming a qualitative methodology brings to the study is not recognized, a case study might appear to be inconsistent with the traditional approaches described by principal authors (Creswell, 2013a ; Merriam, 2009 ; Stake, 1995 ; Yin, 2009 ). If case studies are not methodologically and theoretically situated, then they might appear to be a case report.

Case reports are promoted by university and medical journals as a method of reporting on medical or scientific cases; guidelines for case reports are publicly available on websites ( http://www.hopkinsmedicine.org/institutional_review_board/guidelines_policies/guidelines/case_report.html ). The various case report guidelines provide a general criteria for case reports, which describes that this form of report does not meet the criteria of research, is used for retrospective analysis of up to three clinical cases, and is primarily illustrative and for educational purposes. Case reports can be published in academic journals, but do not require approval from a human research ethics committee. Traditionally, case reports describe a single case, to explain how and what occurred in a selected setting, for example, to illustrate a new phenomenon that has emerged from a larger study. A case report is not necessarily particular or the study of a case in its entirety, and the larger study would usually be guided by a different research methodology.

This description of a case report is similar to what was provided in some studies reviewed. This form of report lacks methodological grounding and qualities of research rigour. The case report has publication value in demonstrating an example and for dissemination of knowledge (Flanagan, 1999 ). However, case reports have different meaning and purpose to case study, which needs to be distinguished. Findings of our review suggest that the medical understanding of a case report has been confused with qualitative case study approaches.

In this review, a number of case studies did not have methodological descriptions that included key characteristics of case study listed in the adapted criteria, and several issues have been discussed. There have been calls for improvements in publication quality of qualitative research (Morse, 2011 ), and for improvements in peer review of submitted manuscripts (Carter & Little, 2007 ; Jasper, Vaismoradi, Bondas, & Turunen, 2013 ). The challenging nature of editor and reviewers responsibilities are acknowledged in the literature (Hames, 2013 ; Wager & Kleinert, 2010b ); however, review of case study methodology should be prioritized because of disputes on methodological value.

Authors using case study approaches are recommended to describe their theoretical framework and methods clearly, and to seek and follow specialist methodological advice when needed (Wager & Kleinert, 2010a ). Adequate page space for case study description would contribute to better publications (Gillard et al., 2011 ). Capitalizing on the ability to publish complementary resources should be considered.

Limitations of the review

There is a level of subjectivity involved in this type of review and this should be considered when interpreting study findings. Qualitative methods journals were selected because the aims and scope of these journals are to publish studies that contribute to methodological discussion and development of qualitative research. Generalist health and social science journals were excluded that might have contained good quality case studies. Journals in business or education were also excluded, although a review of case studies in international business journals has been published elsewhere (Piekkari et al., 2009 ).

The criteria used to assess the quality of the case studies were a set of qualitative indicators. A numerical or ranking system might have resulted in different results. Stake's ( 1995 ) criteria have been referenced elsewhere, and was deemed the best available (Creswell, 2013b ; Crowe et al., 2011 ). Not all qualitative studies are reported in a consistent way and some authors choose to report findings in a narrative form in comparison to a typical biomedical report style (Sandelowski & Barroso, 2002 ), if misinterpretations were made this may have affected the review.

Case study research is an increasingly popular approach among qualitative researchers, which provides methodological flexibility through the incorporation of different paradigmatic positions, study designs, and methods. However, whereas flexibility can be an advantage, a myriad of different interpretations has resulted in critics questioning the use of case study as a methodology. Using an adaptation of established criteria, we aimed to identify and assess the methodological descriptions of case studies in high impact, qualitative methods journals. Few articles were identified that applied qualitative case study approaches as described by experts in case study design. There were inconsistencies in methodology and study design, which indicated that researchers were confused whether case study was a methodology or a method. Commonly, there appeared to be confusion between case studies and case reports. Without clear understanding and application of the principles and key elements of case study methodology, there is a risk that the flexibility of the approach will result in haphazard reporting, and will limit its global application as a valuable, theoretically supported methodology that can be rigorously applied across disciplines and fields.

Conflict of interest and funding

The authors have not received any funding or benefits from industry or elsewhere to conduct this study.

• Adamson S, Holloway M. Negotiating sensitivities and grappling with intangibles: Experiences from a study of spirituality and funerals. Qualitative Research. 2012; 12 (6):735–752. doi: 10.1177/1468794112439008. [ CrossRef ] [ Google Scholar ]
• Ajodhia-Andrews A, Berman R. Exploring school life from the lens of a child who does not use speech to communicate. Qualitative Inquiry. 2009; 15 (5):931–951. doi: 10.1177/1077800408322789. [ CrossRef ] [ Google Scholar ]
• Alexander B. K, Moreira C, Kumar H. S. Resisting (resistance) stories: A tri-autoethnographic exploration of father narratives across shades of difference. Qualitative Inquiry. 2012; 18 (2):121–133. doi: 10.1177/1077800411429087. [ CrossRef ] [ Google Scholar ]
• Austin W, Park C, Goble E. From interdisciplinary to transdisciplinary research: A case study. Qualitative Health Research. 2008; 18 (4):557–564. doi: 10.1177/1049732307308514. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Ayres L, Kavanaugh K, Knafl K. A. Within-case and across-case approaches to qualitative data analysis. Qualitative Health Research. 2003; 13 (6):871–883. doi: 10.1177/1049732303013006008. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Barone T. L. Culturally sensitive care 1969–2000: The Indian Chicano Health Center. Qualitative Health Research. 2010; 20 (4):453–464. doi: 10.1177/1049732310361893. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Bassey M. A solution to the problem of generalisation in educational research: Fuzzy prediction. Oxford Review of Education. 2001; 27 (1):5–22. doi: 10.1080/03054980123773. [ CrossRef ] [ Google Scholar ]
• Bronken B. A, Kirkevold M, Martinsen R, Kvigne K. The aphasic storyteller: Coconstructing stories to promote psychosocial well-being after stroke. Qualitative Health Research. 2012; 22 (10):1303–1316. doi: 10.1177/1049732312450366. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Broyles L. M, Rodriguez K. L, Price P. A, Bayliss N. K, Sevick M. A. Overcoming barriers to the recruitment of nurses as participants in health care research. Qualitative Health Research. 2011; 21 (12):1705–1718. doi: 10.1177/1049732311417727. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Buckley C. A, Waring M. J. Using diagrams to support the research process: Examples from grounded theory. Qualitative Research. 2013; 13 (2):148–172. doi: 10.1177/1468794112472280. [ CrossRef ] [ Google Scholar ]
• Buzzanell P. M, D'Enbeau S. Stories of caregiving: Intersections of academic research and women's everyday experiences. Qualitative Inquiry. 2009; 15 (7):1199–1224. doi: 10.1177/1077800409338025. [ CrossRef ] [ Google Scholar ]
• Carter S. M, Little M. Justifying knowledge, justifying method, taking action: Epistemologies, methodologies, and methods in qualitative research. Qualitative Health Research. 2007; 17 (10):1316–1328. doi: 10.1177/1049732307306927. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Cheek J, Garnham B, Quan J. What's in a number? Issues in providing evidence of impact and quality of research(ers) Qualitative Health Research. 2006; 16 (3):423–435. doi: 10.1177/1049732305285701. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Colón-Emeric C. S, Plowman D, Bailey D, Corazzini K, Utley-Smith Q, Ammarell N, et al. Regulation and mindful resident care in nursing homes. Qualitative Health Research. 2010; 20 (9):1283–1294. doi: 10.1177/1049732310369337. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Coltart C, Henwood K. On paternal subjectivity: A qualitative longitudinal and psychosocial case analysis of men's classed positions and transitions to first-time fatherhood. Qualitative Research. 2012; 12 (1):35–52. doi: 10.1177/1468794111426224. [ CrossRef ] [ Google Scholar ]
• Creswell J. W. Five qualitative approaches to inquiry. In: Creswell J. W, editor. Qualitative inquiry and research design: Choosing among five approaches. 3rd ed. Thousand Oaks, CA: Sage; 2013a. pp. 53–84. [ Google Scholar ]
• Creswell J. W. Qualitative inquiry and research design: Choosing among five approaches. 3rd ed. Thousand Oaks, CA: Sage; 2013b. [ Google Scholar ]
• Crowe S, Cresswell K, Robertson A, Huby G, Avery A, Sheikh A. The case study approach. BMC Medical Research Methodology. 2011; 11 (1):1–9. doi: 10.1186/1471-2288-11-100. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Cunsolo Willox A, Harper S. L, Edge V. L, ‘My Word’: Storytelling and Digital Media Lab, & Rigolet Inuit Community Government Storytelling in a digital age: Digital storytelling as an emerging narrative method for preserving and promoting indigenous oral wisdom. Qualitative Research. 2013; 13 (2):127–147. doi: 10.1177/1468794112446105. [ CrossRef ] [ Google Scholar ]
• De Haene L, Grietens H, Verschueren K. Holding harm: Narrative methods in mental health research on refugee trauma. Qualitative Health Research. 2010; 20 (12):1664–1676. doi: 10.1177/1049732310376521. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• D'Enbeau S, Buzzanell P. M, Duckworth J. Problematizing classed identities in fatherhood: Development of integrative case studies for analysis and praxis. Qualitative Inquiry. 2010; 16 (9):709–720. doi: 10.1177/1077800410374183. [ CrossRef ] [ Google Scholar ]
• Denzin N. K, Lincoln Y. S. Introduction: Disciplining the practice of qualitative research. In: Denzin N. K, Lincoln Y. S, editors. The SAGE handbook of qualitative research. 4th ed. Thousand Oaks, CA: Sage; 2011a. pp. 1–6. [ Google Scholar ]
• Denzin N. K, Lincoln Y. S, editors. The SAGE handbook of qualitative research. 4th ed. Thousand Oaks, CA: Sage; 2011b. [ Google Scholar ]
• Edwards R, Weller S. Shifting analytic ontology: Using I-poems in qualitative longitudinal research. Qualitative Research. 2012; 12 (2):202–217. doi: 10.1177/1468794111422040. [ CrossRef ] [ Google Scholar ]
• Eisenhardt K. M. Building theories from case study research. The Academy of Management Review. 1989; 14 (4):532–550. doi: 10.2307/258557. [ CrossRef ] [ Google Scholar ]
• Fincham B, Scourfield J, Langer S. The impact of working with disturbing secondary data: Reading suicide files in a coroner's office. Qualitative Health Research. 2008; 18 (6):853–862. doi: 10.1177/1049732307308945. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Flanagan J. Public participation in the design of educational programmes for cancer nurses: A case report. European Journal of Cancer Care. 1999; 8 (2):107–112. doi: 10.1046/j.1365-2354.1999.00141.x. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Flyvbjerg B. Five misunderstandings about case-study research. Qualitative Inquiry. 2006; 12 (2):219–245. doi: 10.1177/1077800405284.363. [ CrossRef ] [ Google Scholar ]
• Flyvbjerg B. Case study. In: Denzin N. K, Lincoln Y. S, editors. The SAGE handbook of qualitative research. 4th ed. Thousand Oaks, CA: Sage; 2011. pp. 301–316. [ Google Scholar ]
• Fourie C. L, Theron L. C. Resilience in the face of fragile X syndrome. Qualitative Health Research. 2012; 22 (10):1355–1368. doi: 10.1177/1049732312451871. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Gallagher N, MacFarlane A, Murphy A. W, Freeman G. K, Glynn L. G, Bradley C. P. Service users’ and caregivers’ perspectives on continuity of care in out-of-hours primary care. Qualitative Health Research. 2013; 23 (3):407–421. doi: 10.1177/1049732312470521. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Gerring J. What is a case study and what is it good for? American Political Science Review. 2004; 98 (2):341–354. doi: 10.1017/S0003055404001182. [ CrossRef ] [ Google Scholar ]
• Gillard A, Witt P. A, Watts C. E. Outcomes and processes at a camp for youth with HIV/AIDS. Qualitative Health Research. 2011; 21 (11):1508–1526. doi: 10.1177/1049732311413907. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Grant M, Booth A. A typology of reviews: An analysis of 14 review types and associated methodologies. Health Information and Libraries Journal. 2009; 26 :91–108. doi: 10.1111/j.1471-1842.2009.00848.x. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Gratton M.-F, O'Donnell S. Communication technologies for focus groups with remote communities: A case study of research with First Nations in Canada. Qualitative Research. 2011; 11 (2):159–175. doi: 10.1177/1468794110394068. [ CrossRef ] [ Google Scholar ]
• Hallberg L. Quality criteria and generalization of results from qualitative studies. International Journal of Qualitative Studies on Health and Wellbeing. 2013; 8 :1. doi: 10.3402/qhw.v8i0.20647. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Hames I. Committee on Publication Ethics, 1. 2013, March. COPE Ethical guidelines for peer reviewers. Retrieved April 7, 2013, from http://publicationethics.org/resources/guidelines . [ Google Scholar ]
• Hooghe A, Neimeyer R. A, Rober P. “Cycling around an emotional core of sadness”: Emotion regulation in a couple after the loss of a child. Qualitative Health Research. 2012; 22 (9):1220–1231. doi: 10.1177/1049732312449209. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Jackson C. B, Botelho E. M, Welch L. C, Joseph J, Tennstedt S. L. Talking with others about stigmatized health conditions: Implications for managing symptoms. Qualitative Health Research. 2012; 22 (11):1468–1475. doi: 10.1177/1049732312450323. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Jasper M, Vaismoradi M, Bondas T, Turunen H. Validity and reliability of the scientific review process in nursing journals—time for a rethink? Nursing Inquiry. 2013 doi: 10.1111/nin.12030. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Jensen J. L, Rodgers R. Cumulating the intellectual gold of case study research. Public Administration Review. 2001; 61 (2):235–246. doi: 10.1111/0033-3352.00025. [ CrossRef ] [ Google Scholar ]
• Jorrín-Abellán I. M, Rubia-Avi B, Anguita-Martínez R, Gómez-Sánchez E, Martínez-Mones A. Bouncing between the dark and bright sides: Can technology help qualitative research? Qualitative Inquiry. 2008; 14 (7):1187–1204. doi: 10.1177/1077800408318435. [ CrossRef ] [ Google Scholar ]
• Ledderer L. Understanding change in medical practice: The role of shared meaning in preventive treatment. Qualitative Health Research. 2011; 21 (1):27–40. doi: 10.1177/1049732310377451. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Lincoln Y. S. Emerging criteria for quality in qualitative and interpretive research. Qualitative Inquiry. 1995; 1 (3):275–289. doi: 10.1177/107780049500100301. [ CrossRef ] [ Google Scholar ]
• Luck L, Jackson D, Usher K. Case study: A bridge across the paradigms. Nursing Inquiry. 2006; 13 (2):103–109. doi: 10.1111/j.1440-1800.2006.00309.x. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Mawn B, Siqueira E, Koren A, Slatin C, Devereaux Melillo K, Pearce C, et al. Health disparities among health care workers. Qualitative Health Research. 2010; 20 (1):68–80. doi: 10.1177/1049732309355590. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Merriam S. B. Qualitative research: A guide to design and implementation. 3rd ed. San Francisco, CA: Jossey-Bass; 2009. [ Google Scholar ]
• Meyer C. B. A case in case study methodology. Field Methods. 2001; 13 (4):329–352. doi: 10.1177/1525822x0101300402. [ CrossRef ] [ Google Scholar ]
• Morse J. M. Mixing qualitative methods. Qualitative Health Research. 2009; 19 (11):1523–1524. doi: 10.1177/1049732309349360. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Morse J. M. Molding qualitative health research. Qualitative Health Research. 2011; 21 (8):1019–1021. doi: 10.1177/1049732311404706. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Morse J. M, Dimitroff L. J, Harper R, Koontz A, Kumra S, Matthew-Maich N, et al. Considering the qualitative–quantitative language divide. Qualitative Health Research. 2011; 21 (9):1302–1303. doi: 10.1177/1049732310392386. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Nagar-Ron S, Motzafi-Haller P. “My life? There is not much to tell”: On voice, silence and agency in interviews with first-generation Mizrahi Jewish women immigrants to Israel. Qualitative Inquiry. 2011; 17 (7):653–663. doi: 10.1177/1077800411414007. [ CrossRef ] [ Google Scholar ]
• Nairn K, Panelli R. Using fiction to make meaning in research with young people in rural New Zealand. Qualitative Inquiry. 2009; 15 (1):96–112. doi: 10.1177/1077800408318314. [ CrossRef ] [ Google Scholar ]
• Nespor J. The afterlife of “teachers’ beliefs”: Qualitative methodology and the textline. Qualitative Inquiry. 2012; 18 (5):449–460. doi: 10.1177/1077800412439530. [ CrossRef ] [ Google Scholar ]
• Piekkari R, Welch C, Paavilainen E. The case study as disciplinary convention: Evidence from international business journals. Organizational Research Methods. 2009; 12 (3):567–589. doi: 10.1177/1094428108319905. [ CrossRef ] [ Google Scholar ]
• Ragin C. C, Becker H. S. What is a case?: Exploring the foundations of social inquiry. Cambridge: Cambridge University Press; 1992. [ Google Scholar ]
• Roscigno C. I, Savage T. A, Kavanaugh K, Moro T. T, Kilpatrick S. J, Strassner H. T, et al. Divergent views of hope influencing communications between parents and hospital providers. Qualitative Health Research. 2012; 22 (9):1232–1246. doi: 10.1177/1049732312449210. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Rosenberg J. P, Yates P. M. Schematic representation of case study research designs. Journal of Advanced Nursing. 2007; 60 (4):447–452. doi: 10.1111/j.1365-2648.2007.04385.x. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Rytterström P, Unosson M, Arman M. Care culture as a meaning- making process: A study of a mistreatment investigation. Qualitative Health Research. 2013; 23 :1179–1187. doi: 10.1177/1049732312470760. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Sandelowski M. Whatever happened to qualitative description? Research in Nursing & Health. 2000; 23 (4):334–340. doi: 10.1002/1098-240X. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Sandelowski M. What's in a name? Qualitative description revisited. Research in Nursing & Health. 2010; 33 (1):77–84. doi: 10.1002/nur.20362. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Sandelowski M, Barroso J. Reading qualitative studies. International Journal of Qualitative Methods. 2002; 1 (1):74–108. [ Google Scholar ]
• Snyder-Young D. “Here to tell her story”: Analyzing the autoethnographic performances of others. Qualitative Inquiry. 2011; 17 (10):943–951. doi: 10.1177/1077800411425149. [ CrossRef ] [ Google Scholar ]
• Stake R. E. The case study method in social inquiry. Educational Researcher. 1978; 7 (2):5–8. [ Google Scholar ]
• Stake R. E. The art of case study research. Thousand Oaks, CA: Sage; 1995. [ Google Scholar ]
• Stake R. E. Case studies. In: Denzin N. K, Lincoln Y. S, editors. Strategies of qualitative inquiry. Thousand Oaks, CA: Sage; 1998. pp. 86–109. [ Google Scholar ]
• Sumsion J. Opening up possibilities through team research: Investigating infants’ experiences of early childhood education and care. Qualitative Research. 2013; 14 (2):149–165. doi: 10.1177/1468794112468471.. [ CrossRef ] [ Google Scholar ]
• Thomas G. Doing case study: Abduction not induction, phronesis not theory. Qualitative Inquiry. 2010; 16 (7):575–582. doi: 10.1177/1077800410372601. [ CrossRef ] [ Google Scholar ]
• Thomas G. A typology for the case study in social science following a review of definition, discourse, and structure. Qualitative Inquiry. 2011; 17 (6):511–521. doi: 10.1177/1077800411409884. [ CrossRef ] [ Google Scholar ]
• Tight M. The curious case of case study: A viewpoint. International Journal of Social Research Methodology. 2010; 13 (4):329–339. doi: 10.1080/13645570903187181. [ CrossRef ] [ Google Scholar ]
• Wager E, Kleinert S. Responsible research publication: International standards for authors. A position statement developed at the 2nd World Conference on Research Integrity, Singapore, July 22–24, 2010. In: Mayer T, Steneck N, editors. Promoting research integrity in a global environment. Singapore: Imperial College Press/World Scientific; 2010a. pp. 309–316. [ Google Scholar ]
• Wager E, Kleinert S. Responsible research publication: International standards for editors. A position statement developed at the 2nd World Conference on Research Integrity, Singapore, July 22–24, 2010. In: Mayer T, Steneck N, editors. Promoting research integrity in a global environment. Singapore: Imperial College Press/World Scientific; 2010b. pp. 317–328. [ Google Scholar ]
• Webb C, Kevern J. Focus groups as a research method: A critique of some aspects of their use in nursing research. Journal of Advanced Nursing. 2000; 33 (6):798–805. doi: 10.1046/j.1365-2648.2001.01720.x. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Wimpenny K, Savin-Baden M. Exploring and implementing participatory action synthesis. Qualitative Inquiry. 2012; 18 (8):689–698. doi: 10.1177/1077800412452854. [ CrossRef ] [ Google Scholar ]
• Yeh H.-Y. Boundaries, entities, and modern vegetarianism: Examining the emergence of the first vegetarian organization. Qualitative Inquiry. 2013; 19 (4):298–309. doi: 10.1177/1077800412471516. [ CrossRef ] [ Google Scholar ]
• Yin R. K. Enhancing the quality of case studies in health services research. Health Services Research. 1999; 34 (5 Pt 2):1209–1224. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Yin R. K. Case study research: Design and methods. 4th ed. Thousand Oaks, CA: Sage; 2009. [ Google Scholar ]
• Yin R. K. Applications of case study research. 3rd ed. Thousand Oaks, CA: Sage; 2012. [ Google Scholar ]

Log in using your username and password

• Search More Search for this keyword Advanced search
• Latest content
• Current issue
• BMJ Journals

You are here

• Online First
• Diagnostic accuracy of alternative biomarkers for acute aortic syndrome: a systematic review
• Article Text
• Article info
• Citation Tools
• Rapid Responses
• Article metrics

• http://orcid.org/0000-0002-8358-0607 Joshua Wren 1 ,
• http://orcid.org/0000-0003-0803-8444 Steve Goodacre 2 ,
• Abdullah Pandor 2 ,
• Munira Essat 2 ,
• Mark Clowes 2 ,
• Graham Cooper 3 , 1 ,
• Robert Hinchliffe 4 , 5 ,
• http://orcid.org/0000-0003-1308-4824 Matthew J Reed 6 , 7 ,
• Steven Thomas 1 ,
• http://orcid.org/0000-0003-3964-0809 Sarah Wilson 8
• 1 Sheffield Teaching Hospitals NHS Foundation Trust , Sheffield , UK
• 2 University of Sheffield , Sheffield , UK
• 3 Aortic Dissection Charitable Trust , Sheffield , UK
• 4 Department of Vascular Surgery , North Bristol NHS Trust , Westbury on Trym , UK
• 5 Translational Health Sciences , University of Bristol , Bristol , UK
• 6 Emergency Medicine Research Group Edinburgh (EMERGE) , NHS Lothian , Edinburgh , UK
• 7 Acute Care Group , The University of Edinburgh Usher Institute of Population Health Sciences and Informatics , Edinburgh , UK
• 8 Emergency Department , Wexham Park Hospital , Slough , UK
• Correspondence to Professor Steve Goodacre; s.goodacre{at}sheffield.ac.uk

Background D-dimer is the only biomarker currently recommended in guidelines for the diagnosis of acute aortic syndrome (AAS). We undertook a systematic review to determine whether any alternative biomarkers could be useful in AAS diagnosis.

Methods We searched electronic databases (including MEDLINE, EMBASE and the Cochrane Library) from inception to February 2024. Diagnostic studies were eligible if they examined biomarkers other than D-dimer for diagnosing AAS compared with a reference standard test in people presenting to the ED with symptoms of AAS. Case-control studies were identified but excluded due to high risk of bias. Selection of studies, data extraction and risk of bias assessments using the Quality Assessment of Diagnostic Accuracy Studies 2 (QUADAS-2) tool were undertaken independently by at least two reviewers. We used narrative synthesis to summarise the findings.

Results We identified 2017 citations, included 13 cohort studies (n=76–999), and excluded 38 case-control studies. Methodological quality was variable, with most included studies having unclear or high risk of bias and applicability concerns in at least one item of the QUADAS‐2 tool. Only two studies reported biomarkers with sensitivity and specificity comparable to D-dimer (ie, >90% and >50%, respectively). Wang et al reported 99.1% sensitivity and 84.9% specificity for soluble ST2; however, these findings conflicted with estimates of 58% sensitivity and 70.8% specificity reported in another study. Chun and Siu reported 95.6% sensitivity and 56.1% specificity for neutrophil count, but this has not been confirmed elsewhere.

Conclusion There are many potential alternative biomarkers for AAS but few have been evaluated in more than one study, study designs are often weak and reported biomarker accuracy is modest or inconsistent between studies. Alternative biomarkers to D-dimer are not ready for routine clinical use.

PROSPERO registration number CRD42022252121.

• systematic review
• cardiovascular system

Data availability statement

All data relevant to the study are included in the article or uploaded as supplementary information.

This is an open access article distributed in accordance with the Creative Commons Attribution 4.0 Unported (CC BY 4.0) license, which permits others to copy, redistribute, remix, transform and build upon this work for any purpose, provided the original work is properly cited, a link to the licence is given, and indication of whether changes were made. See: https://creativecommons.org/licenses/by/4.0/ .

https://doi.org/10.1136/emermed-2023-213772

Statistics from Altmetric.com

Request permissions.

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

WHAT IS ALREADY KNOWN ON THIS TOPIC

D-dimer has some diagnostic value in the assessment of suspected acute aortic syndrome (AAS), but the role of other potential biomarkers is unclear.

WHAT THIS STUDY ADDS

Our systematic review showed that the evidence for other biomarkers is weak and estimates of accuracy are generally modest or inconsistent.

HOW THIS STUDY MIGHT AFFECT RESEARCH, PRACTICE OR POLICY

Diagnostic biomarkers for AAS, other than D-dimer, are not ready for routine clinical use.

Large cohort studies are required to evaluate multiple biomarkers in an appropriate population with suspected AAS.

Introduction

Acute aortic syndrome (AAS) is a life-threatening emergency condition affecting the thoracic aorta that includes acute aortic dissection (AAD), intramural haematoma and penetrating ulcer. CT angiography (CTA) scanning of the aorta has high sensitivity and specificity for diagnosing AAS but incurs significant costs and the risks of ionising radiation.

Biomarkers could be used to select patients with suspected AAS for CTA. The pathophysiology of AAS allows researchers to select and investigate various biomarkers. In aortic dissection, intima rupture allows circulating blood to enter the media of the aorta, forming both true and false lumens. This results in an initial inflammatory response, followed by infiltration of inflammatory cells and subsequent vascular smooth muscle cell apoptosis, ultimately leading to aortic media degradation. This process potentiates aortic dilatation, aneurysm formation, progression to dissection and can lead to rupture. 1 Recognising this condition early is paramount to avoid significant morbidity and mortality. Biomarkers can therefore be divided according to the process with which they are associated: clotting, inflammatory response, lipid metabolism, cardiac myocyte damage, vascular extracellular matrix damage and other protein metabolism. 2 Biomarkers may also reflect the consequent effects of organ hypoperfusion. Such biomarkers would therefore play more of a role in severity of sequelae as opposed to identification of AAS.

D-dimer is the most extensively studied biomarker for AAS. The most recent meta-analysis included 18 studies with 7978 patients and reported pooled sensitivity of 96.5% (95% credible interval (CrI) 94.8% to 98%) and specificity of 56.2% (95% CrI 48.3% to 63.9%) for D-dimer above the diagnostic threshold of 500 ng/mL. 3 4 D-dimer sensitivity can be improved by using it alongside a clinical probability score, such as the aortic dissection detection risk score (ADD-RS), which uses clinical features to estimate clinical risk on a score from 0 to 3. The combination of ADD-RS ≥1 and D-dimer >500 ng/mL in a recent meta-analysis of six studies demonstrated pooled sensitivity of 93.1% (95% CrI 87.1% to 96.3%) and specificity 67.1% (95% CrI 54.4% to 77.7%) when diagnosing AAS. 5 These findings suggest a potential role for D-dimer, alone or alongside clinical probability scoring, to select patients for CTA, and suggest that alternative biomarkers with superior accuracy to D-dimer could have an important role in clinical practice.

A number of studies have evaluated biomarkers for AAS other than D-dimer. However, these have not yet been systematically examined to identify the most promising candidates for future research or to determine whether any have the potential to improve on the accuracy of D-dimer. We aimed to systematically review biomarkers other than D-dimer to determine their accuracy for diagnosing AAS.

A systematic review was undertaken in accordance with the general principles recommended in the Preferred Reporting Items for Systematic Reviews and Meta-Analyses statement. 6 7 This review was part of a larger National Institute for Health and Care Research-funded (January to December 2023) Aortic Syndrome Evidence Synthesis (ASES) project on diagnostic strategies for suspected AAS and was registered on the International Prospective Register of Systematic Reviews database (CRD42022252121). 8

Eligibility criteria

Prospective or retrospective studies reporting diagnostic accuracy metrics were eligible if they examined any biomarkers (other than D-dimer) for diagnosing AAS compared with a reference standard test (eg, a definitive imaging modality such as CTA, ECG-gated CTA, echocardiography, and magnetic resonance angiography or confirmed/excluded by operation and autopsy). The study population of interest in our review consisted of people (any age) presenting to the ED with symptoms of AAS, including those with new-onset chest, back or abdominal pain, syncope or symptoms related to perfusion deficit. Studies including people with AAS following major trauma or as incidental findings were excluded. We initially planned to include all study designs in the review but only include cohort studies in any meta-analysis. After undertaking initial searches, we amended the protocol to exclude case-control designs from the review due to the potential for high bias resulting in inaccurate estimates and lack of representativeness of test accuracy in a clinical setting. 9 10

Data sources and searches

Potentially relevant studies were identified through searches of several electronic databases including MEDLINE (OvidSP from 1946 to February 2024), EMBASE (OvidSP from 1974 to February 2024), and the Cochrane Library ( https://www.cochranelibrary.com from inception to February 2024) by an experienced information specialist (MC), who is a member of the research team. The search strategy used free text and thesaurus terms and combined synonyms relating to the topic of interest (eg, AAS and diagnostic strategies) with diagnostic testing terms (adapted Scottish Intercollegiate Guidelines Network filter for identifying diagnostic studies). Searches were supplemented by hand-searching the reference lists of all relevant studies (including existing systematic reviews); forward citation searching of relevant articles; contacting key experts in the field and undertaking targeted searches of the World Wide Web using the Google search engine. No date or language restrictions were applied on any database. Further details on the search strategy can be found in online supplemental appendix S1 .

Supplemental material

Study selection.

All titles were examined for inclusion by one reviewer (ME) and any citations that clearly did not meet the inclusion criteria (eg, non-human, unrelated to AAS) were excluded. All abstracts and full-text articles were then examined independently by two reviewers (ME and AP). Any disagreements in the selection process were resolved through discussion or if necessary, arbitration by a third reviewer (SG) and included by consensus.

Data extraction and quality assessment

Data relating to study design, methodological quality and outcomes were extracted by one reviewer (JW) into a standardised data extraction form and independently checked for accuracy by a second (AP). Any discrepancies were resolved through discussion to achieve agreement. Where differences were unresolved, a third reviewer’s opinion was sought (SG). Where multiple publications of the same study were identified, data were extracted and reported as a single study.

The methodological quality of each included study was assessed using the Quality Assessment of Diagnostic Accuracy Studies-2 (QUADAS-2) tool. 11 This instrument evaluates four key domains: patient selection, index test, reference standard, flow and timing. Each domain is assessed in terms of risk of bias and concerns regarding the applicability of the study results (first three domains only). The subdomains about risk of bias include a number of signaling questions to help guide the overall judgement about whether a study is at high, low or an unclear (in the event of insufficient data in the publication to answer the corresponding question) risk of bias.

Data synthesis and analysis

We were unable to perform meta-analysis due to the limited number of studies per biomarker and variable reporting of items. As a result, a narrative synthesis approach was undertaken, with data being summarised in tables with accompanying narrative summaries that included a description of the included variables, statistical methods and performance measures (eg, sensitivity, specificity). 12 13 All analyses were conducted using Microsoft Excel 2010 (Microsoft, Redmond, Washington, USA).

Patient and public involvement

Two members of the Aortic Dissection Charitable Trust ( https://aorticdissectioncharitabletrust.org/ ) joined the ASES project management team and helped to develop the study proposal. SG presented the findings of this review to a webinar of Aortic Dissection Charitable Trust members and sought their feedback on interpretation of the results.

Figure 1 summarises the process of identifying and selecting relevant literature. Of the 2017 citations identified, 13 studies investigating 17 index tests met the inclusion criteria. 14–26 The majority of the articles were excluded primarily on the basis of an inappropriate target population (patients with AAS or not suspected AAS), investigating intervention not an alternative biomarker or an unsuitable publication type (ie, reviews or abstract of full-text studies). A full list of excluded studies with reasons for exclusion can be found in online supplemental appendix S2 . More specifically, 38 case-control studies were excluded due to the high potential for bias with this design. 9 10 Four case-control studies reported comparisons with unselected controls with suspected AAS, 27–29 whereas 34 case-control studies reported comparisons with healthy controls or controls with other diagnoses. 30–63 A summary of the design and patient characteristics of the 38 excluded case-control studies can be found in online supplemental appendix S3 . These studies evaluated a wide variety of biomarkers using a variety of different control groups. These studies may identify biomarkers for future research but do not provide reliable estimates of accuracy to inform clinical practice.

• Download figure
• Open in new tab
• Download powerpoint

Study flow chart (adapted with permission from reference 7 ). AAS, acute aortic syndrome.

Study and patient characteristics

The design and patient characteristics of the 13 included studies are summarised in table 1 . Sample size ranged from n=76–999, with prevalence of AAS ranging from 1% to 51.2%. A team of researchers in Italy undertook five of the studies, evaluating the following biomarkers in consecutive cohorts: matrix metalloproteinases (MMP) 8 and 9, lactate dehydrogenase (LDH), white blood cell (WBC) count, platelet count, fibrinogen, copeptin and soluble suppression of tumourigenicity-2 (sST2). The other studies were undertaken in China (five studies), Germany, Japan and Canada, evaluating the following biomarkers: troponin, α-smooth muscle actin (α-SMA), smooth muscle myosin heavy chain (smMHC), soluble elastin fragments (sELAF) in serum, polycystin-1 (PC1), acidic and basic calponin at 6 and 24 hours after presentation, sST2, neutrophil-to-lymphocyte ratio, neutrophil count and leucocyte count.

• View inline

Study characteristics of the 13 included cohort studies

Risk of bias and applicability assessment

The overall methodological quality of the 13 included studies is summarised in table 2 and figure 2 15 17–26 and in the online supplemental appendix S4 . The methodological quality of the included studies was variable, with most studies having unclear or high risk of bias and applicability concerns in at least one item of the QUADAS‐2 tool. The following sections attributed a high level of bias: patient selection, primarily due to the use of convenience sampling; index test, due to the absence of prespecified threshold values and flow and timing, principally because patients received different reference standards (imaging or follow-up). The reference standard item attributed an unclear level of bias due to a lack of clarity as to whether the reference standard results were interpreted without knowledge of the index test.

QUADAS-2 quality assessment summary—review authors’ judgements

Quality Assessment of Diagnostic Accuracy Studies 2 (QUADAS-2) assessment summary graph—review authors’ judgements.

Diagnostic performance of alternate biomarkers

The accuracy results (sensitivity, specificity, positive and negative predictive values, positive and negative likelihood ratios and area under the receiver operating characteristic (AUROC) of the 13 included studies are summarised in table 3 and online supplemental appendix S5 . The AUROC for the biomarkers was generally modest, with high sensitivity with tight CIs achieved only when a threshold was used that resulted in low specificity. In general, the alternative biomarkers did not achieve the sensitivity and sensitivity of D-dimer reported in recent meta-analysis. 3 There were two exceptions. Wang et al 25 reported 99.1% sensitivity and 84.9% specificity for soluble ST2, but this differed markedly from the sensitivity of 58% and specificity of 70.8% reported by Morello et al . 21 Chun and Siu 14 reported 95.6% sensitivity and 56.1% specificity for neutrophil count, but this has not been confirmed by other studies. Accuracy improved in the event the biomarkers were combined with D-dimer but was not clearly superior to D-dimer alone.

Summary of accuracy results (sensitivity, specificity and AUROC) along with their respective 95% CIs and cut-off values where provided for the biomarkers investigated in the 13 included cohort studies

Summary of results

This systematic review has shown that biomarkers for AAS, other than D-dimer, currently have insufficient evidence of acceptable accuracy to support routine clinical use. We identified numerous studies evaluating many different biomarkers, but the quality and heterogeneity of the studies limited the conclusions we could draw. The estimates of sensitivity, specificity and AUROC from the cohort studies generally suggested poor diagnostic accuracy for AAS and thus these biomarkers have no current role in clinical practice.

We are aware of one additional review of alternative biomarkers for AAS. 64 Chen et al concluded that microRNA biomarkers may have better specificity than D-dimer but current studies are insufficient. Their review included case-control studies, which are known to overestimate diagnostic accuracy, 10 and undertook limited quality assessment of included studies, so any conclusions should be interpreted with caution.

Interpretation of results

Soluble ST2 is primarily found in inflammatory processes and T-cell-mediated immune responses. The studies of soluble ST2 produced conflicting results, with Wang et al 25 reporting diagnostic accuracy superior to that of D-dimer but Morello et al 21 reporting modest accuracy. These differences could be attributed to differences in ethnicity and age across the two study populations. Further studies are required to determine accuracy in an appropriate cohort.

MMPs are key to aortic remodelling and part of the family of extracellular matrix markers. The study of Giachino et al 15 suggested modest accuracy for AAS, but the combination of D-dimer and MMP-8 improved the sensitivity of D-dimer at the expense of specificity. Morello et al 20 reported that copeptin, a biomarker released by the neurohypophysis in response to stress, provided suboptimal diagnostic accuracy for AAS. Suzuki et al 23 reported that calponin, an analogue of cardiac troponin and released during muscle fibre apoptosis, had modest diagnostic accuracy for AAS. This was replicated by Lian et al , 16 who demonstrated that including ascending aortic root dilation of >40 mm significantly increased the accuracy of calponin.

Peng et al 22 focused on two distinct groups of biomarkers: smooth muscle biomarkers (α-SMA, smMHC and PC1) and extracellular matrix markers (sELAF). smMHC has been found to be released from damaged aortic medial muscle cells during aortic dissection, while PC1 plays a key role in stability and integrity of aortic vessel walls. sELAF is released on rupture of elastic vascular wall fibres. In isolation, none of these markers offered superior performance to D-dimer, but the combination of three of the biomarkers alongside D-dimer could offer better accuracy than D-dimer alone. This requires validation in a new cohort.

Some of the biomarkers evaluated for AAS are already used in routine clinical assessment for other conditions. Morello et al 19 showed that WBC count, platelet count and fibrinogen are not accurate biomarkers for AAS but the modest diagnostic information they provide could be used in pretest probability assessment, while the study of von Kodolitsch et al 24 suggested that leucocyte count provided no useful diagnostic information. Similarly, LDH was found to have poor diagnostic accuracy for AAS. 18 The study of Zhang et al 26 suggested that the neutrophil-to-lymphocyte ratio may have similar or superior accuracy to D-dimer in the diagnosis of AAS. 26 This finding requires replication in other studies. Two studies of troponin showed no diagnostic value for AAS. 17 25 Chun and Siu 14 reported similar accuracy data to that of D-dimer, although these findings will require validation in further studies.

Strengths and limitations of the systematic review

We used established and robust methods to ensure our review was comprehensive and involved objective assessment of study quality. However, it may have some limitations. Indexing and reporting of diagnostic studies may be suboptimal, especially if biomarker analysis is a secondary study objective, so we may have missed some potentially relevant data. The primary studies had important limitations, with insufficient numbers of cohort studies evaluating any individual biomarker to support meta-analysis and potential biases affecting patient selection and reference standard adjudication. There was also a lack of implementation studies, threshold rationale and economic data, thus making it difficult to draw conclusions regarding the practicality and affordability of clinical implementation of these alternative biomarkers. Furthermore, we noted that a number of our selected cohort studies reported high AAS prevalence, compared with an unselected population with possible AAS, such as reported in the Diagnosis of Acute Aortic Syndrome in the Emergency Department (DAShED) study. 65 This likely represents selection of patients who received a definitive imaging reference standard, which limits the applicability of findings to the unselected general population.

The QUADAS-2 assessment of the reference standard did not differentiate between studies using AD and those using AAS as the reference standard. However, between 73% and 86% of cases of AAS were AD in the cohort studies that used AAS as the reference standard, so any differences between the studies using AD and AAS are likely to be modest and unlikely to impact our conclusions.

Implications for policy, practice and future research

More research is needed on biomarkers for AAS. However, the low incidence of AAS presentations in the ED creates a substantial barrier to conducting adequately powered studies with robust designs. The researchers who undertook the studies included in this review deserve credit for their efforts, particularly the Italian researchers who provided 5 of the 13 studies. Our review highlights the need for other research teams to evaluate new biomarkers for AAS so we can determine whether findings are reproduced elsewhere. Case-control studies can provide initial data to identify biomarkers that show an association with AAS but cohort studies are required to provide data to guide clinical practice. Studies should ideally record clinical risk assessment, such as the ADD-RS, and measure D-dimer, to determine the additional contribution to diagnostic assessment of novel biomarkers. An optimal future study might involve recording the ADD-RS and taking blood for measurement of D-dimer and multiple alternative biomarkers from a large prospective cohort of patients with suspected AAS. Analysis could then determine whether and how alternative biomarkers provide additional diagnostic information beyond that provided by the ADD-RS and D-dimer.

Conclusions

Currently available research is insufficient to recommend any biomarker as an alternative or in addition to D-dimer in the diagnostic assessment of AAS. Large cohort studies are required to evaluate multiple biomarkers in an appropriate population with suspected AAS.

Ethics statements

Patient consent for publication.

Not applicable.

Ethics approval

Acknowledgments.

The authors would like to thank all additional members of the core project group for NIHR151853 for their valuable input and commentary throughout the work. The authors are indebted to Joanne Hinde for assistance with logistics and administration. The authors would also like to thank Catherine Fowler, Valerie Lechene and other members of the Aortic Dissection Charitable Trust for their help with this project.

• Li J-J , et al
• Goodacre S ,
• Pandor A , et al
• Yang B , et al
• McInnes MDF ,
• Thombs BD , et al
• McKenzie JE ,
• Bossuyt PM , et al
• Thokala P , et al
• Leeflang MMG
• Lijmer JG ,
• Heisterkamp S , et al
• Whiting PF ,
• Rutjes AWS ,
• Westwood ME , et al
• Brennan SE ,
• Ryan RE , et al
• Schünemann HJ ,
• Brozek J , et al
• Giachino F ,
• Loiacono M ,
• Lucchiari M , et al
• Liu J , et al
• Mellnick VM ,
• Monteiro S , et al
• Morello F ,
• Ravetti A ,
• Nazerian P , et al
• Cavalot G ,
• Giachino F , et al
• Cavalot G , et al
• Bartalucci A ,
• Bironzo M , et al
• Chai X , et al
• Distante A ,
• Zizza A , et al
• von Kodolitsch Y ,
• Schwartz AG ,
• Nienaber CA
• Gao H , et al
• Guo J , et al
• Ohlmann P ,
• Morel O , et al
• Yao C , et al
• Anjum O , et al
• Aksun S , et al
• Goliopoulou A ,
• Oikonomou E ,
• Antonopoulos A , et al
• Hagiwara A ,
• Sakamoto D ,
• Sasaki R , et al
• Wang Y , et al
• Li D , et al
• Li R , et al
• Shinohara T ,
• Okada M , et al
• Luo L , et al
• Watanabe M , et al
• Guo X , et al
• Zhang W , et al
• Feng R , et al
• Eggebrecht H ,
• Bruch C , et al
• Schoenrath F ,
• Torzewski M , et al
• Kimira A , et al
• Li Y , et al
• Fukumoto H ,
• Negoro N , et al
• Li X , et al
• Su X , et al
• Okazaki T ,
• Yamamoto Y ,
• Yoda K , et al
• Yang G , et al
• Sbarouni E ,
• Georgiadou P ,
• Analitis A , et al
• Kosmas E , et al
• Marathias A , et al
• Pan J , et al
• Wang J-J , et al
• Fletcher A ,
• Iskander Z , et al
• Domanovits H ,
• Holzer M , et al
• Trimarchi S ,
• Sawaki D , et al
• Li Z , et al
• McLatchie R ,
• Freeman N , et al

Handling editor Richard Body

X @mattreed73

Contributors SG and AP coordinated the study. SG, AP, ME, MC, GC, RH, MJR, ST and SW were responsible for conception, design and obtaining funding for the study. MC developed the search strategy, undertook searches and organised retrieval of papers. AP, ME, SG and JW were responsible for the acquisition, analysis and interpretation of data. GC, RH, MJR, ST and SW helped interpret and provided a methodological, policy and clinical perspective on the data. JW, AP and SG were responsible for the drafting of this paper, although all authors provided comments on the drafts and read and approved the final version. SG is the guarantor for the paper.

Funding This study was funded by the UK National Institute for Health and Care Research (project number 151853).

Disclaimer The views expressed in this paper are those of the authors and not necessarily those of the NIHR or the Department of Health and Social Care. Any errors are the responsibility of the authors. The funders had no role in the study design, in the collection, analysis and interpretation of data; in the writing of the manuscript and in the decision to submit the manuscript for publication.

Competing interests SG, AP, ME, MC, GC, RH, MJR, ST and SW all declare funding from NIHR Health Technology Assessment (Grant NIHR151853). There are no other competing interests.

Patient and public involvement Patients and/or the public were involved in the design, or conduct, or reporting, or dissemination plans of this research. Refer to the 'Methods' section for further details.

Provenance and peer review Not commissioned; externally peer reviewed.

Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.

Read the full text or download the PDF: