importance of studying research methods

1 Chapter 1: The Importance of Research Methods and Becoming an Informed Consumer of Research

Case study : student apprehension regarding research methods.

Research Study

Understanding and Measuring Student Apprehension in Criminal Justice Research Methods Courses 1

Research Question

How do we measure disinterest, relevance argumentation, and math anxiety experienced by students enrolled in research methods courses?

Methodology

It is said that �misery loves company,� so you are not alone in your apprehension and anxiety regarding your research methods course. The problem of student apprehension and anxiety related to taking a research methods course is not new and has been studied for over 25 years. Previously, such apprehension and anxiety appeared to be caused by math anxiety, especially as it applies to statistics. The authors of this article believe that student apprehension goes beyond math anxiety; that math anxiety is too simplistic of an explanation of student fear of research methods courses. Besides math anxiety, the researchers think that apprehension is caused by student indifference to the subject matter and irrelevance of the course because it does not apply to the �real world.� They state that student apprehension in research methods and statistics courses is due to three main factors:

Disinterest (D.);

Relevance Argumentation (RA.), and;

Math Anxiety (MA.).

Taken together, the reconceptualization is known as D.RA.MA., and the combination of these three factors constitutes the D.RA.MA. scale for research methods and statistics courses.

The researchers developed the D.RA.MA. scale by constructing survey questions to measure each factor in the scale (i.e., disinterest, relevance argumentation, and math anxiety). After they developed the survey, they tested it by distributing the survey to three criminal justice classes, totaling 80 students, from a midsized regional comprehensive university in the southern region of the United States. Higher scale scores demonstrate more disinterest, more relevance argumentation, or more math anxiety.

The D.RA.MA. scale consists of 20 survey questions. Ten questions were borrowed from an existing Math Anxiety scale developed by Betz 2 . The researchers then created five items to assess Disinterest and five items intended to measure Relevance Argumentation. The items for the D.RA.MA. scale are illustrated below.

Math Anxiety 3

I usually have been at ease in math classes.

Math does not scare me at all.

I am no good at math.

I don�t think that I could do advanced math.

Generally, I have been secure about attempting math.

For some reason, even though I study, math seems unusually hard for me.

Math has been my worst subject.

My mind goes blank and I am unable to think clearly when working in mathematics.

I think I could handle more difficult math.

I am not the type to do well in mathematics.

Relevance Argumentation 4

I will need research methods for my future work.

I view research methods as a subject that I will rarely use.

Research methods is not really useful for students who intend to work in Criminal Justice.

Knowing research methods will help me earn a living.

Research methods does not reflect the �real world.�

Research Disinterest 5

I am excited about taking research methods.

It would not bother me at all to take more research methods courses.

I expect a research methods class to be boring.

I don�t expect to learn much in research methods.

I really don�t care if I learn anything in research methods, as long as I get the requirement completed.

The Math Anxiety Scale responses for the 80 students ranged from 0 to 30 with a mean of 14, demonstrating a moderate level of math anxiety among the study participants. The responses for Relevance Argumentation ranged from 0 to 12 with a mean of 5.4 while those for Disinterest ranged from 1 to 15 with a mean of 7.0, demonstrating a moderate level of disinterest and relevance argumentation among students regarding research methods. Based on these findings, the study demonstrated that student apprehension regarding research methods courses goes beyond math anxiety and includes two additional factors; disinterest in the subject matter and irrelevance of research methods to the �real world.�

Limitations with the Study Procedure

This research study was designed to develop a broader measure of student apprehension in criminal justice research methods courses. Moving beyond just math anxiety, the researchers accomplished their objective by developing the D.RA.MA. scale; adding disinterest and relevance argumentation to the understanding of student apprehension regarding research methods. As is true for all research, this study is not without limitations. The biggest limitation of this study is the limited sample size. Only 80 students completed the survey. Although this is certainly a good start, similar research (i.e., replication) needs to be completed with larger student samples in different locations throughout the country before the actual quality of the D.RA.MA. scale can be determined.

Impact on Criminal Justice

The D.RA.MA. scale developed in this study identifies disinterest and relevance argumentation, in addition to math anxiety, as part of student apprehension and resistance to research methods. A variety of instructional strategies can be inferred from the D.RA.MA. survey. However, it is important for professors to recognize that no single approach will reduce research methods resistance and apprehension for all students. For example, discussing research methods in a popular culture framework may resonate with students and lead to engaged students who are more interested in the subject matter and identify with the relevance of research methods to criminal justice in general and the future careers of students, in particular. This approach may provide an effective means for combating student disinterest and relevance argumentation in criminal justice research methods courses. At a minimum, it is critical for professors to explain the relevance of research methods to the policies and practices of police, courts, and corrections. Students need to realize that research methods are essential tools for assessing agency policies and practices. Professors will always have D.RA.MA.-plagued students, but recognizing the problem and then developing effective strategies to connect with these students is the challenge all professors face. Experimenting with a multitude of teaching strategies to alleviate the math anxiety, relevance argumentation, and disinterest of criminal justice research methods students will result in more effective teaching and learning.

In This Chapter You Will Learn

What research is and why it is important to be an informed consumer of research

The sources of knowledge development and problems with each

How research methods can dispel myths about crime and the criminal justice system

The steps in the research process

How research has impacted criminal justice operations

Introduction

As noted in the chapter opening case study, it is expected that you have some anxiety and apprehension about taking this criminal justice research methods course. But, you have taken a significant step toward success in this course by opening up your research methods book, so congratulations are in order. You might have opened this book for a number of reasons. Perhaps it is the first day of class and you are ready to get started on the course material. Perhaps you have a quiz or exam soon. Perhaps the book has been gathering dust on your shelf since the first day of class and you are not doing well in your research methods class and are looking for the book to help with course improvement. Perhaps you are taking a research methods class in the future and are seeing if all the chatter among students is true.

No matter how you got here, two things are probably true. First, you are taking this research methods course because it is a requirement for your major. The bottom line is that most of the students who read this text are required to take a research methods course. While you may think studying research methods is irrelevant to your career goals, unnecessary, overly academic, or perhaps even intimidating, you probably must finish this course in order to graduate. Second, you have heard negative comments about this course. The negative comments mention the difficulty of the course and the relevance of the course (e.g., �I am going to be a police officer, so why do I need to take a research methods course?�). If you are like most students we have experienced in our research methods courses in the past, you are not initially interested in this course and are concerned about whether you will do well in it.

If you are concerned about the course, realize that you are not alone because most students are anxious about taking a research methods course. Also realize that your professor is well aware of student anxiety and apprehension regarding research methods. So, relax and do not think about the entire course and the entire book. Take the course content one chapter, one week at a time. One of the advantages of taking a research methods course is that you learn about the process of research methods. Each chapter builds upon the previous chapters, illustrating and discussing more about the research process. This is certainly an advantage, but it is also critical that you understand the initial chapters in this book so you are not confused with the content discussed in later chapters. In addition to anxiety and apprehension over the course material, research methods can be boring if you only read and learn about it with no particular purpose in mind. Although examples are prevalent throughout the book, as you read this material, it is recommended that you think about the relevancy and application of the topics covered in this book to your specific criminal justice interests. As you continue to read the book, think about how you might use the information you are reading in your current position or your intended profession.

The goal of this research methods book is to develop you into an informed consumer of research. Most, if not all, of your fellow classmates will never conduct their own research studies. However, every one of you will be exposed to research findings in your professional and personal lives for the remainder of your lives. You are exposed to research findings in the media (e.g., television, newspapers, and online), in personal interaction with others (e.g., friends and family, doctors, and professors), as well as in class. You should challenge yourself for this semester to keep a journal and document exposure to research in your daily life outside of college whether through the nightly news, newspapers, magazine articles, Internet, personal conversations, or other means. At the end of the semester, you will be amazed at the amount of research you are exposed to in a short period of time. This book is focused on research exposure and assisting you to become an educated consumer of research by providing you the skills necessary to differentiate between good and not so good research. Why should you believe research findings if the study is faulty? Without being an educated consumer of research, you will not be able to differentiate between useful and not useful research. This book is designed to remedy this problem.

This book was written to make your first encounter with research methods relevant and successful while providing you the tools necessary to become an educated consumer of research. Therefore, this book is written with the assumption that students have not had a prior class on research methods. In addition, this book assumes that practical and evaluative knowledge of research methods is more useful than theoretical knowledge of the development of research methods and the relationship between theory and research. Since the focus of this book is on consumerism, not researcher training, practical and evaluative knowledge is more useful than theoretical knowledge.

It is also important to understand that the professors who design academic programs in criminal justice at the associate and bachelor level believe that an understanding of research methods is important for students. That is why, more than likely, this research methods course is a required course in your degree program. These professors understand that a solid understanding of research methods will enrich the qualifications of students for employment and performance in their criminal justice careers.

As previously stated, the basic goal of this book is to make students, as future and possibly even current practitioners in the criminal justice system, better informed and more capable consumers of the results of criminal justice research. This goal is based on the belief that an understanding of research methods allows criminal justice practitioners to be better able to make use of the results of research as it applies to their work-related duties. In fact, thousands of research questions are asked and answered each year in research involving criminal justice and criminological topics. In addition, thousands of articles are published, papers presented at conferences, and reports prepared that provide answers to these questions. The ability to understand research gives practitioners knowledge of the most current information in their respective fields and the ability to use this knowledge to improve the effectiveness of criminal justice agencies.

How Do We Know What We Know? Sources of Knowledge

The reality is the understanding of crime and criminal justice system operations by the public is frequently the product of misguided assumptions, distorted interpretations, outright myths, and hardened ideological positions. 6 This is a bold statement that basically contends that most people�s knowledge of crime and criminal justice is inaccurate. But, how do these inaccuracies occur? Most people have learned what they know about crime and criminal justice system operations through some other means besides scientific research results and findings. Some of that knowledge is based on personal experience and common sense. Much of it is based on the information and images supported by politicians, governmental agencies, and especially the media. This section will discuss the mechanisms used to understand crime and criminal justice operations by the public. It is important to note that although this section will focus on the failings of these knowledge sources, they each can be, and certainly are, accurate at times, and thus are valuable sources of knowledge.

Knowledge from Authority

We gain knowledge from parents, teachers, experts, and others who are in positions of authority in our lives. When we accept something as being correct and true just because someone in a position of authority says it is true, we are using what is referred to as authority knowledge as a means of knowing. Authorities often expend significant time and effort to learn something, and we can benefit from their experience and work.

However, relying on authority as a means of knowing has limitations. It is easy to overestimate the expertise of other people. A person�s expertise is typically limited to a few core areas of significant knowledge; a person is not an expert in all areas. More specifically, criminal justice professors are not experts on all topics related to criminal justice. One professor may be an expert on corrections but know little about policing. If this professor discusses topics in policing in which he is not an expert, we may still assume he is right when he may be wrong. Authority figures may speak on fields they know little about. They can be completely wrong but we may believe them because of their status as an expert. Furthermore, an expert in one area may try to use his authority in an unrelated area. Other times, we have no idea of how the experts arrived at their knowledge. We just know they are experts in the topic area.

As I am writing this, I recall an example of authority knowledge that was wrong during my police academy training in the late 1980s. My academy training was about four years after the U.S. Supreme Court decision in Tennessee v. Garner. 7 In this case, the Court limited the use of deadly force by police to defense of life situations and incidents where the suspect committed a violent offense. Prior to the decision, the police in several states could use deadly force on any fleeing suspect accused of a felony offense. One day, the academy class was practicing mock traffic stops. During one of my mock traffic stops, I received information that the vehicle I stopped was stolen. The driver and passenger exited the vehicle and fled on foot. I did not use deadly force (this was a training exercise so was not real) against the suspects and was chastised by my instructor who insisted that I should have shot the suspects as they were fleeing. Training instructors, just like professors, convey authority knowledge but, in this case, the instructor was wrong. I was not legally authorized to use deadly force in the traffic stop scenario despite the insistence of my instructor to the contrary.

Politicians are sometimes taken as a source of authority knowledge about the law, crime, and criminal justice issues. Since they enact laws that directly impact the operations of the criminal justice system, we may assume they are an authority on crime and criminal justice. More specifically, we may assume that politicians know best about how to reduce crime and increase the effectiveness of the criminal justice system. However, history is rife with laws that sounded good on paper but had no impact on crime. For example, there is little evidence that sex offender registration protects the public from sexual predators or acts as a deterrent to repeat sex offenders even though every state has a law requiring convicted sex offenders to register with local authorities. Perhaps politicians are not the criminal justice experts some perceive them to be.

History is also full of criminal justice authorities that we now see as being misinformed. For example, Cesare Lombroso is the father of the positivist school of criminology. He is most readily recognized for his idea that some individuals are born criminal. He stated that criminals have certain unique biological characteristics, including large protruding jaws, high foreheads, flattened noses, and asymmetrical faces, to name a few. 8 These characteristics were similar to those found in primitive humans. Therefore, Lombroso argued that some individuals were genetic �throwbacks� to a more primitive time and were less evolved than other people and thus, were more likely to be criminals. Lombroso�s research has been discredited because he failed to compare criminals with noncriminals. By studying only criminals, he found characteristics that were common to criminals. However, if Lombroso had studied a group of noncriminals, he would have discovered that these biological characteristics are just as prevalent among noncriminals. This example involves authority knowledge that is supported by research but the research methods used were flawed. The errors of Lombroso seem obvious now, but what do we know today through authority knowledge that is inaccurate or will be proven wrong in the future?

Knowledge from Tradition

In addition to authority knowledge, people often rely on tradition for knowledge. Tradition knowledge relies on the knowledge of the past. Individuals accept something as true because that is the way things have always been so it must be right. A good example of tradition knowledge is preventive/random patrol. Ever since vehicles were brought into the police patrol function, police administrators assumed that having patrol officers drive around randomly in the communities they serve, while they are not answering calls for service, would prevent crime. If you were a patrol officer in the early 1970s and asked your supervisor, �Why do I drive around randomly throughout my assigned area when I am not answering a call for service?� the answer would have been, �That is the way we have always done patrol and random patrol reduces crime through deterrence.� The Kansas City Preventive Patrol Experiment challenged the tradition knowledge that preventive/random patrol reduces crime. The results of the study made it clear that the traditional practice of preventive/random patrol had little to no impact on reducing crime. This allowed police departments to develop other patrol deployment strategies such as directed patrol and �hot spots� policing since preventive patrol was seen as ineffective. The development of effective patrol deployment strategies continues today.

Knowledge from Common Sense

We frequently rely on common sense knowledge for what we know about crime and the criminal justice system because it �just makes sense.� For example, it �just makes sense� that if we send juvenile delinquents on a field trip to prison where they will see first hand the prison environment as well as be yelled at by actual prisoners, they will refrain from future delinquency. That is exactly what the program Scared Straight, originally developed in the 1970s, is designed to do. Scared Straight programs are still in existence today and are even the premise for the television show Beyond Scared Straight on the A&E television network. As originally created, the program was designed to decrease juvenile delinquency by bringing at-risk and delinquent juveniles into prison where they would be �scared straight� by inmates serving life sentences. Participants in the program were talked to and yelled at by the inmates in an effort to scare them. It was believed that the fear felt by the participants would lead to a discontinuation of their delinquent behavior so that they would not end up in prison themselves. This sounds like a good idea. It makes sense, and the program was initially touted as a success due to anecdotal evidence based on a few delinquents who turned their lives around after participation in the program.

However, evaluations of the program and others like it showed that the program was in fact unsuccessful. In the initial evaluation of the Scared Straight program, Finckenauer used a classic experimental design (discussed in Chapter 5), to evaluate the original �Lifer�s Program� at Rahway State Prison in New Jersey where the program was initially developed. 13 Juveniles were randomly assigned to an experimental group that attended the Scared Straight program and a control group that did not participate in the program. Results of the evaluation were not positive. Post-test measures revealed that juveniles who were assigned to the experimental group and participated in the program were actually more seriously delinquent afterwards than those who did not participate in the program. Also using an experimental design with random assignment, Yarborough evaluated the �Juvenile Offenders Learn Truth� (JOLT) program at the State Prison of Southern Michigan at Jackson. 14 This program was similar to that of the �Lifer�s Program,� only with fewer obscenities used by inmates. Post-test measurements were taken at two intervals, three and six months after program completion. Again, results were not positive. Findings revealed no significant differences in delinquency between those juveniles who attended the program and those who did not. Other experiments conducted on Scared Straight- type programs further revealed their inability to deter juveniles from further delinquency. 15 Despite the common sense popularity of these programs, the evaluations showed that Scared Straight programs do not reduce delinquency and, in some instances, may actually increase delinquency. The programs may actually do more harm than good. I guess that begs the question, �Why do we still do these types of programs?�

Scared Straight programs and other widely held common sense beliefs about crime and the criminal justice system are questionable, based on the available research evidence. Common sense is important in our daily lives and is frequently correct, but, at times, it also contains inaccuracies, misinformation, and even prejudice.

CLASSICS IN CJ RESEARCH

Is It Safe to Put Felons on Probation?

Research Study 9

In the mid-1970s, the number of offenders on probation began to significantly increase. By the mid-1980s, probation was the most frequently used sentence in most states and its use was becoming more common for felons, whereas previously, probation was typically limited to misdemeanor crimes and offenses committed by juveniles. Increasing numbers of felony offenders were being placed on probation because judges had no other alternative forms of punishment. Prisons were already operating above capacity due to rising crime rates. Despite the increase in the use of probation in the 1980s, few empirical studies of probation (particularly its use with felony offenders) had been published. In the early 1980s, the Rand Corporation conducted an extensive study of probation to learn more about the offenders sentenced to probation and the effectiveness of probation as a criminal sanction. At the time the study began, over one-third of California�s probation population were convicted felons. 10 This was the first large-scale study of felony probation.

Is it safe to put felons on probation?

Data for the study were obtained from the California Board of Prison Terms (CBPT). The Board had been collecting comprehensive data on all offenders sentenced to prison since 1978 and on a sample of adult males from 17 counties who received probation. From these two data sources, researchers selected a sample of male offenders who had been convicted of the following crimes: robbery, assault, burglary, theft, forgery, and drug offenses. These crimes were selected because an offender could receive either prison or probation if convicted. Approximately 16,500 male felony offenders were included in the study. For each offender, researchers had access to their personal characteristics, information on their crimes, court proceedings, and disposition.

Two main research questions were answered in this study. First, what were the recidivism rates for felony offenders who received probation? When assessing recidivism rates, the study found that the majority of offenders sentenced to probation recidivated during the follow-up period, which averaged 31 months. Overall, 65% of the sample of probationers were re-arrested and 51 % were charged with and convicted of another offense. A total of 18% were convicted of a violent crime.

The second research question asked, what were the characteristics of the probationers who recidivated? Property offenders were more likely to recidivate compared to violent or drug offenders. Researchers also discovered that probationers tended to recidivate by committing the same crime that placed them on probation. Rand researchers included time to recidivism in their analysis and found that property and violent offenders recidivated sooner than drug offenders. The median time to the first filed charge was five months for property offenders and eight months for violent offenders.

The issue of whether or not the findings would generalize to other counties in California and to other states was raised. Data for the study came from probation and prison records from two counties in California. These two counties were not randomly selected, but were chosen because of their large probation populations and the willingness of departments to provide information. Further, the probation departments in these counties had experienced significant budget cuts. Supervision may have become compromised as a result and this could have explained why these counties had high rates of recidivism. Studies of probation recidivism in other states have found recidivism rates to be much lower, suggesting the Rand results may not have applied elsewhere. 11 Several studies examining the effectiveness of probation and the factors correlated with probation outcomes were published after 1985. Much of this research failed to produce results consistent with the Rand study.

The Rand study of felony probation received a considerable amount of attention within the field of corrections. According to one scholar, the study was acclaimed as �the most important criminological research to be reported since World War II.� 12 The National Institute of Justice disseminated the report to criminal justice agencies across the country and even highlighted the study in their monthly newsletter. Today, the study remains one of the most highly cited pieces of corrections research.

According to Rand researchers, these findings raised serious doubts about the effectiveness of probation for felony offenders. Most of the felons sentenced to probation recidivated and researchers were unable to develop an accurate prediction model to improve the courts� decision-making. The continued use of probation as a sanction for felony offenders appeared to be putting the public at risk. However, without adequate prison space, the courts had no other alternatives besides probation when sentencing offenders.

The researchers made several recommendations to address the limitations of using probation for felony offenders. First, it was recommended that states formally acknowledge that the purpose of probation had changed. Probation was originally used as a means of furthering the goal of rehabilitation in the correctional system. As the United States moved away from that goal in the late 1960s, the expectations of probation changed. Probation was now used as a way to exercise �restrictive supervision� over more serious offenders. Second, probation departments needed to redefine the responsibilities of their probation officers. Probation officers were now expected to be surveillance officers instead of treatment personnel, which required specialized training. In addition, states needed to explore the possibility of broadening the legal authority of its probation officers by allowing them to act as law enforcement officers if necessary. Third, states were advised to adopt a formal client management system that included risk/need assessments of every client. Such a system would help establish uniform, consistent treatment of those on probation and would also help departments allocate their resources efficiently and effectively. Fourth, researchers encouraged states to develop alternative forms of community punishment that offered more public protection than regular probation, which led to the development and use of intensive supervision probation, house arrest, electronic monitoring, day reporting centers, and other intermediate punishments.

Knowledge from Personal Experience

If you personally see something or if it actually happens to you, then you are likely to accept it as true and gain knowledge from the experience. Gaining knowledge through actual experiences is known as personal experience knowledge, and it has a powerful and lasting impact on everyone. Personal experiences are essential building blocks of knowledge and of what we believe to be true. The problem with knowledge gained from personal experiences is that personal experiences can be unique and unreliable, which can distort reality and lead us to believe things that are actually false.

How can events that someone personally experienced be wrong? The events are not wrong. Instead, the knowledge gained from the experience is wrong. For example, the research consistently shows that a person�s demeanor significantly impacts the decision-making of police officers. During a traffic stop, if a person is rude, disrespectful, and uncooperative to the officer, then the driver is more likely to receive a traffic citation than a warning. That is what the research on police discretion shows. However, if a person was rude and uncooperative to a police officer during a traffic stop and was let go without a citation, the person will gain knowledge from this personal experience. The knowledge gained may include that being disrespectful during future traffic stops will get this person out of future tickets. Not likely. The event is not wrong. Instead, the knowledge gained from the experience is wrong because being disrespectful to the police usually leads to more enforcement action taken by the police, not less.

As a student in criminal justice, you have probably experienced something similar in interaction with friends, relatives, and neighbors. Your knowledge of criminal justice that you have developed in your criminal justice classes is trumped by one experience your friend, relative, or neighbor had with the criminal justice system. They believe they are right because they experienced it. However, there are four errors that occur in the knowledge gained from personal experiences: overgeneralization, selective observation, illogical reasoning, and resistance to change.

Overgeneralization happens when people conclude that what they have observed in one or a few cases is true for all cases. For example, you may see that a wealthy businesswomen in your community is acquitted of bribery and may conclude that �wealthy people, especially women, are never convicted in our criminal justice system,� which is an overgeneralization. It is common to draw conclusions about people and society from our personal interactions, but, in reality, our experiences are limited because we interact with just a small percentage of people in society.

The same is true for practitioners in the criminal justice system. Practitioners have a tendency to believe that because something was done a particular way in their agency, it is done that way in all agencies. That may not be true. Although there are certainly operational similarities across criminal justice agencies, there are also nuances that exist across the over 50,000 criminal justice agencies in the United States. Believing that just because it was that way in your agency, it must be that way in all agencies leads to overgeneralization.

Selective observation is choosing, either consciously or unconsciously, to pay attention to and remember events that support our personal preferences and beliefs. In fact, with selective observation, we will seek out evidence that confirms what we believe to be true and ignore the events that provide contradictory evidence. We are more likely to notice pieces of evidence that reinforce and support our ideology. As applied to the criminal justice system, when we are inclined to be critical of the criminal justice system, it is pretty easy to notice its every failing and ignore its successes. For example, if someone believes the police commonly use excessive force, the person is more likely to pay attention to and remember a police brutality allegation on the nightly news than a police pursuit that led to the apprehension of the suspect without incident on the same nightly news. As another example, if you believe treatment efforts on sex offenders are futile, you will pay attention to and remember each sex offender you hear about that recidivates but will pay little attention to any successes. It is easy to find instances that confirm our beliefs, but with selective observation, the complete picture is not being viewed. Therefore, if we only acknowledge the events that confirm our beliefs and ignore those that challenge them, we are falling victim to selective observation.

Besides selective observation, some of our observations may simply be wrong. Consider eyewitness identification. It is a common practice in the criminal justice system, but research has consistently demonstrated inaccuracies in eyewitness identification. The witness feels certain that the person viewed is the person who committed the offense, but sometimes the witness is wrong. Even when our senses of sight, hearing, taste, touch, and smell are fully operational, our minds have to interpret what we have sensed, which may lead to an inaccurate observation.

RESEARCH IN THE NEWS

When Your Criminal Past Isn�t Yours 16

The business of background checks on prospective employees is increasing significantly. According to the Society for Human Resource Management, since the events of September 11, 2001, the percentage of companies that conduct criminal history checks during the hiring process has risen past 90%. Employers spend at least $2 billion a year to look into the pasts of their prospective employees. Problems with the business of background checks were identified through research that included a review of thousands of pages of court filings and interviews with dozens of court officials, data providers, lawyers, victims, and regulators.

The business of background checks is a system weakened by the conversion to digital files and compromised by the significant number of private companies that profit by amassing public records and selling them to employers. The private companies create a system in which a computer program scrapes the public files of court systems around the country to retrieve personal data. Basically, these are automated data-mining programs. Today, half the courts in the United States put criminal records on their public websites. So, the data are there for the taking, but the records that are retrieved typically are not checked for errors�errors that would be obvious to human eyes.

The errors can start with a mistake entered into the logs of a law enforcement agency or a court file. The biggest culprits, though, are companies that compile databases using public information. In some instances, their automated formulas misinterpret the information provided them. Other times, records wind up assigned to the wrong people with a common name. Furthermore, when a government agency erases a criminal conviction after a designated period of good behavior, many of the commercial databases don�t perform the updates required to purge offenses that have been removed from public record. It is clear that these errors can have substantial ramifications, including damaged reputations and loss of job opportunities.

Illogical reasoning occurs when someone jumps to premature conclusions or presents an argument that is based on invalid assumptions. Premature conclusions occur when we feel we have the answer based on a few pieces of evidence and do not need to seek additional information that may invalidate our conclusion. Think of a detective who, after examining only a few pieces of evidence, quickly narrows in on a murder suspect. It is common for a detective to assess the initial evidence and make an initial determination of who committed the murder. However, it is hoped that the detective will continue to sort through all the evidence for confirmation or rejection of his original conclusion regarding the murder suspect. Illogical reasoning by jumping to premature conclusions is common in everyday life. We look for evidence to confirm or reject our beliefs and stop when a small amount of evidence is present; we jump to conclusions. If a person states, �I know four people who have dropped out of high school, and each one of them ended up addicted to drugs, so all dropouts abuse drugs,� the person is jumping to conclusions.

Illogical reasoning also occurs when an argument, based on invalid assumptions, is presented. Let�s revisit the Scared Straight example previously discussed. Program developers assumed that brief exposure to the harsh realities of prison would deter juveniles from future delinquency. The Scared Straight program is an example of illogical reasoning. Four hours of exposure to prison life is not going to counteract years of delinquency and turn a delinquent into a nondelinquent. The program is based on a false assumption and fails to recognize the substantial risk factors present in the lives of most delinquents that must be mediated before the juvenile can live a crime-free lifestyle. A fear of prison, developed through brief exposure, is not enough to counteract the risk factors present in the lives of most delinquents. Although the Scared Straight program sounds good, it is illogical to assume that a brief experience with prison life will have a stronger impact on the decisions made by delinquents than peer support for delinquency, drug abuse, lack of education, poor parental supervision, and other factors that influence delinquency.

Resistance to change is the reluctance to change our beliefs in light of new, accurate, and valid information to the contrary. Resistance to change is common and it occurs for several reasons. First, even though our personal experience may be counter to our belief system, it is hard to admit we were wrong after we have taken a position on an issue. Even when the research evidence shows otherwise, people who work within programs may still believe they are effective. As previously stated, even though the research evidence shows otherwise, Scared Straight programs still exist and there is even a television show devoted to the program. Second, too much devotion to tradition and the argument that this is the way it has always been done inhibits change and hinders our ability to accept new directions and develop new knowledge. Third, uncritical agreement with authority inhibits change. Although authority knowledge is certainly an important means of gaining knowledge, we must critically evaluate the ideas, beliefs, and statements of those in positions of authority and be willing to challenge those statements where necessary. However, people often accept the beliefs of those in positions of authority without question, which hinders change.

Knowledge from Media Portrayals

Television shows, movies, websites, newspapers, and magazine articles are important sources of information. This is especially true for information about crime and the criminal justice system since most people have not had much contact with criminals or the criminal justice system. Instead of gaining knowledge about the criminal justice system through personal experience, most people learn about crime and the operations of the criminal justice system through media outlets. Since the primary goal of many of these media outlets is to entertain, they may not accurately reflect the reality of crime and criminal justice. Despite their inaccuracies, the media has a substantial impact on what people know about crime and the criminal justice system. Most people know what they know about crime and criminal justice through the media, and this knowledge even has an impact on criminal justice system operations.

An example of the potential impact of the media on the actual operations of the criminal justice system involves the CSI: Crime Scene Investigation television shows. The shows have been criticized for their unrealistic portrayal of the role of forensic science in solving criminal cases. Critics claim that CSI viewers accept what they see on the show as an accurate representation of how forensic science works. When summoned for jury duty, they bring with them unrealistic expectations of the forensic evidence they will see in trial. When the expected sophisticated forensic evidence is not presented in the real trial, the juror is more likely to vote to acquit the defendant. This phenomenon is known as the CSI Effect. Has the research shown that the CSI Effect exists and is impacting the criminal justice system? Most of the research shows that the CSI Effect does not exist and thus does not impact juror decision-making, but other research has shown that viewers of CSI have higher expectations related to evidence presented at trial. 17

There are several instances in which media attention on a particular topic created the idea that a major problem existed when it did not. An example is Halloween sadism. Halloween sadism is the practice of giving contaminated treats to children during trick or treating. 18 In 1985, Joel Best wrote an article entitled, �The Myth of the Halloween Sadist.� 19 His article reviewed press coverage of Halloween sadism in the leading papers in the three largest metropolitan areas ( New York Times, Los Angeles Times, and Chicago Tribune ) from 1958�1984. Although the belief in Halloween sadism is widespread, Best found few reported incidents and few reports of children being injured by Halloween sadism. Follow-ups on these reported incidents led to the conclusion that most of these reports were hoaxes. Best concluded, �I have been unable to find a substantiated report of a child being killed or seriously injured by a contaminated treat picked up in the course of trick or treating.� 20 Since 1985, Best has kept his research up to date and has come to the same conclusion. Halloween sadism is an urban legend; it is a story that is told as true, even though there is little or no evidence that the events in the story ever occurred.

Dispelling Myths: The Power of Research Methods

In the prior section, sources of knowledge were discussed along with the limitations of each. A researcher (e.g., criminologist), ideally, takes no knowledge claim for granted, but instead relies on research methods to discover the truth. In the attempt to generate new knowledge, a researcher is skeptical of knowledge that is generated by the sources discussed in the prior section, and this skepticism leads to the questioning of conventional thinking. Through this process, existing knowledge claims are discredited, modified, or substantiated. Research methods provide the researcher with the tools necessary to test current knowledge and discover new knowledge.

Although knowledge developed through research methods is by no means perfect and infallible, it is definitely a more systematic, structured, precise, and evidence-based process than the knowledge sources previously discussed. However, researchers should not dismiss all knowledge from the prior sources discussed, because, as mentioned, these sources of knowledge are sometimes accurate and certainly have their place in the development of knowledge. Researchers should guard against an elitist mind-set in which all knowledge, unless it is research-based knowledge, is dismissed.

To further discuss the importance of research methods in the development of knowledge, this section will discuss myths about crime and criminal justice. Myths are beliefs that are based on emotion rather than rigorous analysis. Take the myth of the Halloween sadist previously discussed. Many believe that there are real examples of children being harmed by razor blades, poison, or other nefarious objects placed in Halloween candy. This belief has changed the practices of many parents on Halloween; not allowing their children to trick-or-treat in their neighborhood and forbidding them from going to the doors of strangers. After careful analysis by Best, there is not a single, known example of children being seriously injured or killed by contaminated candy given by strangers. The Halloween sadist is a myth but it is still perpetuated today, and as the definition states, it is a belief based upon emotion rather than rigorous analysis. People accept myths as accurate knowledge of reality when, in fact, the knowledge is false.

The power of research is the ability to dispel myths. If someone were to assess the research literature on a myth or do their own research, she would find that the knowledge based on the myth is wrong. Perceived reality is contradicted by the facts developed through research. But that does not mean that the myth still doesn�t exist. It is important to keep in mind that the perpetuation and acceptance of myths by the public, politicians, and criminal justice personnel has contributed to the failure of criminal justice practices and policies designed to reduce crime and improve the operations of the criminal justice system. In this section, a detailed example of a myth about crime, police, courts, and corrections will be presented to demonstrate how the myth has been dispelled through research. In addition, several additional myths about crime, police, courts, and corrections will be briefly presented.

The Health Benefits of Alcohol Consumption 21

The press release from Oregon State University is titled �Beer Compound Shows Potent Promise in Prostate Cancer Battle.� The press release leads to several newspaper articles throughout the country written on the preventative nature of drinking beer on prostate cancer development with titles such as �Beer Protects Your Prostate� and �Beer May Help Men Ward Off Prostate Cancer.� By the titles alone, this sounds great; one of the main ingredients in beer appears to thwart prostate cancer.

The study that generated these headlines was conducted by a group of researchers at Oregon State University using cultured cells with purified compounds in a laboratory setting. The research showed that xanthohumol, a compound found in hops, slowed the growth of prostate cancer cells and also the growth of cells that cause enlarged prostates. But you would have to drink more than 17 pints of beer to consume a medically effective dose of xanthohumol, which is almost a case of beer. In addition, although the research is promising, further study is necessary to determine xanthohumol�s true impact on prostate cancer.

These are the types of headlines that people pay attention to and want to believe as true, even if disproven by later research. People want to believe that there are health benefits to alcohol consumption. You have probably heard about the health benefits of drinking red wine, but here is something you should consider. Recently, the University of Connecticut released a statement describing an extensive research misconduct investigation involving a member of its faculty. The investigation was sparked by an anonymous allegation of research irregularities. The comprehensive report of the investigation, which totals approximately 60,000 pages, concludes that the professor is guilty of 145 counts of fabrication and falsification of data. The professor had gained international notoriety for his research into the beneficial properties of resveratrol, which is found in red wine, especially its impact on aging. Obviously, this throws his research conclusions, that red wine has a beneficial impact on the aging process, into question.

Myths about Crime�Drug Users Are Violent

The myth of drug users as violent offenders continues to be perpetuated by media accounts of violent drug users. The public sees drug users as violent offenders who commit violent crimes to get money for drugs or who commit violent crimes while under the intoxicating properties of drugs. The public also recognizes the violent nature of the drug business with gangs and cartels using violence to protect their turf. In May 2012, extensive media attention was given to the case of the Miami man who ate the face of a homeless man for an agonizing 18 minutes until police shot and killed the suspect. The police believed that the suspect was high on the street drug known as �bath salts.� This horrific case definitely leaves the image in the public�s mind about the relationship between violence and drug use.

In recent years, media reports have focused on the relationship between methamphetamine use and violence; before then it was crack cocaine use and violence. 32 However, media portrayals regarding the violent tendencies of drug users date back to the 1930s and the release of Reefer Madness. In 1985, Goldstein suggested that drugs and violence could be related in three different ways:

1. violence could be the direct result of drug ingestion;

2. violence could be a product of the instability of drug market activity; and

3. violence could be the consequence of people having a compulsive need for drugs or money for drugs. 33

So, what does the research show? Studies have found that homicides related to crack cocaine were usually the product of the instability of drug market activity (i.e., buying and selling drugs can be a violent activity) and rarely the result of drug ingestion. 34 After an extensive review of research studies on alcohol, drugs, and violence, Parker and Auerhahn concluded, �Despite a number of published statements to the contrary, we find no significant evidence suggesting that drug use is associated with violence. There is substantial evidence to suggest that alcohol use is significantly associated with violence of all kinds.� 35 The reality is not everyone who uses drugs becomes violent and users who do become violent do not do so every time they use drugs; therefore, the relationship between violence and drug use is a myth.

MYTHS ABOUT CRIME

Some additional myths about crime that research does not support include:

�Crime statistics accurately show what crimes are being committed and what crimes are most harmful. 22

�Most criminals�especially the dangerous ones�are mentally ill. 23

�White-collar crime is only about financial loss and does not hurt anyone. 24

�Serial murderers are middle-aged, white males. 25

�Criminals are significantly different from noncriminals. 26

�People are more likely to be a victim of violent crime committed by a stranger than by someone they know. 27

�Older adults are more likely to be victimized than people in any other age group. 28

�Sex offender registration protects the public from sexual predators. 29

�Juvenile crime rates are significantly increasing. 30

�Only the most violent juveniles are tried as adults. 31

Myths about Police�Female Police Officers Do Not Perform as Well as Males

Female police officers still face the myth that they cannot perform as well as male police officers. Throughout history, females have faced significant difficulties even becoming police officers. In the past, it was common for police agencies to require all police applicants to meet a minimum height requirement to be considered for employment. The minimum height requirement was 5?8? for most agencies, which limited the ability of females to successfully meet the minimum standards to become a police officer. Even if women could meet the minimum height requirements, they were typically faced with a physical-abilities test that emphasized upper body strength (e.g., push-ups and bench presses). Women failed these tests more often than men, and thus were not eligible to be police officers. Minimum height requirements are no longer used in law enforcement, but the perception that female police officers are not as good as males still exists. Today, the myth that women cannot be effective police officers is based largely on the belief that the need to demonstrate superior physical strength is a daily, common occurrence in law enforcement along with the belief that police work is routinely dangerous, violent, and crime-related.

So, what does the research show? On occasion, it is useful for police officers to be able to overpower suspects by demonstrating superior physical strength, but those types of activities are rare in law enforcement. In addition, it is fairly rare for a police officer to have to deal with a dangerous and violent encounter or even an incident involving a crime. The Police Services Study conducted in the 1970s analyzed 26,418 calls for service in three metropolitan areas and found that only 19% of calls for service involve crime and only 2% of the total calls for service involve violent crime. 43 This research study was among the first to assess the types of calls for service received by police agencies.

Despite the belief that women do not make good police officers, consistent research findings show that women are extremely capable as police officers, and in some respects, outperform their male counterparts. 44 Research has demonstrated several advantages to the hiring, retention, and promotion of women in law enforcement. First, female officers are as competent as their male counterparts. Research does not show any consistent differences in how male and female patrol officers perform their duties. Second, female officers are less likely to use excessive force. Research has shown that female patrol officers are less likely to be involved in high-speed pursuits, incidents of deadly force, and the use of excessive force. Female officers are more capable at calming potentially violent situations through communication and also demonstrate heightened levels of caution. Third, female officers can help implement community-oriented policing. Studies have shown that female officers are more supportive of the community-policing philosophy than are their male counterparts. Fourth, female officers can improve law enforcement�s response to violence against women. Studies have shown that female officers are more patient and understanding in handling domestic violence calls, and female victims of domestic violence are more likely to provide positive evaluations of female officers than their male counterparts. 52

MYTHS ABOUT POLICE

Some additional myths about the police that research does not support include:

�Police target minorities for traffic stops and arrests. 36

�Most crimes are solved through forensic science. 37

�COMPSTAT reduces crime. 38

�Intensive law enforcement efforts at the street level will lead to the control of illicit drug use and abuse. 39

�Police work primarily entails responding to crimes in progress or crimes that have just occurred. 40

�Police presence reduces crime. 41

�Detectives are most responsible for solving crimes and arresting offenders. 42

Myths about Courts�The Death Penalty Is Administered Fairly

According to a recent Gallup poll, 52% of Americans say the death penalty is applied fairly in the United States, the lowest mark in almost 40 years. 53 The issue of fairness and the death penalty typically concerns whether the punishment is equally imposed on offenders who are equally deserving based on legal factors (i.e., similar offense, similar prior criminal history, similar aggravating circumstances, and similar mitigating circumstances). 54 Unfairness can be shown if similarly situated offenders are more or less likely to receive death sentences based on age, gender, and race.

So, what does the research show? First, has research shown that a defendant�s age influences his or her chances of being sentenced to death? A study of about 5,000 homicides, controlling for legally relevant variables, found that defendants over the age of 25 were more than twice as likely to receive the death penalty in comparison to those 25 years of age or younger. 55

Second, has research shown that a defendant�s gender influences his or her chance of being sentenced to death? Capital punishment is almost exclusively reserved for male defendants. On December 31, 2010, there were 3,158 prisoners under a sentence of death in the United States: 58 were women, or 1.8%. 56 However, women account for 10�12% of all murders in the United States. 57 One research study found that male defendants were 2.6 times more likely than females to receive a death sentence after controlling for legally relevant factors. 58

Third, has research shown that a defendant�s race influences his or her chance of being sentenced to death? Most of the research on the biased nature of the death penalty has focused on racial inequities in the sentence. Although some research has shown that a defendant�s race has an impact on the likelihood of receiving a death sentence, a significant amount of research has shown that the race of the victim has the most substantial impact on death sentences. The research evidence clearly shows that offenders who murder white victims are more likely to receive a death sentence than offenders who murder black victims. 59 When assessing the race of both the victim and offender, the composition most likely to receive the death penalty is when a black offender murders a white victim. 60

MYTHS ABOUT COURTS

Some additional myths about courts that research does not support include:

�Many criminals escape justice because of the exclusionary rule. 45

�Subjecting juvenile offenders to harsh punishments can reduce crime committed by juveniles. 46

�Public opinion is overwhelmingly in favor of imprisonment and harsh punishment for offenders. 47

�The death penalty brings closure and a sense of justice to the family and friends of murder victims. 48

�Insanity is a common verdict in criminal courts in the United States. 49

�Eyewitness identification is reliable evidence. 50

�Most people who commit crimes based on hatred, bias, or discrimination face hate crime charges and longer sentencing. 51

Myths about Corrections�Imprisonment Is the Most Severe Form of Punishment

It seems clear that besides the death penalty, the most severe punishment available in our criminal justice system is to lock up offenders in prison. On a continuum, it is perceived that sentence severity increases as one moves from fines, to probation, to intermediate sanctions such as boot camps, and finally, to incarceration in prison. The public and politicians support this perception as well.

So, what does the research show? What do criminals think is the most severe form of punishment? A growing body of research has assessed how convicted offenders perceive and experience the severity of sentences in our criminal justice system. 61 Research suggests that alternatives to incarceration in prison (i.e., probation and intermediate sanctions) are perceived by many offenders as more severe due to a greater risk of program failure (e.g., probation revocation). In comparison, serving prison time is easier. 62 �

For example, one study found that about one-third of nonviolent offenders given the option of participating in an Intensive Supervision Probation (ISP) program, chose prison instead because the prospects of working every day and submitting to random drug tests was more punitive than serving time in prison. 73 Prisoners also stated that they would likely be caught violating probation conditions (i.e., high risk of program failure) and be sent to prison anyway. 74 In another research study involving survey responses from 415 inmates serving a brief prison sentence for a nonviolent crime, prison was considered the eighth most severe sanction, with only community service and probation seen as less punitive. Electronic monitoring (seventh), intensive supervision probation (sixth), halfway house (fifth), intermittent incarceration (fourth), day reporting (third), county jail (second), and boot camp (first) were all rated by inmates as more severe sanctions than prison. 75

MYTHS ABOUT CORRECTIONS

Some additional myths about corrections that research does not support include:

�Punishing criminals reduces crime. 63

�Prisons are too lenient in their day-to-day operations (prisons as country clubs). 64

�Prisons can be self-supporting if only prisoners were forced to work. 65

�Private prisons are more cost effective than state-run prisons. 66

�Focus of community corrections is rehabilitation rather than punishment. 67

�Correctional rehabilitation does not work. 68

�Drug offenders are treated leniently by the criminal justice system. 69

�Most death row inmates will be executed eventually. 70

�If correctional sanctions are severe enough, people will think twice about committing crimes. 71

�Sexual violence against and exploitation of inmates of the same gender are primarily the result of lack of heterosexual opportunities. 72

What is Research and Why is It Important to be an Informed Consumer of Research?

We probably should have started the chapter with the question �What is research?� but we wanted to initially lay a foundation for the question with a discussion of the problems with how knowledge is developed and the power of research in discovering the truth. Research methods are tools that allow criminology and criminal justice researchers to systematically study crime and the criminal justice system. The study of research methods is the study of the basic rules, appropriate techniques, and relevant procedures for conducting research. Research methods provide the tools necessary to approach issues in criminal justice from a rigorous standpoint and challenge opinions based solely on nonscientific observations and experiences. Similarly, research is the scientific investigation of an issue, problem, or subject utilizing research methods. Research is a means of knowledge development that is designed to assist in discovering answers to research questions and leads to the creation of new questions.

How Is Knowledge Development through Research Different?

Previously, sources of knowledge development were discussed, including authority, tradition, common sense, personal experience, and media portrayals. The problems generated by each knowledge source were also discussed. Research is another source of knowledge development, but it is different than those previously discussed in several ways. First, research relies on logical and systematic methods and observations to answer questions. Researchers use systematic, well-established research practices to seek answers to their questions. The methods and observations are completed in such a way that others can inspect and assess the methods and observations and offer feedback and criticism. Researchers develop, refine, and report their understanding of crime and the criminal justice system more systematically than the public does through casual observation. Those who conduct scientific research employ much more rigorous methods to gather the information/knowledge they are seeking.

Second, in order to prove that a research finding is correct, a researcher must be able to replicate the finding using the same methods. Only through replication can we have confidence in our original finding. For researchers, it may be important to replicate findings many times over so that we are assured our original finding was not a coincidence or chance occurrence. The Minneapolis Domestic Violence Experiment is an example of this and will be discussed in detail in Chapter 5. In the experiment, the researchers found that arrests for domestic violence lead to fewer repeat incidences in comparison to separation of the people involved and mediation. Five replication studies were conducted and none were able to replicate the findings in the Minneapolis study. In fact, three of the replications found that those arrested for domestic violence had higher levels of continued domestic violence, so arrest did not have the deterrent effect found in the Minneapolis study.

Third, research is objective. Objectivity indicates a neutral and nonbiased perspective when conducting research. Although there are examples to the contrary, the researcher should not have a vested interest in what findings are discovered from the research. The researcher is expected to remain objective and report the findings of the study regardless of whether the findings support their personal opinion or agenda. In addition, research ensures objectivity by allowing others to examine and be critical of the methodology, findings, and results of research studies.

It should be clear that using research methods to answer questions about crime and the criminal justice system will greatly reduce the errors in the development of knowledge previously discussed. For example, research methods reduces the likelihood of overgeneralization by using systematic procedures for selecting individuals or groups to study that are representative of the individuals or groups that we wish to generalize. This is the topic of Chapter 3, which covers sampling procedures. In addition, research methods reduces the risk of selective observation by requiring that we measure and observe our research subjects systematically.

Being an Informed Consumer of Research

Criminal justice and criminological research is important for several reasons. First, it can provide better and more objective information. Second, it can promote better decision-making. Today, more than ever, we live in a world driven by data and in which there is an increasing dependence on the assessment of data when making decisions. As well as possible, research ensures that our decisions are based on data and not on an arbitrary or personal basis. Third, it allows for the objective assessment of programs. Fourth, it has often been the source of innovation within criminal justice agencies. Fifth, it can be directly relevant to criminal justice practice and have a significant impact on criminal justice operations.

Before we apply research results to practices in the criminal justice system, and before we even accept those research results as reasonable, we need to be able to know whether or not they are worthwhile. In other words, should we believe the results of the study? Research has its own limitations, so we need to evaluate research results and the methods used to produce them, and we do so through critical evaluation. Critical evaluation involves identifying both positive and negative aspects of the research study�both the good and the bad. Critical evaluation involves comparing the methodology used in the research with the standards established in research methods.

Through critical evaluation, consumers of research break studies down into their essential elements. What are the research questions and hypotheses? What were the independent and dependent variables? What research design was used? Was probability sampling used? What data-gathering procedures were employed? What type of data analysis was conducted and what conclusions were made? These are some of the questions that are asked by informed consumers of research. The evaluation of research ranges from the manner in which one obtains an idea to the ways in which one writes about the research results, and understanding each step in the research process is useful in our attempts to consume research conducted by others. Located between these two activities are issues concerning ethics, sampling, research design, data analyses, and interpretations.

The research design and procedures are typically the most critically evaluated aspects of research and will likewise receive the greatest amount of attention in this text. Informed consumers of research don�t just take the results of a research study at face value because the study is in an academic journal or written by someone with a Ph.D. Instead, informed consumers critically evaluate research. Taking what is learned throughout this text, critical evaluation of research is covered in Chapter 8, and upon completing this text, it is hoped that you will be an informed consumer of research and will put your research knowledge to use throughout your career.

Although many students will never undertake their own research, all will be governed by policies based upon research and exposed to research findings in their chosen professional positions. Most government agencies, including the criminal justice system, as well as private industry, routinely rely on data analysis. Criminal justice students employed with these agencies will be challenged if not prepared for quantitative tasks. Unfortunately, it is not unusual to find students as well as professionals in criminal justice who are unable to fully understand research reports and journal articles in their own field.

Beyond our criminal justice careers, we are all exposed to and use research to help us understand issues and to make personal decisions. For example, we know that cigarette smoking causes lung cancer and has other significant health impacts, so we don�t smoke. Your doctor tells you that your cholesterol is too high and you need to limit your red meat intake because research shows that consumption of red meat raises cholesterol; so, you quit eating red meat. That is why not all the examples in this text are criminal justice research examples. Some come from the medical field while others come from psychology and other disciplines. This is to remind you that you are probably exposed to much more research than you thought on day one of this class.

Overall, knowledge of research methods will allow you to more appropriately consider and consume information that is important to your career in criminal justice. It will help you better understand the process of asking and answering a question systematically and be a better consumer of the kind of information that you really need to be the best criminal justice professional you can. Once familiar with research methods, your anxiety about reviewing technical reports and research findings can be minimized. As discussed in the next section, research methods involve a process and once you understand the process, you can apply your knowledge to any research study, even those in other disciplines.

The Research Process

One of the nice things about studying research methods is it is about learning a process. Research methods can be seen as a sequential process with the first step being followed by the second step, and so on. There are certainly times when the order of the steps may be modified, but researchers typically follow the same process for each research study they complete regardless of the research topic (as depicted in Figure 2.1 in Chapter 2). Very simply, a research problem or question is identified, and a methodology is selected, developed, and implemented to answer the research question. This sequential process is one of the advantages of understanding research methods, because once you understand the process, you can apply that process to any research question that interests you. In addition, research methods are the same across disciplines. So, sampling is the same in business as it is in health education and as it is in criminal justice. Certainly the use of a particular method will be more common in one discipline in comparison to another, but the protocol for implementing the method to complete the research study is the same. For example, field research (discussed in Chapter 6) is used much more frequently in anthropology than in criminal justice. However, the research protocol to implement field research is the same whether you are studying an indigenous Indian tribe in South America in anthropology or a group of heroin users in St. Louis in criminal justice.

Some authors have presented the research process as a wheel or circle, with no specific beginning or end. Typically, the research process begins with the selection of a research problem and the development of research questions or hypotheses (discussed further in Chapter 2). It is common for the results of previous research to generate new research questions and hypotheses for the researcher. This suggests that research is cyclical, a vibrant and continuous process. When a research study answers one question, the result is often the generation of additional questions, which plunges the researcher right back into the research process to complete additional research to answer these new questions.

In this section, a brief overview of the research process will be presented. The chapters that follow address various aspects of the research process, but it is critical that you keep in mind the overall research process as you read this book, which is why is it presented here. Although you will probably not be expected to conduct a research study on your own, it is important for an educated consumer of research to understand the steps in the research process. The steps are presented in chronological order and appear neatly ordered. In practice, the researcher can go back and forth between the steps in the research process.

Step 1: Select a Topic and Conduct a Literature Review

The first step in the research process is typically the identification of a problem or topic that the researcher is interested in studying. Research topics can arise from a wide variety of sources, including the findings of a current study, a question that a criminal justice agency needs to have answered, or the result of intellectual curiosity. Once the researcher has identified a particular problem or topic, the researcher assesses the current state of the literature related to the problem or topic. The researcher will often spend a considerable amount of time in determining what the existing literature has to say about the topic. Has the topic already been studied to the point that the questions in which the researcher is interested have been sufficiently answered? If so, can the researcher approach the subject from a previously unexamined perspective? Many times, research topics have been previously explored but not brought to completion. If this is the case, it is certainly reasonable to examine the topic again. It is even appropriate to replicate a previous study to determine whether the findings reported in the prior research continue to be true in different settings with different participants. This step in the research process is also discussed in Chapter 2.

Step 2: Develop a Research Question

After a topic has been identified and a comprehensive literature review has been completed on the topic, the next step is the development of a research question or questions. The research question marks the beginning of your research study and is critical to the remaining steps in the research process. The research question determines the research plan and methodology that will be employed in the study, the data that will be collected, and the data analysis that will be performed. Basically, the remaining steps in the process are completed in order to answer the research question or questions established in this step. The development of research questions is discussed in more detail in Chapter 2.

Step 3: Develop a Hypothesis

After the research questions have been established, the next step is the formulation of hypotheses, which are statements about the expected relationship between two variables. For example, a hypothesis may state that there is no relationship between heavy metal music preference and violent delinquency. The two variables stated in the hypothesis are music preference and violent delinquency. Hypothesis development is discussed in more detail in Chapter 2.

Step 4: Operationalize Concepts

Operationalization involves the process of giving the concepts in your study a working definition and determining how each concept in your study will be measured. For example, in Step 3, the variables were music preference and violent delinquency. The process of operationalization involves determining how music preference and violent delinquency will be measured. Operationalization is further discussed in Chapter 2.

Step 5: Develop the Research Plan and Methodology

The next step is to develop the methodology that will be employed to answer the research questions and test the hypotheses. The research methodology is the blueprint for the study, which outlines how the research is to be conducted. The research questions will determine the appropriate methodology for the study. The research design selected should be driven by the research questions asked. In other words, the research questions dictate the methods used to answer them. The methodology is basically a research plan on how the research questions will be answered and will detail:

1. What group, subjects, or population will be studied and selected? Sampling will be discussed in Chapter 3.

2 . What research design will be used to collect data to answer the research questions? Various research designs will be covered in Chapters 4�7.

You need to have familiarity with all research designs so that you can become an educated consumer of research. A survey cannot answer all research questions, so knowing a lot about surveys but not other research designs will not serve you well as you assess research studies. There are several common designs used in criminal justice and criminology research. Brief descriptions of several common research designs are presented below, but each is discussed in detail in later chapters.

Survey research is one of the most common research designs employed in criminal justice research. It obtains data directly from research participants by asking them questions and is often conducted through self-administered questionnaires and personal interviews. For example, a professor might have her students complete a survey during class to understand the relationship between drug use and self-esteem. Survey research is discussed in Chapter 4.

Experimental designs are used when researchers are interested in determining whether a program, policy, practice, or intervention is effective. For example, a researcher may use an experimental design to determine if boot camps are effective at reducing juvenile delinquency. Experimental design is discussed in Chapter 5.

Field research involves researchers studying individuals or groups of individuals in their natural environment. The researcher is observing closely or acting as part of the group under study and is able to describe in depth not only the subject�s behaviors, but also consider the motivations that drive those behaviors. For example, if a researcher wanted to learn more about gangs and their activities, he may �hang out� with a gang in order to observe their behavior. Field research is discussed in Chapter 6.

A case study is an in-depth analysis of one or a few illustrative cases. This design allows the story behind an individual, a particular offender, to be told and then information from cases studies can be extrapolated to a larger group. Often these studies require the review and analysis of documents such as police reports and court records and interviews with the offender and others. For example, a researcher may explore the life history of a serial killer to try and understand why the offender killed. Case studies are discussed in Chapter 6.

Secondary data analysis occurs when researchers obtain and reanalyze data that was originally collected for a different purpose. This can include reanalyzing data collected from a prior research study, using criminal justice agency records to answer a research question, or historical research. For example, a researcher using secondary data analysis may analyze inmate files from a nearby prison to understand the relationship between custody level assignment and disciplinary violations inside prison. Secondary data analysis is discussed in Chapter 7.

Content analysis requires the assessment of content contained in mass communication outlets such as newspapers, television, magazines, and the like. In this research design, documents, publications, or presentations are reviewed and analyzed. For example, a researcher utilizing content analysis might review true crime books involving murder to see how the characteristics of the offender and victim in the true crime books match reality as depicted in the FBI�s Supplemental Homicide Reports. Content analysis is discussed in Chapter 7.

Despite the options these designs offer, other research designs are available and will be discussed later in the text. Ultimately, the design used will depend on the nature of the study and the research questions asked.

Step 6: Execute the Research Plan and Collect Data

The next step in the research process is the collection of the data based on the research design developed. For example, if a survey is developed to study the relationship between gang membership and violent delinquency, the distribution and collection of surveys from a group of high school students would occur in this step. Data collection is discussed in several chapters throughout this text.

Step 7: Analyze Data

After the data have been collected, the next phase in the research process involves analyzing the data through various and appropriate statistical techniques. The most common means for data analysis today is through the use of a computer and statistically oriented software. Data analysis and statistics are discussed in Chapter 9.

Step 8: Report Findings, Results, and Limitations

Reporting and interpreting the results of the study make up the final step in the research process. The findings and results of the study can be communicated through reports, journals, books, or computer presentations. At this step, the results are reported and the research questions are answered. In addition, an assessment is made regarding the support or lack of support for the hypotheses tested. It is also at this stage that the researcher can pose additional research questions that may now need to be answered as a result of the research study. In addition, the limitations of the study, as well as the impact those limitations may have on the results of the study, will be described by the researcher. All research has limitations, so it is incumbent on the researcher to identify those limitations for the reader. The process of assessing the quality of research will be discussed in Chapter 8.

Research in Action: Impacting Criminal Justice Operations

Research in the criminal justice system has had significant impacts on its operations. The following sections provide an example of research that has significantly impacted each of the three main components of the criminal justice system: police, courts, and corrections. The purpose of this section is to demonstrate that research has aided the positive development and progression of the criminal justice system.

Police Research Example 76

The efforts of criminal justice researchers in policing have been important and have created the initial and critical foundation necessary for the further development of effective and productive law enforcement. One seminal study asked: How important is it for the police to respond quickly when a citizen calls? The importance of rapid response was conveyed in a 1973 National Commission on Productivity Report despite the fact that there was very little empirical evidence upon which to base this assumption. In fact, the Commission stated �there is no definitive relationship between response time and deterrence, but professional judgment and logic do suggest that the two are related in a strong enough manner to make more rapid response important.� 77 Basically the Commission members were stating that we don�t have any research evidence that response times are important, but we �know� that they are. Police departments allocated substantial resources to the patrol function and deployed officers in an effort to improve response time through the use of the 9-1-1 telephone number, computer-assisted dispatch, and beat assignment systems. Officers were typically assigned to a patrol beat. When the officers were not answering calls for service, they remained in their assigned beats so they could immediately respond to an emergency.

The data for the project were collected as part of a larger experiment on preventive patrol carried out in Kansas City, Missouri, between October 1972 and September 1973. 78 To determine the impact of response time, researchers speculated that the following variables would be influenced by response time: 1) the outcome of the response, 2) citizen satisfaction with response time, and 3) citizen satisfaction with the responding officer. Several data sources were used in the study. First, surveys were completed after all citizen-initiated calls (excluding automobile accidents) that involved contact with a police officer. The survey instrument consisted of questions to assess the length of time to respond to a call and the outcome of the call (i.e., arrest). Over 1,100 surveys were completed. Second, a follow-up survey was mailed to citizens whom the police had contacted during their response. These surveys asked questions to assess citizen satisfaction with response time and outcome. Over 425 of these surveys were returned.

The data collected during the study showed that response time did not determine whether or not the police made an arrest or recovered stolen property. This was the most surprising finding from the study because it challenged one of the basic underlying principles of police patrol. Researchers attributed the lack of significance to the fact that most citizens waited before calling the police. Rapid response simply did not matter in situations where citizens delayed in reporting the crime.

Rapid response time was not only believed to be important in determining the outcome of a response (i.e., more likely to lead to an arrest), it was also considered an important predictor of citizen satisfaction. Data from the study showed that when the police arrived sooner than expected, citizens were more satisfied with response time. However, subsequent research has shown that citizens are also satisfied with a delayed response as long as the dispatcher sets a reasonable expectation for when the patrol officer will arrive. Response time was also the best predictor of how satisfied a citizen was with the responding officer. It was further revealed that citizens became dissatisfied with the police when they were not informed of the outcome (i.e., someone was arrested). Again, these findings indicate the need for dispatchers and patrol officers to communicate with complainants regarding when they should expect an officer to arrive and the outcome of the call.

Based on the results of the response time study, the researchers concluded that rapid response was not as important as police administrators had thought. Response time was not related to an officer�s ability to make an arrest or recover stolen property. Results from the response time study challenged traditional beliefs about the allocation of patrol in our communities. Based on tradition knowledge, as previously discussed, rapidly responding to calls for service is what the police had always done since they started using patrol vehicles. In addition, common sense, as previously discussed, played a role in the practice of rapid response to calls for service; it just made sense that if a patrol officer arrives sooner, she will be more likely to make an arrest.

Prior to the research, police departments operated under the assumption that rapid response was a crucial factor in the ability of an officer to solve a crime and an important predictor of citizen satisfaction. In response to the research on rapid response, many police departments changed the way they responded to calls for service. Many departments adopted a differential police response approach. Differential police response protocols allow police departments to prioritize calls and rapidly dispatch an officer only when an immediate response is needed (i.e., crimes in progress). For crimes in progress, rapid response is critical and may reduce the injuries sustained by the victim as well, but these emergency calls usually account for less than 2% of all 9-1-1 calls for police service. For nonemergency calls, an officer is either dispatched at a later time when the officer is available or a report is taken over the phone or through some other means. Differential police response has been shown to save departments money and give patrol officers more time to engage in community-oriented and proactive policing activities. The benefits for a department are not at the expense of the public. In fact, a study by Robert Worden found a high degree of citizen satisfaction with differential police response. 79

Courts Research Example 80

Research on the courts component of the criminal justice system, while far from complete, has produced direct effects on the operations of the criminal justice system. The study reviewed in this section asked the following research question: Are jurors able to understand different legal rules for establishing a defendant�s criminal responsibility? The study described below explored the issue of criminal responsibility as it applies to the insanity defense in the United States. For several years, the M � Naghten rule was the legal rule applied in all courts of the United States. Under M � Naghten, criminal responsibility was absent when the offender did not understand the nature of his actions due to failure to distinguish �right� from �wrong.� This is known as the �right/wrong test� for criminal responsibility. The case of Durham v. United States was heard in the U.S. Court of Appeals for the District of Columbia and offered an alternative test for criminal responsibility and insanity. The legal rule emerging from Durham was that criminal responsibility was absent if the offense was a product of mental disease or defect. This ruling provided psychiatrists with a more important role at trial because of the requirement that the behavior be linked to a mental disorder that only a psychiatrist could officially determine.

At the time of Simon�s 1967 study, most courts across the country still followed the M � Naghten rule. Questions arose, however, regarding whether juries differed in their understanding of M � Naghten versus Durham and, in turn, whether this resulted in differences in their ability to make informed decisions regarding criminal responsibility in cases involving the insanity defense. The study was designed to determine the effect of different legal rules on jurors� decision-making in cases where the defense was insanity. There was a question of whether there was a difference between the rules to the extent that jurors understood each rule and could capably apply it.

Simon conducted an experimental study on jury deliberations in cases where the only defense was insanity. 81 Utilizing a mock jury approach, Simon took the transcripts of two actual trials with one reflecting the use of the M � Naghten rule and the other the Durham rule. Both cases were renamed and the transcripts were edited to constitute a trial of 60�90 minutes in length. These edited transcripts were then recorded, with University of Chicago Law School faculty as the attorneys, judges, and witnesses involved in each case. Groups of 12 jurors listened to each trial with instruction provided at the end regarding the particular rule of law ( M � Naghten or Durham) for determining criminal responsibility. Each juror submitted a written statement with his or her initial decision on the case before jury deliberations, and the juries� final decisions after deliberation were also reported.

Simon found significant differences in the verdicts across the two groups ( M � Naghten rule applied and Durham rule applied) even when the case was the same. For the M � Naghten version of the case, the psychiatrists stated that the defendant was mentally ill yet knew right from wrong during the crime. These statements/instructions should have led to a guilty verdict on the part of the mock jury. As expected, the M � Naghten juries delivered guilty verdicts in 19 of the 20 trials, with one hung jury. For the Durham version of the case, the psychiatrists stated that the crime resulted from the defendant�s mental illness, which should have lead to acquittal. However, the defendant was acquitted in only five of the 26 Durham trials. Twenty-six groups of 12 jurors were exposed to the Durham version of the trial and the case was the same each time. Simon interpreted these results as suggesting that jurors were unambiguous in their interpretations and applications of M � Naghten (due to the consistency in guilty verdicts), but they were less clear on the elements of Durham and how to apply it (reflected by the mix of guilty, not guilty, and hung verdicts). 82

After Simon�s study, most states rejected the Durham test. Recall her finding that the Durham rule produced inconsistent verdicts. She interpreted this finding as Durham being no better than providing no guidance to jurors on how to decide the issue of insanity. The observation helped to fuel arguments against the use of Durham, which, in turn, contributed to its demise as a legal rule. Today, only New Hampshire uses a version of the Durham rule in insanity cases.

WHAT RESEARCH SHOWS: IMPACTING CRIMINAL JUSTICE OPERATIONS

The Punishment Cost of Being Young, Black, and Male

Steffensmeier, Ulmer, and Kramer 83 hypothesized that African Americans overall were not likely to be treated more harshly than white defendants by the courts because it was only particular subgroups of minority defendants that fit with court actors� stereotypes of �more dangerous� offenders. In particular, they argued that younger African American males not only fulfilled this stereotype more than any other age, race, and gender combination, they were also more likely to be perceived by judges as being able to handle incarceration better than other subgroups.

In order to test their hypotheses, the researchers examined sentencing data from Pennsylvania spanning four years (1989�1992). Almost 139,000 cases were examined. The sentences they examined included whether a convicted defendant was incarcerated in prison or jail, and the length of incarceration in prison or jail. The researchers found that offense severity and prior record were the most important predictors of whether a convicted defendant was incarcerated and the length of incarceration. The authors found that the highest likelihood of incarceration and the longest sentences for males were distributed to African Americans aged 18�29 years. Their analysis of females revealed that white females were much less likely than African American females to be incarcerated, regardless of the age group examined. Taken altogether, the analysis revealed that African American males aged 18�29 years maintained the highest odds of incarceration and the longest sentences relative to any other race, sex, and age group.

Overall, this research showed that judges focused primarily on legal factors (offense severity and prior record) when determining the sentences of convicted offenders. These are the factors we expect judges to consider when making sentencing decisions. However, the research also found that judges base their decisions in part on extralegal factors, particularly the interaction of a defendant�s age, race, and gender. This research expanded our knowledge beyond the impact of singular factors on sentencing to expose the interaction effects of several variables (race, gender, and age). Court personnel are aware of these interaction effects based on this study, and others that followed, as well as their personal experiences in the criminal justice system. Identification and recognition of inequities in our justice system (in this case that young, African American males are punished more severely in our justice system) is the first step in mitigating this inequity.

Corrections Research Example 84

Although the research in corrections is far from complete, it has contributed greatly to the development of innovative programs and the professional development of correctional personnel. The contributions of academic and policy-oriented research can be seen across the whole range of correctional functions from pretrial services through probation, institutional corrections, and parole.

Rehabilitation remained the goal of our correctional system until the early 1970s, when the efficacy of rehabilitation was questioned. Violent crime was on the rise, and many politicians placed the blame on the criminal justice system. Some believed the system was too lenient on offenders. Interest in researching the effectiveness of correctional treatment remained low until 1974 when an article written by Robert Martinson and published in Public Interest titled �What Works? Questions and Answers about Prison Reform� generated enormous political and public attention to the effectiveness of correctional treatment. 85

Over a six-month period, Martinson and his colleagues reviewed all of the existing literature on correctional treatment published in English from 1945 to 1967. Each of the articles was evaluated according to traditional standards of social science research. Only studies that utilized an experimental design, included a sufficient sample size, and could be replicated were selected for review. A total of 231 studies examining a variety of different types of treatment were chosen, including educational and vocational training, individual and group counseling, therapeutic milieus, medical treatment, differences in length and type of incarceration, and community corrections. All of the treatment studies included at least one measure of offender recidivism, such as whether or not offenders were rearrested or violated their parole. The recidivism measures were used to examine the success or failure of a program in terms of reducing crime.

After reviewing all 231 studies, Martinson reported that there was no consistent evidence that correctional treatment reduced recidivism. Specifically, he wrote, �with few and isolated exceptions, the rehabilitative efforts that have been reported so far have had no appreciable effect on recidivism.� 86 Martinson further indicated that the lack of empirical support for correctional treatment could be a consequence of poorly implemented programs. If the quality of the programs were improved, the results may have proved more favorable, but this conclusion was for the most part ignored by the media and policy-makers.

Martinson�s report became commonly referred to as �nothing works� and was subsequently used as the definitive study detailing the failures of rehabilitation. The article had implications beyond questioning whether or not specific types of correctional treatment reduced recidivism. The entire philosophy of rehabilitation was now in doubt because of Martinson�s conclusion that �our present strategies … cannot overcome, or even appreciably reduce, the powerful tendencies of offenders to continue in criminal behavior.� 87

Martinson�s article provided policy makers the evidence to justify spending cuts on rehabilitative programs. Furthermore, it allowed politicians to respond to growing concerns about crime with punitive, get-tough strategies. States began implementing strict mandatory sentences that resulted in more criminals being sent to prison and for longer periods of time. Over the next several years, Martinson�s article was used over and over to support abandoning efforts to treat offenders until rehabilitation became virtually nonexistent in our correctional system.

Chapter Summary

This chapter began with a discussion of sources of knowledge development and the problems with each. To depict the importance of research methods in knowledge development, myths about crime and the criminal justice system were reviewed along with research studies that have dispelled myths. As the introductory chapter in this text, this chapter also provided an overview of the steps in the research process from selecting a topic and conducting a literature review at the beginning of a research study to reporting findings, results, and limitations at the end of the study. Examples of actual research studies in the areas of police, courts, and corrections were also provided in this chapter to demonstrate the research process in action and to illustrate how research has significantly impacted practices within the criminal justice system. In addition, this chapter demonstrated the critical importance of becoming an informed consumer of research in both your personal and professional lives.

Critical Thinking Questions

1. What are the primary sources of knowledge development, and what are the problems with each?

2. How is knowledge developed through research methods different from other sources of knowledge?

3. What myths about crime and criminal justice have been dispelled through research? Give an example of a research study that dispelled a myth.

4. Why is it important to be an informed consumer of research?

5. What are the steps in the research process, and what activities occur at each step?

authority knowledge: Knowledge developed when we accept something as being correct and true just because someone in a position of authority says it is true

case study: An in-depth analysis of one or a few illustrative cases

common sense knowledge: Knowledge developed when the information �just makes sense�

content analysis: A method requiring the analyzing of content contained in mass communication outlets such as newspapers, television, magazines, and the like

CSI Effect: Due to the unrealistic portrayal of the role of forensic science in solving criminal cases in television shows, jurors are more likely to vote to acquit a defendant when the expected sophisticated forensic evidence is not presented

differential police response: Methods that allow police departments to prioritize calls and rapidly dispatch an officer only when an immediate response is needed (i.e., crimes in progress)

experimental designs: Used when researchers are interested in determining whether a program, policy, practice, or intervention is effective

field research: Research that involves researchers studying individuals or groups of individuals in their natural environment

Halloween sadism: The practice of giving contaminated treats to children during trick or treating

hypotheses: Statements about the expected relationship between two concepts

illogical reasoning: Occurs when someone jumps to premature conclusions or presents an argument that is based on invalid assumptions

myths: Beliefs that are based on emotion rather than rigorous analysis

operationalization: The process of giving a concept a working definition; determining how each concept in your study will be measured

overgeneralization: Occurs when people conclude that what they have observed in one or a few cases is true for all cases

personal experience knowledge: Knowledge developed through actual experiences

research: The scientific investigation of an issue, problem, or subject utilizing research methods

research methods: The tools that allow criminology and criminal justice researchers to systematically study crime and the criminal justice system and include the basic rules, appropriate techniques, and relevant procedures for conducting research

resistance to change: The reluctance to change our beliefs in light of new, accurate, and valid information to the contrary

secondary data analysis: Occurs when researchers obtain and reanalyze data that were originally collected for a different purpose

selective observation: Choosing, either consciously or unconsciously, to pay attention to and remember events that support our personal preferences and beliefs

survey research: Obtaining data directly from research participants by asking them questions, often conducted through self-administered questionnaires and personal interviews

tradition knowledge: Knowledge developed when we accept something as true because that is the way things have always been, so it must be right

variables: Concepts that have been given a working definition and can take on different values

1 Briggs, Lisa T., Stephen E. Brown, Robert B. Gardner, and Robert L. Davidson. (2009). �D.RA.MA: An extended conceptualization of student anxiety in criminal justice research methods courses.� Journal of Criminal Justice Education 20 (3), 217�226.

2 Betz, N. E. (1978). �Prevalence, distribution, and correlates of math anxiety in college students. Journal of Counseling Psychology 25 (5), 441�448.

3 Briggs, et al., 2009, p. 221.

4 Ibid, p. 221.

5 Ibid, p. 221.

6 Kappeler, Victor E., and Gary W. Potter. (2005). The mythology of crime and criminal justice. Prospect Heights, IL: Waveland.

7 Tennessee v. Gamer, 471 U.S. 1 (1985).

8 Lombroso-Ferrero, Gina. (1911). Criminal man, according to the classification of Cesare Lombroso. New York: Putnam.

9 This study was included in Amy B. Thistlethwaite and John D. Wooldredge. (2010). Forty studies that changed criminal justice: Explorations into the history of criminal justice research. Upper Saddle River, NJ: Prentice Hall.

10 Petersilia, J., S. Turner, J. Kahan, and J. Peterson. (1985). Granting felons probation: Public risks and alternatives. Santa Monica, CA: Rand.

11 Vito, G. (1986). �Felony probation and recidivism: Replication and response.� Federal Probation 50, 17�25.

12 Conrad, J. (1985). �Research and development in corrections.� Federal Probation 49, 69�71.

13 Finckenauer, James O. (1982). Scared straight! and the panacea phenomenon. Englewood Cliffs, NJ: Prentice Hall.

14 Yarborough, J.C. (1979). Evaluation of JOLT (Juvenile Offenders Learn Truth) as a deterrence program. Lansing, MI: Michigan Department of Corrections.

15 Petrosino, Anthony, Carolyn Turpin-Petrosino, and James O. Finckenauer. (2000). �Well-meaning programs can have harmful effects! Lessons from experiments of programs such as Scared Straight,� Crime & Delinquency 46, 354�379.

16 Robertson, Jordan. �I�m being punished for living right�: Background check system is haunted by errors. December 20, 2011. http://finance.yahoo.com/news /ap-impact-criminal-past-isnt-182335059.html. Retrieved on December 29, 2011.

17 Shelton, D. E. (2008). �The �CSI Effect�: Does it really exist?� NIJ Journal 259 [NCJ 221501].

18 Best, Joel. (2011). �Halloween sadism: The evidence.� http://dspace.udel.edu:8080/dspace/bitstream/handle/ 19716/726/Halloween%20sadism.revised%20thru%20201l.pdf?sequence=6. Retrieved on May 7, 2012.

19 Best, Joel. (1985, November). �The myth of the Halloween sadist. Psychology Today 19 (11), p. 14.

21 �Beer compound shows potent promise in prostate cancer battle.� Press release from Oregon State University May 30, 2006. http://oregonstate.edu/ua/ncs/archives/2006/ may/beer-compound-shows-potent-promise-prostate-cancer-battle. Retrieved on January 6, 2012; Colgate, Emily C., Cristobal L. Miranda, Jan F. Stevens, Tammy M. Bray, and Emily Ho. (2007). �Xanthohumol, a prenylflavonoid derived from hops induces apoptosis and inhibits NF-kappaB activation in prostate epithelial cells,� Cancer Letters 246, 201�209; �Health benefits of red wine exaggerated� http://health.yahoo.net/articles /nutrition/health-benefits-red-wine-exaggerated. Retrieved on January 14, 2012; �Scientific journals notified following research misconduct investigation.� January 11, 2012. http://today.uconn.edu/blog/2012/01/scientific-journals -notified-following-research-misconduct-investigation/. Retrieved on January 14, 2012.

22 Pepinsky, Hal. �The myth that crime and criminality can be measured.� 3�11 in Bohm, Robert M., and Jeffrey T. Walker. (2006). Demystifying crime and criminal justice. Los Angeles: Roxbury.

23 Bullock, Jennifer L., and Bruce A. Arrigo. �The myth that mental illness causes crime.� 12�19 in Bohm, Robert M., and Jeffrey T. Walker. (2006). Demystifying crime and criminal justice. Los Angeles: Roxbury.

24 Friedrichs, David O. �The myth that white-collar crime is only about financial loss.� 20�28 in Bohm, Robert M., and Jeffrey T. Walker. (2006). Demystifying crime and criminal justice. Los Angeles: Roxbury.

25 Kuhns III, Joseph B., and Charisse T. M. Coston. �The myth that serial murderers are disproportionately white males.� 37�44 in Bohm, Robert M., and Jeffrey T. Walker. (2006). Demystifying crime and criminal justice. Los Angeles: Roxbury.

26 Longmire, Dennis R., Jacqueline Buffington-Vollum, and Scott Vollum. �The myth of positive differentiation in the classification of dangerous offenders.� 123�131 in Bohm, Robert M., and Jeffrey T. Walker. (2006). Demystifying crime and criminal justice. Los Angeles: Roxbury.

27 Masters, Ruth E., Lori Beth Way, Phyllis B. Gerstenfeld, Bernadette T. Muscat, Michael Hooper, John P. J. Dussich, Lester Pincu, and Candice A. Skrapec. (2013). CJ realities and challenges, 2nd ed. New York: McGraw-Hill.

32 Brownstein, Henry H. �The myth of drug users as violent offenders.� 45�53 in Bohm, Robert M., and Jeffrey T. Walker. (2006). Demystifying crime and criminal justice. Los Angeles: Roxbury.

33 Goldstein, P. (1985). �The drugs/violence nexus: A tripartite conceptual framework.� Journal of Drug Issues 15, 493�506.

34 Goldstein, P, H. Brownstein, and P. Ryan. (1992). �Drug-related homicide in New York City: 1984 and 1988.� Crime & Delinquency 38, 459�476.

35 Parker, R., and K. Auerhahn. (1998). �Alcohol, drugs, and violence.� Annual Review of Sociology 24, 291�311, p. 291.

36 Buerger, Michael. �The myth of racial profiling.� 97�103 in Bohm, Robert M., and Jeffrey T. Walker. (2006). Demystifying crime and criminal justice. Los Angeles: Roxbury.

37 Cordner, Gary, and Kathryn E. Scarborough. �The myth that science solves crimes.� 104�110 in Bohm, Robert M., and Jeffrey T. Walker. (2006). Demystifying crime and criminal justice. Los Angeles: Roxbury.

38 Willis, James J., Stephen D. Mastrofski, and David Weisburd. �The myth that COMPSTAT reduces crime and transforms police organizations.� 111�119 in Bohm, Robert M., and Jeffrey T. Walker. (2006). Demystifying crime and criminal justice. Los Angeles: Roxbury.

39 Masters, et al., 2013.

43 Scott, Eric J. (1981). Calls for service: Citizen demand and initial police response. Washington, DC: Government Printing Office.

44 Lersch, Kim. �The myth of policewomen on patrol.� 89�96 in Bohm, Robert M., and Jeffrey T. Walker. (2006). Demystifying crime and criminal justice. Los Angeles: Roxbury.

45 Janikowski, Richard. �The myth that the exclusionary rule allows many criminals to escape justice.� 132�139 in Bohm, Robert M., and Jeffrey T. Walker. (2006). Demystifying crime and criminal justice. Los Angeles: Roxbury.

46 Bishop, Donna M. �The myth that harsh punishments reduce juvenile crime.� 140�148 in Bohm, Robert M., and Jeffrey T. Walker. (2006). Demystifying crime and criminal justice. Los Angeles: Roxbury.

47 Immarigeon, Russ. �The myth that public attitudes are punitive.� 149�157 in Bohm, Robert M., and Jeffrey T. Walker. (2006). Demystifying crime and criminal justice. Los Angeles: Roxbury.

48 Acker, James R. �The myth of closure and capital punishment.� 167�175 in Bohm, Robert M., and Jeffrey T. Walker. (2006). Demystifying crime and criminal justice. Los Angeles: Roxbury.

49 Masters, et al., 2013.

52 Lersch, 2006.

53 Newport, Frank. �In U.S., support for death penalty falls to 39-year low.� October 13, 2011. http://www.gallup .com/poll/150089/support-death-penalty-falls-year-low.aspx. Retrieved on April 16, 2012.

54 Applegate, Brandon. �The myth that the death penalty is administered fairly.� 158�166 in Bohm, Robert M., and Jeffrey T. Walker. (2006). Demystifying crime and criminal justice. Los Angeles: Roxbury.

55 Williams, M. R., and J. E. Holcomb. (2001). �Racial disparity and death sentences in Ohio.� Journal of Criminal Justice 29, 207�218.

56 Snell, Tracy L. (2011, December). Capital punishment, 2010�statistical tables. Washington, DC: Bureau of Justice Statistics.

57 Applegate, 2006.

58 Williams and Holcomb, 2001.

59 Applegate, 2006.

61 Wood, Peter B. �The myth that imprisonment is the most severe form of punishment.� 192�200 in Bohm, Robert M., and Jeffrey T. Walker. (2006). Demystifying crime and criminal justice. Los Angeles: Roxbury.

63 Michalowski, Raymond. �The myth that punishment reduces crime.� 179�191 in Bohm, Robert M., and Jeffrey T. Walker. (2006). Demystifying crime and criminal justice. Los Angeles: Roxbury.

64 McShane, Marilyn, Frank P. Williams III, and Beth Pelz. �The myth of prisons as country clubs.� 201�208 in Bohm, Robert M., and Jeffrey T. Walker. (2006). Demystifying crime and criminal justice. Los Angeles: Roxbury.

65 Parker, Mary. �The myth that prisons can be self-supporting.� 209�213 in Bohm, Robert M., and Jeffrey T. Walker. (2006). Demystifying crime and criminal justice. Los Angeles: Roxbury.

66 Blakely, Curtis, and John Ortiz Smykla. �Correctional privatization and the myth of inherent efficiency.� 214�220 in Bohm, Robert M., and Jeffrey T. Walker. (2006). Demystifying crime and criminal justice. Los Angeles: Roxbury.

67 Jones, G. Mark. �The myth that the focus of community corrections is rehabilitation.� 221�226 in Bohm, Robert M., and Jeffrey T. Walker. (2006). Demystifying crime and criminal justice. Los Angeles: Roxbury.

68 Cullen, Francis T., and Paula Smith. �The myth that correctional rehabilitation does not work.� 227�238 in Bohm, Robert M., and Jeffrey T. Walker. (2006). Demystifying crime and criminal justice. Los Angeles: Roxbury.

69 Masters, et al., 2013.

73 Petersilia, Joan. (1990). �When probation becomes more dreaded than prison. Federal Probation 54, 23�27.

75 Wood, P. B., and H. G. Grasmick. (1999). �Toward the development of punishment equivalencies: Male and female inmates rate the severity of alternative sanctions compared to prison.� Justice Quarterly 16, 19�50.

76 Example is excerpted from Amy B. Thistlethwaite and John D. Wooldredge. (2010). Forty studies that changed criminal justice: Explorations into the history of criminal justice research. Upper Saddle River, NJ: Prentice Hall. This is an excellent book that demonstrates the impact research has had on criminal justice operations.

77 National Commission on Productivity. (1973). Opportunities for improving productivity in police services. Washington, DC: United States Government Printing Office, p. 19.

78 Pate, T., A. Ferrara, R. Bowers, and J. Lorence. (1976). Police response time: Its determinants and effects. Washington, DC: Police Foundation.

79 Worden, R. (1993). �Toward equity and efficiency in law enforcement: Differential police response. American Journal of Police 12, 1�32.

80 Example is excerpted from Amy B. Thistlethwaite and John D. Wooldredge. (2010). Forty studies that changed criminal justice: Explorations into the history of criminal justice research. Upper Saddle River, NJ: Prentice Hall.

81 Simon, R. (1967). The jury and the defense of insanity. Boston: Little, Brown.

83 Steffensmeier, D., J. Ulmer, & J. Kramer. (1998). �The interaction of race, gender, and age in criminal sentencing: The punishment cost of being young, black, and male. Criminology 36, 763�797.

84 Example is excerpted from Amy B. Thistlethwaite and John D. Wooldredge. (2010). Forty studies that changed criminal justice: Explorations into the history of criminal justice research. Upper Saddle River, NJ: Prentice Hall.

85 Martinson, R. (1974). �What works? Questions and answers about prison reform.� The Public Interest 10, 22�54.

86 Ibid, p. 25.

87 Ibid, p. 49.

Applied Research Methods in Criminal Justice and Criminology by University of North Texas is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License , except where otherwise noted.

Share This Book

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings
• My Bibliography
• Collections
• Citation manager

Save citation to file

Email citation, add to collections.

• Create a new collection
• Add to an existing collection

Add to My Bibliography

Your saved search, create a file for external citation management software, your rss feed.

• Search in PubMed
• Search in NLM Catalog
• Add to Search

A tutorial on methodological studies: the what, when, how and why

Affiliations.

• 1 Department of Health Research Methods, Evidence and Impact, McMaster University, Hamilton, ON, Canada. [email protected].
• 2 Biostatistics Unit/FSORC, 50 Charlton Avenue East, St Joseph's Healthcare-Hamilton, 3rd Floor Martha Wing, Room H321, Hamilton, Ontario, L8N 4A6, Canada. [email protected].
• 3 Centre for the Development of Best Practices in Health, Yaoundé, Cameroon. [email protected].
• 4 Department of Health Research Methods, Evidence and Impact, McMaster University, Hamilton, ON, Canada.
• 5 Center for Evidence-Based Medicine and Health Care, Catholic University of Croatia, Ilica 242, 10000, Zagreb, Croatia.
• 6 Department of Epidemiology and Biostatistics, School of Public Health - Bloomington, Indiana University, Bloomington, IN, 47405, USA.
• 7 Biostatistics Unit/FSORC, 50 Charlton Avenue East, St Joseph's Healthcare-Hamilton, 3rd Floor Martha Wing, Room H321, Hamilton, Ontario, L8N 4A6, Canada.
• 8 Departments of Paediatrics and Anaesthesia, McMaster University, Hamilton, ON, Canada.
• 9 Centre for Evaluation of Medicine, St. Joseph's Healthcare-Hamilton, Hamilton, ON, Canada.
• 10 Population Health Research Institute, Hamilton Health Sciences, Hamilton, ON, Canada.
• PMID: 32894052
• PMCID: PMC7487909
• DOI: 10.1186/s12874-020-01107-7

Background: Methodological studies - studies that evaluate the design, analysis or reporting of other research-related reports - play an important role in health research. They help to highlight issues in the conduct of research with the aim of improving health research methodology, and ultimately reducing research waste.

Main body: We provide an overview of some of the key aspects of methodological studies such as what they are, and when, how and why they are done. We adopt a "frequently asked questions" format to facilitate reading this paper and provide multiple examples to help guide researchers interested in conducting methodological studies. Some of the topics addressed include: is it necessary to publish a study protocol? How to select relevant research reports and databases for a methodological study? What approaches to data extraction and statistical analysis should be considered when conducting a methodological study? What are potential threats to validity and is there a way to appraise the quality of methodological studies?

Conclusion: Appropriate reflection and application of basic principles of epidemiology and biostatistics are required in the design and analysis of methodological studies. This paper provides an introduction for further discussion about the conduct of methodological studies.

Keywords: Meta-epidemiology; Methodological study; Research methods; Research-on-research.

PubMed Disclaimer

Conflict of interest statement

DOL, DBA, LM, LP and LT are involved in the development of a reporting guideline for methodological studies.

Trends in the number studies…

Trends in the number studies that mention “methodological review” or “meta- epidemiological study”…

A proposed framework for methodological…

A proposed framework for methodological studies

• Chalmers I, Glasziou P. Avoidable waste in the production and reporting of research evidence. Lancet. 2009;374(9683):86–89. - PubMed
• Chan AW, Song F, Vickers A, Jefferson T, Dickersin K, Gotzsche PC, Krumholz HM, Ghersi D, van der Worp HB. Increasing value and reducing waste: addressing inaccessible research. Lancet. 2014;383(9913):257–266. - PMC - PubMed
• Ioannidis JP, Greenland S, Hlatky MA, Khoury MJ, Macleod MR, Moher D, Schulz KF, Tibshirani R. Increasing value and reducing waste in research design, conduct, and analysis. Lancet. 2014;383(9912):166–175. - PMC - PubMed
• Higgins JP, Altman DG, Gotzsche PC, Juni P, Moher D, Oxman AD, Savovic J, Schulz KF, Weeks L, Sterne JA. The Cochrane Collaboration's tool for assessing risk of bias in randomised trials. BMJ. 2011;343:d5928. - PMC - PubMed
• Moher D, Schulz KF, Altman DG. The CONSORT statement: revised recommendations for improving the quality of reports of parallel-group randomised trials. Lancet. 2001;357. - PubMed

Publication types

• Search in MeSH

Related information

Linkout - more resources, full text sources.

• BioMed Central
• Europe PubMed Central
• PubMed Central

• Citation Manager

NCBI Literature Resources

MeSH PMC Bookshelf Disclaimer

The PubMed wordmark and PubMed logo are registered trademarks of the U.S. Department of Health and Human Services (HHS). Unauthorized use of these marks is strictly prohibited.

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, generate accurate citations for free.

• Knowledge Base

Methodology

Research Methods | Definitions, Types, Examples

Research methods are specific procedures for collecting and analyzing data. Developing your research methods is an integral part of your research design . When planning your methods, there are two key decisions you will make.

First, decide how you will collect data . Your methods depend on what type of data you need to answer your research question :

• Qualitative vs. quantitative : Will your data take the form of words or numbers?
• Primary vs. secondary : Will you collect original data yourself, or will you use data that has already been collected by someone else?
• Descriptive vs. experimental : Will you take measurements of something as it is, or will you perform an experiment?

Second, decide how you will analyze the data .

• For quantitative data, you can use statistical analysis methods to test relationships between variables.
• For qualitative data, you can use methods such as thematic analysis to interpret patterns and meanings in the data.

Table of contents

Methods for collecting data, examples of data collection methods, methods for analyzing data, examples of data analysis methods, other interesting articles, frequently asked questions about research methods.

Data is the information that you collect for the purposes of answering your research question . The type of data you need depends on the aims of your research.

Qualitative vs. quantitative data

Your choice of qualitative or quantitative data collection depends on the type of knowledge you want to develop.

For questions about ideas, experiences and meanings, or to study something that can’t be described numerically, collect qualitative data .

If you want to develop a more mechanistic understanding of a topic, or your research involves hypothesis testing , collect quantitative data .

You can also take a mixed methods approach , where you use both qualitative and quantitative research methods.

Primary vs. secondary research

Primary research is any original data that you collect yourself for the purposes of answering your research question (e.g. through surveys , observations and experiments ). Secondary research is data that has already been collected by other researchers (e.g. in a government census or previous scientific studies).

If you are exploring a novel research question, you’ll probably need to collect primary data . But if you want to synthesize existing knowledge, analyze historical trends, or identify patterns on a large scale, secondary data might be a better choice.

Descriptive vs. experimental data

In descriptive research , you collect data about your study subject without intervening. The validity of your research will depend on your sampling method .

In experimental research , you systematically intervene in a process and measure the outcome. The validity of your research will depend on your experimental design .

To conduct an experiment, you need to be able to vary your independent variable , precisely measure your dependent variable, and control for confounding variables . If it’s practically and ethically possible, this method is the best choice for answering questions about cause and effect.

Prevent plagiarism. Run a free check.

Your data analysis methods will depend on the type of data you collect and how you prepare it for analysis.

Data can often be analyzed both quantitatively and qualitatively. For example, survey responses could be analyzed qualitatively by studying the meanings of responses or quantitatively by studying the frequencies of responses.

Qualitative analysis methods

Qualitative analysis is used to understand words, ideas, and experiences. You can use it to interpret data that was collected:

• From open-ended surveys and interviews , literature reviews , case studies , ethnographies , and other sources that use text rather than numbers.
• Using non-probability sampling methods .

Qualitative analysis tends to be quite flexible and relies on the researcher’s judgement, so you have to reflect carefully on your choices and assumptions and be careful to avoid research bias .

Quantitative analysis methods

Quantitative analysis uses numbers and statistics to understand frequencies, averages and correlations (in descriptive studies) or cause-and-effect relationships (in experiments).

You can use quantitative analysis to interpret data that was collected either:

• During an experiment .
• Using probability sampling methods .

Because the data is collected and analyzed in a statistically valid way, the results of quantitative analysis can be easily standardized and shared among researchers.

Here's why students love Scribbr's proofreading services

Discover proofreading & editing

If you want to know more about statistics , methodology , or research bias , make sure to check out some of our other articles with explanations and examples.

• Chi square test of independence
• Statistical power
• Descriptive statistics
• Degrees of freedom
• Pearson correlation
• Null hypothesis
• Double-blind study
• Case-control study
• Research ethics
• Data collection
• Hypothesis testing
• Structured interviews

Research bias

• Hawthorne effect
• Unconscious bias
• Recall bias
• Halo effect
• Self-serving bias
• Information bias

Quantitative research deals with numbers and statistics, while qualitative research deals with words and meanings.

Quantitative methods allow you to systematically measure variables and test hypotheses . Qualitative methods allow you to explore concepts and experiences in more detail.

In mixed methods research , you use both qualitative and quantitative data collection and analysis methods to answer your research question .

A sample is a subset of individuals from a larger population . Sampling means selecting the group that you will actually collect data from in your research. For example, if you are researching the opinions of students in your university, you could survey a sample of 100 students.

In statistics, sampling allows you to test a hypothesis about the characteristics of a population.

The research methods you use depend on the type of data you need to answer your research question .

• If you want to measure something or test a hypothesis , use quantitative methods . If you want to explore ideas, thoughts and meanings, use qualitative methods .
• If you want to analyze a large amount of readily-available data, use secondary data. If you want data specific to your purposes with control over how it is generated, collect primary data.
• If you want to establish cause-and-effect relationships between variables , use experimental methods. If you want to understand the characteristics of a research subject, use descriptive methods.

Methodology refers to the overarching strategy and rationale of your research project . It involves studying the methods used in your field and the theories or principles behind them, in order to develop an approach that matches your objectives.

Methods are the specific tools and procedures you use to collect and analyze data (for example, experiments, surveys , and statistical tests ).

In shorter scientific papers, where the aim is to report the findings of a specific study, you might simply describe what you did in a methods section .

In a longer or more complex research project, such as a thesis or dissertation , you will probably include a methodology section , where you explain your approach to answering the research questions and cite relevant sources to support your choice of methods.

Is this article helpful?

Other students also liked, writing strong research questions | criteria & examples.

• What Is a Research Design | Types, Guide & Examples
• Data Collection | Definition, Methods & Examples

More interesting articles

• Between-Subjects Design | Examples, Pros, & Cons
• Cluster Sampling | A Simple Step-by-Step Guide with Examples
• Confounding Variables | Definition, Examples & Controls
• Construct Validity | Definition, Types, & Examples
• Content Analysis | Guide, Methods & Examples
• Control Groups and Treatment Groups | Uses & Examples
• Control Variables | What Are They & Why Do They Matter?
• Correlation vs. Causation | Difference, Designs & Examples
• Correlational Research | When & How to Use
• Critical Discourse Analysis | Definition, Guide & Examples
• Cross-Sectional Study | Definition, Uses & Examples
• Descriptive Research | Definition, Types, Methods & Examples
• Ethical Considerations in Research | Types & Examples
• Explanatory and Response Variables | Definitions & Examples
• Explanatory Research | Definition, Guide, & Examples
• Exploratory Research | Definition, Guide, & Examples
• External Validity | Definition, Types, Threats & Examples
• Extraneous Variables | Examples, Types & Controls
• Guide to Experimental Design | Overview, Steps, & Examples
• How Do You Incorporate an Interview into a Dissertation? | Tips
• How to Do Thematic Analysis | Step-by-Step Guide & Examples
• How to Write a Literature Review | Guide, Examples, & Templates
• How to Write a Strong Hypothesis | Steps & Examples
• Inclusion and Exclusion Criteria | Examples & Definition
• Independent vs. Dependent Variables | Definition & Examples
• Inductive Reasoning | Types, Examples, Explanation
• Inductive vs. Deductive Research Approach | Steps & Examples
• Internal Validity in Research | Definition, Threats, & Examples
• Internal vs. External Validity | Understanding Differences & Threats
• Longitudinal Study | Definition, Approaches & Examples
• Mediator vs. Moderator Variables | Differences & Examples
• Mixed Methods Research | Definition, Guide & Examples
• Multistage Sampling | Introductory Guide & Examples
• Naturalistic Observation | Definition, Guide & Examples
• Operationalization | A Guide with Examples, Pros & Cons
• Population vs. Sample | Definitions, Differences & Examples
• Primary Research | Definition, Types, & Examples
• Qualitative vs. Quantitative Research | Differences, Examples & Methods
• Quasi-Experimental Design | Definition, Types & Examples
• Questionnaire Design | Methods, Question Types & Examples
• Random Assignment in Experiments | Introduction & Examples
• Random vs. Systematic Error | Definition & Examples
• Reliability vs. Validity in Research | Difference, Types and Examples
• Reproducibility vs Replicability | Difference & Examples
• Reproducibility vs. Replicability | Difference & Examples
• Sampling Methods | Types, Techniques & Examples
• Semi-Structured Interview | Definition, Guide & Examples
• Simple Random Sampling | Definition, Steps & Examples
• Single, Double, & Triple Blind Study | Definition & Examples
• Stratified Sampling | Definition, Guide & Examples
• Structured Interview | Definition, Guide & Examples
• Survey Research | Definition, Examples & Methods
• Systematic Review | Definition, Example, & Guide
• Systematic Sampling | A Step-by-Step Guide with Examples
• Textual Analysis | Guide, 3 Approaches & Examples
• The 4 Types of Reliability in Research | Definitions & Examples
• The 4 Types of Validity in Research | Definitions & Examples
• Transcribing an Interview | 5 Steps & Transcription Software
• Triangulation in Research | Guide, Types, Examples
• Types of Interviews in Research | Guide & Examples
• Types of Research Designs Compared | Guide & Examples
• Types of Variables in Research & Statistics | Examples
• Unstructured Interview | Definition, Guide & Examples
• What Is a Case Study? | Definition, Examples & Methods
• What Is a Case-Control Study? | Definition & Examples
• What Is a Cohort Study? | Definition & Examples
• What Is a Conceptual Framework? | Tips & Examples
• What Is a Controlled Experiment? | Definitions & Examples
• What Is a Double-Barreled Question?
• What Is a Focus Group? | Step-by-Step Guide & Examples
• What Is a Likert Scale? | Guide & Examples
• What Is a Prospective Cohort Study? | Definition & Examples
• What Is a Retrospective Cohort Study? | Definition & Examples
• What Is Action Research? | Definition & Examples
• What Is an Observational Study? | Guide & Examples
• What Is Concurrent Validity? | Definition & Examples
• What Is Content Validity? | Definition & Examples
• What Is Convenience Sampling? | Definition & Examples
• What Is Convergent Validity? | Definition & Examples
• What Is Criterion Validity? | Definition & Examples
• What Is Data Cleansing? | Definition, Guide & Examples
• What Is Deductive Reasoning? | Explanation & Examples
• What Is Discriminant Validity? | Definition & Example
• What Is Ecological Validity? | Definition & Examples
• What Is Ethnography? | Definition, Guide & Examples
• What Is Face Validity? | Guide, Definition & Examples
• What Is Non-Probability Sampling? | Types & Examples
• What Is Participant Observation? | Definition & Examples
• What Is Peer Review? | Types & Examples
• What Is Predictive Validity? | Examples & Definition
• What Is Probability Sampling? | Types & Examples
• What Is Purposive Sampling? | Definition & Examples
• What Is Qualitative Observation? | Definition & Examples
• What Is Qualitative Research? | Methods & Examples
• What Is Quantitative Observation? | Definition & Examples
• What Is Quantitative Research? | Definition, Uses & Methods

Get unlimited documents corrected

✔ Free APA citation check included ✔ Unlimited document corrections ✔ Specialized in correcting academic texts

Understanding research methods: An overview of the essentials, tenth edition

• ISBN: 9781315213033
• This person is not on ResearchGate, or hasn't claimed this research yet.

• Mt. San Antonio College

Discover the world's research

• 25+ million members
• 160+ million publication pages
• 2.3+ billion citations

• Akhtar Ebrahimi Ghassemi

• Sara M. Awaad
• Leonard Jackson
• Towan Dicks
• Jeremiah Mugambi Ananga
• Dr. Tobias Mwalili
• Samson Nyang’au Paul
• Melissa Xiaohui Qin

• Ketebo Abdiyo Ensene
• Tobia Mwalili
• QUAL HEALTH RES

• Recruit researchers
• Join for free
• Login Email Tip: Most researchers use their institutional email address as their ResearchGate login Password Forgot password? Keep me logged in Log in or Continue with Google Welcome back! Please log in. Email · Hint Tip: Most researchers use their institutional email address as their ResearchGate login Password Forgot password? Keep me logged in Log in or Continue with Google No account? Sign up

Reference management. Clean and simple.

What is research methodology?

The basics of research methodology

Why do you need a research methodology, what needs to be included, why do you need to document your research method, what are the different types of research instruments, qualitative / quantitative / mixed research methodologies, how do you choose the best research methodology for you, frequently asked questions about research methodology, related articles.

When you’re working on your first piece of academic research, there are many different things to focus on, and it can be overwhelming to stay on top of everything. This is especially true of budding or inexperienced researchers.

If you’ve never put together a research proposal before or find yourself in a position where you need to explain your research methodology decisions, there are a few things you need to be aware of.

Once you understand the ins and outs, handling academic research in the future will be less intimidating. We break down the basics below:

A research methodology encompasses the way in which you intend to carry out your research. This includes how you plan to tackle things like collection methods, statistical analysis, participant observations, and more.

You can think of your research methodology as being a formula. One part will be how you plan on putting your research into practice, and another will be why you feel this is the best way to approach it. Your research methodology is ultimately a methodological and systematic plan to resolve your research problem.

In short, you are explaining how you will take your idea and turn it into a study, which in turn will produce valid and reliable results that are in accordance with the aims and objectives of your research. This is true whether your paper plans to make use of qualitative methods or quantitative methods.

The purpose of a research methodology is to explain the reasoning behind your approach to your research - you'll need to support your collection methods, methods of analysis, and other key points of your work.

Think of it like writing a plan or an outline for you what you intend to do.

When carrying out research, it can be easy to go off-track or depart from your standard methodology.

Tip: Having a methodology keeps you accountable and on track with your original aims and objectives, and gives you a suitable and sound plan to keep your project manageable, smooth, and effective.

With all that said, how do you write out your standard approach to a research methodology?

As a general plan, your methodology should include the following information:

• Your research method.  You need to state whether you plan to use quantitative analysis, qualitative analysis, or mixed-method research methods. This will often be determined by what you hope to achieve with your research.
• Explain your reasoning. Why are you taking this methodological approach? Why is this particular methodology the best way to answer your research problem and achieve your objectives?
• Explain your instruments.  This will mainly be about your collection methods. There are varying instruments to use such as interviews, physical surveys, questionnaires, for example. Your methodology will need to detail your reasoning in choosing a particular instrument for your research.
• What will you do with your results?  How are you going to analyze the data once you have gathered it?
• Advise your reader.  If there is anything in your research methodology that your reader might be unfamiliar with, you should explain it in more detail. For example, you should give any background information to your methods that might be relevant or provide your reasoning if you are conducting your research in a non-standard way.
• How will your sampling process go?  What will your sampling procedure be and why? For example, if you will collect data by carrying out semi-structured or unstructured interviews, how will you choose your interviewees and how will you conduct the interviews themselves?
• Any practical limitations?  You should discuss any limitations you foresee being an issue when you’re carrying out your research.

In any dissertation, thesis, or academic journal, you will always find a chapter dedicated to explaining the research methodology of the person who carried out the study, also referred to as the methodology section of the work.

A good research methodology will explain what you are going to do and why, while a poor methodology will lead to a messy or disorganized approach.

You should also be able to justify in this section your reasoning for why you intend to carry out your research in a particular way, especially if it might be a particularly unique method.

Having a sound methodology in place can also help you with the following:

• When another researcher at a later date wishes to try and replicate your research, they will need your explanations and guidelines.
• In the event that you receive any criticism or questioning on the research you carried out at a later point, you will be able to refer back to it and succinctly explain the how and why of your approach.
• It provides you with a plan to follow throughout your research. When you are drafting your methodology approach, you need to be sure that the method you are using is the right one for your goal. This will help you with both explaining and understanding your method.
• It affords you the opportunity to document from the outset what you intend to achieve with your research, from start to finish.

A research instrument is a tool you will use to help you collect, measure and analyze the data you use as part of your research.

The choice of research instrument will usually be yours to make as the researcher and will be whichever best suits your methodology.

There are many different research instruments you can use in collecting data for your research.

Generally, they can be grouped as follows:

• Interviews (either as a group or one-on-one). You can carry out interviews in many different ways. For example, your interview can be structured, semi-structured, or unstructured. The difference between them is how formal the set of questions is that is asked of the interviewee. In a group interview, you may choose to ask the interviewees to give you their opinions or perceptions on certain topics.
• Surveys (online or in-person). In survey research, you are posing questions in which you ask for a response from the person taking the survey. You may wish to have either free-answer questions such as essay-style questions, or you may wish to use closed questions such as multiple choice. You may even wish to make the survey a mixture of both.
• Focus Groups.  Similar to the group interview above, you may wish to ask a focus group to discuss a particular topic or opinion while you make a note of the answers given.
• Observations.  This is a good research instrument to use if you are looking into human behaviors. Different ways of researching this include studying the spontaneous behavior of participants in their everyday life, or something more structured. A structured observation is research conducted at a set time and place where researchers observe behavior as planned and agreed upon with participants.

These are the most common ways of carrying out research, but it is really dependent on your needs as a researcher and what approach you think is best to take.

It is also possible to combine a number of research instruments if this is necessary and appropriate in answering your research problem.

There are three different types of methodologies, and they are distinguished by whether they focus on words, numbers, or both.

➡️ Want to learn more about the differences between qualitative and quantitative research, and how to use both methods? Check out our guide for that!

If you've done your due diligence, you'll have an idea of which methodology approach is best suited to your research.

It’s likely that you will have carried out considerable reading and homework before you reach this point and you may have taken inspiration from other similar studies that have yielded good results.

Still, it is important to consider different options before setting your research in stone. Exploring different options available will help you to explain why the choice you ultimately make is preferable to other methods.

If proving your research problem requires you to gather large volumes of numerical data to test hypotheses, a quantitative research method is likely to provide you with the most usable results.

If instead you’re looking to try and learn more about people, and their perception of events, your methodology is more exploratory in nature and would therefore probably be better served using a qualitative research methodology.

It helps to always bring things back to the question: what do I want to achieve with my research?

Once you have conducted your research, you need to analyze it. Here are some helpful guides for qualitative data analysis:

➡️  How to do a content analysis

➡️  How to do a thematic analysis

➡️  How to do a rhetorical analysis

Research methodology refers to the techniques used to find and analyze information for a study, ensuring that the results are valid, reliable and that they address the research objective.

Data can typically be organized into four different categories or methods: observational, experimental, simulation, and derived.

Writing a methodology section is a process of introducing your methods and instruments, discussing your analysis, providing more background information, addressing your research limitations, and more.

Your research methodology section will need a clear research question and proposed research approach. You'll need to add a background, introduce your research question, write your methodology and add the works you cited during your data collecting phase.

The research methodology section of your study will indicate how valid your findings are and how well-informed your paper is. It also assists future researchers planning to use the same methodology, who want to cite your study or replicate it.

• Research Process
• Manuscript Preparation
• Manuscript Review
• Publication Process
• Publication Recognition
• Language Editing Services
• Translation Services

Choosing the Right Research Methodology: A Guide for Researchers

• 3 minute read
• 46.8K views

Table of Contents

Choosing an optimal research methodology is crucial for the success of any research project. The methodology you select will determine the type of data you collect, how you collect it, and how you analyse it. Understanding the different types of research methods available along with their strengths and weaknesses, is thus imperative to make an informed decision.

Understanding different research methods:

There are several research methods available depending on the type of study you are conducting, i.e., whether it is laboratory-based, clinical, epidemiological, or survey based . Some common methodologies include qualitative research, quantitative research, experimental research, survey-based research, and action research. Each method can be opted for and modified, depending on the type of research hypotheses and objectives.

Qualitative vs quantitative research:

When deciding on a research methodology, one of the key factors to consider is whether your research will be qualitative or quantitative. Qualitative research is used to understand people’s experiences, concepts, thoughts, or behaviours . Quantitative research, on the contrary, deals with numbers, graphs, and charts, and is used to test or confirm hypotheses, assumptions, and theories. 

Qualitative research methodology:

Qualitative research is often used to examine issues that are not well understood, and to gather additional insights on these topics. Qualitative research methods include open-ended survey questions, observations of behaviours described through words, and reviews of literature that has explored similar theories and ideas. These methods are used to understand how language is used in real-world situations, identify common themes or overarching ideas, and describe and interpret various texts. Data analysis for qualitative research typically includes discourse analysis, thematic analysis, and textual analysis. 

Quantitative research methodology:

The goal of quantitative research is to test hypotheses, confirm assumptions and theories, and determine cause-and-effect relationships. Quantitative research methods include experiments, close-ended survey questions, and countable and numbered observations. Data analysis for quantitative research relies heavily on statistical methods.

Analysing qualitative vs quantitative data:

The methods used for data analysis also differ for qualitative and quantitative research. As mentioned earlier, quantitative data is generally analysed using statistical methods and does not leave much room for speculation. It is more structured and follows a predetermined plan. In quantitative research, the researcher starts with a hypothesis and uses statistical methods to test it. Contrarily, methods used for qualitative data analysis can identify patterns and themes within the data, rather than provide statistical measures of the data. It is an iterative process, where the researcher goes back and forth trying to gauge the larger implications of the data through different perspectives and revising the analysis if required.

When to use qualitative vs quantitative research:

The choice between qualitative and quantitative research will depend on the gap that the research project aims to address, and specific objectives of the study. If the goal is to establish facts about a subject or topic, quantitative research is an appropriate choice. However, if the goal is to understand people’s experiences or perspectives, qualitative research may be more suitable. 

Conclusion:

In conclusion, an understanding of the different research methods available, their applicability, advantages, and disadvantages is essential for making an informed decision on the best methodology for your project. If you need any additional guidance on which research methodology to opt for, you can head over to Elsevier Author Services (EAS). EAS experts will guide you throughout the process and help you choose the perfect methodology for your research goals.

Why is data validation important in research?

When Data Speak, Listen: Importance of Data Collection and Analysis Methods

You may also like.

Descriptive Research Design and Its Myriad Uses

Five Common Mistakes to Avoid When Writing a Biomedical Research Paper

Making Technical Writing in Environmental Engineering Accessible

To Err is Not Human: The Dangers of AI-assisted Academic Writing

Writing a good review article

Scholarly Sources: What are They and Where can You Find Them?

Input your search keywords and press Enter.

How To Choose Your Research Methodology

Qualitative vs quantitative vs mixed methods.

By: Derek Jansen (MBA). Expert Reviewed By: Dr Eunice Rautenbach | June 2021

Without a doubt, one of the most common questions we receive at Grad Coach is “ How do I choose the right methodology for my research? ”. It’s easy to see why – with so many options on the research design table, it’s easy to get intimidated, especially with all the complex lingo!

In this post, we’ll explain the three overarching types of research – qualitative, quantitative and mixed methods – and how you can go about choosing the best methodological approach for your research.

Overview: Choosing Your Methodology

Understanding the options – Qualitative research – Quantitative research – Mixed methods-based research

Choosing a research methodology – Nature of the research – Research area norms – Practicalities

1. Understanding the options

Before we jump into the question of how to choose a research methodology, it’s useful to take a step back to understand the three overarching types of research – qualitative , quantitative and mixed methods -based research. Each of these options takes a different methodological approach.

Qualitative research utilises data that is not numbers-based. In other words, qualitative research focuses on words , descriptions , concepts or ideas – while quantitative research makes use of numbers and statistics. Qualitative research investigates the “softer side” of things to explore and describe, while quantitative research focuses on the “hard numbers”, to measure differences between variables and the relationships between them.

Importantly, qualitative research methods are typically used to explore and gain a deeper understanding of the complexity of a situation – to draw a rich picture . In contrast to this, quantitative methods are usually used to confirm or test hypotheses . In other words, they have distinctly different purposes. The table below highlights a few of the key differences between qualitative and quantitative research – you can learn more about the differences here.

• Uses an inductive approach
• Is used to build theories
• Takes a subjective approach
• Adopts an open and flexible approach
• The researcher is close to the respondents
• Interviews and focus groups are oftentimes used to collect word-based data.
• Generally, draws on small sample sizes
• Uses qualitative data analysis techniques (e.g. content analysis , thematic analysis , etc)
• Uses a deductive approach
• Is used to test theories
• Takes an objective approach
• Adopts a closed, highly planned approach
• The research is disconnected from respondents
• Surveys or laboratory equipment are often used to collect number-based data.
• Generally, requires large sample sizes
• Uses statistical analysis techniques to make sense of the data

Mixed methods -based research, as you’d expect, attempts to bring these two types of research together, drawing on both qualitative and quantitative data. Quite often, mixed methods-based studies will use qualitative research to explore a situation and develop a potential model of understanding (this is called a conceptual framework), and then go on to use quantitative methods to test that model empirically.

In other words, while qualitative and quantitative methods (and the philosophies that underpin them) are completely different, they are not at odds with each other. It’s not a competition of qualitative vs quantitative. On the contrary, they can be used together to develop a high-quality piece of research. Of course, this is easier said than done, so we usually recommend that first-time researchers stick to a single approach , unless the nature of their study truly warrants a mixed-methods approach.

The key takeaway here, and the reason we started by looking at the three options, is that it’s important to understand that each methodological approach has a different purpose – for example, to explore and understand situations (qualitative), to test and measure (quantitative) or to do both. They’re not simply alternative tools for the same job. 

Right – now that we’ve got that out of the way, let’s look at how you can go about choosing the right methodology for your research.

2. How to choose a research methodology

To choose the right research methodology for your dissertation or thesis, you need to consider three important factors . Based on these three factors, you can decide on your overarching approach – qualitative, quantitative or mixed methods. Once you’ve made that decision, you can flesh out the finer details of your methodology, such as the sampling , data collection methods and analysis techniques (we discuss these separately in other posts ).

The three factors you need to consider are:

• The nature of your research aims, objectives and research questions
• The methodological approaches taken in the existing literature
• Practicalities and constraints

Let’s take a look at each of these.

Factor #1: The nature of your research

As I mentioned earlier, each type of research (and therefore, research methodology), whether qualitative, quantitative or mixed, has a different purpose and helps solve a different type of question. So, it’s logical that the key deciding factor in terms of which research methodology you adopt is the nature of your research aims, objectives and research questions .

But, what types of research exist?

Broadly speaking, research can fall into one of three categories:

• Exploratory – getting a better understanding of an issue and potentially developing a theory regarding it
• Confirmatory – confirming a potential theory or hypothesis by testing it empirically
• A mix of both – building a potential theory or hypothesis and then testing it

As a rule of thumb, exploratory research tends to adopt a qualitative approach , whereas confirmatory research tends to use quantitative methods . This isn’t set in stone, but it’s a very useful heuristic. Naturally then, research that combines a mix of both, or is seeking to develop a theory from the ground up and then test that theory, would utilize a mixed-methods approach.

Let’s look at an example in action.

If your research aims were to understand the perspectives of war veterans regarding certain political matters, you’d likely adopt a qualitative methodology, making use of interviews to collect data and one or more qualitative data analysis methods to make sense of the data.

If, on the other hand, your research aims involved testing a set of hypotheses regarding the link between political leaning and income levels, you’d likely adopt a quantitative methodology, using numbers-based data from a survey to measure the links between variables and/or constructs .

So, the first (and most important thing) thing you need to consider when deciding which methodological approach to use for your research project is the nature of your research aims , objectives and research questions. Specifically, you need to assess whether your research leans in an exploratory or confirmatory direction or involves a mix of both.

The importance of achieving solid alignment between these three factors and your methodology can’t be overstated. If they’re misaligned, you’re going to be forcing a square peg into a round hole. In other words, you’ll be using the wrong tool for the job, and your research will become a disjointed mess.

If your research is a mix of both exploratory and confirmatory, but you have a tight word count limit, you may need to consider trimming down the scope a little and focusing on one or the other. One methodology executed well has a far better chance of earning marks than a poorly executed mixed methods approach. So, don’t try to be a hero, unless there is a very strong underpinning logic.

Need a helping hand?

Factor #2: The disciplinary norms

Choosing the right methodology for your research also involves looking at the approaches used by other researchers in the field, and studies with similar research aims and objectives to yours. Oftentimes, within a discipline, there is a common methodological approach (or set of approaches) used in studies. While this doesn’t mean you should follow the herd “just because”, you should at least consider these approaches and evaluate their merit within your context.

A major benefit of reviewing the research methodologies used by similar studies in your field is that you can often piggyback on the data collection techniques that other (more experienced) researchers have developed. For example, if you’re undertaking a quantitative study, you can often find tried and tested survey scales with high Cronbach’s alphas. These are usually included in the appendices of journal articles, so you don’t even have to contact the original authors. By using these, you’ll save a lot of time and ensure that your study stands on the proverbial “shoulders of giants” by using high-quality measurement instruments .

Of course, when reviewing existing literature, keep point #1 front of mind. In other words, your methodology needs to align with your research aims, objectives and questions. Don’t fall into the trap of adopting the methodological “norm” of other studies just because it’s popular. Only adopt that which is relevant to your research.

Factor #3: Practicalities

When choosing a research methodology, there will always be a tension between doing what’s theoretically best (i.e., the most scientifically rigorous research design ) and doing what’s practical , given your constraints . This is the nature of doing research and there are always trade-offs, as with anything else.

But what constraints, you ask?

When you’re evaluating your methodological options, you need to consider the following constraints:

• Data access
• Equipment and software
• Your knowledge and skills

Let’s look at each of these.

Constraint #1: Data access

The first practical constraint you need to consider is your access to data . If you’re going to be undertaking primary research , you need to think critically about the sample of respondents you realistically have access to. For example, if you plan to use in-person interviews , you need to ask yourself how many people you’ll need to interview, whether they’ll be agreeable to being interviewed, where they’re located, and so on.

If you’re wanting to undertake a quantitative approach using surveys to collect data, you’ll need to consider how many responses you’ll require to achieve statistically significant results. For many statistical tests, a sample of a few hundred respondents is typically needed to develop convincing conclusions.

So, think carefully about what data you’ll need access to, how much data you’ll need and how you’ll collect it. The last thing you want is to spend a huge amount of time on your research only to find that you can’t get access to the required data.

Constraint #2: Time

The next constraint is time. If you’re undertaking research as part of a PhD, you may have a fairly open-ended time limit, but this is unlikely to be the case for undergrad and Masters-level projects. So, pay attention to your timeline, as the data collection and analysis components of different methodologies have a major impact on time requirements . Also, keep in mind that these stages of the research often take a lot longer than originally anticipated.

Another practical implication of time limits is that it will directly impact which time horizon you can use – i.e. longitudinal vs cross-sectional . For example, if you’ve got a 6-month limit for your entire research project, it’s quite unlikely that you’ll be able to adopt a longitudinal time horizon. 

Constraint #3: Money

As with so many things, money is another important constraint you’ll need to consider when deciding on your research methodology. While some research designs will cost near zero to execute, others may require a substantial budget .

Some of the costs that may arise include:

• Software costs – e.g. survey hosting services, analysis software, etc.
• Promotion costs – e.g. advertising a survey to attract respondents
• Incentive costs – e.g. providing a prize or cash payment incentive to attract respondents
• Equipment rental costs – e.g. recording equipment, lab equipment, etc.
• Travel costs
• Food & beverages

These are just a handful of costs that can creep into your research budget. Like most projects, the actual costs tend to be higher than the estimates, so be sure to err on the conservative side and expect the unexpected. It’s critically important that you’re honest with yourself about these costs, or you could end up getting stuck midway through your project because you’ve run out of money.

Constraint #4: Equipment & software

Another practical consideration is the hardware and/or software you’ll need in order to undertake your research. Of course, this variable will depend on the type of data you’re collecting and analysing. For example, you may need lab equipment to analyse substances, or you may need specific analysis software to analyse statistical data. So, be sure to think about what hardware and/or software you’ll need for each potential methodological approach, and whether you have access to these.

Constraint #5: Your knowledge and skillset

The final practical constraint is a big one. Naturally, the research process involves a lot of learning and development along the way, so you will accrue knowledge and skills as you progress. However, when considering your methodological options, you should still consider your current position on the ladder.

Some of the questions you should ask yourself are:

• Am I more of a “numbers person” or a “words person”?
• How much do I know about the analysis methods I’ll potentially use (e.g. statistical analysis)?
• How much do I know about the software and/or hardware that I’ll potentially use?
• How excited am I to learn new research skills and gain new knowledge?
• How much time do I have to learn the things I need to learn?

Answering these questions honestly will provide you with another set of criteria against which you can evaluate the research methodology options you’ve shortlisted.

So, as you can see, there is a wide range of practicalities and constraints that you need to take into account when you’re deciding on a research methodology. These practicalities create a tension between the “ideal” methodology and the methodology that you can realistically pull off. This is perfectly normal, and it’s your job to find the option that presents the best set of trade-offs.

Recap: Choosing a methodology

In this post, we’ve discussed how to go about choosing a research methodology. The three major deciding factors we looked at were:

• Exploratory
• Confirmatory
• Combination
• Research area norms
• Hardware and software
• Your knowledge and skillset

If you have any questions, feel free to leave a comment below. If you’d like a helping hand with your research methodology, check out our 1-on-1 research coaching service , or book a free consultation with a friendly Grad Coach.

Psst... there’s more!

This post was based on one of our popular Research Bootcamps . If you're working on a research project, you'll definitely want to check this out ...

Very useful and informative especially for beginners

Nice article! I’m a beginner in the field of cybersecurity research. I am a Telecom and Network Engineer and Also aiming for PhD scholarship.

I find the article very informative especially for my decitation it has been helpful and an eye opener.

Hi I am Anna ,

I am a PHD candidate in the area of cyber security, maybe we can link up

The Examples shows by you, for sure they are really direct me and others to knows and practices the Research Design and prepration.

I found the post very informative and practical.

I struggle so much with designs of the research for sure!

I’m the process of constructing my research design and I want to know if the data analysis I plan to present in my thesis defense proposal possibly change especially after I gathered the data already.

Thank you so much this site is such a life saver. How I wish 1-1 coaching is available in our country but sadly it’s not.

Submit a Comment Cancel reply

Your email address will not be published. Required fields are marked *

Save my name, email, and website in this browser for the next time I comment.

• Print Friendly

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

• View all journals
• Explore content
• About the journal
• Publish with us
• Sign up for alerts
• Perspective
• Published: 29 November 2022

The fundamental importance of method to theory

• Rick Dale   ORCID: orcid.org/0000-0001-7865-474X 1 ,
• Anne S. Warlaumont   ORCID: orcid.org/0000-0001-9450-1372 1 &
• Kerri L. Johnson   ORCID: orcid.org/0000-0002-1458-2019 1 , 2  

Nature Reviews Psychology volume  2 ,  pages 55–66 ( 2023 ) Cite this article

788 Accesses

3 Citations

15 Altmetric

Metrics details

• Communication
• Human behaviour

Many domains of inquiry in psychology are concerned with rich and complex phenomena. At the same time, the field of psychology is grappling with how to improve research practices to address concerns with the scientific enterprise. In this Perspective, we argue that both of these challenges can be addressed by adopting a principle of methodological variety. According to this principle, developing a variety of methodological tools should be regarded as a scientific goal in itself, one that is critical for advancing scientific theory. To illustrate, we show how the study of language and communication requires varied methodologies, and that theory development proceeds, in part, by integrating disparate tools and designs. We argue that the importance of methodological variation and innovation runs deep, travelling alongside theory development to the core of the scientific enterprise. Finally, we highlight ongoing research agendas that might help to specify, quantify and model methodological variety and its implications.

This is a preview of subscription content, access via your institution

Access options

Subscribe to this journal

Receive 12 digital issues and online access to articles

55,14 € per year

only 4,60 € per issue

Buy this article

• Purchase on Springer Link
• Instant access to full article PDF

Prices may be subject to local taxes which are calculated during checkout

Similar content being viewed by others

Comparing meta-analyses and preregistered multiple-laboratory replication projects

High replicability of newly discovered social-behavioural findings is achievable

The replication crisis has led to positive structural, procedural, and community changes

Hacking, I. Representing and Intervening: Introductory Topics in The Philosophy of Natural Science (Cambridge Univ. Press, 1983).

Mayo, D. G. in PSA: Proceedings of the Biennial Meeting of the Philosophy of Science Association Vol. 1994, 270–279 (Cambridge Univ. Press, 1994).

Ackermann, R. The new experimentalism. Brit. J. Philos. Sci. 40 , 185–190 (1989).

Article   Google Scholar  

Simons, M. & Vagelli, M. Were experiments ever neglected? Ian Hacking and the history of philosophy of experiment. Phil. Inq. 9 , 167–188 (2021).

Google Scholar  

Nosek, B. A. et al. Promoting an open research culture. Science 348 , 1422–1425 (2015).

Richardson, D. C., Dale, R. & Tomlinson, J. M. Conversation, gaze coordination, and beliefs about visual context. Cogn. Sci. 33 , 1468–1482 (2009).

Laidlaw, K. E., Foulsham, T., Kuhn, G. & Kingstone, A. Potential social interactions are important to social attention. Proc. Natl Acad. Sci. USA 108 , 5548–5553 (2011).

Richardson, D. C. et al. Joint perception: gaze and social context. Front. Hum. Neurosci. 6 , 194 (2012).

Risko, E. F., Richardson, D. C. & Kingstone, A. Breaking the fourth wall of cognitive science: real-world social attention and the dual function of gaze. Curr. Dir. Psychol. Sci. 25 , 70–74 (2016).

Levin, I. P., Schneider, S. L. & Gaeth, G. J. All frames are not created equal: a typology and critical analysis of framing effects. Organ. Behav. Hum. Decis. Process. 76 , 149–188 (1998).

Pothos, E. M. & Busemeyer, J. R. A quantum probability explanation for violations of ‘rational’ decision theory. Proc. R. Soc. B 276 , 2171–2178 (2009).

Pärnamets, P. et al. Biasing moral decisions by exploiting the dynamics of eye gaze. Proc. Natl Acad. Sci. USA 112 , 4170–4175 (2015).

Vinson, D. W., Dale, R. & Jones, M. N. Decision contamination in the wild: sequential dependencies in online review ratings. Behav. Res. Methods 51 , 1477–1484 (2019).

Longino, H. E. Gender, politics, and the theoretical virtues. Synthese 104 , 383–397 (1995).

Hansson, S. O. in Stanford Encyclopedia of Philosophy (ed. Zalta, E. N.) https://plato.stanford.edu/archives/fall2021/entries/pseudo-science/ (Stanford Univ., 2015).

Dupré, J. The Disorder of Things: Metaphysical Foundations of the Disunity of Science (Harvard Univ. Press, 1993).

Cartwright, N. The Dappled World: A Study of the Boundaries of Science (Cambridge Univ. Press, 1999).

McCauley, R. N. & Bechtel, W. Explanatory pluralism and heuristic identity theory. Theory Psychol. 11 , 736–760 (2001).

Mitchell, S. D. Integrative pluralism. Biol. Phil. 17 , 55–70 (2002).

Abrahamsen, A. & Bechtel, W. in Contemporary Debates in Cognitive Science (ed. Stainton, R.) 159–187 (Blackwell, 2006).

Kellert, S. H., Longino, H. E. & Waters, C. K. Scientific Pluralism (Univ. Minnesota Press, 2006).

Dale, R., Dietrich, E. & Chemero, A. Explanatory pluralism in cognitive science. Cogn. Sci. 33 , 739–742 (2009).

Weisberg, M. & Muldoon, R. Epistemic landscapes and the division of cognitive labor. Phil. Sci. 76 , 225–252 (2009).

Zollman, K. J. S. The epistemic benefit of transient diversity. Erkenntnis 72 , 17–35 (2010).

Horst, S. Beyond reduction: from naturalism to cognitive pluralism. Mind Matter 12 , 197–244 (2014).

Open Science Collaboration. Estimating the reproducibility of psychological science. Science 349 , 943 (2015).

Lin, H., Werner, K. M. & Inzlicht, M. Promises and perils of experimentation: the mutual-internal-validity problem. Perspect. Psychol. Sci. 16 , 854–863 (2021).

van Rooij, I. & Baggio, G. Theory before the test: how to build high-verisimilitude explanatory theories in psychological science. Perspect. Psychol. Sci. 16 , 682–697 (2021).

MacLeod, C. M. The Stroop task: the gold standard of attentional measures. J. Exp. Psychol. Gen. 121 , 12–14 (1992).

Eriksen, C. W. The flankers task and response competition: a useful tool for investigating a variety of cognitive problems. Vis. Cogn. 2 , 101–118 (1995).

Dickter, C. L. & Bartholow, B. D. Ingroup categorization and response conflict: interactive effects of target race, flanker compatibility, and infrequency on N2 amplitude. Psychophysiology 47 , 596–601 (2010).

Parris, B. A., Hasshim, N., Wadsley, M., Augustinova, M. & Ferrand, L. The loci of Stroop effects: a critical review of methods and evidence for levels of processing contributing to color-word Stroop effects and the implications for the loci of attentional selection. Psychol. Res. 86 , 1029–1053 (2022).

Barzykowski, K., Wereszczyski, M., Hajdas, S. & Radel, R. Cognitive inhibition behavioral tasks in online and laboratory settings: data from Stroop, SART and Eriksen Flanker tasks. Data Brief. 43 , 108398 (2022).

Gobel, M. S., Kim, H. S. & Richardson, D. C. The dual function of social gaze. Cognition 136 , 359–364 (2015).

Gallup, A. C., Chong, A. & Couzin, I. D. The directional flow of visual information transfer between pedestrians. Biol. Lett. 8 , 520–522 (2012).

Lick, D. J. & Johnson, K. L. Straight until proven gay: a systematic bias toward straight categorizations in sexual orientation judgments. J. Pers. Soc. Psychol. 110 , 801 (2016).

Alt, N. P., Lick, D. J. & Johnson, K. L. The straight categorization bias: a motivated and altruistic reasoning account. J. Pers. Soc. Psychol. https://doi.org/10.1037/pspi0000232 (2020).

Popper, K. Conjectures and Refutations: The Growth of Scientific Knowledge (Routledge, 2002).

Heit, E. & Hahn, U. Diversity-based reasoning in children. Cogn. Psychol. 43 , 243–273 (2001).

Heit, E., Hahn, U. & Feeney, A. in Categorization Inside and Outside the Laboratory: Essays in Honor of Douglas L. Medin 87–99 (American Psychological Association, 2005).

MacWhinney, B. in The Handbook of Linguistics 2nd edn (eds. Aronoff, M. & Rees-Miller, J.) 397–413 (Wiley, 2017).

Stivers, T. et al. Universals and cultural variation in turn-taking in conversation. Proc. Natl Acad. Sci. USA 106 , 10587–10592 (2009).

Louwerse, M. M., Dale, R., Bard, E. G. & Jeuniaux, P. Behavior matching in multimodal communication is synchronized. Cogn. Sci. 36 , 1404–1426 (2012).

Fusaroli, R., Bjørndahl, J. S., Roepstorff, A. & Tylén, K. A heart for interaction: shared physiological dynamics and behavioral coordination in a collective, creative construction task. J. Exp. Psychol. Hum. Percept. Perform. 42 , 1297 (2016).

Rasenberg, M., Özyürek, A. & Dingemanse, M. Alignment in multimodal interaction: an integrative framework. Cogn. Sci. 44 , e12911 (2020).

Dunn, M., Greenhill, S. J., Levinson, S. C. & Gray, R. D. Evolved structure of language shows lineage-specific trends in word-order universals. Nature 473 , 79–82 (2011).

Hua, X., Greenhill, S. J., Cardillo, M., Schneemann, H. & Bromham, L. The ecological drivers of variation in global language diversity. Nat. Commun. 10 , 1–10 (2019).

Christiansen, M. H. & Chater, N. The now-or-never bottleneck: a fundamental constraint on language. Behav. Brain Sci. 39 , e62 (2016).

Fitch, W. T., De Boer, B., Mathur, N. & Ghazanfar, A. A. Monkey vocal tracts are speech-ready. Sci. Adv. 2 , e1600723 (2016).

Hauser, M. D., Chomsky, N. & Fitch, W. T. The faculty of language: what is it, who has it, and how did it evolve? Science 298 , 1569–1579 (2002).

Cowley, S. J. Distributed Language (John Benjamins, 2011).

Samuels, R. Nativism in cognitive science. Mind Lang. 17 , 233–265 (2002).

Behme, C. & Deacon, S. H. Language learning in infancy: does the empirical evidence support a domain specific language acquisition device? Phil. Psychol. 21 , 641–671 (2008).

Chomsky, N. 4. A Review Of BF Skinner’s Verbal Behavior (Harvard Univ. Press, 2013).

Vihman, M. M. Phonological Development: The Origins of Language in the Child (Blackwell, 1996).

Oller, D. K. The Emergence of the Speech Capacity (Psychology Press, 2000).

Clark, E. V. & Casillas, M. First Language Acquisition (Routledge, 2015).

Goldstein, M. H., King, A. P. & West, M. J. Social interaction shapes babbling: testing parallels between birdsong and speech. Proc. Natl Acad. Sci. USA 100 , 8030–8035 (2003).

Warlaumont, A. S. Modeling the emergence of syllabic structure. J. Phonet. 53 , 61–65 (2015).

VanDam, M. et al. HomeBank: An online repository of daylong child-centered audio recordings. Semin. Speech Lang. 37 , 128–142 (2016).

Elmlinger, S. L., Schwade, J. A. & Goldstein, M. H. The ecology of prelinguistic vocal learning: parents simplify the structure of their speech in response to babbling. J. Child Lang. 46 , 998–1011 (2019).

Roy, B. C., Frank, M. C., DeCamp, P., Miller, M. & Roy, D. Predicting the birth of a spoken word. Proc. Natl Acad. Sci. USA 112 , 12663–12668 (2015).

McClelland, J. L. The place of modeling in cognitive science. Top. Cogn. Sci. 1 , 11–38 (2009).

Smaldino, P. E. in Computational Social Psychology (eds Vallacher’, R., Read, S. J. & Nowak, A.) 311–331 (Routledge, 2017).

Guest, O. & Martin, A. E. How computational modeling can force theory building in psychological science. Perspect. Psychol. Sci. 16 , 789–802 (2021).

Elman, J. L., Bates, E. A. & Johnson, M. H. Rethinking Innateness: A Connectionist Perspective on Development Vol. 10 (MIT Press, 1996).

Warlaumont, A. S., Westermann, G., Buder, E. H. & Oller, D. K. Prespeech motor learning in a neural network using reinforcement. Neural Netw. 38 , 64–75 (2013).

Warlaumont, A. S. & Finnegan, M. K. Learning to produce syllabic speech sounds via reward-modulated neural plasticity. PLoS One 11 , e0145096 (2016).

MacWhinney, B. & Snow, C. The child language data exchange system: an update. J. Child. Lang. 17 , 457–472 (1990).

MacWhinney, B. The CHILDES Project: Tools for Analyzing Talk 3rd edn (Psychology Press, 2014).

Kachergis, G., Marchman, V. A. & Frank, M. C. Toward a “standard model” of early language learning. Curr. Dir. Psychol. Sci. 31 , 20–27 (2022).

Lewis, J. D. & Elman, J. L. A connectionist investigation of linguistic arguments from the poverty of the stimulus: learning the unlearnable. In Proc. Annual Meeting of the Cognitive Science Society Vol. 23, 552–557 (eds. Moore, J. D. & Stenning, K.) (2001).

Regier, T. & Gahl, S. Learning the unlearnable: the role of missing evidence. Cognition 93 , 147–155 (2004).

Reali, F. & Christiansen, M. H. Uncovering the richness of the stimulus: structure dependence and indirect statistical evidence. Cogn. Sci. 29 , 1007–1028 (2005).

Foraker, S., Regier, T., Khetarpal, N., Perfors, A. & Tenenbaum, J. Indirect evidence and the poverty of the stimulus: the case of anaphoric one . Cognit. Sci. 33 , 287–300 (2009).

Saffran, J. R., Aslin, R. N. & Newport, E. L. Statistical learning by 8-month-old infants. Science 274 , 1926–1928 (1996).

McMurray, B. & Hollich, G. Core computational principles of language acquisition: can statistical learning do the job? Dev. Sci. 12 , 365–368 (2009).

Frost, R., Armstrong, B. C., Siegelman, N. & Christiansen, M. H. Domain generality versus modality specificity: the paradox of statistical learning. Trends Cogn. Sci. 19 , 117–125 (2015).

Isbilen, E. S., Frost, R. L. A., Monaghan, P. & Christiansen, M. H. Statistically based chunking of nonadjacent dependencies. J. Exp. Psychol. Gen. https://doi.org/10.1037/xge0001207 (2022).

Ruba, A. L., Pollak, S. D. & Saffran, J. R. Acquiring complex communicative systems: Statistical learning of language and emotion. Top. Cogn. Sci. https://doi.org/10.1111/tops.12612 (2022).

Abney, D. H., Warlaumont, A. S., Oller, D. K., Wallot, S. & Kello, C. T. Multiple coordination patterns in infant and adult vocalizations. Infancy 22 , 514–539 (2017).

Mendoza, J. K. & Fausey, C. M. Everyday music in infancy. Dev. Sci. 24 , e13122 (2019).

Ritwika, V. et al. Exploratory dynamics of vocal foraging during infant-caregiver communication. Sci. Rep. 10 , 10469 (2020).

Mendoza, J. K. & Fausey, C. M. Quantifying everyday ecologies: principles for manual annotation of many hours of infants’ lives. Front. Psychol. 12 , 710636 (2021).

Fernald, A., Zangl, R., Portillo, A. L. & Marchman, V. A. in Developmental Psycholinguistics: On-line Methods in Children’s Language Processing (eds. Sekerina, I. A., Fernández, E. M. & Clahsen, H.) Vol. 44, 97 (John Benjamins, 2008).

Weisleder, A. & Fernald, A. Talking to children matters: early language experience strengthens processing and builds vocabulary. Psychol. Sci. 24 , 2143–2152 (2013).

Bergelson, E. & Aslin, R. N. Nature and origins of the lexicon in 6-mo-olds. Proc. Natl Acad. Sci. USA 114 , 12916–12921 (2017).

Brennan, S. E., Galati, A. & Kuhlen, A. K. in Psychology of Learning and Motivation (ed. Ross, B. H.) Vol. 53, 301–344 (Elsevier, 2010).

Streeck, J., Goodwin, C. & LeBaron, C. Embodied Interaction: Language and Body in the Material World (Cambridge Univ. Press, 2011).

Goodwin, C. Co-operative Action (Cambridge Univ. Press, 2017).

Dale, R., Spivey, M. J. in Eye-Tracking In Interaction. Studies On The Role Of Eye Gaze In Dialogue (eds. Oben, B. and Brône, G.) 67–90 (John Benjamins, 2018).

Richardson, D. C. & Spivey, M. J. in Encyclopedia of Biomaterials and Biomedical Engineering 573–582 (CRC Press, 2004).

Tanenhaus, M. K., Spivey-Knowlton, M. J., Eberhard, K. M. & Sedivy, J. C. Integration of visual and linguistic information in spoken language comprehension. Science 268 , 1632–1634 (1995).

Spivey, M. J., Tanenhaus, M. K., Eberhard, K. M. & Sedivy, J. C. Eye movements and spoken language comprehension: effects of visual context on syntactic ambiguity resolution. Cogn. Psychol. 45 , 447–481 (2002).

Richardson, D. C., Dale, R. & Spivey, M. J. Eye movements in language and cognition. Methods Cogn. Linguist. 18 , 323–344 (2007).

Ferreira, F. & Clifton, C. Jr The independence of syntactic processing. J. Mem. Lang. 25 , 348–368 (1986).

Kamide, Y., Altmann, G. T. & Haywood, S. L. The time-course of prediction in incremental sentence processing: evidence from anticipatory eye movements. J. Mem. Lang. 49 , 133–156 (2003).

Coco, M. I., Keller, F. & Malcolm, G. L. Anticipation in real‐world scenes: the role of visual context and visual memory. Cogn. Sci. 40 , 1995–2024 (2016).

Coco, M. I. & Keller, F. Scan patterns predict sentence production in the cross‐modal processing of visual scenes. Cogn. Sci. 36 , 1204–1223 (2012).

Kieslich, P. J., Henninger, F., Wulff, D. U., Haslbeck, J. M. & Schulte-Mecklenbeck, M. in A Handbook of Process Tracing Methods 111–130 (Routledge, 2019).

Spivey, M. J., Grosjean, M. & Knoblich, G. Continuous attraction toward phonological competitors. Proc. Natl Acad. Sci. USA 102 , 10393–10398 (2005).

Freeman, J., Dale, R. & Farmer, T. Hand in motion reveals mind in motion. Front. Psychol. 2 , 59 (2011).

Freeman, J. B. & Johnson, K. L. More than meets the eye: split-second social perception. Trends Cogn. Sci. 20 , 362–374 (2016).

Goodale, B. M., Alt, N. P., Lick, D. J. & Johnson, K. L. Groups at a glance: perceivers infer social belonging in a group based on perceptual summaries of sex ratio. J. Exp. Psychol. Gen. 147 , 1660–1676 (2018).

Sneller, B. & Roberts, G. Why some behaviors spread while others don’t: a laboratory simulation of dialect contact. Cognition 170 , 298–311 (2018).

Atkinson, M., Mills, G. J. & Smith, K. Social group effects on the emergence of communicative conventions and language complexity. J. Lang. Evol. 4 , 1–18 (2019).

Raviv, L., Meyer, A. & Lev‐Ari, S. The role of social network structure in the emergence of linguistic structure. Cogn. Sci. 44 , e12876 (2020).

Lupyan, G. & Dale, R. Language structure is partly determined by social structure. PLoS One 5 , e8559 (2010).

Lupyan, G. & Dale, R. Why are there different languages? The role of adaptation in linguistic diversity. Trends Cogn. Sci. 20 , 649–660 (2016).

Wu, L., Waber, B. N., Aral, S., Brynjolfsson, E. & Pentland, A. Mining face-to-face interaction networks using sociometric badges: predicting productivity in an IT configuration task. Inf. Syst. Behav. Soc. Methods https://doi.org/10.2139/ssrn.1130251 (2008).

Paxton, A. & Dale, R. Argument disrupts interpersonal synchrony. Q. J. Exp. Psychol. 66 , 2092–2102 (2013).

Alviar, C., Dale, R. & Galati, A. Complex communication dynamics: exploring the structure of an academic talk. Cogn. Sci. 43 , e12718 (2019).

Joo, J., Bucy, E. P. & Seidel, C. Automated coding of televised leader displays: detecting nonverbal political behavior with computer vision and deep learning. Int. J. Commun. 13 , 4044–4066 (2019).

Metallinou, A. et al. The USC CreativeIT database of multimodal dyadic interactions: from speech and full body motion capture to continuous emotional annotations. Lang. Res. Eval.   50 , 497–521 (2016).

Pouw, W., Paxton, A., Harrison, S. J. & Dixon, J. A. Acoustic information about upper limb movement in voicing. Proc. Natl Acad. Sci. USA 117 , 11364–11367 (2020).

Enfield, N., Levinson, S. C., De Ruiter, J. P. & Stivers, T. in Field Manual Vol. 10, 96–99 (ed. Majid, A.) (Max Planck Institute for Psycholinguistics, 2007).

Enfield, N. & Sidnell, J. On the concept of action in the study of interaction. Discourse Stud. 19 , 515–535 (2017).

Duran, N. D., Paxton, A. & Fusaroli, R. ALIGN: analyzing linguistic interactions with generalizable techNiques — a Python library. Psychol. Methods 24 , 419 (2019).

Brennan, S. E. & Clark, H. H. Conceptual pacts and lexical choice in conversation. J. Exp. Psychol. Learn. Mem. Cogn. 22 , 1482–1493 (1996).

Hasson, U., Nir, Y., Levy, I., Fuhrmann, G. & Malach, R. Intersubject synchronization of cortical activity during natural vision. Science 303 , 1634–1640 (2004).

Huth, A. G., De Heer, W. A., Griffiths, T. L., Theunissen, F. E. & Gallant, J. L. Natural speech reveals the semantic maps that tile human cerebral cortex. Nature 532 , 453–458 (2016).

Shain, C. et al. Robust effects of working memory demand during naturalistic language comprehension in language-selective cortex. J. Neurosci.   42 , 7412–7430 (2022).

Fedorenko, E., Blank, I. A., Siegelman, M. & Mineroff, Z. Lack of selectivity for syntax relative to word meanings throughout the language network. Cognition 203 , 104348 (2020).

Stephens, G. J., Silbert, L. J. & Hasson, U. Speaker–listener neural coupling underlies successful communication. Proc. Natl Acad. Sci. USA 107 , 14425–14430 (2010).

Schilbach, L. et al. Toward a second-person neuroscience. Behav. Brain Sci. 36 , 393–414 (2013).

Redcay, E. & Schilbach, L. Using second-person neuroscience to elucidate the mechanisms of social interaction. Nat. Rev. Neurosci. 20 , 495–505 (2019).

Riley, M. A., Richardson, M., Shockley, K. & Ramenzoni, V. C. Interpersonal synergies. Front. Psychol. 2 , 38 (2011).

Dale, R., Fusaroli, R., Duran, N. D. & Richardson, D. C. in Psychology of Learning and Motivation (ed. Ross, B. H.) Vol. 59, 43–95 (Elsevier, 2013).

Fusaroli, R., Rączaszek-Leonardi, J. & Tylén, K. Dialog as interpersonal synergy. N. Ideas Psychol. 32 , 147–157 (2014).

Hadley, L. V., Naylor, G. & Hamilton, A. F. D. C. A review of theories and methods in the science of face-to-face social interaction. Nat. Rev. Psychol. 1 , 42–54 (2022).

Cornejo, C., Cuadros, Z., Morales, R. & Paredes, J. Interpersonal coordination: methods achievements and challenges. Front. Psychol. https://doi.org/10.3389/fpsyg.2017.01685  (2017).

Smaldino, P. E. in Computational Social Psychology 311–331 (Routledge, 2017).

Devezer, B., Nardin, L. G., Baumgaertner, B. & Buzbas, E. O. Scientific discovery in a model-centric framework: reproducibility, innovation, and epistemic diversity. PLoS One 14 , e0216125 (2019).

Sulik, J., Bahrami, B. & Deroy, O. The diversity gap: when diversity matters for knowledge. Perspect. Psychol. Sci. 17 , 752–767 (2022).

O’Connor, C. & Bruner, J. Dynamics and diversity in epistemic communities. Erkenntnis 84 , 101–119 (2019).

Longino, H. in The Stanford Encyclopedia of Philosophy (ed. Zalta, E. N.) https://plato.stanford.edu/archives/sum2019/entries/scientific-knowledge-social/ (Stanford Univ., 2019).

Van Rooij, I. The tractable cognition thesis. Cogn. Sci. 32 , 939–984 (2008).

Kwisthout, J., Wareham, T. & Van Rooij, I. Bayesian intractability is not an ailment that approximation can cure. Cogn. Sci. 35 , 779–784 (2011).

Contreras Kallens, P. & Dale, R. Exploratory mapping of theoretical landscapes through word use in abstracts. Scientometrics 116 , 1641–1674 (2018).

Methods for methods’ sake. Nat. Methods https://doi.org/10.1038/nmeth1004-1 (2004).

Oberauer, K. & Lewandowsky, S. Addressing the theory crisis in psychology. Psychon. Bull. Rev. 26 , 1596–1618 (2019).

Meehl, P. E. Theory-testing in psychology and physics: a methodological paradox. Phil. Sci. 34 , 103–115 (1967).

Klein, O. et al. A practical guide for transparency in psychological science. Collabra Psychol. 4 , 20 (2018).

Muthukrishna, M. & Henrich, J. A problem in theory. Nat. Hum. Behav. 3 , 221–229 (2019).

Eronen, M. I. & Bringmann, L. F. The theory crisis in psychology: how to move forward. Perspect. Psychol. Sci. https://doi.org/10.1177/1745691620970586 (2021).

Borsboom, D., van der Maas, H. L. J., Dalege, J., Kievit, R. A. & Haig, B. D. Theory construction methodology: a practical framework for building theories in psychology. Perspect. Psychol. Sci. https://doi.org/10.1177/1745691620969647 (2021).

Kyvik, S. & Reymert, I. Research collaboration in groups and networks: differences across academic fields. Scientometrics 113 , 951–967 (2017).

Tebes, J. K. & Thai, N. D. Interdisciplinary team science and the public: steps toward a participatory team science. Am. Psychol. 73 , 549 (2018).

Falk-Krzesinski, H. J. et al. Mapping a research agenda for the science of team science. Res. Eval. 20 , 145–158 (2011).

da Silva, J. A. T. The Matthew effect impacts science and academic publishing by preferentially amplifying citations, metrics and status. Scientometrics 126 , 5373–5377 (2021).

Scheel, A. M., Tiokhin, L., Isager, P. M. & Lakens, D. Why hypothesis testers should spend less time testing hypotheses. Perspect. Psychol. Sci. 16 , 744–755 (2020).

Jones, M. N. Big Data in Cognitive Science (Psychology Press, 2016).

Paxton, A. & Griffiths, T. L. Finding the traces of behavioral and cognitive processes in big data and naturally occurring datasets. Behav. Res. Methods 49 , 1630–1638 (2017).

Lupyan, G. & Goldstone, R. L. Beyond the lab: using big data to discover principles of cognition. Behav Res. Methods , 51 , 1554–3528 (2019).

Haspelmath, M., Dryer, M. S., Gil, D. & Comrie, B. (eds) The World Atlas of Language Structures (Max Planck Digital Library, 2013).

Eberhard, D. M., Simons, G. F. & Fennig, C. D. Ethnologue: Languages of the World (SIL International, 2021).

MacCorquodale, K. & Meehl, P. E. On a distinction between hypothetical constructs and intervening variables. Psychol. Rev. 55 , 95–107 (1948).

van Rooij, I. & Baggio, G. Theory before the test: how to build high-verisimilitude explanatory theories in psychological science. Perspect. Psychol. Sci . https://doi.org/10.1177/1745691620970604 (2021).

Christiansen, M. H. & Chater, N. Creating Language: Integrating Evolution, Acquisition, and Processing (MIT Press, 2016).

Berwick, R. C. & Chomsky, N. Why Only Us: Language and Evolution (MIT Press, 2016).

Gopnik, A. Scientific thinking in young children: theoretical advances, empirical research, and policy implications. Science 337 , 1623–1627 (2012).

Pereira, A. F., James, K. H., Jones, S. S. & Smith, L. B. Early biases and developmental changes in self-generated object views. J. Vis. 10 , 22–22 (2010).

Fagan, M. K. & Iverson, J. M. The influence of mouthing on infant vocalization. Infancy 11 , 191–202 (2007).

Martin, J., Ruthven, M., Boubertakh, R. & Miquel, M. E. Realistic dynamic numerical phantom for MRI of the upper vocal tract. J. Imaging 6 , 86 (2020).

Spivey, M. J. & Dale, R. Continuous dynamics in real-time cognition. Curr. Dir. Psychol. Sci. 15 , 207–211 (2006).

Publication Manual of the American Psychological Association 3rd edn (American Psychological Association, 1983).

Publication Manual of the American Psychological Association 6th edn (American Psychological Association, 2010).

Ashby, W. R. An Introduction to Cybernetics (Martino, 1956).

de Raadt, J. D. R. Ashby’s law of requisite variety: an empirical study. Cybern. Syst. 18 , 517–536 (1987).

Ward, L. M. Dynamical Cognitive Science (MIT Press, 2002).

Regier, T., Carstensen, A. & Kemp, C. Languages support efficient communication about the environment: words for snow revisited. PLoS One 11 , e0151138 (2016).

Newell, A. Unified Theories of Cognition (Harvard Univ. Press, 1990).

Rich, P., de Haan, R., Wareham, T. & van Rooij, I. in Proc. Annual Meeting of the Cognitive Science Society Vol. 43, 3034–3040 (eds. Fitch, T., Lamm, C., Leder, H., and Teßmar-Raible, K.) (2021).

Potochnik, A. & Sanches de Oliveira, G. Patterns in cognitive phenomena and pluralism of explanatory styles. Top. Cogn. Sci. 12 , 1306–1320 (2020).

Leydesdorff, L. & Schank, T. Dynamic animations of journal maps: indicators of structural changes and interdisciplinary developments. J. Am. Soc. Inf. Sci. Technol. 59 , 1810–1818 (2008).

Leydesdorff, L. & Goldstone, R. L. Interdisciplinarity at the journal and specialty level: the changing knowledge bases of the journal Cognitive Science . J. Assoc. Inf. Sci. Technol. 65 , 164–177 (2014).

DeStefano, I., Oey, L. A., Brockbank, E. & Vul, E. Integration by parts: collaboration and topic structure in the CogSci community. Top. Cogn. Sci. 13 , 399–413 (2021).

Cummins, R. in Explanation And Cognition (eds Keil, F. C. & Wilson, R.) 117–144 (MIT Press, 2000).

Boyd, N. M. & Bogen, J. in Stanford Encyclopedia of Philosophy (ed. Zalta, E. N.) https://plato.stanford.edu/archives/win2021/entries/science-theory-observation/ (Stanford Univ., 2021).

Smaldino, P. E. How to build a strong theoretical foundation. Psychol. Inq. 31 , 297–301 (2020).

Chang, H. Inventing Temperature: Measurement and Scientific Progress (Oxford Univ. Press, 2004).

Download references

Acknowledgements

A.S.W. was supported by the National Science Foundation (grants 1529127 and 1539129/1827744) and by the James S. McDonnell Foundation ( https://doi.org/10.37717/220020507 ). K.L.J. was supported by the National Science Foundation (grant 2017245).

Author information

Authors and affiliations.

Department of Communication, University of California, Los Angeles, Los Angeles, CA, USA

Rick Dale, Anne S. Warlaumont & Kerri L. Johnson

Department of Psychology, University of California, Los Angeles, Los Angeles, CA, USA

Kerri L. Johnson

You can also search for this author in PubMed   Google Scholar

Contributions

R.D. discussed the submission theme with the editor and wrote the first draft. A.S.W. and K.L.J. refined and added to this plan and contributed major new sections of writing and revision. All authors contributed to developing the figures.

Corresponding author

Correspondence to Rick Dale .

Ethics declarations

Competing interests.

The authors declare no competing interests.

Peer review

Peer review information.

Nature Reviews Psychology thanks Berna Devezer; Michael Frank, who co-reviewed with Anjie Cao; and Justin Sulik for their contribution to the peer review of this work.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Cite this article.

Dale, R., Warlaumont, A.S. & Johnson, K.L. The fundamental importance of method to theory. Nat Rev Psychol 2 , 55–66 (2023). https://doi.org/10.1038/s44159-022-00120-5

Download citation

Accepted : 22 September 2022

Published : 29 November 2022

Issue Date : January 2023

DOI : https://doi.org/10.1038/s44159-022-00120-5

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

This article is cited by

Mechanisms linking social media use to adolescent mental health vulnerability.

• Adrian Meier
• Sarah-Jayne Blakemore

Nature Reviews Psychology (2024)

Quick links

• Explore articles by subject
• Guide to authors
• Editorial policies

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

What is Research Methodology? Definition, Types, and Examples

Research methodology 1,2 is a structured and scientific approach used to collect, analyze, and interpret quantitative or qualitative data to answer research questions or test hypotheses. A research methodology is like a plan for carrying out research and helps keep researchers on track by limiting the scope of the research. Several aspects must be considered before selecting an appropriate research methodology, such as research limitations and ethical concerns that may affect your research.

The research methodology section in a scientific paper describes the different methodological choices made, such as the data collection and analysis methods, and why these choices were selected. The reasons should explain why the methods chosen are the most appropriate to answer the research question. A good research methodology also helps ensure the reliability and validity of the research findings. There are three types of research methodology—quantitative, qualitative, and mixed-method, which can be chosen based on the research objectives.

What is research methodology ?

A research methodology describes the techniques and procedures used to identify and analyze information regarding a specific research topic. It is a process by which researchers design their study so that they can achieve their objectives using the selected research instruments. It includes all the important aspects of research, including research design, data collection methods, data analysis methods, and the overall framework within which the research is conducted. While these points can help you understand what is research methodology, you also need to know why it is important to pick the right methodology.

Why is research methodology important?

Having a good research methodology in place has the following advantages: 3

• Helps other researchers who may want to replicate your research; the explanations will be of benefit to them.
• You can easily answer any questions about your research if they arise at a later stage.
• A research methodology provides a framework and guidelines for researchers to clearly define research questions, hypotheses, and objectives.
• It helps researchers identify the most appropriate research design, sampling technique, and data collection and analysis methods.
• A sound research methodology helps researchers ensure that their findings are valid and reliable and free from biases and errors.
• It also helps ensure that ethical guidelines are followed while conducting research.
• A good research methodology helps researchers in planning their research efficiently, by ensuring optimum usage of their time and resources.

Writing the methods section of a research paper? Let Paperpal help you achieve perfection

Types of research methodology.

There are three types of research methodology based on the type of research and the data required. 1

• Quantitative research methodology focuses on measuring and testing numerical data. This approach is good for reaching a large number of people in a short amount of time. This type of research helps in testing the causal relationships between variables, making predictions, and generalizing results to wider populations.
• Qualitative research methodology examines the opinions, behaviors, and experiences of people. It collects and analyzes words and textual data. This research methodology requires fewer participants but is still more time consuming because the time spent per participant is quite large. This method is used in exploratory research where the research problem being investigated is not clearly defined.
• Mixed-method research methodology uses the characteristics of both quantitative and qualitative research methodologies in the same study. This method allows researchers to validate their findings, verify if the results observed using both methods are complementary, and explain any unexpected results obtained from one method by using the other method.

What are the types of sampling designs in research methodology?

Sampling 4 is an important part of a research methodology and involves selecting a representative sample of the population to conduct the study, making statistical inferences about them, and estimating the characteristics of the whole population based on these inferences. There are two types of sampling designs in research methodology—probability and nonprobability.

• Probability sampling

In this type of sampling design, a sample is chosen from a larger population using some form of random selection, that is, every member of the population has an equal chance of being selected. The different types of probability sampling are:

• Systematic —sample members are chosen at regular intervals. It requires selecting a starting point for the sample and sample size determination that can be repeated at regular intervals. This type of sampling method has a predefined range; hence, it is the least time consuming.
• Stratified —researchers divide the population into smaller groups that don’t overlap but represent the entire population. While sampling, these groups can be organized, and then a sample can be drawn from each group separately.
• Cluster —the population is divided into clusters based on demographic parameters like age, sex, location, etc.
• Convenience —selects participants who are most easily accessible to researchers due to geographical proximity, availability at a particular time, etc.
• Purposive —participants are selected at the researcher’s discretion. Researchers consider the purpose of the study and the understanding of the target audience.
• Snowball —already selected participants use their social networks to refer the researcher to other potential participants.
• Quota —while designing the study, the researchers decide how many people with which characteristics to include as participants. The characteristics help in choosing people most likely to provide insights into the subject.

What are data collection methods?

During research, data are collected using various methods depending on the research methodology being followed and the research methods being undertaken. Both qualitative and quantitative research have different data collection methods, as listed below.

Qualitative research 5

• One-on-one interviews: Helps the interviewers understand a respondent’s subjective opinion and experience pertaining to a specific topic or event
• Document study/literature review/record keeping: Researchers’ review of already existing written materials such as archives, annual reports, research articles, guidelines, policy documents, etc.
• Focus groups: Constructive discussions that usually include a small sample of about 6-10 people and a moderator, to understand the participants’ opinion on a given topic.
• Qualitative observation : Researchers collect data using their five senses (sight, smell, touch, taste, and hearing).

Quantitative research 6

• Sampling: The most common type is probability sampling.
• Interviews: Commonly telephonic or done in-person.
• Observations: Structured observations are most commonly used in quantitative research. In this method, researchers make observations about specific behaviors of individuals in a structured setting.
• Document review: Reviewing existing research or documents to collect evidence for supporting the research.
• Surveys and questionnaires. Surveys can be administered both online and offline depending on the requirement and sample size.

Let Paperpal help you write the perfect research methods section. Start now!

What are data analysis methods.

The data collected using the various methods for qualitative and quantitative research need to be analyzed to generate meaningful conclusions. These data analysis methods 7 also differ between quantitative and qualitative research.

Quantitative research involves a deductive method for data analysis where hypotheses are developed at the beginning of the research and precise measurement is required. The methods include statistical analysis applications to analyze numerical data and are grouped into two categories—descriptive and inferential.

Descriptive analysis is used to describe the basic features of different types of data to present it in a way that ensures the patterns become meaningful. The different types of descriptive analysis methods are:

• Measures of frequency (count, percent, frequency)
• Measures of central tendency (mean, median, mode)
• Measures of dispersion or variation (range, variance, standard deviation)
• Measure of position (percentile ranks, quartile ranks)

Inferential analysis is used to make predictions about a larger population based on the analysis of the data collected from a smaller population. This analysis is used to study the relationships between different variables. Some commonly used inferential data analysis methods are:

• Correlation: To understand the relationship between two or more variables.
• Cross-tabulation: Analyze the relationship between multiple variables.
• Regression analysis: Study the impact of independent variables on the dependent variable.
• Frequency tables: To understand the frequency of data.
• Analysis of variance: To test the degree to which two or more variables differ in an experiment.

Qualitative research involves an inductive method for data analysis where hypotheses are developed after data collection. The methods include:

• Content analysis: For analyzing documented information from text and images by determining the presence of certain words or concepts in texts.
• Narrative analysis: For analyzing content obtained from sources such as interviews, field observations, and surveys. The stories and opinions shared by people are used to answer research questions.
• Discourse analysis: For analyzing interactions with people considering the social context, that is, the lifestyle and environment, under which the interaction occurs.
• Grounded theory: Involves hypothesis creation by data collection and analysis to explain why a phenomenon occurred.
• Thematic analysis: To identify important themes or patterns in data and use these to address an issue.

How to choose a research methodology?

Here are some important factors to consider when choosing a research methodology: 8

• Research objectives, aims, and questions —these would help structure the research design.
• Review existing literature to identify any gaps in knowledge.
• Check the statistical requirements —if data-driven or statistical results are needed then quantitative research is the best. If the research questions can be answered based on people’s opinions and perceptions, then qualitative research is most suitable.
• Sample size —sample size can often determine the feasibility of a research methodology. For a large sample, less effort- and time-intensive methods are appropriate.
• Constraints —constraints of time, geography, and resources can help define the appropriate methodology.

Got writer’s block? Kickstart your research paper writing with Paperpal now!

How to write a research methodology .

A research methodology should include the following components: 3,9

• Research design —should be selected based on the research question and the data required. Common research designs include experimental, quasi-experimental, correlational, descriptive, and exploratory.
• Research method —this can be quantitative, qualitative, or mixed-method.
• Reason for selecting a specific methodology —explain why this methodology is the most suitable to answer your research problem.
• Research instruments —explain the research instruments you plan to use, mainly referring to the data collection methods such as interviews, surveys, etc. Here as well, a reason should be mentioned for selecting the particular instrument.
• Sampling —this involves selecting a representative subset of the population being studied.
• Data collection —involves gathering data using several data collection methods, such as surveys, interviews, etc.
• Data analysis —describe the data analysis methods you will use once you’ve collected the data.
• Research limitations —mention any limitations you foresee while conducting your research.
• Validity and reliability —validity helps identify the accuracy and truthfulness of the findings; reliability refers to the consistency and stability of the results over time and across different conditions.
• Ethical considerations —research should be conducted ethically. The considerations include obtaining consent from participants, maintaining confidentiality, and addressing conflicts of interest.

Streamline Your Research Paper Writing Process with Paperpal

The methods section is a critical part of the research papers, allowing researchers to use this to understand your findings and replicate your work when pursuing their own research. However, it is usually also the most difficult section to write. This is where Paperpal can help you overcome the writer’s block and create the first draft in minutes with Paperpal Copilot, its secure generative AI feature suite.  

With Paperpal you can get research advice, write and refine your work, rephrase and verify the writing, and ensure submission readiness, all in one place. Here’s how you can use Paperpal to develop the first draft of your methods section.  

• Generate an outline: Input some details about your research to instantly generate an outline for your methods section 
• Develop the section: Use the outline and suggested sentence templates to expand your ideas and develop the first draft.  
• P araph ras e and trim : Get clear, concise academic text with paraphrasing that conveys your work effectively and word reduction to fix redundancies. 
• Choose the right words: Enhance text by choosing contextual synonyms based on how the words have been used in previously published work.  
• Check and verify text : Make sure the generated text showcases your methods correctly, has all the right citations, and is original and authentic. .   

You can repeat this process to develop each section of your research manuscript, including the title, abstract and keywords. Ready to write your research papers faster, better, and without the stress? Sign up for Paperpal and start writing today!

Frequently Asked Questions

Q1. What are the key components of research methodology?

A1. A good research methodology has the following key components:

• Research design
• Data collection procedures
• Data analysis methods
• Ethical considerations

Q2. Why is ethical consideration important in research methodology?

A2. Ethical consideration is important in research methodology to ensure the readers of the reliability and validity of the study. Researchers must clearly mention the ethical norms and standards followed during the conduct of the research and also mention if the research has been cleared by any institutional board. The following 10 points are the important principles related to ethical considerations: 10

• Participants should not be subjected to harm.
• Respect for the dignity of participants should be prioritized.
• Full consent should be obtained from participants before the study.
• Participants’ privacy should be ensured.
• Confidentiality of the research data should be ensured.
• Anonymity of individuals and organizations participating in the research should be maintained.
• The aims and objectives of the research should not be exaggerated.
• Affiliations, sources of funding, and any possible conflicts of interest should be declared.
• Communication in relation to the research should be honest and transparent.
• Misleading information and biased representation of primary data findings should be avoided.

Q3. What is the difference between methodology and method?

A3. Research methodology is different from a research method, although both terms are often confused. Research methods are the tools used to gather data, while the research methodology provides a framework for how research is planned, conducted, and analyzed. The latter guides researchers in making decisions about the most appropriate methods for their research. Research methods refer to the specific techniques, procedures, and tools used by researchers to collect, analyze, and interpret data, for instance surveys, questionnaires, interviews, etc.

Research methodology is, thus, an integral part of a research study. It helps ensure that you stay on track to meet your research objectives and answer your research questions using the most appropriate data collection and analysis tools based on your research design.

Accelerate your research paper writing with Paperpal. Try for free now!

• Research methodologies. Pfeiffer Library website. Accessed August 15, 2023. https://library.tiffin.edu/researchmethodologies/whatareresearchmethodologies
• Types of research methodology. Eduvoice website. Accessed August 16, 2023. https://eduvoice.in/types-research-methodology/
• The basics of research methodology: A key to quality research. Voxco. Accessed August 16, 2023. https://www.voxco.com/blog/what-is-research-methodology/
• Sampling methods: Types with examples. QuestionPro website. Accessed August 16, 2023. https://www.questionpro.com/blog/types-of-sampling-for-social-research/
• What is qualitative research? Methods, types, approaches, examples. Researcher.Life blog. Accessed August 15, 2023. https://researcher.life/blog/article/what-is-qualitative-research-methods-types-examples/
• What is quantitative research? Definition, methods, types, and examples. Researcher.Life blog. Accessed August 15, 2023. https://researcher.life/blog/article/what-is-quantitative-research-types-and-examples/
• Data analysis in research: Types & methods. QuestionPro website. Accessed August 16, 2023. https://www.questionpro.com/blog/data-analysis-in-research/#Data_analysis_in_qualitative_research
• Factors to consider while choosing the right research methodology. PhD Monster website. Accessed August 17, 2023. https://www.phdmonster.com/factors-to-consider-while-choosing-the-right-research-methodology/
• What is research methodology? Research and writing guides. Accessed August 14, 2023. https://paperpile.com/g/what-is-research-methodology/
• Ethical considerations. Business research methodology website. Accessed August 17, 2023. https://research-methodology.net/research-methodology/ethical-considerations/

Paperpal is a comprehensive AI writing toolkit that helps students and researchers achieve 2x the writing in half the time. It leverages 21+ years of STM experience and insights from millions of research articles to provide in-depth academic writing, language editing, and submission readiness support to help you write better, faster.  

Get accurate academic translations, rewriting support, grammar checks, vocabulary suggestions, and generative AI assistance that delivers human precision at machine speed. Try for free or upgrade to Paperpal Prime starting at US$19 a month to access premium features, including consistency, plagiarism, and 30+ submission readiness checks to help you succeed.  

Experience the future of academic writing – Sign up to Paperpal and start writing for free!  

Related Reads:

• Dangling Modifiers and How to Avoid Them in Your Writing 
• Webinar: How to Use Generative AI Tools Ethically in Your Academic Writing
• Research Outlines: How to Write An Introduction Section in Minutes with Paperpal Copilot
• How to Paraphrase Research Papers Effectively

Language and Grammar Rules for Academic Writing

Climatic vs. climactic: difference and examples, you may also like, how to write a research proposal: (with examples..., apa format: basic guide for researchers, how to choose a dissertation topic, how to write a phd research proposal, how to write an academic paragraph (step-by-step guide), maintaining academic integrity with paperpal’s generative ai writing..., research funding basics: what should a grant proposal..., how to write an abstract in research papers..., how to write dissertation acknowledgements, how to structure an essay.

What Is Research, and Why Do People Do It?

• Open Access
• First Online: 03 December 2022

Cite this chapter

You have full access to this open access chapter

• James Hiebert 6 ,
• Jinfa Cai 7 ,
• Stephen Hwang 7 ,
• Anne K Morris 6 &
• Charles Hohensee 6  

Part of the book series: Research in Mathematics Education ((RME))

20k Accesses

Abstractspiepr Abs1

Every day people do research as they gather information to learn about something of interest. In the scientific world, however, research means something different than simply gathering information. Scientific research is characterized by its careful planning and observing, by its relentless efforts to understand and explain, and by its commitment to learn from everyone else seriously engaged in research. We call this kind of research scientific inquiry and define it as “formulating, testing, and revising hypotheses.” By “hypotheses” we do not mean the hypotheses you encounter in statistics courses. We mean predictions about what you expect to find and rationales for why you made these predictions. Throughout this and the remaining chapters we make clear that the process of scientific inquiry applies to all kinds of research studies and data, both qualitative and quantitative.

You have full access to this open access chapter,  Download chapter PDF

Part I. What Is Research?

Have you ever studied something carefully because you wanted to know more about it? Maybe you wanted to know more about your grandmother’s life when she was younger so you asked her to tell you stories from her childhood, or maybe you wanted to know more about a fertilizer you were about to use in your garden so you read the ingredients on the package and looked them up online. According to the dictionary definition, you were doing research.

Recall your high school assignments asking you to “research” a topic. The assignment likely included consulting a variety of sources that discussed the topic, perhaps including some “original” sources. Often, the teacher referred to your product as a “research paper.”

Were you conducting research when you interviewed your grandmother or wrote high school papers reviewing a particular topic? Our view is that you were engaged in part of the research process, but only a small part. In this book, we reserve the word “research” for what it means in the scientific world, that is, for scientific research or, more pointedly, for scientific inquiry .

Exercise 1.1

Before you read any further, write a definition of what you think scientific inquiry is. Keep it short—Two to three sentences. You will periodically update this definition as you read this chapter and the remainder of the book.

This book is about scientific inquiry—what it is and how to do it. For starters, scientific inquiry is a process, a particular way of finding out about something that involves a number of phases. Each phase of the process constitutes one aspect of scientific inquiry. You are doing scientific inquiry as you engage in each phase, but you have not done scientific inquiry until you complete the full process. Each phase is necessary but not sufficient.

In this chapter, we set the stage by defining scientific inquiry—describing what it is and what it is not—and by discussing what it is good for and why people do it. The remaining chapters build directly on the ideas presented in this chapter.

A first thing to know is that scientific inquiry is not all or nothing. “Scientificness” is a continuum. Inquiries can be more scientific or less scientific. What makes an inquiry more scientific? You might be surprised there is no universally agreed upon answer to this question. None of the descriptors we know of are sufficient by themselves to define scientific inquiry. But all of them give you a way of thinking about some aspects of the process of scientific inquiry. Each one gives you different insights.

Exercise 1.2

As you read about each descriptor below, think about what would make an inquiry more or less scientific. If you think a descriptor is important, use it to revise your definition of scientific inquiry.

Creating an Image of Scientific Inquiry

We will present three descriptors of scientific inquiry. Each provides a different perspective and emphasizes a different aspect of scientific inquiry. We will draw on all three descriptors to compose our definition of scientific inquiry.

Descriptor 1. Experience Carefully Planned in Advance

Sir Ronald Fisher, often called the father of modern statistical design, once referred to research as “experience carefully planned in advance” (1935, p. 8). He said that humans are always learning from experience, from interacting with the world around them. Usually, this learning is haphazard rather than the result of a deliberate process carried out over an extended period of time. Research, Fisher said, was learning from experience, but experience carefully planned in advance.

This phrase can be fully appreciated by looking at each word. The fact that scientific inquiry is based on experience means that it is based on interacting with the world. These interactions could be thought of as the stuff of scientific inquiry. In addition, it is not just any experience that counts. The experience must be carefully planned . The interactions with the world must be conducted with an explicit, describable purpose, and steps must be taken to make the intended learning as likely as possible. This planning is an integral part of scientific inquiry; it is not just a preparation phase. It is one of the things that distinguishes scientific inquiry from many everyday learning experiences. Finally, these steps must be taken beforehand and the purpose of the inquiry must be articulated in advance of the experience. Clearly, scientific inquiry does not happen by accident, by just stumbling into something. Stumbling into something unexpected and interesting can happen while engaged in scientific inquiry, but learning does not depend on it and serendipity does not make the inquiry scientific.

Descriptor 2. Observing Something and Trying to Explain Why It Is the Way It Is

When we were writing this chapter and googled “scientific inquiry,” the first entry was: “Scientific inquiry refers to the diverse ways in which scientists study the natural world and propose explanations based on the evidence derived from their work.” The emphasis is on studying, or observing, and then explaining . This descriptor takes the image of scientific inquiry beyond carefully planned experience and includes explaining what was experienced.

According to the Merriam-Webster dictionary, “explain” means “(a) to make known, (b) to make plain or understandable, (c) to give the reason or cause of, and (d) to show the logical development or relations of” (Merriam-Webster, n.d. ). We will use all these definitions. Taken together, they suggest that to explain an observation means to understand it by finding reasons (or causes) for why it is as it is. In this sense of scientific inquiry, the following are synonyms: explaining why, understanding why, and reasoning about causes and effects. Our image of scientific inquiry now includes planning, observing, and explaining why.

We need to add a final note about this descriptor. We have phrased it in a way that suggests “observing something” means you are observing something in real time—observing the way things are or the way things are changing. This is often true. But, observing could mean observing data that already have been collected, maybe by someone else making the original observations (e.g., secondary analysis of NAEP data or analysis of existing video recordings of classroom instruction). We will address secondary analyses more fully in Chap. 4 . For now, what is important is that the process requires explaining why the data look like they do.

We must note that for us, the term “data” is not limited to numerical or quantitative data such as test scores. Data can also take many nonquantitative forms, including written survey responses, interview transcripts, journal entries, video recordings of students, teachers, and classrooms, text messages, and so forth.

Exercise 1.3

What are the implications of the statement that just “observing” is not enough to count as scientific inquiry? Does this mean that a detailed description of a phenomenon is not scientific inquiry?

Find sources that define research in education that differ with our position, that say description alone, without explanation, counts as scientific research. Identify the precise points where the opinions differ. What are the best arguments for each of the positions? Which do you prefer? Why?

Descriptor 3. Updating Everyone’s Thinking in Response to More and Better Information

This descriptor focuses on a third aspect of scientific inquiry: updating and advancing the field’s understanding of phenomena that are investigated. This descriptor foregrounds a powerful characteristic of scientific inquiry: the reliability (or trustworthiness) of what is learned and the ultimate inevitability of this learning to advance human understanding of phenomena. Humans might choose not to learn from scientific inquiry, but history suggests that scientific inquiry always has the potential to advance understanding and that, eventually, humans take advantage of these new understandings.

Before exploring these bold claims a bit further, note that this descriptor uses “information” in the same way the previous two descriptors used “experience” and “observations.” These are the stuff of scientific inquiry and we will use them often, sometimes interchangeably. Frequently, we will use the term “data” to stand for all these terms.

An overriding goal of scientific inquiry is for everyone to learn from what one scientist does. Much of this book is about the methods you need to use so others have faith in what you report and can learn the same things you learned. This aspect of scientific inquiry has many implications.

One implication is that scientific inquiry is not a private practice. It is a public practice available for others to see and learn from. Notice how different this is from everyday learning. When you happen to learn something from your everyday experience, often only you gain from the experience. The fact that research is a public practice means it is also a social one. It is best conducted by interacting with others along the way: soliciting feedback at each phase, taking opportunities to present work-in-progress, and benefitting from the advice of others.

A second implication is that you, as the researcher, must be committed to sharing what you are doing and what you are learning in an open and transparent way. This allows all phases of your work to be scrutinized and critiqued. This is what gives your work credibility. The reliability or trustworthiness of your findings depends on your colleagues recognizing that you have used all appropriate methods to maximize the chances that your claims are justified by the data.

A third implication of viewing scientific inquiry as a collective enterprise is the reverse of the second—you must be committed to receiving comments from others. You must treat your colleagues as fair and honest critics even though it might sometimes feel otherwise. You must appreciate their job, which is to remain skeptical while scrutinizing what you have done in considerable detail. To provide the best help to you, they must remain skeptical about your conclusions (when, for example, the data are difficult for them to interpret) until you offer a convincing logical argument based on the information you share. A rather harsh but good-to-remember statement of the role of your friendly critics was voiced by Karl Popper, a well-known twentieth century philosopher of science: “. . . if you are interested in the problem which I tried to solve by my tentative assertion, you may help me by criticizing it as severely as you can” (Popper, 1968, p. 27).

A final implication of this third descriptor is that, as someone engaged in scientific inquiry, you have no choice but to update your thinking when the data support a different conclusion. This applies to your own data as well as to those of others. When data clearly point to a specific claim, even one that is quite different than you expected, you must reconsider your position. If the outcome is replicated multiple times, you need to adjust your thinking accordingly. Scientific inquiry does not let you pick and choose which data to believe; it mandates that everyone update their thinking when the data warrant an update.

Doing Scientific Inquiry

We define scientific inquiry in an operational sense—what does it mean to do scientific inquiry? What kind of process would satisfy all three descriptors: carefully planning an experience in advance; observing and trying to explain what you see; and, contributing to updating everyone’s thinking about an important phenomenon?

We define scientific inquiry as formulating , testing , and revising hypotheses about phenomena of interest.

Of course, we are not the only ones who define it in this way. The definition for the scientific method posted by the editors of Britannica is: “a researcher develops a hypothesis, tests it through various means, and then modifies the hypothesis on the basis of the outcome of the tests and experiments” (Britannica, n.d. ).

Notice how defining scientific inquiry this way satisfies each of the descriptors. “Carefully planning an experience in advance” is exactly what happens when formulating a hypothesis about a phenomenon of interest and thinking about how to test it. “ Observing a phenomenon” occurs when testing a hypothesis, and “ explaining ” what is found is required when revising a hypothesis based on the data. Finally, “updating everyone’s thinking” comes from comparing publicly the original with the revised hypothesis.

Doing scientific inquiry, as we have defined it, underscores the value of accumulating knowledge rather than generating random bits of knowledge. Formulating, testing, and revising hypotheses is an ongoing process, with each revised hypothesis begging for another test, whether by the same researcher or by new researchers. The editors of Britannica signaled this cyclic process by adding the following phrase to their definition of the scientific method: “The modified hypothesis is then retested, further modified, and tested again.” Scientific inquiry creates a process that encourages each study to build on the studies that have gone before. Through collective engagement in this process of building study on top of study, the scientific community works together to update its thinking.

Before exploring more fully the meaning of “formulating, testing, and revising hypotheses,” we need to acknowledge that this is not the only way researchers define research. Some researchers prefer a less formal definition, one that includes more serendipity, less planning, less explanation. You might have come across more open definitions such as “research is finding out about something.” We prefer the tighter hypothesis formulation, testing, and revision definition because we believe it provides a single, coherent map for conducting research that addresses many of the thorny problems educational researchers encounter. We believe it is the most useful orientation toward research and the most helpful to learn as a beginning researcher.

A final clarification of our definition is that it applies equally to qualitative and quantitative research. This is a familiar distinction in education that has generated much discussion. You might think our definition favors quantitative methods over qualitative methods because the language of hypothesis formulation and testing is often associated with quantitative methods. In fact, we do not favor one method over another. In Chap. 4 , we will illustrate how our definition fits research using a range of quantitative and qualitative methods.

Exercise 1.4

Look for ways to extend what the field knows in an area that has already received attention by other researchers. Specifically, you can search for a program of research carried out by more experienced researchers that has some revised hypotheses that remain untested. Identify a revised hypothesis that you might like to test.

Unpacking the Terms Formulating, Testing, and Revising Hypotheses

To get a full sense of the definition of scientific inquiry we will use throughout this book, it is helpful to spend a little time with each of the key terms.

We first want to make clear that we use the term “hypothesis” as it is defined in most dictionaries and as it used in many scientific fields rather than as it is usually defined in educational statistics courses. By “hypothesis,” we do not mean a null hypothesis that is accepted or rejected by statistical analysis. Rather, we use “hypothesis” in the sense conveyed by the following definitions: “An idea or explanation for something that is based on known facts but has not yet been proved” (Cambridge University Press, n.d. ), and “An unproved theory, proposition, or supposition, tentatively accepted to explain certain facts and to provide a basis for further investigation or argument” (Agnes & Guralnik, 2008 ).

We distinguish two parts to “hypotheses.” Hypotheses consist of predictions and rationales . Predictions are statements about what you expect to find when you inquire about something. Rationales are explanations for why you made the predictions you did, why you believe your predictions are correct. So, for us “formulating hypotheses” means making explicit predictions and developing rationales for the predictions.

“Testing hypotheses” means making observations that allow you to assess in what ways your predictions were correct and in what ways they were incorrect. In education research, it is rarely useful to think of your predictions as either right or wrong. Because of the complexity of most issues you will investigate, most predictions will be right in some ways and wrong in others.

By studying the observations you make (data you collect) to test your hypotheses, you can revise your hypotheses to better align with the observations. This means revising your predictions plus revising your rationales to justify your adjusted predictions. Even though you might not run another test, formulating revised hypotheses is an essential part of conducting a research study. Comparing your original and revised hypotheses informs everyone of what you learned by conducting your study. In addition, a revised hypothesis sets the stage for you or someone else to extend your study and accumulate more knowledge of the phenomenon.

We should note that not everyone makes a clear distinction between predictions and rationales as two aspects of hypotheses. In fact, common, non-scientific uses of the word “hypothesis” may limit it to only a prediction or only an explanation (or rationale). We choose to explicitly include both prediction and rationale in our definition of hypothesis, not because we assert this should be the universal definition, but because we want to foreground the importance of both parts acting in concert. Using “hypothesis” to represent both prediction and rationale could hide the two aspects, but we make them explicit because they provide different kinds of information. It is usually easier to make predictions than develop rationales because predictions can be guesses, hunches, or gut feelings about which you have little confidence. Developing a compelling rationale requires careful thought plus reading what other researchers have found plus talking with your colleagues. Often, while you are developing your rationale you will find good reasons to change your predictions. Developing good rationales is the engine that drives scientific inquiry. Rationales are essentially descriptions of how much you know about the phenomenon you are studying. Throughout this guide, we will elaborate on how developing good rationales drives scientific inquiry. For now, we simply note that it can sharpen your predictions and help you to interpret your data as you test your hypotheses.

Hypotheses in education research take a variety of forms or types. This is because there are a variety of phenomena that can be investigated. Investigating educational phenomena is sometimes best done using qualitative methods, sometimes using quantitative methods, and most often using mixed methods (e.g., Hay, 2016 ; Weis et al. 2019a ; Weisner, 2005 ). This means that, given our definition, hypotheses are equally applicable to qualitative and quantitative investigations.

Hypotheses take different forms when they are used to investigate different kinds of phenomena. Two very different activities in education could be labeled conducting experiments and descriptions. In an experiment, a hypothesis makes a prediction about anticipated changes, say the changes that occur when a treatment or intervention is applied. You might investigate how students’ thinking changes during a particular kind of instruction.

A second type of hypothesis, relevant for descriptive research, makes a prediction about what you will find when you investigate and describe the nature of a situation. The goal is to understand a situation as it exists rather than to understand a change from one situation to another. In this case, your prediction is what you expect to observe. Your rationale is the set of reasons for making this prediction; it is your current explanation for why the situation will look like it does.

You will probably read, if you have not already, that some researchers say you do not need a prediction to conduct a descriptive study. We will discuss this point of view in Chap. 2 . For now, we simply claim that scientific inquiry, as we have defined it, applies to all kinds of research studies. Descriptive studies, like others, not only benefit from formulating, testing, and revising hypotheses, but also need hypothesis formulating, testing, and revising.

One reason we define research as formulating, testing, and revising hypotheses is that if you think of research in this way you are less likely to go wrong. It is a useful guide for the entire process, as we will describe in detail in the chapters ahead. For example, as you build the rationale for your predictions, you are constructing the theoretical framework for your study (Chap. 3 ). As you work out the methods you will use to test your hypothesis, every decision you make will be based on asking, “Will this help me formulate or test or revise my hypothesis?” (Chap. 4 ). As you interpret the results of testing your predictions, you will compare them to what you predicted and examine the differences, focusing on how you must revise your hypotheses (Chap. 5 ). By anchoring the process to formulating, testing, and revising hypotheses, you will make smart decisions that yield a coherent and well-designed study.

Exercise 1.5

Compare the concept of formulating, testing, and revising hypotheses with the descriptions of scientific inquiry contained in Scientific Research in Education (NRC, 2002 ). How are they similar or different?

Exercise 1.6

Provide an example to illustrate and emphasize the differences between everyday learning/thinking and scientific inquiry.

Learning from Doing Scientific Inquiry

We noted earlier that a measure of what you have learned by conducting a research study is found in the differences between your original hypothesis and your revised hypothesis based on the data you collected to test your hypothesis. We will elaborate this statement in later chapters, but we preview our argument here.

Even before collecting data, scientific inquiry requires cycles of making a prediction, developing a rationale, refining your predictions, reading and studying more to strengthen your rationale, refining your predictions again, and so forth. And, even if you have run through several such cycles, you still will likely find that when you test your prediction you will be partly right and partly wrong. The results will support some parts of your predictions but not others, or the results will “kind of” support your predictions. A critical part of scientific inquiry is making sense of your results by interpreting them against your predictions. Carefully describing what aspects of your data supported your predictions, what aspects did not, and what data fell outside of any predictions is not an easy task, but you cannot learn from your study without doing this analysis.

Analyzing the matches and mismatches between your predictions and your data allows you to formulate different rationales that would have accounted for more of the data. The best revised rationale is the one that accounts for the most data. Once you have revised your rationales, you can think about the predictions they best justify or explain. It is by comparing your original rationales to your new rationales that you can sort out what you learned from your study.

Suppose your study was an experiment. Maybe you were investigating the effects of a new instructional intervention on students’ learning. Your original rationale was your explanation for why the intervention would change the learning outcomes in a particular way. Your revised rationale explained why the changes that you observed occurred like they did and why your revised predictions are better. Maybe your original rationale focused on the potential of the activities if they were implemented in ideal ways and your revised rationale included the factors that are likely to affect how teachers implement them. By comparing the before and after rationales, you are describing what you learned—what you can explain now that you could not before. Another way of saying this is that you are describing how much more you understand now than before you conducted your study.

Revised predictions based on carefully planned and collected data usually exhibit some of the following features compared with the originals: more precision, more completeness, and broader scope. Revised rationales have more explanatory power and become more complete, more aligned with the new predictions, sharper, and overall more convincing.

Part II. Why Do Educators Do Research?

Doing scientific inquiry is a lot of work. Each phase of the process takes time, and you will often cycle back to improve earlier phases as you engage in later phases. Because of the significant effort required, you should make sure your study is worth it. So, from the beginning, you should think about the purpose of your study. Why do you want to do it? And, because research is a social practice, you should also think about whether the results of your study are likely to be important and significant to the education community.

If you are doing research in the way we have described—as scientific inquiry—then one purpose of your study is to understand , not just to describe or evaluate or report. As we noted earlier, when you formulate hypotheses, you are developing rationales that explain why things might be like they are. In our view, trying to understand and explain is what separates research from other kinds of activities, like evaluating or describing.

One reason understanding is so important is that it allows researchers to see how or why something works like it does. When you see how something works, you are better able to predict how it might work in other contexts, under other conditions. And, because conditions, or contextual factors, matter a lot in education, gaining insights into applying your findings to other contexts increases the contributions of your work and its importance to the broader education community.

Consequently, the purposes of research studies in education often include the more specific aim of identifying and understanding the conditions under which the phenomena being studied work like the observations suggest. A classic example of this kind of study in mathematics education was reported by William Brownell and Harold Moser in 1949 . They were trying to establish which method of subtracting whole numbers could be taught most effectively—the regrouping method or the equal additions method. However, they realized that effectiveness might depend on the conditions under which the methods were taught—“meaningfully” versus “mechanically.” So, they designed a study that crossed the two instructional approaches with the two different methods (regrouping and equal additions). Among other results, they found that these conditions did matter. The regrouping method was more effective under the meaningful condition than the mechanical condition, but the same was not true for the equal additions algorithm.

What do education researchers want to understand? In our view, the ultimate goal of education is to offer all students the best possible learning opportunities. So, we believe the ultimate purpose of scientific inquiry in education is to develop understanding that supports the improvement of learning opportunities for all students. We say “ultimate” because there are lots of issues that must be understood to improve learning opportunities for all students. Hypotheses about many aspects of education are connected, ultimately, to students’ learning. For example, formulating and testing a hypothesis that preservice teachers need to engage in particular kinds of activities in their coursework in order to teach particular topics well is, ultimately, connected to improving students’ learning opportunities. So is hypothesizing that school districts often devote relatively few resources to instructional leadership training or hypothesizing that positioning mathematics as a tool students can use to combat social injustice can help students see the relevance of mathematics to their lives.

We do not exclude the importance of research on educational issues more removed from improving students’ learning opportunities, but we do think the argument for their importance will be more difficult to make. If there is no way to imagine a connection between your hypothesis and improving learning opportunities for students, even a distant connection, we recommend you reconsider whether it is an important hypothesis within the education community.

Notice that we said the ultimate goal of education is to offer all students the best possible learning opportunities. For too long, educators have been satisfied with a goal of offering rich learning opportunities for lots of students, sometimes even for just the majority of students, but not necessarily for all students. Evaluations of success often are based on outcomes that show high averages. In other words, if many students have learned something, or even a smaller number have learned a lot, educators may have been satisfied. The problem is that there is usually a pattern in the groups of students who receive lower quality opportunities—students of color and students who live in poor areas, urban and rural. This is not acceptable. Consequently, we emphasize the premise that the purpose of education research is to offer rich learning opportunities to all students.

One way to make sure you will be able to convince others of the importance of your study is to consider investigating some aspect of teachers’ shared instructional problems. Historically, researchers in education have set their own research agendas, regardless of the problems teachers are facing in schools. It is increasingly recognized that teachers have had trouble applying to their own classrooms what researchers find. To address this problem, a researcher could partner with a teacher—better yet, a small group of teachers—and talk with them about instructional problems they all share. These discussions can create a rich pool of problems researchers can consider. If researchers pursued one of these problems (preferably alongside teachers), the connection to improving learning opportunities for all students could be direct and immediate. “Grounding a research question in instructional problems that are experienced across multiple teachers’ classrooms helps to ensure that the answer to the question will be of sufficient scope to be relevant and significant beyond the local context” (Cai et al., 2019b , p. 115).

As a beginning researcher, determining the relevance and importance of a research problem is especially challenging. We recommend talking with advisors, other experienced researchers, and peers to test the educational importance of possible research problems and topics of study. You will also learn much more about the issue of research importance when you read Chap. 5 .

Exercise 1.7

Identify a problem in education that is closely connected to improving learning opportunities and a problem that has a less close connection. For each problem, write a brief argument (like a logical sequence of if-then statements) that connects the problem to all students’ learning opportunities.

Part III. Conducting Research as a Practice of Failing Productively

Scientific inquiry involves formulating hypotheses about phenomena that are not fully understood—by you or anyone else. Even if you are able to inform your hypotheses with lots of knowledge that has already been accumulated, you are likely to find that your prediction is not entirely accurate. This is normal. Remember, scientific inquiry is a process of constantly updating your thinking. More and better information means revising your thinking, again, and again, and again. Because you never fully understand a complicated phenomenon and your hypotheses never produce completely accurate predictions, it is easy to believe you are somehow failing.

The trick is to fail upward, to fail to predict accurately in ways that inform your next hypothesis so you can make a better prediction. Some of the best-known researchers in education have been open and honest about the many times their predictions were wrong and, based on the results of their studies and those of others, they continuously updated their thinking and changed their hypotheses.

A striking example of publicly revising (actually reversing) hypotheses due to incorrect predictions is found in the work of Lee J. Cronbach, one of the most distinguished educational psychologists of the twentieth century. In 1955, Cronbach delivered his presidential address to the American Psychological Association. Titling it “Two Disciplines of Scientific Psychology,” Cronbach proposed a rapprochement between two research approaches—correlational studies that focused on individual differences and experimental studies that focused on instructional treatments controlling for individual differences. (We will examine different research approaches in Chap. 4 ). If these approaches could be brought together, reasoned Cronbach ( 1957 ), researchers could find interactions between individual characteristics and treatments (aptitude-treatment interactions or ATIs), fitting the best treatments to different individuals.

In 1975, after years of research by many researchers looking for ATIs, Cronbach acknowledged the evidence for simple, useful ATIs had not been found. Even when trying to find interactions between a few variables that could provide instructional guidance, the analysis, said Cronbach, creates “a hall of mirrors that extends to infinity, tormenting even the boldest investigators and defeating even ambitious designs” (Cronbach, 1975 , p. 119).

As he was reflecting back on his work, Cronbach ( 1986 ) recommended moving away from documenting instructional effects through statistical inference (an approach he had championed for much of his career) and toward approaches that probe the reasons for these effects, approaches that provide a “full account of events in a time, place, and context” (Cronbach, 1986 , p. 104). This is a remarkable change in hypotheses, a change based on data and made fully transparent. Cronbach understood the value of failing productively.

Closer to home, in a less dramatic example, one of us began a line of scientific inquiry into how to prepare elementary preservice teachers to teach early algebra. Teaching early algebra meant engaging elementary students in early forms of algebraic reasoning. Such reasoning should help them transition from arithmetic to algebra. To begin this line of inquiry, a set of activities for preservice teachers were developed. Even though the activities were based on well-supported hypotheses, they largely failed to engage preservice teachers as predicted because of unanticipated challenges the preservice teachers faced. To capitalize on this failure, follow-up studies were conducted, first to better understand elementary preservice teachers’ challenges with preparing to teach early algebra, and then to better support preservice teachers in navigating these challenges. In this example, the initial failure was a necessary step in the researchers’ scientific inquiry and furthered the researchers’ understanding of this issue.

We present another example of failing productively in Chap. 2 . That example emerges from recounting the history of a well-known research program in mathematics education.

Making mistakes is an inherent part of doing scientific research. Conducting a study is rarely a smooth path from beginning to end. We recommend that you keep the following things in mind as you begin a career of conducting research in education.

First, do not get discouraged when you make mistakes; do not fall into the trap of feeling like you are not capable of doing research because you make too many errors.

Second, learn from your mistakes. Do not ignore your mistakes or treat them as errors that you simply need to forget and move past. Mistakes are rich sites for learning—in research just as in other fields of study.

Third, by reflecting on your mistakes, you can learn to make better mistakes, mistakes that inform you about a productive next step. You will not be able to eliminate your mistakes, but you can set a goal of making better and better mistakes.

Exercise 1.8

How does scientific inquiry differ from everyday learning in giving you the tools to fail upward? You may find helpful perspectives on this question in other resources on science and scientific inquiry (e.g., Failure: Why Science is So Successful by Firestein, 2015).

Exercise 1.9

Use what you have learned in this chapter to write a new definition of scientific inquiry. Compare this definition with the one you wrote before reading this chapter. If you are reading this book as part of a course, compare your definition with your colleagues’ definitions. Develop a consensus definition with everyone in the course.

Part IV. Preview of Chap. 2

Now that you have a good idea of what research is, at least of what we believe research is, the next step is to think about how to actually begin doing research. This means how to begin formulating, testing, and revising hypotheses. As for all phases of scientific inquiry, there are lots of things to think about. Because it is critical to start well, we devote Chap. 2 to getting started with formulating hypotheses.

Agnes, M., & Guralnik, D. B. (Eds.). (2008). Hypothesis. In Webster’s new world college dictionary (4th ed.). Wiley.

Google Scholar  

Britannica. (n.d.). Scientific method. In Encyclopaedia Britannica . Retrieved July 15, 2022 from https://www.britannica.com/science/scientific-method

Brownell, W. A., & Moser, H. E. (1949). Meaningful vs. mechanical learning: A study in grade III subtraction . Duke University Press..

Cai, J., Morris, A., Hohensee, C., Hwang, S., Robison, V., Cirillo, M., Kramer, S. L., & Hiebert, J. (2019b). Posing significant research questions. Journal for Research in Mathematics Education, 50 (2), 114–120. https://doi.org/10.5951/jresematheduc.50.2.0114

Article   Google Scholar  

Cambridge University Press. (n.d.). Hypothesis. In Cambridge dictionary . Retrieved July 15, 2022 from https://dictionary.cambridge.org/us/dictionary/english/hypothesis

Cronbach, J. L. (1957). The two disciplines of scientific psychology. American Psychologist, 12 , 671–684.

Cronbach, L. J. (1975). Beyond the two disciplines of scientific psychology. American Psychologist, 30 , 116–127.

Cronbach, L. J. (1986). Social inquiry by and for earthlings. In D. W. Fiske & R. A. Shweder (Eds.), Metatheory in social science: Pluralisms and subjectivities (pp. 83–107). University of Chicago Press.

Hay, C. M. (Ed.). (2016). Methods that matter: Integrating mixed methods for more effective social science research . University of Chicago Press.

Merriam-Webster. (n.d.). Explain. In Merriam-Webster.com dictionary . Retrieved July 15, 2022, from https://www.merriam-webster.com/dictionary/explain

National Research Council. (2002). Scientific research in education . National Academy Press.

Weis, L., Eisenhart, M., Duncan, G. J., Albro, E., Bueschel, A. C., Cobb, P., Eccles, J., Mendenhall, R., Moss, P., Penuel, W., Ream, R. K., Rumbaut, R. G., Sloane, F., Weisner, T. S., & Wilson, J. (2019a). Mixed methods for studies that address broad and enduring issues in education research. Teachers College Record, 121 , 100307.

Weisner, T. S. (Ed.). (2005). Discovering successful pathways in children’s development: Mixed methods in the study of childhood and family life . University of Chicago Press.

Download references

Author information

Authors and affiliations.

School of Education, University of Delaware, Newark, DE, USA

James Hiebert, Anne K Morris & Charles Hohensee

Department of Mathematical Sciences, University of Delaware, Newark, DE, USA

Jinfa Cai & Stephen Hwang

You can also search for this author in PubMed   Google Scholar

Rights and permissions

Open Access This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License ( http://creativecommons.org/licenses/by/4.0/ ), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

Reprints and permissions

Copyright information

© 2023 The Author(s)

About this chapter

Hiebert, J., Cai, J., Hwang, S., Morris, A.K., Hohensee, C. (2023). What Is Research, and Why Do People Do It?. In: Doing Research: A New Researcher’s Guide. Research in Mathematics Education. Springer, Cham. https://doi.org/10.1007/978-3-031-19078-0_1

Download citation

DOI : https://doi.org/10.1007/978-3-031-19078-0_1

Published : 03 December 2022

Publisher Name : Springer, Cham

Print ISBN : 978-3-031-19077-3

Online ISBN : 978-3-031-19078-0

eBook Packages : Education Education (R0)

Share this chapter

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Publish with us

Policies and ethics

• Find a journal
• Track your research

In order to continue enjoying our site, we ask that you confirm your identity as a human. Thank you very much for your cooperation.

• Harvard Library
• Research Guides
• Faculty of Arts & Sciences Libraries

Finding and Reading Journal Articles

• Journal Articles: Why You Use Them

Why are articles so important to research?

• Subject Databases: Organizing Research Conversations
• Databases We Recommend For You
• Sources in the Disciplines
• Reading in the Disciplines

Journal articles are the academic's stock in trade, t he basic means of communicating research findings to an audience of one’s peers. That holds true across the disciplinary spectrum, so no matter where you land as a concentrator, you can expect to rely on them heavily. 

Regardless of the discipline, moreover,  journal articles perform an important knowledge-updating function .

Textbooks and handbooks and manuals will have a secondary function for chemists and physicists and biologists, of course. But in the sciences, articles are the standard and  preferred publication form. 

In the social sciences and humanities , where knowledge develops a little less rapidly or is driven less by issues of time-sensitivity , journal articles and books are more often used together.

Not all important and influential ideas warrant book-length studies, and some inquiry is just better suited to the size and scope and concentrated discussion that the article format offers.

Journal articles sometimes just present the most  appropriate  solution for communicating findings or making a convincing argument.  A 20-page article may perfectly fit a researcher's needs.  Sustaining that argument for 200 pages might be unnecessary -- or impossible.

The quality of a research article and the legitimacy of its findings are verified by other scholars, prior to publication, through a rigorous evaluation method called peer-review . This seal of approval by other scholars doesn't mean that an article is the best, or truest, or last word on a topic. If that were the case, research on lots of things would cease. Peer review simply means other experts believe the methods, the evidence, the conclusions of an article have met important standards of legitimacy, reliability, and intellectual honesty.

Searching the journal literature is part of being a responsible researcher at any level: professor, grad student, concentrator, first-year. Knowing why academic articles matter will help you make good decisions about what you find -- and what you choose to rely on in your work.

Think of journal articles as the way you tap into the ongoing scholarly conversation , as a way of testing the currency of  a finding, analysis, or argumentative position, and a way of bolstering the authority (or plausibility) of explanations you'll offer in the papers and projects you'll complete at Harvard. 

• Next: Subject Databases: Organizing Research Conversations >>

Except where otherwise noted, this work is subject to a Creative Commons Attribution 4.0 International License , which allows anyone to share and adapt our material as long as proper attribution is given. For details and exceptions, see the Harvard Library Copyright Policy ©2021 Presidents and Fellows of Harvard College.

• Open access
• Published: 02 August 2024

“I didn’t even wonder why I was on the floor” – mixed methods exploration of stroke awareness and help-seeking behaviour at stroke symptom onset

• Loraine Busetto   ORCID: orcid.org/0000-0002-9228-7875 1 , 2 ,
• Christina Stang 1 ,
• Franziska Herzog   ORCID: orcid.org/0000-0002-2504-294X 3 ,
• Melek Sert 1 ,
• Johanna Hoffmann 1 ,
• Jan Purrucker   ORCID: orcid.org/0000-0003-2978-4972 1 ,
• Fatih Seker   ORCID: orcid.org/0000-0001-6072-0438 4 ,
• Martin Bendszus   ORCID: orcid.org/0000-0002-9094-6769 4 ,
• Wolfgang Wick   ORCID: orcid.org/0000-0002-6171-634X 1 ,
• Matthias Ungerer 1   na1 &
• Christoph Gumbinger   ORCID: orcid.org/0000-0002-6137-1169 1   na1  

BMC Health Services Research volume  24 , Article number:  880 ( 2024 ) Cite this article

Metrics details

Introduction

To better target stroke awareness efforts (pre and post first stroke) and thereby decrease the time window for help-seeking, this study aims to assess quantitatively whether stroke awareness is associated with appropriate help-seeking at symptom onset, and to investigate qualitatively why this may (not) be the case.

This study conducted in a German regional stroke network comprises a convergent quantitative-dominant, hypothesis-driven mixed methods design including 462 quantitative patient questionnaires combined with qualitative interviews with 28 patients and seven relatives. Quantitative associations were identified using Pearson’s correlation analysis. Open coding was performed on interview transcripts before the quantitative results were used to further focus qualitative analysis. Joint display analysis was conducted to mix data strands. Cooperation with the Patient Council of the Department of Neurology ensured patient involvement in the study.

Our hypothesis that stroke awareness would be associated with appropriate help-seeking behaviour at stroke symptom onset was partially supported by the quantitative data, i.e. showing associations between some dimensions of stroke awareness and appropriate help-seeking, but not others. For example, knowing stroke symptoms is correlated with recognising one’s own symptoms as stroke ( r  = 0.101; p  = 0.030*; N  = 459) but not with no hesitation before calling help ( r  = 0.003; p  = 0.941; N  = 457). A previous stroke also makes it more likely to recognise one’s own symptoms as stroke ( r  = 0.114; p  = 0.015*; N  = 459), but not to be transported by emergency ambulance ( r  = 0.08; p  = 0.872; N  = 462) or to arrive at the hospital on time ( r  = 0.02; p  = 0.677; N  = 459). Qualitative results showed concordance, discordance or provided potential explanations for quantitative findings. For example, qualitative data showed processes of denial on the part of patients and the important role of relatives in initiating appropriate help-seeking behaviour on patients’ behalf.

Conclusions

Our study provides insights into the complexities of the decision-making process at stroke symptom onset. As our findings suggest processes of denial and inabilities to translate abstract disease knowledge into correct actions, we recommend to address relatives as potential saviours of loved ones, increased use of specific situational examples (e.g. lying on the bathroom floor) and the involvement of patient representatives in the preparation of informational resources and campaigns. Future research should include mixed methods research from one sample and more attention to potential reporting inconsistencies.

Peer Review reports

Acute ischemic stroke is one of the leading causes of death and acquired disability worldwide. Acute treatment options include stroke unit treatment, intravenous thrombolysis (IVT) and endovascular thrombectomy (EVT), all with strongly time-dependent treatment effects. While institutional and regulatory efforts have addressed the time frames from emergency call to treatment initiation [ 1 , 2 , 3 , 4 , 5 ], the time from symptom onset to first help-seeking is largely determined by decisions made by individual medical laypeople. Efforts for raising awareness of stroke are usually based on the assumption that increased stroke awareness will contribute to an increased likelihood of patients behaving correctly, and thereby an increased likelihood of timely treatment access.

However, a positive effect of these efforts has not been shown consistently [ 4 , 6 , 7 , 8 , 9 , 10 , 11 , 12 , 13 , 14 , 15 ]. Moreover, evaluations use a wide range of outcome measures, including knowledge of risk factors, symptoms and treatments [ 9 , 10 , 11 , 12 , 13 , 14 ], action taken [ 9 ], emergency department visits [ 8 ], thrombolysis rates [ 8 ], initiation of reperfusion therapy [ 15 ] or functional outcome at discharge [ 7 ] – all capturing different aspects of how well a person is informed about stroke, knows what to do or actually implements the recommended action. This means that it is not clear to what extent knowledge of stroke symptoms can actually predict good health outcomes, or whether timely presentation to emergency services can really be attributed to higher stroke awareness. Several qualitative studies have pointed out the complexity of the decision-making-process, which in addition to patient-specific factors, is also subject to outside influences [ 16 , 17 , 18 ].

This study aims to (1) assess quantitatively whether different aspects of stroke awareness were associated with appropriate help-seeking behaviour at stroke symptom onset, and to (2) investigate qualitatively why this may (not) have been the case. We expect our results to help inform outreach campaigns and awareness efforts to better reach its target groups and intended goals for improved stroke outcomes.

Mixed methods research design

This study used a convergent quantitative-dominant, hypothesis-driven mixed methods design including patient questionnaires and semi-structured interviews with patients and relatives (Fig.  1 ). The theoretical framework is informed by the COMIC Model, developed for the evaluation of complex care interventions, such as stroke care provision. It focuses on aspects beyond the medical (such as patient-centeredness) and specifically considers the context in which an intervention is implemented, as needed for the current study [ 19 ]. The study was conducted in a German regional stroke network (FAST; www.fast-schlaganfall.de ). Ethics approval was obtained from the Medical Faculty of Heidelberg University (S-306/2016; S-682/2017). All study participants provided written informed consent. We report our findings in line with applicable standards [ 20 ].

The mixed methods integration strategy was to compare (especially regarding patient data) and to expand (especially regarding relative data) [ 21 ]. The mixed methods research data inventory [ 21 ] is shown in Table  1 . We hypothesised that stroke awareness would be associated with appropriate help-seeking behaviour at stroke symptom onset. We defined “stroke awareness” as having information about stroke before the stroke occurred using the concepts “knowing stroke symptoms”, “familiarity with information campaigns”, “having experienced one or more previous strokes” and “knowing other stroke patients” or “having discussed stroke symptoms with other stroke patients”. We defined appropriate help-seeking as responding to a suspected stroke by seeking the appropriate help immediately upon symptom onset, measured using the concepts “recognising the symptoms as stroke”, “no hesitation before calling for help”, “transportation to hospital by emergency ambulance”, and “arrival at hospital within the 4.5 h therapeutic time window”. For mixing of data strands, we conducted a joint display analysis to assess for “fit” and draw meta-inferences according to the categories of concordance, expansion, complementarity or discordance of quantitative and qualitative findings which are addressed in the Discussion [ 22 , 23 ].

Data collection and analysis

Quantitative and qualitative data were collected separately. The quantitative data collection consisted of a questionnaire for patients admitted with acute stroke at an urban university hospital or a rural primary stroke center. Patients were recruited consecutively over a period of 6 months, starting in January 2017. The questionnaires were completed on the day after admission by the patients and their treating physician. Quantitative data were analysed using standard descriptive statistics. Associations were identified using Pearson’s correlation analysis. More detailed information on the quantitative questionnaire is published elsewhere [ 26 ].

For the qualitative data collection, semi-structured interviews were conducted with stroke patients and their relatives. A purposive sampling strategy was used to include interviewees with different stroke pathway experiences such as different transfer modes (helicopter or ambulance), admission at more or less specialised hospitals as well as different health outcomes. Recruitment and data collection place from May to July 2018 at Heidelberg University hospital and from July to September 2019 at two primary stroke centers. Interviews were conducted in German, approximately one month after stroke. The interview guide was piloted in advance with members of a regional stroke self-help group. For qualitative intra-method analysis, interview transcripts were coded by at least two researchers using MaxQDA-software (2018, VERBI, Berlin, Germany). After coding of all transcripts was completed, the quantitative results were used to focus the qualitative analysis on the aspects of stroke awareness and help-seeking behaviour as outlined for the questionnaires.

More detailed information on the respective methods of data collection and intra-method data analysis are shown in Table  2 .

Patient and public involvement

A stroke self-help group consulted on the qualitative design and helped pilot the interviews. Stakeholder validation of preliminary results was conducted with the Patient Council of the Department of Neurology on 17 November 2020, which showed agreement with findings outside the study sample and provided insights into discordance between quantitative and qualitative findings (see Discussion).

Baseline characteristics (questionnaires)

In total, 462 patients were included in the quantitative analysis. Median age was 71.5 years (IQR: 60–79) and 47.4% of patients were female. Median premorbid Rankin scale (pmRS) was 0 (0–2). Other baseline characeristics including primary admission hospital, health status and risk factors are reported in Table  3 .

Summary of main findings

Patient and relative characteristics (interviews)

We conducted 35 interviews, including 28 patient interviews and seven relative interviews. In 8 of the patient interviews, a relative was also present and occasionally participated. The interviews lasted between 20 and 82 min (median: 47 min, IQR: 32–59). Eleven patients were female (39%), and median age was 66 years (IQR: 60–78). Most patients had no prestroke disabilities as indicated by a pmRS of 0 (IQR 0–1). The mean NIHSS at admission was 8.7 (SD 7.7), indicating that most patients had not experienced a severe stroke. The primary admission hospital of eleven patients was an EVT-capable hospital; whereas the others were admitted at an IVT-capable hospital. Mean NIHSS at discharge was 2.6 (SD 2.6) while median mRS at discharge was 2 (IQR 1–3), showing a relatively good outcome after stroke. Of the seven relatives, six were female, and median age was 58, ranging from 23 to 72 years.

Help-seeking and stroke awareness

Main findings are summarised in an integrated visual display in Fig.  2 . This includes statistical results as well as qualitative interview quotes.

Knowing stroke symptoms

Questionnaires showed a positive correlation between knowing stroke symptoms and recognising symptoms as stroke ( N  = 459; r  = 0.101; p  = 0.030*) and arrival at hospital within 4.5 h ( N  = 459; r  = 0.093; p  = 0.046*), but not with no hesitation before calling for help ( N  = 457; r  = 0.003; p  = 0,941) and transportation by emergency ambulance ( N  = 462; r  = 0.014; p  = 0.764).

Five patient interviewees reported immediately knowing or strongly suspecting that they experienced a stroke. One recognized the stroke when he felt a sudden, strong stab of pain in the head and could not hold a water bottle. The other patient recognised the stroke when she saw her drooping cheek in the mirror. Of the five patients who recognized their stroke, four patients immediately called an ambulance or told their spouse to do so. The fifth patient was alone at home and could not physically react appropriately.

In contrast, eight patients who consciously experienced their symptoms stated that they had no idea it was a stroke, e.g. specifying that “[it] was the last thing [he] would have thought of” (Patient, Interview 12). These patients reported slurred speech, not being able to speak or answer questions, not being able to sit/stand/get up or walk (properly), not being able to use their leg(s), lying on the floor, and not being able to use their arm or hand (including dropping things). Another patient specified that even though she was aware of common stroke symptoms, she did not recognise them in her own case.

I know this thing , that you hold up both arms. But for myself , it would never have crossed my mind . Patient , Interview 1

She and another patient emphasised that even though they consciously experienced one or more symptoms, they did not feel that something was wrong.

I thought I had got up to go to the bathroom. I didn’t even wonder why I was on the floor. […] I just felt so comfortably sleepy and thought: Hm , why can’t I get up? Patient , Interview 1

Sometimes patients also initially attributed their symptoms to alternative explanations, i.e. an epileptic attack or hangover. Eight patients were unconscious or too confused to notice their symptoms or did not remember the situation. In these cases, other people called for help on their behalf. Twelve relatives present at symptom onset immediately knew or strongly suspected a stroke based on the symptoms, which included slurred speech, drooping mouth, not being able to speak, paresis, not being able to get up or walk properly, a cramped-up hand, and tingling feelings in one arm.

All relatives suspecting a stroke immediately called for help without waiting for the symptoms to improve or otherwise delaying the process.

I saw that something was wrong with [her] mouth and that’s when I knew it was a stroke . Relative , Interview 6 .

Familiarity with stroke information campaigns

Questionnaires showed a positive correlation between familiarity with stroke information campaigns and recognising symptoms as stroke ( r  = 0.203; p  ≤ 0.001*; N  = 457) but no correlation with no hesitation before calling for help ( r  = 0.009; p  = 0.847; N  = 456), transportation by emergency ambulance ( r  = 0.046; p  = 0.323; N  = 460), and arrival at hospital within 4.5 h ( r  = 0.014; p  = 0.769; N  = 457).

In the interviews, patients were asked about their prior knowledge about the disease stroke and if so, their information sources. Twelve patients indicated that they had had prior information about the disease stroke, naming information sources such as television shows, books and magazines on health topics, knowing other stroke patients, medical conditions because of which they had been told they were at risk for stroke, a previous (own) stroke, and working or volunteering in health care. Of these patients, two patients reported having recognised their stroke, both immediately asking their husbands to call help. Stroke information campaigns were not mentioned by the interviewees.

Many patients who answered “no” to the question “Did you have any prior information about the disease stroke?”, also reported knowing other stroke patients or having discussed their stroke risk or suspected stroke symptoms with a health professional in the months or years before their stroke. Two patients reported actively avoiding information on the topic

When I saw those news articles , I did not read them. […] I skipped them. […] I did not want to know about that. […] I had the feeling […] that I wanted nothing to do with it. Patient , Interview 9 When there was information on TV , I often switched channels. I can’t watch it […] , it upsets me too much. Patient , Interview 34

The latter patient is one of two patients who, despite indicating no prior information about stroke, recognized their stroke at symptom onset. The other patient reported that because of his regular check-up appointments for heart disease he was aware of his stroke risk. The patient was alone at home when the stroke happened but was found by a neighbour who immediately called an ambulance.

Only few patients who indicated having no prior information about stroke also reported not knowing any stroke patients and not having been aware that they were at risk of stroke. In these cases, it was the patient’s partner who initiated help-seeking. In one case, the patient’s wife called an ambulance because of the severity of the symptoms even though she did not realise it was a stroke at the time.

Nine relatives present at symptom onset said they had prior information about stroke, also citing television shows and books on health topics, knowing other stroke patients, the patient’s previous stroke, and volunteering in health care as their main information sources .

Speaking to patient: I saved you. Because I know […]. I do read a lot , and I watch [shows] on TV Relative , Interview 33

All of these relatives recognised the patient’s stroke based on their symptoms and sought help immediately.

Previous stroke

Questionnaire data for having experienced one or more previous strokes showed a positive correlation with recognising symptoms as stroke ( r  = 0.114; p  = 0.015*; N  = 459) but no correlation with no hesitation before calling for help ( r  = 0.027; p  = 0.565; N  = 457), transportation by emergency ambulance ( r  = 0.008; p  = 0.872; N  = 462), and arrival at hospital within 4.5 h ( r  = 0.02; p  = 0.677; N  = 459).

In the qualitative patient sample, four patients had previously experienced a stroke. None of them recognised their second stroke, with two unconscious at symptom onset or unable to recall the situation later. In two cases, patients knew that a stroke had been discovered previously during a routine scan, but they had not been aware of it when it happened (so-called “silent infarctions”). A third patient had experienced his first stroke just a few weeks prior to his second while he was still in rehabilitation for the first. A fourth patient had experienced an acute stroke two years previously. This latter patient did not seem to (want to) realise that this would put him at risk for another stroke:

Interviewer: “Were you aware that having had a previous stroke would put you at risk for another one?” Interviewee: I thought it’s enough now. I […] suppressed it , [put it] out of my mind […]. I thought it would be over now. Patient , Interview 9

In one of the above cases, Patient 9’s wife recognized the stroke and alerted emergency services immediately. In the other cases, no relatives were present and emergency services were instead alerted by unrelated witnesses. A fifth case of a previous stroke was reported by the daughter of a stroke patient who was herself not included in this study. This patient had experienced a severe acute stroke approximately twelve years previously. The daughter reported this as the reason why she recognized her mother’s second stroke and called for help immediately:

She had major speech problems after her first stroke […]. And [this time] I noticed the exact same thing. […] I said: it’s a stroke again. Relative , Interview 24

Knowing other stroke patients

Questionnaires showed no correlation between knowing other stroke patients and recognising symptoms as stroke ( r  = 0.082; p  = 0.081; N  = 455), no hesitation before calling for help ( r  = 0.031; p  = 0.514; N  = 453), transportation by emergency ambulance ( r  = 0.052; p  = 0.264; N  = 458), and arrival at hospital within 4.5 h ( r  = 0.052; p  = 0.272; N  = 455). For those patients who did know other stroke patients and who reported having discussed stroke symptoms with them, a positive correlation was found with recognising symptoms as stroke ( r  = 0.152; p  = 0.026*; N  = 215), and arrival at hospital within 4.5 h ( r  = 0.230; p  = 0.001*; N  = 217) but not with no hesitation before calling for help ( r  = 0.045; p  = 0.506; N  = 216) and transportation by emergency ambulance ( r  = 0.037; p  = 0.588; N  = 217).

In the interviews, thirteen patients reported knowing other stroke patients before, mostly family members and friends, but also colleagues, neighbours and acquaintances. Of these, two patients had recognised their own stroke and called for help immediately. One spoke in detail about her son-in-law’s stroke and thrombectomy treatment as well as the stroke experience of a friend, stating this as the reason “[…] why [she and her husband] had known about stroke since then and also knew about the time window” (Patient , Interview 7) . This was not the case for the other patient who first reported no prior information about stroke before mentioning that his mother had had one at a much older age:

Interviewer: Did you have general prior information about the disease stroke? Interviewee: No. […] Well , [my] mother had a stroke at [88]. Of course , I was aware of that. But , well , riding your motorcycle at [57] , you don’t think about a stroke Patient , Interview 25

A similar pattern was also visibile with other interviewees, who initially responded that they did not know other stroke patients before realising that this was not the case. Nine patients specifically stated that they did not know other stroke patients before their own stroke. Of these, three patients were able to recognise their own stroke, however citing other information sources such as check-ups for heart disease, working in health care, and TV programs.

Seven relatives present at symptom onset reported knowing other stroke patients, with several identifying this as the reason why they recognised their spouse’s stroke and responded appropriately.

We reacted immediately […] because several people in our family already had a stroke , so I know the symptoms. Relative , Interview 29

We explored patients’ and relatives’ help-seeking behaviour at stroke symptom onset using quantitative questionnaires and qualitative interviews. Our hypothesis that having stroke awareness would be positively associated with appropriate help-seeking behaviour was partially supported by quantitative and qualitative data, which confirmed and contradicted each other and sometimes provided potential explanations for apparent inconsistencies, as we discuss below.

Summary and discussion of main findings

Qualitative findings around the impact of knowing stroke symptoms were found to be partially in discordance with quantitative findings. Specifically, questionnaires showed patients with knowledge of stroke symptoms to be more likely to recognise their symptoms as stroke and to arrive at hospital on time. In contrast, interviews showed many patients to not have recognized their symptoms as stroke, even when they knew of common stroke symptoms. Two patients explained that they did not feel ill and even that they felt comfortable. This was confirmed by a former stroke patient in the Patient Council who reported not linking their general knowledge to their acute experience and inexplicably feeling safe and seeing everything through rose-tinted glasses. While the literature shows that lack of pain or perceived symptom severity can contribute to a diminished feeling of urgency, we were not able to find published descriptions of these feelings of comfort or safety [ 16 , 27 , 28 , 29 ].

Regarding the importance of familiarity with information campaigns , our qualitative and quantitative findings complemented each other. While questionnaires showed that patients familiar with campaigns were more likely to recognise their stroke, interviewed patients reported other information sources. Findings from the published literature show a variety of results in terms the impact of stroke information campaigns, e.g. reporting (partial) effectiveness [ 7 , 8 , 10 ] but also rather limited impact [ 6 , 9 ]. Notably, in our study, patient reporting of prior stroke information sometimes appeared inconsistent, e.g. when patients later spoke about a relative with stroke. This suggests that patients have better recall of some types of information than others [ 28 ]. It may also be suggestive of individual patient characteristics contributing to avoidance behaviour. Moloczij et al. called this the desire to “[maintain] a sense of normalcy”, describing several strategies used by patients to support their decision not to take any action, including denial, minimisation of symptoms, and compensating or adapting [ 16 ]. Wang et al. use descriptors such as “hesitating and puzzling” and “doubting – it may only be a minor problem” to describe this process experienced by stroke patients before initiating help-seeking [ 30 ].

Partial discordance was also found for previous strokes . While questionnaires showed patients with one or more previous strokes more likely to recognise their current symptoms as stroke, none of the five patients in the qualitative sample had recognised their current stroke. In their literature review of factors influence prehospital delay and stroke knowledge, Teuschl and Brainin (2010) find that only few studies report shorter time delays or better stroke knowledge in persons having suffered a previous stroke [ 27 ]. While silent (previous) infarctions may explain some of these instances, one patient who actively experienced their previous stroke reported avoidance behaviour before the second stroke. This was also reflected in Mackintosh et al.’s study of why people do (not) immediately contact emergency services, including several patients who recognised their second stroke but did not take action [ 28 ]. This observation was discussed in the Patient Council whose patient representatives showed surprise at the apparent lack of impact of previous stroke experiences. It was discussed whether stroke patients may not perceive themselves as living with a long-term condition requiring ongoing vigilance, but instead an isolated and completed incident.

Finally, qualitative and quantitative data were found to overlap and expand each other for knowing other stroke patients and having discussed the disease stroke . Interviews provided additional insights into possible reasons for when patients did not relate to others’ experiences and showed the importance of relatives knowing other stroke patients. Questionnaires showed no significant associations between knowing other stroke patients and the four dimensions of appropriate help-seeking behaviour, but patients who had discussed symptoms with other stroke patients were found to be more likely to recognise their stroke and to arrive at hospital on time. Again, there appeared to be inconsistencies in the interviews, with patients forgetting and then remembering knowing someone with stroke, and with many patients not relating others’ stroke experiences to their own situation. In contrast, several relatives identified knowing other stroke patients as the specific reason why they recognized the patient’s stroke and knew how to react. The importance of bystander involvement was explored by Mellon et al., identifying symptom recognition and help-seeking by witnesses as critical for a fast response [ 31 ]. For instance, Geffner et al. found that the decision to seek medical help was taken by patients in only 20.4% of cases [ 32 ]. Iverson et al. also found the presence of a bystander at symptom onset to be associated with appropriate help-seeking [ 15 ]. However, other qualitative findings are more nuanced, e.g. with Mc Sharry et al. reporting actions taken by others as having the potential to override patients’ own identification of symptoms and Moloczij et al. finding that sometimes the presence of another person contributed to delayed help-seeking, while at other times facilitating contact with medical services [ 16 , 29 ]. In addition to patients’ and relatives’ own behaviour and decisions, studies also show the importance of system factors, such as inefficient pre-hospital triage for treatment delay [ 33 ].

Strengths and limitations

As data collection was prepared and conducted independently, it was not always perfectly matched. One example of this is the fact that the rural-urban divide was not considered in detail in the qualitative data collection. This means that potentially important qualitative explanations of quantitative findings related to rural vs. urban differences were not explored in the current study, such as potential differences in information access, transport time or time-to-access to emergency services. Moreover, as is appropriate for qualitative interviews, prompting for more detailed information depended on the specific context and was therefore not feasible for all interviewees and all sub-questions. In the questionnaires, patients were asked about prior knowledge of stroke systems after they had their stroke. However, since it was completed on the day itself or day one after treatment, there would not have been much time for extended patient education. Additionally, the quantitative questionnaire was analysed with a pre-defined analysis plan and was collected over a (pre-defined) time period of six months. However, no power or sensitivity analysis was conducted in advance. Finally, our qualitative sample showed very good recovery, which probably affected the range of experiences and reactions covered in the interviews. One might assume that this overrepresentation of good outcomes could suggest a similar overrepresentation of study participants who “acted correctly”. However, given the importance of luck, bystander help, patients’ physical incapability to react and additional factors other than informed decision-making reported in this study, our results indicate that caution is warranted when interpreting good outcomes or arrival inside the time-window as proxies for having acted quickly or correctly (and vice versa). The main strengths of this study are its two-site design covering hospitals in urban and rural areas with differences in acute stroke treatment options, ensuring good external validity for Germany and countries covering larger geographical areas, its mixed methods approach allowing for integration of findings and generation of new perspectives of inquiry, and the involvement of patient representatives in the study preparation and conclusion.

Recommendations

As quantitative and qualitative findings sometimes seemed contradictory, we recommend that future studies collect data from one patient sample (instead of two separate samples, as here), allowing for direct back-and-forth iterations.As qualitative interviews pointed towards relevant inconsistencies in patient reporting, e.g. of prior stroke knowledge even with regard to close family members, it might be worth re-examining the reliability of common quantitative measures of stroke awareness and help-seeking behaviour where these inconsistencies would remain hidden and potentially incorrect. Following the Patient Council discussions, future research may investigate the “comfortable lull” reported by two patients from the study sample and one patient from the Council. If found in more instances, this could contribute to patients not recognizing a situation as highly problematic and requiring urgent action. In terms of practice recommendations, a more family- or community-based approach to stroke information provision may be helpful, emphasising the opportunity to be a loved one’s saviour. This could lessen the impact of avoidance behaviour and increase the positive impact of the presence of a family member on the decision-making process. This may necessitate critical discussions of whether and how relatives should be able to override patient preferences for delayed or no help-seeking behaviour, especially when the patient’s capacity for decision-making is impaired. As many patients seemed unable to apply general knowledge of stroke symptoms in the acute situation, we suggest exploring an example-based approach to risk communication. Specific situational examples (e.g. lying on the floor in the middle of the night or falling down without knowing why) may be a more accessible source of information compared to paresis of the arms or legs. To provide this type of information in the most appropriate way to future patients and their relatives, it seems relevant to involve former stroke patients in the preparation and provision of these informational resources.

Our study provides insights into the complexity of a decision-making process that is influenced by certain factors, but not others – e.g. a previous stroke makes it more likely that a patient recognises their symptoms as stroke, but not that they call for help without hesitation or arrive at the hospital on time. Interviews with patients and relatives provided in-depth insights into these seemingly contradictory findings, e.g. suggesting processes of denial or the inability to translate abstract knowledge into correct actions. We therefore recommend to address relatives as potential saviours of loved ones, increased use of specific situational examples (e.g. lying on the bathroom floor) and the involvement of patient representatives in the preparation of informational resources and campaigns.

Data availability

The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request. This excludes interview transcripts as ethics requirements to ensure confidentiality do not allow for data sharing outside the research team.

Schönenberger S, Weber D, Ungerer MN, Pfaff J, Schieber S, Uhlmann L, et al. The KEEP SIMPLEST Study: improving In-House delays and Periinterventional Management in Stroke Thrombectomy-A Matched Pair Analysis. Neurocrit Care. 2019;31(1):46–55.

Article   PubMed   Google Scholar  

Wu TY, Coleman E, Wright SL, Mason DF, Reimers J, Duncan R, et al. Helsinki Stroke Model is transferrable with Real-World resources and reduced stroke Thrombolysis Delay to 34 min in Christchurch. Front Neurol. 2018;9:290.

Article   PubMed   PubMed Central   Google Scholar  

Meretoja A, Weir L, Ugalde M, Yassi N, Yan B, Hand P, et al. Helsinki model cut stroke thrombolysis delays to 25 minutes in Melbourne in only 4 months. Neurology. 2013;81(12):1071–6.

Article   CAS   PubMed   Google Scholar  

Willeit J, Geley T, Schöch J, Rinner H, Tür A, Kreuzer H, et al. Thrombolysis and clinical outcome in patients with stroke after implementation of the Tyrol Stroke pathway: a retrospective observational study. Lancet Neurol. 2015;14(1):48–56.

Ebinger M, Winter B, Wendt M, Weber JE, Waldschmidt C, Rozanski M, et al. Effect of the use of ambulance-based thrombolysis on time to thrombolysis in acute ischemic stroke: a randomized clinical trial. JAMA. 2014;311(16):1622–31.

Morrow A, Miller CB, Dombrowski SU. Can people apply ‘FAST’ when it really matters? A qualitative study guided by the common sense self-regulation model. BMC Public Health. 2019;19(1):643.

Rasura M, Baldereschi M, Di Carlo A, Di Lisi F, Patella R, Piccardi B, et al. Effectiveness of public stroke educational interventions: a review. Eur J Neurol. 2014;21(1):11–20.

Flynn D, Ford GA, Rodgers H, Price C, Steen N, Thomson RG. A time series evaluation of the FAST National Stroke awareness campaign in England. PLoS ONE. 2014;9(8):e104289.

Wolters FJ, Li L, Gutnikov SA, Mehta Z, Rothwell PM. Medical attention seeking after transient ischemic attack and minor stroke before and after the UK Face, Arm, Speech, Time (FAST) Public Education campaign: results from the Oxford Vascular Study. JAMA Neurol. 2018;75(10):1225–33.

Nordanstig A, Asplund K, Norrving B, Wahlgren N, Wester P, Rosengren L. Impact of the Swedish National Stroke Campaign on stroke awareness. Acta Neurol Scand. 2017;136(4):345–51.

Metias MM, Eisenberg N, Clemente MD, Wooster EM, Dueck AD, Wooster DL, et al. Public health campaigns and their effect on stroke knowledge in a high-risk urban population: a five-year study. Vascular. 2017;25(5):497–503.

Bray JE, O’Connell B, Gilligan A, Livingston PM, Bladin C. Is FAST stroke smart? Do the content and language used in awareness campaigns describe the experience of stroke symptoms? Int J Stroke: Official J Int Stroke Soc. 2010;5(6):440–6.

Article   Google Scholar  

Hartigan I, O’Connell E, O’Brien S, Weathers E, Cornally N, Kilonzo B, et al. The Irish national stroke awareness campaign: a stroke of success? Appl Nurs Research: ANR. 2014;27(4):e13–9.

Kraywinkel K, Heidrich J, Heuschmann PU, Wagner M, Berger K. Stroke risk perception among participants of a stroke awareness campaign. BMC Public Health. 2007;7:39.

Iversen AB, Blauenfeldt RA, Johnsen SP, Sandal BF, Christensen B, Andersen G, et al. Understanding the seriousness of a stroke is essential for appropriate help-seeking and early arrival at a stroke centre: a cross-sectional study of stroke patients and their bystanders. Eur Stroke J. 2020;5(4):351–61.

Moloczij N, McPherson KM, Smith JF, Kayes NM. Help-seeking at the time of stroke: stroke survivors’ perspectives on their decisions. Health Soc Care Commun. 2008;16(5):501–10.

Zock E, Kerkhoff H, Kleyweg RP, van de Beek D. Intrinsic factors influencing help-seeking behaviour in an acute stroke situation. Acta Neurol Belgica. 2016;116(3):295–301.

Zock E, Kerkhoff H, Kleyweg RP, van Bavel-Ta TBV, Scott S, Kruyt ND, et al. Help seeking behavior and onset-to-alarm time in patients with acute stroke: sub-study of the preventive antibiotics in stroke study. BMC Neurol. 2016;16(1):241.

Article   CAS   PubMed   PubMed Central   Google Scholar  

Busetto L, Luijkx K, Vrijhoef HJM. Development of the COMIC Model for the comprehensive evaluation of integrated care interventions. Int J Care Coord. 2016;19(1–2):47–58.

Google Scholar  

Levitt HM, Bamberg M, Creswell JW, Frost DM, Josselson R, Suárez-Orozco C. Journal article reporting standards for qualitative primary, qualitative meta-analytic, and mixed methods research in psychology: the APA Publications and Communications Board task force report. Am Psychol. 2018;73(1):26–46.

Fetters MD. The mixed methods research workbook: activities for designing. implementing, and publishing projects: SAGE; 2019.

Fetters MD, Curry LA, Creswell JW. Achieving integration in mixed methods designs-principles and practices. Health Serv Res. 2013;48(6 Pt 2):2134–56.

Moseholm E, Rydahl-Hansen S, Lindhardt BØ, Fetters MD. Health-related quality of life in patients with serious non-specific symptoms undergoing evaluation for possible cancer and their experience during the process: a mixed methods study. Qual Life Res. 2017;26(4):993–1006.

Busetto L, Sert M, Herzog F, Hoffmann J, Stang C, Amiri H, et al. But it’s a nice compromise - qualitative multi-center study of barriers and facilitators to acute telestroke cooperation in a regional stroke network. Eur J Neurol. 2021;n/a(n/a):1–9.

Busetto L, Stang C, Hoffmann J, Amiri H, Seker F, Purrucker J, et al. Patient-centredness in acute stroke care – a qualitative study from the perspectives of patients, relatives and staff. Eur J Neurol. 2020;27(8):1638–46.

Ungerer MN, Busetto L, Begli NH, Riehle K, Regula J, Gumbinger C. Factors affecting prehospital delay in rural and urban patients with stroke: a prospective survey-based study in Southwest Germany. BMC Neurol. 2020;20(1):441.

Teuschl Y, Brainin M. Stroke education: discrepancies among factors influencing prehospital delay and stroke knowledge. Int J Stroke: Official J Int Stroke Soc. 2010;5(3):187–208.

Mackintosh JE, Murtagh MJ, Rodgers H, Thomson RG, Ford GA, White M. Why people do, or do not, Immediately Contact Emergency Medical Services following the onset of Acute Stroke: qualitative interview study. PLoS ONE. 2012;7(10):e46124.

Mc Sharry J, Baxter A, Wallace LM, Kenton A, Turner A, French DP. Delay in seeking medical help following transient ischemic attack (TIA) or mini-stroke: a qualitative study. PLoS ONE. 2014;9(8):e104434.

Wang PY, Tsao LI, Chen YW, Lo YT, Sun HL. Hesitating and puzzling: the experiences and decision process of Acute ischemic stroke patients with Prehospital Delay after the onset of symptoms. Healthc (Basel). 2021;9(8):1061. https://doi.org/10.3390/healthcare9081061 . PMID: 34442198; PMCID: PMC8391298.

Mellon L, Doyle F, Williams D, Brewer L, Hall P, Hickey A. Patient behaviour at the time of stroke onset: a cross-sectional survey of patient response to stroke symptoms. Emerg Med J. 2016;33(6):396–402.

Geffner D, Soriano C, Pérez T, Vilar C, Rodríguez D. Delay in seeking treatment by patients with stroke: who decides, where they go, and how long it takes. Clin Neurol Neurosurg. 2012;114(1):21–5. Epub 2011 Sep 23. PMID: 21944574.

Iversen AB, Johnsen SP, Blauenfeldt RA, Gude MF, Dalby RB, Christensen B, Andersen G, Christensen MB. Help-seeking behaviour and subsequent patient and system delays in stroke. Acta Neurol Scand. 2021;144(5):524–34. https://doi.org/10.1111/ane.13484 . Epub 2021 Jun 14. PMID: 34124770.

Download references

Acknowledgements

The authors thank all study participants for their participation and valuable contribution to this study. For the publication fee, we acknowledge financial support by Heidelberg University.

There was not external funding for this study.

Open Access funding enabled and organized by Projekt DEAL.

Author information

Matthias Ungerer and Christoph Gumbinger contributed equally to this work.

Authors and Affiliations

Department of Neurology, Heidelberg University Hospital, Im Neuenheimer Feld 400, 69120, Heidelberg, Germany

Loraine Busetto, Christina Stang, Melek Sert, Johanna Hoffmann, Jan Purrucker, Wolfgang Wick, Matthias Ungerer & Christoph Gumbinger

Institute of Medical Virology, Goethe University Frankfurt, University Hospital, Paul-Ehrlich-Str. 40, 60590, Frankfurt am Main, Germany

Loraine Busetto

Department of Paraplegia, Heidelberg University Hospital, Schlierbacher Landstraße 200a, 69118, Heidelberg, Germany

Franziska Herzog

Department of Neuroradiology, Heidelberg University Hospital, Im Neuenheimer Feld 400, 69120, Heidelberg, Germany

Fatih Seker & Martin Bendszus

You can also search for this author in PubMed   Google Scholar

Contributions

LB drafted the manuscript, conceptualised the overall mixed methods study design, and was responsible for the qualitative study design including qualitative data collection and analysis. CS, FH, MS and JH conducted the qualitative interviews and contributed significantly to qualitative data analysis. JP, FS, MB and WW provided medical expertise, contributed to quantitative analysis and revised the manuscript. MU conducted the quantitative analysis and contributed to the mixed methods analysis. CG had a supervisory role, contributed significantly to quantitative analysis and mixed methods design and analysis and relevantly revised different manuscript versions.

Corresponding author

Correspondence to Loraine Busetto .

Ethics declarations

Ethics approval and consent to participate.

Ethics approval was obtained from the ethics committee of the Medical Faculty of Heidelberg University (S-306/2016; S-682/2017). All study participants provided written informed consent.

Consent for publication

Not applicable.

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Busetto, L., Stang, C., Herzog, F. et al. “I didn’t even wonder why I was on the floor” – mixed methods exploration of stroke awareness and help-seeking behaviour at stroke symptom onset. BMC Health Serv Res 24 , 880 (2024). https://doi.org/10.1186/s12913-024-11276-6

Download citation

Received : 23 February 2024

Accepted : 03 July 2024

Published : 02 August 2024

DOI : https://doi.org/10.1186/s12913-024-11276-6

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• All clinical neurology
• Medical care
• Stroke awareness
• Help-seeking behaviour
• Mixed methods research
• Qualitative research
• Acute stroke

BMC Health Services Research

ISSN: 1472-6963

• General enquiries: [email protected]

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• Lippincott Open Access

How to study effectively

Alexander fowler.

a Guy’s and St Thomas’ NHS Foundation Trust

Katharine Whitehurst

b Royal Devon and Exeter Hospital

Yasser Al Omran

c East Anglia NHS Deanery

Shivanchan Rajmohan

d Imperial College London

Yagazie Udeaja

e UCL Medical School, University College London

Kiron Koshy

f Brighton and Sussex University Hospitals

Buket Gundogan

g University College London Medical School, UK

The ability to study effectively is an essential part of completing a medical degree. To cope with the vast amount of information and skills needed to be acquired, it is necessary develop effective study techniques. In this article we outline the various methods students can use to excel in upcoming examinations.

The ability to study effectively is essential in a medical degree. Firstly, having an effective way of learning is key to completing medical school, to cope with the vast volume of information taught. Secondly, medicine and surgery are careers that require constant learning; best practice is ever changing and it is important to be able to integrate these changes into your clinical practice. Thirdly, setting up efficient learning techniques while at medical school will be beneficial to you as and when you approach membership examinations, where study must be fitted around clinical commitments.

Studying effectively depends upon 2 factors: the content you intend to study and how you learn. Learning styles classically fall into 4 groups according to the VARK model (Visual, Aural, Read/Write, and Kinesthetic) but medical students seem to be multimodal in their learning style 1 . This implies that different learning techniques typically ascribed to certain learning styles may be beneficial to students of other learning styles and thus attempting to determine your unique learning style may help to consolidate your methods of study.

Broadly, study is divided into “Book Work” and “Practical Work.” As a medical student, this translates to “Written Papers” and “Clinical Exams,” respectively, although there is often significant overlap. Irrespective of what is to be studied, a plan must be considered first. A solid plan and revision timetable are critical to success upon examination. First, find the date of your examination/s, then work back to deduce how long you have to prepare. At this time, you must also consider the format of the examination, either by reading supplementary material offered by the medical school or by asking for first-hand accounts from students in the years above you who have experienced the examination and can provide extra tips and information. These quick tasks ensure that your preparation and prospective study is well suited to the examination you will do.

Some like to dedicate specific days of the week to certain topics and others, different times in a day and this will vary from person to person. It is possibly best to implement a mixture of the 2, where there is an initial block session to establish the basics, followed by a number of consolidation periods over time to go over and reinforce your learning 2 . For big topics, it is often easier and more time efficient to try and establish a pattern of learning that involves regular, small periods of work. Switching between topics when studying may also aid in effective learning 2 .

Chunking is essentially breaking a big topic into smaller components, which are more manageable with regards to study. Depending on the topic, it may be appropriate to break it down in different ways, for example, anatomy may be broken down by location (pelvis/upper limb), whereas pediatrics may be broken down by body system (gastrointestinal/genitourinary). Emergencies can be divided according to the dysfunctional body system or the symptoms that the patient may present with. Chunking allows you to move swiftly between topics. Making links between these different topics or ideas consolidates them within your mind, which also makes information recall easier, a skill much desired for examinations.

It is not unusual to see medical students with small medical pocketbooks on the wards and clinical placements. There are many books that have been made for this intention—to be read when you have a few minutes spare, enough time to read a few key facts but not enough to have a “revision session.” Apps on smart phones also offer a means for this opportunistic learning. It may also be worth carrying some notes with some static points you have to learn, for example, drug doses. Carrying allows you to consolidate your learning around clinical commitments, some of which are often considerably delayed.

Learning techniques

There is a wide range of techniques people use for their learning:

• Studying in different or new locations—Students often have an ideal location where they feel comfortable to study. However, it is proposed that studying in different locations can aid in memory recall and learning 3 , 4 . This could be different areas in the same building or completely separate locations but changing location may reap significant benefits.
• Working in groups—Each person may learn a different topic to teach to the student group later. The preparation involved in teaching and the interaction you have with your fellow students can help to consolidate your learning 3 .
• Stick men—A simple outline of a stick man can be used and arrows drawn to demonstrate various signs and symptoms of disease. These can aid pattern recognition associated with making diagnoses.
• Spider diagrams/mind maps—Mind maps are a revision technique often not utilized by medical students yet they are a good revision technique for enabling factual recall compared with other study methods 5 .
• Flash cards—These can be made to cover systems/diseases or specific questions. They are very easy to carry around and can be used alone or as part of a group. There are a number of web programs that create flashcards based on the content you are learning, some of which allow flashcards to be distributed electronically between fellow students 2 , 6 .
• Post-it-notes—Around examination times, some students have found sticking post it notes, on which key facts are written, on their walls, desks, or places where they will view them regularly, which may be around the home.

It is important that your learning is derived from a range of resources, including past papers. Past papers should be used early to gauge where you are before your revision and then used later when you have covered most of the required material to identify your unique areas of weakness. Online test services and question banks have exploded in recent years. They now enable you to test your knowledge by domain and even by question type (for example, extended matching questions and singe best answers, which each require a different examination technique). Some question banks even offer practice questions outside of the medical degree curriculum but integral for your medical career, such as the examinations, which are done when applying for your foundation post.

It is imperative that the any books and websites you use are up to date with current medical guidelines and best practice. Most medical students use online resources as much, if not more, than book-based resources. Social media, such as Facebook and Instagram, is also rapidly becoming a platform by which you can access medical resources.

Practical work

The practical examinations of medical school require a slightly different approach to the written examinations. That said, they exist symbiotically, such that a solid basis from your book work will set you up really well for practical examinations. The key to learning for practical examinations (which includes communication history taking, long cases, short cases, clinical examination, and practical procedures) is knowing the format for your examination, and recurrent practice. Recently, practical examinations in medical schools have taken place as OSCEs (Objective Structured Clinical Examination). OSCE practice sessions between students is invaluable and have been shown to provide high-quality feedback when compared with feedback from the senior tutors but it is imperative that the feedback is constructive, recognizing faults with immediate suggestions for improvement 7 , 8 . There are many textbooks available that provide clinical scenarios and mock OSCE stations in order for students to practice among themselves.

Planning how to approach your examinations is critical. You should be aware of what you are expected to know, and how you’re going to be assessed on it. The best way to pass practical examinations is to have actually done what they are expecting you to be able to do and receive feedback. For example, if your first time placing a cannula is the week before your examination, you’re unlikely to be as confident as someone doing 5 a week throughout the year during clinical placements. In the lead up to examinations, sketch out a structure of what you need to practice, and how you’re going to do it. It really helps to have a senior colleague who can practice with you—observing and advising as you go.

Practical skills

This is all practice based so it is best to watch an expert doing it, or access online resources explaining how to do it, then continually practice. Many medical schools will have dedicated teaching sessions run by clinical skill tutors, allowing you learn from a professional approved by the medical school and an opportunity to practice between yourselves, a method found to be very effective 7 . Upon examination, your marks will be based on your technique but also your manner with the supposed patient. You have to be prepared to talk to a mannequin like it is a real patient with the same level of respect.

Communication skills

There are a number of books detailing how communication skills are examined (the best and most appropriate to use are those written for PACES preparation). The key here is knowing the structures of communication stations, key facts, and practice. Practice with friends going through the scenarios provided. Communication stations take many forms: focused history, explaining a procedure, gaining consent, establishing capacity, explaining a diagnosis, and many others. For history stations, ensure that you have a good structure for your history and key headings, making sure you show empathy for the patient as you go. All communication stations run on a backbone of a clear introduction, good rapport with the patient, and checking for any ideas, concerns, or expectations they may have throughout the consultation.

Examinations

You must practice examinations with close observation by someone who either examines final year examinations or has recently completed finals/membership examinations. Ideally, set it up so you have short, regular sessions to enable you to develop your technique. Practice needs to be in the time you have allocated, on real patients, with real signs, and with a following presentation/viva as appropriate. Rigorous questioning after each examination regarding patient presentation, diagnosis, investigation, and management can really set you up for any difficult questions that may come up in your end of year OSCE. Around this dedicated practice time, you should be examining every major organ system at least once a week in the build up to finals, and ideally once a month when you are on firms (though this is heavily firm dependent). Some medical schools offer simulation training especially for ABCDE training, a method by which students may feel better prepared for examinations involving problem-solving decisions 9 .

Looking after yourself

Examination performance, irrespective of the preparation you may have done, is optimized by staying healthy. For many students, examination time translates to prolonged periods of erratic sleep patterns, reduced exercise, poor diet options, and copious amounts of caffeine.

Sleep is key for examination preparation as sleep deprivation reduces the effectiveness of study and can considerably hinder your performance on the day 10 . It is recommended that regular sleep patterns are adopted, aiming for 6 hours of sleep or more daily. Naps in-between study sessions may also aid in effective study. Alongside sleep deprivation, reduced recreational exercise has also been shown to hinder examination performance 11 . It is essential that you keep a schedule of exercise during examination season. Exercising reduces stress, prevents burnout, and delays the onset of mental health conditions such as depression 10 .

A balanced diet must be maintained during examination season. A “bad” diet can affect your energy levels and thus can be detriment to the amount of time you are capable of efficient study. A bad diet may constitute as high energy, high fat with a reduction in protein consumption and this has been shown to correlate to reduced academic performance 12 . Eating a nonbalanced diet may also actually reduce the number of hours of effective study 13 . A balanced diet will also maintain your immune system, to stop you from acquiring an illness, which may slow or effectively halt study and revision times. You must remain hydrated. Being dehydrated is linked to reduced concentration, tiredness, and headaches, none of which constitute to an effective study session.

Finally if you are feeling stressed, you must talk to someone about it. It is most likely that there are numerous other students feeling the same way. You need to ensure you have a robust support network during medical school, particularly around examination time. This network can be made up of family members, fellow students, or friends or the welfare office at your medical school or university. There is a wide range of services that can be offered to you and some of which you can access independently.

• Know your examination dates and the amount of time you have to prepare.
• Use a range of learning techniques and study in chunks to ensure effective study sessions.
• Practise for your practical examinations, either with fellow colleagues or sessions run by your medical school.
• Sleep well, exercise, and maintain a healthy, balanced diet especially during examination season.
• Have a robust support network, which can be relied on during periods of stress.

Sponsorships or competing interests that may be relevant to content are disclosed at the end of this article.

Published online 15 June 2017

Conflict of interest statement

The authors declare that they have no financial conflict of interest with regard to the content of this report.

• Research article
• Open access
• Published: 29 July 2024

Unveiling the epigenetic impact of vegan vs. omnivorous diets on aging: insights from the Twins Nutrition Study (TwiNS)

• Varun B. Dwaraka 1   na1 ,
• Lucia Aronica 2   na1 ,
• Natalia Carreras-Gallo 1 ,
• Jennifer L. Robinson 2 ,
• Tayler Hennings 3 ,
• Matthew M. Carter 4 ,
• Michael J. Corley 5 ,
• Aaron Lin 1 ,
• Logan Turner 1 ,
• Ryan Smith 1 ,
• Tavis L. Mendez 1 ,
• Hannah Went 1 ,
• Emily R. Ebel 4 ,
• Erica D. Sonnenburg 4 ,
• Justin L. Sonnenburg 4 , 6 , 7 &
• Christopher D. Gardner 2  

BMC Medicine volume  22 , Article number:  301 ( 2024 ) Cite this article

9096 Accesses

2669 Altmetric

Metrics details

Geroscience focuses on interventions to mitigate molecular changes associated with aging. Lifestyle modifications, medications, and social factors influence the aging process, yet the complex molecular mechanisms require an in-depth exploration of the epigenetic landscape. The specific epigenetic clock and predictor effects of a vegan diet, compared to an omnivorous diet, remain underexplored despite potential impacts on aging-related outcomes.

This study examined the impact of an entirely plant-based or healthy omnivorous diet over 8 weeks on blood DNA methylation in paired twins. Various measures of epigenetic age acceleration (PC GrimAge, PC PhenoAge, DunedinPACE) were assessed, along with system-specific effects (Inflammation, Heart, Hormone, Liver, and Metabolic). Methylation surrogates of clinical, metabolite, and protein markers were analyzed to observe diet-specific shifts.

Distinct responses were observed, with the vegan cohort exhibiting significant decreases in overall epigenetic age acceleration, aligning with anti-aging effects of plant-based diets. Diet-specific shifts were noted in the analysis of methylation surrogates, demonstrating the influence of diet on complex trait prediction through DNA methylation markers. An epigenome-wide analysis revealed differentially methylated loci specific to each diet, providing insights into the affected pathways.

Conclusions

This study suggests that a short-term vegan diet is associated with epigenetic age benefits and reduced calorie intake. The use of epigenetic biomarker proxies (EBPs) highlights their potential for assessing dietary impacts and facilitating personalized nutrition strategies for healthy aging. Future research should explore the long-term effects of vegan diets on epigenetic health and overall well-being, considering the importance of proper nutrient supplementation.

Trial registration

Clinicaltrials.gov identifier: NCT05297825

Peer Review reports

While advances in technology and medicine have allowed the average person to live longer, age-related disease and impairment remain an issue that greatly impacts individuals and healthcare systems. Aging is associated with increases in health care costs and financial stress on social insurance systems [ 1 ]. In light of these challenges, the field of geroscience has emerged, proposing interventions aimed at slowing down or reversing the molecular changes that occur with aging. These interventions encompass a wide range of factors, including lifestyle modifications, nutrition, medications, sleep, and social factors, all of which can influence the aging process and potentially delay or prevent the onset of multiple chronic diseases, ultimately extending healthy lifespan [ 2 , 3 , 4 ]. Consequently, the exploration of nutritional and dietary recommendations has become an increasingly significant area of research within the broader field of aging, providing insights into how dietary choices can impact the aging process and overall health outcomes.

However, unraveling the intricate molecular mechanisms through which diets influence aging necessitates a deeper understanding of the epigenetic landscape [ 5 ]. Epigenetic modifications, such as DNA methylation, have emerged as pivotal regulators of gene expression and provide a promising avenue for investigating the effects of vegan diets on the aging process [ 6 ]. The epigenetic effects of a vegan diet, in comparison to an omnivore diet, remain largely unexplored, with limited available evidence. Although certain studies have indicated potential positive impacts of specific components of a vegan diet, such as heightened intake of vegetables and fruits, on epigenetic aging, concerns have been raised regarding potential deficiencies in essential “epi-nutrients” necessary for effective epigenetic regulation [ 7 ]. Notably, vitamins and nutrients, including vitamin B12, vitamin B+, choline, vitamin D, omega-3 fatty acids, and zinc, are among the concerns associated with a vegan diet, as their availability may be compromised. Furthermore, other work on diets has aimed to discover the association between diets and longevity [ 8 , 9 ]. For instance, the Mediterranean diet has been documented to slow the progression of frailty with aging [ 10 ]. Dietary protein intake is another important factor considered in aging and frailty, with many studies showing beneficial impacts of protein regardless of animal or plant origin [ 11 ]. These and other studies have provided mixed notions of a healthy vegan diet, necessitating additional interrogation of its impact on aging and disease outcomes, as measured by aging markers.

Epigenetic clocks, derived from DNA methylation patterns, have emerged as powerful tools for estimating biological age and predicting age-related outcomes. These clocks have also been refined over time to incorporate known clinical factors, making them sensitive and reliable indicators of aging-related changes [ 12 ]. Additionally, epigenetic interpretation algorithms have proven valuable in predicting relative immune cell levels and protein expressions, providing insights into immune system functionality through immune deconvolution [ 13 , 14 , 15 ]. Moreover, these clocks can estimate the number of cell cycle divisions, reflecting cellular senescence and potential disease susceptibility [ 16 ].

While aging intervention studies face the challenge of requiring sufficiently long periods to show statistically significant effect, advancements in DNAm-based analysis, such as phenotypically and clinically trained DNAm clocks, have allowed for changes in the pace of aging and risk factors related to aging to be studied [ 17 ]. Epigenetic age trials using these epigenetic clocks have found that different diets such as a Mediterranean diet and DASH diet have shown improvements of aging pathways and markers, including protective effects of immunosenescence markers, activation of mTOR pathway, and epigenetic aging [ 18 , 19 ]. In particular, a Mediterranean diet has been shown to both slow aging and delay the onset of frailty [ 20 ].

Given the discussion on which diets are most beneficial to longevity, this study aims to identify the effect of an 8-week plant-based or healthy omnivorous diet on blood DNA methylation in twins and evaluate age-related risk factors and health biomarkers. The novelty of this study includes the twin-pair study design which controls for genetic, age, and sex differences, while highlighting the methylation changes based on diet. Furthermore, this is the first study assessing the impact of epigenetic measures on twin-pair study design, and specifically addressed whether diet impacts such measures. Finally, we conducted a differential methylation analysis using the twin-pair design to identify potential DNAm markers which are related to the application of a healthy vegan or omnivorous diet, while also identifying DNAm markers which differentiate between diets. This comprehensive approach will provide insights into how diet type influences epigenetic dynamics and contribute to our understanding of potential interventions in the process of nutrition.

Ethical approval and study design

Procedures adhered to the ethical standards of the Helsinki Declaration, approved by the Stanford University Human Subjects Committee (IRB protocol 63955, approved March 9, 2022). Written informed consent was obtained from all participants. The study, a single-site, parallel-group dietary intervention trial, randomized generally healthy adult twins to either a healthy vegan or omnivorous diet for 8 weeks. Enrollment commenced in March 2022, concluding in May 2022, with the final follow-up in July 2022. The trial employed the CONSORT reporting guideline for randomized clinical trials, focusing on the primary outcome: the 8-week change in DNA methylation profiles from baseline. Secondary outcomes encompassed triglycerides, HDL-C, glucose, insulin, TMAO, vitamin B12, and body weight, serving as controls for relevant methylation risk scores and were published previously [ 21 ]. Diet quality, adherence, and study design are illustrated in Fig.  1 .

Timeline diagram for the study design. A total of 21 pairs of twins ( N =42) were subjected to a vegan diet ( N = 21, labeled in green) and an omnivore diet ( N = 21, labeled in orange). Blood was collected for baseline at the start of the trial (week 0) and at the end of the trial (week 8) and methylation states were quantified using the EPIC 850k array

Participant recruitment and eligibility

The goal was to recruit 22 pairs of identical twins—controlled for sex, age, and ethnicity—primarily from the Stanford Twin Registry and other twin registries, including Netflix’s pre-recruited participants interested in a documentary on vegan diets. Inclusion criteria involved participants aged ≥18, part of a willing twin pair, with BMI <40, and LDL-C <190 mg/dL. Exclusions included uncontrolled hypertension, metabolic disease, diabetes, cancer, heart/renal/liver disease, pregnancy, lactation, and medication use affecting body weight or energy. Eligibility was determined via online screening, followed by an orientation meeting and in-person clinic visit. Randomization occurred only after completing baseline visits, dietary recalls, and questionnaires for both twins. One twin pair (which started the study, did not abide by the above requirements and thus was removed from the study. Ultimately blood samples from 21 numbers of twin pairs ( N = 42) were considered for downstream analyses. Full details of the participant profiles are detailed in Landry et al. 2023 [ 21 ].

Dietary intervention and lifestyle changes

The study comprised two 4-week phases: delivered meals and self-provided meals. Trifecta Nutrition supplied meals for the first 4 weeks, tailored to omnivorous and vegan diets. Health educators facilitated nutrition classes via Zoom, emphasizing principles like choosing minimally processed foods and building balanced plates. The omnivorous group received animal product targets (e.g., 6–8 ounces of meat, 1 egg, and 1.5 servings of dairy), while the vegan group avoided all animal products. Dietary intake was assessed through unannounced 24-h recalls and participant logs on the Cronometer app, capturing food intake details at baseline, week 4, and week 8. Dietary data quality was ensured through trained dietitian interviews and app records, used to evaluate diet quality and adherence. To account for lifestyle changes, participants filled out surveys on global health, fatigue, stress, and physical activity, at baseline and week 8. Participants exhibiting notable changes in any of these factors were not considered in the analyses.

PCR-based telomere estimation

DNA was extracted from whole blood stored at −80 °C with the QIAamp DNA blood mini kit (QIAGEN cat# 51106). Relative telomere length was measured by quantitative polymerase chain reaction (qPCR), expressed as the ratio of telomere to single-copy gene abundance (T/S ratio) [ 22 , 23 ]. A detailed protocol can be found on the Telomere Research Network’s website ( https://trn.tulane.edu/wp-content/uploads/sites/445/2021/07/Lin-qPCR-protocol-01072020.pdf ). The inter-assay coefficient of variation (CV) for this study is 2.0%±1.7%. The baseline and follow-up samples of the same participant were processed in the same batch throughout the whole assay. Lab personnel is blind to all the demographic and clinical data.

DNA methylation assessment

Whole blood was collected at baseline and at week 8 for DNA methylation preparation and analysis. Majority of twin pairs (20 twin pairs, N = 40) were collected as biological replicates per time point and individual using Dried Blood Spot cards; one twin pair ( N =2 patients) which had triplicate collections in which two were collected by dried blood spot and one using the tasso. Blood collected by the clinics was sent to TruDiagnostic labs in Lexington, KY, for DNA extraction and methylation processing. Using the EZ DNA Methylation kit (Zymo Research), 500 ng of DNA was bisulfite-converted following the manufacturer’s instructions. Bisulfite-converted DNA samples were randomly assigned to wells on the Infinium HumanMethylationEPIC BeadChip, and the subsequent steps included amplification, hybridization, staining, washing, and imaging with the Illumina iScan SQ instrument to acquire raw image intensities. Longitudinal DNA samples for each participant were assessed on the same array to mitigate batch effects. Raw image intensities were saved as IDATs for further processing.

DNAm data processing

Raw IDATs underwent processing using the minfi pipeline [ 24 ]. Samples of low quality were identified with ENMix based on variance of internal controls, flagging samples showing more than 3 standard deviations away from the mean control probe value [ 25 ]. However, no outlier samples were identified, and thus, all samples were considered for analysis. DNAm normalization involves Gaussian mixed quantile normalization (GMQN) to correct between batch collections and BMIQ normalization to normalize intra-sample variance within chips [ 26 ]. Probe-level analysis masked probe sets without at least one intensity fluorescence above the background as implemented by pOOBAH. Missing beta values were imputed using K nearest neighbor (KNN) imputation.

Deriving estimates of epigenetic clocks and methylation-based metrics

Epigenetic clocks were calculated from cleaned beta values, focusing on clocks like Horvath multi-tissue [ 27 ], Horvath skin and blood [ 28 ], Hannum [ 29 ], PhenoAge [ 30 ], GrimAge v1 and v2 [ 31 , 32 ], and DNAmTL [ 33 ]. To ensure that values were highly reproducible, the principal component versions of these clocks were utilized as described by Higgins-Chen et al. [ 12 ]. Individual systems clocks were calculated using the framework presented by Sehgal et al. [ 34 ]. Clocks were calculated using a custom R script available on Github. DunedinPACE was calculated using a custom script available from Github ( https://github.com/danbelsky/DunedinPACE , [ 35 ]). Additional non-epigenetic age metrics included relative percentages of 12 immune cell subsets imputed using EpiDISH [ 15 ], 116 methylation-based predictions of biochemical and lifestyle risk factors using MethylDetectR [ 36 ], and 396 epigenetic biomarker proxies [ 14 ]. All epigenetic metrics such as clocks, telomere length, immune deconvolution, EpiScore, and EBPs, were residualized prior to statistical analysis by using the lmer() R package as such:

Residualized epigenetic metric = resid(lmer(Epigenetic predictor ~ Chronological Age + Sex + PC1 + PC2)

Estimates of EpiScores and EBPs were calculated using multivariate models described previously. Briefly, these estimates were derived by modeling DNAm beta values to predict relative protein estimates, as quantified by Olink and SEER/Mass Spec; metabolite estimates, as quantified by the Metabolon panel; clinical values; and clinical and laboratory protein estimates collected from various clinics and panels [ 14 , 37 ]. Resulting scores and estimates were then used for statistical analyses. All comparisons utilized paired Wilcoxon-rank sum tests faceted by diet type, with significance set at unadjusted p < 0.05.

Assessment of concordance for DNAm and surrogate values

Analyses of telomere and BMI values performed between the reported clinical/qPCR values and DNAm predicted values were conducted in R. Values were scaled using the scale() function prior to comparison. Cohen’s d statistics were calculated by inputting scaled values into the cohen.d() function available in the effsize library. Statistical significance was assessed using paired Wilcoxon-rank sum tests implemented in the wilcox.test() package. Spearman correlations and associated p -values were calculated using the cor.test() package in R and setting the method = “spearman”.

Differentially methylated analysis

Differential methylation analysis was conducted using processed beta values logit-transformed to M-values with the BetaValueToMValue function from the sesame R package. No additional probes (e.g., sex associated probes) were pre-filtered in order to prior to the analysis. However, technical variation and sex were considered in the final model for differential methylation. Limma package was applied across the four comparisons: within vegan (week 0 vs. week 8), within omnivore (week 0 vs. week 8), cross-sectional Vegan vs. Omnivore (at week 8) and cross-sectional Vegan vs. Omnivore (week 0, or baseline). Differentially methylated loci (DMLs) were identified using different modeling types based on comparison. For within diet comparisons which were longitudinal, multivariate linear models were controlled for fixed effects such as chronological age, BMI, sex, beadchip, 5 immune cell percentages (basophils, CD8T naive, eosinophils, NK, and Neu), the first three principal components of technical probes. For the cross-sectional comparisons, the same fixed effects and PC components were used; however, the individual ID was used in the longitudinal comparison. The inflation or deflation of P -values across the methylome was assessed with Q-Q plots and lambda values [ 38 ]. DMLs were identified with a significance level of unadjusted p < 0.001. False discovery rate (FDR) were also calculated as implemented within the limma package and reported.

GREAT analysis

Functional annotation of DMLs was performed using the GREAT pipeline to identify significant gene ontology terms, as implemented in the rGREAT R package [ 39 ]. Significant enrichment terms were identified using a Hyper_Raw_PValue < 0.0001; however, only those passing a correct p -value (FDR < 0.05) were discussed.

Description of study population

To investigate the impact of diet on the methylome, blood samples from a randomized clinical trial were used to quantify methylation [ 21 ]. As shown in Fig.  1 , to quantify methylation, whole blood was collected to establish a baseline measure of methylation at the time of starting the trial (week 0) and at the conclusion of the study (week 8). Baseline characteristics by diet group appear in Table 1 . Among 21 pairs of twins, the randomized mean age was 39.9 (SD 13) years, 77.3% were women, and the mean body mass index was 26 (SD 5). The BMI of both cohorts were largely equivalent due to each group matched to paired-twins with equivalent BMI and genetic makeup (average Vegan BMI = 26.3, average Omnivore BMI = 26.2). The paired-twin design developed here is unique as it controls for genetic and physiological differences between individuals surveyed, which ultimately increases the power of statistical comparisons across the two groups. Full descriptions and characteristics of the study population are detailed previously [ 21 ].

Diet type impacts changes in epigenetic age

To investigate the response to diet on biological age and telomere length, we quantified and analyzed several biological age and telomere length predictors derived from DNAm. These included the principal component (PC)-based clocks: the first-generation multi-tissue Horvath (Horvath1) and skin+blood Horvath (Horvath2); and the second-generation PhenoAge, GrimAge, and DNAm telomere clocks. Additionally, several non-PC clocks were included as well: the first-generation Zhang clock based on the elastic net (Zhang-EN) and BLUP (Zhang-BLUP) method; the second-generation multi-omic informed OMICmAge, and the third generation DunedinPACE clock. To better understand the impact of diet on the epigenetic age of specific organ systems, we also calculated the individual ages of 11 organ systems: Heart, Lung, Kidney, Liver, Brain, Immune, Inflammatory, Blood, Musculoskeletal, Hormone, and Metabolic. In addition, a composite age of the system was also calculated as Systems Age. In the vegan group, we observed significant decreases in the following epigenetic age metrics: PC GrimAge (mean Δ EAA = −0.3011, p = 0.033), PC PhenoAge (mean ΔEAA = −0.7824, p = 0.014), and DunedinPACE (mean Δ PACE residual = −0.0312, p = 0.00061) significantly decreased at 8 weeks relative to 0 weeks (Fig.  2 A–C). Similarly, we observed significant reductions in the composite systems age metric, which was corroborated by significant reductions of 5 out of 11 systems: Inflammation, Heart, Hormone, Liver, and Metabolic (F  i gure 2 D–I). In contrast, no epigenetic clock or telomere measure exhibited significant changes in the omnivorous cohort, suggesting that the omnivorous diet did not induce any epigenetic age methylation changes. Taken together, these findings suggest that the observed DNA methylation changes may contribute to the overall decreases in epigenetic age in response to a vegan diet, which is not observed among omnivores.

Boxplot showing the evolution of epigenetic age acceleration (EAA)/residuals among the significant epigenetic age clocks and systems-specific clocks based on diet type. Clocks assessed include the A PC GrimAge, B PC PhenoAge, C DunedinPACE, D Systems Age, E Inflammation Age, F Heart Age, G Liver Age, H Metabolic Age, and I Musculoskeletal Age. On the X -axis, the time points of measurements in weeks. On the Y -axis, the EAA/residual measure. EAA, or residual, is defined as the residual calculated between the raw value regressed upon chronological age, and adjusted by sex, technical principal components 1 and 2. On the top, the mean and median values of the EAA at each time point. The p -values of the paired Wilcoxon-rank sums test are also displayed in the plots. Lines that connect both boxplots represent the average of each patient’s tests

Telomere length quantified by Telomere Shortening Rate (TSR) exhibits changes, but not epigenetic telomere length

We next sought to elucidate the potential impact of vegan diets on telomere length. First, we established an understanding of concordance between estimated telomere length using quantitative polymerase chain reaction (qPCR) and the estimated PC DNAmTL values by quantifying the correlations between the values of all samples, and assessing significant differences agnostic to groups. Overall, we observed an overall correlation > 0.5 between the scaled qPCR and PC DNAmTL values calculated ( ⍴ = 0.564; p < 1.22e−7), and no significant differences (Wilcoxon-rank sum p = 0.9427). This was further supported by a negligible effect size difference between the scaled values (Cohen’s d estimate = −2.45e−15, 95% CI = −0.321–0.321). These results suggest that the values generated by both methods are comparable.

Next, changes in TSR were assessed between timepoints within each diet among the TSR data. The average telomere length was significantly longer at week 8 than at week 0 for Vegans ( p = 0.045, Δ T/S ratio = 0.0361, Fig.  3 A) but not for omnivores ( p = 0.86, Δ T/S ratio = −0.0045, Fig.  3 A). Furthermore, paired analyses comparing twins between diets within each time point were conducted among the TSR samples, which found that the Vegan group had significantly longer telomeres than their Omnivore twins at week 8 ( p = 0.01, Δ T/S ratio = 0.042) but not at week 0 ( p = 0.54, Δ T/S ratio = 0.0013), further confirming that the telomere extension was specific to the vegan diet. This contrasted the findings observed among the PC DNAmTL values, which showed that there were no significant changes between week 8 and week 0 measures among the Vegan or Omnivore cohort (Fig.  3 B).

Boxplot showing the change between relative telomere levels as quantified by qPCR and DNA methylation (PC DNAmTL). Telomere qPCR value is reported in panel A , while the PC DNAmTL values are reported in B . On the X -axis, the time points of measurements in weeks. On the Y -axis, the T/S ratio is shown for qPCR, or the residual of PC DNAmTL. The PC DNAmTL residual is defined as the residual calculated between the raw PC DNAmTL value regressed upon chronological age, and adjusted by sex, technical principal components 1 and 2. On the top, the mean and medians of the Y -axis values at each time point are reported. The p -values of the paired Wilcoxon-rank sums test are also displayed in the plots. Lines that connect both boxplots represent the average of each patient’s tests

Analysis of cell cycle changes shows no significant changes based on diet

We next assessed whether diet type exhibited differences in overall mitotic rate as quantified by the mitotic clock output. Using the epiTOC2 algorithm, we observed no significant changes in either the vegan or omnivore diet when assessing the total number of stem cell replication cycles estimated ( tnsc ) or the intrinsic stem cell cycle rate based on tissue ( irS ). This suggests that diet type did not have an impact on the overall mitotic clock values from the data.

Vegan diets exhibit significant changes in relative basophil levels

The immune system undergoes distinctive changes based on dietary choices, with vegan and omnivore diets influencing immune cell behavior in unique ways. Exploring this interplay provides valuable insights into the intricate relationship between diet and the body’s immune defenses. To investigate the impact of diet on the immune system, we next analyzed relative immune cell subset changes throughout the trial among 12 immune cell subsets quantified by the EPIDISH frame: CD8T-naive, CD8T-memory, CD4T-naive, CD4T-memory, basophils, B naive, B memory, T-regulatory, monocytes, neutrophils, natural killer, and eosinophils. We observed significant changes in basophil levels in the vegan and omnivore diets. However, the basophil levels increased in the vegan group (Δ mean = 0.0014, p = 0.04, Fig.  4 ) and decreased in the omnivore group (Δ mean = −0.0018, p = 0.048).

Boxplots showing the evolution of basophil cell subset percentages based on diet type. On the X -axis, the time points of measurements in weeks. On the Y -axis, the basophil measure. The basophil measure is residualized, which is defined as the residual of the raw deconvolution value regressed upon chronological age, and adjusted by sex, technical principal components 1 and 2. On the top, the mean and median values of the residual at each time point. The p -values of the paired Wilcoxon-rank sums test are also displayed in the plots. Lines that connect both boxplots represent the average of each patient’s tests

Assessment of type 2 diabetes risk based on loci

Previous studies have shown plant-based diets associated with reduced type 2 diabetes risk [ 40 , 41 ]. To test whether epigenetic changes are consistent with previous findings, we analyzed two DNA methylation loci, ABCG1 (cg06500161) and PHOSPHO1 (cg02650017), which are implicated in predicting T2D risk [ 42 ]; increased methylation in ABCG1 correlates with a higher T2D risk, while heightened PHOSPHO1 methylation is linked to a reduced risk. In our study, the vegan group displayed a significant increase in methylation at the ABCG1 loci (Δ beta value mean = 0.0105, p = 0.0093, Fig.  5 A), indicating a potentially elevated T2D risk. Concurrently, an increase in PHOSPHO1 cg02650017 methylation (Δ beta value mean = 0.0079, p = 0.011, Fig.  5 B) suggests a decreased T2D risk for the vegan cohort. This dichotomy in methylation changes for the two loci within the vegan group underscores a complex relationship between diet and T2D biomarkers, necessitating further investigation for a comprehensive understanding. None of these CpG sites were differentially methylated over time in the omnivore group.

Boxplots showing the relative beta value change of two weight loss methylation sites on the ABCG1 gene (reported on the left) and PHOSPHO1 gene (reported on the right). On the X -axis, the time points of measurements in weeks, and the loci beta value which is reported on the Y -axis. On the top, the mean and median beta values of the loci at each time point. The p -values of the paired Wilcoxon-rank sums test are also displayed in the plots. Lines that are connecting both boxplots represent the average of each patient’s tests

Analysis of EpiScore markers

Recent efforts have expanded DNA methylation proxies to predict proteins, complex behavioral, and physiological traits [ 13 , 14 , 26 ]. To this end, we utilized these DNAm-based surrogate markers to assess relative changes in response to diet type (Additional file 1 : Table S1). In an initial analysis, we utilized EpiScore models previously described: multivariable linear models of beta values used to predict the estimates of the 116 modeled proteins, behavioral and physiological traits [ 26 ]. Comparison of EpiScore values between time 8 and time 0 samples detected significant changes in seven EpiScores in the Vegan group, using a unadjusted p <0.05: CCL21, MMP1, ENPP7, Testican 2, ADAMTS, CD163, and MMP2. Notably, these EpiScores were not evident in the omnivore group analysis, underscoring diet-specific variations. Conversely, in the omnivore-specific analysis, six EpiScores—Ectodysplasin A, PAPP-A, VEGFA, HGF, Body Fat %, and TNFRSF17—exhibited exclusive significant change at an unadjusted threshold of p < 0.05. However, it must be noted that in both analyses, none of the EpiScore values met a multi-comparison corrected significance threshold (adjusted BH, p < 0.05). In summary, the methylation-based surrogate markers of complex physiological and behavioral traits identified here suggest that while common markers are present, the majority of changes among the EpiScore are unique among diet types.

Analysis of Epigenetic Biomarker Proxies (EBP)

We also assessed changes in EBPs: DNAm proxy scores of metabolites, proteins, and clinical values estimated using multivariate linear models composed of DNA methylation values that were previously described [ 14 ]. Of the 396, we identified a total of 76 and 89 EBPs which showed significant changes among the vegans and omnivores, respectively, using an unadjusted p < 0.05 (Additional file 2 : Table S2). After correcting for multiple comparisons (BH < 0.05), 13 and 19 EBPs satisfied the adjusted threshold. In the following independent analyses were performed between vegan and omnivore diets respectively to identify EBPs which showed (1) unique changes among diet types, (2) consistent changes among diet types, and (3) opposing changes among diet types.

We identified 33 EBPs that showed uniquely significant changes (unadjusted p < 0.05) among the vegan cohort: androsterone glucuronide , homovanillate (HVA) , branched-chain , straight-chain , or cyclopropyl 10:1 fatty acid (2)* , Liver albumin , CCL18 , PON1 , dehydroepiandrosterone sulfate (DHEA-S) , PON1 , glutamine_degradant , leucine , 1 , 5-anhydroglucitol (1,5-AG) , CRP , arabitol/xylitol , retinol (vitamin A) , 3-hydroxyindolin-2-one sulfate , 2-methylcitrate/homocitrate , deoxycholic acid glucuronide , 7-hydroxyindole sulfate , alpha-CMBHC glucuronide , PCOC1 , riboflavin (vitamin B2) , 1-palmitoyl-GPC (16:0) , PCOC1 , GRN , S-carboxyethylcysteine , FETUA , CSPG2 , dimethyl sulfone , carotene diol (2) , guanidinosuccinate , 6-oxopiperidine-2-carboxylate . Among these, 3 EBPs - androsterone glucuronide , homovanillate (HVA) , branched-chain , straight-chain , or cyclopropyl 10:1 fatty acid (2)* - further passed an adjusted p-value threshold (BH < 0.05), suggesting that the EBPs identified here represent potential biomarkers uniquely altered in response to a vegan diet at 8 weeks.

Among omnivores, we observed 46 EBPs which showed significant changes only among the omnivore diet cohort: 4-methoxyphenol sulfate , N-methylpipecolate , N-acetylcitrulline , sucrose , vanillactate , uridine , N-acetyltyrosine , 3-hydroxybutyroylglycine , Liver_ALP , tryptophan , dihydroferulic acid sulfate , salicyluric glucuronide* , picolinate , 3,5-dichloro-2,6-dihydroxybenzoic acid , urea , galactonate , thyroxine , 2-acetamidophenol sulfate , cystathionine , sphinganine-1-phosphate , choline phosphate , picolinoylglycine , N,N,N-trimethyl-5-aminovalerate , 1-pentadecanoyl-GPC (15:0)* , TLL1 , PCOC1 , glycochenodeoxycholate 3-sulfate , trans-4-hydroxyproline , gentisate , catechol glucuronide , citramalate , ferulic acid 4-sulfate , PLMN , sedoheptulose , vanillic acid glycine , PCOC1 , BMP1 , linoleoylcarnitine (C18:2)* , 1-methylguanidine , isobutyrylcarnitine (C4) , indolebutyrate , hypoxanthine , Smoking_PackYears , 3-hydroxyoctanoylcarnitine (1) , eicosenoylcarnitine (C20:1)* , and BMP1 . Among these, 8 EBPs passed an adjusted p -value threshold (BH < 0.05), with 4-methoxyphenol sulfate , N-methylpipecolate , N-acetylcitrulline , sucrose , vanillactate , uridine , and N-acetyltyrosine exhibiting a significant increase among the omnivore group at week 8, and a significant decrease in uridine and 3-hydroxybutyroylglycine . These EBPs represent biomarkers uniquely associated with the omnivore diet but not vegan diet.

We also identified several EBPs which showed consistent changes among the diet types. Approximately 16 of the vegan EBPs showed significant increase in both vegan and omnivore diet types which included CCL16, glucuronide of C12H22O4 (2)*, 2-methoxyhydroquinone sulfate (1), adenosine , lactosyl-N-palmitoyl-sphingosine (d18:1/16:0) , 1-stearoyl-2-dihomo-linolenoyl-GPC (18:0/20:3n3 or 6)* , N-acetylalliin , N-carbamoylalanine , caffeine , carnitine , 1-palmitoyl-2-arachidonoyl-GPE (16:0/20:4)* , FETUA , 2,3-dihydroxy-2-methylbutyrate , LYSC , eicosenedioate (C20:1-DC)* , and 1-methyl-5-imidazoleacetate) . Conversely, approximately 21 exhibited decreases among both diets, which included 10-undecenoate (11:1n1) , 1,2-dipalmitoyl-GPC (16:0/16:0) , 3-carboxy-4-methyl-5-propyl-2-furanpropanoate (CMPF) , salicylate , succinylcarnitine (C4-DC) , 1-margaroyl-2-arachidonoyl-GPC (17:0/20:4)* , 5-methyluridine (ribothymidine) , Glucose , 2-aminoheptanoate , stearoyl-arachidonoyl-glycerol (18:0/20:4) [ 1 ] *, PCOC1 , proline , ibuprofen , 11-ketoetiocholanolone glucuronide , homoarginine , Triglyceride , PCOC1 , PCOC1 , 1-stearoyl-2-adrenoyl-GPC (18:0/22:4)* , BMI , and 3-hydroxyphenylacetoylglutamine ). These EBPs represent surrogate markers of metabolite, clinical, and proteins which changed regardless of diet type, suggesting these as non-diet associated EBP markers.

However, we observed 6 EBPs which showed opposing changes in EBP levels: serine , 1-margaroyl-GPE(17:0)* , and 4-acetamidophenol , showed significant increases among vegans, and significant decreases among omnivores, while ergothioneine , indoleacetylglutamine , and creatinine showed a significant decrease among vegans compared to the increase observed among omnivores. The significant, and opposing, changes between diets suggest that these represent diet-based interactions significant in one diet but not the other.

Assessing congruence among BMI and BMI-EBP measures

To better assess the reproducibility of the EBPs calculating clinical measures, we compared the BMI-EBP changes relative to the BMI-clinical values that were collected within this study. First, we assessed the correlation between all BMI-clinical values with the BMI-EBP counterparts, which resulted in significant correlations among the values ( ⍴ = 0.275, p = 0.0022) and negligible difference in mean difference (Cohen’s d = 1.10e−16, 95% CI = −0.252–0.252) after scaling. Analysis of the longitudinal data identified that both BMI measurements showed consistent significant decreases in both diet types ( p < 0.05). However, the magnitude of change was higher in the BMI-clinical values compared to the BMI-EBP values (Fig.  6 ). Taken together, these findings exhibit the reproducibility of the BMI metrics among the EBPs relative to their clinical counterparts.

Boxplot showing the evolution of BMI values calculated from clinical measures (reported on the left) and epigenetic biomarker proxy (EBP) measures (reported on the right). On the X -axis, the time points of measurements in weeks. On the Y -axis, the BMI measure. The BMI-EBP measurements are reported as residuals, which are defined as the residual of the raw BMI value regressed upon chronological age, and adjusted by sex, technical principal components 1 and 2. No residual calculation was done for the clinical EBP. On the top, the mean and median values of the BMI at each time point. The p -values of the paired Wilcoxon-rank sums test are also displayed in the plots. Lines that connect both boxplots represent the average of each patient’s tests

Global EWAS analysis identifies epigenetic markers of vegan and omnivorous diets

We utilized an exploratory epigenome-wide analysis approach of 866,836 CpGs to identify candidate differentially methylated loci associated with a vegan or omnivore diet. To run the correct EWAS model for each comparison, we first tested for test-statistic inflation (lambda) with each EWAS model adjusted by different fixed effects [ 27 ]. The final models ultimately chosen reported lambdas closest to 1 in each of the comparison: within vegan (week 0 vs week 8) lambda chosen is 0.97; within omnivore (week 0 vs week 8) lambda chosen is 0.89; cross-sectional week 8 comparison lambda chosen is 1.03; and cross-sectional week 0 comparison chosen is 1.06.

Utilizing the optimal EWAS models, differentially methylated loci DMLs were identified. In the first comparison, we identified a total of 607 differentially methylated loci (DMLs) associated with 8 weeks of a vegan diet ( p -value < 0.001) compared to week 0 (Fig.  7 A). Among these vegan-diet associated loci, 322 CpG sites showed hypomethylation at 8 weeks, and 312 loci exhibited hypermethylation at week 8. Among the omnivore cohort, a total of 494 DMLs were associated with 8 weeks of an omnivore diet ( p -value < 0.001) (Fig.  7 B), in which 309 CpGs showed increases in DNA methylation and 185 CpGs exhibited loss in DNA methylation at week 8. The full list of DMLs associated with 8 weeks of a vegan or omnivore diet is listed in Additional file 3 : Tables S3 and S4 for both analyses. The DMLs identified here represent potential methylation markers of specific dietary interventions in response to the consumption of vegan diet or omnivorous diet, respectively.

Manhattan plots for the vegan and the omnivore epigenome-wide association studies. The Manhattan plot illustrates genes associated with CpG sites identified in the A vegan and B omnivore comparison, with each dot representing a CpG site and its vertical position corresponding to the negative logarithm (base 10) of the unadjusted p -value for DNA methylation association (significance set at p = 0.001). The x -axis denotes genomic positions organized by chromosomes, with color-coded dots indicating specific chromosomes, and prominently peaked dots represent significantly associated CpG sites surpassing the genome-wide significance threshold

To better understand the specific DNA methylation patterns that differentiated vegan diet samples and omnivorous diets, a cross-sectional analysis comparing these groups at the week 8 time points was conducted. We identified a total of 980 DMLs that were differentially methylated between the participants on an omnivore diet at week 8 and the participants on a vegan diet at week 8 ( p -value < 0.001). Of the DMLs identified, 317 exhibited hypermethylation in the week 8 vegan samples relative to the week 8 omnivore samples, while 663 DMLs exhibited hypomethylation in the week 8 vegan sample (or greater methylation in the omnivore group) (Fig.  8 , Additional file 3 : Table S5). Similarly, a cross-sectional analysis at week 0 was also conducted to identify the base difference in methylation between the vegan and omnivore twins at the time of starting the trial. A total of 834 DMLs were identified between the diets at week 0, with 385 hypermethylated loci in the vegan samples compared to the omnivores (average logFC difference of 0.498), and 452 hypomethylated DMLs (or greater in omnivores compared to vegans) exhibiting an average logFC difference of −0.355. Baseline DMLs represent methylation differences of twins at their base and are reported in Additional file 3 : Table S6.

Volcano plot of DMLs identified in the comparison between vegan and omnivore diet at the week 8 time point. The volcano plot illustrates DMLs identified in the Vegan vs. Omnivore comparison, with each dot representing a CpG site and its vertical position corresponding to the negative logarithm (base 10) of the unadjusted p -value for DNA methylation association. The x -axis denotes the relative log fold change (logFC) of the m -values between the vegan and the omnivore diets. Values greater than 0 represent CpGs with greater methylation among vegans (blue), compared to the negative values which represent greater methylation among omnivores (red)

Finally, to answer if the CpGs identified at week 8 were uniquely differentially methylated to the identified week 0 DMLs, we compared the cross-sectional comparison lists. Only 2 CpGs (cg04227789, cg18301717), or 0.2%, overlapped in both comparisons, suggesting that the DMLs identified at week 8 are likely due to a diet-based effect.

Gene ontology pathway analysis of diet-associated DMLs

To better understand the transcriptionally relevant biological processes associated with methylation changes, gene ontology (GO) enrichment analyses were conducted among DMLs identified in the week 8 vegan and omnivore comparison. To ensure transcriptional relevance, DMLs were inputted into the GREAT software by direction of methylation and CpGs overlapping with cis-regulatory or other regulatory regions were linked to genes and thus assessed for their relationship to various biological processes (BP), molecular function (MF), and cellular component (CC) associations. Significantly hypermethylated DMLs in the vegan group, or hypomethylated among omnivores, were reported as significantly enriched for GO-BP terms such as paracrine signaling , response to beta-amyloid , neuron apoptosis , and several developmental GO-BP terms (adjusted BH p -value < 0.05). In addition, molecular function (GO-MF) terms such as Ras guanyl-nucleotide exchange factor activity were enriched for sites that exhibited significant hypermethylation among the vegans, and lower among the omnivores (adjusted BH p -value < 0.05, Additional file 4 : Table S7). CpGs that were hypermethylated among omnivores, and hypomethylated among vegans, were enriched for GO-BP terms associated with cell cycle (negative regulation of the G0-G1 transition), genomic imprinting ( regulation of gene expression by genetic imprinting ), cytosolic calcium ion transport , and cellular response to alcohol. Cell cycle and transcriptional activity were further supported by the enrichment of GO-MF terms associated with RNA polymerase activity and transcriptional processes ( protein phosphatase inhibitor activity , RNA polymerase II regulatory region DNA binding , and promoter-specific chromatin binding ). Full results for biological associations to CpGs differentially methylated between diet types are listed in Additional file 4 : Table S8. In summary, the significant gene ontology terms identified reveal distinct associations with key biological processes and molecular functions, shedding light on the epigenetic mechanisms altered in response to dietary choices.

In this study, we sought to elucidate the impact of a “healthy vegan” or a “healthy omnivorous diet” on epigenetic age, telomere length, immune cell subsets, and type 2 diabetes (T2D) risk-associated CpGs, building on current knowledge of nutrition on both diets. Our findings reveal distinct responses to vegan and omnivore diets, aligning with existing literature on the subject. Notably, the vegan cohort exhibited a significant decrease in epigenetic age acceleration, as demonstrated by reductions in multiple epigenetic aging clocks, all of which were trained upon clinical [ 28 ] and phenotypic scores (PC GrimAge, PC PhenoAge, 28-31). The usage of systems-specific aging predictors further specified which organ systems showed age improvements, resolving five specific systems that showed aging improvements among the vegan cohort and not omnivores. These findings are consistent with previous research highlighting the potential anti-aging effects of plant-based diets, known for their rich antioxidant content and anti-inflammatory properties [ 43 , 44 , 45 ]. However, the significant impact of basophils in the vegan group contrasts with studies emphasizing the immunomodulatory benefits of plant-based diets, suggesting that further investigation into the nuanced interactions is warranted [ 46 ]. These comprehensive findings underscore the complex interplay between diet, epigenetic regulation, immune function, and metabolic health, offering valuable insights for future research and personalized health interventions.

The measures investigated in our study offer a holistic perspective on biological aging without isolating system-specific aging processes, as highlighted by Ahadi in 2020 [ 47 ]. However, the incorporation of the systems age clock in our research addresses this limitation by providing valuable, system-specific insights into aging changes [ 34 ]. Notably, our findings reveal significant reductions in key system-specific disease processes, including inflammation, heart, liver, metabolic, and hormonal systems. This nuanced approach aligns with previous research demonstrating that vegan and plant-based diets are associated with lower levels of inflammatory markers [ 46 ], lower risk of cardiovascular diseases [ 48 , 49 ], reduced risk of non-alcoholic fatty liver disease (NAFLD) [ 50 ], improve glycemic control and other metabolic factors in individuals with type 2 diabetes [ 51 ], and regulated hormonal level outputs in responses such as hot flashes [ 52 ]. This approach allows for a more comprehensive understanding of the impact of the studied interventions on specific facets of aging, shedding light on potential areas of targeted intervention for promoting overall health and longevity. The identification of these system-specific changes contributes to a more nuanced and actionable comprehension of the aging process, underscoring the significance of our results in advancing our knowledge of interventions that may influence distinct physiological systems and enhance overall well-being.

One notable difference we observed was the magnitude of telomere length change within the vegan diet, where qPCR-TL analysis identified a statistically significant increase in telomere length, while PC DNAmTL exhibited an insignificant increase. While this finding is consistent with previous investigations that have reported mixed congruency between qPCR-TL and PC DNAmTL values [ 53 , 54 , 55 ], our assessment of telomere congruency between the two methods showed moderate correlation ( ⍴ > 0.56) and no significant difference between PC DNAmTL and qPCR-TL, as evidenced by the Wilcoxon-rank sum and Cohen’s d tests. This suggests that the incongruency between the telomere length changes observed between the two methods could be attributed more to the different signals of telomere biology captured, such as telomere maintenance mechanisms and not telomere length [ 54 ].

While there is no gold standard measure of biological aging [ 56 ], we analyzed several measures that represent the current DNAm predictors of biological aging. Nevertheless, these measures are acknowledged to be incomplete summaries of biological changes that occur with aging and to have technical limitations [ 57 , 58 ]. Treatment effects on aspects of biological aging not captured by the DNAm measures are not included in effect estimates; measurement error due to technical limitations of DNAm assays may bias effect estimates towards the null. Treatment effect estimates may therefore represent a lower bound of the true impact of vegan or omnivore dietary intervention on biological aging.

A notable contribution of this study is the assessment of Epigenetic Biomarker Proxies (EBP), which were previously described [ 14 ]. Firstly, the notable consistency in significant decreases observed in both BMI-EBP and BMI-clinical values across diet types highlights the reproducibility of BMI metrics within the epigenetic context. Despite a slightly higher magnitude of change in BMI-clinical values, the parallel trends emphasize the reliability of BMI-EBPs as reflective markers of body mass index alterations. Secondly, six EBPs exhibited divergent alterations between the vegan and omnivore diets, shedding light on diet-specific impacts on the epigenome. Ergothioneine, indoleacetylglutamine, and creatinine demonstrated a noteworthy decrease in the vegan group but an increase in the omnivore group. Ergothioneine, a potent antioxidant guarding cells against oxidative stress, potentially decreased in the vegan diet due to reduced intake from sources like mushrooms and certain grains [ 44 ]. Indoleacetylglutamine, derived from tryptophan, showcased elevated levels in the omnivore diet and a decline in the vegan diet, possibly mirroring the distinct abundance of protein-rich foods in each diet. The analogous patterns in creatinine, a marker of muscle metabolism, might also be linked to variations in protein intake and muscle turnover between the two diets. Conversely, serine, 1-margaroyl-GPE(17:0), and 4-acetamidophenol saw a significant rise in the vegan group but a decrease in the omnivore group. Serine, a non-essential amino acid abundant in plant sources, such as soybeans and nuts, likely increased on the vegan diet due to elevated consumption. The opposite trends in 1-margaroyl-GPE(17:0), a relatively novel metabolite predicted to function as a glycerophospholipid involved in cellular membranes and signaling pathways, suggest diet-induced variations in membrane composition and function. 4-acetamidophenol, a derivative of paracetamol widely used in analgesic and antipyretic medications, may reflect increased usage in the vegan compared to the omnivore group. Further studies are needed to identify the health implications of these changes and whether specific dietary components are responsible for them. Thirdly, the analysis of the previously published EpiScores provided insights into the potential of DNA methylation markers for predicting complex physiological and behavioral traits influenced by diet [ 37 ]. While seven EpiScores showed small effect changes exclusively in the vegan group and six exhibited exclusive significance in the omnivore group, the failure to achieve the corrected p -value provides an avenue for further interrogation of their utility in interventional data. However, the EpiScores identified by the uncorrected p -value threshold do act as targets for further assessment in clinical and lab-based protein studies. Nevertheless, the significant changes in EBP values highlight the potential of DNA methylation-based surrogate markers in delineating diet-related impacts on complex traits. This underscores the necessity for further exploration to refine and validate these markers for their predictive utility.

Several metabolites EBPs exhibited noteworthy changes, providing insights into differences and commonalities of diet response between the two groups. Among the top markers showing significant alterations in the vegan group, C-reactive protein (CRP), deoxycholic acid glucuronide, and spermidine stood out. A decrease in predicted CRP levels suggests a potential reduction in systemic inflammation. Spermidine, a polyamine associated with various health benefits, demonstrated an increase, potentially indicating an increased intake of vegetables like soy, legumes, and mushrooms. Deoxycholic acid glucuronide, a bile acid metabolite, displayed a decrease, suggesting an expected potential reduction in bile acid metabolism in response to a reduced intake of animal fat. Additionally, the vegan group demonstrated significant changes in other markers, such as N-acetyl-cadaverine and carnitine. Whereas the elevated levels of N-acetyl-cadaverine decreased as expected, given that this marker is associated with amino acid fermentation in the gut, the increase in carnitine levels contradicts the anticipated decrease in response to a vegan diet, since carnitine is mainly derived from meat and dairy products [ 59 ].

Several metabolites exhibited significant decreases in both diet groups, pointing to shared metabolic responses across diverse dietary patterns. Both salicylate, a component found in various plant foods, and its metabolite salicyluric glucuronide, demonstrated a reduction in both groups potentially reflecting a decrease in salicylate rich food such as legumes (e.g., lentils, beans), vegetables (e.g., cauliflowers, pickled vegetables), and fruits (e.g., strawberries, plums, watermelons). Reductions in quinate, a compound derived from the metabolism of coffee polyphenols [ 60 , 61 ] and 10-undecenoate (11:1n1), a fatty acid related to butter intake [ 62 , 63 ], suggest potential reduction in coffee and butter intake, respectively. Interestingly, both groups exhibited a decrease in predicted body mass index (BMI), which is consistent with the decrease in BMI in both groups.

In the omnivore group, we observed several intriguing shifts in key metabolic markers. The increase in tryptophan and serotonin, a neurotransmitter synthesized from tryptophan, suggests potential impacts on mood regulation and other serotonin-mediated functions in response to increased intake of tryptophan-rich animal protein in the omnivore diet. Choline phosphate, a vital component in cell membrane structure, exhibited an increase, hinting at increased dietary intake from meat, fish, and eggs. Indolebutyrate, a microbial metabolite, displayed an increase, suggesting potential shifts in gut microbial metabolism influenced by the diverse dietary components. Adenosine, a nucleoside that promotes sleep and reduces anxiety, exhibited an increase, indicating potential changes in endogenous metabolism on an omnivore diet [ 64 ]. These findings underscore the nuanced interplay of neurotransmitter synthesis, lipid metabolism, microbial activity, and purine metabolism associated with omnivorous dietary patterns.

Previous studies have suggested vegan diets associated with lower T2D risk [ 40 , 41 ]. Interestingly, our investigation into T2D risk-associated methylation loci revealed that the vegan diet led to increased methylation in ABCG1 and PHOSPHO1 , which provided relatively conflicting results; increase in ABCG1 indicates a reduced T2D risk, which is contradicted with the increase in PHOSPHO1 , which indicates increased T2D risk. These results call for the need to develop disease-specific epigenetic predictors for T2D risk which go beyond single loci risk predictors, to potential multi-loci risk predictors exhibit significant association to disease risk.

Finally, the exploration of global DNA methylation patterns across the entire epigenome revealed significant differences between the vegan and omnivore cohorts, and identified 607 and 494 differentially methylated loci (DMLs) across the genome, respectively. Notably, the models accounted for potential confounding factors such as BMI, age, and sex, making it likely that these DMLs are more closely associated with the diet change. This comprehensive epigenome-wide analysis aligns with a growing body of literature examining the epigenetic effects of different dietary patterns [ 65 , 66 , 67 ]. When we analyzed each diet group independently, we observed 322 hypomethylated probes in the vegan diet and 185 in the omnivore diet. These CpG sites represent the epigenetic targets that changed during the trial, but independent of diet. However, to compare the evolution of each of the twin pairs, we compared the week 8 time point for those individuals in the vegan diet and those in the omnivore diet. This analysis unraveled 980 DMLs, with 317 demonstrating higher methylation in the vegan group and 663 in the omnivore group. Using the significant CpG sites from the twin-pair comparison at week 8, we performed an enrichment analysis to elucidate the biological relevance of these methylation patterns. Hypermethylated sites in vegans revealed enrichment of paracrine signaling, response to beta-amyloid, neuron apoptosis, and developmental processes. These findings imply that a vegan diet may influence pathways associated with cellular communication, neuroprotective mechanisms, and development. In contrast, hypermethylation in omnivores was linked to cell cycle regulation, genomic imprinting, cytosolic calcium ion transport, and cellular response to alcohol. This suggests that an omnivorous diet may impact pathways related to cell division, genetic regulation, cellular signaling, and responses to environmental stimuli. These insights contribute to a deeper understanding of how diet can impact the epigenome and, consequently, influence various aspects of cellular activity and health outcomes. Future investigations linking the epigenetic sites identified here in the context of gene expression may identify gene regulatory networks altered due to diet, further providing a molecular perspective in nutrition and diet.

Our exploratory longitudinal differential methylation analyses were focused on identifying candidate DNA methylation loci associated with 8 weeks of a vegan or herbivore diet. Hence, we utilized a more stringent P value cutoff of less than 0.001 which has been utilized by other EWAS studies [ 68 , 69 ]. Our differential methylation analyses also controlled for twin structure and other potential confounding factors of age, sex, BMI, batch, immune cell composition, and accounted for the repeated measures by considering twin pairs as a random effect. However, this approach may have identified DML by chance and is a limitation of this approach compared to more stringent false discovery rate correction of all CpG loci. Due to the limited sample size of our study, when p -values were adjusted for multiple correction using the false-discovery rate method as has been utilized in large epigenome-wide association studies, in all three comparisons, this approach appeared too conservative as no DMLs were identified. Future studies are needed to validate DML associated with vegan and herbivore diets.

It is crucial to acknowledge that the observed epigenetic age and biomarker differences between the vegan and omnivore groups may be predominantly attributed to the variations in weight loss rather than solely reflecting the distinct dietary compositions. Throughout the “Food Delivery” phase, the vegan group consumed ~ 200 calories less per day than their omnivorous counterparts, resulting in an average weight loss of 2 kg greater than the omnivore group by the end of the 8-week intervention. Extensive population studies and Mendelian randomization analyses have underscored the impact of BMI changes on inducing epigenetic alterations linked to metabolic health [ 70 , 71 ]. However, it should be noted that while we saw significant decreases in both clinical-BMI and EBP-BMI values, only the vegan cohort exhibited significant reductions in epigenetic age. This calls for a nuanced interpretation of our findings and emphasizes the need for future investigations to disentangle the complex interrelationships between dietary factors, weight dynamics, and epigenetic modifications.

While our study provides valuable insights into the short-term effects of weight loss on two different diets on epigenetic markers, it is important to acknowledge that the long-term impact of a vegan diet on epigenetic processes may carry adverse effects in the absence of sufficient intake of crucial vitamins and nutrients essential for supporting these intricate molecular reactions. In particular, all vegans and a substantial portion of vegetarians, if not supplemented, are at risk of developing vitamin B12 deficiency, resulting in elevated levels of homocysteine—an established marker of dysfunctional methylation associated with increased cardiovascular risk, including coronary artery disease (CAD) and heightened stroke susceptibility [ 72 , 73 , 74 ]. Vitamin B12 deficiency has been implicated in disease-related epigenetic alterations in both animals and humans [ 75 , 76 , 77 , 78 ]. In our cohort, the vegan group exhibited a lower intake of vitamin B12, although serum vitamin B12 levels did not demonstrate statistical differences compared to omnivores at the 8-week mark, likely due to preserved stores [ 21 ]. It is crucial to emphasize that long-term adherence to vegan diets typically necessitates vitamin B12 supplementation to mitigate the risk of deficiency and its consequential impact on epigenetic processes. Furthermore, the vegan cohort exhibited lower caloric intake, consumed less saturated fats, more polyunsaturated fats, and more fiber than the omnivorous group, suggesting these as the potential drivers of age reductions, rather than the vegan diet only [ 21 ]. This highlights the imperative role of nutritional considerations in optimizing the health outcomes associated with plant-based dietary choices. Within the context of these limitations, our findings have implications for future geroscience research. Aging biology research has identified multiple therapies with the potential to improve healthy lifespan in humans. A barrier to advancing the translation of these therapies through human trials is that intervention studies run for months or years, but human aging takes decades to cause disease [ 79 , 80 , 81 ].

We also acknowledge the potential for differences in behavior and lifestyle factors which may have impacted the study findings here. As previously described (21, online Supplement 2, eAppendix), majority of factors which may alter methylation changes were controlled for among the individuals who participated in the trial: routine dietary checks throughout the duration of the trail, a fixed checklist for diet adherence, and assessment of diet adherence at the end of each 4-week phase. In one sensitivity analysis which identified non-normal changes in a group of twins featured in a documentary compared to the rest of the group. These analyses identified differences in TMAO levels in a set of twins which were removed from the study, indicating a potential confounding factor of non-adherence to the preset diet which was corrected for. A subset of twins ( N = 4) contributed to the filming of a documentary and thus were encouraged to exercise more, which may affect caloric outputs and thus epigenetic changes [ 82 ]. While the current analyses accounted for the large effects using a pair-wise and random effect statistical design, minor effects in the cross-sectional analyses may not have been accounted for.

In this epigenetic analysis of an initial randomized clinical trial, we observed significant changes using epigenetic age clocks among healthy identical twins, suggesting short-term advantageous aging benefits for a calorie-restricted vegan diet compared to an omnivorous diet. The use of EBPs in this study showcases the potential of epigenetic testing to provide personalized insights into the impact of nutrition on cellular aging, enabling targeted dietary interventions to optimize health and well-being. Differential methylation analysis of diet type identified methylation changes unique to each diet implementation, potentially representing methylation markers of diet. However, it is still uncertain whether the observed benefits may be primarily due to greater weight loss in the vegan group; thus long-term effects of unsupplemented vegan diets on epigenetic processes require further investigation. Future research utilizing a long-term, well-controlled study design will further highlight the complex relationships between diet, epigenetics, and health outcomes such as weight loss, while emphasizing the importance of proper nutrient supplementation in vegan diets.

Availability of data and materials

The data that support the findings of this study are not publicly available due to protection of patient data in accordance to maintaining HIPAA compliance. However, the corresponding authors can provide the data upon reasonable request after signing a Data Use Agreement.

Jin K, Simpkins JW, Ji X, Leis M, Stambler I. The critical need to promote research of aging and aging-related diseases to improve health and longevity of the elderly population. Aging Dis. 2014;6(1):1–5.

PubMed   PubMed Central   Google Scholar  

Bauman A, Merom D, Bull FC, Buchner DM, Fiatarone Singh MA. Updating the evidence for physical activity: summative reviews of the epidemiological evidence, prevalence, and interventions to promote “active aging.” Gerontologist. 2016;56(Suppl 2):S268-280.

Article   PubMed   Google Scholar  

Kennedy BK, Berger SL, Brunet A, Campisi J, Cuervo AM, Epel ES, et al. Aging: a common driver of chronic diseases and a target for novel interventions. Cell. 2014;159(4):709–13.

Article   CAS   PubMed   PubMed Central   Google Scholar  

Campisi J, Kapahi P, Lithgow GJ, Melov S, Newman JC, Verdin E. From discoveries in ageing research to therapeutics for healthy ageing. Nature. 2019;571(7764):183–92.

Sen P, Shah PP, Nativio R, Berger SL. Epigenetic mechanisms regulating longevity and aging. Cell. 2016;166(4):822–39.

Miles FL, Mashchak A, Filippov V, Orlich MJ, Duerksen-Hughes P, Chen X, et al. DNA methylation profiles of vegans and non-vegetarians in the adventist health study-2 cohort. Nutrients. 2020;12(12): 3697.

Craig WJ. Nutrition concerns and health effects of vegetarian diets. Nutr Clin Pract. 2010;25(6):613–20.

Trichopoulou A, Vasilopoulou E. Mediterranean diet and longevity. Br J Nutr. 2000;84(Suppl 2):S205-209.

Article   CAS   PubMed   Google Scholar  

Roberts MN, Wallace MA, Tomilov AA, Zhou Z, Marcotte GR, Tran D, et al. A ketogenic diet extends longevity and healthspan in adult mice. Cell Metab. 2017;26(3):539-546.e5.

Capurso C, Bellanti F, Lo Buglio A, Vendemiale G. The Mediterranean diet slows down the progression of aging and helps to prevent the onset of frailty: a narrative review. Nutrients. 2019;12(1): 35.

Article   PubMed   PubMed Central   Google Scholar  

Coelho-Junior HJ, Marzetti E, Picca A, Cesari M, Uchida MC, Calvani R. Protein intake and frailty: a matter of quantity, quality, and timing. Nutrients. 2020;12(10): 2915.

Higgins-Chen AT, Thrush KL, Wang Y, Minteer CJ, Kuo PL, Wang M, et al. A computational solution for bolstering reliability of epigenetic clocks: implications for clinical trials and longitudinal tracking. Nat Aging. 2022;2(7):644–61.

McCartney DL, Hillary RF, Stevenson AJ, Ritchie SJ, Walker RM, Zhang Q, et al. Epigenetic prediction of complex traits and death. Genome Biol. 2018;19(1):136.

Chen Q, Dwaraka VB, Carreras-Gallo N, Mendez K, Chen Y, Begum S, Kachroo P, Prince N, Went H, Mendez T, Lin A. OMICmAge: an integrative multi-omics approach to quantify biological age with electronic medical records. bioRxiv. 2023.

Luo Q, Dwaraka VB, Chen Q, Tong H, Zhu T, Seale K, et al. A meta-analysis of immune-cell fractions at high resolution reveals novel associations with common phenotypes and health outcomes. Genome Med. 2023;15(1):59.

Teschendorff AE. A comparison of epigenetic mitotic-like clocks for cancer risk prediction. Genome Med. 2020;12(1):56.

Rutledge J, Oh H, Wyss-Coray T. Measuring biological age using omics data. Nat Rev Genet. 2022;23(12):715–27.

Gensous N, Garagnani P, Santoro A, Giuliani C, Ostan R, Fabbri C, et al. One-year Mediterranean diet promotes epigenetic rejuvenation with country- and sex-specific effects: a pilot study from the NU-AGE project. GeroScience. 2020;42(2):687–701.

Galkin F, Kovalchuk O, Koldasbayeva D, Zhavoronkov A, Bischof E. Stress, diet, exercise: common environmental factors and their impact on epigenetic age. Ageing Res Rev. 2023;88: 101956.

Kim K, Zheng Y, Joyce BT, Jiang H, Greenland P, Jacobs DR, et al. Relative contributions of six lifestyle- and health-related exposures to epigenetic aging: the Coronary Artery Risk Development in Young Adults (CARDIA) study. Clin Epigenetics. 2022;14(1):85.

Landry MJ, Ward CP, Cunanan KM, Durand LR, Perelman D, Robinson JL, et al. Cardiometabolic effects of omnivorous vs vegan diets in identical twins: a randomized clinical trial. JAMA Netw Open. 2023;6(11):e2344457.

Cawthon RM. Telomere measurement by quantitative PCR. Nucleic Acids Res. 2002;30(10):e47.

Lin J, Epel E, Cheon J, Kroenke C, Sinclair E, Bigos M, et al. Analyses and comparisons of telomerase activity and telomere length in human T and B cells: insights for epidemiology of telomere maintenance. J Immunol Methods. 2010;352(1–2):71–80.

Aryee MJ, Jaffe AE, Corrada-Bravo H, Ladd-Acosta C, Feinberg AP, Hansen KD, et al. Minfi: a flexible and comprehensive Bioconductor package for the analysis of Infinium DNA methylation microarrays. Bioinformatics. 2014;30(10):1363–9.

Xu Z, Niu L, Taylor JA. The ENmix DNA methylation analysis pipeline for Illumina BeadChip and comparisons with seven other preprocessing pipelines. Clin Epigenetics. 2021;13(1):216.

Xiong Z, Li M, Ma Y, Li R, Bao Y. GMQN: a reference-based method for correcting batch effects and probe bias in HumanMethylation BeadChip. Front Genet. 2022;12: 810985.

Horvath S. DNA methylation age of human tissues and cell types. Genome Biol. 2013;14(10):3156.

Article   Google Scholar  

Horvath S, Oshima J, Martin GM, Lu AT, Quach A, Cohen H, et al. Epigenetic clock for skin and blood cells applied to Hutchinson Gilford Progeria Syndrome and ex vivo studies. Aging. 2018;10(7):1758–75.

Hannum G, Guinney J, Zhao L, Zhang L, Hughes G, Sadda S, et al. Genome-wide methylation profiles reveal quantitative views of human aging rates. Mol Cell. 2013;49(2):359–67.

Levine ME, Lu AT, Quach A, Chen BH, Assimes TL, Bandinelli S, et al. An epigenetic biomarker of aging for lifespan and healthspan. Aging. 2018;10(4):573–91.

Lu AT, Quach A, Wilson JG, Reiner AP, Aviv A, Raj K, et al. DNA methylation GrimAge strongly predicts lifespan and healthspan. Aging. 2019;11(2):303–27.

Lu AT, Binder AM, Zhang J, Yan Q, Reiner AP, Cox SR, et al. DNA methylation GrimAge version 2. Aging. 2022;14(23):9484–549.

CAS   PubMed   PubMed Central   Google Scholar  

Lu AT, Seeboth A, Tsai PC, Sun D, Quach A, Reiner AP, et al. DNA methylation-based estimator of telomere length. Aging. 2019;11(16):5895–923.

Sehgal R, Markov Y, Qin C, Meer M, Hadley C, Shadyab AH, Casanova R, Manson JE, Bhatti P, Crimmins EM, Hagg S. Systems age: a single blood methylation test to quantify aging heterogeneity across 11 physiological systems. bioRxiv. 2023:2023-07.

Belsky DW, Caspi A, Corcoran DL, Sugden K, Poulton R, Arseneault L, et al. DunedinPACE, a DNA methylation biomarker of the pace of aging. Deelen J, Tyler JK, Suderman M, Deelen J, editors. eLife. 2022;11:e73420.

Hillary RF, Marioni RE. MethylDetectR: a software for methylation-based health profiling. Wellcome Open Res. 2020;5:283.

Gadd DA, Hillary RF, McCartney DL, Zaghlool SB, Stevenson AJ, Cheng Y, et al. Epigenetic scores for the circulating proteome as tools for disease prediction. eLife. 2022;11: e71802.

Guintivano J, Shabalin AA, Chan RF, Rubinow DR, Sullivan PF, Meltzer-Brody S, Aberg KA, van den Oord EJ. Test-statistic inflation in methylome-wide association studies. Epigenetics. 2020;15(11):1163–6.

Gu Z, Hübschmann D. rGREAT: an R/bioconductor package for functional enrichment on genomic regions. Bioinforma Oxf Engl. 2023;39(1):btac745.

Article   CAS   Google Scholar  

Satija A, Bhupathiraju SN, Rimm EB, Spiegelman D, Chiuve SE, Borgi L, et al. Plant-based dietary patterns and incidence of type 2 diabetes in US men and women: results from three prospective cohort studies. PLoS Med. 2016;13(6): e1002039.

Tonstad S, Stewart K, Oda K, Batech M, Herring RP, Fraser GE. Vegetarian diets and incidence of diabetes in the Adventist Health Study-2. Nutr Metab Cardiovasc Dis NMCD. 2013;23(4):292–9.

Dayeh T, Tuomi T, Almgren P, Perfilyev A, Jansson PA, de Mello VD, Pihlajamäki J, Vaag A, Groop L, Nilsson E, Ling C. DNA methylation of loci within ABCG1 and PHOSPHO1 in blood DNA is associated with future type 2 diabetes risk. Epigenetics. 2016;11(7):482–8.

Wang S, Li W, Li S, Tu H, Jia J, Zhao W, et al. Association between plant-based dietary pattern and biological aging trajectory in a large prospective cohort. BMC Med. 2023;21(1):310.

Smith E, Ottosson F, Hellstrand S, Ericson U, Orho-Melander M, Fernandez C, et al. Ergothioneine is associated with reduced mortality and decreased risk of cardiovascular disease. Heart. 2020;106(9):691–7.

Ornish D, Lin J, Chan JM, Epel E, Kemp C, Weidner G, et al. Effect of comprehensive lifestyle changes on telomerase activity and telomere length in men with biopsy-proven low-risk prostate cancer: 5-year follow-up of a descriptive pilot study. Lancet Oncol. 2013;14(11):1112–20.

Turner-McGrievy GM, Wirth MD, Shivappa N, Wingard EE, Fayad R, Wilcox S, et al. Randomization to plant-based dietary approaches leads to larger short-term improvements in Dietary Inflammatory Index scores and macronutrient intake compared with diets that contain meat. Nutr Res N Y N. 2015;35(2):97–106.

Ahadi S, Zhou W, Schüssler-Fiorenza Rose SM, Sailani MR, Contrepois K, Avina M, Ashland M, Brunet A, Snyder M. Personal aging markers and ageotypes revealed by deep longitudinal profiling. Nat Med. 2020;26(1):83–90.

Orlich MJ, Singh PN, Sabaté J, Jaceldo-Siegl K, Fan J, Knutsen S, Beeson WL, Fraser GE. Vegetarian dietary patterns and mortality in Adventist Health Study 2. JAMA Intern Med. 2013;173(13):1230–8.

Kim H, Caulfield LE, Garcia-Larsen V, Steffen LM, Coresh J, Rebholz CM. Plant-based diets are associated with a lower risk of incident cardiovascular disease, cardiovascular disease mortality, and all-cause mortality in a general population of middle-aged adults. J Am Heart Assoc. 2019;8(16):e012865.

Lv Y, Rong S, Deng Y, Bao W, Xia Y, Chen L. Plant-based diets, genetic predisposition and risk of non-alcoholic fatty liver disease. BMC Med. 2023;21(1):351.

Barnard ND, Cohen J, Jenkins DJ, Turner-McGrievy G, Gloede L, Green A, Ferdowsian H. A low-fat vegan diet and a conventional diabetes diet in the treatment of type 2 diabetes: a randomized, controlled, 74-wk clinical trial. Am J Clin Nutr. 2009;89(5):1588S-1596S.

Kahleova H, Znayenko-Miller T, Uribarri J, Schmidt N, Kolipaka S, Hata E, Holtz DN, Sutton M, Holubkov R, Barnard ND. Dietary advanced glycation end-products and postmenopausal hot flashes: a post-hoc analysis of a 12-week randomized clinical trial. Maturitas. 2023;172:32–8.

Pearce EE, Horvath S, Katta S, Dagnall C, Aubert G, Hicks BD, Spellman SR, Katki H, Savage SA, Alsaggaf R, Gadalla SM. DNA-methylation-based telomere length estimator: comparisons with measurements from flow FISH and qPCR. Aging (Albany NY). 2021;13(11):14675.

Hastings WJ, Etzel L, Heim CM, Noll JG, Rose EJ, Schreier HM, Shenk CE, Tang X, Shalev I. Comparing qPCR and DNA methylation-based measurements of telomere length in a high-risk pediatric cohort. Aging (Albany NY). 2022;14(2):660.

Safaee MM, Dwaraka VB, Lee JM, Fury M, Mendez TL, Smith R, Lin J, Smith DL, Burke JF, Scheer JK, Went H. Epigenetic age biomarkers and risk assessment in adult spinal deformity: a novel association of biological age with frailty and disability. J Neurosurg. 2023;1(aop):1–2.

Google Scholar  

Ferrucci L, Gonzalez-Freire M, Fabbri E, Simonsick E, Tanaka T, Moore Z, Salimi S, Sierra F, de Cabo R. Measuring biological aging in humans: a quest. Aging Cell. 2020;19(2):e13080.

Belsky DW, Domingue BW, Wedow R, Arseneault L, Boardman JD, Caspi A, Conley D, Fletcher JM, Freese J, Herd P, Moffitt TE. Genetic analysis of social-class mobility in five longitudinal studies. Proc Natl Acad Sci. 2018;115(31):E7275-84.

Bell CG, Lowe R, Adams PD, Baccarelli AA, Beck S, Bell JT, Christensen BC, Gladyshev VN, Heijmans BT, Horvath S, Ideker T. DNA methylation aging clocks: challenges and recommendations. Genome Biol. 2019;20:1–24.

Lombard KA, Olson AL, Nelson SE, Rebouche CJ. Carnitine status of lactoovovegetarians and strict vegetarian adults and children. Am J Clin Nutr. 1989;50(2):301–6.

Guertin KA, Loftfield E, Boca SM, Sampson JN, Moore SC, Xiao Q, Huang WY, Xiong X, Freedman ND, Cross AJ, Sinha R. Serum biomarkers of habitual coffee consumption may provide insight into the mechanism underlying the association between coffee consumption and colorectal cancer. Am J Clin Nutr. 2015;101(5):1000–11.

Playdon MC, Sampson JN, Cross AJ, Sinha R, Guertin KA, Moy KA, Rothman N, Irwin ML, Mayne ST, Stolzenberg-Solomon R, Moore SC. Comparing metabolite profiles of habitual diet in serum and urine. Am J Clin Nutr. 2016;104(3):776–89.

Pallister T, Haller T, Thorand B, Altmaier E, Cassidy A, Martin T, Jennings A, Mohney RP, Gieger C, MacGregor A, Kastenmüller G. Metabolites of milk intake: a metabolomic approach in UK twins with findings replicated in two European cohorts. Eur J Nutr. 2017;56:2379–91.

Guertin KA, Moore SC, Sampson JN, Huang WY, Xiao Q, Stolzenberg-Solomon RZ, Sinha R, Cross AJ. Metabolomics in nutritional epidemiology: identifying metabolites associated with diet and quantifying their potential to uncover diet-disease relations in populations. Am J Clin Nutr. 2014;100(1):208–17.

Masino SA, Kawamura M Jr, Wasser CA, Pomeroy LT, Ruskin DN. Adenosine, ketogenic diet and epilepsy: the emerging therapeutic relationship between metabolism and brain activity. Curr Neuropharmacol. 2009;7(3):257–68.

Do WL, Whitsel EA, Costeira R, Masachs OM, Le Roy CI, Bell JT, et al. Epigenome-wide association study of diet quality in the Women’s Health Initiative and TwinsUK cohort. Int J Epidemiol. 2021;50(2):675–84.

Hellbach F, Sinke L, Costeira R, Baumeister SE, Beekman M, Louca P, et al. Pooled analysis of epigenome-wide association studies of food consumption in KORA TwinsUK and LLS. Eur J Nutr. 2023;62(3):1357–75.

Hellbach F, Freuer D, Meisinger C, Peters A, Winkelmann J, Costeira R, Hauner H, Baumeister SE, Bell JT, Waldenberger M, Linseisen J. Usual dietary intake and change in DNA methylation over years: EWAS in KORA FF4 and KORA fit. Front Nutr. 2024;10: 1295078.

Sundar IK, Yin Q, Baier BS, Yan L, Mazur W, Li D, Susiarjo M, Rahman I. DNA methylation profiling in peripheral lung tissues of smokers and patients with COPD. Clin Epigenet. 2017;9:1–8.

Go RC, Corley MJ, Ross GW, Petrovitch H, Masaki KH, Maunakea AK, He Q, Tiirikainen MI. Genome-wide epigenetic analyses in Japanese immigrant plantation workers with Parkinson’s disease and exposure to organochlorines reveal possible involvement of glial genes and pathways involved in neurotoxicity. BMC Neurosci. 2020;21:1–8.

Wahl S, Drong A, Lehne B, Loh M, Scott WR, Kunze S, Tsai PC, Ried JS, Zhang W, Yang Y, Tan S. Epigenome-wide association study of body mass index, and the adverse outcomes of adiposity. Nature. 2017;541(7635):81–6.

Mendelson MM, Marioni RE, Joehanes R, Liu C, Hedman ÅK, Aslibekyan S, Demerath EW, Guan W, Zhi D, Yao C, Huan T. Association of body mass index with DNA methylation and gene expression in blood cells and relations to cardiometabolic disease: a Mendelian randomization approach. PLoS Med. 2017;14(1): e1002215.

Pawlak R. Is vitamin B12 deficiency a risk factor for cardiovascular disease in vegetarians? Am J Prev Med. 2015;48(6):e11-26.

Tong TYN, Appleby PN, Bradbury KE, Perez-Cornago A, Travis RC, Clarke R, et al. Risks of ischaemic heart disease and stroke in meat eaters, fish eaters, and vegetarians over 18 years of follow-up: results from the prospective EPIC-Oxford study. BMJ. 2019;366:l4897.

Kwok T, Chook P, Tam L, Qiao M, Woo JLF, Celermajer DS, et al. Vascular dysfunction in chinese vegetarians: an apparent paradox? J Am Coll Cardiol. 2005;46(10):1957–8.

Adaikalakoteswari A, Finer S, Voyias PD, McCarthy CM, Vatish M, Moore J, et al. Vitamin B12 insufficiency induces cholesterol biosynthesis by limiting s-adenosylmethionine and modulating the methylation of SREBF1 and LDLR genes. Clin Epigenet. 2015;7(1):14.

Waterland RA, Jirtle RL. Transposable elements: targets for early nutritional effects on epigenetic gene regulation. Mol Cell Biol. 2003;23(15):5293–300.

Mahajan A, Sapehia D, Thakur S, Mohanraj PS, Bagga R, Kaur J. Effect of imbalance in folate and vitamin B12 in maternal/parental diet on global methylation and regulatory miRNAs. Sci Rep. 2019;9:17602.

Ge Y, Zadeh M, Mohamadzadeh M. Vitamin B12 regulates the transcriptional, metabolic, and epigenetic programming in human ileal epithelial cells. Nutrients. 2022;14(14):2825.

Sierra F, Caspi A, Fortinsky RH, Haynes L, Lithgow GJ, Moffitt TE, et al. Moving geroscience from the bench to clinical care and health policy. J Am Geriatr Soc. 2021;69(9):2455–63.

Moffitt TE, Belsky DW, Danese A, Poulton R, Caspi A. The longitudinal study of aging in human young adults: knowledge gaps and research agenda. J Gerontol A Biol Sci Med Sci. 2017;72(2):210–5.

Justice J, Miller JD, Newman JC, Hashmi SK, Halter J, Austad SN, et al. Frameworks for proof-of-concept clinical trials of interventions that target fundamental aging processes. J Gerontol A Biol Sci Med Sci. 2016;71(11):1415–23.

Waziry R, Ryan CP, Corcoran DL, Huffman KM, Kobor MS, Kothari M, Graf GH, Kraus VB, Kraus WE, Lin DT, Pieper CF. Effect of long-term caloric restriction on DNA methylation measures of biological aging in healthy adults from the CALERIE trial. Nat Aging. 2023;3(3):248–57.

Download references

Acknowledgements

We are grateful to all participants and researchers who took part in this study.

This study was funded in part by the Vogt Foundation (JLR, TH, JLS, and CDG).

Author information

Varun B. Dwaraka and Lucia Aronica contributed equally to this work.

Authors and Affiliations

TruDiagnostic, Inc, 881 Corporate Dr, Lexington, KY, 40503, USA

Varun B. Dwaraka, Natalia Carreras-Gallo, Aaron Lin, Logan Turner, Ryan Smith, Tavis L. Mendez & Hannah Went

Stanford Prevention Research Center, Department of Medicine, School of Medicine, Stanford University, 3180 Porter Dr, Palo Alto, Stanford, CA, 94305, USA

Lucia Aronica, Jennifer L. Robinson & Christopher D. Gardner

Seattle Children’s Research Institute, Seattle, WA, 98101, USA

Tayler Hennings

Department of Microbiology and Immunology, School of Medicine, Stanford University, Stanford University, Palo Alto, CA, USA

Matthew M. Carter, Emily R. Ebel, Erica D. Sonnenburg & Justin L. Sonnenburg

Department of Medicine, Division of Infectious Diseases, Weill Cornell Medicine, New York, NY, USA

Michael J. Corley

Chan Zuckerberg Biohub, San Francisco, CA, USA

Justin L. Sonnenburg

Center for Human Microbiome Studies, Stanford University School of Medicine, Stanford, CA, USA

You can also search for this author in PubMed   Google Scholar

Contributions

VBD, LA, and CDG had full access to the data and verified the data integrity and accuracy of the analysis. Writing of manuscript—VBD, LA, MJC, AL, RS, ERA. Conceived and designed the study, and provided funding—VBD, CDG, JLS. Sample Selection, Patient Recruitment, and sample analysis—TH, JLR, CDG. Sample processing and Methylation Data generation—TLM, HW. Telomere data contribution—MMC, JLS, EDS. Data processing, normalization, epigenetic clock quantification, statistical analysis, EWAS analysis—VBD. Results interpretation—VBD, LA, MJC, NCG. Figure generation—VBD, NCG, AL, LT. Edited and revised manuscript—all authors. All authors read and approved the final manuscript.

Corresponding authors

Correspondence to Varun B. Dwaraka or Christopher D. Gardner .

Ethics declarations

Ethics approval and consent to participate.

Procedures adhered to the ethical standards of the Helsinki Declaration, approved by the Stanford University Human Subjects Committee (IRB protocol 63955, approved March 9, 2022). Written informed consent was obtained from all participants.

Consent for publication

Not applicable.

Competing interests

Dr. Dwaraka, Dr. Carreras-Gallo, Aaron Lin, Logan Turner, Dr. Mendez, Hannah Went, and Ryan Smith are all employees of TruDiagnostic Inc. Dr Gardner reported receiving funding from Beyond Meat outside of the submitted work. Dr J. L. Sonnenburg is a Chan Zuckerberg Biohub investigator. No other disclosures were reported.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

12916_2024_3513_moesm1_esm.xlsx.

Additional file 1: Table S1. EpiScore analysis between baseline and 8-week test in the Stanford TWINs trial.The first column reports the EpiScore that was assessed, followed by the unadjusted p -value, and the direction of difference of the residual values between Week 8 from Week 0 for the Vegan (columns 2 and 3), and omnivore samples (columns 4 and 5). The final two columns show adjusted p -values for the vegan (column 6) and omnivore analyses (column 7). The statistical test run here was the Wilcoxon-rank sum test. The direction of change is represented as a + (representing a higher value at Week 8 relative to Week 0) or a - (representing a lower value at Week 8 relative to Week 0). Abbreviations: NS = not significant.

12916_2024_3513_MOESM2_ESM.xlsx

Additional file 2: Table S2. Epigenetic Biomarker Proxy (EBP) analysis between baseline and 8-week test in the Stanford TWINs trial. The first column reports the EBP that was assessed, followed by the unadjusted p -value, and the direction of difference of the residual values between Week 8 from Week 0 for the Vegan (columns 2 and 3), and omnivore samples (columns 4 and 5). The final two columns show adjusted p -values for the vegan (column 6) and omnivore analyses (column 7). The statistical test run here was the Wilcoxon-rank sum test. The direction of change is represented as a + (representing a higher value at Week 8 relative to Week 0) or a - (representing a lower value at Week 8 relative to Week 0). Abbreviations: NS = not significant. 

12916_2024_3513_MOESM3_ESM.xlsx

Additional file 3: Table S3. Excel file contains the significant results for the differential methylation analysis results from the EWAS time point analysis of Week 8 vs Week 0 of the Vegan diet. Column headers of each sheet are listed as follows: Column A represents the CpGs identified; Column B shows the log fold change of the m-value between Week 0 vs. Week 8 for each timepoint comparison, in which positive values are higher methylation at week 8 relative to week 0; Column C shows the average M-value for the CpG; Column D reports the t-statistic; Column E reports the unadjusted p -value; Column F reports the false-discovery rate (FDR) corrected p -value; Column G reports the B value outputted from limma; and Column H reports the gene ID overlapping the specific CpG loci. Table S4. Excel file contains the significant results for the differential methylation analysis results from the EWAS time-point analysis of Week 8 vs Week 0 of the Omnivore diet. Column headers of each sheet are listed as follows: Column A represents the CpGs identified; Column B shows the log fold change of the m-value between Week 0 vs. Week 8 for each timepoint comparison, in which positive values are higher methylation at week 8 relative to week 0; Column C shows the average M-value for the CpG; Column D reports the t-statistic; Column E reports the unadjusted p -value; Column F reports the false-discovery rate (FDR) corrected p -value; Column G reports the B value outputted from limma; and Column H reports the gene ID overlapping the specific CpG loci. Table S5. Excel file contains the significant results for the differential methylation analysis results from the EWAS time-point analysis of the Week 8 Vegan compared to the Week 8 of the Omnivore diet. Column headers of each sheet are listed as follows: Column A represents the CpGs identified; Column B shows the log fold change of the m-value between the Vegan vs. Omnivore at Week 8, in which positive values are higher methylation in the vegans relative to omnivores; Column C shows the average M-value for the CpG; Column D reports the t-statistic; Column E reports the unadjusted p -value; Column F reports the false-discovery rate (FDR) corrected p -value; Column G reports the B value outputted from limma; and Column H reports the gene ID overlapping the specific CpG loci. Table S6. Excel file contains the significant results for the differential methylation analysis results from the EWAS time-point analysis of the Week 0 Vegan compared to the Week 0 of the Omnivore diet. Column headers of each sheet are listed as follows: Column A represents the CpGs identified; Column B shows the log fold change of the m-value between the Vegan vs. Omnivore at Week 8, in which positive values are higher methylation in the vegans relative to omnivores; Column C shows the average M-value for the CpG; Column D reports the t-statistic; Column E reports the unadjusted p -value; Column F reports the false-discovery rate (FDR) corrected p -value; Column G reports the B value outputted from limma; and Column H reports the gene ID overlapping the specific CpG loci. 

12916_2024_3513_MOESM4_ESM.xlsx

Additional file 4: Table S7. GREAT results for DMLs hypermethylated in Vegan samples at 8 weeks, compared to Omnivore samples. Columns presented are as follows: Column A represents the GO ID of the term identified; Column B exhibits the name of the GO ID; Column C exhibits the total number of genes matched from the list of CpGs; Column D represents the hypergeometric coefficient expected; Column E represents the hypergeometric coefficient observed from the CPG set;  Column G represents the fold enrichment of the hypergeometric value; Column G represents the raw hypergeometric test p -value; Column H represents the adjusted hypergeometric test p -value (BH); and Column I represents the class of gene ontology (GO) term - Molecular Function (MF), Biological Process (BP), and Cellular Component (CC). Table S8. GREAT results for DMLs hypomethylated in Vegan samples at 8 weeks, compared to Omnivore samples. Columns presented are as follows: Column A represents the GO ID of the term identified; Column B exhibits the name of the GO ID; Column C exhibits the total number of genes matched from the list of CpGs; Column D represents the hypergeometric coefficient expected; Column E represents the hypergeometric coefficient observed from the CPG set;  Column G represents the fold enrichment of the hypergeometric value; Column G represents the raw hypergeometric test p -value; Column H represents the adjusted hypergeometric test p -value (BH); and Column I represents the class of gene ontology (GO) term - Molecular Function (MF), Biological Process (BP), and Cellular Component (CC).

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Dwaraka, V.B., Aronica, L., Carreras-Gallo, N. et al. Unveiling the epigenetic impact of vegan vs. omnivorous diets on aging: insights from the Twins Nutrition Study (TwiNS). BMC Med 22 , 301 (2024). https://doi.org/10.1186/s12916-024-03513-w

Download citation

Received : 07 February 2024

Accepted : 02 July 2024

Published : 29 July 2024

DOI : https://doi.org/10.1186/s12916-024-03513-w

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Diet and Nutrition
• Epigenetic clocks
• Epigenome-wide association study

BMC Medicine

ISSN: 1741-7015

• Submission enquiries: [email protected]
• General enquiries: [email protected]

Information

• Author Services

Initiatives

You are accessing a machine-readable page. In order to be human-readable, please install an RSS reader.

All articles published by MDPI are made immediately available worldwide under an open access license. No special permission is required to reuse all or part of the article published by MDPI, including figures and tables. For articles published under an open access Creative Common CC BY license, any part of the article may be reused without permission provided that the original article is clearly cited. For more information, please refer to https://www.mdpi.com/openaccess .

Feature papers represent the most advanced research with significant potential for high impact in the field. A Feature Paper should be a substantial original Article that involves several techniques or approaches, provides an outlook for future research directions and describes possible research applications.

Feature papers are submitted upon individual invitation or recommendation by the scientific editors and must receive positive feedback from the reviewers.

Editor’s Choice articles are based on recommendations by the scientific editors of MDPI journals from around the world. Editors select a small number of articles recently published in the journal that they believe will be particularly interesting to readers, or important in the respective research area. The aim is to provide a snapshot of some of the most exciting work published in the various research areas of the journal.

Original Submission Date Received: .

• Active Journals
• Find a Journal
• Proceedings Series
• For Authors
• For Reviewers
• For Editors
• For Librarians
• For Publishers
• For Societies
• For Conference Organizers
• Open Access Policy
• Institutional Open Access Program
• Special Issues Guidelines
• Editorial Process
• Research and Publication Ethics
• Article Processing Charges
• Testimonials
• Preprints.org
• SciProfiles
• Encyclopedia

Article Menu

• Subscribe SciFeed
• Recommended Articles
• Google Scholar
• on Google Scholar
• Table of Contents

Find support for a specific problem in the support section of our website.

Please let us know what you think of our products and services.

Visit our dedicated information section to learn more about MDPI.

JSmol Viewer

Reverse logistics and the circular economy: a study before and after the implementation of the national solid waste policy in brazil.

1. Introduction

2. materials and methods.

• Applied nature: as it examines the RL practices in a sample of companies from the South Fluminense region before and after the implementation of NSWP.
• Exploratory: The literature review focused on interrelating concepts of CE and RL linked to the NSWP. Subsequently, qualitative and quantitative data collected by Hernández [ 36 ] and Bitencourt [ 54 ] were processed and organized to analyze the relationship between RL and CE and to determine if there were changes after the implementation of the NSWP. Although two cases were selected for the study, the research is not considered a multiple case study because there is no in-depth investigation, which, according to Miguel [ 55 ], is characteristic of longitudinal research with more extensive temporal analysis, something that is not the aim of the present study.
• Mixed Method: With a sequential exploratory strategy, prioritizing initially collected qualitative data [ 56 ]. In this approach, quantitative data are used to assist in interpretation and to corroborate qualitative results. The qualitative strategy poses a broad question for the research, identified from establishing the relationship between RL and CE concepts. Both qualitative and quantitative strategies coexist from the definition of objectives, which were expressed by verbs that keep the investigation open, such as “analyze”, and phrases like “measurement of importance”. According to authors Günther [ 57 ] and Creswell and Creswell [ 56 ], as research questions are multifaceted, they accommodate more than one method.
• Hierarchy Representation: The defined objective was to prioritize RL practices, including criteria (RL programs) and alternatives (RL activities). This hierarchy was specific to each company and later grouped according to the study objectives;
• Judgment of criteria and alternatives: This judgment was carried out through pairwise comparison of all criteria and alternatives using Saaty’s Fundamental Scale, where numbers range from 1 (equal importance) to 9 (absolute importance) [ 58 ];
• CI : Consistency Index
• λ m a x : Maximum Eigenvalue
• n : Number of Criteria
• Aggregation of priorities and determination of global priority: Aggregation of relative priorities was performed to evaluate the outcome achieved concerning the objective. Priorities were calculated for each interviewed expert; since the companies belong to different sectors, their objectives also varied. To aggregate the results, the Aggregating Individual Priorities (AIP) method using arithmetic mean was employed. This aggregation method is used when decision vectors from individuals, with different perspectives on values and objectives, are combined to form an overall priority vector, as occurred in this case.
• CASE A: Study by Hernandez [ 36 ]
• CASE B: Study by Bitencourt [ 45 ]

4. Discussion

• In Hernández [ 36 ], economic programs (EP) had the highest importance, occupying the first position in priority order, while LP was ranked last in priority. The weight vectors of EP and LP had values of 52.88% and 2.28%, respectively ( Table 5 ).
• In Bitencourt [ 45 ], the EP continues to occupy the first position in priority order, but with a weight of 45.75%. However, the second position is now taken by the LP with a weight of 24.76% ( Table 6 ).
• The recycling of agricultural pesticide packaging increased from 37.4 thousand tons processed in 2012 to over 53.5 thousand tons in 2021.
• Similarly, plastic packaging from lubricating oil increased from 2538 tons recycled in 2012 to 4774 tons recycled in 2021.
• The collection and environmentally correct disposal for recycling of household electronic waste, or e-waste, in 2021, amounted to 1245 tons, surpassing the target established by Federal Decree No. 10240 of 2020.
• A total of 22,336.65 tons of paper and cardboard packaging were recovered, along with 8194.43 tons of post-consumer plastic packaging from discarded electronic products.
• In 2021, approximately 303,000 tons of dry recyclable waste were recovered: 46.3% comprised paper and cardboard, 26.5% comprised plastic, 14.5% comprised metal, and 12.2% comprised glass.
• Emerging concepts are being integrated: RL has already been practiced by many organizations, but CE is a relatively new and disruptive concept that deviates from the traditional linear economy model. Therefore, studying its relationship with RL offers insights on how to optimize and expand these practices and helps understand how this transition can be operationalized.
• The integration of the two concepts can help minimize the negative environmental impacts associated with the production and disposal of products, contributing to ecosystem conservation and the reduction in carbon emissions.
• The intersection between RL and CE can encourage innovation in product design, manufacturing processes, and business models, creating new opportunities for companies and startups.
• Analyzing this relationship can provide valuable insights for policymakers seeking to promote sustainability through regulations that encourage RL and CE practices, thereby supporting governmental strategies.

5. Conclusions

Author contributions, data availability statement, conflicts of interest, abbreviations.

• Hernández, C.T.; Marins, F.A.S.; Castro, R.C. Reverse Logistics Management Model. Gest. Prod. 2012 , 19 , 445–456. (In Portuguese) [ Google Scholar ] [ CrossRef ]
• De Brito, M.P.; Dekker, R. A Framework for Reverse Logistics, ERIM Report Series Research in Management. Erasmus Research Institute of Management . 2004, pp. 3–27. Available online: https://repub.eur.nl/pub/354/ERS-2003-045-LIS.pdf (accessed on 5 January 2024).
• Leite, P.R. Logística Reversa: Sustentabilidade e Competitividade ; Saraiva Educação SA: São Paulo, Brazil, 2017. [ Google Scholar ]
• Ye, F.; Zhao, X.; Prahinski, C.; Li, Y. The impact of institutional pressures, top managers posture and reverse logistics on performance—Evidence from China. Int. J. Prod. Econ. 2013 , 143 , 132–143. [ Google Scholar ] [ CrossRef ]
• Farooque, M.; Zhang, A.; Thürer, M.; Qu, T.; Huisingh, D. Circular supply chain management: A definition and structured literature review. J. Clean. Prod. 2019 , 228 , 882–900. [ Google Scholar ] [ CrossRef ]
• Geissdoerfer, M.; Savaget, P.; Bocken, N.M.; Hultink, E.J. The Circular Economy—A new sustainability paradigm? J. Clean. Prod. 2017 , 143 , 757–768. [ Google Scholar ] [ CrossRef ]
• Ghisellini, P.; Cialani, C.; Ulgiati, S. A review on circular economy: The expected transition to a balanced interplay of environmental and economic systems. J. Clean. Prod. 2016 , 114 , 11–32. [ Google Scholar ] [ CrossRef ]
• Kalmykova, Y.; Sadagopan, M.; Rosado, L. Circular econom—From review of theories and practices to development of implementation tools. Resour. Conserv. Recycl. 2018 , 135 , 190–201. [ Google Scholar ] [ CrossRef ]
• Oliveira, N.P. Use of the analytic hierarchy process methodology for selecting suppliers: Case study in tire reverse logistics. Rev. Gestão Sustentabilidade Ambient. 2020 , 9 , 1011–1020. (In Portuguese) [ Google Scholar ] [ CrossRef ]
• Queiroz, F.C.B.P.; Lima, N.C.; da Silva, C.L.; Queiroz, J.V.; de Souza, G.H.S. Article purchase intentions for brazilian recycled pet products—Circular economy opportunities. Recycling 2021 , 6 , 75. [ Google Scholar ] [ CrossRef ]
• Islam, M.T.; Iyer-Raniga, U. Lithium-ion battery recycling in the circular economy: A review. Recycling 2022 , 7 , 33. [ Google Scholar ] [ CrossRef ]
• Ellen MacArthur Foundation. EMF—Towards a Circular Economy: Business Rationale for an Accelerated Transition—2015. Available online: https://kidv.nl/media/rapportages/towards_a_circular_economy.pdf?1.2.1 (accessed on 14 June 2023).
• Figge, F.; Thorpe, A.; Gutberlet, M. Definitions of the circular economy-circularity matters. Ecol. Econ. 2023 , 208 . Available online: https://ssrn.com/abstract=4398717 (accessed on 15 January 2024). [ CrossRef ]
• Rosa, P.; Sassanelli, C.; Urbinati, A.; Chiaroni, D.; Terzi, S. Assessing relations between Circular Economy and Industry 4.0: A systematic literature review. Int. J. Prod. Res. 2020 , 58 , 1662–1687. [ Google Scholar ] [ CrossRef ]
• Pizzi, S.; Corbo, L.; Caputo, A. Fintech and SMEs sustainable business models: Reflections and considerations for a circular economy. J. Clean. Prod. 2021 , 281 , 125217. [ Google Scholar ] [ CrossRef ]
• Yu, Z.; Khan, S.A.R.; Umar, M. Circular economy practices and industry 4.0 technologies: A strategic move of automobile industry. Bus. Strategy Environ. 2022 , 31 , 796–809. [ Google Scholar ] [ CrossRef ]
• Dev, N.K.; Shankar, R.; Qaiser, F.H. Industry 4.0 and circular economy: Operational excellence for sustainable reverse supply chain performance. Resour. Conserv. Recycl. 2020 , 153 , 104583. [ Google Scholar ] [ CrossRef ]
• Hsu, C.C.; Choon Tan, K.; Hanim Mohamad Zailani, S.; Jayaraman, V. Supply chain drivers that foster the development of green initiatives in an emerging economy. Int. J. Oper. Prod. Manag. 2013 , 33 , 656–688. [ Google Scholar ] [ CrossRef ]
• Huang, Y.C.; Yang, M.L. Reverse logistics innovation, institutional pressures and performance. Manag. Res. Rev. 2014 , 37 , 615–641. [ Google Scholar ] [ CrossRef ]
• Gollnhofer, J.F.; Turkina, E. Cultural distance and entry modes: Implications for global expansion strategy. Cross Cult. Manag. 2015 , 22 , 21–41. [ Google Scholar ] [ CrossRef ]
• Huang, Y.C.; Yang, M.L.; Wong, Y.J. Institutional pressures, resources commitment, and returns management. Supply Chain Manag. Int. J. 2016 , 21 , 398–416. [ Google Scholar ] [ CrossRef ]
• Chu, S.H.; Yang, H.; Lee, M.; Park, S. The impact of institutional pressures on green supply chain management and firm performance: Top management roles and social capital. Sustainability 2017 , 9 , 764. [ Google Scholar ] [ CrossRef ]
• Dubey, R.; Gunasekaran, A.; Childe, S.J.; Papadopoulos, T.; Hazen, B.; Giannakis, M.; Roubaud, D. Examining the effect of external pressures and organizational culture on shaping performance measurement systems (PMS) for sustainability benchmarking: Some empirical findings. Int. J. Prod. Econ. 2017 , 193 , 63–76. [ Google Scholar ] [ CrossRef ]
• Famiyeh, S.; Kwarteng, A. Implementation of environmental management practices in the Ghanaian mining and manufacturing supply chains. Int. J. Product. Perform. Manag. 2018 , 67 , 1091–1112. [ Google Scholar ] [ CrossRef ]
• Demajorovic, J.; Huertas, M.K.Z.; Boueres, J.A.; Silva, A.G.d.; Sotano, A.S. Reverse logistics: How do companies report the disposal of batteries and cell phones? Rev. Adm. Empres. 2012 , 52 , 165–178. (In Portuguese) [ Google Scholar ] [ CrossRef ]
• Gomes, A.C.A.; Lobo, D.d.S.; Cardoso, B.F.; Shikida, P.F.A. Analysis of competitiveness of a dairy property through reverse logistics: A case study. Int. J. Glob. Energy Issues 2014 , 37 , 253–266. [ Google Scholar ] [ CrossRef ]
• de Jesus, F.S.M.; Barbieri, J.C. Role of recyclable material collector cooperatives in business reverse logistics for recycling through direct marketing. Rev. Gestão Soc. Ambient. 2013 , 7 , 1–15. (In Portuguese) [ Google Scholar ] [ CrossRef ]
• Oliveira, M.d.P.S.L.; de Oliveira, E.A.; Fonseca, A.M. Strategies to promote circular economy in the management of construction and demolition waste at the regional level: A case study in Manaus, Brazil. Clean Technol. Environ. Policy 2021 , 23 , 2713–2725. [ Google Scholar ] [ CrossRef ]
• de Almeida Virginio, L.K.; Fontgalland, I.L.; de Fátima Nóbrega Barbosa, M. Report of Experiences of the Sugar and Alcohol Sector in the Northeast of Brazil: The Debate of Nuances of the Circular Economy. Rev. De Gestão Soc. E Ambient. 2023 , 17 , e03964. (In Portuguese) [ Google Scholar ] [ CrossRef ]
• Sznitowski, A.M.; de Queiroz, A.A.F.S.L.; Padgett, R.C.M.L. Producing sustainably: A study on circular practices adopted on a rural property in Mato Grosso, Brazil. Contextus 2023 , 21 , 1–15. (In Portuguese) [ Google Scholar ] [ CrossRef ]
• Agrawal, S.; Singh, R.K. Analyzing disposition decisions for sustainable reverse logistics: Triple Bottom Line approach. Resour. Conserv. Recycl. 2019 , 150 , 104448. [ Google Scholar ] [ CrossRef ]
• Geisendorf, S.; Pietrulla, F. The circular economy and circular economic concepts—A literature analysis and redefinition. Thunderbird Int. Bus. Rev. 2018 , 60 , 771–782. [ Google Scholar ] [ CrossRef ]
• Vlachos, I.P. Reverse logistics capabilities and firm performance: The mediating role of business strategy. Int. J. Logist. Res. Appl. 2016 , 19 , 424–442. [ Google Scholar ] [ CrossRef ]
• Sellitto, M.A.; Hermann, F.F. Influence of Green Practices on Organizational Competitiveness: A Study of the Electrical and Electronics Industry. Eng. Manag. J. 2019 , 31 , 98–112. [ Google Scholar ] [ CrossRef ]
• Leite, P.R. Strategic drivers in reverse logistics programs in Brazil. Rev. Alcance 2012 , 19 , 182–201. (In Portuguese) [ Google Scholar ]
• Hernández, C. Reverse Logistics Management: A Conceptual Model ; Blucher Acadêmico: São Paulo, Brazil, 2011. (In Portuguese) [ Google Scholar ]
• Govindan, K.; Muduli, K.; Devika, K.; Barve, A. Investigation of the influential strength of factors on adoption of green supply chain management practices: An Indian mining scenario. Resour. Conserv. Recycl. 2016 , 107 , 185–194. [ Google Scholar ] [ CrossRef ]
• Couto, M.C.L.; Lange, L.C. Analysis of reverse logistics systems in Brazil. Eng. Sanitária E Ambient. 2017 , 22 , 889–898. (In Portuguese) [ Google Scholar ] [ CrossRef ]
• de Oliveira, C.T.; Luna, M.M.; Campos, L.M. Understanding the Brazilian expanded polystyrene supply chain and its reverse logistics towards circular economy. J. Clean. Prod. 2019 , 235 , 562–573. [ Google Scholar ] [ CrossRef ]
• Luna, R.A.; Viana, F.L.E. The role of national solid waste policy in reverse logistics in pharmaceutical companies. Rev. De Gestão Soc. E Ambient. 2019 , 13 , 40–56. (In Portuguese) [ Google Scholar ] [ CrossRef ]
• Orji, I.J. Examining barriers to organizational change for sustainability and drivers of sustainable performance in the metal manufacturing industry. Resour. Conserv. Recycl. 2019 , 140 , 102–114. [ Google Scholar ] [ CrossRef ]
• Shibao, F.Y.; Moori, R.G.; Santos, M.d. Reverse logistics and business sustainability. Semin. Em Adm. Ao 2010 , 13 , 1–17. (In Portuguese) [ Google Scholar ]
• National Industry Confederation. NIC—Circular Economy: A Strategic Path for the Brazilian Industry. Available online: https://static.portaldaindustria.com.br/media/filer_public/69/a7/69a762d3-ff66-4bfe-9eb1-452fd4566415/publicacao_caminho_estrategico_economia_circular.pdf (accessed on 14 June 2023).
• Zhu, Q.; Geng, Y.; Lai, K.H. Circular economy practices among Chinese manufacturers varying in environmental-oriented supply chain cooperation and the performance implications. J. Environ. Manag. 2010 , 91 , 1324–1331. [ Google Scholar ] [ CrossRef ]
• Bitencourt, J.d.S. Reverse Logistics in Companies in the Southern Region of Rio de Janeiro after the Implementation of the National Solid Waste Policy. 2021. Available online: https://app.uff.br/riuff/handle/1/23737 (accessed on 14 June 2023). (In Portuguese).
• Su, B.; Heshmati, A.; Geng, Y.; Yu, X. A review of the circular economy in China: Moving from rhetoric to implementation. J. Clean. Prod. 2013 , 42 , 215–227. [ Google Scholar ] [ CrossRef ]
• Wang, Z.; Hao, H.; Gao, F.; Zhang, Q.; Zhang, J.; Zhou, Y. Multi-attribute decision making on reverse logistics based on DEA-TOPSIS: A study of the Shanghai End-of-life vehicles industry. J. Clean. Prod. 2019 , 214 , 730–737. [ Google Scholar ] [ CrossRef ]
• da Silva Sousa, M.; Serra, J.C.V. Environmental indicators of urban solid waste associated with improving public policies. Rev. Gestão Sustentabilidade Ambient. 2019 , 8 , 707–724. (In Portuguese) [ Google Scholar ] [ CrossRef ]
• Cetrulo, T.B.; Marques, R.C.; Cetrulo, N.M.; Pinto, F.S.; Moreira, R.M.; Mendizábal-Cortés, A.D.; Malheiros, T.F. Effectiveness of solid waste policies in developing countries: A case study in Brazil. J. Clean. Prod. 2018 , 205 , 179–187. [ Google Scholar ] [ CrossRef ]
• Cunha, V.; Caixeta Filho, J.V. Management of solid urban waste collection: Structuring and application of a non-linear goal programming model. Gest. Prod. 2002 , 9 , 143–161. (In Portuguese) [ Google Scholar ] [ CrossRef ]
• Souza, M.T.S.d.; Paula, M.B.d.; Souza-Pinto, H.d. The role of recycling cooperatives in the reverse channel for postconsumer recyclables. Rev. De Adm. Ao De Empres. 2012 , 52 , 246–262. (In Portuguese) [ Google Scholar ] [ CrossRef ]
• RECICLUS. Who We Are—2021—Panorama 2022. Available online: https://reciclus.org.br/sobre/ (accessed on 10 June 2023). (In Portuguese).
• Brazilian Association of Public Cleaning and Special Waste Companies. ABRELPE—Panorama 2022. Available online: https://www.abrema.org.br/download/90935/?tmstv=1718136285 (accessed on 14 June 2023). (In Portuguese).
• Bitencourt, J.d.S.; Hernández, C.T. Analysis of reverse logistics practices in the South Fluminense region, after the implementation of the National Solid Waste Policy. Rev. De Adm. Ao Soc. E Inovac Ao 2022 , 8 . (In Portuguese) [ Google Scholar ] [ CrossRef ]
• Miguel, P.A.C. A case study in production engineering: Structuring and recommendations for conducting it. Production 2007 , 17 , 216–229. (In Portuguese) [ Google Scholar ] [ CrossRef ]
• Creswell, J.W.; Creswell, J.D. Research Design: Qualitative, Quantitative and Mixed Methods ; SAGE Publications: Thousand Oaks, CA, USA, 2021. (In Portuguese) [ Google Scholar ]
• Günther, H. Qualitative research versus quantitative research: Is that really the question? Psicol. Teor. E Pesqui. 2006 , 22 , 201–209. (In Portuguese) [ Google Scholar ] [ CrossRef ]
• Saaty, T.L. The analytic hierarchy process (AHP). J. Oper. Res. Soc. 1980 , 41 , 1073–1076. [ Google Scholar ]
• Guimarães, J.L.d.S. Multi-Criteria Analysis of Reverse Logistics Indicators in the Footwear Industry in Juazeiro do Norte. 2017. Available online: https://repositorio.unesp.br/handle/11449/151557 (accessed on 20 December 2023). (In Portuguese).
• Tosarkani, B.M.; Amin, S.H. A multi-objective model to configure an electronic reverse logistics network and third party selection. J. Clean. Prod. 2018 , 198 , 662–682. [ Google Scholar ] [ CrossRef ]
• Amaral, T.S.; de Andrade, V.S.; Junior, G.P. Use of the AHP methodology to choose a city to implement a logistical zone in the Rio Doce Valley region (MG). Braz. J. Dev. 2021 , 7 , 52562–52582. (In Portuguese) [ Google Scholar ]
• Tian, G.; Liu, X.; Zhang, M.; Yang, Y.; Zhang, H.; Lin, Y.; Ma, F.; Wang, X.; Qu, T.; Li, Z. Selection of take-back pattern of vehicle reverse logistics in China via Grey-DEMATEL and Fuzzy-VIKOR combined method. J. Clean. Prod. 2019 , 220 , 1088–1100. [ Google Scholar ] [ CrossRef ]
• Leite, P.R.; Brito, E.P.Z.; Macau, F.R.; Povoa, Â. The role of economic gains and corporate image in structuring reverse channels. Gestão. Org 2006 , 4 , 230–246. (In Portuguese) [ Google Scholar ]
• Silva, T.E.; Hernández, C.T. Circular economy and reverse logistics: Challenges and opportunities for sustainability in the context of industry 4.0. In Proceedings of the XVII CNEG—Congresso Nacional de Excelência em Gestão, INOVARSE—Simpósio de Inovação e Responsabilidade Social & ESD—Economic and Social Development Conference, Rio de Janeiro, Brazil, 23–25 November 2023. (In Portuguese). [ Google Scholar ] [ CrossRef ]
• Beiler, B.C.; de Arruda Ignácio, P.S.; Pacagnella Júnior, A.C.; Anholon, R.; Rampasso, I.S. Reverse logistics system analysis of a Brazilian beverage company: An exploratory study. J. Clean. Prod. 2020 , 274 , 122624. [ Google Scholar ] [ CrossRef ]
• Berssaneti, F.T.; Berger, S.; Saut, A.M.; Vanalle, R.M.; Santana, J.C.C. Value generation of remanufactured products: Multi-case study of third-party companies. Sustainability 2019 , 11 , 584. [ Google Scholar ] [ CrossRef ]
• Cassol, M.; Sellitto, M.A. Socio-biodiversity supply chain: Sustainable practices of a Brazilian cosmetic company. Environ. Qual. Manag. 2020 , 30 , 25–31. [ Google Scholar ] [ CrossRef ]
• Hammes, G.; De Souza, E.D.; Taboada Rodriguez, C.M.; Rojas Millan, R.H.; Mojica Herazo, J.C. Evaluation of the reverse logistics performance in civil construction. J. Clean. Prod. 2020 , 248 , 119212. [ Google Scholar ] [ CrossRef ]
• Kaviani, M.A.; Tavana, M.; Kumar, A.; Michnik, J.; Niknam, R.; de Campos, E.A.R. An integrated framework for evaluating the barriers to successful implementation of reverse logistics in the automotive industry. J. Clean. Prod. 2020 , 272 , 122714. [ Google Scholar ] [ CrossRef ]
• Cosenza, J.P.; de Andrade, E.M.; de Assunção, G.M. A circular economy as an alternative for Brazil’s sustainable growth: Analysis of the national solid waste policy. Rev. De Gestão Ambient. E Sustentabilidade 2020 , 9 , e16147. (In Portuguese) [ Google Scholar ] [ CrossRef ]
• Pinheiro, A.B.; Lameu, E.V.M.; Sampaio, T.S.L.; dos Santos, S.M. Revisiting the relationship between financial performance and environmental disclosure: A comparative analysis between Brazil and the Netherlands. Race Rev. De Adm. Ao Contab. Eeconomia 2022 , 21 , 51–76. (In Portuguese) [ Google Scholar ]

Click here to enlarge figure

Share and Cite

Aguirre Rodríguez, E.C.; Hernández, C.T.; Aguirre-Rodríguez, E.Y.; da Silva, A.F.; Marins, F.A.S. Reverse Logistics and the Circular Economy: A Study before and after the Implementation of the National Solid Waste Policy in Brazil. Recycling 2024 , 9 , 64. https://doi.org/10.3390/recycling9040064

Aguirre Rodríguez EC, Hernández CT, Aguirre-Rodríguez EY, da Silva AF, Marins FAS. Reverse Logistics and the Circular Economy: A Study before and after the Implementation of the National Solid Waste Policy in Brazil. Recycling . 2024; 9(4):64. https://doi.org/10.3390/recycling9040064

Aguirre Rodríguez, Elias Carlos, Cecilia Toledo Hernández, Elen Yanina Aguirre-Rodríguez, Aneirson Francisco da Silva, and Fernando Augusto Silva Marins. 2024. "Reverse Logistics and the Circular Economy: A Study before and after the Implementation of the National Solid Waste Policy in Brazil" Recycling 9, no. 4: 64. https://doi.org/10.3390/recycling9040064

Article Metrics

Further information, mdpi initiatives, follow mdpi.

Subscribe to receive issue release notifications and newsletters from MDPI journals

Log in using your username and password

• Search More Search for this keyword Advanced search
• Latest content
• Current issue
• Browse by collection
• BMJ Journals

You are here

• Online First
• Expert CONsensus on Visual Evaluation in Retinal disease manaGEment: the CONVERGE study
• Article Text
• Article info
• Citation Tools
• Rapid Responses
• Article metrics

• http://orcid.org/0000-0003-3981-707X Roger S Anderson 1 , 2 ,
• Mark Roark 3 ,
• Rose Gilbert 2 , 4 ,
• Dayyanah Sumodhee 2 , 4
• 1 Centre for Optometry and Vision Science , Ulster University , Coleraine , UK
• 2 NIHR Moorfields Biomedical Research Centre , London , UK
• 3 Allisonville Eye Care Center, Inc., Indiana USA , Allisonville , Indiana , USA
• 4 UCL Institute of Ophthalmology , London , UK
• Correspondence to Professor Roger S Anderson; rs.anderson{at}ulster.ac.uk

Background/Aims Recent decades have seen significant advances in both structural and functional testing of retinal disease. However, the current clinical value of specific testing modalities, as well as future trends, need to be clearly identified in order to highlight areas for further development in routine care and clinical trials.

Methods We designed a modified two-round Delphi study to obtain the opinion of a multidisciplinary group of 33 international experts involved in the field of retinal disease management/research to determine the level of agreement and consensus regarding the value and performance of specific structural and functional testing methods for retinal disease. On a Likert scale, a median of 1–2 indicated disagreement with the statement, and 5–6 indicated agreement with the statement. An IQR of ≤2 indicated consensus in the responses. Several questions also allowed comments on responses.

Results There was overall agreement that structural testing currently predominates for detection and monitoring. There was moderate agreement that functional testing remains important and will continue to do so in the future because it provides complementary information. Certain respondents considered that properly designed and applied psychophysical tests are as reliable and repeatable as structural observations and that functional changes are the most important in the long run. Respondents considered future care and research to require a combination of structural and functional testing with strong consensus that the relative importance will depend on disease type and stage.

Conclusion The study obtained important insights from a group of international experts regarding current and future needs in the management of retinal disease using a mix of quantitative and qualitative approaches. Responses provide a rich range of opinions that will be of interest to researchers seeking to design tests for future patient care and clinical trials.

• Diagnostic tests/Investigation
• Psychophysics
• Surveys and Questionnaires

Data availability statement

All data relevant to the study are included in the article or uploaded as supplementary information.

This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See:  http://creativecommons.org/licenses/by-nc/4.0/ .

https://doi.org/10.1136/bjo-2024-325310

Statistics from Altmetric.com

Request permissions.

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

WHAT IS ALREADY KNOWN ON THIS TOPIC

While recent decades have seen great advances in both structural and functional testing for retinal disease, the relative importance of these modalities, both now and in future, remains poorly explored and understood, with little guidance on what future requirements will be.

WHAT THIS STUDY ADDS

Using a mix of quantitative and qualitative approaches, this Delphi study acquired the opinions of 33 international experts (ophthalmologists, optometrists, psychophysicists) and found high levels of agreement and consensus about the relative importance of structural versus functional testing, both now and in future, as well as opinions about the future importance of home testing, artificial intelligence and patient reported outcome measures.

HOW THIS RESEARCH MIGHT AFFECT RESEARCH, PRACTICE OR POLICY

The study provides timely insights regarding current and future needs/priorities in the management of retinal disease. Responses also provide a rich range of opinions that will be of interest to researchers seeking to design tests for future patient care and clinical trials.

Introduction

The past two decades have seen important developments in the diagnosis and monitoring of retinal disease. Advances in structural imaging technology, most notably optical coherence tomography (OCT) now permit the in-vivo visualisation and study of retinal and subretinal layers, previously invisible to fundus photography. Along with further structural imaging developments such as OCT angiography and autofluorescence imaging, this has aided both disease detection and monitoring, but also improved understanding of the disease process itself.

Alongside these advances in structural imaging, functional measures of visual integrity have also, in parallel, significantly improved. In fact, functional testing still remains an essential component of retinal disease detection and monitoring in clinical care. Crucially, it measures what the patient cares the most about, namely the preservation of their vision. The current and future relative preference for structural or functional testing in the clinic is therefore very important.

Conventional visual acuity (VA) remains the most widely adopted test of visual function in clinical care and the most commonly adopted endpoint in clinical trials, 1 2 this despite its widely acknowledged high test–retest variability (TRV) 3–7 and poor sensitivity to conditions such as early age-related macular degeneration (AMD) and early retinal disease stages. 8 9 The realisation that even logMAR charts display high TRV and poor sensitivity to early AMD has led to further attempts to improve VA reliability by different scoring and termination rules, 10 11 computerised control of termination and scoring 12 and the use of high-pass filtered letters. 13 14 Tests to measure disease-specific aspects of visual loss such as contrast sensitivity (CS), low contrast letter acuity, cone function, flicker sensitivity, retinal adaptation and shape discrimination 15 have also been designed. Although these tests displayed early promise in detecting deficits of visual function not detected by conventional VA in clinical trials, they have not yet made their way into routine clinical care. It is therefore crucial to identify and understand the barriers to wider clinical adoption.

CS in particular has existed for more than 40 years and is reported to be more sensitive to subtle changes of visual function than VA 16–18 and to better relate to subjective visual impairment and quality of life. 18–20 Despite its availability in various commercial forms, the test continues to be largely absent from routine clinical care. 21 Whether this is due to poor repeatability 22 or some other combination of factors is not entirely clear.

Similarly, microperimetry is commonly used as an endpoint in trials and research 23 as it is effective at measuring functional loss in conditions such as AMD 24–26 and other retinal conditions. 27 Again, exploring the potential barriers to its wider adoption in a clinical setting is essential.

The shortcomings of VA have led to the adoption of structural imaging as a secondary (surrogate) endpoint in clinical trials. Increasingly there have even been attempts to develop structural primary endpoints 8 28 often because, as Schaal et al 28 note, ‘it is unrealistic to use visual acuity as a clinical trial endpoint in non-exudative AMD because vision loss takes many years to develop’. More recently there have been attempts to develop combined/composite endpoints which combine different types of functional and structural endpoints. 29 However, Terheyden et al 29 point out that there remains no agreement with regards to their implementation.

There has been an increase in studies examining the impact of retinal disease on patient quality of life from both a functional and psychological perspective, 30 and the adoption of questionnaire-based patient-related outcome measures (PROMs). 15 18 26

Several studies have indicated that patient reports of visual difficulty in low light/low contrast environments are predictive of disease progression across a range of AMD stages. 31 32 However, Finger et al 33 state that current PROMs need to be improved to detect or stage disease, and that these are not yet accepted by regulators as clinical endpoints, in part due to insufficient adherence to development guidelines. The existing and potential role of PROMs in clinical care is therefore a question of interest.

We attempted to provide answers to some of these important points by designing a Delphi study during which we obtained the opinion of internationally recognised experts involved in the field of retinal disease management and research. The aim of this study was to determine the level of agreement and consensus to a series of questions to identify current deficiencies in the assessment of retinal disease and how these need to change in the future, especially for the evaluation of novel therapies.

An initial literature search was undertaken of research studies involving different retinal or optic nerve diseases that employed various structural and/or functional outcome measures. Conditions included AMD, diabetic retinopathy, optic neuritis (ON), inherited retinal dystrophy, cystoid macular oedema, central or branch retinal vein occlusion as well as several glaucoma studies. The purpose was to initially review the abstracts and/or manuscripts to identify the most common primary and secondary outcome measures employed for various conditions with a view to formulating the questions for the first round of the study. 453 studies/papers were identified and reviewed in total by the authors.

Participants

A modified online Delphi study was designed that closely followed the recommendations of previously published ‘how to’ studies. 34 35 A flowchart of the process can be seen in figure 1 . 60 individuals were initially identified by the authors as potential expert participants in the study. Criteria for inclusion were that they were either:

Senior clinicians (ophthalmologists, optometrists, nurses) involved in the care of retinal disease, including active involvement in research and/or clinical trials of such conditions, evidenced by a significant publication record in the field, or recognised as key opinion leaders in retinal disease management. The majority of candidates were in this group.

Senior scientific researchers (psychologists, psychophysicists, neurophysiologists) involved in the research and/or design of tests of visual structure or function in retinal disease. Again, these individuals were required to be experienced experts in their field and display significant involvement in clinical retinal research and its publication.

• Download figure
• Open in new tab
• Download powerpoint

Flowchart demonstrating the Delphi process employed by the study.

Potential expert panel members were first approached by email to explain the purpose of the study and obtain their consent to participate. They were then given a link to a survey where they could access and respond to the questions for the first round. 33 of the identified experts consented to participate in the study and become part of the CONVERGE expert panel. The names and affiliations of those who consented to be identified are listed in table 1 , but for the duration of the study they were anonymous to each other. Of these, 14 were ophthalmologists, 13 were optometrists, 1 was a specialist ophthalmic research nurse, 3 were psychologists specialising in psychophysics and 2 others were from a neurophysiology and quantitative methods background. 22 were senior clinicians specialising in medical retina care and/or research who held the rank of consultant/head of service. Others were clinically qualified academics working in clinical visual research (psychophysics/visual function) related to medical retina. The non-clinicians consisted of senior academics specialising in clinical psychophysics or neurophysiology research. 21 of the total group also held the rank of full professor at their institution.

• View inline

Members of the CONVERGE expert panel

Summary graphs relating to the expert panel profile can be seen in figure 2 . Throughout the duration of the study the responses were collected and curated by staff from the study sponsor and made available to the authors in such a way that individual responses or comments were anonymous. In this way, the authors could not be influenced in their subsequent design or interpretation of questions by responses from individual panel members whom they regarded as authoritative opinion leaders, or with whom they were more personally acquainted.

Plots displaying the background and specialist interests of the CONVERGE expert panel, including: (A) professional background, (B) clinical and/or research involvement, (C) specialist area of interest, (D) years in specialty area. AMD, age-related macular degeneration.

Round 1 methods

The first round of the study was comprised of more general, open-ended questions to determine the initial panel member opinions and experience with regard to the sort of questions posed in the introduction above, and to identify issues requiring further specific probing.

Questions for this round related to such topics as the importance of structural versus functional testing in the future, any experience of VA being poorly correlated with symptoms/retinal appearance, barriers to the use of functional tests other than VA, reasons why CS is not tested more commonly and how this could be improved. Questions also explored functional tests that panel members considered most valuable in both clinical and research environments, and the importance of PROMs. The panel members were also afforded the opportunity to expand on their responses using comments in text boxes.

Round 2 methods

Based on the responses to round 1, a series of more focused questions was drawn up for round 2 with the goal of determining the level of agreement and consensus to various statements. We followed the design recommendation of Trevelyan and Robinson 35 where panel members were forwarded the (unattributed) responses and comments from all panel member in round 1, and were then asked to indicate their level of agreement to the round 2 questions on a 6-point Likert scale where, for most questions, a score of 1 meant ‘strongly disagree’ and a score of 6 meant ‘strongly agree’ with the statement. The responses were quantitatively analysed and a group median of 1–2 was taken to indicate disagreement with the statement, and 5–6 was taken to indicate agreement with the statement. An IQR of ≤2 was taken to indicate consensus in the responses, that is, the degree to which the experts agreed with each other, with 1 taken to mean strong consensus and 2 taken to mean moderate consensus. Several questions also afforded the opportunity to comment on foregoing responses.

Round 1 results

All 33 panel members completed round 1. We did not seek to quantitatively analyse the responses at this stage but the questions, summary of responses and unattributed additional comments can be seen in online supplemental material .

Supplemental material

Notable outcomes included:

Majority agreement (26/33, 79%) that both structural and functional testing will be required in the future. The themes identified in the comments are illustrated with 11 quotes in table 2 ( online supplemental material ). One of the dominant themes was that panel members considered the two types of testing to each have their relative strengths and weaknesses and could potentially complement each other (quote 1). Generally, most agreed that structural testing would predominate over functional to detect early stage disease (quotes 1, 2 and 3). Nevertheless, their respective usage varied depending on the type and stage of the disease with some disagreement as to which one performs better in the early stages (quote 2 vs quote 6).

Unanimous agreement that VA often poorly correlates with symptoms and/or disease severity.

Panel members further commented that VA does not necessarily mirror patients’ reported day-to-day visual function. They noted that a poor correlation is more likely to be found in certain diseases where the regions of the retina affected differ from that determining VA, usually the fovea (quote 3).

A majority of panel members agreed with the potential of CS to improve understanding of visual function in retinal disease due to its high sensitivity to early vision loss and gradual change in visual function. Nevertheless, various barriers were given for why it is not as widely used. They highlighted the complexity of employing the test including staff training, repeatability, lack of standardisation and time constraints (quote 4).

A wide range of functional tests, such as VA, colour vision, CS, visual field and microperimetry, are used in clinical care and research/clinical trials. However, the relative ranking of the tests in terms of importance is different in clinical care compared with research/clinical trials. For example, colour vision is considered more important in a clinical care setting, whereas in research/clinicals it is CS (round 1 online supplemental material , questions 13 and 15). Various other tests were suggested as important such as electrophysiology, reading function and low luminance VA.

Most respondents considered PROMs to be important in future assessment but the level of importance ranged from ‘essential’ to ‘fairly unimportant’. They further explained in the comments that PROMs, a subjective measure of the visual impairment impact on patients, should be used in conjunction with visual function testing (quote 5).

Selected round 1 and round 2 quotes

Round 2 results

All 33 experts from round 1 also participated in round 2. The complete round 2 questionnaire with responses displayed graphically can be found in online supplemental material . The median and IQR results of the Likert responses are displayed in table 3 , with selected quotes in table 2 ( online supplemental material ). Missing question numbers in the table relate to questions asking for follow-up comments.

Summary responses to questions from round 2, indicating the median (agreement) and IQR (consensus) of the responses (n=33)

Structural versus functional testing for detection and monitoring

Questions 2–16 dealt with the relative importance of structural versus functional testing both now and in the future. For question 2, there was moderate agreement and consensus among panel members that structural testing dominated current detection and monitoring of retinal disease but not necessarily future (median 2.5–3). Panel members seemed to have greater confidence in structural than functional testing as it offers more reliable and objective testing (Q4 and quote 7). There was strong consensus for current dominance of structural testing (Q2.a,b, median 2, IQR 1), but more moderate consensus for the future (Q2.c,d, median 2.5/3, IQR 2).

The notion that the dominance of structural testing would increase in the future with the inclusion of artificial intelligence (AI) reached agreement and consensus, but affordability, software interface and compactness of equipment will remain limitations (Q5).

There was agreement and consensus that functional testing remains essential for some conditions that structural testing cannot detect (Q7) but, conversely, panel members did not agree that structural testing has reached its zenith, thus requiring new functional tests to detect changes that are invisible to structural testing (Q8). Panel members highlighted the need to improve functional testing to make it easier to administer and more cost effective (Q10).

There was agreement and consensus that functional testing provides information that is complementary to structural testing (Q9) and that future detection (Q10) and monitoring (Q11) will require both for a fuller picture of retinal health. There was also agreement and consensus that the relative value of each will depend on both the disease type (Q12) and particularly stage (Q13).

The idea that the value of either test type was dependent on its ability to assess the efficacy of future treatment also reached agreement (Q14). There was also agreement and strong consensus that the relative value of each could change as new tests of each type are developed (Q15).

Interestingly, the notion that more emphasis should be placed on developing new functional rather than structural tests showed moderate agreement and consensus (Q16).

Visual acuity

Question 18 explored the reasons why VA did not always correlate with patient symptoms or retinal appearance. Panel members suggested three reasons for this, namely TRV, relative vulnerability to optical blur/retinal changes and the localised nature of retinal disease. Only the last two reasons reached agreement with moderate consensus.

Contrast sensitivity

Questions 20–24 investigated the role of CS testing. Question 20 interrogated the potential versus current usage of CS which is less wide than VA in clinical practice . There was agreement and consensus that it was time-consuming and required special equipment (Q20.a), that it was non-specific and thus susceptible to other conditions such as cataract (Q20.c), that it is poorly understood by clinicians (Q20.d), and that current tests lack uniformity in design and testing methods (Q20.f). This latter suggestion (Q20.f) was the only answer that had an IQR of 0. The idea that either it has poor repeatability (Q20.b) or that target sizes do not correspond to patient difficulty did not reach agreement (Q20.e and quote 9).

In question 22 panel members elaborated on how CS might change if these problems were resolved, in particular the possibility that CS could detect early change not seen by imaging (Q22.a), monitor advanced retinal disease (Q22.b), predict real-world impact on patient tasks (Q22.c) or better correlate with structural change (Q22.d). Agreement was reached only for Q22.c (median 5, IQR 2).

Question 24 examined why CS is more commonly used in clinical research or trials . There was agreement and consensus that this was because it was easier to administer in research because there is more time to devote to careful CS testing (Q24.a) but not necessarily that researchers have a better understanding of CS testing than the average clinician (Q24.b), or because clinical trial subjects are carefully selected (Q24.c).

Other functional tests

Question 26 asked about the importance of other functional tests in the management of retinal disease, both now and in future. Only reading speed and low luminance/contrast VA reached agreement and consensus. There was no agreement for retinal densitometry, CFF, electrophysiology or glare testing.

Home testing

For question 28, there was agreement and consensus that there would be a future increase in both functional (Q28.a) and structural testing (Q28.b) in the home in the future, using technologies such as tablet/VR and smart-phone/hand-held OCT, respectively. Although, this may lead to digital exclusion of elderly people (quote 10), home monitoring offers the opportunity to ease the burden on the hospital and prevent better sight loss (quote 11).

Patient-related outcome measures

Question 30 indicated no agreement with regard to the use of PROMs to either (a) detect problems that other tests miss, (b) tailor treatment regimens to individual patient needs or (c) the notion that they will become more valuable as care becomes more personalised. However, responses did not agree that PROMs provide little additional clinical information over current structure/function testing.

The study was successful in gaining the opinion of an expert group of clinicians and scientists, from a range of different professional groupings. The panel responses were analysed for both agreement, defined as whether the experts agreed with the question as it was posed, and consensus, indicating the degree to which they agreed with each other. Both are important but the latter is, we believe, more indicative of present and future trends.

Structure versus function

The overall position of the panel seems to be that structural testing currently holds sway in terms of detection and monitoring, owing to its less variable results, but functional testing remains important. Clinicians (ophthalmologists and optometrists) strongly agreed that structural testing is more dependable and repeatable, whereas psychophysicists were not so sure.

However, several respondents had a strong opinion that imaging will in future be able to predict function.

On the other hand, others consider that robustly designed and applied psychophysical tests are as reliable and repeatable as structural observations and that functional changes are the most important in the long run. One respondent noted that structural testing cannot yet detect colour vision changes. Some respondents acknowledged that there was a lack of research and development into practical functional tests, particularly tests that can detect cellular dysfunction rather than cell death.

Overall, panel members considered future care and research to require a combination of structural and functional testing as these provide complementary information, with strong consensus that the relative importance will depend on disease type and stage.

There was consensus that AI would continue to grow in importance and several comments related to this, with some panel members responding that AI could potentially help provide better prediction of function from structure and also understanding of individual disease progression.

The optometrists and psychophysicists were in strong agreement that future detection of retinal disease will require a combination of structural and functional testing, whereas the ophthalmologists did not quite reach agreement. It may be that the clinicians, and ophthalmologists in particular, are more familiar with, and better trained in, structural rather than functional testing.

Specific functional tests

Despite its recognised limitations, no panel member commented that VA would become less important as a functional test in the future. Responses indicated its main limitations to be its lack of ability to discriminate between neural and optical losses of vision, and its poor correlation with symptoms/retinal appearance. The clinicians were close to agreement that the poor correlation between VA, visual symptoms and retinal appearance is because of VA’s poor TRV, whereas the psychophysicists were much closer to disagreement. Comments commonly referred to the lack of localisation of retinal disease compared with the foveally dominant nature of VA as a test.

Nonetheless, a poor correlation between two tests may merely indicate that they are each measuring a different aspect of vision and are providing complementary information about ocular health. However, a VA test that employs a stimulus that better taps into the parafoveal damage associated with early AMD would appear to be very welcome.

The results and comments from both rounds of the study indicate that CS has the potential to provide significant further understanding of retinal disease and its impact, but is currently let down by its perceived complicated technical nature, lack of testing uniformity and susceptibility to optical/cortical changes. Several comments also alluded to the poor resolution of early stage disease.

Comments also indicated that it has the ability to provide additional clinical information and better predict real world task performance, but remains more associated with clinical trials where there is more time for testing and more technical assistance with set-up. Many clinicians perhaps continue to lack understanding of the test.

Reading speed and low luminance VA seemed more significant than others with one comment highlighting the importance of understanding what people see beyond distance VA. The variation in answers was perhaps not surprising given the variation in disease specialty of the panel members, and the likelihood that different tests are more appropriate for different retinal diseases, for example, electrophysiology.

There was agreement and consensus that home testing will likely become more prevalent in the future as both structural and functional tests become more compact and suitable for patient operation.

Some respondents who previously considered structural tests to have greater value considered functional home testing to be important in future. Others, however, considered home structural testing to be the future solution to reducing clinic visits and better detecting change.

There was consensus that PROMs will be of limited importance in future in terms of detecting and managing patient problems. This result may be due to the fact that most panel members are clinicians or psychophysicists who by training rely more on their examinations than what the patient reports. There were no noticeable differences in the responses between the various professional groups.

Strengths and weaknesses of the study

This study managed to obtain important insight from a group of international experts regarding current and future needs in the management of retinal disease using a mix of quantitative and qualitative approaches. The strength of a Delphi study is that it can rapidly identify the core problems/solutions without resorting to lengthy experimental research or clinical trials. The weakness is that it relies on subjective opinions, and even experts are not always correct. Several of the experts engaged in interesting discussions before consenting to take part. One concern expressed was that smaller discussion panels or working groups, typically convened to define guidelines are often dominated by a few prominent individuals with strong opinions. To address this risk, our study, (a) recruited a large panel of experts, (b) the panel members were unlikely to be influenced by others as they remained unknown to each other for the duration of the study. Another strength of our study was that we included experts representing all relevant professional groupings and specialist interests, allowing for a diverse range of opinions.

Our findings suggest a moderate overall agreement that structural rather than functional testing dominates current practice, but not necessarily future practice. Overall, respondents tend to favour the tests they are most familiar with. Clinicians (ophthalmologists, optometrists) tend to favour structural testing in the future, whereas those with a psychophysical background tend to favour functional testing. Trained psychophysicists may see future solutions to current functional testing limitations and the potential for more targeted tests, particularly for different conditions and disease stages, and thus substantial untapped potential in functional testing. New improved tests may have subsequently come on stream, and will continue to do so in the future, and the need for expert psychophysical understanding and input will only grow if appropriate decisions are to be made. Most of our respondents also agreed that AI and home testing will become increasingly relevant in retinal disease management.

Ethics statements

Patient consent for publication.

Not applicable.

Ethics approval

This study involves human participants and the methods for this study were reviewed and approved by the institutional legal and ethics committee of Alliance Pharmaceuticals Ltd. All participating experts granted personal approval for their name to be included in the manuscript. Participants gave informed consent to participate in the study before taking part.

Acknowledgments

The authors would like to specifically thank Sophie Fairweather and Grace Evans for their support in facilitating meetings and assisting with administrative details.

• Lesmes LA ,
• Jackson ML ,
• Rosser DA ,
• Cousens SN ,
• Murdoch IE , et al
• Kaiser PK ,
• Zamora D , et al
• Rubin GS , et al
• Lovie-Kitchin JE ,
• falkenstein IA ,
• cochran DE ,
• azen SP , et al
• Chew EY , et al
• Huisingh C ,
• Clark ME , et al
• Whitaker HL , et al
• Laidlaw DAH ,
• Shah N , et al
• Redmond T , et al
• Dobinson S , et al
• Chakravarthy U
• Ramirez LMV ,
• Pimentel SLG , et al
• Pondorfer SG ,
• Terheyden JH ,
• Heinemann M , et al
• Mitchell P ,
• Bandeen-Roche K ,
• Huang GH , et al
• Luu CD , et al
• Steinberg JS ,
• Fitzke FW ,
• Fimmers R , et al
• Sundaram V ,
• Ripamonti C , et al
• Corradetti G ,
• Esmaeilkhanian H , et al
• Schaal KB ,
• Rosenfeld PJ ,
• Gregori G , et al
• Schmitz-Valckenberg S ,
• Crabb DP , et al
• Taylor DJ ,
• Binns AM , et al
• Maguire MG ,
• Liu C , et al
• Finger RP ,
• Schmid M , et al
• Sandford BA
• Trevelyan EG ,
• Robinson PN

Deceased Mark Roark passed away shortly before acceptance of the manuscript

Contributors RSA, RG and MR were involved in the initial design of the study and formulation of the questionnaires and recruitment of the subjects. All authors were involved in the analysis of the results and the writing of the paper. RSA is the guarantor.

Competing interests The study was funded and facilitated by Alliance Pharmaceuticals Ltd. The authors received financial payment from the funder, but the study objectives, design, analysis and subsequent write-up was entirely undertaken by the authors. Alliance Pharmaceuticals Ltd played no part in the study design, the formulation of the questions, or the analysis and respondents were not paid for their responses. The online questionnaires were administered by the funder and the data curated by Alliance staff to render the authors blind to the identity of respondents until final analysis.

Provenance and peer review Not commissioned; externally peer reviewed.

Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.

Read the full text or download the PDF:

ChatGPT in the classroom: navigating the generative AI wave in management education

Journal of Research in Innovative Teaching & Learning

ISSN : 2397-7604

Article publication date: 9 July 2024

The study aims to explore the role of ChatGPT, an artificial intelligence (AI) language model, in the field of management education. Specifically, the goal is to evaluate ChatGPT's effectiveness in facilitating active learning, promoting critical thinking, and fostering creativity among students. Additionally, the study seeks to investigate the potential of ChatGPT as a novel tool for enhancing traditional teaching methods within the framework of management education.

Design/methodology/approach

This research systematically explores ChatGPT's impact on student engagement in management education, considering AI integration benefits and limitations. Ethical dimensions, including information authenticity and bias, are scrutinized, alongside educators' roles in guiding AI-augmented learning.

The study reveals ChatGPT's effectiveness in engaging students, nurturing critical thinking, and fostering creativity in management education. Ethical concerns regarding information authenticity and bias are addressed. Insights from student and teacher perceptions offer valuable pedagogical implications for AI's role in management education.

Research limitations/implications

While this study offers valuable insights into the role of ChatGPT in management education, it is essential to acknowledge certain limitations. Firstly, the research primarily focuses on a specific AI model (ChatGPT), and findings may not be generalized to other AI language models. Additionally, the study relies on a specific set of educational contexts and may not fully capture the diverse landscape of management education globally. The duration of the research and the sample size could also impact the generalizability of the findings.

Practical implications

The findings of this study hold practical significance for educators and institutions engaged in management education. The integration of ChatGPT into teaching strategies has the potential to improve active learning, critical thinking, and creativity. Educators can utilize this AI tool to diversify instructional methods and accommodate diverse learning styles. However, the practical implementation of AI in the classroom necessitates meticulous consideration of infrastructure, training, and ongoing support for both educators and students. Furthermore, institutions should proactively tackle ethical concerns and establish guidelines for the responsible use of AI in education.

Social implications

The incorporation of AI, such as ChatGPT, in management education carries broader social implications. The study underscores the significance of addressing ethical concerns associated with AI, including issues related to information authenticity and bias. As AI becomes more widespread in educational settings, there is a necessity for societal discussions on the role of technology in shaping learning experiences. This research advocates for a thoughtful approach to AI adoption, emphasizing the importance of transparency, accountability, and inclusivity in the development and deployment of AI technologies within the educational sphere. The findings prompt reflections on the societal impact of AI-driven education and the potential consequences for students' skills, employment prospects, and societal values.

Originality/value

Originality/Values: This research contributes to the academic discourse by systematically examining the role of ChatGPT in management education, providing insights into both its advantages and potential ethical challenges. The study offers original perspectives on the use of AI in educational settings, paving the way for well-informed decision-making that can shape the future of management education in the evolving landscape of technological progress.

• Artificial intelligence
• Management education
• AI-Augmented
• Pedagogical innovation
• Educational tool and future of learning

Leelavathi, R. and Surendhranatha, R.C. (2024), "ChatGPT in the classroom: navigating the generative AI wave in management education", Journal of Research in Innovative Teaching & Learning , Vol. ahead-of-print No. ahead-of-print. https://doi.org/10.1108/JRIT-01-2024-0017

Emerald Publishing Limited

Copyright © 2024, R. Leelavathi and Reddy C. Surendhranatha

Published in Journal of Research in Innovative Teaching & Learning . Published by Emerald Publishing Limited. This article is published under the Creative Commons Attribution (CC BY 4.0) licence. Anyone may reproduce, distribute, translate and create derivative works of this article (for both commercial and non-commercial purposes), subject to full attribution to the original publication and authors. The full terms of this licence may be seen at http://creativecommons.org/licences/by/4.0/legalcode

1. Introduction

The rapid advancement of artificial intelligence (AI) technologies, including ChatGPT, presents transformative opportunities across various sectors, including education. The integration of ChatGPT in management education holds promise for reshaping traditional teaching methods. As industries adapt to an AI-driven landscape, exploring the potential of incorporating such technologies becomes crucial. Generative AI, exemplified by ChatGPT, has generated enthusiasm for its applications in diverse areas, offering revolutionary paths to enhance learning experiences in management education. However, this integration raises ethical questions, necessitating a comprehensive exploration of ChatGPT's benefits and challenges in the classroom.

Technological progress has significantly impacted the education sector, with AI tools like ChatGPT altering teaching and learning processes for both students and teachers. Understanding the challenges and benefits of implementing ChatGPT in education is crucial. Responsible usage of AI tools can enhance the capabilities of teachers and students, but vigilance is required due to issues such as inconsistency, incorrect information, and misleading facts. Careful implementation and a reevaluation of academic integrity policies are essential for ChatGPT's effective use in academic settings.

ChatGPT offers capabilities such as providing code, solutions, and opportunities to enhance learning, but it also comes with limitations like limited data sources and potential misinformation. Students need to understand both the capabilities and limitations of ChatGPT for progressive knowledge utilization. The future of ChatGPT appears promising, with opportunities for personalization, quick assessment, and strategic improvement, but challenges such as integrity issues and ethical considerations persist.

The significance of reproductive AI and ChatGPT tools in higher education is emphasized, with the potential to enhance learning capabilities and skill sets. The role of generative AI tools is evolving rapidly, and educational institutions need to incorporate them for improved learning and collaboration. ChatGPT's integration into design knowledge acquisition is highlighted, facilitating interaction, support, and collaboration in the workplace. While ChatGPT brings opportunities like personalized learning and teaching assistance, concerns about authorship, plagiarism, and biased information need attention.

In management education, opportunities and challenges from ChatGPT are identified, prompting the need for further research. While ChatGPT provides learning opportunities, educating students on its proper use, integrating AI tools into the learning environment, and re-engineering assessments are essential. This study explores the dynamic use of ChatGPT to enhance student engagement, foster critical thinking, and navigate AI ethics in the context of management education, addressing a gap in understanding the effective integration of advanced tools like ChatGPT.

2. Research aims

To assess the effectiveness of integrating ChatGPT into management education for enhancing students' learning.

To examine the ramifications of ChatGPT on fostering critical thinking and creativity among management students.

To investigate the ethical dimensions linked to the incorporation of ChatGPT into management education.

3. Theoretical framework

3.1 integration of ai into education.

The gradual integration of artificial intelligence (AI) technologies, such as ChatGPT, is reshaping traditional teaching approaches and unlocking AI-powered tools to enhance student engagement and personalized learning experiences ( Johnson et al ., 2020 ). The effectiveness of incorporating AI-driven platforms to promote critical thinking skills among students is well-established ( Smith et al. , 2019 ). The ethical dimensions of AI integration within educational settings emphasize the importance of transparent AI decision-making processes ( Akgun and Greenhow, 2022 ). A growing body of literature recognizes the need to equip future professionals with AI literacy and suggests that integrating AI tools like ChatGPT can cultivate creativity and innovative thinking among business students ( Reddy, 2022 ). The survey-based approach, as utilized in this study, has previously been employed to assess student perceptions of AI integration ( Kumar Ravi and Raman, 2022 ). Notably, academic viewpoints on the challenges and possibilities of AI in education have been underscored ( Kim and Kim, 2022 ).

One prominent benefit of incorporating ChatGPT in the educational setting is its capacity to boost learning engagement and creativity. AI models like ChatGPT can generate interactive and dynamic content, providing students with personalized knowledge experiences tailored to their individual preferences. This personalization fosters increased engagement and creativity among students, making the learning process more effective ( Anderson et al. , 2021 ). ChatGPT's natural language comprehension and generation abilities have shown promise in assisting students in decision-making and problem-solving tasks. Interacting with ChatGPT, students were able to gain insights into complex management scenarios and develop critical thinking skills, making it a valuable tool for management education ( Zhu et al ., 2023 ; Chen et al ., 2020 ).

3.2 Opportunities and obstacles of implementing ChatGPT in management education

Despite its benefits, the integration of ChatGPT in education raises ethical concerns. It is argued that generative AI models may perpetuate biases inherent in their training data, potentially reinforcing stereotypes or providing inaccurate information. Therefore, educators must be cognizant of these ethical issues and actively work to mitigate bias when implementing ChatGPT in the classroom ( Johnson et al. , 2019 ). Effective implementation of ChatGPT in management education requires addressing technical challenges and limitations. The emphasis should be on proper training and integration strategies to enable educators to fully leverage the capabilities of AI models like ChatGPT ( Smith et al. , 2019 ).

The guidance provided by ChatGPT was evidence-based, but the absence of cited sources made it unverifiable. Additionally, it was found to offer incorrect and incomplete information, which may require expert verification ( Oviedo-Trespalacios et al ., 2023 ). The product recommendations made by ChatGPT have implications for customers, as their decisions are influenced by them, highlighting the need for incorporating AI tools in online shopping ( Kim et al ., 2023 ; Lo, 2023 ).While ChatGPT serves as a source of quick and relevant data, it lacks the ability to think like a human being, raising concerns about the integrity of the information it provides ( Vázquez-Cano et al ., 2023 ).

Literature review and source citation remain critical issues in scientific writing using ChatGPT, as it provides sources with fabricated, non-existing titles. This may be attributed to a lack of training in ChatGPT for generating language models relevant to the query ( Varghese and Chapiro, 2023 ). The results generated by ChatGPT are not consistently satisfactory across domains, and the information provided lacks consistency. It can be integrated into education by developing learning models and instructions ( Alves de Castro et al ., 2023 ). Generative AI technologies like ChatGPT offer opportunities for higher education, providing continuous access to information, personalized learning, learning experience support for instructors, and data analysis. However, they also pose challenges regarding ethical issues, integrity, transparency, accountability, and quality ( Pisica et al ., 2023 ; Michel-Villarreal et al ., 2023 ).

3.3 Ethical concerns and application of ChatGPT

Human ethics is pivotal in the ethical use of generative AI tools in education, as highlighted by Heyder et al . (2023) . Despite the benefits of personalized learning, ChatGPT raises ethical concerns and integrity issues, noted by Rasul et al . (2023) . Educators, as emphasized by Benuyenah (2023) , must assess ChatGPT's potential and challenges for thoughtful integration into educational processes. Despite hurdles like privacy concerns and regulatory gaps, Sabzalieva and Valentini (2023) , suggests higher education institutions can utilize ChatGPT for teaching and research. While AI offers benefits like aiding in learning measurement and curriculum organization, there's limited research on its implications in higher education, requiring further exploration ( Crompton and Burke, 2023 ; Hinojo-Lucena et al. , 2019 ). Wazan et al . (2023) suggest integrating generative AI tools like ChatGPT to enhance learning experiences. However, Bozkurt et al . (2021) stress considering ethical and privacy implications. Overall, ChatGPT's primary applications in education include teaching, assessment, research, and development ( Wilfred and Ade-Ibijola, 2021 ).

3.4 Enhancing critical thinking and creativity in management education through ChatGPT

Recent research explores ChatGPT's role in improving critical thinking among management students by facilitating meaningful dialogue and inquiry ( Van Inwagen, 2020 ). Interacting with ChatGPT prompts students to question assumptions and construct reasoned arguments ( Kirschner and van Merriënboer, 2013 ). Similarly, ChatGPT stimulates creativity by providing a platform for brainstorming and collaboration ( Wiggins and McTighe, 2013 ). This enhances students' ability to innovate solutions ( Bereiter and Scardamalia, 2014 ). Engaging with ChatGPT fosters interactive learning and collaborative problem-solving, crucial in management education. Assessing its impact aids understanding of technology's role in effective learning outcomes. While promising, integrating ChatGPT necessitates addressing ethical and technical challenges for optimal use in education.

4. Methodology

4.1 methodological framework.

The research methodology for this study involves the use of a survey-based approach to gather insights from both teachers and students within the realm of management education. A structured questionnaire was designed to generate quantitative responses, addressing various aspects of ChatGPT integration. The questionnaire is divided into three parts: the first section collects socio-demographic information, the second section focuses on basic usage of ChatGPT, and the final section explores various constructs related to the study, utilizing a 5-point Likert scale. In this scale, 1 signifies strong disagreement (SD), and 5 signifies strong agreement (SA).

Students' perspectives on learning enhancement, engagement, and critical thinking development were explored, while academicians' insights into instructional methodologies, challenges, and ethical considerations were assessed. The collected data underwent quantitative analysis, employing statistical measures to provide a comprehensive understanding of the inferences and effectiveness of incorporating ChatGPT in the context of management education. Throughout the research process, ethical considerations and data privacy were ensured.

4.2 Population and sample

The research assesses the impact of ChatGPT on creative and critical thinking abilities, as well as the ethical considerations related to integrating ChatGPT in management education by faculty members. The study population comprised undergraduate and postgraduate students, along with faculty members from various management education institutions in Bangalore city. Study participants were recruited through college-wide email and personal connections. An online structured questionnaire was distributed to both students and faculty members through a dedicated online platform, specifically using a Google Form.

The final sample included 331 individuals: 282 students (both undergraduate and postgraduate) and 49 faculty members with postgraduate or Doctor of Philosophy (PhD) degrees, selected from a diverse range of management institutes, including colleges and universities. Among the faculty members, there were teaching assistants, assistant professors, associate professors, heads of departments, and deans. Participants voluntarily took part in the study, with guarantees of the confidentiality of their data. The fieldwork was conducted over a period of five months, during which the questionnaire was distributed and responses were gathered effectively. Subsequently, data from the collected responses were extracted from the database, and statistical analysis was performed using appropriate tools.

4.3 Hypotheses

There is no substantial relationship between the incorporation of ChatGPT in the educational setting and the enhancement of critical thinking abilities, creativity, and ethical consciousness in management students.

There is a meaningful correlation between the utilization of ChatGPT in the classroom and the advancement of critical thinking skills, creativity, and ethical awareness among management students.

The integration of ChatGPT in the classroom does not positively influence the level of AI integration, enhancement of learning outcomes, and Educator guidance to management studies in management education.

The integration of ChatGPT in the classroom positively influences the level of AI integration, enhancement of learning outcomes, and Educator guidance to management studies in management education.

4.4 Study constructs

In this section, we detail the constructs analyzed in the study to provide a thorough understanding for the reader. The research scrutinized six key dataset items, each representing a unique facet of the interaction between students and ChatGPT in management education. These constructs encompass:

Critical Thinking: This assesses students' capability to evaluate evidence and make informed decisions, influenced by their engagement with ChatGPT.

Creativity: Examining students' ability to generate innovative ideas and solutions, shaped by their interaction with ChatGPT.

Ethical Awareness: Evaluating students' recognition of moral considerations in decision-making, particularly regarding AI integration.

AI Integration: Assessing the degree of incorporation of AI, specifically ChatGPT, into the management education curriculum.

Learning Enhancement: Measuring the impact of AI integration on the overall learning experience, including factors like engagement, knowledge retention, and understanding.

Educator Guidance: Reflecting on educators' role in facilitating students' effective use of AI tools like ChatGPT in teaching, guiding their interactions.

By defining these independent and dependent variables, the study aims to systematically explore the relationships between critical thinking, creativity, and ethical awareness as catalysts for AI integration, learning enhancement, and educator guidance in management education. This structured framework enables a data-driven analysis of how these elements interconnect and contribute to the educational experience as a whole.

4.5 Reliability statistics

Statistical reliability is crucial to ensure the validity and accuracy of statistical analysis. The statistical tools utilized in the research must consistently produce reliable results. This consistency is essential for building trust in the statistical analysis and the outcomes it generates.

Table 1 data shows that both Cronbach's Alpha coefficients (0.848 and 0.860) are relatively high indicates a robust level of internal consistency within the dataset's six items. These elevated Cronbach's Alpha values are positive indicators, suggesting that the set of items in the study is internally consistent. This reflects the reliability and quality of study measurement tool.

5. Results and discussion

Data analysis was conducted on information obtained through a structured questionnaire to derive results and corresponding interpretations for the purpose of study and future research. From the analysis of the data, it was determined that there were 157 female and 164 male participants in the survey. Given that the generative AI tool, ChatGPT, is in its early stages, the frequency of its usage by both students and faculty members is comparatively low. About 53% of students mentioned using it occasionally, while among faculty members, 44.2% expressed that they also use ChatGPT occasionally. Interestingly, among both faculty members and students, the majority (87.9%) indicated that they are not ready to purchase the premium version of ChatGPT.

5.1 Frequency of using ChatGPT

Table 2 displays the frequency of ChatGPT usage categorized by gender, occupation, and highest qualification. The majority of users from both genders employ ChatGPT occasionally, with 54% of females and 56% of males falling into this category. Doctoral program participants from both genders are the least frequent users. Chi-square tests for gender do not indicate a statistically significant relationship between gender and ChatGPT usage. The p -values for both the Pearson Chi-Square (“ χ 2 ”) and Likelihood Ratio (“G”) tests exceed the alpha level of 0.05, suggesting that gender has no statistically significant impact on ChatGPT usage.

There are six occupation categories: Faculty, Postgraduate (PG) Students, Undergraduate (UG) Students, Bachelor of Business Administration (BBA), Bachelor of Commerce (BCOM), and Master of Business Administration (MBA). PG Students and UG Students, irrespective of their specific major, are the most frequent users of ChatGPT. Faculty members and Doctoral Program participants use ChatGPT less frequently. The chi-square tests for occupation also do not reveal a statistically significant relationship between occupation and ChatGPT usage, with p -values for both tests exceeding 0.05.

Qualifications range from BBA, BCOM, and MBA to Doctoral Program. Individuals with an MBA qualification are the most frequent users of ChatGPT, followed by those with a BCOM qualification. Doctoral Program participants use ChatGPT the least. In this case, the chi-square tests for the highest qualification indicate a statistically significant relationship between the highest qualification and ChatGPT usage. The p -values for both the Pearson Chi-Square (" χ 2 ") and Likelihood Ratio ("G") tests are below 0.05, indicating that the highest qualification significantly influences ChatGPT usage.

In summary, the analysis suggests that while gender and occupation do not appear to have a significant impact on ChatGPT usage patterns, there is a statistically vital association between the highest qualifications attained by respondents and their frequency of using ChatGPT ( Crompton and Burke, 2023 ). Further investigation may be needed to understand the reasons behind this association.

5.2 Intention behind using ChatGPT

The ranking and percentages reflect the diverse range of purposes for which users employ ChatGPT. It is notably valuable for academic-related tasks such as assignment writing, note preparation, and exploring new concepts, but it also serves functions like grammar checking and report writing. Additionally, it finds utility in language-related tasks such as translation and paraphrasing. ChatGPT has extensive applications as it can be used in higher education in many areas by both students and faculty members ( Holmes and Tuomi, 2022 ) (see Figure 1 ).

5.3 Readiness to invest in a premium edition of ChatGPT

Table 3 presents the willingness to invest in a premium version, categorized by gender, occupation, and highest qualification. Among the 157 female respondents, 18 expressed a willingness to pay for the premium version, while 20 out of the 164 male respondents were open to the idea. Notably, the majority of respondents from both genders are not inclined to pay for the premium version. Chi-square tests for gender failed to reveal a significant association between gender and the willingness to pay. Both the “Pearson Chi-Square ( χ 2 )” and “Likelihood Ratio tests (G)” yielded p -values exceeding 0.05, indicating that gender does not significantly impact the willingness to pay.

Among faculty members, 28 expressed interest in paying, while among PG students, 18 were willing to do so. Interestingly, UG students are the category least inclined to pay for the premium version.

The chi-square tests related to occupation revealed a statistically meaningful connection between occupation and the inclination to invest in the premium version. The p -values for both the Pearson Chi-Square and Likelihood Ratio tests fall below 0.05, indicating that occupation indeed exerts a substantial impact on the willingness to pay. Qualifications range from BBA, BCOM, and MBA to Doctoral Programs. Among these, individuals with MBA qualifications are the most inclined to pay for the premium version, followed by those with BCOM qualifications. Participants in Doctoral Programs exhibit the least willingness to pay.

Similarly, the chi-square tests concerning the highest qualification also establish a statistically significant link between qualification level and readiness to pay for the premium version. The p -values for both the Pearson Chi-Square and Likelihood Ratio tests are less than 0.05. In summary, the analysis suggests that while gender does not substantially impact the willingness to pay for a premium version, both occupation and highest qualification play a statistically significant role in influencing individuals' readiness to pay. Further research may be beneficial to understand the underlying factors behind these connections and to formulate targeted strategies for marketing premium versions of the product or service.

5.4 Factor analysis

The study uses factor analysis to understand how ChatGPT fits into management education by looking at different parts of the data. It helps answer the research questions by finding hidden patterns or groups in the data.

The KMO Measure Table 4 suggests that the dataset is suitable for factor analysis, and Bartlett's Test affirms the presence of substantial correlations among the variables, providing further justification for conducting factor analysis on the dataset.

Table 5 displays, Variables with higher commonalities, such as “AI Integration” (0.385), exhibit stronger relationships with the extracted factors, whereas variables like “Learning Enhancement” (0.269) have weaker associations with the extracted factors.

Table 6 provides detailed correlation analysis matrix, Critical Thinking demonstrates moderately strong positive correlations with Creativity (0.616), Learning Enhancement (0.772), and, to a lesser extent, Ethical Awareness (0.453). It exhibits weaker positive correlations with AI Integration (0.349) and Educator Guidance (0.279).

Similarly, Creativity displays a comparable pattern of moderately strong positive correlations with Critical Thinking (0.616), Learning Enhancement (0.765), and, to a lesser extent, Ethical Awareness (0.508). It also shows a weaker positive correlation with AI Integration (0.352) and Educator Guidance (0.280).

Ethical Awareness shows moderate positive correlations with Critical Thinking (0.453) and Creativity (0.508). It has a weaker positive correlation with AI Integration (0.406) and a somewhat stronger positive correlation with Educator Guidance (0.419).

AI Integration exhibits a moderate positive correlation with Learning Enhancement (0.709) but weaker positive correlations with Critical Thinking (0.349), Creativity (0.352), Ethical Awareness (0.406), and Educator Guidance (0.438).

Learning Enhancement demonstrates strong positive correlations with Critical Thinking (0.772) and Creativity (0.765). It also displays moderate positive correlations with Ethical Awareness (0.750) and AI Integration (0.709), along with a weaker positive correlation with Educator Guidance (0.506). Educator Guidance, in turn, shows weaker positive correlations with all other variables, with the highest correlation being 0.506 with Learning Enhancement.

This Inter-Item Correlation Matrix provides insights into how the variables in study dataset are related to one another. It suggests that Critical Thinking, Creativity, and Learning Enhancement are closely related, while Ethical Awareness, AI Integration, and Educator Guidance have weaker but still meaningful relationships with the other variables. These findings can assist researchers in understanding the patterns of association between these constructs and guide further analyses or the development of measurement instruments.

5.5 Hypothesis testing

As illustrated in Table 7 , AI Integration and Critical Thinking (p label: 0.83): The high p label value of 0.83 indicates a robust positive relationship between “AI Integration” and “Critical Thinking.” This suggests that AI integration in a learning context is strongly associated with the development or enhancement of critical thinking skills. It's noteworthy that a p label value close to 1 indicates a strong positive correlation.

AI Integration and Creativity (p label: 0.67): The p label value of 0.67 reflects a moderately positive relationship between “AI Integration” and “Creativity.” This implies that AI integration in education may moderately influence fostering creativity among learners. While the correlation is positive, it is not as robust as the correlation observed with critical thinking.

AI Integration and Ethical Awareness (p label: 0.67): Similar to creativity, “AI Integration” also exhibits a moderate positive relationship with “Ethical Awareness,” with a p label of 0.67. This suggests that the incorporation of AI ( Rasul et al ., 2023 ) in management educational settings may moderately contribute to raising awareness of ethical considerations in learning and technology.

Learning Enhancement and Critical Thinking (p label: 0.87): The high p label of 0.87 indicates a strong positive correlation between “Learning Enhancement” and “Critical Thinking.” This suggests that learning enhancement measures are strongly associated with the development or improvement of critical thinking skills among students ( Farrokhnia et al ., 2023 ).

Learning Enhancement and Creativity (p label: 0.57): “Learning Enhancement” has a positive correlation with “Creativity,” but the correlation is weaker compared to critical thinking, with a p label of 0.57. While it suggests a positive relationship, it may not be as impactful as critical thinking.

Educator Guidance and Ethical Awareness (p label: 0.56): “Educator Guidance” demonstrates a moderate positive correlation with “Ethical Awareness,” as indicated by a p label of 0.56. This implies that guidance from educators may moderately contribute to the development of ethical awareness among learners ( Heyder et al ., 2023 ). The above results suggest varying degrees of positive associations between items related to AI integration ( Hinojo-Lucena et al ., 2019 ), learning enhancement, and specific skills or attributes. The strength of these associations can help educators and researchers gain a more comprehensive understanding of the potential influence of incorporating AI and improving learning approaches on critical thinking, creativity, and ethical consciousness in management educational environments (see Figure 2 ).

Considering the observed indices ( Table 8 Model Fit Index Evaluation), it is evident that the Structural Equation Model (SEM) applied to the integration of ChatGPT in the classroom, specifically in navigating the generative AI wave within management education, exhibits commendable alignment with the dataset. The notable performance, as indicated by indices such as “Chi-Square/degrees of freedom (3.31), Goodness of Fit Index (0.92), Normed Fit Index (0.91), Comparative Fit Index (0.94), and Root Mean Square Error of Approximation (0.16),” affirms the model's proficiency in elucidating the underlying relationships in the data. It is essential to benchmark these values against established standards in the field of study, taking into consideration the specific research area and aims.

6. Findings and recommendations

The study shows that using ChatGPT in management education can make learning better and improve critical thinking skills. Even though some people worry about the ethics of using ChatGPT, many students and teachers use it for things like writing assignments and exploring ideas. It's interesting that how much ChatGPT is used varies between different groups of students and teachers. Undergraduates and postgraduates use it more than doctoral students and faculty members. This might be because different people have different ideas about ethics and what tasks they need help with. But, gender and job don't seem to make a big difference in how much ChatGPT is used. The study also found that when ChatGPT is used more, it helps students think critically and become more aware of ethical issues. To use ChatGPT well, teachers need to think about ethics and give students good guidance. It's important to keep checking how well ChatGPT and other AI tools are working and to be honest and responsible about using them. By doing this, management education can make the most of ChatGPT to make learning more interesting, helpful, and fair for everyone.

7. Conclusions

Integrating ChatGPT into management education presents an innovative approach to enhancing the learning journey. This paper delves into the multifaceted integration of generative AI, like ChatGPT, within management education, highlighting its ability to engage students, tailor learning experiences, and foster critical thinking skills. As we embrace the surge of generative AI in education, it's vital to consider insights and recommendations from field experts. Maintaining a balance between human and AI-driven instruction is crucial; AI should enhance, not replace, educators' roles ( Anderson et al. , 2021 ).

Usage of ChatGPT is notably higher among undergraduate and postgraduate students compared to doctoral candidates and faculty. This difference may stem from ethical concerns associated with ChatGPT in management education and research activities ( Ray, 2023 ). Gender and occupation seem to have minimal impact on ChatGPT usage, but respondents' highest qualifications correlate with usage frequency ( Crompton and Burke, 2023 ).

Despite ethical considerations, ChatGPT sees widespread use in academic tasks like assignment writing, note preparation, and exploring new concepts ( Holmes and Tuomi, 2022 ). Ethical considerations advocate for transparency and ongoing assessment of AI-powered tools to ensure alignment with educational objectives ( Johnson et al ., 2019 ; Jones and Brown, 2021 ).

Integrating AI generative tools like ChatGPT into management education could positively impact critical thinking skills and moderately contribute to fostering creativity and ethical awareness ( Rasul et al ., 2023 ). A strong positive correlation exists between “Learning Enhancement” and “Critical Thinking,” indicating their close relationship ( Farrokhnia et al ., 2023 ). Similarly, “Educator Guidance” moderately correlates with “Ethical Awareness,” emphasizing educators' role in fostering ethical awareness among learners ( Heyder et al ., 2023 ).

In conclusion, the effective integration of ChatGPT and other generative AI technologies into management education requires an informed, balanced, and ethical approach. By leveraging research and expert guidance, educators and institutions can maximize ChatGPT's potential to enrich management education.

8. Scope for future study

The future of the study, called 'ChatGPT in the Classroom: Navigating the Generative AI Wave in Management Education,' looks into many things. It wants to see how using ChatGPT affects learning, how we can use creative AI, and if there are any problems with ethics. It also wants to know how AI affects getting a job, where else we can use it, and how to keep our information safe. Checking rules, new technology, and working together between schools and companies is important to keep management education useful as AI changes. This study starts a big look at how AI is changing management education. We need more research to understand ChatGPT better and make sure it's used right in schools.

Intentions for using ChatGPT

SEM model on ChatGPT in the classroom: Navigating the generative AI wave in management education

Reliability statistics for study constructs