systematic review of research on access and use

• Research article
• Open access
• Published: 03 January 2014

A systematic review of barriers to and facilitators of the use of evidence by policymakers

• Kathryn Oliver 1 ,
• Simon Innvar 2 ,
• Theo Lorenc 3 ,
• Jenny Woodman 4 &
• James Thomas 5  

BMC Health Services Research volume  14 , Article number:  2 ( 2014 ) Cite this article

83k Accesses

783 Citations

376 Altmetric

Metrics details

The gap between research and practice or policy is often described as a problem. To identify new barriers of and facilitators to the use of evidence by policymakers, and assess the state of research in this area, we updated a systematic review.

Systematic review. We searched online databases including Medline, Embase, SocSci Abstracts, CDS, DARE, Psychlit, Cochrane Library, NHSEED, HTA, PAIS, IBSS (Search dates: July 2000 - September 2012). Studies were included if they were primary research or systematic reviews about factors affecting the use of evidence in policy. Studies were coded to extract data on methods, topic, focus, results and population.

145 new studies were identified, of which over half were published after 2010. Thirteen systematic reviews were included. Compared with the original review, a much wider range of policy topics was found. Although still primarily in the health field, studies were also drawn from criminal justice, traffic policy, drug policy, and partnership working. The most frequently reported barriers to evidence uptake were poor access to good quality relevant research, and lack of timely research output. The most frequently reported facilitators were collaboration between researchers and policymakers, and improved relationships and skills. There is an increasing amount of research into new models of knowledge transfer, and evaluations of interventions such as knowledge brokerage.

Conclusions

Timely access to good quality and relevant research evidence, collaborations with policymakers and relationship- and skills-building with policymakers are reported to be the most important factors in influencing the use of evidence. Although investigations into the use of evidence have spread beyond the health field and into more countries, the main barriers and facilitators remained the same as in the earlier review. Few studies provide clear definitions of policy, evidence or policymaker. Nor are empirical data about policy processes or implementation of policy widely available. It is therefore difficult to describe the role of evidence and other factors influencing policy. Future research and policy priorities should aim to illuminate these concepts and processes, target the factors identified in this review, and consider new methods of overcoming the barriers described.

Peer Review reports

Despite an increasing body of research on the uptake and impact of research on policy, and encouragement for policymaking to be evidence-informed [ 1 ], research often struggles to identify a policy audience. The research-policy gap’ is the subject of much commentary and research activity [ 2 – 4 ]. Interventions to bridge this gap are the focus of recent systematic reviews [ 5 – 7 ]. To ensure these interventions are appropriately designed and effective, it is important that they address genuine barriers to research uptake, and utilise facilitators which are likely to affect research uptake.

It is now well recognized that policy is determined as much by the decision-making context (and other influences) as by research evidence [ 8 , 9 ]. Policymakers’ perceptions form an important part of this story, but not the whole. Innvaer [ 10 ] aimed to review studies about the health sector, but the influence of the evidence-based policy movement is now recognized to be important across many policy areas. In the UK, with the creation of Clinical Commissioning Groups, Health and Well-Being Boards, and private providers moving into areas traditionally occupied by the NHS, a broader range of policymakers are becoming potential evidence-users’ than ever. Researchers need to take stock of what we know about evidence-based policy, what we don’t know, and what can be done to assist these users.

The last systematic review looking at policymakers’ perceptions about the barriers to, and facilitators of research use was Innvaer [ 10 ]. The findings from this review were corroborated by later research [ 11 , 12 ], but no systematic update has yet been undertaken. In addition to updating this review in the area of policymakers’ perceptions of barriers and facilitators to use of evidence in policy, we also wished to include perceptions from other stakeholder groups than policymakers, such as researchers, managers, and other research users. Furthermore, it may be possible to identify factors affecting research use without relying on the perceptions of research participants – for example, ethnographic studies may produce observational data about knowledge exchange. In addition, we acknowledge that interest in using evidence to inform policy has spread beyond the health sector. Therefore, we aimed to update Innvaer [ 10 ] to include studies identifying all barriers and facilitators of the use of evidence in all policy fields.

This review aimed to update and expand Innvaer [ 10 ], and broaden the scope of the review to:

 Identify factors which act as barriers to and facilitators of the use of evidence in public policy, including factors perceived by different stakeholder groups.

 Describe the focus, methods, populations, and findings of the new evidence in this area.

Because this review has a larger scope that Innvaer [ 10 ], caution must be used in drawing direct comparisons; discussed further in the results.

A protocol for the review was developed and sent to an advisory group of senior academics (available from KO) in order to ensure that the methods and search strategies were exhaustive.

To be included, studies had to be:

 Primary research (any study design) or systematic reviews categorising, describing or explaining how evidence is used in policymaking. Intervention studies were included.

 About policy (defined as decisions made by a state organisation, or a group of state organisations, at a national, regional or conurbation level). Studies of clinical decision-making for individual patients, or protocols for single clinical sites were excluded.

 About barriers or facilitators to the use of evidence (relational, organisational, factors related to researchers, policymakers, policy or research directly, or others).

We did not exclude any studies on the basis of population. These criteria are therefore broader than those for Innvaer [ 10 ] by including all study designs, all populations and all policy areas.

The following electronic databases were searched using adapted search strings from Innvaer [ 10 ] from July 2000 (the cut-off point for the earlier review) - September 2012: Medline, Embase, SocSci Abstracts, CDS, DARE, Psychlit, Cochrane Library, NHSEED, HTA, PAIS, IBSS. Searches combined policy’ terms with utilisation/use’ terms in the first instance. The full search strategy is available from the corresponding author on request; sample search available here (Additional file 1 ). Authors in the field were contacted and key websites were hand-searched. In order to pick up a range of study designs and theoretical papers a methodological filter was not applied.

All studies were screened initially on title and abstract. 100 studies were double screened to ensure consistency, and revisions were made to definitions and criteria accordingly. Relevant studies were retrieved and screened on full text by one reviewer.

Studies were stored, screened and keyworded using the EPPI Reviewer software [ 13 ]. Data were extracted on study characteristics, sampling and recruitment, theoretical framework, methods, and results, with all studies being coded by one reviewer, and two reviewers coding 10-25% each (67 studies were double-coded in total). Because we were not aiming to determine the size of an effect, but instead to describe a body of literature, no risk of bias assessment was made. Quality appraisal in this case would have made no difference to this systematic descriptive synthesis.

Studies were keyworded using a data extraction tool which collected information on study characteristics, topic and focus, and theoretical background. Factors which affected evidence use were coded as barriers or facilitators against a pre-defined list of factors, which was iteratively updated as new factors were identified. All studies were therefore coded at least twice, once with the initial tool, and once with the finalised list of factors.

6879 unique records were retrieved, of which 430 were screened on full text. 145 studies were included on full text, of which half were published between September 2010 and September 2012. Figure  1 describes the flow of studies through searching and screening for inclusion.

PRISMA flowchart detailing flow of studies through the review.

Characteristics of included studies

For a full description of the included studies, see Additional file 2 . Studies were undertaken in a wide range of countries (145 studies in over 59 countries,). A significant proportion (n = 33, 23%) were from low- and middle-income countries in regions such as sub-Saharan Africa and Central America (n = 32), and several were conducted in Middle-Eastern states (n = 4).

Eleven studies used observational (ethnographic) methods to collect data, and 37 used documentary analysis. However, these represent less than a quarter of included studies, the majority of which were or included semi-structured interviews (n = 79), or included a survey (n = 44). Twelve studies were longitudinal while the rest were cross-sectional. Thirteen systematic reviews and fifty-three case studies were included.

Most studies reported perceptions or experiences of respondents (n = 109; n = 64 respectively), rather than documentary proof or observational results about the use of evidence in policy (n = 14; n = 11 respectively). Evidence’ was defined in 121 studies. Where it was possible to identify what kinds of evidence were being discussed, most focused on the use of research evidence (n = 90) with 33 focusing specifically on systematic reviews. However, 59 studies looked at the use of non-formal evidence, which included local data, surveillance data, personal experience, clinical expertise, or other informal knowledge.

The context of the study was usually non-specific, referring to general policy (n = 84) or practice (n = 37). Changes to specific policy areas or policies were explored in 22 and 14 studies respectively, and information/evidence diffusion in 13. Some studies explicitly set out to look at uptake or adoption of research (n = 41), and others described interventions aiming to increase uptake [ 14 ], or the context after a specific piece of research or policy (such as after the introduction of the 1999 White Paper “Saving Lives: our healthier nation” [ 15 , 16 ]. The vast majority of studies were conducted in health or health-related fields. Most new evidence in the area focused on the health sector, but research was also conducted in areas including traffic [ 17 , 18 ], criminal justice [ 19 – 23 ], drugs policy [ 22 , 24 – 37 ], and environmental conservation [ 20 , 22 , 38 – 42 ] (see Figure  2 ).

Policy focus of study.

Who are these studies written by and for?

137 study reports were written by researchers or people with academic affiliations, with clinical researchers co-authoring a proportion of these (n = 57). Policymakers were credited as authors in 3 studies, [ 25 , 39 , 43 ] and one of those was a governmental report.

The population samples themselves were predominantly policymakers or advisors (n = 86, n = 32 respectively), health care managers (n = 49), or researchers (49), although many other groups were also included (see Figure  3 ). Where researchers were included in the study population (n = 49), they often outnumbered the policy and practice participants. Other participants included commissioners, health economists, third sector workers, patients, industry and business representatives, and justice and criminal workers. Because it was not always clear who had been involved and what their roles were, it was not possible to give numbers for all these groups. Also included in this other’ category (n = 62) were all documents analysed.

Sample population.

What factors affect use of evidence?

All studies reported either barriers, facilitators, or both, of the use of evidence. Studies also described processes of research use (n = 50), strategies and interventions to increase research use (n = 24), and assessments of the uptake of research (n = 33) (see Figure  4 ).

Main barriers and facilitators of the use of evidence by policymakers.

Studies reported a range of factors which acted as barriers and/or facilitators of evidence use. The most frequently reported barriers were the lack of availability to research, lack of relevant research, having no time or opportunity to use research evidence, policymakers’ and other users not being skilled in research methods, and costs (see Table  1 ). The most frequently reported facilitators also included access to and improved dissemination of research, and existence of and access to relevant research. Collaboration and relationships between policymakers and research staff were all reported as important factors.

To interpret all the factors reported by included studies, the barriers and facilitators were categorised into themes depending on content: Organisations and resources’ , Contact and collaboration’ , Research and researcher characteristics’ , Policymaker characteristics’ , Policy characteristics’ , and Other’ (see Table  2 ). Below, we describe the main barriers and facilitators reported within each theme, and we give some supplementary information not mentioned in the table.

Contact and relationships

Contact, collaboration and relationships are a major facilitator of evidence use, reported in over two thirds of all studies. Timing and opportunity was the most prominent barrier (n = 42) within this theme. Many studies also discussed the role of relationships, trust, and mutual respect. The serendipitous nature of the policy process was emphasised in some studies, which discussed the role of informal, unplanned contact in policy development and in finding evidence.

Organisations and resources

Organisational factors such as lack of access to research, poor dissemination and costs were highly reported factors affecting the use of research. Other barriers were lack of managerial support, professional bodies, material and personnel resources, managerial will and staff turnover. Professional bodies were seen as barriers where useful guidelines were not available, or where they were perceived to be political or biased. In the case of the WHO, it was seen as unreliable, unsupportive, and with dubious claims to be evidence-based’ [ 31 , 44 ]. Other factors mentioned in connection with organisational and resource barriers included poor long term policy planning [ 45 ], inflexible and non-transparent policy processes [ 46 , 47 ] and in developing countries, lack of effective health care systems [ 24 ]. Leadership and authority were reported as facilitators, with emphasis on community leadership [ 48 ] and policy entrepreneurialism of policy champions [ 43 , 49 ].

Among the facilitators under the theme Organisation and Resources , availability, access and dissemination were considered important facilitators, as was managerial support (n = 22).

Research and researcher characteristics

Characteristics of research evidence were widely reported as factors affecting uptake of research, with clarity, relevance and reliability of research findings reported as important factors. The format of research output was also an important factor in uptake. The importance of the research findings themselves was discussed in 19 studies, usually studies describing the uptake of health inequalities research. The quality and authoritativeness of research was clearly a factor in uptake, particularly where other evidence in the area was poor quality [ 50 ].

Emerging as a new stream of research, eleven studies evaluated or described knowledge broker roles or related concepts [ 6 , 35 , 37 , 51 – 57 ] with dedicated dissemination strategies evaluated in 7 studies and mentioned as a facilitator in 43. Incentives to use evidence and client demand for research evidence were described as facilitators in one study each [ 10 , 58 ].

Researchers themselves were described as factors affecting uptake of their research. Having a good understanding of the policy process and the context surrounding policy priorities was supportive of research uptake [ 17 , 18 , 59 – 61 ]. A barrier to uptake was identified where researchers were described as having different priorities from policymakers, with pressure to publish in peer-reviewed journals [ 27 , 62 , 63 ]. Researchers were valued more when it was clear they were non-partisan and producing unbiased results [ 40 , 57 , 64 ], and provision of expert advice was also reported as helpful.

Policymaker characteristics

Policymakers’ characteristics were also reported to play a role in evidence uptake, with their research skills and awareness (or lack of) reported as a barrier in 34 studies. Some studies reported that policymakers’ beliefs about the utility of evidence-use was a major factor in evidence use (barrier: n = 2, facilitator: n = 3), and, in general, personal experiences, judgments, and values were reported as important factors in whether evidence was used. However, these findings were nearly all (91%) based on studies of perceptions, of which half were perceptions of researchers.

Some studies reported that left-leaning, younger and/or female policymakers were more likely to use research evidence [ 65 , 66 ]. Being more highly educated was reported as a barrier [ 67 ], but there was no consensus about the effect of being clinically trained [ 61 , 68 ].

Policy characteristics

Perhaps surprisingly, legal support and the existence of guidelines for the use of evidence were scarcely reported as factors affecting uptake of evidence. The importance and complexity of the policy area was also discussed, especially in comparison with the relative simplicity of clinical problems.

However, competing pressures (economic, political, social, and cultural factors) were seen to impact on the policy process and hinder the development of evidence-based policy. Political pressures, finances, and competing priorities were all discussed (n = 12), with the media (n = 3) vested interest and pressure/lobby groups (n = 3) and unclear decision-making practices (n = 2) also reported as barriers.

Other factors

One study which studied use of evidence in prisons reported potential security breaches from data loss as a potential barrier to evidence use [ 19 ]. Other studies reported consumer-related barriers (such as issues around privacy and choice [ 49 , 55 , 69 ]), differences between types of policymaker (such as civil servants vs. managers) [ 29 , 70 ] and public opinion. External events were reported as a facilitator in one study. The role of local context, contingency, and serendipity in influencing policy processes and outcomes overall emerged as a theme throughout the results.

Comparing with Innvaer (2002): focus of new evidence in the area

There are differences between the reviews (see Table  3 ), in part reflecting the broader inclusion criteria for this update. However, it is clear that interest in studying the use of evidence has spread beyond the health sector, with more attention from other public policy domains. In addition, there is an increase in publications from low-and middle income countries, where the contexts, barriers and pressures on policymakers in these countries are likely to be very different from those in high-income countries. However, the main research methods used by included studies, and the results generated by those methods, are similar. Despite this increase in research attention, there is still a remarkable dearth of reliable empirical evidence about the actual processes and impacts of research and other evidence use in policy.

This systematic review aimed to identify and describe research about the barriers and facilitators of the use of evidence for policy, expanding on and updating Innvaer [ 10 ]. It found that organisational factors, including availability and access to research were considered to be important influences on whether evidence was used in policy, and the quality of the relationship and collaboration between researchers and policymakers to be the single most mentioned facilitator.

The findings of the updated systematic review presented here were consistent with the original review. We can have a high degree of confidence that it is possible to identify factors likely to influence research uptake, as the expanded field of research synthesised here demonstrates. However, it is less clear what we can learn from this research. For example, there was a high degree of consistency in the findings, even though studies from very different contexts were included. It seems plausible that developing countries would have different barriers from wealthy countries; or that criminal justice would have systematically different pressures from health policy. The similarities reported in these studies may be accounted for by the similarity in approach and methods used. Indeed, the impact and contributions of research to policy (and vice versa) are still unclear, with few studies exploring how, when and why different facilitators and barriers come into play during the policymaking process, or developing an understanding of how research impact on policy and populations might be evaluated. However, there are undoubtedly wider questions about how impact may be defined and measured which are, as yet, unanswered. While perceptions and attitudes are of course important to illuminating the policy process, but there are likely to be other ways - for example documentary, historical, ethnographic or network analyses - in which the role of evidence could be, unpicked [ 71 ].

Over a third of the included studies mentioned use of informal evidence such as local data or tacit knowledge. Researchers are starting to recognize that research evidence is just one source of information for policymakers [ 72 ]. Identifying these sources and types of information are a crucial step in describing and ultimately influencing the policy process. However, most studies do not define what they mean by evidence’, hampering attempts to understand the process. Interventions addressing barriers specifically are unlikely to influence policy without a detailed understanding of all these factors.

Studies in this area continue to be mainly written by and for researchers, with a lack of attention given to the policy process or policymakers’ priorities. Most studies asked researchers about their perspectives. Where mixed populations were included, the researchers often outnumbered the other participants. Involving policymakers in designing and writing a study which looks at these issues in conjunction with barriers and facilitators may be fruitful. Until then, it is hard to defend academics from the charge of misunderstanding policy priorities or processes – a charge first made explicit over 20 years ago [ 73 ].

Strengths and weaknesses of the study

The review is exhaustive, and we followed a pre-published protocol and rigorous review methods, including the advice of an advisory group (details available from the corresponding author) (see Additional file 3 for a PRISMA checklist report). However, this paper has only counted the frequencies with which factors are mentioned without any weighting. Without more research, it is difficult to say what impact different factors might have.

Most studies still employ relatively superficial methods such as surveys or short interviews. These were all based on self-reports, however, so given the contentious nature of the topic combined with understandable fear of audit/performance monitoring these results may not be reliable. However, there is some evidence that researchers are employing impact assessment, intervention, or observational studies as well to explore how evidence and policy are related. We were unable to double-screen and double-code all studies due to lack of resources. However, all studies were data-extracted at least twice (once at the beginning, and again with the finalized list of factors which was developed iteratively) so we have confidence in the consistency of approach. No methodological assessment of included studies was undertaken, as this was primarily a descriptive exercise. In addition, the heterogeneity of study designs and the difficulty of comparing quality across these domains limited the usefulness of such an exercise. Quality appraisal would be a valuable step in any in-depth review of a subset of these studies.

Strengths and weaknesses in relation to other studies

Recent systematic reviews in the area have focused on the use of research evidence, [ 74 ] or on the impact of research evidence on policy [ 5 ]. Orton [ 75 ], included in this review, does not include any evaluations of evidence use, ethnographies, or case studies, relying only on self-report questionnaires and interviews to provide the results. Without empirical data exploring access to information and perceived impact [ 74 ], and without investigating the policy process, or testing current theories about knowledge utilization, it is hard to draw useful conclusions. Few studies have systematically appraised the use of evidence in this wider sense.

The reviews all found similar findings with regard to barriers and facilitators of the use of evidence. There still appears to be a need for high-quality, simple, clear and relevant research summaries, to be delivered by known and trusted researchers.

Possible change in future research practice and policymaking

Most studies in this review are descriptive. Because most studies do not go into the content of the facilitators and barriers they identified, we know little about when, why and how the identified barriers and facilitators come into play in the use of evidence in policymaking. Based on this review, future research can use Table  2 to identify themes and factors relevant for their field of research, be it organisations, collaboration, research, researchers, policymakers or policy. Identifying the content and relative importance of these factors and new undiscovered factors in different contexts, at different levels, or in different countries, may contribute to our understanding of evidence use in policy.

One future objective for researchers can be drawn from the results found in Table  1 , namely that four of the five top barriers to the use of evidence is a lack of relevance and importance. If research becomes available, the possibility of increased use improves. If policymakers’ research skills improve, calculations of costs will become more accurate. The natural question is to explore why policymakers do not prioritize overcoming barriers relating to themselves. The barrier called lack of clarity, relevance and reliability of research calls for change in researchers’ objectives and methods, but we need to know what policymakers define as clear, relevant and reliable research, and why and when policymakers will use such research. Of special relevance to this question, is the research on knowledge translation done in the past five to ten years, which formed a new strand of research. This body of work draws on the theory that interpersonal relations are important for knowledge exchange, through employing knowledge brokers or similar. There has also been a growth in resources aimed at helping decision-makers to navigate research evidence, such as Cochrane-produced evidence summaries. These are not only aimed at practitioners within the health field, and the knowledge translation field will hopefully soon make efforts at addressing the broader issues around evidence use in policy more widely to identify underlying mechanisms behind knowledge use.

This review looked for all barriers and facilitators of the used of evidence in policy. Most studies collected research and policy actors’ perceptions about factors affecting the use of research evidence, with a large minority surveying only researchers. Understanding how to alleviate these barriers is hampered by a lack of clarity about how evidence’ is defined by studies, with fewer than half specifying what kinds of information were discussed. Most studies however focused on uptake of research evidence, as opposed to evidence more widely. Research into how to alleviate organisational and resource barriers effectively would be welcomed. Additionally, all such research should be based on an understanding that a broader interpretation of “evidence” than “research-based” evidence is also essential.

Stakeholders perceive relationships to be essential elements of the policy process. However, few studies use dedicated relational methods such as network analysis to study policy communities or the policy process, with a few exceptions [ 75 , 76 ].

Several new strands of research offer encouragement to researchers in the area. Firstly, learning from political sciences and management studies is filtering into the EBP debates, as can be seen from the attention paid to leadership and organisational factors. Research into policy entrepreneurship and knowledge brokerage also formed a significant subset of studies. However, there remains a need for empirical evidence to be generated about the policy process. The barriers and facilitators generated above refer specifically to the use of evidence; however, it is equally possible that similar factors affect the policy process in general (for example, constraints on resources, personnel and costs are likely to affect all policy decisions). Identification and exploration of all factors influencing policy, not just those relating to evidence, should be of interest to researchers; however, this is outside the scope of this review.

Finally, little empirical evidence about the processes or impact of the use of evidence by policy is presented by these studies. Despite the increased amount of research on interventions to increase research use in policy, this is not linked with research about the impact of policy on populations, or of evidence use on population outcomes. Much of the literature is concerned with policymaking; but policymakers’ time is spend on implementation. To justify the continuing rhetoric about the importance of research use, and the ever-increasing amount of research into the area, it is surely essential that we practise what we preach and generate evidence about the process and effectiveness of research use in policy.

“What this paper adds” box

Section 1: what is already known on this subject.

Little is known about the role of research in policymaking. A previous systematic review (Innvaer [ 10 ]) identified the main barriers and facilitators of the use of evidence. Although subsequent reviews have been conducted, they have focused on specific types of evidence, such as economic analyses (Williams) or systematic reviews (Best), or on first-world countries (Orton). Given the explosion of research in the area, an update of the original review was carried out.

Section 2: What this study add

The most often mentioned facilitators of the use of evidence are still reported to be relationships, contact and collaboration, availability and access to research, and relevant, reliable and clear research findings. A lack of relevant, reliable and clear research findings, and poor availability and access to research, are the most often mentioned barriers to policymakers’ use of research.

Research into EBP has spread across a wide range of policy areas and countries, including those from low and middle-income countries. New strands of research focus on knowledge translation, knowledge brokerage, and other interventions to increase uptake of evidence. Little research exists about the process, impact or effectiveness of how, when and why research is used during the policy process.

This study did not require ethics approval.

Data sharing: Full dataset and search strategies are available from Kathryn Oliver at [email protected]. Consent was not obtained as this study had no participants.

Macintyre S, Chalmers I, Horton R, Smith R: Using evidence to inform health policy: case study. BMJ. 2001, 322: 222-225. 10.1136/bmj.322.7280.222.

Article   CAS   PubMed   PubMed Central   Google Scholar  

Nutbeam D: Getting evidence into policy and practice to address health inequalities. Health Promot Int. 2004, 19: 137-140.

Article   PubMed   Google Scholar  

Oakley A: Evidence-informed policy and practive: challenges for social science. Educational Research and Evidence-Based Practive. Edited by: Hammersley M. 2007, London: SAGE, 91-105.

Google Scholar  

Pawson R: Evidence-Based Policy: A Realist Perspective. 2006, London: Sage Publications Ltd

Book   Google Scholar  

Boaz A, Baeze J, Fraser A: Effective implementation of research into practice: an overview of systematic reviews of the health literature. BMC Res Notes. 2011, 4.1: 212.

Article   Google Scholar  

Murthy L, Shepperd S, Clarke M, Garner S, Lavis J, Perrier L, et al: Interventions to improve the use of systematic reviews in decision-making by health system managers, policymakers and clinicians. Cochrane Database Syst Rev. 2012, 12: CD009401.

Perrier L, Mrklas K, Lavis J, Straus S: Interventions encouraging the use of systematic reviews by health policymakers and managers: a systematic review. Implementation Sci. 2011, 6: 43-10.1186/1748-5908-6-43.

Dobrow MJ, Goel V, Lemieux-Charles L, Black NA: The impact of context on evidence utilization: a framework for expert groups developing health policy recommendations. Soc Sci Med. 2006, 63: 1811-1824. 10.1016/j.socscimed.2006.04.020.

Morrato E, Elias M, Gericke C: Using population-based routine data for evidence-based health policy decisions: lessons from three examples of setting and evaluating national health policy in Australia, the UK and the USA. J Public Health. 2008, 29 (4): 463-471.

Innvaer S, Vist G, Trommald M, Oxman A: Health policymakers’ perceptions of their use of evidence: a systematic review. J Health Serv Res Policy. 2002, 7: 239-244. 10.1258/135581902320432778.

Lavis J, Davies H, Oxman A, Denis JL, Golden-Biddle K, Ferlie E: Towards systematic reviews that inform health care management and policymaking. J Health Serv Res Policy. 2005, 10: 35-48. 10.1258/1355819054308549.

Orton L, Lloyd-Williams F, Taylor-Robinson D, O’Flaherty M, Capewell S: The use of research evidence in public health decision making processes: systematic review. PLoS One. 2011, 6: e21704-10.1371/journal.pone.0021704.

Thomas J, Brunton J: EPPI-Reviewer 3.0: Analysis and Management of Data for Research Synthesis. EPPI-Centre Software. [3.0]. 2006, London: Social Science Research Unit, Institute of Education, Ref Type: Computer Program

Gagliardi AR, Fraser N, Wright FC, Lemieux-Charles L, Davis D: Fostering knowledge exchange between researchers and decision-makers: exploring the effectiveness of a mixed-methods approach. Health Policy. 2008, 86: 53-63. 10.1016/j.healthpol.2007.09.002.

Department of Health: Saving Lives: Our Healthier Nation. 1999, London: HMSO, Ref Type: Report

Learmonth AM: Utilizing research in practice and generating evidence from practice. Health Educ Res. 2000, 15: 743-756. 10.1093/her/15.6.743.

Article   CAS   PubMed   Google Scholar  

Hinchcliff R, Ivers R, Poulos R, Senserrick T: Utilization of research in policymaking for graduated driver licensing. Am J Public Health. 2010, 100: 2052-2058. 10.2105/AJPH.2009.184713.

Article   PubMed   PubMed Central   Google Scholar  

Hinchcliff R, Poulos R, Ivers RQ, Senserrick T: Understanding novice driver policy agenda setting. Public Health. 2011, 125: 217-221. 10.1016/j.puhe.2011.01.001.

Anaraki S, Plugge E: Delivering primary care in prison: the need to improve health information. Inform Prim Care. 2003, 11: 191-194.

PubMed   Google Scholar  

Bédard P, Ouimet M: Cognizance and consultation of randomized controlled trials among ministerial policy analysts. Rev Policy Res. 2012, 29: 625-644. 10.1111/j.1541-1338.2012.00581.x.

Henderson CE, Young DW, Farrell J, Taxman FS: Associations among state and local organizational contexts: Use of evidence-based practices in the criminal justice system. Drug Alcohol Depend. 2009, 103 (Suppl 1): S23-S32.

Jennings ET, Hall JL: Evidence-based practice and the use of information in state agency decision making. J Public Adm Res Theory. 2012, 22: 245-266. 10.1093/jopart/mur040.

Stevens A: Telling policy stories: an ethnographic study of the use of evidence in policymaking in the UK. J Soc Policy. 2011, 40: 237-255. 10.1017/S0047279410000723.

Aaserud M, Lewin S, Innvaer S, Paulsen E, Dahlgren A, Trommald M, et al: Translating research into policy and practice in developing countries: a case study of magnesium sulphate for pre-eclampsia. BMC Health Serv Res. 2005, 5: 68-10.1186/1472-6963-5-68.

Albert MA, Fretheim A, Maiga D: Factors influencing the utilization of research findings by health policymakers in a developing country: the selection of Mali’s essential medicines. Health Res Policy Syst. 2007, 5: 2-10.1186/1478-4505-5-2.

Bickford J, Kothari A: Research and knowledge in Ontario tobacco control networks. Canadian J Public Health Rev Can Sante Publique. 2008, 99: 297-300.

Burris H, Parkhurst J, Du-Sarkodie Y, Mayaud P: Getting research into policy - herpes simplex virus type-2 (HSV-2) treatment and HIV infection: international guidelines formulation and the case of Ghana. Health Res Policy Syst. 2011, 9 (Suppl 1): S5-10.1186/1478-4505-9-S1-S5.

Currie L, Clancy L: The road to smoke-free legislation in Ireland. [References]. Addiction. 2011, 106: 15-24. 10.1111/j.1360-0443.2010.03157.x.

Frey K, Widmer T: Revising swiss policies: the influence of efficiency analyses. Am J Eval. 2011, 32: 494-517. 10.1177/1098214011401902.

Greyson DL, Cunningham C, Morgan S: Information behaviour of Canadian pharmaceutical policymakers. Health Info Libr J. 2012, 29: 16-27. 10.1111/j.1471-1842.2011.00969.x.

Hutchinson E, Parkhurst J, Phiri S, Gibb DM, Chishinga N, Droti B, et al: National policy development for cotrimoxazole prophylaxis in Malawi, Uganda and Zambia: the relationship between context, evidence and links. Health Res Policy Syst. 2011, 9 (Suppl 1): S6-10.1186/1478-4505-9-S1-S6.

Innvaer S: The use of evidence in public governmental reports on health policy: an analysis of 17 Norwegian official reports (NOU). BMC Health Serv Res. 2009, 9: 177-10.1186/1472-6963-9-177.

Kurko T, Silvast A, Wahlroos H, Pietila K, Airaksinen M: Is pharmaceutical policy evidence-informed? A case of the deregulation process of nicotine replacement therapy products in Finland. Health Policy. 2012, 105: 246-255. 10.1016/j.healthpol.2012.02.013.

Rieckmann TR, Kovas AE, Cassidy EF, McCarty D: Employing policy and purchasing levers to increase the use of evidence-based practices in community-based substance abuse treatment settings: Reports from single state authorities. Eval Program Plann. 2011, 34: 366-374. 10.1016/j.evalprogplan.2011.02.003.

Ritter A: How do drug policymakers access research evidence?. Int J Drug Policy. 2009, 20: 70-75. 10.1016/j.drugpo.2007.11.017.

Rocchi A, Menon D, Verma S, Miller E: The role of economic evidence in Canadian oncology reimbursement decision-making: to lambda and beyond. Value Health. 2008, 11: 771-783. 10.1111/j.1524-4733.2007.00298.x.

Wang A, Baerwaldt T, Kuan R, Nordyke R, Halbert R: Payer perspectives on evidence for formulary decision making in the United States. Value Health. 2011, 14: A350.

Carneiro M, Silva-Rosa T: The use of Scientific Knowledge in the Decision Making Process of Environmental Public Policies in Brazil. 2011, Science Communication: Journal of, 10.

Comptroller and Auditor General of the National Audit Office: Getting the Evidence: Using Research in Policymaking. 2003, London: Stationery Office

Deelstra Y, Nooteboom SG, Kohlmann HR, Berg J, Innanen S: Using knowledge for decision-making purposes in the context of large projects in The Netherlands. Environ Impact Assess Rev. 2003, 23: 517-541. 10.1016/S0195-9255(03)00070-2.

Ortega-Argueta A, Baxter G, Hockings M: Compliance of Australian threatened species recovery plans with legislative requirements. J Environ Manage. 2011, 92: 2054-2060. 10.1016/j.jenvman.2011.03.032.

Weitkamp G, Van den Berg AE, Bregt AK, Van Lammeren RJA: Evaluation by policymakers of a procedure to describe perceived landscape openness. J Environ Manage. 2012, 95: 17-28. 10.1016/j.jenvman.2011.09.022.

Lomas J, Brown A: Research and advice giving: a functional view of evidence-informed policy advice in a canadian ministry of health. Milbank Q. 2009, 87: 903-926. 10.1111/j.1468-0009.2009.00583.x.

Bryce J, Victora C, Habicht J, Vaghan J, Black R: The multi-country evaluation of the integrated management of childhood illness strategy: lessons for the evaluation of public health interventions. Am J Public Health. 2004, 20: 94-105.

Hivon ML: Use of health technology assessment in decision making: coresponsibility of users and producers?. Int J Technol Assess Health Care. 2005, 21: 268-275.

Flitcroft K, Gillespie J, Salkeld G, Carter S, Trevena L: Getting evidence into policy: the need for deliberative strategies?. Soc Sci Med. 2011, 72: 1039-1046. 10.1016/j.socscimed.2011.01.034.

Galani C: Self-reported healthcare decision-makers’ attitudes towards economic evaluations of medical technologies. Curr Med Res Opin. 2008, 24: 3049-3058. 10.1185/03007990802442695.

Lencucha R, Kothari AR, Hamel N: Extending collaborations for knowledge translation: lessons from the community-based participatory research literature. Evid Policy. 2010, 6: 75.

Brambila C, Ottolenghi E, Marin C, Bertrand J: Getting results used: evidence from reproductive health programmatic research in Guatemala. Health Policy Plan. 2007, 22: 234-245. 10.1093/heapol/czm013.

Hamel N, Schrecker T: Unpacking capacity to utilize research: a tale of the burkina faso public health association. Soc Sci Med. 2011, 72: 31-38. 10.1016/j.socscimed.2010.09.051.

Bunn F: Strategies to promote the impact of systematic reviews on healthcare policy: a systematic review of the literature. Evid Policy. 2011, 7: 428.

Campbell D, Donald B, Moore G, Frew D: Evidence check: knowledge brokering to commission research reviews for policyAN - 857120904; 4175308. Evid Policy. 2011, 7: 97-107. 10.1332/174426411X553034.

Dobbins M, Robeson P, Ciliska D, Hanna S, Cameron R, O’ Mara L, et al: A description of a knowledge broker role implemented as part of a randomized controlled trial evaluating three knowledge translation strategies. Implementation Science. 2009, 4.23: 1-16.

El-Jardali F, Lavis JN, Ataya N, Jamal D: Use of health systems and policy research evidence in the health policymaking in eastern Mediterranean countries: views and practices of researchers. Implementation Science. 2012, 7: 2-10.1186/1748-5908-7-2.

Jack SM: Knowledge transfer and exchange processes for environmental health issues in Canadian Aboriginal communities. Int J Environ Res Public Health. 2010, 7: 651-674. 10.3390/ijerph7020651.

Jonsson K: Health systems research in Lao PDR: capacity development for getting research into policy and practice. Health Res Policy Syst. 2007, 5.

Ward V, Smith S, House A, Hamer S: Exploring knowledge exchange: a useful framework for practice and policy. Soc Sci Med. 2012, 74: 297-304. 10.1016/j.socscimed.2011.09.021.

Williams I, McIver S, Moore D, Bryan S: The use of economic evaluations in NHS decision-making: a review and empirical investigation. Health Technol Assess. 2008, 12: iii-175.

Friese B, Bogenschneider K: The voice of experience: How social scientists communicate family research to policymakers. Family Relations. 2009, 58: 229-243. 10.1111/j.1741-3729.2008.00549.x.

Petticrew M, Whitehead M, Macintyre SJ, Graham H, Egan M: Evidence for public health policy on inequalities: 1: the reality according to policymakers. J Epidemiol Community Health. 2004, 58: 811-816. 10.1136/jech.2003.015289.

Smith K, Joyce K: Capturing complex realities: understanding efforts to achieve evidence-based policy and practice in public health. Evidence and Policy. 2012, 8: 78.

Bunn F, Kendall S: Does nursing research impact on policy? A case study of health visiting research and UK health policy. [References]. J Res Nurs. 2011, 16: 169-191. 10.1177/1744987110392627.

Hyder AA, Corluka A, Winch PJ, El-Shinnawy A, Ghassany H, Malekafzali H, et al: National policymakers speak out: are researchers giving them what they need?. Health Policy Plan. 2011, 26: 73-82. 10.1093/heapol/czq020.

Hird JA: Policy analysis for what? The effectiveness of nonpartisan policy research organizations. Policy Stud J. 2005, 33: 83-105. 10.1111/j.1541-0072.2005.00093.x.

Olson B, Armstrong EP, Grizzle AJ, Nichter MA: Industry’s perception of presenting pharmacoeconomic models to managed care organizations. J Manag Care Pharm. 2003, 9: 159-167.

Brownson RC, Dodson EA, Stamatakis KA, Casey CM, Elliott MB, Luke DA, et al: Communicating evidence-based information on cancer prevention to state-level policymakers. J Natl Cancer Inst. 2011, 103: 306-316. 10.1093/jnci/djq529.

Larsen M, Gulis G, Pedersen KM: Use of evidence in local public health work in Denmark. Int J Public Health. 2012, 57: 477-483. 10.1007/s00038-011-0324-y.

Dobbins M, Cockerill R, Barnsley J, Ciliska D: Factors of the innovation, organization, environment, and individual that predict the influence five systematic reviews had on public health decisions. Int J Technol Assess Health Care. 2001, 17: 467-478.

Jewell CJ, Bero LA: “Developing good taste in evidence”: facilitators of and hindrances to evidence-informed health policymaking in state government. Milbank Q. 2008, 86: 177-208. 10.1111/j.1468-0009.2008.00519.x.

McDavid JC, Huse I: Legislator uses of public performance reports: Findings from a five-year study. Am J Eval. 2012, 33: 7-25. 10.1177/1098214011405311.

Wieshaar H, Collin J, Smith K, Gruning T, Mandal S, Gilmore A: Global health governance and the commercial sector: a documentary analysis of tobacco company strategies to influence the WHO framework convention on tobacco control. PLoS Med. 2012, 9: e1001249-10.1371/journal.pmed.1001249.

Haynes AS, Gillespie JA, Derrick GE, Hall WD, Redman S, Chapman S, et al: Galvanizers, guides, champions, and shields: the many ways that policymakers use public health researchers. Milbank Q. 2011, 89: 564-598. 10.1111/j.1468-0009.2011.00643.x.

Weiss C: The many meanings of research utilisation. Public Adm Rev. 1979, 39: 426-431. 10.2307/3109916.

Orton L: The Use of research evidence in public health decision making processes: systematic review. PLoS One. 2011, 6: e21704-10.1371/journal.pone.0021704.

Lewis JM: Being around and knowing the players: networks of influence in health policy. Soc Sci Med. 2006, 62: 2125-2136. 10.1016/j.socscimed.2005.10.004.

Oliver K, de Vocht F, Money A, Everett MG: Who runs public health? A mixed-methods study combining network and qualitative analyses. J Public Health. 2013, 35: 453-459. 10.1093/pubmed/fdt039.

Pre-publication history

The pre-publication history for this paper can be accessed here: http://www.biomedcentral.com/1472-6963/14/2/prepub

Download references

Acknowledgements

We thank Frank de Vocht for his comments on the manuscript.

This systematic review was not funded, and no funders had input into design or conduct of the review. KO was part-funded by the EU Commission, under the 7th Framework Programme (200802013 DG Research) as part of the EURO-URHIS 2 project (FP7 HEALTH-2, 223711). Theo Lorenc was funded by the NIHR School for Primary Care, London School of Hygiene and Tropical Medicine. We received guidance from an advisory group of academics and practitioners, for which we are grateful.

All authors have access to the data and can take responsibility for the integrity and accuracy of the study.

Author information

Authors and affiliations.

School of Social Sciences, University of Manchester, Bridgeford Street, M13 9PL, Manchester, UK

Kathryn Oliver

Faculty of Social Sciences, Oslo University College, P.B. 4, St. Olavs Plass, NO-0130, Oslo, Norway

Simon Innvar

Department of Science, Technology, Engineering, and Public Policy (UCL STEaPP), University College London, 66-72 Gower Street, London, WC1E 6EA, UK

Theo Lorenc

MRC Centre of Epidemiology for Child Health, Institute of Child Health, London, WC1N 1EH, UK

Jenny Woodman

University of London, Institute of Education, 20 Bedford Way, London, WC1H 0AL, UK

James Thomas

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Kathryn Oliver .

Additional information

Competing interests.

All authors have completed the Unified Competing Interest form at http://www.icmje.org/coi_disclosure.pdf (available on request from the corresponding author) and declare that none of the authors (KO, SI, TL, JW or JT) have no non-financial interests that may be relevant to the submitted work. All authors declare they have no competing interests.

Authors’ contribution

KO designed the study, carried out the searches, screened and data-extracted studies, and prepared the manuscript. She is the guarantor. SI provided the search strings, data-extracted studies, helped to analyse the data and helped prepare the manuscript. TL data-extracted studies, helped to analyse the data and prepared the manuscript. JW screened studies, helped design the scope and helped prepare the manuscript. JT helped to analyse the data and helped prepare the manuscript. All authors read and approved the final manuscript.

Electronic supplementary material

Additional file 1: sample search strategy.(docx 11 kb), additional file 2: characteristics of included studies.(docx 203 kb), additional file 3: research checklist.(docx 18 kb), authors’ original submitted files for images.

Below are the links to the authors’ original submitted files for images.

Authors’ original file for figure 1

Authors’ original file for figure 2, authors’ original file for figure 3, authors’ original file for figure 4, rights and permissions.

This article is published under license to BioMed Central Ltd. This is an open access article distributed under the terms of the Creative Commons Attribution License ( http://creativecommons.org/licenses/by/2.0 ), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Reprints and permissions

About this article

Cite this article.

Oliver, K., Innvar, S., Lorenc, T. et al. A systematic review of barriers to and facilitators of the use of evidence by policymakers. BMC Health Serv Res 14 , 2 (2014). https://doi.org/10.1186/1472-6963-14-2

Download citation

Received : 02 July 2013

Accepted : 20 December 2013

Published : 03 January 2014

DOI : https://doi.org/10.1186/1472-6963-14-2

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Criminal Justice
• Research Evidence
• Knowledge Translation
• Policy Process
• Policy Area

BMC Health Services Research

ISSN: 1472-6963

• General enquiries: [email protected]

• Open access
• Published: 28 June 2022

Rapid systematic review to identify key barriers to access, linkage, and use of local authority administrative data for population health research, practice, and policy in the United Kingdom

• Sowmiya Moorthie 1 , 2 ,
• Shabina Hayat 1 ,
• Yi Zhang 3 ,
• Katherine Parkin 1 , 3 ,
• Veronica Philips 4 ,
• Amber Bale 5 ,
• Robbie Duschinsky 3 ,
• Tamsin Ford 6 &
• Anna Moore 6 , 7 , 8  

BMC Public Health volume  22 , Article number:  1263 ( 2022 ) Cite this article

2240 Accesses

2 Citations

Metrics details

Improving data access, sharing, and linkage across local authorities and other agencies can contribute to improvements in population health. Whilst progress is being made to achieve linkage and integration of health and social care data, issues still exist in creating such a system. As part of wider work to create the Cambridge Child Health Informatics and Linked Data (Cam-CHILD) database, we wanted to examine barriers to the access, linkage, and use of local authority data.

A systematic literature search was conducted of scientific databases and the grey literature. Any publications reporting original research related to barriers or enablers of data linkage of or with local authority data in the United Kingdom were included. Barriers relating to the following issues were extracted from each paper: funding, fragmentation, legal and ethical frameworks, cultural issues, geographical boundaries, technical capability, capacity, data quality, security, and patient and public trust.

Twenty eight articles were identified for inclusion in this review. Issues relating to technical capacity and data quality were cited most often. This was followed by those relating to legal and ethical frameworks. Issue relating to public and patient trust were cited the least, however, there is considerable overlap between this topic and issues relating to legal and ethical frameworks.

Conclusions

This rapid review is the first step to an in-depth exploration of the barriers to data access, linkage and use; a better understanding of which can aid in creating and implementing effective solutions. These barriers are not novel although they pose specific challenges in the context of local authority data.

Peer Review reports

Data analysis for improving healthcare is established practice in the United Kingdom (UK). However, the era of big data has led to a greater emphasis on harnessing data from a variety of sources for improved healthcare [ 1 , 2 ]. This means that increasingly efforts are being made in the UK to access, link and use healthcare data from different sources, including general practice, hospitals and community health services. The value of this approach is especially pertinent to addressing complex health issues such as mental health, which are impacted by a wide variety of factors and where service provision may span several agencies (e.g. health and social care) and may be outside of traditional healthcare settings (e.g. third sector, schools or the workplace). Thus, addressing and improving services for those experiencing complex health problems can be better informed by analysis of data from a range of local public services such as education and social care. In the UK, these data usually sit within local authorities (LA) which are local government organisations responsible for public services in particular areas.

Improving data access, linkage and integration across LAs and other agencies can contribute to improvements in population health. Analysis of health datasets that are routinely collected in the course of public service delivery are an important resource for population health and epidemiological research. They can enable research and analysis to better understand social and biological determinants of health, as well as mechanisms to intervene, either through service or policy development [ 3 , 4 ].

National policy prioritises appropriate access to, and use of administrative health and LA data [ 5 , 6 , 7 ], beyond population health management, to patients, service providers, academics, industry, and policy makers [ 8 ]. Such access would optimise health and care outcomes [ 9 ], management of integrated pathways [ 10 ], cost-effectiveness [ 11 ], and service user experience and satisfaction [ 12 , 13 ]. Furthermore, achieving this is particularly important as inequality has widened over the past 10 years in the UK, and accelerated during 2020 and the COVID-19 pandemic [ 14 , 15 ]. For health inequalities to be addressed, the Marmot report has recommended the creation of linked administrative databases of health, social care, education, and environmental data that are locally embedded, whole-population, patient-level, granular, and geographically-bound [ 15 ].

Finally, the Health and Social Care Bill currently being passed in England will create Integrated Care System Partnerships (ICSP). The Bill includes a requirement for all ICSPs to develop cross-system intelligence functions to support operational and strategic decision-making, underpinned by linked data and accessible analytical resources [ 16 ]. Thus, building longitudinal, whole-population, patient-level databases linking social, environment, and health information is a priority to better understand, monitor and enable interventions.

The Cambridge Child Health Informatics and Linked Data (Cam-CHILD) database aims to do this for children’s and young people’s health, by building an anonymized, linked database of health, education, social care and genetic data for the population of Cambridgeshire and Peterborough. This database will be utilised to develop informatics-driven approaches to early identification and intervention for mental health problems in children and young people within the region. Our preliminary analyses of the data requirements for model building indicate a prominent requirement for information relating to social and environmental domains. Many of the required variables are located in existing local authority datasets. However, there are significant barriers to access, linkage and use of this data, with few examples of their routine use to support public health decision making, research, or direct patient care [ 17 ].

Understanding the barriers faced in accessing and linking LA data is an important step towards developing solutions to its more efficient use. Thus, the primary aim of this rapid review was to examine reported barriers to the access, linkage and use of such data. A greater understanding of these barriers is important as we embark on the process of bringing together data from a variety of sources. It will also enable others involved in such initiatives, to develop effective and locally driven solutions for use of local authority data.

Search strategy

This review was conducted using systematic review methods and in accordance with the PRISMA statement where possible [ 18 ]. The protocol was registered on PROSPERO ( www.crd.york.ac.uk/PROSPERO , CRD42021245528) in April 2021.

A systematic literature search was conducted of the following databases; Medline (via Ovid), Embase (via Ovid), Cochrane Library, Global Health (via EBSCO host), PsycINFO (via EBSCO host), CINAHL (via EBSCO host), and Informit Health Collection in January 2021. Searches were date limited from January 2010 to January 2021, using the search terms listed in Table  1 and tailoring them for each database (detailed search terms in Additional file 1 ). In addition, the PROSPERO registry was searched for newly registered protocols. All results were limited to the UK regions and English language papers.

The initial database search revealed 551 records. In addition, 44 papers were identified through other sources (hand-searching references, expert communications and grey literature searches). The grey literature search was conducted using the advanced search function in Google and the search terms listed in Table 1 . The search was customised by restricting to English language pages, in the UK region and in PDF format. Date restrictions were set as for the initial database searches. Potentially relevant records were identified by examining the first five pages of the search results. Following de-duplication, the titles and abstracts of these records were screened by two authors (S.M. and A.M.) separately. Discussion was used to resolve discrepancies and a final list of 81 articles were identified as eligible for full text screening. Fig.  1 shows the search and selection outcomes for each stage of the review process.

PRISMA flow diagram

Given the large proportion of policy papers and grey literature documents, papers were not assessed using traditional quality assurance measures.

Inclusion and data extraction

Five of the authors (S.M., S.H., Y.Z., A.B. and K.P.) assessed full texts of papers for inclusion in the review. Of these papers, 10% ( n  = 5) were assessed by all reviewers to ensure a consistent approach to inclusion. Any publications reporting original research related to barriers or enablers of data linkage of or with local authority data in the United Kingdom were included. Research could be either qualitative or quantitative, and from any phase or study design, including grey literature. Publications focused on countries outside the UK, opinion pieces, letters, commentaries and editorials were excluded. Reviewers met regularly to discuss and resolve any uncertainties or disagreements by consensus.

A standardised form was developed for data extraction. Barriers relating to the following issues were extracted from each paper: funding, fragmentation, legal and ethical frameworks, cultural issues, geographical boundaries, technical, capacity, data quality, security, and patient and public trust. These issues were chosen based on our initial scoping work involving preliminary examination of the literature, which indicated these to be the main areas of discussion. Through a more thorough review, our intention was to elicit a more nuanced understanding of how barriers relating to each of these issues were contributing impacting access, linkage and use of LA data. Data on additional issues beyond these was also extracted, if available. For the most part, the issues cited in papers were the same as those identified in our preliminary scoping work. However, we noted that there was considerable inter-relationship and overlap between these issues. Thus, we subsequently collapsed identified barriers into themes to enable better depiction of the main findings. These themes are: technical capability and data quality; legal and ethical frameworks; funding and capacity; cultural factors; data fragmentation plus public and patient trust.

A narrative synthesis approach was taken to data analysis. The extracted data was tabulated under the headings indicated above. The analysis included the extent to which barriers to access, linkage and use of local authority data have been examined, the types of publication reporting on this (e.g. grey literature or peer-reviewed), and the main barriers to access, linkage and use of LA data.

Forty-nine studies were excluded following assessment of full texts. A summary of these articles and reasons for exclusion can be found in Additional file 3 . The lack of information on barriers to access, use or linkage of data was the most frequent reason for exclusion. A total of twenty-eight reports were included in the final review, and Table  2 provides a summary of the characteristics of these reports. Of these, 10 were grey literature and the remainder were academic publications. Most (16 publications) of these covered the entire UK, while 6 focused on Scotland, 1 on Wales, and 5 on England (including 2 on London specifically).

Below we provide a synthesis of our main findings for each grouped theme and Fig.  2 provides a summary of the data that was available relating to each (Additional file  2 has tabulated data). Issues relating to technical capability and data quality were cited most often. This was followed by those relating to legal and ethical frameworks. Issue relating to public and patient trust were cited the least, however, there is considerable overlap between this topic and issues relating to legal and ethical frameworks.

Frequency or reported barriers across citations

Technical capability and data quality issues

Twenty-one papers cited technical constraints in data linkage. These constraints were either due to legacy systems hindering data sharing across organisations, or the absence of secure methods of data transfer, and issues in creating standardised interoperable systems between organisations. The lack of funding and capacity, as discussed further below also influence the ability to create safe, secure and interoperable systems.

Many reports acknowledged the variable quality of data collected by different organisations, [ 5 , 20 , 21 , 22 , 23 , 24 , 26 , 28 , 31 , 36 ] with the consequences that effective linkage is much harder to achieve [ 38 , 39 ], especially as it is a challenge to understand how data are coded and there is potential for missing or unavailable data [ 20 , 37 ]. Much of local authority data, for example social care data, contain a high proportion of data recorded in an unstructured format [ 38 ]. This serves as an additional challenge to its use, with reports that up to 90% of unstructured data is never analysed [ 21 ]. Standardisation does not address the problem of how to access and use unstructured data.

A few papers explicitly discussed issues relating to bias and inequalities as acting as a barrier to linkage and use of local authority data. These included overrepresentation [ 34 ], as well as underrepresentation of particular groups, for example, women, children, the very elderly, ethnic minorities and those with multiple co-morbidities [ 33 ]. In addition, reports also discussed the potential for explicit consent processes to lead to selection bias [ 3 , 19 ], given differences in which service users are likely to consent to broader use of data [ 19 ].

Legal and ethical frameworks

Both legal frameworks such as the General Data Protection Regulation (GDPR), and ethical principles govern and impact on data access, linkage and use. Existing legal frameworks are designed to address ethical concerns on data processing and require activities involving data to be ethically reviewed. Thus, it can often be difficult to disentangle ethical and legal frameworks with relation to data protection, leading to discussions on this topic being interconnected. As such, we grouped legal and ethical frameworks together for the purposes of data analysis.

Nineteen records identified existing ethical and legal frameworks as a barrier, particularly the complexity of the regulatory landscape pertaining to data protection. Notable variation in the processes for information governance and ethical approvals used to manage compliance with regulatory frameworks were reported between regions and organisations, with inconsistencies in both interpretation and operationalisation. Contributing to this, were the different legal provisions applying to various categories of public sector data, often including sensitive personal data, which may be identifiable, pseudonymised or anonymous [ 3 , 21 , 22 ]. Each must be considered differently by the law, and thus by information governance and ethics committees. Furthermore, the purpose of initial data collection often differ by team and organisation, limiting the ways in which it can subsequently be used.

Accessing or sharing of data, even between public agencies is a complex process that requires a clear understanding of legal frameworks that govern data access and use. Extending this to sharing between agencies for alternative uses, such as research, requires significant expertise which is often not available within local authorities [ 5 , 30 , 38 ]. Lack of familiarity with frameworks that must be applied to enable inter-agency sharing and use contributes to a risk-averse approach [ 22 , 23 ] and the lack of capacity and resources available hinder problem solving [ 26 ]. Understandable efforts to ensure privacy, confidentiality and consent often leads to hesitancy or concerns by organisation in sharing data [ 3 , 5 , 17 , 19 , 22 ], and where processes were in place, the approval processes, together with the capacity demands within the systems to process these make data access and linkage too time-consuming and resource intensive, and many projects fail [ 28 , 29 , 30 ] or are prevented from even starting [ 3 , 21 ].

Funding and capacity

Seventeen papers discussed issues related to funding either explicitly or indicated that funding posed a barrier. Reports discussed funding for data linkage initiatives, as well as access to research or strategic funding. In particular, within local authorities the need for funding to build capacity was discussed, for example to upskill staff, and build linked data systems and IT infrastructures, as well as to sustain existing systems [ 21 , 30 , 33 , 38 ].

Twelve reports identified capacity constraints stemming from a lack of personnel in government departments with expertise in addressing data access and creating infrastructure for data management. This included the lack of personnel with the expertise to link data within local authorities, and with expertise across sectors, for example health and social care [ 20 , 41 ]. Linkage across sectors can be particularly challenging and time-consuming. It requires time from those with domain-specific knowledge (e.g. social workers), as well as dedicated informatics expertise. Finding sufficient time for frontline staff to contribute to these projects can be challenging, particularly in an already stretched and busy working environment [ 36 , 41 ]. In addition, the report by the Local Government Association in 2019 [ 38 ] noted that building capacity was hindered by funding cuts to local authorities.

From a research perspective, attempts to create linked datasets were hampered by grant deadlines and high costs associated with accessing data as a result of the many different data access agreements and procedures that need to be navigated [ 28 , 35 , 42 ].

Cultural factors

Sixteen papers discussed cultural factors as a barrier to creating and utilising linked datasets. These include both individual and organisational cultural factors that impact on data access for subsequent use in linkage initiatives. Data are often owned by different organisations, which have different cultures in relation to willingness to share, and attitudes to data linkage and sharing which influence the ease of data access and linkage [ 21 , 23 ]. Specifically, willingness to share data, risk aversion and concerns about data breaches were cited as problematic issues [ 3 , 21 , 22 , 28 , 35 , 38 ]. Variation in interpretation of ethical and legal frameworks, as well as the degree of concern about inadvertently going against them impacted on negotiating access to data and in its subsequent linkage and use. The lack of trust between the different parties involved in initiatives to build linked datasets also contributes to issues [ 3 , 21 , 22 , 28 , 29 , 35 , 38 ].

Relatedly, the lack of a clear vision for use of data, not treating it as an asset and lack of leadership within organisations [ 5 , 21 , 26 , 43 , 44 ] also serve to contribute to cultural barriers. This can lead to uncertainty as to what is permissible or desirable by data holders for safe and appropriate use of data.

Data fragmentation

Fourteen papers discussed data fragmentation and data silos as a barrier to linkage and use of data. The variety of data holders and fragmentation across government departments can contribute to delays in access and use of data [ 26 , 28 , 29 , 38 ]. As noted above, cultural factors may influence individual departments’ or organisations’ interpretation of what is permissible or where responsibilities need to be fulfilled to access data. These in turn influence the practicalities of data access such as the requirement for different permissions between and within organisations that need to be granted [ 3 ]. In addition, organisational silos can lead to a lack of understanding of available data across local authorities [ 23 , 43 ].

Related to data fragmentation across data holders, are data silos created by the use of different IT infrastructures within local authorities [ 17 , 21 , 23 , 26 , 33 , 38 ]. The predominance of bespoke and legacy IT systems has led to data being recorded in specific ways, often unique to teams, in a wide range of formats with different coding systems that are not compatible with each other [ 5 , 21 , 23 ]. Furthermore, changes in coding practices within teams or to care processes over time require frequent local system reconfigurations, which are time-consuming and costly [ 24 , 38 ]. This variety in storage formats leads to difficulties with sharing and linkage even between teams within a council, let alone linkage with other external agencies [ 20 ].

Data can also be fragmented across geographical boundaries such as local authorities, counties or countries in the UK. Differences in institutional digital maturity across these boundaries is problematic [ 3 , 21 , 26 , 32 , 38 ]. The report published in 2012 by the Administrative Data Research UK (ADRUK) noted that addressing this in Scotland and Wales had led to significant gains compared to the rest of the UK enabling a county-wide approach to linkage [ 3 ]. Linking between health and social care is significantly more challenging given the minimal use of the National Health Service NHS number within social care across much of England [ 23 , 31 , 38 ]. While there has been significant progress in some areas in the use of the NHS number [ 38 ], this has predominantly involved adult social care data, where sharing has been of basic core information such as demographics, allocated case worker and information about services accessed by individuals. To address this, there are examples of linkage methods that used alternative methods of matching records, for example based on hashed de-identified personal identifiers such as birth date and postcode [ 25 , 28 , 34 , 39 ]. These methods were shown to be effective for a large proportion of records, but require access either to specialist software, or expensive safe havens that provide linkage services. These approaches require orchestrators to navigate the balance between privacy, confidentiality and scalability [ 5 , 17 ], and all are resource intensive in terms of cost and time.

Furthermore, variable digital maturity can also lead to implementation of different ethical and legal frameworks as a means to accommodate this, which in themselves can act as a barrier to data access by creating different processes for applying for use of data. Finally, geographical differences in IT infrastructure risks introducing significant regional inequality, for example in remote areas with poor connectivity [ 32 ].

Lack of patient and public trust

Ten reports identified lack of patient and public trust as a barrier to access, linkage and use of local authority data. These reports discussed several factors that contribute to erosion of public confidence and trust in the use of data for research. These include heightened public understanding of rights to privacy, concerns around misuse and exploitation of data, access to private information by commercial organisations, and limited control over uses of their data. This combined with negative publicity about some government data programmes such as care.data underpinned concerns around data sharing and linkage [ 27 , 33 ]. Furthermore, one report [ 21 ] cited that there can be a lack of public trust in local authorities to efficiently manage and achieve full potential from their data. This is because individuals are required to provide the same information multiple times, and there is a lack of clarity on the purpose of each point of data collection.

Overall, negative publicity around administrative data, local authorities’ inefficiency in data exploitation, lack of transparency, and not obtaining informed consent may all reduce public trust in those handling the data, hence reducing public support for data access, use and linkage and use, acting as a barrier. This in turn can also indirectly influence the willingness and extent to which local authorities engage in data linkage and sharing initiatives.

Over the past 10 years, there has been a policy push in the UK towards digitisation and the more effective use of data across organisations to improve healthcare. Data held outside of the NHS, within other agencies such as local authorities, can provide important information that can be used to improve health. However, it is apparent from the papers we reviewed that whilst there has been much progress made to achieve the ambition of joint-working, significant barriers still exist in accessing, linking and using data across these sectors. Addressing these barriers is now imperative to aid the move towards creating integrated care systems.

The objective of this review was to gain a better understanding of the key barriers to accessing, linking and using local authority data for population health research, practice and policy using a systematic approach. Examination of the reports included in this review led to the identification of barriers which were grouped into key themes. Consideration of these barriers and the themes together, suggests there are a core set of interlinked, cross-cutting factors which impact on the ability to access, link and use local authority data for population health research, practice and policy. These are trust between different stakeholders; leadership to make the best use of data and capacity to deliver on data-led initiatives. Although these are not novel, nor specific to local authorities, their impact on the identified barriers is particular to the context in which LAs function.

While local authorities may be considered as a single organisation, in reality many different services and/or agencies contribute to this landscape. In addition, whilst data from these agencies can contribute either directly or indirectly in diverse ways to healthcare, they may not be collected for this purpose. Thus, cross-organisational working is a key element in efforts to harness LA data for health. Cross-organisational working requires trust between data suppliers, holders and users. Sexton et al. [ 37 ] discuss how securing public trust is dependent on achieving broader trust between those involved in data initiatives. This can be a challenge given the diversity of stakeholders that may be involved in data initiatives. Legislative and ethical processes are in place to prevent misuse, and are one way of gaining trust and transparency on data uses. However, these are complex and often not clearly communicated, understood or operationalised by many [ 3 , 20 , 22 , 29 ]. Underpinning this is ensuring transparency and clarity about how the data will be used. Several of the reports we identified highlighted that where data initiatives involve parties from different organisations, time is needed to build trust and to navigate this landscape in a mutually beneficial fashion.

A repeated theme that related to access, linkage and use of the data was the lack of dedicated leadership for data use and incentives for such leadership within local authorities [ 5 , 21 , 34 , 44 ]. In the reviewed literature, there were no examples of a designated senior officer for data issues within an LA. This role was identified as important, and should provide leadership responsibility extending beyond technical requirements into advocacy for the use and reuse of data [ 21 ], the budgeting required [ 5 ], the adoption of rules and standards [ 5 ], establishing the changes in business processes required [ 5 , 34 , 40 ], and navigating what should be linked and shared with whom [ 5 ]. Specific gaps were identified in relation to access, linkage, and use, as described above. However, across all domains there was a consistent lack of prioritisation within LAs to embed data skills within workforce strategies. This led to reports of lack of confidence by staff and elected members in the quality of the information being made available for decision-making, reporting and scrutiny [ 21 ]. Changes in the political landscape also impact much more on councils and local authorities. This is likely to influence creation of a stable and dedicated leadership for data use.

Many of the barriers we identified were also linked to capacity issues in the form of funding, personnel and skills to deliver on data initiatives. Whilst there is a policy push for more integrated working between health and social care, and there is recognition of the rich contributions that data outside of the health system can make to improving population health, investment in achieving this is not optimal. LA budget cuts and lack of prioritisation of informatics projects means they can struggle to retain the skills required, leading to difficulties in maintaining integrity and accuracy of databases [ 21 ]. Budget cuts have often forced IT teams to work across councils, or for IT support to be outsourced, which has led to reduced knowledge of the data systems involved in commissioning and delivery of care [ 38 ]. Furthermore, non-integrated technology and legacy systems mean that nearly all local authority IT systems are unique. This raises technology costs, creates duplication and redundancy of capacity, and leads to vendor lock-in, preventing councils from taking advantage of economies of scale. Despite shrinking budgets, LAs are estimated to spend ~ 3–6% on IT, almost twice as much as utility and transport sectors [ 5 , 23 , 28 ]. In spite of this proportionally high spend, a lack of long-term planning to identify future data and intelligence needs means councils are failing to invest in the infrastructure and capability required to enable use of their data for translation into cost-saving initiatives.

The lack of long-term planning extends to consideration of potential secondary uses of data. This can increase costs as additional work needs to be undertaken to make data suitable for sharing and linking, and other expertise such as legal advice and additional technology are required [ 5 ]. Lack of awareness of the benefits of linkage means funding to enable interoperability between systems is not a priority. When it is prioritised, a lack of stability of budgets means that novel technologies are not always maintained due to funding cuts. Indeed, some providers decline to operate in the sector due to complexity of the systems, pressures to keep costs down and scarcity of long-term contracts [ 38 ]. Collaborative working between councils and other partners can identify opportunities for savings, with examples of the costs associated with data sharing being shared between councils [ 28 ]. Government-funded infrastructure projects encouraging collaboration between councils and supporting initial costs of enabling data sharing have been successful. However, it is important that councils concurrently ensure the value is recognised locally and ongoing budgets are allocated to ensure project sustainability [ 32 ]. Expanding this sharing also depends on the adoption of common information standards. However, currently many programmes are led by health services, and are often based on medical episodes of care for individuals. This does not always translate to the LA, where care is based more around families, carers and individuals, and is rarely medically-driven [ 38 ]. Greater initiation by LAs could help shape data use in a way which benefits them more.

While we were able to find a range of articles discussing barriers to access, linkage and use of data, there are limitations to our work. It is likely that there are reports that we missed, especially in the form of grey literature addressing this issue. Due to the nature of the subject, the search terms used were broad, leading to many non-specific hits. In addition, we employed a rapid review methodology; whilst this followed a systematic review approach, it is less rigorous but was a more pragmatic given time constraints.

Given the heterogeneity of the literature reviewed, the depth of discussion on the barriers to access, linkage and use of data varied, with few academic publications discussing issues relating to access or linkage in detail. This might reflect the focus of such publications in describing research, rather than reporting in detail on barriers faced in conducting research involving linked datasets. The grey literature, especially those reports that were aimed at developing recommendations to move towards the use of linked datasets, provided more substance. However, the focus was not always on population health research.

Finally, the Health and Care Bill is currently being passed to establish Integrated Care Partnerships (ICPs) across England, which will take on the commissioning functions of CCGs and be accountable for NHS spend and performance within the system [ 16 ]. These ICPs establish the NHS and local government as equal partners, required to jointly facilitate action to address the wider determinants of health, as well as broader economic development, in order to improve health outcomes. Underpinning these will be digital and data transformation plans that must enable a cross-system approach and provide clear accountability for digital and data use to support population health management, resource planning and performance management. The Bill recognises that improvement requires access to whole-system data. Whilst progress is being made to achieve integration of health and social care data, issues still exist in creating such a system and there remain few examples of successful integration of health and care data. This is also reflected in our experience of developing the Cam-CHILD database, and in a recent paper which describes the issues in relation to creation of LAUNCHES QI which aimed to link audit and national datasets in Congenital Heart Services [ 45 ]. Thus, this review provides a starting point for moving towards a better and more nuanced understanding of a broad set of issues impacting on better use of local authority data.

This rapid review is the first step to an in-depth exploration of the barriers to data access, linkage, and use, a better understanding of which can aid in creating and implementing effective solutions. These barriers are not novel and have been identified by others in relation to health data sharing. However, how these manifest in relation to local authority data and the context in which they are experienced are novel, as are the solutions put forward to address them [ 3 , 20 , 38 , 43 ]. As we move towards creation of ICPs, it becomes imperative to share learning on approaches to effectively addressing these barriers, so that we can unlock the potential of LA data for health. The potential for making the most of data across systems has been alluded to by many [ 2 , 8 ], but achieved by few. As local authorities undertake more research activities [ 46 ], joint efforts to address these barriers and achieve effective data sharing in a safe and acceptable way is important to enhance research that feeds into service provision and improves population health.

Availability of data and materials

The authors confirm that the data supporting the findings of this review are available within the article or in supplementary materials.

Wellcome Trust. Public health research data forum: Enabling data linkage to maximise the value of public health research data. 2015. https://wellcomeacuk/sites/default/files/enabling-datalinkage-to-maximise-value-of-public-health-research-data-phrdfmar15pdf .

Google Scholar  

Jones KH, Laurie G, Stevens L, Dobbs C, Ford DV, Lea N. The other side of the coin: harm due to the non-use of health-related data. Int J Med Inform. 2017;97:43–51.

Article   Google Scholar  

Administrative Data Taskforce. Improving access for research and policy. Report from the administrative data taskforce 2012.

Roos LL, Walld R, Burchill C, Roos NP, Nickel N. Linkable administrative files: family information and existing data. Longitud Life Course. 2017;8(3):262–80.

Office for National Statistics & Government Analysis Function. Joined up data in government_ the future of data linking methods. 2020.

Academy of Medical Sciences. Improving the health of the public by 2040. 2016.

National Health Service. The NHS long term plan 2019.

Verhulst S, Noveck B, Caplan R, Brown K, Paz C. The open data Era in health and social care. NYLS legal studies research paper No 2563788. 2014; Available at SSRN: https://ssrn.com/abstract=2563788 or https://doi.org/10.2139/ssrn.2563788 .

Woodman J, Brandon M, Bailey S, Belderson P, Gilbert R. Healthcare utilisation by children fatally or seriously injured following child maltreatment: analysis of national case series 2005–7 for England. Arch Dis Child. 2010;95:A90–A.

Lewer D, Bourne T, George A, Abi-Aad G, Taylor C, George J. Data resource: the Kent integrated dataset (KID). Int J Popul Data Sci. 2018;3(1):427.

CAS   PubMed   PubMed Central   Google Scholar  

Asaria M, Grasic K, Walker S. Using linked electronic health records to estimate healthcare costs: key challenges and opportunities. Pharmacoeconomics. 2016;34(2):155–60.

Henderson DAG, Atherton I, McCowan C, Mercer SW, Bailey N. Linkage of national health and social care data: a cross-sectional study of multimorbidity and social care use in people aged over 65 years in Scotland. Age Ageing. 2021;50(1):176–82.

Ainsworth J, Buchan I. Combining health data uses to ignite health system learning. Methods Inf Med. 2015;54(6):479–87.

Article   CAS   Google Scholar  

Marmot M. Fair society, healthy lives: the Marmot review. London: Marmot Review; 2010.

Marmot M, Allen J, Boyce T, Goldbatt P, Morrison J. Health equity in England : the Marmot review 10 years on. London: Institute of Health Equity; 2020.

National Health Service. Integrated care systems: design framework version 1. 2021.

Mansfield KL, Gallacher JE, Mourby M, Fazel M. Five models for child and adolescent data linkage in the UK: a review of existing and proposed methods. Evid Based Ment Health. 2020;23(1):39–44.

Knobloch K, Yoon U, Vogt PM. Preferred reporting items for systematic reviews and meta-analyses (PRISMA) statement and publication bias. J Craniomaxillofac Surg. 2011;39(2):91–2.

Aitken M, de St. Jorre J, Pagliari C, Jepson R, Cunningham-Burley S. Public responses to the sharing and linkage of health data for research purposes: A systematic review and thematic synthesis of qualitative studies. BMC Medical Ethics. 2016;17:73. https://doi.org/10.1186/s12910-016-0153-x .

Atherton IM, Lynch E, Williams AJ, Witham MD. Barriers and Solutions to Linking and Using Health and Social Care Data in Scotland. Br J Soc Work. 2015;45(5):1614–22.

Auditor General for Wales. The maturity of local government in use of data Archwilydd Cyffredinol Cymru. 2018.

Centre for Data Ethics and Innovation. Addressing trust in public sector data use. 2020. Available at: https://www.gov.uk/government/publications/cdei-publishes-its-first-report-on-public-sector-data-sharing/addressing-trust-in-public-sector-data-use .

Copeland E. Small Pieces Loosely Joined: How smarter use of technology and data can deliver real reform of local government. 2015.

Davies JM, Gao W, Sleeman KE, Lindsey K, Murtagh FE, Teno JM, et al. Using routine data to improve palliative and end of life care. BMJ Support Palliat Care. 2016;6(3):257–62.

Downs JM, Ford T, Stewart R, Epstein S, Shetty H, Little R, et al. An approach to linking education, social care and electronic health records for children and young people in South London: A linkage study of child and adolescent mental health service data. BMJ Open. 2019;9(1).

Comptroller and Auditor General. Challenges in using data in governments. National Audit Office. 2019. Available at: https://www.nao.org.uk/report/challenges-in-using-data-across-government/ .

Heitmueller A, Henderson S, Warburton W, Elmagarmid A, Pentland A, Darzi A. Developing public policy to advance the use of big data in health care. Health Affairs. 2014;33(9):1523–30.

Higgins C, Matthews K. Electronic linkage and interrogation of administrative health, social care, and criminal justice datasets: feasibility concerning process and content. Informatics for Health & Social Care. 2020;45(4):444–60.

Office for Statistics Regulation, Joining Up Data for Better Statistics. 2018. Available at: https://osr.statisticsauthority.gov.uk/publication/joining-up-data/ .

Iveson MH, Deary IJ. Navigating the landscape of non-health administrative data in scotland: A researcher’s narrative. Wellcome Open Research. 2019;4:97

Kemm JR, Robinson J, Verne J. Social care data in England: What they tell us and what they do not tell us. Public Health. 2010;124(5):265–8.

King G, O'Donnell C, Boddy D, Smith F, Heaney D, Mair FS. Boundaries and e-health implementation in health and social care. Bmc Medical Informatics and Decision Making. 2012;12.

Kneale D, Khatwa M, Thomas J. Identifying and appraising promising sources of UK clinical, health and social care data for use by NICE. 2016. Report No.: 9781907345937.

Malomo F, Sena V. Data Intelligence for Local Government? Assessing the Benefits and Barriers to Use of Big Data in the Public Sector. Policy Internet. 2017;9(1):7–27.

Mourby MJ, Doidge J, Jones KH, Aidinlis S, Smith H, Bell J, et al. Health Data Linkage for UK Public Interest Research: Key Obstacles and Solutions. Int J Popul Data Science. 2019;4(1):1093.

Oyeyemi A, Scott P. Interoperability in health and social care: Organisational issues are the biggest challenge. J Innovation Health Informatics. 2018;25(3):196–8.

Sexton A, Shepherd E, Duke-Williams O, Eveleigh A. A balance of trust in the use of government administrative data. Archival Science. 2017;17(4):305–30.

Local government association. Local government social care data and interoperability standards: executive summary for social care professionals 2019. Available at: https://www.local.gov.uk/publications/local-government-social-care-data-standards-and-interoperability .

Stewart CH, Dundas R, Leyland AH. The Scottish school leavers cohort: linkage of education data to routinely collected records for mortality, hospital discharge and offspring birth characteristics. BMJ Open. 2017;7(7):e015027.

Symons T. Datavores of Local Government: Using data to make services more personalised, effective and efficient. Nesta. 2016;(7):1–32.

Wistow G, Gaskins M, Holder H, Smith J. Why Implementing Integrated Care is so much harder than designing it: experience in North West London. England. Int J Integr Care (IJIC). 2016;16(6):1–3.

Witham MD, Frost H, McMurdo M, Donnan PT, McGilchrist M. Construction of a linked health and social care database resource—Lessons on process, content and culture. Informatics for Health & Social Care. 2015;40(3):229–39.

Muirhead A, Ward DG, Howard B. The Digital House of Care: information solutions for integrated care. J Integr Care. 2016;24(5/6):237–48.

National Audit Office. Challenges in using data across government. 2019.

Taylor JA, Crowe S, Espuny Pujol F, Franklin RC, Feltbower RG, Norman LJ, et al. The road to hell is paved with good intentions: the experience of applying for national data for linkage and suggestions for improvement. Bmj Open. 2021;11(8):e047575.

Fynn JF, Jones J, Jones A. A systems approach to the exploration of research activity and relationships within a local authority. Health Res Policy Syst. 2021;19(1):137.

Download references

Acknowledgements

MRC Adolescent Engagement Awards MR/T046430/1.

Medical Humanities Collaborative Award from the Wellcome Trust [Grant Number: 218025/A/19/Z].

SM and KP are supported by the National Institute for Health Research (NIHR) Applied Research Collaboration East of England (ARC EoE). KP is also funded by the National Institute for Health Research (NIHR School for Public Health Research (SPHR) (Grant: PD-SPH-2015). AM is funded through an NIHR Clinical Lectureship funded by Anna Freud National Centre for Children and Families. The views expressed are those of the author(s) and not necessarily those of the NHS, the NIHR or the Department of Health and Social Care. RD holds a Collaborative Award from the Wellcome Trust.

Author information

Authors and affiliations.

Cambridge Public Health, Interdisciplinary Research Centre, Forvie Site, Cambridge Biomedical Campus, Cambridge, UK

Sowmiya Moorthie, Shabina Hayat & Katherine Parkin

PHG Foundation, 2 Worts Causeway, University of Cambridge, Cambridge, UK

Sowmiya Moorthie

Department of Public Health and Primary Care, University of Cambridge, Cambridge, UK

Yi Zhang, Katherine Parkin & Robbie Duschinsky

University of Cambridge Medical Library, Cambridge, UK

Veronica Philips

Department of Psychology, University of Northumbria, Newcastle upon Tyne, UK

Department of Psychiatry, University of Cambridge, Herschel Smith Building, Robinson Way, Cambridge, UK

Tamsin Ford & Anna Moore

Anna Freud National Centre for Children and Families, London, UK

Cambridgeshire and Peterborough NHS Foundation Trust, Cambridge, Peterborough, UK

You can also search for this author in PubMed   Google Scholar

Contributions

SM, SH and AM conceived and developed the review protocol with assistance from VP. Searches were performed by VP. SM and AM carried out initial screening. SM, SH, KP, YZ and AB screened citations and full text articles for inclusion and extracted data. SM, SH, KP, YZ and AM analysed and interpreted the data. All authors contributed to drafting and revising the manuscript. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Sowmiya Moorthie .

Ethics declarations

Ethics approval and consent to participate.

Not applicable.

Consent for publication

Competing interests.

The authors declare no conflict of interest.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1..

Search terms.

Additional file 2.

Summary of included studies against each theme.

Additional file 3.

Rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit  http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Moorthie, S., Hayat, S., Zhang, Y. et al. Rapid systematic review to identify key barriers to access, linkage, and use of local authority administrative data for population health research, practice, and policy in the United Kingdom. BMC Public Health 22 , 1263 (2022). https://doi.org/10.1186/s12889-022-13187-9

Download citation

Received : 18 October 2021

Accepted : 31 March 2022

Published : 28 June 2022

DOI : https://doi.org/10.1186/s12889-022-13187-9

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Linked data
• Integrated care
• Health informatics
• Public health

BMC Public Health

ISSN: 1471-2458

• General enquiries: [email protected]

Advertisement

A Systematic Review of Interventions to Improve Healthcare Experiences and Access in Autism

• Review Paper
• Open access
• Published: 26 August 2021
• Volume 10 , pages 185–202, ( 2023 )

Cite this article

You have full access to this open access article

• Chloe Walsh 1 , 2 ,
• Paul O’Connor 1 , 2 ,
• Ellen Walsh 2 &
• Sinéad Lydon   ORCID: orcid.org/0000-0001-5700-4350 2 , 3  

13k Accesses

13 Citations

2 Altmetric

Explore all metrics

Autistic individuals report barriers to accessing and receiving healthcare, and experience increased morbidity and mortality. This systematic review synthesizes 31 research studies evaluating interventions implemented to improve the healthcare experiences and/or access of autistic persons. Interventions were most commonly patient-focused (58.1%), focused on supporting the autistic individual to engage with, tolerate, or anticipate medical procedures, care, or settings. Fewer studies were provider-focused (48.4%) or organization-focused (6.5%). Interventions were typically evaluated using measures of reactions (45.2%) or behavior (48.4%), and outcomes were predominantly positive (80.6%). Further research is imperative and should look to how providers and organizations must change. Future research must be inclusive of the autistic community, must measure what matters, and must offer complete detail on interventions implemented.

Similar content being viewed by others

Working Beyond Capacity: a Qualitative Review of Research on Healthcare Providers’ Experiences with Autistic Individuals

Tackling healthcare access barriers for individuals with autism from diagnosis to adulthood

Healthcare Providers’ Experiences with Autism: A Scoping Review

Avoid common mistakes on your manuscript.

Introduction

Autism is a neurodevelopmental condition characterized by differences in social and communication skills, along with the presence of repetitive and/or restricted behaviors and interests (American Psychiatric Association, 2013 ). Autistic Footnote 1 individuals are more likely to experience illness or disease than others. Poor health is estimated to be more than five times more likely among autistic adults than among neurotypical adults (Rydzewska et al., 2019 ). Autistic individuals have a higher likelihood of experiencing a variety of medical conditions such as autoimmune disease, cardiovascular diseases, neurological conditions, and gastrointestinal disorders (Croen et al., 2015 ). A number of studies, analyzing data from different countries, have also evidenced substantially increased mortality, in some cases up to twice as high, among autistic persons (Hirvikoski et al., 2016 ; Hwang et al., 2019 ; Mouridsen et al., 2008 ; Shavelle et al., 2001 ). This means autistic individuals are more likely to die than their peers over a period of time. Further, compared to neurotypical controls, autistic individuals have a greater number of unmet healthcare needs (Nicolaidis et al., 2013 ), receive less preventative care (Nicolaidis et al., 2013 ), report more frequent use of emergency care (Nicolaidis et al., 2013 ; Tregnago & Cheak-Zamora, 2012 ), and have a lower health-related quality of life (Khanna et al., 2014 ; Kuhlthau et al., 2010 ). However, poor health is not an inevitable consequence of autism but instead reflects important health inequities experienced by autistic individuals that exist internationally (Bishop-Fitzpatrick & Kind, 2017 ; Tregnago & Cheak-Zamora, 2012 ).

Health inequities are defined as “unjust and avoidable differences in healthcare access, quality, and outcomes” (Scott & Rawal, 2018 , p.1; Whitehead, 1991 ). Potential determinants of, or contributors to, the health inequities observed among persons with intellectual disabilities are well-delineated and are suggested to include increased experience of undesirable social determinants of health (e.g., poverty and unemployment), communication problems and health illiteracy, poor health-related behaviors, and poor access to, and quality of, healthcare (Emerson, 2011 ; Emerson & Baines, 2011 ). It is likely that such factors also play a role in perpetuating health inequities for autistic individuals (Brugha et al., 2016 ; Mason et al., 2019 ; Muskat et al., 2015 ; Walsh et al., 2020a ) making this a complex issue. For instance, research shows that autistic people are more likely to be less educated, unmarried, and more socially deprived than the general population (Brugha et al., 2016 ). Similarly, it is well-recognized that autistic individuals experience issues in accessing and receiving high quality healthcare (Mason et al., 2019 ; Muskat et al., 2015 ; Walsh et al., 2020a ).

Barriers to healthcare access can occur at the level of the patient, the healthcare provider, and the healthcare system (Walsh et al., 2020a ). Key barriers to healthcare access occurring at the level of the patient can include sensory sensitivity which makes it difficult for the individual to experience novel or stimulating settings or to tolerate physical exams or investigations (Mason et al., 2019 ; Walsh et al., 2020a ); communication difficulties which complicate identification of pain or symptoms and engagement with healthcare providers regarding investigations or treatments (Mason et al., 2019 ; Walsh et al., 2020a ), and; challenging behaviors (e.g., self-injurious behavior; aggression; stereotypy) which can cause the discontinuation of medical procedures, result in the use of physical and/or chemical restraint (e.g., Kamat et al., 2018 ), or make caregivers less likely to attend for medical appointments (Walsh et al., 2020a ). Barriers to healthcare access occurring at the level of the healthcare provider include a lack of provider knowledge or skill relating to autism which impedes their ability to adapt care for autistic patients or to understand their experiences or needs (Mason et al., 2019 ; Walsh et al., 2020a ); inflexibility of healthcare providers or a lack of willingness to make accommodations or change their behavior to facilitate the autistic patient (Mason et al., 2019 ; Walsh et al., 2020a ), and; healthcare providers ignoring, or not taking seriously, the concerns or expertise of the autistic patient or their caregiver (Mason et al., 2019 ; Walsh et al., 2020a ). Barriers also occur at the level of the organization or health system and these include a lack of continuity of care or collaboration between the different healthcare providers or services involved in the management of the autistic patient’s care (Walsh et al., 2020a ); time or resource constraints which impede the delivery of high quality care to the autistic patient (Walsh et al., 2020a ), and; financial and/or insurance issues which can preclude autistic individuals from accessing required care or making use of preventative care services (Walsh et al., 2020a ). Systematic reviews (Mason et al., 2019 ; Walsh et al., 2020a ) of barriers to healthcare access showcase the range of barriers that can impede care and emphasize the complexity of the challenge of improving healthcare access and experiences for autistic persons.

It has been mandated in the UK that public services adjust their practices to accommodate autistic people (Department of Health, 2010 ). Further, the development of autism-friendly healthcare services is a recognized research priority of the autistic community (Warner et al., 2019 ). There is an increasing focus internationally on making environments and processes “autism-friendly” which involves identifying the challenges or difficulties that autistic individuals may experience when engaging with them and making adaptations to facilitate the participation of autistic individuals (e.g., addressing sensory sensitivity by reducing noise levels or removing fluorescent light to make the environment less stimulating; Nguyẽ̂n, 2009 ). However, although the barriers to healthcare access have been well-considered and shown to exist across many countries (Mason et al., 2019 ; Muskat et al., 2015 ; Walsh et al., 2020a ), there is a lack of guidance on how to improve the care of autistic patients. To date, suggestions for improving healthcare have been described (Barber, 2017 ; Bellando et al., 2016 ; Nicolaidis et al., 2014 ; Souders et al., 2002 ; Venkat et al., 2012 ; Walsh et al., 2020a ) but there has been no attempt to synthesize interventional research in this area and to examine the types of interventions which have been employed and their outcomes. Understanding what type of interventions have been trialed to date, and their effects, is important for informing both research and practice in this area. Accordingly, the purpose of this systematic review was to synthesize the research evaluating interventions to improve the healthcare experiences and/or access of autistic persons.

Study Design

This systematic review was conducted, and is reported, in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-analysis (PRISMA) guidelines (Moher et al., 2009 ). The protocol was registered in the International Prospective Register of Systematic Reviews (PROSPERO; CRD42020163200).

Search Strategy

A systematic search of five electronic databases was completed in January 2020. The databases searched were Medline, CINAHL, Web of Science, Psychology and Behavioral Sciences Collection, and PsycInfo. The search protocol, developed with the assistance of a research librarian, included Medical Subject Headings (MeSH) search terms along with other keywords that related to autism, healthcare, and interventions. The search strategy for Medline is presented in Supplemental Digital Content 1 and was adapted as necessary for the other databases. Searches were limited to the English language but no restriction was placed upon year of publication.

It is well-recognized that the exclusion of grey literature (i.e., materials not published in peer-reviewed journals) can lead to the overestimation of intervention effects within systematic reviews (Hopewell et al., 2007 ). Accordingly, grey literature searches were undertaken in November 2019 to identify unpublished materials suitable for inclusion in this review. Searches were conducted across: (1) Google (first 100 returns; location set to UK); (2) Google Scholar (first 100 returns; location set to UK); (3) Ethos (e-theses online service; all returns examined), and (4) OpenGrey (System for Information on Grey Literature in Europe; all returns examined). Across each of these databases, the keyword “autism” was entered along with one of the following terms: health; healthcare; hospital; general practice; doctor; physician, and nurse.

In order to identify additional relevant studies, the reference lists of a number of related reviews (Koski et al., 2016 ; Kupzyk & Allen, 2019 ; Lydon et al., 2015 ; McCormack et al., 2019 ) were screened along with the reference lists of all studies determined to be suitable for inclusion following the electronic searches.

Study Selection

Inclusion and exclusion criteria.

In order to be included in this review, studies were required to (a) be written in the English language; (b) describe the implementation and evaluation of an intervention to improve healthcare access or care experiences for autistic persons in relation to physical healthcare specifically (defined as any medical care related to the physical wellbeing of the participants; Walsh et al., 2020a ), and (c) use an established research design. The interventions described could include autistic individuals or their caregivers, healthcare staff, or could target change or improvement within organizations or health systems.

Studies were excluded if they involved the development or consideration of relevant interventions but did not evaluate these (e.g., Carter et al., 2017 ). In addition, studies were excluded if their focus was not specific to autism and/or it was not possible to extract data pertaining to the outcomes of the intervention for autistic participants or their caregivers (e.g., Drake et al., 2012 ). Other reasons for exclusion included the following: implementation of an intervention with a focus on improving healthcare providers’ diagnostic or screening capabilities (e.g., Bauer et al., 2015 ); interventions implemented in mental health or dental settings (e.g., Kuriakose et al., 2018 ); case reports or studies or multiple case studies reported together (e.g., Edwards & Northway, 2011 ); and no original, empirical data provided (e.g., Ailey et al., 2017 ). In some cases, multiple exclusion criteria were relevant to an individual study.

Titles and abstracts of all records returned during the electronic searches were screened by one author. If a study appeared relevant, or it was not possible to determine whether it met the inclusion and exclusion criteria, the full text was accessed. Full text review was completed by the full research team in tandem, and decisions regarding inclusion or exclusion were documented (see Supplemental Digital Content 2 ).

Data Extraction and Synthesis

Two authors independently completed data extraction with any disagreements resolved through discussion until consensus was achieved. Data were extracted on country, intervention setting(s), participant(s), research approach, intervention characteristics, evaluation measures, and outcomes.

As the included papers were diverse with regard to research design, and interventions implemented, it was necessary to develop and apply codes to facilitate data synthesis. Throughout the coding process, researchers were careful to ensure that the context and meaning of data was retained in spite of the application of codes (Sandelowski et al., 2012 ). Coding was completed by the research team together following completing of initial data extraction, discussing each study in detail prior to developing and/or selecting the appropriate code(s). The importance of dialogue between researchers during coding has been outlined previously (Graneheim & Lundman, 2004 ). Table 1 presents a complete summary of codes applied to synthesize data on participants (e.g., autistic children/adults and healthcare providers), setting (e.g., educational setting for autistic children/adults and primary care), evaluation measures (e.g., measures of reactions and measures of behavior), the intervention implemented (e.g., patient-focused and provider-focused), and the outcomes of the intervention (e.g., positive and mixed). With regard to evaluation measures employed, Kirkpatrick’s evaluation hierarchy (Kirkpatrick, 1967 ) was used to categorize the level of evaluation completed as has been done in other systematic reviews previously (Hammick et al., 2010 ; O’Dea et al., 2014 ). Further, outcome data within each paper were coded using a grading system of positive/mixed/negative/no clear effect, a similar process to that employed in other systematic reviews (e.g., McCormack et al., 2019 ) previously. Multiple codes were applied if required to capture the data presented.

Methodological Rigor

Methodological rigor was assessed by two researchers working in tandem and using the Quality Assessment Tool for Studies with Diverse Designs (QATSDD; Sirriyeh et al., 2012 ) which allows for the appraisal of studies using quantitative, qualitative, or mixed methods approaches. This 16-item tool has been widely applied in systematic reviews pertaining to autism and/or health services research (Ayres et al., 2018 ; McCormack et al., 2019 ; O’Dowd et al., 2019 ; Walsh et al., 2020a ). Items are rated on a four-point scale (0–3), with a higher score indicating greater rigor.

Figure  1 presents a PRISMA flow diagram depicting study selection. Titles and abstracts were examined for more than 3,800 records with full texts considered for 191 papers and 30 papers (Birkan et al., 2011 ; Broder-Fingert et al., 2016 ; Carbone et al., 2016 ; Cavalari et al., 2013 ; Chebuhar et al., 2013 ; Clark et al., 2016 ; Cox et al., 2017 ; Cuvo et al., 2010 ; Davit et al., 2011 ; Ellis et al., 2006 ; Ghuman et al., 2004 ; Giarelli et al., 2012 ; Gillis et al., 2009 ; Golnik et al., 2012 ; Havercamp et al., 2016 ; Iannuzzi et al., 2019 ; Lucarelli et al., 2018 ; Mazurek et al., 2017 ; Mazurek et al., 2019 ; McGonigle et al., 2014 ; McIntosh et al., 2015 ; Mills, 2017 ; Nicolaidis et al., 2016 ; Riosa et al., 2017 ; Riviere et al., 2011 ; Schiff et al., 2011 ; Shabani & Fisher, 2006 ; Swartz et al., 2017 ; Whippey et al., 2019 ; Wolff & Symons, 2013 ), describing 31 studies, ultimately included. Almost all studies (96.8%) were peer-reviewed with the exception of one study (Mills, 2017 ) which comprised a student thesis. Of the 31 studies included, more than 70% were conducted in the USA and 16% conducted in Canada. Studies conducted outside of North America were much fewer in number (< 15% of included studies). Table 2 provides a summary of the characteristics of included studies with a study-by-study summary available in Supplemental Digital Content 3 .

PRISMA flow diagram depicting study selection

Participants

As shown in Table 2 , participants in the studies were most commonly autistic children or adults (61.3% of studies), followed by healthcare providers (29% of studies). The participation of caregivers of autistic children or adults, healthcare administrators, and health profession students was less frequent (all < 15% of studies) across the studies reviewed.

Studies were most commonly conducted in primary care, secondary care, or educational settings (all in 22.6% of studies) that served autistic children or adults (see Table 2 ). Studies conducted in participants’ homes or the community, health profession education settings, prehospital care, or elsewhere were less frequent.

Research Approach

As outlined in Table 2 , studies were predominantly quantitative only, typically using groups research designs (51.6%; e.g., pretest–posttest design; quasi-randomized controlled trial) though single-subject research designs (e.g., changing criterion design; multiple baseline across participants design) were also common (32.3%). Five studies (16.1%) used a mixed methods research approach, collecting both qualitative and quantitative data as part of the intervention evaluation conducted.

Level of Evaluation

Full detail on the evaluations of interventions within studies is provided in Supplemental Digital Content 3 while Table 3 provides a summary of evaluation across the included studies, examples of evaluations conducted, and associated outcomes. Studies most commonly assessed behavior (54.8% of studies; e.g., compliance with components of a medical exam) followed by the assessment of reactions (45.2%; e.g., caregiver satisfaction with care) and then changes in attitudes (29%; e.g., confidence working with autistic patients). As can be seen in Table 3 , for each Kirkpatrick 1967 , level of evaluation, the majority of outcomes observed were positive.

In total, 80.6% ( n  = 25) of studies demonstrated positive outcomes. A further four studies showed positive outcomes on some measures and either no clear effects ( n  = 3; 9.7%) or mixed effects ( n  = 1; 3.2%). The two final studies reported outcome data that was classified as no clear effects.

Nature of the Intervention

For complete information on interventions delivered, please see Supplemental Digital Content 3 . Table 4 provides an overview of the types of interventions reported, along with examples and information on outcomes. A majority of studies were patient-focused (58.1%) with the intervention predicated on supporting the autistic patient to engage with, tolerate, or anticipate medical procedures, care, or settings. Targets in these studies varied but included teaching cooperation with injections (Birkan et al., 2011 ) and improving communication with autistic patients during medical exams (Mills, 2017 ). These interventions typically took the form of a behavioral intervention (72.2%). Outcomes of patient-focused interventions were predominantly positive (88.9% showing positive outcomes only).

Interventions were also commonly provider-focused (48.4%), or intended to equip healthcare providers with the knowledge and skills required to effectively support and care for autistic patients. Targets within studies varied but it was most common for studies to seek to improve staff knowledge of autism or attitudes towards autistic patients. These interventions were typically educational interventions (66.6%), though a small number of studies did describe care plans or quality improvement deemed to be provider-focused. Outcomes of provider-focused interventions were typically positive (66.6% of studies showing positive outcomes only).

Only two studies (6.5%) reported the use of interventions classified as organization-focused, and which dealt with altering the physical environment or organization in ways that created a more inclusive and supportive environment for autistic individuals. Both studies described the use of care plans for autistic patients, with one study documenting positive outcomes only (Whippey et al., 2019 ) and the other showing a positive effect on two outcome measures and no clear effect on a third (Swartz et al., 2017 ).

Methodological rigor was variable. QATSDD scores can range from 0 to 42 (qualitative or quantitative studies) or 46 (mixed methods studies). Mean QATSDD score was 18.3 (SD = 5.5, range = 9–28). Studies performed best on items assessing description of the research setting, aims and objectives, and data collection procedures. Studies performed most poorly on items appraising consideration of sample size, justification of analyses, and user involvement in design.

Comparison by Research Design

Of the 31 studies included, 11 utilized a single-subject research design (35.5%; e.g., changing criterion design; reversal design) and 21 utilized some form of groups design (67.7%; e.g., randomized controlled pilot trial study). Table 5 presents a comparison of the level of evaluation, nature of intervention, outcomes, and methodological rigor by research design. As can be seen, there appeared to be less variability within the methods and outcomes of studies utilizing single-subject research designs.

Morbidity and mortality are substantially increased among autistic individuals internationally. Relatedly, autistic individuals experience issues in accessing and receiving high quality healthcare that are not experienced by neurotypical individuals (Mason et al., 2019 ; Muskat et al., 2015 ; Walsh et al., 2020a ). There is therefore a need, and in some countries a legal requirement, to develop and implement interventions that can improve the healthcare experience or access of autistic persons. Accordingly, the current systematic review offers a synthesis of research on interventions to improve healthcare experiences and/or access for autistic persons. Key findings include the small body of research describing evaluations of relevant interventions, the predominant focus on changing the behavior of the patient with comparatively little work focused on changing healthcare provider behavior and/or healthcare organizations and systems, and relatively weak forms of outcome measurement.

Although many papers exist that provide recommendations for providing healthcare to autistic individuals (Barber, 2017 ; Bellando et al., 2016 ; Nicolaidis et al., 2014 ; Souders et al., 2002 ; Venkat et al., 2012 ), a much smaller body of literature ( n  = 30 papers) has evaluated interventions to improve access to, or experiences of, healthcare for autistic persons. Of the studies reviewed, only 13% were conducted outside of North America and just under half (48.3%) took place in a healthcare setting. A recent workshop (Warner et al., 2019 ) that brought together autistic people, their relatives, clinicians, healthcare managers, and others identified the development of autism-friendly healthcare services as a research priority. While it is encouraging that a majority of studies reviewed herein (80.6%) demonstrated positive outcomes of interventions, the autistic community is heterogeneous (Nicolaidis et al., 2014 ) and additional research will be required to explore, and establish, a variety of evidence-based practices suitable for facilitating care of autistic individuals of all ages, and across all health specialties and settings. This is particularly true given that none of the included studies employed a full randomized controlled trial design (two studies employed randomization but were described as pilot studies) which continues to constitute the “gold standard” for interventional research and to afford the most trustworthy evidence on effectiveness (Bothwell et al., 2016 ; Hariton & Locascio, 2018 ). However, research has now effectively delineated the barriers to healthcare access (Calleja et al., 2020 ; Mason et al., 2019 ; Walsh et al., 2020a ), and there exist a number of tools to facilitate data collection in relation to the barriers to healthcare access experienced by autistic adults (Nicolaidis et al., 2013 ) or the caregivers of autistic persons (Walsh et al., 2020b ) or healthcare providers (Walsh et al., 2021 ) which should facilitate future research in this area. This research must move beyond developing understanding on, and collecting data relating to experiences of, healthcare access and experiences of autistic persons to “actioning” data to inform the development and implementation of interventions to improve the care and health of autistic persons.

It is notable that interventions were most commonly categorized as patient-focused (58.1% of studies) and sought to support the autistic individual in engaging with, tolerating, or anticipating medical procedures, care, or settings. All studies using single-subject research designs reported on patient-focused interventions which likely reflects the popularity of the design type in behavior analytic research (Scruggs & Mastropieri, 1998 ) and the frequency with which behavior analysts work directly with autistic children and adults (Keenan et al., 2015 ). It is certainly known that autism-related characteristics (e.g., sensory issues, communication issues, and challenging behavior) are barriers to healthcare access and may impede the provision of high quality care for autistic patients (Walsh et al., 2020a ). However, autistic individuals and caregivers have also highlighted healthcare provider-related barriers (e.g., lack of knowledge/skills and inflexibility) and healthcare system-related barriers (e.g., lack of continuity/collaboration and lack of time/resources), which were addressed less frequently in included studies (48.3% of studies and 6.5% of studies, respectively). Future research must look beyond approaches targeting only the behavior of the autistic individual and must instead ensure that interventions target the other factors that are known to impact upon access to, and experiences of, care (Walsh et al., 2020a ). Although interventions seeking to change healthcare provider behavior, or management, coordination, or facilities within organizations or healthcare services, may be more complex or resource-intensive, they are essential to ensure that care is improved for the many diverse autistic persons accessing healthcare services daily and addressing unmet needs and issues of poor care which persist in the autistic community (Nicolaidis et al., 2013 ). Researchers, or quality professionals, may wish to use existing tools to establish the barriers to care in advance of intervention development to ensure that interventions are likely to yield perceptible improvements for autistic patients. For example, Raymaker and colleagues ( 2017 ) have developed a tool that allows autistic individuals to self-report barriers experienced in healthcare settings, Walsh et al. ( 2020b ) present a tool which allows the caregivers of autistic patients to report on the frequency and severity of barriers experienced in physical healthcare settings, and Walsh et al. ( 2021 ) have completed initial validation of a tool which allows healthcare providers to identify barriers to care provision or access for autistic patients. The use of such tools will allow providers or organizations to determine the most frequent (and in some instances, the most severe) barriers impacting care for autistic patients within their organization or setting. This may allow prioritization and evaluation of interventions or strategies which are likely to address these specific barriers. Walsh and colleagues (Walsh et al., 2020a ) have offered recommendations for interventions to address the most common barriers to healthcare access for autistic patients (e.g., the use of picture or visual schedules before and during medical encounters to address communication challenges; use of a shared care plan to facilitate continuity of care or collaboration between healthcare providers). There is additional work required, however, to develop complete and comprehensive guidelines on how to address particular barriers and what interventions might be most appropriate in what instances as this will be crucial to support organizations in implementing effective practices to improve care. Finally, within studies evaluating patient-focused interventions that were intended to change, or teach, behaviors, participants typically received 1:1 behavioral interventions in their educational setting which yielded uniformly positive outcomes. There is some research (Law et al., 2018 , 2019 ; Rogerson et al., 2019 ) demonstrating the efficacy of technologies in facilitating the delivery of parent-mediated behavioral intervention to autistic children. Researchers could explore capitalizing on technology to allow such interventions to be accessed more readily by autistic individuals and caregivers or provided more commonly through healthcare services or organizations.

The most important measure of the impact of an intervention to improve access to care and care experiences must be whether it actually improves delivery of care, care experiences, and health outcomes for autistic persons. Included studies typically relied on measures of behavior (54.8%; Kirkpatrick Level 3) when evaluating intervention outcomes, most commonly the behavior of the autistic individual undergoing a medical procedure, followed by measures of reactions (45.2%; Kirkpatrick Level 1), typically of healthcare providers to an educational intervention. More than 90% of studies using single-subject research designs used measures of behavior which, again, likely reflects that these studies were typically behavior analytic in nature and that the measurement of behavior is a cornerstone of applied behavior analysis (Dillenburger & Keenan, 2009 ). There was greater variability in outcome measures utilized within studies employing groups designs which likely reflects that these studies have emerged from a variety of disciplines and that a variety of constructs (e.g., knowledge and attitudes) were targeted within the interventions. Notably, less than one-fifth of studies used measures that were classified as Level 4 measures on Kirkpatrick’s evaluation hierarchy (Kirkpatrick, 1967 ). Level 4 measures consider “business results” (Reio et al., 2017 ) or impact upon the practice of the organization and/or improvements for employees or service-users. Given data on increased morbidity and mortality (Croen et al., 2015 ; Hirvikoski et al., 2016 ; Hwang et al., 2019 ; Mouridsen et al., 2008 ; Shavelle et al., 2001 ), unmet healthcare needs (Nicolaidis et al., 2013 ), and poorer care experiences (Nicolaidis et al., 2013 ; Tregnago & Cheak-Zamora, 2012 ) in autism, we argue that future studies evaluating interventions must assess for stronger indicators of change and impact (e.g., measures of health of autistic persons and measures of patient comfort and/or distress during healthcare provider consultations). It is crucial also that studies measure what matters to autistic patients (Coulter, 2017 ). Research which engages the autistic community to determine preferred outcome measures, and/or which considers what routinely collected healthcare data may offer useful insights, would be of much use in advancing knowledge on how to most efficiently and effectively determine the effect an intervention has had for patients. The measurement of intervention outcomes at the higher Kirkpatrick levels is essential for engaging policymakers and securing financial support (Gordon et al., 2018 ). Stronger data to support intervention effectiveness may also encourage adoption or uptake of interventions thereby “spreading” positive change in the physical healthcare of persons with autism.

Limitations

This review had a number of limitations. First, we excluded studies that delivered interventions focused solely on improving screening and/or diagnosis of autism by healthcare professionals and within included studies any related data were not extracted. We recognize that the identification of a patient as autistic may result in better accommodation of the patient’s needs and improved healthcare provision. However, these studies are well-reviewed elsewhere (Daniels et al., 2014 ; McCormack et al., 2019 ) and were not considered to have the specific focus on improving physical healthcare access or care experiences required.

Second, the focus on interventions to improve physical healthcare access and/or experiences only should be noted. Autistic individuals are at a greater risk for experiencing psychiatric conditions (Croen et al., 2015 ) so quality of care in mental healthcare services is important. Similarly, autistic individuals may be more likely to experience poor dental or oral health (e.g., caries and receding gums; Blomqvist et al., 2015 ; Loo et al., 2008 ). However, as the barriers to physical healthcare access have been delineated (Mason et al., 2019 ; Walsh et al., 2020a ), and may differ from the barriers experiences within other types of services, it was considered most appropriate to focus on interventions that were implemented and evaluated in such settings. Future research that synthesizes data on the barriers to accessing mental, dental, or other forms of healthcare services for autistic individuals, and interventions to address these issues, is recommended.

Finally, the decision to include grey literature within a review can be contentious. There is a lack of established best practice in how to search and engage with grey literature, searches of the grey literature may not be replicable by others, the return on resources invested for identification of grey literature can be limited, and issues with the interpretation or extraction of data within grey literature can arise due to poor methodological quality and/or poor reporting (Benzies et al., 2006 ; Egger et al., 2003 ; Mahood et al., 2014 ; Martin et al., 2005 ). There is some empirical evidence to suggest that the methodological quality of grey literature studies is poorer than that of peer-reviewed studies (Egger et al., 2003 ; MacLean et al., 2003 ). This is an important finding as it suggests the inclusion of grey literature could constitute a threat to the internal validity of a systematic review as it may lack the methodology and controls to address the research question. These are this issues of which a reader should be aware. However, prestigious evidence-synthesis organizations such as Cochrane (Higgins et al., 2019 ) and the Campbell Collaboration ( 2016 ) do recommend searches of the grey literature. Within systematic reviews focused on interventions, and their effectiveness, the non-inclusion of grey literature can lead to a problematic overestimation of effect size or overly favorable judgments of intervention effects (Hopewell et al., 2007 ). Further, grey literature searches are particularly recommended where a small body of research exists, there is a lack of consensus on appropriate outcome measurement, and where the context of the intervention is important (Benzies et al., 2006 ). Accordingly, we did consider grey literature for inclusion in the current review. Our search tactics were intended to identify grey literature from English-speaking countries through the use of Google (location set to UK; first 100 returns), Google Scholar (location set to UK; first 100 returns), Ethos, and OpenGrey. Specified search locations impact ordering of returns on Google databases. Ethos and OpenGrey are largely focused on grey literature arising from Europe or the UK. Therefore, it is important to note that our grey literature search tactics may not have adequately identified papers published outside of Europe. In total, one grey literature study (Mills, 2017 ) met our inclusion criteria and has been integrated within the current review to provide as complete as possible of an overview of work to date in this area and to showcase the full extent of interventions which have been trialed in this area. This study was reviewed using the QATSDD, and the quality score achieved (17) was similar to many of the peer-reviewed studies (e.g., mean QATSDD scores for groups designs studies = 17.7, range 9–28).

Recommendations for Future Research

The synthesis conducted allows for a number of recommendations for future research to be offered. First, the knowledge, behavior, and attitudes of healthcare providers are a recognized barrier to healthcare access for autistic individuals (Walsh et al., 2020a ). However, a majority of provider-focused interventions consisted of educational interventions only, and in most cases, these were largely lecture-based. However, it is well-recognized that the change resulting from education alone is minimal (Soong & Shojania, 2020 ). It is essential that future research on educational interventions requires active engagement by participants and affords attendees relevant opportunities to practice target behaviors/knowledge/attitudes and receive feedback if training is to result in changes in behavior that generalize to the clinical environment and into patient consultations. The use of simulation is becoming increasingly commonplace in healthcare research and practice (Walsh et al., 2018 ) as a means of recreating or replicating patients, clinical situations, and/or facilities. Simulation has the potential to allow physicians to engage in targeted behaviors outside of the clinical setting and to receive corrective feedback which may result in improved clinical performance. Indeed, one included study described the use of an autistic standardized patient (i.e., actor portraying a patient with a particular condition) among nursing students which was reported to impact on communication skills, along with students’ critical thinking, prioritization skills, and patient assessment abilities (McIntosh et al., 2015 ). Simulation could be coupled with in vivo coaching and feedback from an expert to teach healthcare workers how to appropriately perform key non-technical skills (e.g., communication, empathy, and shared decision-making) and technical skills (e.g., physical exams, blood draws, and MRI scans) when caring for autistic patients. Other applications of simulation are possible too. For example, the use of virtual patients to improve knowledge of autism or the use of high-fidelity simulations to explore healthcare workers’ implementation of accommodations as outlined in personalized care plans for autistic patients. Future research which further considers the potential for simulation to yield more effective educational interventions is therefore recommended. However, education or training is just one means of improving healthcare delivery and services for autistic patients. Researchers seeking to develop, implement, and evaluate provider-focused interventions should therefore also consider how other forms of provider-focused interventions such as individualized care plans, checklists, guidelines, and technological supports may serve to more effectively improve the experience of autistic patients.

Second, studies scored poorly in the quality assessment in relation to user involvement in the design of the research. Only three studies (Chebuhar et al., 2013 ; Nicolaidis et al., 2016 ; Whippey et al., 2019 ) indicated any involvement of autistic persons or their families in the development of the research or intervention that was reported upon. The rationale for including autistic individuals in the prioritization, planning, and conduct of research relating to autism has been explicated and well-argued (Fletcher-Watson et al., 2019 ; Jivraj et al., 2014 ). Although such partnerships may encounter challenges (Pellicano et al., 2014 ), such involvement will yield data that are more socially valid and may result in improved translation of research and better outcomes for autistic people (Fletcher-Watson et al., 2019 ). Social validity relates to the importance of intervention targets (i.e., is this an appropriate or useful goal?), the appropriateness of the intervention procedures (i.e., are the intervention procedure acceptable?), and the perceived importance of the outcomes observed (i.e., are stakeholders pleased with the results of the intervention?) (Wolf, 1978 ), and should be assessed via engagement with stakeholders (i.e., the individual participating within the interventions and those close to them). Social validity has been long discussed in relation to behavioral interventions but its measurement or consideration has not been optimal (Schwartz & Baer, 1991 ; Wolf, 1978 ). It is essential that the design of future research, and interventions in this area, is inclusive of autistic individuals, and the caregivers or family members of autistic individuals as appropriate. This will contribute to ensuring that interventions are maximally socially valid and may contribute to clarifying what interventions should be prioritized.

Relatedly, it would be of much use to engage stakeholders (e.g., autistic persons, the caregivers of autistic persons, healthcare providers, and healthcare managers) in a process involving the mapping of barriers to relevant interventions. Researchers have previously attempted to provide suggestions for interventions to address specific barriers to physical healthcare access (e.g., use of picture/visual schedules as a means of addressing communication difficulties, development, and delivery of training or education to address issues relating to provider knowledge or skills; Walsh et al., 2020a ). However, there is a need to complete this process more systematically, to include a comprehensive overview of existing interventions such as that offered in the current review, and to conduct the process in a way that is inclusive of the stakeholders who will be responsible for the delivery of the intervention or who will experience the implementation of the intervention. Intervention mapping approaches such as this are growing in popularity and have been used with patient populations including patients with cancer, patients experiencing mental health issues, patients with HIV, and others (Majid et al., 2018 ). Such a process may valuably identify frequent or pervasive barriers that are not adequately addressed by existing or previously trialed interventions, may lead to suggestions for novel interventions that could be evaluated, and, through the consideration of the frequency with which specific barriers are reported within research, may assist with the prioritization of interventions for evaluation. There is a clear interest in contributing to the improvement of healthcare services among the autistic community (Warner et al., 2019 ) so such an exercise may be well received and yield good engagement.

Next, given the relative dearth of literature focused on interventions to improve physical healthcare access, this review has synthesized interventions implemented across all types of healthcare settings. However, it is important to consider differences, or specific challenges or procedures, which may exist across healthcare settings and how these may be addressed. For example, the experience of an autistic patient receiving routine preventative care within a primary care setting from a known family doctor/general practitioner is likely to be quite different to that of an autistic patient admitted to an emergency department while acutely unwell or an autistic patient admitted to hospital for scheduled/elective surgery. The use of patient narratives is a recognized means of informing quality improvement initiatives in healthcare (Tsianakas et al., 2012 ; Wilcock et al., 2003 ) and may be a useful means of furthering understanding of care within specific settings or in relation to specific care services. Alternatively, other qualitative methodologies may usefully elucidate patient experiences. For example, Donovan ( 2020 ) describes engagement with autistic women via semi-structured interview in order to develop an understanding of their experiences and particular challenges or difficulties encountered. Nicholas et al. ( 2016 ) employed semi-structured interviews to develop an understanding of the experiences of families with an autistic child required to attend the emergency department. As research on the healthcare experiences of individuals with autism increases, such data may usefully facilitate quality improvement in specific services or specialties.

Finally, included studies typically provided limited detail on the implementation of the intervention. This issue of insufficient description of interventions has been identified previously (Glasziou et al., 2010 ; Michie et al., 2009 ), has been demonstrated to hinder replication of interventions (Glasziou et al., 2008 ) by other researchers, and is suggested to contribute substantially to the “waste” of healthcare research (Glasziou et al., 2010 ) as it precludes the implementation or re-creation of interventions by those working on-the-ground. Further, the effects of interventions which are informed, or delivered, by individuals with high levels of training or expertise, or which are well resourced in terms of personnel or funding, may not be generalizable to other contexts or settings. Therefore, it is essential that future research provides complete detail on intervention content (e.g., components and materials), agent (e.g., expertise/qualifications and supports provided), setting, delivery of intervention (e.g., frequency and schedule), and any flexibility in delivery of interventions (Glasziou et al., 2010 ).

The need to adapt healthcare services for autistic persons is recognized as a priority. However, only a small body of literature internationally has considered how this can be effectively done. Further research is imperative and should look beyond the autistic individual to how provider behavior and healthcare organizations must alter. The design of future research must be inclusive of the autistic community, must measure what matters, and must offer complete detail on interventions implemented.

We have deliberately opted to use identity-first language (i.e., autistic individual) rather than person-first language (i.e., individual with autism) in this manuscript. The appropriate use of language around autism is recognized as a complex issue (Shakes & Cashin, 2019 ). However, we have made this decision as, in recent years, autistic individuals have expressed a strong preference for the use of identity-first language (Kenny et al., 2016 ; Robison, 2019 ) and the use of person-first language has been suggested to perpetuate or sustain stigma around disability (Gernsbacher, 2017 ).

Ailey, S. H., Brown, P. J., & Ridge, C. M. (2017). Improving hospital care of patients with intellectual and developmental disabilities. Disability & Health Journal, 10 (2), 169–172. https://doi.org/10.1016/j.dhjo.2016.12.019

Article   Google Scholar  

American Psychiatric Association. (2013). Diagnostic and statistical manual of mental disorders (5th ed). https://doi.org/10.1176/appi.books.9780890425596

Ayres, M., Parr, J. R., Rodgers, J., Mason, D., Avery, L., & Flynn, D. (2018). A systematic review of quality of life of adults on the autism spectrum. Autism, 22 (7), 774–783. https://doi.org/10.1177/1362361317714988

Article   PubMed   Google Scholar  

Barber, C. (2017). Meeting the healthcare needs of adults on the autism spectrum. British Journal of Nursing, 26 (7), 420–425. https://doi.org/10.12968/bjon.2017.26.7.420

Bauer, N. S., Carroll, A. E., Saha, C., & Downs, S. M. (2015). Computer decision support changes physician practice but not knowledge regarding autism spectrum disorders. Applied Clinical Informatics, 6 (3), 454–465. https://doi.org/10.4338/aci-2014-09-ra-0084

Article   PubMed   PubMed Central   Google Scholar  

Bellando, J., Fussell, J. J., & Lopez, M. (2016). Autism speaks toolkits: Resources for busy physicians. Clinical Pediatrics, 55 (2), 171–175. https://doi.org/10.1177/0009922815594587

Benzies, K. M., Premji, S., Hayden, K. A., & Serrett, K. (2006). State-of-the-evidence reviews: Advantages and challenges of including grey literature. Worldviews on Evidence-Based Nursing, 3 (2), 55–61. https://doi.org/10.1111/j.1741-6787.2006.00051.x

Birkan, B., Krantz, P. J., & McClannahan, L. E. (2011). Teaching children with autism spectrum disorders to cooperate with injections. Research in Autism Spectrum Disorders, 5 (2), 941–948. https://doi.org/10.1016/j.rasd.2010.11.004

Bishop-Fitzpatrick, L., & Kind, A. J. (2017). A scoping review of health disparities in autism spectrum disorder. Journal of Autism and Developmental Disorders, 47 (11), 3380–3391. https://doi.org/10.1007/s10803-017-3251-9

Blomqvist, M., Bejerot, S., & Dahllöf, G. (2015). A cross-sectional study on oral health and dental care in intellectually able adults with autism spectrum disorder. BMC Oral Health, 15 (1), 1–8. https://doi.org/10.1186/s12903-015-0065-z

Bothwell, L. E., Greene, J. A., Podolsky, S. H., & Jones, D. S. (2016). Assessing the gold standard—Lessons from the history of RCTs. New England Journal of Medicine, 374 (22), 2175–2181.

Broder-Fingert, S., Shui, A., Ferrone, C., Iannuzzi, D., Cheng, E. R., Giauque, A., Connors, S., McDougle, C. J., Donelan, K., Neumeyer, A., & Kuhlthau, K. (2016). A pilot study of autism-specific care plans during hospital admission. Pediatrics, 137 , S196–S204. https://doi.org/10.1542/peds.2015-2851R

Brugha, T. S., Spiers, N., Bankart, J., Cooper, S.-A., McManus, S., Scott, F. J., Smith, J., & Tyrer, F. (2016). Epidemiology of autism in adults across age groups and ability levels. The British Journal of Psychiatry, 209 (6), 498–503. https://doi.org/10.1192/bjp.bp.115.174649

Calleja, S., Islam, F. M. A., Kingsley, J., & McDonald, R. (2020). Healthcare access for autistic adults: A systematic review. Medicine, 99 (29), e20899. https://doi.org/10.1097/MD.0000000000020899

Carbone, P. S., Norlin, C., & Young, P. C. (2016). Improving early identification and ongoing care of children with autism spectrum disorder. Pediatrics, 137 (6), e20151850. https://doi.org/10.1542/peds.2015-1850

Carter, J., Broder-Fingert, S., Neumeyer, A., Giauque, A., Kao, A., & Iyasere, C. (2017). Brief report: Meeting the needs of medically hospitalized adults with autism: A provider and patient toolkit. Journal of Autism and Developmental Disorders, 47 (5), 1510–1529. https://doi.org/10.1007/s10803-017-3040-5

Cavalari, R. N., DuBard, M., Luiselli, J. K., & Birtwell, K. (2013). Teaching an adolescent with autism and intellectual disability to tolerate routine medical examination: Effects of a behavioral compliance training package. Clinical Practice in Pediatric Psychology, 1 (2), 121–128. https://doi.org/10.1037/cpp0000013

Chebuhar, A., McCarthy, A. M., Bosch, J., & Baker, S. (2013). Using picture schedules in medical settings for patients with an autism spectrum disorder. Journal of Pediatric Nursing, 28 (2), 125–134. https://doi.org/10.1016/j.pedn.2012.05.004

Clark, A., Browne, S., Boardman, L., Hewitt, L., & Light, S. (2016). Implementing UK autism policy & national institute for health and care excellence guidance-assessing the impact of autism training for frontline staff in community learning disabilities teams. British Journal of Learning Disabilities, 44 (2), 103–110. https://doi.org/10.1111/bld.12116

Coulter, A. (2017). Measuring what matters to patients. BMJ, 356 , j816.

Cox, A. D., Virues-Ortega, J., Julio, F., & Martin, T. L. (2017). Establishing motion control in children with autism and intellectual disability: Applications for anatomical and functional MRI. Journal of Applied Behavior Analysis, 50 (1), 8–26. https://doi.org/10.1002/jaba.351

Croen, L. A., Zerbo, O., Qian, Y., Massolo, M. L., Rich, S., Sidney, S., & Kripke, C. (2015). The health status of adults on the autism spectrum. Autism, 19 (7), 814–823. https://doi.org/10.1177/1362361315577517

Cuvo, A. J., Reagan, A. L., Ackerlund, J., Huckfeldt, R., & Kelly, C. (2010). Training children with autism spectrum disorders to be compliant with a physical exam. Research in Autism Spectrum Disorders, 4 (2), 168–185. https://doi.org/10.1016/j.rasd.2009.09.001

Daniels, A. M., Halladay, A. K., Shih, A., Elder, L. M., & Dawson, G. (2014). Approaches to enhancing the early detection of autism spectrum disorders: A systematic review of the literature. Journal of the American Academy of Child & Adolescent Psychiatry, 53 (2), 141–152. https://doi.org/10.1016/j.jaac.2013.11.002

Davit, C. J., Hundley, R. J., Bacic, J. D., & Hanson, E. M. (2011). A pilot study to improve venipuncture compliance in children and adolescents with autism spectrum disorders. Journal of Developmental and Behavioral Pediatrics, 32 (7), 521–525. https://doi.org/10.1097/DBP.0b013e3182245b09

Department of Health (2010). Fulfilling and rewarding lives: The strategy for adults with autism in England . Crown. https://webarchive.nationalarchives.gov.uk/20130104203954/http://www.dh.gov.uk/prod_consum_dh/groups/dh_digitalassets/@dh/@en/@ps/documents/digitalasset/dh_113405.pdf . Accessed July 2021.

Dillenburger, K., & Keenan, M. (2009). None of the As in ABA stand for autism: Dispelling the myths. Journal of Intellectual and Developmental Disability, 34 (2), 193–195. https://doi.org/10.1080/13668250902845244

Donovan, J. (2020). Childbirth experiences of women with autism spectrum disorder in an acute care setting. Nursing for Women’s Health, 24 (3), 165–174. https://doi.org/10.1016/j.nwh.2020.04.001

Drake, J., Johnson, N., Stoneck, A. V., Martinez, D. M., & Massey, M. (2012). Evaluation of a coping kit for children with challenging behaviors in a pediatric hospital. Pediatric Nursing, 38 (4), 215–221.

PubMed   Google Scholar  

Edwards, J., & Northway, R. (2011). Helping a person with autism to overcome her fear of needles. Primary Health Care, 21 (10), 26–29. https://doi.org/10.7748/phc2011.12.21.10.26.c8850

Egger, M., Juni, P., Bartlett, C., Holenstein, F., & Sterne, J. (2003). How important are comprehensive literature searches and the assessment of trial quality in systematic reviews? Empirical study. Health Technololgy Assessment, 7 (1), 1–76.

Ellis, E. M., Ala’i-Rosales, S. S., Glenn, S. S., Rosales-Ruiz, J., & Greenspoon, J. (2006). The effects of graduated exposure, modeling, and contingent social attention on tolerance to skin care products with two children with autism. Research in Developmental Disabilities, 27 (6), 585–598. https://doi.org/10.1016/j.ridd.2005.05.009

Emerson, E. (2011). Health status and health risks of the “hidden majority” of adults with intellectual disability. Intellectual and Developmental Disabilities, 49 (3), 155–165. https://doi.org/10.1352/1934-9556-49.3.155

Emerson, E., & Baines, S. (2011). Health inequalities and people with learning disabilities in the UK. Tizard Learning Disability Review, 16 (1), 42–48.

Fletcher-Watson, S., Adams, J., Brook, K., Charman, T., Crane, L., Cusack, J., Leekam, S., Milton, D., Parr, J. R., & Pellicano, E. (2019). Making the future together: Shaping autism research through meaningful participation. Autism, 23 (4), 943–953. https://doi.org/10.1177/1362361318786721

Gernsbacher, M. A. (2017). Editorial perspective: The use of person-first language in scholarly writing may accentuate stigma. Journal of Child Psychology and Psychiatry, 58 (7), 859–861. https://doi.org/10.1111/jcpp.12706

Ghuman, J. K., Cataldo, M. D., Beck, M. H., & Slifer, K. J. (2004). Behavioral training for pill-swallowing difficulties in young children with autistic disorder. Journal of Child & Adolescent Psychopharmacology, 14 (4), 601–611. https://doi.org/10.1089/cap.2004.14.601

Giarelli, E., Ruttenberg, J., & Segal, A. (2012). Continuing education for nurses in the clinical management of autism spectrum disorders: Results of a pilot evaluation. The Journal of Continuing Education in Nursing, 43 (4), 169–176. https://doi.org/10.3928/00220124-20111115-01

Gillis, J. M., Hammond Natof, T., Lockshin, S. B., & Romanczyk, R. G. (2009). Fear of routine physical exams in children with autism spectrum disorders: Prevalence and intervention effectiveness. Focus on Autism and Other Developmental Disabilities, 24 (3), 156–168. https://doi.org/10.1177/1088357609338477

Glasziou, P., Chalmers, I., Altman, D. G., Bastian, H., Boutron, I., Brice, A., Jamtvedt, G., Farmer, A., Ghersi, D., & Groves, T. (2010). Taking healthcare interventions from trial to practice. BMJ, 341 , c3852. https://doi.org/10.1136/bmj.c3852

Glasziou, P., Meats, E., Heneghan, C., & Shepperd, S. (2008). What is missing from descriptions of treatment in trials and reviews? BMJ, 336 (7659), 1472–1474. https://doi.org/10.1136/bmj.39590.732037.47

Golnik, A., Scal, P., Wey, A., & Gaillard, P. (2012). Autism-specific primary care medical home intervention. Journal of Autism & Developmental Disorders, 42 (6), 1087–1093. https://doi.org/10.1007/s10803-011-1351-5

Gordon, M., Hill, E., Stojan, J. N., & Daniel, M. (2018). Educational interventions to improve handover in health care: An updated systematic review. Academic Medicine, 93 (8), 1234–1244. https://doi.org/10.1097/ACM.0000000000002236

Graneheim, U. H., & Lundman, B. (2004). Qualitative content analysis in nursing research: Concepts, procedures and measures to achieve trustworthiness. Nurse Education Today, 24 (2), 105–112. https://doi.org/10.1016/j.nedt.2003.10.001

Hammick, M., Dornan, T., & Steinert, Y. (2010). Conducting a best evidence systematic review. Part 1: From idea to data coding. BEME Guide No. 13. Medical Teacher, 32 (1), 3–15. https://doi.org/10.3109/01421590903414245

Hariton, E., & Locascio, J. J. (2018). Randomised controlled trials—The gold standard for effectiveness research. British Journal of Obstetrics and Gynaecology, 125 (13), 1716.

Havercamp, S. M., Ratliff-Schaub, K., Macho, P. N., Johnson, C. N., Bush, K. L., & Souders, H. T. (2016). Preparing tomorrow’s doctors to care for patients with autism spectrum disorder. Intellectual & Developmental Disabilities, 54 (3), 202–216. https://doi.org/10.1352/1934-9556-54.3.202

Higgins, J. P., Thomas, J., Chandler, J., Cumpston, M., Li, T., Page, M. J., & Welch, V. A. (2019). Cochrane handbook for systematic reviews of interventions . Wiley.

Book   Google Scholar  

Hirvikoski, T., Mittendorfer-Rutz, E., Boman, M., Larsson, H., Lichtenstein, P., & Bölte, S. (2016). Premature mortality in autism spectrum disorder. The British Journal of Psychiatry, 208 (3), 232–238. https://doi.org/10.1192/bjp.bp.114.160192

Hopewell, S., McDonald, S., Clarke, M. J., Egger, M. (2007). Grey literature in meta‐analyses of randomized trials of health care interventions. Cochrane Database of Systematic Reviews, (2) . https://doi.org/10.1002/14651858.MR000010.pub3

Hwang, Y. I., Srasuebkul, P., Foley, K. R., Arnold, S., & Trollor, J. N. (2019). Mortality and cause of death of Australians on the autism spectrum. Autism Research, 12 (5), 806–815. https://doi.org/10.1002/aur.2086

Iannuzzi, D., Rissmiller, P., Duty, S. M., Feeney, S., Sullivan, M., & Curtin, C. (2019). Addressing a gap in healthcare access for transition-age youth with autism: A pilot educational intervention for family nurse practitioner students. Journal of Autism & Developmental Disorders, 49 (4), 1493–1504. https://doi.org/10.1007/s10803-018-3846-9

Jivraj, J., Sacrey, L.-A., Newton, A., Nicholas, D., & Zwaigenbaum, L. (2014). Assessing the influence of researcher–partner involvement on the process and outcomes of participatory research in autism spectrum disorder and neurodevelopmental disorders: A scoping review. Autism, 18 (7), 782–793. https://doi.org/10.1177/1362361314539858

Kamat, P. P., Bryan, L. N., McCracken, C. E., Simon, H. K., Berkenbosch, J. W., & Grunwell, J. R. (2018). Procedural sedation in children with autism spectrum disorders: A survey of current practice patterns of the society for pediatric sedation members. Pediatric Anesthesia, 28 (6), 552–557. https://doi.org/10.1111/pan.13387

Keenan, M., Dillenburger, K., Röttgers, H. R., Dounavi, K., Jónsdóttir, S. L., Moderato, P., Schenk, J. J., Virués-Ortega, J., Roll-Pettersson, L., & Martin, N. (2015). Autism and ABA: The gulf between North America and Europe. Review Journal of Autism and Developmental Disorders, 2 (2), 167–183. https://doi.org/10.1007/s40489-014-0045-2

Kenny, L., Hattersley, C., Molins, B., Buckley, C., Povey, C., & Pellicano, E. (2016). Which terms should be used to describe autism? Perspectives from the UK autism community. Autism, 20 (4), 442–462. https://doi.org/10.1177/1362361315588200

Khanna, R., Jariwala-Parikh, K., West-Strum, D., & Mahabaleshwarkar, R. (2014). Health-related quality of life and its determinants among adults with autism. Research in Autism Spectrum Disorders, 8 (3), 157–167. https://doi.org/10.1016/j.rasd.2013.11.003

Kirkpatrick, D. L. (1967). Evaluation of training. In R. L. Craig & L. R. Bittel (Eds.), Training and development handbook  (pp. 87–112). McGraw Hill.

Koski, S., Gabriels, R. L., & Beresford, C. (2016). Interventions for paediatric surgery patients with comorbid autism spectrum disorder: A systematic literature review. Archives of Disease in Childhood, 101 (12), 1090–1094. https://doi.org/10.1136/archdischild-2016-310814

Kuhlthau, K., Orlich, F., Hall, T. A., Sikora, D., Kovacs, E. A., Delahaye, J., & Clemons, T. E. (2010). Health-related quality of life in children with autism spectrum disorders: Results from the autism treatment network. Journal of Autism and Developmental Disorders, 40 (6), 721–729. https://doi.org/10.1007/s10803-009-0921-2

Kupzyk, S., & Allen, K. D. (2019). A review of strategies to increase comfort and compliance with medical/dental routines in persons with intellectual and developmental disabilities. Journal of Developmental and Physical Disabilities, 31 (2), 231–249. https://doi.org/10.1007/s10882-018-09656-y

Kuriakose, S., Filton, B., Marr, M., Okparaeke, E., Cervantes, P., Siegel, M., Horwitz, S., & Havens, J. (2018). Does an autism spectrum disorder care pathway improve care for children and adolescents with ASD in inpatient psychiatric units? Journal of Autism & Developmental Disorders, 48 (12), 4082–4089. https://doi.org/10.1007/s10803-018-3666-y

Law, G. C., Dutt, A., & Neihart, M. (2019). Increasing intervention fidelity among special education teachers for autism intervention: A pilot study of utilizing a mobile-app-enabled training program. Research in Autism Spectrum Disorders, 67 , 101411. https://doi.org/10.1016/j.rasd.2019.101411

Law, G. C., Neihart, M., & Dutt, A. (2018). The use of behavior modeling training in a mobile app parent training program to improve functional communication of young children with autism spectrum disorder. Autism, 22 (4), 424–439. https://doi.org/10.1177/1362361316683887

Loo, C. Y., Graham, R. M., & Hughes, C. V. (2008). The caries experience and behavior of dental patients with autism spectrum disorder. The Journal of the American Dental Association, 139 (11), 1518–1524. https://doi.org/10.14219/jada.archive.2008.0078

Lucarelli, J., Welchons, L., Sideridis, G., Sullivan, N. R., Chan, E., & Weissman, L. (2018). Development and evaluation of an educational initiative to improve hospital personnel preparedness to care for children with autism spectrum disorder. Journal of Developmental and Behavioral Pediatrics, 39 (5), 358–364. https://doi.org/10.1097/DBP.0000000000000580

Lydon, S., Healy, O., O’Callaghan, O., Mulhern, T., & Holloway, J. (2015). A systematic review of the treatment of fears and phobias among children with autism spectrum disorders. Review Journal of Autism and Developmental Disorders, 2 (2), 141–154. https://doi.org/10.1007/s40489-014-0043-4

MacLean, C. H., Morton, S. C., Ofman, J. J., Roth, E. A., & Shekelle, P. G. (2003). How useful are unpublished data from the Food and Drug Administration in meta-analysis? Journal of Clinical Epidemiology, 56 (1), 44–51. https://doi.org/10.1016/s0895-4356(02)00520-6

Mahood, Q., Van Eerd, D., & Irvin, E. (2014). Searching for grey literature for systematic reviews: Challenges and benefits. Research Synthesis Methods, 5 (3), 221–234. https://doi.org/10.1002/jrsm.1106

Majid, U., Kim, C., Cako, A., & Gagliardi, A. R. (2018). Engaging stakeholders in the co-development of programs or interventions using intervention mapping: A scoping review. PLoS ONE, 13 (12), e0209826. https://doi.org/10.1371/journal.pone.0209826

Martin, J. L. R., Pérez, V., Sacristán, M., & Álvarez, E. (2005). Is grey literature essential for a better control of publication bias in psychiatry? An example from three meta-analyses of schizophrenia. European Psychiatry, 20 (8), 550–553. https://doi.org/10.1016/j.eurpsy.2005.03.011

Mason, D., Ingham, B., Urbanowicz, A., Michael, C., Birtles, H., Woodbury-Smith, M., Brown, T., James, I., Scarlett, C., & Nicolaidis, C. (2019). A systematic review of what barriers and facilitators prevent and enable physical healthcare services access for autistic adults. Journal of Autism and Developmental Disorders, 49 (8), 3387–3400. https://doi.org/10.1007/s10803-019-04049-2

Mazurek, M. O., Brown, R., Curran, A., & Sohl, K. (2017). ECHO autism. Clinical Pediatrics, 56 (3), 247–256. https://doi.org/10.1177/0009922816648288

Mazurek, M. O., Curran, A., Burnette, C., & Sohl, K. (2019). ECHO autism STAT: Accelerating early access to autism diagnosis. Journal of Autism & Developmental Disorders, 49 (1), 127–137. https://doi.org/10.1007/s10803-018-3696-5

McCormack, G., Dillon, A. C., Healy, O., Walsh, C., & Lydon, S. (2019). Primary care physicians’ knowledge of autism and evidence-based interventions for autism: A systematic review. Review Journal of Autism and Developmental Disorders, 7 , 226–241. https://doi.org/10.1007/s40489-019-00189-4

McGonigle, J., Migyanka, J., Glor-Scheib, S., Cramer, R., Fratangeli, J., Hegde, G., Shang, J., & Venkat, A. (2014). Development and evaluation of educational materials for pre-hospital and emergency department personnel on the care of patients with autism spectrum disorder. Journal of Autism & Developmental Disorders, 44 (5), 1252–1259. https://doi.org/10.1007/s10803-013-1962-0

McIntosh, C. E., Thomas, C. M., Allen, R. A., & Edwards, J. A. (2015). Using a combination of teaching and learning strategies and standardized patient for a successful autism simulation. Clinical Simulation in Nursing, 11 (3), 143–152. https://doi.org/10.1016/j.ecns.2014.11.008

Michie, S., Jochelson, K., Markham, W. A., & Bridle, C. (2009). Low-income groups and behaviour change interventions: A review of intervention content, effectiveness and theoretical frameworks. Journal of Epidemiology & Community Health, 63 (8), 610–622.

Mills, H. (2017). Evaluating the use of the ‘show me where’ picture communication system when conducting medical examinations on children with autistic spectrum disorder [Unpublished doctoral dissertation]. University of Cardiff.

Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & Group, P. (2009). Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement. PLoS Medicine, 6 (7), e1000097. https://doi.org/10.1371/journal.pmed.1000097

Mouridsen, S. E., Brønnum-Hansen, H., Rich, B., & Isager, T. (2008). Mortality and causes of death in autism spectrum disorders: An update. Autism, 12 (4), 403–414. https://doi.org/10.1177/1362361308091653

Muskat, B., Burnham Riosa, P., Nicholas, D. B., Roberts, W., Stoddart, K. P., & Zwaigenbaum, L. (2015). Autism comes to the hospital: The experiences of patients with autism spectrum disorder, their parents and health-care providers at two Canadian paediatric hospitals. Autism, 19 (4), 482–490. https://doi.org/10.1177/1362361314531341

Nguyẽ̂n, A. (2009). Environment and surroundings: How to make them autism-friendly . National Autistic Society.

Nicholas, D. B., Zwaigenbaum, L., Muskat, B., Craig, W. R., Newton, A. S., Kilmer, C., Greenblatt, A., Roberts, W., & Cohen-Silver, J. (2016). Experiences of emergency department care from the perspective of families in which a child has autism spectrum disorder. Social Work in Health Care, 55 (6), 409–426. https://doi.org/10.1080/00981389.2016.1178679

Nicolaidis, C., Kripke, C. C., & Raymaker, D. (2014). Primary care for adults on the autism spectrum. Medical Clinics of North America, 98 (5), 1169–1191. https://doi.org/10.1016/j.mcna.2014.06.011

Nicolaidis, C., Raymaker, D., McDonald, K., Dern, S., Boisclair, W. C., Ashkenazy, E., & Baggs, A. (2013). Comparison of healthcare experiences in autistic and non-autistic adults: A cross-sectional online survey facilitated by an academic-community partnership. Journal of General Internal Medicine, 28 (6), 761–769. https://doi.org/10.1007/s11606-012-2262-7

Nicolaidis, C., Raymaker, D., McDonald, K., Kapp, S., Weiner, M., Ashkenazy, E., Gerrity, M., Kripke, C., Platt, L., & Baggs, A. (2016). The development and evaluation of an online healthcare toolkit for autistic adults and their primary care providers. Journal of General Internal Medicine, 31 (10), 1180–1189. https://doi.org/10.1007/s11606-016-3763-6

O’Dea, A., O’Connor, P., & Keogh, I. (2014). A meta-analysis of the effectiveness of crew resource management training in acute care domains. Postgraduate Medical Journal, 90 (1070), 699–708. https://doi.org/10.1136/postgradmedj-2014-132800

O’Dowd, E., Lydon, S., O’Connor, P., Madden, C., & Byrne, D. (2019). A systematic review of 7 years of research on entrustable professional activities in graduate medical education, 2011–2018. Medical Education, 53 (3), 234–249. https://doi.org/10.1111/medu.13792

Pellicano, E., Dinsmore, A., & Charman, T. (2014). Views on researcher-community engagement in autism research in the United Kingdom: A mixed-methods study. PLoS ONE, 9 (10), e109946. https://doi.org/10.1371/journal.pone.0109946

Raymaker, D. M., McDonald, K. E., Ashkenazy, E., Gerrity, M., Baggs, A. M., Kripke, C., ... & Nicolaidis, C. (2017). Barriers to healthcare: Instrument development and comparison between autistic adults and adults with and without other disabilities. Autism, 21 (8), 972–984.

Reio, T. G., Rocco, T. S., Smith, D. H., & Chang, E. (2017). A critique of Kirkpatrick’s evaluation model. New Horizons in Adult Education and Human Resource Development, 29 (2), 35–53. https://doi.org/10.1002/nha3.20178

Riosa, P. B., Greenblatt, A., & Muskat, B. (2017). An online ASD learning module for pediatric health care professionals. Advances in Autism, 3 (3), 154–162. https://doi.org/10.1108/aia-03-2017-0007

Riviere, V., Becquet, M., Peltret, E., Facon, B., & Darcheville, J. C. (2011). Increasing compliance with medical examination requests directed to children with autism: Effects of a high-probability request procedure. Journal of Applied Behavior Analysis, 44 (1), 193–197. https://doi.org/10.1901/jaba.2011.44-193

Robison, J. E. (2019). Talking about autism—Thoughts for researchers. Autism Research, 12 (7), 1004–1006. https://doi.org/10.1002/aur.2119

Rogerson, J., Falkmer, M., Cuomo, B., Falkmer, T., Whitehouse, A. J. O., Granich, J., & Vaz, S. (2019). Parental experiences using the Therapy Outcomes by You (TOBY) application to deliver early intervention to their child with autism. Developmental Neurorehabilitation, 22 (4), 219–227. https://doi.org/10.1080/17518423.2018.1440259

Rydzewska, E., Hughes-McCormack, L. A., Gillberg, C., Henderson, A., MacIntyre, C., Rintoul, J., & Cooper, S.-A. (2019). General health of adults with autism spectrum disorders—A whole country population cross-sectional study. Research in Autism Spectrum Disorders, 60 , 59–66. https://doi.org/10.1016/j.rasd.2019.01.004

Sandelowski, M., Voils, C. I., Leeman, J., & Crandell, J. L. (2012). Mapping the mixed methods–mixed research synthesis terrain. Journal of Mixed Methods Research, 6 (4), 317–331. https://doi.org/10.1177/1558689811427913

Schiff, A., Tarbox, J., Lanagan, T., & Farag, P. (2011). Establishing compliance with liquid medication administration in a child with autism. Journal of Applied Behavior Analysis, 44 (2), 381–385. https://doi.org/10.1901/jaba.2011.44-381

Schwartz, I. S., & Baer, D. M. (1991). Social validity assessments: Is current practice state of the art? Journal of Applied Behavior Analysis, 24 (2), 189–204. https://doi.org/10.1901/jaba.1991.24-189

Scott, M., & Rawal, S. (2018). The promise of equity: A review of health equity research in high-impact quality improvement journals. American Journal of Medical Quality, 33 (3), 269–273. https://doi.org/10.1177/1062860617726854

Scruggs, T. E., & Mastropieri, M. A. (1998). Summarizing single-subject research: Issues and applications. Behavior Modification, 22 (3), 221–242. https://doi.org/10.1177/01454455980223001

Shabani, D. B., & Fisher, W. W. (2006). Stimulus fading and differential reinforcement for the treatment of needle phobia in a youth with autism. Journal of Applied Behavior Analysis, 39 (4), 449–452. https://doi.org/10.1901/jaba.2006.30-05

Shakes, P., & Cashin, A. (2019). Identifying language for people on the autism spectrum: A scoping review. Issues in Mental Health Nursing, 40 (4), 317–325. https://doi.org/10.1080/01612840.2018.1522400

Shavelle, R. M., Strauss, D. J., & Pickett, J. (2001). Causes of death in autism. Journal of Autism and Developmental Disorders, 31 (6), 569–576. https://doi.org/10.1023/A:1013247011483

Sirriyeh, R., Lawton, R., Gardner, P., & Armitage, G. (2012). Reviewing studies with diverse designs: The development and evaluation of a new tool. Journal of Evaluation in Clinical Practice, 18 (4), 746–752. https://doi.org/10.1111/j.1365-2753.2011.01662.x

Soong, C., & Shojania, K. G. (2020). Education as a low-value improvement intervention: Often necessary but rarely sufficient. BMJ Quality and Safety, 29 , 353–357.

Souders, M. C., Freeman, K. G., DePaul, D., & Levy, S. E. (2002). Caring for children and adolescents with autism who require challenging procedures. Pediatric Nursing, 28 (6), 555–583.

Swartz, J. S., Amos, K. E., Brindas, M., Girling, L. G., & Ruth Graham, M. (2017). Benefits of an individualized perioperative plan for children with autism spectrum disorder. Pediatric Anesthesia, 27 (8), 856–862. https://doi.org/10.1111/pan.13189

The Methods Group of the Campbell Collaboration (2016). Methodological expectations of Campbell Collaboration intervention reviews: Conduct standards. Campbell Policies and Guidelines Series, 3 . https://doi.org/10.4073/cpg.2016.3

Tregnago, M. K., & Cheak-Zamora, N. C. (2012). Systematic review of disparities in health care for individuals with autism spectrum disorders in the United States. Research in Autism Spectrum Disorders, 6 (3), 1023–1031. https://doi.org/10.1016/j.rasd.2012.01.005

Tsianakas, V., Maben, J., Wiseman, T., Robert, G., Richardson, A., Madden, P., Griffin, M., & Davies, E. A. (2012). Using patients’ experiences to identify priorities for quality improvement in breast cancer care: Patient narratives, surveys or both? BMC Health Services Research, 12 (1), 271. https://doi.org/10.1186/1472-6963-12-271

Venkat, A., Jauch, E., Russell, W. S., Crist, C. R., & Farrell, R. (2012). Care of the patient with an autism spectrum disorder by the general physician. Postgraduate Medical Journal, 88 (1042), 472–481. https://doi.org/10.1136/postgradmedj-2011-130727

Walsh, C., Lydon, S., Byrne, D., Madden, C., Fox, S., & O’Connor, P. (2018). The 100 most cited articles on healthcare simulation: A bibliometric review. Simulation in Healthcare, 13 (3), 211–220. https://doi.org/10.1097/SIH.0000000000000293

Walsh, C., Lydon, S., O’Dowd, E., & O’Connor, P. (2020a). Barriers to healthcare for persons with autism: A systematic review of the literature and development of a taxonomy. Developmental Neurorehabilitation, 23 , 413–430. https://doi.org/10.1080/17518423.2020.1716868

Walsh, C., Lydon, S., Geoghegan, R., Carey, C., Creed, M., O’Loughlin, L., Walsh, E., Byrne, D., O’Connor, P. (2021). Development and preliminary evaluation of a novel physician-report tool for assessing barriers to providing care to autistic patients. Manuscript under review.

Walsh, C., Lydon, S., Hehir, A., & O’Connor, P. (2020b). Development and evaluation of a novel caregiver-report tool to assess barriers to physical healthcare for persons on the autism spectrum. Research in Autism Spectrum Disorders, 79 , 101680. https://doi.org/10.1016/j.rasd.2020.101680

Warner, G., Parr, J. R., & Cusack, J. (2019). Workshop report: Establishing priority research areas to improve the physical health and well-being of autistic adults and older people. Autism in Adulthood, 1 (1), 20–26. https://doi.org/10.1089/aut.2018.0003

Whippey, A., Bernstein, L. M., O’Rourke, D., & Reddy, D. (2019). Enhanced perioperative management of children with autism: A pilot study. Canadian Journal of Anesthesia-Journal Canadien D Anesthesie, 66 (10), 1184–1193. https://doi.org/10.1007/s12630-019-01410-y

Whitehead, M. (1991). The concepts and principles of equity and health. Health Promotion International, 6 (3), 217–228. https://doi.org/10.2190/986L-LHQ6-2VTE-YRRN

Wilcock, P. M., Stewart Brown, G. C., Bateson, J., Carver, J., & Machin, S. (2003). Using patient stories to inspire quality improvement within the NHS Modernization Agency collaborative programmes. Journal of Clinical Nursing, 12 (3), 422–430. https://doi.org/10.1046/j.1365-2702.2003.00780.x

Wolf, M. M. (1978). Social validity: The case for subjective measurement or how applied behavior analysis is finding its heart. Journal of Applied Behavior Analysis, 11 (2), 203–214. https://doi.org/10.1901/jaba.1978.11-203

Wolff, J. J., & Symons, F. J. (2013). An evaluation of multi-component exposure treatment of needle phobia in an adult with autism and intellectual disability. Journal of Applied Research in Intellectual Disabilities, 26 (4), 344–348. https://doi.org/10.1111/jar.12002

Download references

Open Access funding provided by the IReL Consortium. Chloe Walsh’s doctoral research is funded by the Irish Research Council (Grant number: GOIPG/2017/1356).

Author information

Authors and affiliations.

Department of General Practice, School of Medicine, National University of Ireland Galway, Galway, Ireland

Chloe Walsh & Paul O’Connor

Irish Centre for Applied Patient Safety and Simulation, National University of Ireland Galway, Galway, Ireland

Chloe Walsh, Paul O’Connor, Ellen Walsh & Sinéad Lydon

School of Medicine, National University of Ireland Galway, Galway, Ireland

Sinéad Lydon

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Sinéad Lydon .

Ethics declarations

Competing interests.

The authors declare no competing interests.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Below is the link to the electronic supplementary material.

Supplementary file1 (DOCX 15 KB)

Supplementary file2 (xlsx 39 kb), supplementary file3 (docx 53.6 kb), rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Walsh, C., O’Connor, P., Walsh, E. et al. A Systematic Review of Interventions to Improve Healthcare Experiences and Access in Autism. Rev J Autism Dev Disord 10 , 185–202 (2023). https://doi.org/10.1007/s40489-021-00279-2

Download citation

Received : 24 September 2020

Accepted : 25 June 2021

Published : 26 August 2021

Issue Date : June 2023

DOI : https://doi.org/10.1007/s40489-021-00279-2

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Health equity
• Quality of care
• Find a journal
• Publish with us
• Track your research

Last updated 10th July 2024: Online ordering is currently unavailable due to technical issues. We apologise for any delays responding to customers while we resolve this. For further updates please visit our website https://www.cambridge.org/news-and-insights/technical-incident

We use cookies to distinguish you from other users and to provide you with a better experience on our websites. Close this message to accept cookies or find out how to manage your cookie settings .

Login Alert

• > Journals
• > BJPsych Advances
• > Volume 24 Issue 2
• > How to carry out a literature search for a systematic...

Article contents

• LEARNING OBJECTIVES
• DECLARATION OF INTEREST

Defining the clinical question

Scoping search, search strategy, sources to search, developing a search strategy, searching electronic databases, supplementary search techniques, obtaining unpublished literature, conclusions, how to carry out a literature search for a systematic review: a practical guide.

Published online by Cambridge University Press:  01 March 2018

Performing an effective literature search to obtain the best available evidence is the basis of any evidence-based discipline, in particular evidence-based medicine. However, with a vast and growing volume of published research available, searching the literature can be challenging. Even when journals are indexed in electronic databases, it can be difficult to identify all relevant studies without an effective search strategy. It is also important to search unpublished literature to reduce publication bias, which occurs from a tendency for authors and journals to preferentially publish statistically significant studies. This article is intended for clinicians and researchers who are approaching the field of evidence synthesis and would like to perform a literature search. It aims to provide advice on how to develop the search protocol and the strategy to identify the most relevant evidence for a given research or clinical question. It will also focus on how to search not only the published but also the unpublished literature using a number of online resources.

• Understand the purpose of conducting a literature search and its integral part of the literature review process

• Become aware of the range of sources that are available, including electronic databases of published data and trial registries to identify unpublished data

• Understand how to develop a search strategy and apply appropriate search terms to interrogate electronic databases or trial registries

A literature search is distinguished from, but integral to, a literature review. Literature reviews are conducted for the purpose of (a) locating information on a topic or identifying gaps in the literature for areas of future study, (b) synthesising conclusions in an area of ambiguity and (c) helping clinicians and researchers inform decision-making and practice guidelines. Literature reviews can be narrative or systematic, with narrative reviews aiming to provide a descriptive overview of selected literature, without undertaking a systematic literature search. By contrast, systematic reviews use explicit and replicable methods in order to retrieve all available literature pertaining to a specific topic to answer a defined question (Higgins Reference Higgins and Green 2011 ). Systematic reviews therefore require a priori strategies to search the literature, with predefined criteria for included and excluded studies that should be reported in full detail in a review protocol.

Performing an effective literature search to obtain the best available evidence is the basis of any evidence-based discipline, in particular evidence-based medicine (Sackett Reference Sackett 1997 ; McKeever Reference McKeever, Nguyen and Peterson 2015 ). However, with a vast and growing volume of published research available, searching the literature can be challenging. Even when journals are indexed in electronic databases, it can be difficult to identify all relevant studies without an effective search strategy (Hopewell Reference Hopewell, Clarke and Lefebvre 2007 ). In addition, unpublished data and ‘grey’ literature (informally published material such as conference abstracts) are now becoming more accessible to the public. It is important to search unpublished literature to reduce publication bias, which occurs because of a tendency for authors and journals to preferentially publish statistically significant studies (Dickersin Reference Dickersin and Min 1993 ). Efforts to locate unpublished and grey literature during the search process can help to reduce bias in the results of systematic reviews (Song Reference Song, Parekh and Hooper 2010 ). A paradigmatic example demonstrating the importance of capturing unpublished data is that of Turner et al ( Reference Turner, Matthews and Linardatos 2008 ), who showed that using only published data in their meta-analysis led to effect sizes for antidepressants that were one-third (32%) larger than effect sizes derived from combining both published and unpublished data. Such differences in findings from published and unpublished data can have real-life implications in clinical decision-making and treatment recommendation. In another relevant publication, Whittington et al ( Reference Whittington, Kendall and Fonagy 2004 ) compared the risks and benefits of selective serotonin reuptake inhibitors (SSRIs) in the treatment of depression in children. They found that published data suggested favourable risk–benefit profiles for SSRIs in this population, but the addition of unpublished data indicated that risk outweighed treatment benefits. The relative weight of drug efficacy to side-effects can be skewed if there has been a failure to search for, or include, unpublished data.

In this guide for clinicians and researchers on how to perform a literature search we use a working example about efficacy of an intervention for bipolar disorder to demonstrate the search techniques outlined. However, the overarching methods described are purposefully broad to make them accessible to all clinicians and researchers, regardless of their research or clinical question.

The review question will guide not only the search strategy, but also the conclusions that can be drawn from the review, as these will depend on which studies or other forms of evidence are included and excluded from the literature review. A narrow question will produce a narrow and precise search, perhaps resulting in too few studies on which to base a review, or be so focused that the results are not useful in wider clinical settings. Using an overly narrow search also increases the chances of missing important studies. A broad question may produce an imprecise search, with many false-positive search results. These search results may be too heterogeneous to evaluate in one review. Therefore from the outset, choices should be made about the remit of the review, which will in turn affect the search.

A number of frameworks can be used to break the review question into concepts. One such is the PICO (population, intervention, comparator and outcome) framework, developed to answer clinical questions such as the effectiveness of a clinical intervention (Richardson Reference Richardson, Wilson and Nishikawa 1995 ). It is noteworthy that ‘outcome’ concepts of the PICO framework are less often used in a search strategy as they are less well defined in the titles and abstracts of available literature (Higgins Reference Higgins and Green 2011 ). Although PICO is widely used, it is not a suitable framework for identifying key elements of all questions in the medical field, and minor adaptations are necessary to enable the structuring of different questions. Other frameworks exist that may be more appropriate for questions about health policy and management, such as ECLIPSE (expectation, client group, location, impact, professionals, service) (Wildridge Reference Wildridge and Bell 2002 ) or SPICE (setting, perspective, intervention, comparison, evaluation) for service evaluation (Booth Reference Booth 2006 ). A detailed overview of frameworks is provided in Davies ( Reference Davies 2011 ).

Before conducting a comprehensive literature search, a scoping search of the literature using just one or two databases (such as PubMed or MEDLINE) can provide valuable information as to how much literature for a given review question already exists. A scoping search may reveal whether systematic reviews have already been undertaken for a review question. Caution should be taken, however, as systematic reviews that may appear to ask the same question may have differing inclusion and exclusion criteria for studies included in the review. In addition, not all systematic reviews are of the same quality. If the original search strategy is of poor quality methodologically, original data are likely to have been missed and the search should not simply be updated (compare, for example, Naughton et al ( Reference Naughton, Clarke and O'Leary 2014 ) and Caddy et al ( Reference Caddy, Amit and McCloud 2015 ) on ketamine for treatment-resistant depression).

The first step in conducting a literature search should be to develop a search strategy. The search strategy should define how relevant literature will be identified. It should identify sources to be searched (list of databases and trial registries) and keywords used in the literature (list of keywords). The search strategy should be documented as an integral part of the systematic review protocol. Just as the rest of a well-conducted systematic review, the search strategy used needs to be explicit and detailed such that it could reproduced using the same methodology, with exactly the same results, or updated at a later time. This not only improves the reliability and accuracy of the review, but also means that if the review is replicated, the difference in reviewers should have little effect, as they will use an identical search strategy. The PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) statement was developed to standardise the reporting of systematic reviews (Moher Reference Moher, Liberati and Tetzlaff 2009 ). The PRISMA statement consists of a 27-item checklist to assess the quality of each element of a systematic review (items 6, 7 and 8 relate to the quality of literature searching) and also to guide authors when reporting their findings.

There are a number of databases that can be searched for literature, but the identification of relevant sources is dependent on the clinical or research question (different databases have different focuses, from more biology to more social science oriented) and the type of evidence that is sought (i.e. some databases report only randomised controlled trials).

• MEDLINE and Embase are the two main biomedical literature databases. MEDLINE contains more than 22 million references from more than 5600 journals worldwide. In addition, the MEDLINE In-Process & Other Non-Indexed Citations database holds references before they are published on MEDLINE. Embase has a strong coverage of drug and pharmaceutical research and provides over 30 million references from more than 8500 currently published journals, 2900 of which are not in MEDLINE. These two databases, however, are only available to either individual subscribers or through institutional access such as universities and hospitals. PubMed, developed by the National Center for Biotechnology Information of the US National Library of Medicine, provides access to a free version of MEDLINE and is accessible to researchers, clinicians and the public. PubMed comprises medical and biomedical literature indexed in MEDLINE, but provides additional access to life science journals and e-books.

In addition, there are a number of subject- and discipline-specific databases.

• PsycINFO covers a range of psychological, behavioural, social and health sciences research.

• The Cochrane Central Register of Controlled Trials (CENTRAL) hosts the most comprehensive source of randomised and quasi-randomised controlled trials. Although some of the evidence on this register is also included in Embase and MEDLINE, there are over 150 000 reports indexed from other sources, such as conference proceedings and trial registers, that would otherwise be less accessible (Dickersin Reference Dickersin, Manheimer and Wieland 2002 ).

• The Cumulative Index to Nursing and Allied Health Literature (CINAHL), British Nursing Index (BNI) and the British Nursing Database (formerly BNI with Full Text) are databases relevant to nursing, but they span literature across medical, allied health, community and health management journals.

• The Allied and Complementary Medicine Database (AMED) is a database specifically for alternative treatments in medicine.

The examples of specific databases given here are by no means exhaustive, but they are popular and likely to be used for literature searching in medicine, psychiatry and psychology. Website links for these databases are given in Box 1 , along with links to resources not mentioned above. Box 1 also provides a website link to a couple of video tutorials for searching electronic databases. Box 2 shows an example of the search sources chosen for a review of a pharmacological intervention of calcium channel antagonists in bipolar disorder, taken from a recent systematic review (Cipriani Reference Cipriani, Saunders and Attenburrow 2016a ).

BOX 1 Website links of search sources to obtain published and unpublished literature

Electronic databases

• MEDLINE/PubMed: www.ncbi.nlm.nih.gov/pubmed

• Embase: www.embase.com

• PsycINFO: www.apa.org/psycinfo

• Cochrane Central Register of Controlled Trials (CENTRAL): www.cochranelibrary.com

• Cumulative Index of Nursing and Allied Health Literature (CINAHL): www.cinahl.com

• British Nursing Index: www.bniplus.co.uk

• Allied and Complementary Medicine Database: https://www.ebsco.com/products/research-databases/amed-the-allied-and-complementary-medicine-database

Grey literature databases

• BIOSIS Previews (part of Thomson Reuters Web of Science): https://apps.webofknowledge.com

Trial registries

• ClinicalTrials.gov: www.clinicaltrials.gov

• Drugs@FDA: www.accessdata.fda.gov/scripts/cder/daf

• European Medicines Agency (EMA): www.ema.europa.eu

• World Health Organization International Clinical Trials Registry Platform (WHO ICTRP): www.who.int/ictrp

• GlaxoSmithKline Study Register: www.gsk-clinicalstudyregister.com

• Eli-Lilly clinical trial results: https://www.lilly.com/clinical-study-report-csr-synopses

Guides to further resources

• King's College London Library Services: http://libguides.kcl.ac.uk/ld.php?content_id=17678464

• Georgetown University Medical Center Dahlgren Memorial Library: https://dml.georgetown.edu/core

• University of Minnesota Biomedical Library: https://hsl.lib.umn.edu/biomed/help/nursing

Tutorial videos

• Searches in electronic databases: http://library.buffalo.edu/hsl/services/instruction/tutorials.html

• Using the Yale MeSH Analyzer tool: http://library.medicine.yale.edu/tutorials/1559

BOX 2 Example of search sources chosen for a review of calcium channel antagonists in bipolar disorder (Cipriani Reference Cipriani, Saunders and Attenburrow 2016a )

Electronic databases searched:

• MEDLINE In-Process and Other Non-Indexed Citations

For a comprehensive search of the literature it has been suggested that two or more electronic databases should be used (Suarez-Almazor Reference Suarez-Almazor, Belseck and Homik 2000 ). Suarez-Almazor and colleagues demonstrated that, in a search for controlled clinical trials (CCTs) for rheumatoid arthritis, osteoporosis and lower back pain, only 67% of available citations were found by both Embase and MEDLINE. Searching MEDLINE alone would have resulted in 25% of available CCTs being missed and searching Embase alone would have resulted in 15% of CCTs being missed. However, a balance between the sensitivity of a search (an attempt to retrieve all relevant literature in an extensive search) and the specificity of a search (an attempt to retrieve a more manageable number of relevant citations) is optimal. In addition, supplementing electronic database searches with unpublished literature searches (see ‘Obtaining unpublished literature’ below) is likely to reduce publication bias. The capacity of the individuals or review team is likely largely to determine the number of sources searched. In all cases, a clear rationale should be outlined in the review protocol for the sources chosen (the expertise of an information scientist is valuable in this process).

Important methodological considerations (such as study design) may also be included in the search strategy. Dependent on the databases and supplementary sources chosen, filters can be used to search the literature by study design (see ‘Searching electronic databases’). For instance, if the search strategy is confined to one study design term only (e.g. randomised controlled trial, RCT), only the articles labelled in this way will be selected. However, it is possible that in the database some RCTs are not labelled as such, so they will not be picked up by the filtered search. Filters can help reduce the number of references retrieved by the search, but using just one term is not 100% sensitive, especially if only one database is used (i.e. MEDLINE). It is important for systematic reviewers to know how reliable such a strategy can be and treat the results with caution.

Identifying search terms

Standardised search terms are thesaurus and indexing terms that are used by electronic databases as a convenient way to categorise articles, allowing for efficient searching. Individual database records may be assigned several different standardised search terms that describe the same or similar concepts (e.g. bipolar disorder, bipolar depression, manic–depressive psychosis, mania). This has the advantage that even if the original article did not use the standardised term, when the article is catalogued in a database it is allocated that term (Guaiana Reference Guaiana, Barbui and Cipriani 2010 ). For example, an older paper might refer to ‘manic depression’, but would be categorised under the term ‘bipolar disorder’ when catalogued in MEDLINE. These standardised search terms are called MeSH (medical subject headings) in MEDLINE and PubMed, and Emtree in Embase, and are organised in a hierarchal structure ( Fig. 1 ). In both MEDLINE and Embase an ‘explode’ command enables the database to search for a requested term, as well as specific related terms. Both narrow and broader search terms can be viewed and selected to be included in the search if appropriate to a topic. The Yale MeSH Analyzer tool ( mesh.med.yale.edu ) can be used to help identify potential terms and phrases to include in a search. It is also useful to understand why relevant articles may be missing from an initial search, as it produces a comparison grid of MeSH terms used to index each article (see Box 1 for a tutorial video link).

FIG 1 Search terms and hierarchical structure of MeSH (medical subject heading) in MEDLINE and PubMed.

In addition, MEDLINE also distinguishes between MeSH headings (MH) and publication type (PT) terms. Publication terms are less about the content of an article than about its type, specifying for example a review article, meta-analysis or RCT.

Both MeSH and Emtree have their own peculiarities, with variations in thesaurus and indexing terms. In addition, not all concepts are assigned standardised search terms, and not all databases use this method of indexing the literature. It is advisable to check the guidelines of selected databases before undertaking a search. In the absence of a MeSH heading for a particular term, free-text terms could be used.

Free-text terms are used in natural language and are not part of a database’s controlled vocabulary. Free-text terms can be used in addition to standardised search terms in order to identify as many relevant records as possible (Higgins Reference Higgins and Green 2011 ). Using free-text terms allows the reviewer to search using variations in language or spelling (e.g. hypomani* or mania* or manic* – see truncation and wildcard functions below and Fig. 2 ). A disadvantage of free-text terms is that they are only searched for in the title and abstracts of database records, and not in the full texts, meaning that when a free-text word is used only in the body of an article, it will not be retrieved in the search. Additionally, a number of specific considerations should be taken into account when selecting and using free-text terms:

• synonyms, related terms and alternative phrases (e.g. mood instability, affective instability, mood lability or emotion dysregulation)

• abbreviations or acronyms in medical and scientific research (e.g. magnetic resonance imaging or MRI)

• lay and medical terminology (e.g. high blood pressure or hypertension)

• brand and generic drug names (e.g. Prozac or fluoxetine)

• variants in spelling (e.g. UK English and American English: behaviour or behavior; paediatric or pediatric).

FIG 2 Example of a search strategy about bipolar disorder using MEDLINE (Cipriani Reference Cipriani, Saunders and Attenburrow 2016a ). The strategy follows the PICO framework and includes MeSH terms, free-text keywords and a number of other techniques, such as truncation, that have been outlined in this article. Numbers in bold give the number of citations retrieved by each search.

Truncation and wildcard functions can be used in most databases to capture variations in language:

• truncation allows the stem of a word that may have variant endings to be searched: for example, a search for depress* uses truncation to retrieve articles that mention both depression and depressive; truncation symbols may vary by database, but common symbols include: *, ! and #

• wild cards substitute one letter within a word to retrieve alternative spellings: for example, ‘wom?n’ would retrieve the terms ‘woman’ and ‘women’.

Combining search terms

Search terms should be combined in the search strategy using Boolean operators. Boolean operators allow standardised search terms and free-text terms to be combined. There are three main Boolean operators – AND, OR and NOT ( Fig. 3 ).

• OR – this operator is used to broaden a search, finding articles that contain at least one of the search terms within a concept. Sets of terms can be created for each concept, for example the population of interest: (bipolar disorder OR bipolar depression). Parentheses are used to build up search terms, with words within parentheses treated as a unit.

• AND – this can be used to join sets of concepts together, narrowing the retrieved literature to articles that contain all concepts, for example the population or condition of interest and the intervention to be evaluated: (bipolar disorder OR bipolar depression) AND calcium channel blockers. However, if at least one term from each set of concepts is not identified from the title or abstract of an article, this article will not be identified by the search strategy. It is worth mentioning here that some databases can run the search also across the full texts. For example, ScienceDirect and most publishing houses allow this kind of search, which is much more comprehensive than abstract or title searches only.

• NOT – this operator, used less often, can focus a search strategy so that it does not retrieve specific literature, for example human studies NOT animal studies. However, in certain cases the NOT operator can be too restrictive, for example if excluding male gender from a population, using ‘NOT male’ would also mean that any articles about both males and females are not obtained by the search.

FIG 3 Example of Boolean operator concepts (the resulting search is the light red shaded area).

The conventions of each database should be checked before undertaking a literature search, as functions and operators may differ slightly between them (Cipriani Reference Cipriani, Saunders and Attenburrow 2016b ). This is particularly relevant when using limits and filters. Figure 2 shows an example search strategy incorporating many of the concepts described above. The search strategy is taken from Cipriani et al ( Reference Cipriani, Zhou and Del Giovane 2016a ), but simplified to include only one intervention.

Search filters

A number of filters exist to focus a search, including language, date and study design or study focus filters. Language filters can restrict retrieval of articles to the English language, although if language is not an inclusion criterion it should not be restricted, to avoid language bias. Date filters can be used to restrict the search to literature from a specified period, for example if an intervention was only made available after a certain date. In addition, if good systematic reviews exist that are likely to capture all relevant literature (as advised by an information specialist), date restrictions can be used to search additional literature published after the date of that included in the systematic review. In the same way, date filters can be used to update a literature search since the last time it was conducted. Reviewing the literature should be a timely process (new and potentially relevant evidence is produced constantly) and updating the search is an important step, especially if collecting evidence to inform clinical decision-making, as publications in the field of medicine are increasing at an impressive rate (Barber Reference Barber, Corsi and Furukawa 2016 ). The filters chosen will depend on the research question and nature of evidence that is sought through the literature search and the guidelines of the individual database that is used.

• Google Scholar

Google Scholar allows basic Boolean operators to be used in strings of search terms. However, the search engine does not use standardised search terms that have been tagged as in traditional databases and therefore variations of keywords should always be searched. There are advantages and disadvantages to using a web search engine such as Google Scholar. Google Scholar searches the full text of an article for keywords and also searches a wider range of sources, such as conference proceedings and books, that are not found in traditional databases, making it a good resource to search for grey literature (Haddaway Reference Haddaway, Collins and Coughlin 2015 ). In addition, Google Scholar finds articles cited by other relevant articles produced in the search. However, variable retrieval of content (due to regular updating of Google algorithms and the individual's search history and location) means that search results are not necessarily reproducible and are therefore not in keeping with replicable search methods required by systematic reviews. Google Scholar alone has not been shown to retrieve more literature than other traditional databases discussed in this article and therefore should be used in addition to other sources (Bramer Reference Bramer, Giustini and Kramer 2016 ).

Citation searching

Once the search strategy has identified relevant literature, the reference lists in these sources can be searched. This is called citation searching or backward searching, and it can be used to see where particular research topics led others. This method is particularly useful if the search identifies systematic reviews or meta-analyses of a similar topic.

Conference abstracts

Conference abstracts are considered ‘grey literature’, i.e. literature that is not formally published in journals or books (Alberani Reference Alberani, De Castro Pietrangeli and Mazza 1990 ). Scherer and colleagues found that only 52.6% of all conference abstracts go on to full publication of results, and factors associated with publication were studies that had RCT designs and the reporting of positive or significant results (Scherer Reference Scherer, Langenberg and von Elm 2007 ). Therefore, failure to search relevant grey literature might miss certain data and bias the results of a review. Although conference abstracts are not indexed in most major electronic databases, they are available in databases such as BIOSIS Previews ( Box 1 ). However, as with many unpublished studies, these data did not undergo the peer review process that is often a tool for assessing and possibly improving the quality of the publication.

Searching trial registers and pharmaceutical websites

For reviews of trial interventions, a number of trial registers exist. ClinicalTrials.gov ( clinicaltrials.gov ) provides access to information on public and privately conducted clinical trials in humans. Results for both published and unpublished studies can be found for many trials on the register, in addition to information about studies that are ongoing. Searching each trial register requires a slightly different search strategy, but many of the basic principles described above still apply. Basic searches on ClinicialTrials.gov include searching by condition, specific drugs or interventions and these can be linked using Boolean operators: for example, (bipolar disorder OR manic depressive disorder) AND lithium. As mentioned above, parentheses can be used to build up search terms. More advanced searches allow one to specify further search fields such as the status of studies, study type and age of participants. The US Food and Drug Administration (FDA) hosts a database providing information about FDA-approved drugs, therapeutic products and devices ( www.fda.gov ). The database (with open access to anyone, not only in the USA) can be searched by the drug name, its active ingredient or its approval application number and, for most drugs approved in the past 20 years or so, a review of clinical trial results (some of which remain unpublished) used as evidence in the approval process is available. The European Medicines Agency (EMA) hosts a similar register for medicines developed for use in the European Union ( www.ema.europa.eu ). An internet search will show that many other national and international trial registers exist that, depending on the review question, may be relevant search sources. The World Health Organization International Clinical Trials Registry Platform (WHO ICTRP; www.who.int/ictrp ) provides access to a central database bringing a number of these national and international trial registers together. It can be searched in much the same way as ClinicalTrials.gov.

A number of pharmaceutical companies now share data from company-sponsored clinical trials. GlaxoSmithKline (GSK) is transparent in the sharing of its data from clinical studies and hosts its own clinical study register ( www.gsk-clinicalstudyregister.com ). Eli-Lilly provides clinical trial results both on its website ( www.lillytrialguide.com ) and in external registries. However, other pharmaceutical companies, such as Wyeth and Roche, divert users to clinical trial results in external registries. These registries include both published and previously unpublished studies. Searching techniques differ for each company and hand-searching through documents is often required to identify studies.

Communication with authors

Direct communication with authors of published papers could produce both additional data omitted from published studies and other unpublished studies. Contact details are usually available for the corresponding author of each paper. Although high-quality reviews do make efforts to obtain and include unpublished data, this does have potential disadvantages: the data may be incomplete and are likely not to have been peer-reviewed. It is also important to note that, although reviewers should make every effort to find unpublished data in an effort to minimise publication bias, there is still likely to remain a degree of this bias in the studies selected for a systematic review.

Developing a literature search strategy is a key part of the systematic review process, and the conclusions reached in a systematic review will depend on the quality of the evidence retrieved by the literature search. Sources should therefore be selected to minimise the possibility of bias, and supplementary search techniques should be used in addition to electronic database searching to ensure that an extensive review of the literature has been carried out. It is worth reminding that developing a search strategy should be an iterative and flexible process (Higgins Reference Higgins and Green 2011 ), and only by conducting a search oneself will one learn about the vast literature available and how best to capture it.

Acknowledgements

We thank Sarah Stockton for her help in drafting this article. Andrea Cipriani is supported by the NIHR Oxford cognitive health Clinical Research Facility.

Select the single best option for each question stem

a an explicit and replicable method used to retrieve all available literature pertaining to a specific topic to answer a defined question

b a descriptive overview of selected literature

c an initial impression of a topic which is understood more fully as a research study is conducted

d a method of gathering opinions of all clinicians or researchers in a given field

e a step-by-step process of identifying the earliest published literature through to the latest published literature.

a does not need to be specified in advance of a literature search

b does not need to be reported in a systematic literature review

c defines which sources of literature are to be searched, but not how a search is to be carried out

d defines how relevant literature will be identified and provides a basis for the search strategy

e provides a timeline for searching each electronic database or unpublished literature source.

a the Cochrane Central Register of Controlled Trials (CENTRAL)

d the Cumulative Index to Nursing and Allied Health Literature (CINAHL)

e the British Nursing Index.

a bipolar disorder OR treatment

b bipolar* OR treatment

c bipolar disorder AND treatment

d bipolar disorder NOT treatment

e (bipolar disorder) OR (treatment).

a publication bias

b funding bias

c language bias

d outcome reporting bias

e selection bias.

MCQ answers

1 a 2 d 3 b 4 c 5 a

FIG 2 Example of a search strategy about bipolar disorder using MEDLINE (Cipriani 2016a). The strategy follows the PICO framework and includes MeSH terms, free-text keywords and a number of other techniques, such as truncation, that have been outlined in this article. Numbers in bold give the number of citations retrieved by each search.

This article has been cited by the following publications. This list is generated based on data provided by Crossref .

View all Google Scholar citations for this article.

Save article to Kindle

To save this article to your Kindle, first ensure [email protected] is added to your Approved Personal Document E-mail List under your Personal Document Settings on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part of your Kindle email address below. Find out more about saving to your Kindle .

Note you can select to save to either the @free.kindle.com or @kindle.com variations. ‘@free.kindle.com’ emails are free but can only be saved to your device when it is connected to wi-fi. ‘@kindle.com’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.

Find out more about the Kindle Personal Document Service.

• Volume 24, Issue 2
• Lauren Z. Atkinson and Andrea Cipriani
• DOI: https://doi.org/10.1192/bja.2017.3

Save article to Dropbox

To save this article to your Dropbox account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you used this feature, you will be asked to authorise Cambridge Core to connect with your Dropbox account. Find out more about saving content to Dropbox .

Save article to Google Drive

To save this article to your Google Drive account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you used this feature, you will be asked to authorise Cambridge Core to connect with your Google Drive account. Find out more about saving content to Google Drive .

Reply to: Submit a response

- No HTML tags allowed - Web page URLs will display as text only - Lines and paragraphs break automatically - Attachments, images or tables are not permitted

Your details

Your email address will be used in order to notify you when your comment has been reviewed by the moderator and in case the author(s) of the article or the moderator need to contact you directly.

You have entered the maximum number of contributors

Conflicting interests.

Please list any fees and grants from, employment by, consultancy for, shared ownership in or any close relationship with, at any time over the preceding 36 months, any organisation whose interests may be affected by the publication of the response. Please also list any non-financial associations or interests (personal, professional, political, institutional, religious or other) that a reasonable reader would want to know about in relation to the submitted work. This pertains to all the authors of the piece, their spouses or partners.

A systematic literature review for authorization and access control: definitions, strategies and models

International Journal of Web Information Systems

ISSN : 1744-0084

Article publication date: 15 August 2022

Issue publication date: 25 October 2022

Authorization and access control have been a topic of research for several decades. However, existing definitions are inconsistent and even contradicting each other. Furthermore, there are numerous access control models and even more have recently evolved to conform with the challenging requirements of resource protection. That makes it hard to classify the models and decide for an appropriate one satisfying security needs. Therefore, this study aims to guide through the plenty of access control models in the current state of the art besides this opaque accumulation of terms meaning and how they are related.

Design/methodology/approach

This study follows the systematic literature review approach to investigate current research regarding access control models and illustrate the findings of the conducted review. To provide a detailed understanding of the topic, this study identified the need for an additional study on the terms related to the domain of authorization and access control.

The authors’ research results in this paper are the distinction between authorization and access control with respect to definition, strategies, and models in addition to the classification schema. This study provides a comprehensive overview of existing models and an analysis according to the proposed five classes of access control models.

Originality/value

Based on the authors’ definitions of authorization and access control along with their related terms, i.e. authorization strategy, model and policy as well as access control model and mechanism, this study gives an overview of authorization strategies and propose a classification of access control models providing examples for each category. In contrast to other comparative studies, this study discusses more access control models, including the conventional state-of-the-art models and novel ones. This study also summarizes each of the literature works after selecting the relevant ones focusing on the database system domain or providing a survey, a classification or evaluation criteria of access control models. Additionally, the introduced categories of models are analyzed with respect to various criteria that are partly selected from the standard access control system evaluation metrics by the National Institute of Standards and Technology.

• Authorization
• Access control
• Authorization strategy
• Access control model
• Classification

Mohamed, A.K.Y.S. , Auer, D. , Hofer, D. and Küng, J. (2022), "A systematic literature review for authorization and access control: definitions, strategies and models", International Journal of Web Information Systems , Vol. 18 No. 2/3, pp. 156-180. https://doi.org/10.1108/IJWIS-04-2022-0077

Emerald Publishing Limited

Copyright © 2022, Aya Khaled Youssef Sayed Mohamed, Dagmar Auer, Daniel Hofer and Josef Küng.

Published by Emerald Publishing Limited. This article is published under the Creative Commons Attribution (CC BY 4.0) licence. Anyone may reproduce, distribute, translate and create derivative works of this article (for both commercial and non-commercial purposes), subject to full attribution to the original publication and authors. The full terms of this licence may be seen at http://creativecommons.org/licences/by/4.0/legalcode

1. Introduction

Access control ensures data security by protecting assets and private information against unauthorized access by defined subjects. It helps to avoid information leaks or improper modification by potentially malicious parties. Besides traditional well-known access control models, there are many others that recently evolved to match advanced security requirements. Because of the increase of access control models, it seems promising to classify the models to enhance the selection of an appropriate model to fulfill the requirements of the overall system. Thus, it is necessary to clarify the core concepts of authorization and access control (e.g. definitions, strategies and models) along with the commonly used, partly ambiguous, synonyms.

definition of authorization and access control along with their related terms;

distinction between authorization strategies and access control models;

rough classification schema for access control models;

illustration of classification schema by providing state of the art as well as not commonly discussed models for each class of access control models;

review of a selected list of comparative studies on access control that are in the context of databases, include a survey of models, provide evaluation criteria and/or introduce a taxonomy of models; and

analysis of the classification schema based on selected criteria of access control models.

Concerning the methodology, we conduct a systematic literature review (SLR) which is a formal repeatable method to identify, analyze and interpret the existing research related to a particular topic of interest. According to the SLR definition in Kuhrmann et al. (2017) , we started our research with an extensive literature study on access control models. We selected a specific range of publications according to our filter criteria and studied them in detail. Because of the differences in the definition of authorization and access control along with their related terms, we discuss the various views for each concept and state which of them we follow. Then, we identified authorization strategies and derived categories for classifying all these models. Finally, we analyzed the resulting selections in addition to the main features of each category.

The remainder of this paper is organized as follows. Section 2 defines the related terms we use throughout this work. Section 3 explains authorization strategy and illustrates existing discretionary, mandatory and hybrid strategies. We introduce a classification of access control models along with examples in Section 4. We provide a summary of survey works comparing the included access control models in Section 5. In Section 6, we analyze the proposed categories with respect to selected criteria before concluding our paper in Section 7.

2. Related terms

Although authorization and access control have already been important in theory and practice for several decades, there are still differences concerning the understanding of basic terms in this domain. Therefore, we discuss the most important ones for our research.

Starting with authorization and access control, we see the following common differences. While Kane and Browne (2006) , some publications by the National Institute of Standards and Technology (NIST) such as Hu et al. (2014) and sources from IBM ( IBM-Corporation, 2015 ) use them as synonyms, Bertino et al. (2011) , Ferrari (2009) , Josang (2017) , Kizza (2020) and other NIST publications ( Ross et al. , 2021 ) clearly differentiate between them.

We follow ( Bertino et al. , 2011 ; Ferrari, 2009 ; Josang, 2017 ; Ross et al. , 2021 ; Kizza, 2020 ) to clearly distinguish between these two terms and discuss our view including associated concepts in the following. A brief overview is given in Figure 1 , which also shows how we assign the terms according to the two dimensions:

authorization and access control; and

strategy, model and instance level.

2.1 Authorization

While authorization not only refers to the result (authorizations, authorization policy), but also to the process of specifying access policies ( Josang, 2017 ; Kizza, 2020 ), most sources including ( Bertino et al. , 2011 ; Ferrari, 2009 ; Kane and Browne, 2006 ; Ross et al. , 2021 ; Hu et al. , 2014 ; Ahmad and Whitworth, 2011 ) focus on the result only.

We are aware that authorizations are usually developed in an iterative process of requirements analysis, definition and authorization specification at different levels of granularity and with respect to the access control model.

Definition 1 (Authorization). Is about the specification of access rights and combines authorization strategy as well as authorization model and authorization policy including their components (i.e. subject, object and action). It also considers the process of defining the authorization policy with respect to the selected model framed by its strategy.

In the following discussion and definitions, we focus on the artifacts in their final level of details. We start from the strategic level down to the instance level.

2.1.1 Authorization strategy.

Bertino (2016) uses the term access control technique to summarize discretionary (DAC) and mandatory access control (MAC) as the” fundamental building blocks”. Kizza (2020) uses the term authorization mechanism accordingly. In contrast, Eckert (2014) considers them as access control strategies, among which she includes role-based access control (RBAC). From our perspective, these fundamental viewpoints are strategic and not a matter of technique or mechanism. Therefore, we follow Eckert (2014) concerning the term strategy. However, as it is about specifying access rights, we follow Kizza (2020) and use the term authorization rather than access control like Eckert (2014) and Bertino (2016) . Furthermore, we do not consider RBAC a strategy, but an access control model (Section 4). Authorization strategies are discussed in Section 3 in more detail and further used in the analysis of the access control models along with their underlying authorization models in Section 6.

Definition 2 (Authorization Strategy). Defines the overall perspective of how to specify authorizations, i.e. owner-centric, administration-centric or hybrid and thus sets the frame for the authorization models.

2.1.2 Authorization model.

Subject is the active entity (e.g. user, group, organizational role, process, application program) to which access rights are granted. With a focus rather on the implementation, the term principal is also used as a synonym.

Object is the passive entity of the system, which needs to be protected, e.g. a file, a database table or record, an object in an object-oriented system, a node in a graph database. Different levels of granularity as well as sensitivity can be considered. Alternative terms for object are resource or asset.

Action states what the subject can perform on the object. There are several alternative terms for action such as privilege, (access) right, type of operations or activity, access mode and property. Privilege is the preferred term not only in the context of databases where privileges are granted to users ( Bertino et al. , 2011 ), but also in the context of basic discussions such as Ferrari (2009) and Josang (2017) . However, privileges are often used with slightly differing semantics ( Center, Computer Security Resource, 2022 ), which may lead to misconceptions. Atlam et al. (2020) even explicitly distinguish between action and privilege. While actions represent the types of activities subjects can perform on objects, the privileges are the permissions granted to a subject to be able to perform particular activities on certain objects. Thus, privileges are considered as synonyms to authorizations ( Josang, 2017 ).

Additional components are often demanded by specific access control models such as roles and sessions for RBAC, environmental attributes for ABAC or conflict of interest classes for the Chinese Wall model.

Definition 3 (Authorization Model). Defines the model for the specification of the access rights, i.e. the components needed and their interactions with respect to the core authorization strategy.

2.1.3 Authorization policy.

We have also seen differences with the term policy in terms of authorization and access control. Often authorization policy, access policy and access control policy are used synonymously. Authorization policies are considered on different levels of detail. For example, Hu (2016) describes access control policies as the “high-level requirements that specify how access is managed and who may access information under what circumstances”. Also Ferrari (2009) and Bertino et al. (2011) consider access control policies as the (high-level) rules to which access control must occur or authorizations are granted. The OASIS eXtensible Access Control Markup Language (XACML) technical committee provides an example with the OASIS standard ( OASIS, 2013 ), which shows an access control policy in plain text and the respective XACML policy in the XACML policy language. Thus, in this case it only differs in the level of granularity and language.

We follow the OASIS view in our definition of authorization policy considering the focus and the openness with regard to nearly any granularity of the specification. The authorization model defines all components and their required dependencies to specify the authorization policy. Thus, Figure 1 contains not only an edge from the authorization model to the set of components, but also from the authorization policy.

Definition 4 (Authorization Policy). Is an instance-level artifact, which specifies the access rights for a system according to the selected authorization model, on nearly any level of granularity, i.e. from simple text up to a policy definition language.

2.2 Access control

While authorization is about specifying the access rights, access control is about their enforcement ( Josang, 2017 ). Kizza (2020) describes this as “… a process to determine who does what to what based on a policy”. We follow these sources and consider access control as the process of enforcing access rights defined in an authorization policy.

2.2.1 Access control model.

The enforcement of access rights, which are defined according to some specific authorization model, is defined in the access control model. It states what needs to be done to determine the decision whether allow or deny access. Ferrari (2009) proposes the term authorization verification as a synonym to access control model. This view contrasts other sources such as Hu et al. (2017a ), who rely on a different definition of policy and further do not distinguish between authorization and access control model. They, for example, consider the access control model to “bridge the gap in abstraction between policy and mechanism” as their policies rather define the organizational frame for the access rights and not the access rights themselves.

Definition 5 (Access Control Model). Defines the enforcement of the authorization model, i.e. what needs to be checked to determine whether to allow or deny access for a subject to a protected resource.

2.2.2 Access control mechanism.

Is an instance-level artifact, i.e. a piece of software implementing a certain access control model ( Bertino et al. , 2011 ; Ferrari, 2009 ). When receiving a request (i.e. a subject requesting a specific kind of access on an object), the mechanism determines whether it can be allowed or must be denied ( Bertino et al. , 2011 ; Ferrari, 2009 ; Hu et al. , 2014 ). As it typically works as a reference monitor intercepting all requests to the system, thus, this term is also used as a synonym ( Bertino et al. , 2011 ; Ferrari, 2009 ; Samarati and De Capitani di Vimercati, 2001 ).

Definition 6 (Access Control Mechanism). Is an implementation of an access control model and thus, an instance-level artifact. It enforces an authorization policy which fits to the access control model of the mechanism. The mechanism determines if an access request evaluation is allowing or restricting the access.

Although there are numerous other technical terms with divergent definitions, the selection presented in this paper forms the basis for our further work.

3. Authorization strategies

An authorization strategy (Definition 2) defines the view point of describing authorization policies, i.e. owner-centric, administration-centric or hybrid. Different terms are used as alternatives to authorization strategy. Eckert (2014) refers to it as access control strategy, Samarati and De Capitani di Vimercati (2001) as well as Benantar (2005) called it access control policies class while others regard it as an access control model ( Bertino and Sandhu, 2005 ). Even though we use the term authorization strategy, we keep the well-established DAC (i.e. Discretionary Access Control) and MAC (i.e. Mandatory Access Control) abbreviations.

3.1 Discretionary strategy

The discretionary strategy (DAC) is owner-centric, i.e. ownership of each resource is assigned to one or more entities. The subject, who is allowed to access a resource, is either the object creator (i.e. the default owner) or a principal with delegated ownership rights. The resource can be only destroyed by the owner and its ownership may optionally be shared with other subjects as well ( Benantar, 2005 ).

DAC systems provide more flexibility to the user, but less administration control. Moreover, they do not scale well and are hard to manage in large environments. Because the propagation and usage of information cannot be controlled after giving access to the legitimate subjects, they are insecure and vulnerable to Trojan Horse attacks. A trojan horse program executes more actions, unknown to users, than it seems and should do ( Bertino et al. , 2011 ; Harris, 2012 ).

3.2 Mandatory strategy

The mandatory strategy (MAC) is non-discretionary because access decisions are not made at the discretion of the user. A MAC policy is obligatory as the access rights are regulated by a central authority. The owner and subject users can neither control the defined access nor override the policy. This strategy often is based on the security label concept where the subjects are associated to security clearance and objects to sensitivity classifications ( Hu et al. , 2017b ; Benantar, 2005 ).

Although MAC systems provide stronger security than the DAC ones and overcome the trojan horse problem, they are vulnerable to covert channels (i.e. tunnels created for transferring information in an unauthorized manner). Furthermore, the required administrative overhead makes it more costly.

3.3 Hybrid strategy

The advanced access control models are typically based on a middle ground strategy mixing DAC and MAC because the pure mandatory and discretionary strategies are often no longer sufficient. For instance, the originator-controlled strategy (ORCON or ORGCON) ( Abrams, 1995 ; Park and Sandhu, 2002 ) combines DAC and MAC such that only the originator (i.e. original owner) can alter the privileges on a subject/object basis ( Matt, 2018 ) (cp. DAC). On the other hand, access restrictions on original resources are automatically copied to derived objects without owner control (cp. MAC).

4. Access control models

An access control model defines the enforcement of the authorization model to decide whether to allow or deny access for a subject to a protected resource. We grouped the access control models into five main classes based on their characteristics. In the following subsections, we explain these categories and provide an overview of a selected subset of the access control models including some recent models that are not previously discussed in other surveys (Section 5).

4.1 Access control by explicit Object-Subject assignment

The oldest and simplest access control model is the access matrix (ACM) proposed by Lampson in 1971. It is built upon the strategy of DAC (i.e. identity-based) where the subjects’ privileges are described over the objects in a matrix data structure. A single entry in the matrix A[s,o] represents the access rights (i.e. actions) a subject s can take upon an object o ( Benantar, 2005 ). The access rights representation is straightforward and was commonly used in practice, but typically the matrix becomes sparse and oversized because of lots of empty cells. In the following, we give an overview of available access matrix model variants.

4.1.1 Authorization table.

Typically used in database management systems. The non-empty matrix authorization entries are stored as tuples in a table with three columns for the subject, object and action ( Petkovic and Jonker, 2007 ).

4.1.2 Access control list (ACL).

The most common and basic form of access control for limiting access to data on shared systems. It represents the access matrix in a column perspective (i.e. resource view) where the objects to be accessed are associated with a list of subjects along with the operations allowed to be executed on these objects ( Petkovic and Jonker, 2007 ).

4.1.3 Capability list.

The conceptual approach is similar to ACL, but with the access matrix stored by row (i.e. subject view). Each subject holds a list of capability certificates containing the access rights to be performed by this principal over a set of resources ( Petkovic and Jonker, 2007 ).

4.2 Access control by model-specific rules

Traditionally, this class of models has been used in MAC systems enforcing the concept of rules. A set of predefined rules must be met to grant/deny the subject access to a particular resource. The models in this category have fixed rules that apply all the time for all users regardless of their identity. The rules are an implicit part of the access control model specifying detailed situations, i.e. whether a given subject can or cannot access an object and what that subject can do once access is granted. For example, the subject’s security level determines the classes of objects to be accessed in the Bell-LaPadula (BLP) and Biba models. Administrators can only manage the basic parameters (e.g. security level) whereas users have no control at all on the rules. In the following, we give an overview of some model-specific rule examples.

4.2.1 Bell-LaPadula (BLP).

Simple security : also known as the no-read-up (i.e. read-down) property such that a subject is not allowed to read objects with higher sensitivity. The subject security clearance must dominate the object security classification.

Star property : also known as the no-write-down (i.e. write-up) policy where it is not possible for a subject with some security level to write any object with lower sensitivity. To avoid the leakage of confidential information, the object security classification has to dominate the subject security clearance.

Strong star property (optional) : read and write operations are performed at a single security level such that the subject and object sensitivity are equal.

4.2.2 Biba.

Simple integrity : read-up rule controls a subject’s access from reading lower integrity level data, so that bad information will not flow upwards from lower clearance levels.

Star integrity : also known as write-down such that subjects are not allowed to write data or pass information to higher classified levels than theirs.

Invocation property : a service can only be invoked by subjects at a lower integrity level.

4.2.3 Lipner.

Based on the BLP: subjects and objects are assigned to one of the two confidentiality levels. In this case, five defined compartments are responsible for integrity and access control.

Full Model: is a hybrid combination of the BLP and Biba integrity models. Three integrity levels and two categories are added to Lipner’s first mechanism, after collapsing some confidentiality compartments, to be assigned to subjects and objects. This is to prevent low-integrity data or programs from impacting those with higher integrity. The purpose of integrity levels is to avoid unauthorized modification of system programs whereas the categories are used to separate domains according to functional areas.

4.2.4 Clark–Wilson.

Subject : (ER3): is an authenticated user who attempts to initiate a transformation procedure (TP). (ER4): Only the certifier of a TP can change the list of entities associated with that TP to prevent violating the integrity constraints by changing the qualifications of a TP.

Object : is either classified as a constrained data item (CDI) with high protection level or unconstrained data item (UDI) representing untrusted information entered to the system. (ER2): The system must associate a user with each TP and set of CDIs. (CR1): The validity of CDIs is ensured by integrity verification procedures (IVPs).

Transformation procedure (TP) : a set of operations performed on data items. (CR2): It transforms CDIs in the system from one valid state to another. (CR5): The TP can also take an UDI as input and either produce a CDI or reject the UDI. (ER1): Only TPs certified to run on a CDI can manipulate it so that the certified relations are maintained. (CR3): The relations allowed by the system must enforce the separation of duty principle. (CR4): Transactions are logged using a CDI and the TP only appends to it.

4.2.5 Chinese-Wall (CW).

Simple security : the object can be accessed by a specific subject if it belongs to either the same company dataset of the previously accessed objects or a different conflict of interest class.

Star property : for the write access, the simple security rule must be satisfied besides the permission to read the objects which are sanitized (i.e. filtered from sensitive data) and belong to the same company dataset as the one for which write access is requested.

4.2.6 Multi-level security (MLS) database.

The MLS database model ( Keefe et al. , 1993 ) follows the MAC authorization strategy and extends the concept of the BLP model to apply fine-grained access control to database systems at the level of relations (i.e. tables), attributes (i.e. columns), tuples (i.e. rows) and elements (i.e. cells). This is done by regulating access to data resources by subjects according to their predefined classification in the system. The classification is based on a partially ordered set of access classes (i.e. labels) such that an access class c i dominates an access class c j if and only if the security level of c i is greater than or equal to that of c j ( Bertino and Sandhu, 2005 ). Having a classification associated to a value represents the sensitivity of that attribute value for a particular entity not the value itself. For example, a classification (e.g. Secret ) associated to a salary value is not for this absolute value, but rather the salary of the given employee. There are three MLS architectures: kernalized architecture , distributed architecture and trusted subject architecture . They are classified according to whether access control is enforced by the database management system (DBMS) or delegated to a trusted operating system ( Rjaibi, 2004 ).

However, the MLS database introduces complications in real-world cases because of polyinstantiation . This problem arises when there are multiple instances of the same entity with different access classes in the system. Possible options are either notify the subject or accept the change replacing the existing value. The first solution compromises secrecy because of revealing protected information causing a covert channel while the overwriting approach compromises integrity because high classified data would be lost. Because both solutions are not viable, the only applicable option is to have the original and the new tuples coexist and manage their presence. Thus, polyinstantiated tuples result and the database loses its semantics after executing few operations. Accordingly, current DBMSs do not support element-level classification, but rather on the tuple level (e.g. Trusted Oracle, DB2 for z/OS and SYBASE Secure SQL Server) ( Samarati and De Capitani di Vimercati, 2001 ).

4.3 Access control by roles

Flat RBAC : applies basic RBAC, but considers many-to-many relations between users and roles such that a user can have many roles and vice versa. The same applies to the permission-role assignment.

Hierarchical RBAC : organizational and administrative roles are defined in a general or limited hierarchy (i.e. tree) for structuring authorities and responsibilities within the organization. The hierarchies are reflexive and transitive, but anti-symmetric.

Constrained RBAC : adds constraints associated with the user-role assignment relations and/or role activation within user sessions to the hierarchical RBAC. The separation of duty (SoD) concept is applied to prevent the users from being over-authorized and enforce conflict of interest policies. The SoD can be static (SSoD) or dynamic (DSoD). In SSoD, the user cannot be a member of roles having shared principles. However, this is allowed in the DSoD without activating these exclusive roles at the same time even across multiple simultaneous sessions initiated by the same subject.

Besides the standard RBAC, advanced models are proposed to structure the RBAC model and manage its policies. Moreover, several works extended the capabilities of RBAC to deal with contextual information.

4.3.1 Administration role-based access control.

This model uses roles as a central concept, but dedicated to the management of policies in RBAC. Sandhu et al. (1999) proposed the first RBAC administration model called ARBAC97 . Administrative roles and permissions are independent of the regular ones. ARBAC97 is decentralized, but without compromising the broad policy objectives. It has three components to deal with different RBAC administration aspects: URA97 for user-role assignment, PRA97 for permission-role assignment and RRA97 for role-role assignment. URA97 and PRA97 are based on a ternary relation (i.e. can_assign ) with prerequisite conditions. For instance, an administration role member can assign a user to a regular role in URA97 only if this user satisfies the prerequisite role(s) condition. URA97 and PRA97 control user-role and permission-role weak and strong revocation by means of a relation called can_revoke without involving prerequisite conditions. Weak revocation applies only to explicit membership in a single role whereas strong revocation cascades upwards in the role hierarchy, however, both apply downward cascading. Last but not least, the role hierarchy is constructed in the RRA97 sub-model.

Additionally, two extensions (i.e. ARBAC99 and ARBAC02 ) have been proposed by the same research group to address the shortcomings in ARBAC97. The objective of ARBAC99 is to manage mobile and immobile users (e.g. visitor or consultant) and permissions. ARBAC02 focuses on resolving the multi-step user assignment, duplicated permission-role assignment (PA) information and restricted composition of permission pools. This is done by creating an organization structure as a user and permission pools independent of the role hierarchy in addition to introducing a bottom-up permission–role administration unlike the top-down approach in the ARBAC97 model. On the other hand, scoped administration of role-based access control (SARBAC) is proposed by Crampton and Loizon (2002) as an extension of RRA97 and an alternative to ARBAC97 ( Cuppens and Miège, 2003 ). It relies on administrative scope which dynamically changes upon changing the role hierarchy. This improves rule consistency when deleting roles. Finally, administrative roles are not separated from the regular ones anymore.

4.3.2 Organization-based access control (OrBAC).

The OrBAC model has evolved because of the need to structure a given organization into sub-organizations and specify their different authorization policies within one framework. It is based on the concept of organization as an organized group of subjects such that each of them plays some role in the organization. In OrBAC, policies associated with different organizations can be handled simultaneously ( Kalam et al. , 2003 ).

Permissions are typically applied directly to subjects, actions and objects while in OrBAC, subject, action and object are respectively abstracted into role, activity and view. The subject in this context is either an active entity (i.e. user) or an organization. A role acts as the link between subjects and organizations in a ternary relationship called Employ . A view corresponds to a set of objects satisfying a common property whereas an activity joins actions that partake of the same principles ( Kalam et al. , 2003 ). As organizations can define views differently, the object, view and organization entities are related to each other via the Use ternary relationship. The same applies to the action, activity and organization using another ternary relationship, i.e. Consider .

An authorization policy is specified as a set of facts, i.e. in an organization, a given role is permitted to perform a given activity on a given view in a given context (e.g. working hours, night and urgency) ( Cuppens and Miège, 2003 ). In addition to permissions, obligations and prohibitions can be specified using OrBAC. It is assumed that any organization is a subject ( Org ⊆ S ), any subject is an object ( S ⊆ O ) and entity attributes are represented as functions, e.g. the name of a subject s is represented as name(s) . Furthermore, an administration model for OrBAC (AdOr-BAC) is proposed by Cuppens and Miège (2003) . The AdOr-BAC model includes the URA and PRA components as in the ARBAC model, but has an additional component called UPA (i.e. user-permission administration). The two variations of the UPA component are UPA1 and UPA2 for enabling a user to delegate a permission to perform an action on an object and an activity on a view respectively.

4.3.3 Role-based access control extensions.

Several RBAC models are proposed to consider context information for access control decisions. For instance, temporal RBAC (T-RBAC) model extends RBAC such that users are limited to only use the role permissions in specific temporal periods. Depending on the specified time interval(s), the roles are either in an active or inactive state. Furthermore, role triggers are supported for controlling the time of action execution. The priority resolves the conflicts between triggers and periodic activation/deactivation ( Bertino et al. , 2011 ). The language is formally defined and checked for inconsistencies or ambiguities in ( Bertino et al. , 2000 ).

Another extension is GEO-RBAC ( Damiani et al. , 2007 ) that evolved because of the increasing need for securing mobile applications and location-based services. Spatial capabilities are added to the conventional RBAC model to support location-specific constraints in which a given role can be accessed by a user. The location can be physically or logically expressed in terms of absolute coordinates or relative to spatial objects respectively. In this model, the role is only enabled if the user is located within the spatial boundary of that role ( Bertino et al. , 2011 ).

Tie-RBAC ( Tapiador et al. , 2012 ) extends RBAC to be applied in social networks. It gives full control to the resource owner by allowing users to define their social circle (i.e. contacts) and establish in-between relations to grant access. Thus, the users control which requestor has access to their resources. The access control policies for all users are stored and enforced by a central server.

new elements to model historical as well as deleted information;

analyzing functions; and

historical constraints such as historical separation of duty (HSD) to deny undesired requests according to the previous and current information.

To sum up, the models in the role-based access control category have many forms and extensions, but all of them are based on the concept of roles which are associated to access permissions and assigned to users.

4.4 Access control by content

This category applies the concept of comprehensive data protection where access control decisions are based on data content (e.g. attribute values) ( Zeng et al. , 2014 ). Besides the flexible policy definition, authorizations are dynamically granted and revoked ( Bertino et al. , 2011 ). In content-based models, the policies are only applicable to the users satisfying specific criteria according to the rules defined by users or administrators. On the other hand, the model-specific rules category has static rules that can neither be modified nor controlled by administrators. Selected content-based models are explained in the rest of this section.

4.4.1 Attribute-based access control.

The ABAC model overcomes the limitations of other models concerning long-term maintenance as well as representing complex access control requirements. In ABAC, a given subject can have access to a wide range of objects without specifying individual relationships to each resource. Authorization policies are specified in terms of the subject, resource and environment condition (e.g. time and location) attributes. The access decision is determined by evaluating the attribute values of the applicable policy (or policies). The ACL and RBAC models are even considered as a special case of ABAC using an attribute for the identity and role respectively ( Hu et al. , 2014 ).

Although there are several proposed ABAC policy models, the eXtensible access control markup language (XACML) ( OASIS, 2013 ) has become the defacto standard not only in specifying ABAC policies, but also enforcing them in a multi-step authorization process using XACML’s reference architecture ( Hu et al. , 2017b ; Ferraiolo et al. , 2016 ). The second commonly used approach is based on next generation access control (NGAC) ( Council , Information Technology Industry, 2018, 2016 ) and its functional architecture.

4.4.2 View-based access control (VBAC).

This model is specific to databases. Theoretically, a view is a static typed language construct while from the technical perspective, it is a virtual table having rows and columns defined by a query based on the database tables, but without physical storage. The access control policy is based on a set of predefined interfaces (i.e. views).

the user is not permitted to access the view; or

the operation to be performed on the object is explicitly denied within one of the views to which the principal or the relevant role is associated ( Bertino et al. , 2011 ; Sahafizadeh and Parsa, 2010 ).

New data that satisfy a given policy will be automatically included in the view result. However, new views are created upon modifications to access control policies and their number further increase because users have different permissions. The Oracle VPD mechanism ( Browder and Davidson, 2002 ) addresses some of these problems where the queries are initially written against the base tables and then, automatically rewritten by the system against the view available to the subject user.

4.4.3 Relation-based access control.

The ReBAC model ( Fong, 2011 ; Gates, 2007 ) addresses the limitations of ABAC to deal with the interpersonal relationships between users in expressing authorization policies. The access control decision is based on the type, depth and trust level of the relationship between the owner and access requester of the resource. This model has been typically applied in social networks focusing on the privacy of end users ( Hu et al. , 2012 ). A policy language based on modal logic and an access control model formulated as a state transition system are introduced in ( Fong, 2011 ) for specifying and enforcing complex relations (e.g. friends-of-friends). However, ReBAC supports neither fine-grained access control at attribute level nor entities other than subjects and objects.

4.4.4 Entity-based access control (EBAC).

This model takes into account both attributes and relationships in policy evaluation using the concept of entities. The EBAC model addresses the expressiveness limitations of ABAC and ReBAC such that the relationships between entities can be navigated reasoning about these entities along paths of arbitrary length by comparing their corresponding properties. In EBAC, an entity-relationship (ER) model and logical expressions, including logical operators (e.g. or and, not ) and quantifiers (i.e. ∀ and ∃), are introduced into the policy expressions as a generalization to ABAC. The ER-model describes the entities along with their properties and relationships for a particular application which is then represented in an entity graph. This is a directed multi-labelled graph mapping the entities and relationship types to vertices and edges respectively. Authorization policies are specified in terms of the entity model which is then instantiated for evaluating attribute values of the relevant entities (i.e. subject, object, action and environment). An authorization system called Auctoritas provides a policy language and an evaluation engine for EBAC ( Bogaerts et al. , 2015 ). However, this model is neither popular nor commonly used like the other conventional access control models.

4.5 Access control by context

The access decision is not only relying on the policy in terms of subject and resource, but also contextual parameters, such as the sequence of events preceding the access attempt (i.e. history), location, time and sequence of responses, are taken into account. The permission to access resources is dependent on these contextual information, unlike the content-based access control, which makes access decisions according to the data values. The final decision is based on the result of reviewing the situation ( Harris, 2012 ). The models in this class, as in the following, are often used as a complement to conventional access control models.

4.5.1 Emotion-based access control.

A system could be in danger when an angry user is granted access despite being an authorized subject. The opposite scenario is also valid as there could be unauthorized individuals who need access urgently to save the system from risky incidents. Hence, this model introduces the concept of sensibility to access control systems instead of relying on the authorization component only.

The emotion factor (i.e. feelings of the person trying to access the protected resources) can be used as a complement to the existing access control mechanisms. Firstly, the spontaneous brain signals are recorded from the scalp of the requesting user in the sensing layer. This is primarily a hardware component called Emotiv EPOC headset which collects the EEG signals and transmits them to a listener module. The received data is then analyzed in a signal processing module where the emotions are classified into positive or negative. According to the emotion level, the decision maker determines whether to allow access to the requested resource or not ( Almehmadi and El-Khatib, 2013 ). Although the emotion detection technology is a novel method in access control, it is still an ongoing research and not commonly used in practice.

4.5.2 Risk-based access control.

This model, also referred to as risk-adaptive access control (RAdAC ) , originated from the need of the enterprise to real-time assessment of the current situation and possible risks even when the subjects lack proper permissions. A possible strategy is to deny the access in this case, however, emergency data access is crucial in some domains (e.g. healthcare and military). Hence, this model introduces risk levels into the process of access decision such that the access is determined by computing the security risk and operational need (e.g. subject trustworthiness, information sensitivity and history events) instead of only using the rigid policies which provide the same decision in different circumstances. After the risky access event, the system will take some mitigating actions for minimizing possible information disclosure in the future. Several methods for estimating access risks are proposed by various works including machine learning ( Molloy et al. , 2012 ), probability theory ( Rajbhandari and Snekkenes, 2010 ) and fuzzy logic ( Cheng et al. , 2007 and Ni et al. , 2010 ). The work of Atlam et al. (2020) provides a survey of the state-of-the-art risk-based access control model along with the existing risk estimation techniques (Section 5).

4.5.3 Sequence-based access control (SeqBAC).

The SeqBAC model is used as a complement to existing access control models to restrict the order of actions performed on databases by legitimate users, e.g. read or write data. It is based on the fact that actions are not totally independent. In some cases, they are used to collect values for the subsequent ones. Thus, it is crucial to audit the sequences in which actions are executed to avoid bypassing the dependency logic between actions. Although this can be handled in the application or even using stored procedures, they are hard to manage and prone to implementation mistakes that can compromise the system correctness. Furthermore, the model allows more flexibility in terms of reusing and branching the sequences of actions which can be validated automatically in real-time. SeqBAC is based on a CRUD expression driven access control model where the policy defines a set of authorized users and a set of actions along with their input parameters and directed transition relations between them (i.e. action flowchart). The model concept and formal definition are proposed in Regateiro et al. (2018) , however, it is still in the research phase, i.e. no actual implementation is provided.

4.5.4 History-based access control (HBAC).

This model aims to maintain a selective history of the access requests initiated by individual subjects and use this history to identify potentially malicious requests. It protects software execution environments (e.g. operating systems and mobile code platforms) against potential damages caused by codes with inappropriate behavior. For example, a previously killed malicious program can be denied from execution on a specific machine by specifying a HBAC policy that keeps track of aborted programs identity along with their past termination events ( Edjlali et al. , 1998 ).

History information is a sequence of events which are an abstraction of security-relevant activities (e.g. opening a socket connection and reading/writing a file) ( Bartoletti et al. , 2005 ). Based on the sequence of requests, each program is dynamically classified to one of the defined equivalence classes and only access the allowed resources associated to the programs of that class. Authorization decisions depend on the real-time evaluation of access history of the inquiring party, e.g. behavior, time between requests, and content of requests. Execution monitors are the typical run-time mechanisms for enforcing history-based policies. They are responsible for observing computations and terminating them upon violating the defined authorization policy. HBAC has been the focus of many researches on the conceptual ( Banerjee and Naumann, 2004 ; Fong, 2004 ) and implementation ( Abadi and Fournet, 2003 ; Edjlali et al. , 1998 ) levels.

5. Comparative Studies

In this section, we review a selected list of access control model literature studies. The related works are sorted ascendingly by their publishing year. The older surveys focus on data security whereas the newer ones deal with access control in specific domains, e.g. cloud computing, social networks and internet of things (IoT). In the following, we summarize each of the survey works and map the presented access control models into our classification categories.

Access control: principle and practice ( Sandhu and Samarati, 1994 ) . This work is one of the earliest works in the area of access control. It provided a concrete explanation for authentication, administration (i.e. centralized, hierarchical, cooperative, ownership and decentralized), access control and auditing in addition to how they are related to each other. The difference between policy and mechanism is also illustrated. The DAC and MAC strategies are explained along with the access matrix including its implementation approaches (i.e. ACL, capabilities and authorization relations). They had a different perspective regarding RBAC because of being relatively recent at the time of publishing this work. They considered the role-based approach as an alternative to traditional DAC and MAC policies with several advantages, e.g. authorization management, hierarchical roles, least privilege, separation of duties and object classes.

Data security ( Bertino, 1998 ). In this paper, Bertino surveyed the state of the art in access control for database systems and outlined the main research issues. The System R ( Astrahan et al. , 1976 ) access control is discussed as the basic DAC model for protecting tables and views with specific access modes (i.e. select, insert, update and delete) in addition to the existing extensions for supporting negative authorizations ( Bertino et al. , 1997 ), non-cascading revoke and temporal duration of authorizations ( Bertino et al. , 1996a , 1996b ). Moreover, RBAC as an extension to access control models is described as well as how MAC strategy is applied in databases enforcing the BLP principles and multilevel relational model using views. Finally, the research directions of access control for database systems are addressed with respect to data protection against intrusions (e.g. trojan horses and covert channels) besides developing authorization and access control models for advanced data management systems.

Access control: policies, models and mechanisms ( Samarati and De Capitani di Vimercati, 2001 ). One of the earliest literature reviews providing definitions for security policy, model and mechanism. They clarified the basic concepts and explained the access control models along with the current implementations in the context of MAC, DAC and RBAC categories. Some models belong to only one category while others are hybrid. For instance, the Access Matrix is a DAC model whereas the Chinese Wall combines DAC with MAC policies. We partly relied on this classification, especially for the DAC and MAC strategies and the basic state-of-the-art access control models.

Database security – concepts, approaches and challenges ( Bertino and Sandhu, 2005 ). In 2005, Bertino and Sandhu discussed database security with focus on confidentiality and integrity. They gave an overview of the System R model along with its extensions in the context of DAC, BLP principles as well as the MLS database model for applying MAC policies, RBAC models and content-based access control using views to enforce fine-grained authorization policies. Further, requirements and features for XML and object-based database systems are presented.

support for fine/coarse grained specification;

evaluation using conditions;

least privilege;

support for single/multiple policy types;

information used for making authorization decisions;

use of application-specific information only while processing the client request;

enterprise-wide consistent access control policy enforcement; and

support for changes.

Microsoft’s SQL Server 2008 DBMS with its base authorization model and access control administration features;

the Oracle Virtual Private Database (VPD) technology ( Browder and Davidson, 2002 ) for controlling database access at the level of columns and rows; and

Oracle Label security mechanism implementing the strategy of MAC.

Last but not least, they gave an overview of access control models for object databases, XML data, Geographical data and digital libraries.

Database security and access control models: a brief overview ( Kriti, 2013 ). The work of Kriti discussed the access control models in the context of databases presenting security threats and policy requirements as a motivation. An overview of the security models basic terms (i.e. subjects, objects, access modes, policies, authorizations, administrative rights and axioms) as well as the access control principles of administration (i.e. centralized vs decentralized), system (i.e. open vs closed) and privilege (i.e. minimum vs maximum) are provided. The DAC, MAC and RBAC are explained in addition to how they are applied in databases along with their vulnerabilities. For instance, the DAC authorization is applied in databases using System R model and its extensions, but vulnerable to trojan horse attacks.

Taxonomy and classification of access control models for cloud environments ( Majumder et al ., 2014 ). The authors classified various existing access control models according to a proposed taxonomy of access control schemes for cloud environments. They discussed the access control challenges in cloud computing regarding cost, granularity, data loss, taking the data sensitivity into account, data theft by malicious users and accessing data from an outside server. Furthermore, they explained 11 models ( Table 1 ) and analyzed them based on:

identity vs nonidentity in terms of whether the model is tree-structured or not; and

centralized (i.e. per user, group users and all users) vs collaborative.

Different access control mechanisms ( Sifou et al ., 2017 ). This work analyzed and compared different access control models in the context of cloud computing. Based on the National Institute of Standards and Technology’s (NIST) view in ( Mell et al. , 2011 ), the authors illustrated the main features of cloud computing service and deployment models. They demonstrated DAC, MAC, RBAC, ABAC and OrBAC along with the advantages and disadvantages of each access control model. According to the current cloud computing requirements, they defined nine criteria to evaluate the current access control models: dynamicity, flexibility, reliability, ease of administration, security policy implementation, global management, support scalability, computational costs and fine-grained access.

Survey on access control mechanisms in cloud computing ( Karatas and Akbulut, 2018 ). The work of Karatas and Abkulut provided a survey of access control approaches and works related to cloud computing. They reviewed 109 research papers in that domain throughout the past decade. They provided not only a comparative explanation for the existing access control models, but also a unique evaluation using NIST access control metrics ( Hu and Scarfone, 2012 ). For each access control model, an overview followed by an analysis with respect to the applicable criteria is given. The models are reviewed according to the satisfaction degree for each metric (i.e. low, medium, high, optional, not applicable and not mentioned). Additionally, their study is compared versus seven other survey works in terms of the presented approaches, graphical definitions, advantages/disadvantages, the use of NIST metrics, number of reviewed articles and queried databases (e.g. IEEE, ACM, Springer, etc).

A Survey on access control in the age of internet of things ( Qiu et al ., 2020 ). The article presented a survey on the access control characteristics, technologies, a taxonomy of access control models requirements and future development direction in the IoT research field. In the IoT environment, the data are dynamic, massive, need strong privacy and continuously exchanged between different cooperation organizations. This work is compared with other literature reviews with similar focus in terms of access control policy description method, combination, conflict resolution and authoring (i.e. attribute discovery mechanism, policy mining and authorization model) explaining each requirement in detail. They described the authorization by categories based on the following: ABAC, RBAC, capability-based access control (CapBAC), usage control-based access model (UCON), OrBAC, blockchain and open authorization (OAuth).

the aim of access control;

the difference between authentication, authorization and access control;

the five core elements of access control models (i.e. subjects, objects, actions, privileges and access policies); and

the access control process flow.

Furthermore, they compared the static and dynamic access control models with respect to features, decision, pros/cons, examples and applications. For the traditional access control approaches, they just mentioned ACL, DAC, MAC and RBAC with a basic description. Then, an overview of the risk-based access control model and its elements is provided. Finally, they addressed the research methodology phases and analyzed the results providing answers to the research questions through comparing all the selected works.

In summary, Table 1 presents the models addressed in the previously discussed surveys with respect to our classification. We indicate whether all the models (✓) listed for each category in Section 4, some of them (O) or none (✗) are addressed in a given citation. For the roles category, all the works discuss the standard RBAC in different levels of details, however, only three of them addressed the OrBAC model while the ARBAC model is not mentioned at all.

We also include the models that are not stated in our work under the column Other . Only one access control model related to databases, i.e. System R ( Griffiths and Wade, 1976 ), is not mentioned. The rest belong to cloud computing and IoT domains, e.g. gateway-based access control (GBAC) ( Wu et al. , 2012 ), novel data access control (NDAC) ( Gao et al. , 2013 ), usage control-based access model (UCON) ( Danwei et al. , 2009 ), purpose-based usage access control (PBAC) ( Sun and Wang, 2010 ), toward temporal-based access control (TTAC) ( Zhu et al. , 2012 ), fine-grained access control (FGAC) ( Li et al. , 2010 ), capability-based access control (CapBAC) ( Hota et al. , 2011 ), hierarchical attribute-based access control (HABE) ( Xie et al. , 2015 ), attribute-based encryption fine-grained access control (ABE-FGAC) ( Tamizharasi et al. , 2016 ) and privacy-preserving ABAC (P-ABAC) ( Xu et al. , 2018 ).

6. Analysis

Authorization strategy : whether MAC, DAC or hybrid (recall Section 3);

Dynamic authorization : represents the dynamic definition of access rights in terms of rules and policies evaluating their attributes in real-time;

Granularity of control : indicates the objects’ levels of granularity, i.e. fine and/or coarse grained;

Least privilege principle support : the minimum access rights required for performing a task;

Separation of duty : ensures that access is only granted to subjects that are duty-related to the objects to limit power and avoid conflict of interests;

Vulnerable to attacks : is for ensuring the safety of the model to avoid the leakage of permissions to an unauthorized principal, e.g. trojan horse and covert channel attacks;

Bypass : is about whether policy rules are allowed to be bypassed for critical access decisions in emergency situations or not and how tolerant the risk is;

Conflict resolution or prevention : deals with preventing or resolving deadlocks and conflicting rules from the same or different policies;

Operational/situational awareness : considers operational/situational factors (e.g. some environment variables) in access rules specification and enforcement (i.e. decision-making);

Privileges/capabilities discovery : is the discovery of capabilities/objects (or object groups) of a given subject (or subject group) and vice versa.

After defining the selected criteria, we summarize them against our access control models categories in Table 2 . For each criterion, we indicate whether it is satisfied by all the models within a given classification group (✓), partially supported either with further considerations (i.e. based on access control requirements and model implementation) or by specific access control models within that category (O) or not at all (✗). If applicable, the level of satisfaction is indicated, i.e. low (L), medium (M) or high (H).

7. Conclusion

Access control mitigates the risks of unauthorized access attempts to data, resources and systems. The definition of authorization and access control in addition to their related concepts (i.e. strategy, model, policy and mechanism) are inconsistent in the literature. Besides, there are a lot of existing access control models; some of them are commonly known and used in practice while others have evolved recently and are not yet popular like the conventional models. The already available access control survey works are either including the state-of-the-art models at the publishing time or focusing on the taxonomy and classification of models for a particular domain.

In this paper, we first discussed authorization and access control along with the terms related to our research on the level of strategy, model and instance. We then explained authorization strategies and proposed a general classification for access control models without being restricted to a specific field (e.g. cloud computing and IoT). Moreover, we provided some examples of access control models along with the current implementations and extensions for the five categories, i.e. explicit object-subject assignment, model-specific rules, roles, content and context. We selected a list of comparative studies about survey, taxonomy and evaluation of access control models. Then, we summarized each work and compared the included models according to our classification. Finally, we analyzed the proposed classes of models with respect to several criteria; some of them are selected from the NIST standard access control system evaluation metrics, according to the level of support and considerations (if any).

The comparison result ( Table 1 ) shows that we discussed more models than other works for all the categories. We did not include advanced domain-specific models as we focus on general access control models with a view to databases.

Authorization and access control

Models included for each category in the survey works list