All applications are due by 5:00 PM local time of the applicant organization.
Applicants are encouraged to apply early to allow adequate time to make any corrections to errors found in the application during the submission process by the due date.
It is critical that applicants follow the Research (R) Instructions in the SF424 (R&R) Application Guide , except where instructed to do otherwise (in this NOFO or in a Notice from the NIH Guide for Grants and Contracts ). Conformance to all requirements (both in the Application Guide and the NOFO) is required and strictly enforced. Applicants must read and follow all application instructions in the Application Guide as well as any program-specific instructions noted in Section IV and follow the AHRQ Grants Policy and Guidance found on the AHRQ website at http://www.ahrq.gov/funding/policies/nofoguidance/index.html .
When the program-specific instructions deviate from those in the Application Guide, follow the program-specific instructions.
Applications that do not comply with these instructions may be delayed or not accepted for review.
There are several options available to submit your application through Grants.gov to NIH and Department of Health and Human Services partners. You must use one of these submission options to access the application forms for this opportunity.
Part 1. Overview Information Part 2. Full Text of the Announcement
Section I. Notice of Funding Opportunity Description Section II. Award Information Section III. Eligibility Information Section IV. Application and Submission Information Section V. Application Review Information Section VI. Award Administration Information Section VII. Agency Contacts Section VIII. Other Information
The mission of the Agency for Healthcare Research and Quality (AHRQ) is to produce evidence to make health care safer, higher quality, and more accessible, equitable, and affordable, and to work within the U.S. Department of Health and Human Services and with other partners to make sure the evidence is understood and used.
Many factors and conditions must align effectively to enable patient safety, including access to capable, motivated and healthy healthcare workers. However, the safety and well-being of healthcare workers, and therefore their ability to provide safe care for patients is challenged by many conditions, including limited resources, conflicting clinical goals (e.g., taking the best care of each patient versus being the best steward of healthcare resources, being efficient versus being thorough, caring for patients despite personal risk of injury or illness), changing priorities (e.g., emergencies can necessitate rapid changes in prioritization, with cascading effects on other patients and other providers), poorly designed or implemented technologies, tools and processes, time pressures, shift work and changing schedules, and lack of personal physical and psychological safety.
Numerous studies have documented that lack of healthcare worker safety and well-being are important current problems, with measurable adverse effects not only on healthcare workers, but also on patient care, patient experiences, and patient outcomes. In recognition of the important consequences of these relationships, improving healthcare worker safety and well-being are pillars of both the Quadruple Aim and the National Action Plan to Advance Patient Safety. Systems-based approaches to improve healthcare worker safety and well-being are essential and are expected to provide valuable benefits for both healthcare workers and patients. Research is urgently needed to better understand the effectiveness of systems-based interventions designed to improve healthcare worker safety and well-being, and to explore their impacts on healthcare workers and on patients.
The overarching goal of this NOFO is to improve healthcare worker safety and well-being so as to improve patient safety. Providing safe care for patients in all care delivery settings is not just about avoiding harm and attempting to prevent errors. Safe care should also be considered from a resilience engineering perspective as attempting to ensure "as many things as possible go right" so that healthcare delivery achieves outcomes that are successful for healthcare workers as well as patients and their families.
The National Academy of Medicine states that healthcare workers are among the most educated and dedicated workers in any industry. However, many healthcare workers struggle with burnout, moral injury, emotional exhaustion, depersonalization, and low levels of personal accomplishment. Burn-out is an occupational syndrome resulting from chronic workplace stress that has not been successfully addressed, characterized by 1) feelings of energy depletion or exhaustion; 2) increased mental distance from one’s job, or feelings of negativism or cynicism related to one's job; and 3) reduced professional efficacy. Moral distress, which can progress to moral injury, can manifest when healthcare workers are unable to provide the ethically driven quality of care they were trained to provide because of structural barriers and challenges. Sustained moral distress can lead to moral injury, which has been linked to feelings of profound guilt, shame, anger, and other psychological impacts.
In contrast, definitions of healthcare worker well-being are evolving. A proposed conceptual definition suggests that well-being (wellness) is defined by quality of life, which includes the absence of ill-being and the presence of positive physical, mental, social, and integrated well-being experienced in connection with activities and environments that allow healthcare workers to develop their full potentials across personal and work-life domains.
Poor healthcare worker well-being is often associated with medical errors and patient safety incidents (which may or may not involve errors). Patient safety and healthcare worker safety and well-being are synergistic, and excessive workload, long hours, burgeoning bureaucracy, physical hazards, bullying, and burnout experienced by healthcare workers have adverse effects on patient outcomes. Accordingly, efforts to minimize harm and provide safe care for patients must include efforts to address healthcare worker safety and well-being.
Factors that threaten healthcare worker safety and well-being extend beyond physical hazards and include moral distress, pandemic-related exhaustion, time pressure, lack of psychological safety, shortages of staff, equipment and supplies, poorly designed or poorly implemented technology, workflow, tools and processes, increasing patient acuity and complexity, structural racism and health inequities, conflicting clinical goals, excessive clerical and administrative burdens, and financial pressures. Furthermore, challenges with work life balance, the healthcare worker's own illness or disability, caregiving responsibilities for children, older adults, and other family members, and lack of sensitivity to each of these factors can also contribute to burnout. Although these challenges were exacerbated during the COVID-19 pandemic, the issues are not new. These factors impact individual healthcare workers, care teams, and entire healthcare delivery organizations.
While the importance of healthcare worker well-being is frequently recognized, efforts to achieve well-being often focus on supporting the individual healthcare worker’s use of coping mechanisms such as yoga, physical exercise, and meditation. While these interventions may be helpful, they do not address the systemic factors that challenge healthcare worker safety and well-being.
Applications responding to this NOFO must test strategies to improve healthcare delivery system safety by addressing the factors that weaken healthcare worker safety and well-being and/or by making systemic changes to improve healthcare worker safety and well-being. Applications must include an operational definition of healthcare worker safety or well-being.
Several organizations, including AHRQ, the National Academy of Medicine, the World Health Organization, and the Institute for Healthcare Improvement, as well as the US Surgeon General and others, have recommended system approaches to improve patient safety through efforts targeting healthcare worker safety and well-being. This NOFO encourages applications focused on improving healthcare worker safety and well-being as a path to improving patient safety.
Healthcare delivery occurs in systems that are simultaneously complex and adaptive. Healthcare delivery is affected by the difficulty, urgency and resource requirements of the intended task, and the availability, accessibility, capabilities and characteristics of patients and the care team, technology, tools and devices, and medications, diverse patient care settings, physical plants and organizational cultures, and the environment, processes, policies, and expectations within the healthcare organization, as well as the requirements and expectations placed on the organization from owners, payers and payment structures, regulators, and others. Relationships, resources, and patient care goals change over time - sometimes slowly and sometimes abruptly. Each of these factors impacts healthcare worker and patient experiences, and each, in turn, is impacted by healthcare worker and patient experiences. In complex systems, the consequences of decisions and actions may emerge in unpredictable, non-linear ways. Although subsystem analysis may provide rich insights, the whole of healthcare delivery cannot be completely understood by deconstruction into subsystem components.
Because of the complexity and adaptive nature of healthcare delivery, and to ensure that healthcare worker well-being enhancements are effective and sustainable, the input and unique insights of healthcare workers must be included in the development and implementation of system level improvements. Involving licensed healthcare professionals in the development and implementation of interventions to support healthcare worker safety and well-being will take advantage of their lived experiences, address competing priorities, and increase their investment in improvement processes and sustainable outcomes. AHRQ requires applications to include licensed healthcare professionals in project planning and the development of interventions or strategies, including those who deliver care in ambulatory, prehospital, emergency, inpatient, post-acute or long-term care settings, including medical, surgical, and mental health care, to patients of any age.
AHRQ encourages research projects that address more effective ways to improve healthcare worker safety and well-being and promote the wide-scale adoption of evidence-based approaches. This NOFO will support Large Research Projects (R01) in areas of healthcare worker safety and well-being focused on healthcare delivery system processes and informed by the participation of licensed healthcare professionals.
This NOFO invites applications that seek to improve healthcare worker safety and well-being by:
1) improving healthcare delivery systems with collaboration at multiple levels, which may include some combination of unit, specialty, department, facility and/or organizational systems, and
2) requiring licensed healthcare professional participation as project PIs, Co-PIs, Senior and/or Key Personnel, to benefit from their input, experience and insights in the development and implementation of improvements, to optimize both healthcare worker capabilities and patient outcomes.
The involvement of many other people who provide essential contributions to patient safety, such as executive leaders, administrators, healthcare delivery support staff (e.g., staff providing environmental and security services, and information technology, biomedical, equipment, and supply resources), and patients, families, and their communities, may also be components of applications responding to this NOFO.
Healthcare workers, as individuals and members of teams, fulfill a variety of roles with a range of responsibilities, thus affording diverse insights into hazards and latent safety threats, first-hand understanding of work-as-done, and invaluable perspectives about potential interventions to enhance patient safety through interventions targeting healthcare worker safety and well-being.
In the following examples, the first list includes processes or qualities that may be desirable, and the second list includes processes or qualities that can be problematic. Researchers may address one or more items from one or both lists. The lists are not intended to be exclusive.
Examples of factors relevant to this NOFO include, but are not limited to, processes, interventions or insights related to understanding, developing, improving, assessing, or maintaining:
1) healthcare worker safety, well-being, satisfaction, joy, recognition, engagement, respect, management of work-home relationships
4) care planning, transitions (e.g., anticipatory planning or transfers of responsibility such as discharges, handoffs, consultations)
5) adaptation, margin (e.g., capacity for adaptive response), or graceful degradation (e.g., ability to prioritize, and defer less essential tasks, but not collapse under pressure)
6) value, priority alignment (e.g., ability to address or resolve conflicting patient care goals)
8) user-friendly/user-informed processes (e.g., clinical decision support, or other electronic or non-electronic processes), user-friendly/user-informed technology
Processes, interventions or insights related to understanding, assessing, preventing, managing, or mitigating the following healthcare worker concerns are also examples of factors relevant to this NOFO:
3) gridlock, brittleness (e.g., propensity for processes to collapse under pressure)
Grant applications may address conditions and circumstances such as patient care surges, acute or chronic crises, and resource limitations as well as ordinary and/or successful work. Grant applications may address, but are not limited to, cognitive, physical, or social/ behavioral processes.
Research, measurement, and practice improvement to enhance healthcare worker safety and well-being may address adverse events and harms or may encompass approaches that support learning from how and why things go right and how to anticipate, recognize, monitor, and respond to hazards and opportunities. Projects can be designed from a Safety-I or a Safety-II lens. The Safety-I perspective typically addresses what went wrong and attempts to prevent future occurrences of similar adverse events, generally by means of constraints. Grant applications may address preventing or mitigating conditions or processes that adversely impact healthcare worker well-being. The Safety-II perspective seeks a deep understanding of what went well with an appreciation of the importance of adaptability in both ordinary and extraordinary healthcare delivery. Healthcare workers doing complex everyday work are successful more often than they are unsuccessful. Studying how healthcare workers constantly adapt to create successful outcomes despite dynamic environments, incomplete information and limited resources may help define and further improve the factors and conditions that underpin that success.
Methods may include the use of simulation and/or debriefing as interventions or research methodologies. A variety of surveys are available, addressing burnout, fulfillment, work environments, and healthcare worker and organizational safety, such as components of AHRQ’s Surveys on Patient Safety Culture (SOPS). Less traditional research methods, such as stepped wedge cluster randomized trial, statistical process control, adaptive trials, agile design, implementation, mixed methods or solely qualitative research methods (e.g., interviews, observation, ethnography) may be appropriate. All research methods must be justified and include a theoretical or conceptual justification.
To increase the consideration of scientific principles from diverse sciences or industries that design or implement methods to improve safety for humans, applicants must include collaboration with safety experts outside of surgery, internal medicine, pediatric, pathology or other traditional medical fields. Examples of collaborators include, but are not limited to, engineers with human factors, systems or industrial safety expertise, design or implementation scientists, informatics experts, social scientists such as anthropologists and sociologists, architects, economists, and psychologists with expertise in safety for humans.
Seek to improve healthcare delivery systems to improve healthcare worker safety and well-being;
Include at least one PI, Co-PI, Senior and/or Key Person who is a licensed healthcare professional in project planning and the development of interventions or strategies,
Include at least one PI, Co-PI, Senior and/or Key Person from outside of traditional medical fields who has unique knowledge relevant to healthcare worker safety and well-being.
See Section VIII. Other Information for award authorities and regulations.
The and the SF424 (R&R) Application Guide provide details on these application types. Only those application types listed here are allowed for this NOFO.
Future year funding is contingent upon the availability of funds for each year of support.
Because the nature and scope of the proposed research will vary from application to application, it is anticipated that the size and duration of each award will also vary.
Funds may be used only for those expenses that are directly related and necessary to the project and must be expended in compliance with applicable Uniform Administrative Requirements, Cost Principles, and Audit Requirements for HHS Awards (45 CFR Part 75) and the HHS .
These projects are being funded pursuant to 42 U.S.C. 299a, which provides that AHRQ shall conduct and support research, support demonstration projects, and disseminate information on health care and on systems for the delivery of such care, including activities with respect to the quality, effectiveness, efficiency, appropriateness, and value of health care services.
All applications submitted and AHRQ grants made in response to this NOFO are subject to 45 CFR Part 75 (Uniform Administrative Requirements, Cost Principles and Audit Requirements for HHS Awards; https://www.ecfr.gov/cgi-bin/text-idx?node=pt45.1.75) , the HHS Grants Policy Statement (see https://www.ahrq.gov/funding/policies/hhspolicy/index.html ), and the terms and conditions set forth in the Notice of Award.
Higher Education Institutions
- Public/State Controlled Institutions of Higher Education
- Private Institutions of Higher Education
The following types of Higher Education Institutions are always encouraged to apply for NIH support as Public or Private Institutions of Higher Education:
- Hispanic-serving Institutions
- Historically Black Colleges and Universities (HBCUs)
- Tribally Controlled Colleges and Universities (TCCUs)
- Alaska Native and Native Hawaiian Serving Institutions
- Asian American Native American Pacific Islander Serving Institutions (AANAPISIs)
Nonprofits Other Than Institutions of Higher Education
- Nonprofits with 501(c)(3) IRS Status (Other than Institutions of Higher Education)
- Nonprofits without 501(c)(3) IRS Status (Other than Institutions of Higher Education)
Local Governments
- State Governments
- County Governments
- City or Township Governments
- Special District Governments
- Indian/Native American Tribal Governments (Federally Recognized)
- Indian/Native American Tribal Governments (Other than Federally Recognized)
Federal Governments
- Eligible Agencies of the Federal Government
- U.S. Territory or Possession
- Independent School Districts
- Public Housing Authorities/Indian Housing Authorities
- Native American Tribal Organizations (other than Federally recognized tribal governments)
- Faith-based or Community-based Organizations
- Regional Organizations
AHRQ's authorizing legislation does not allow for-profit organizations to be eligible to lead applications under this research mechanism. For-profit organizations may participate in projects as members of consortia or as subcontractors only. Because the purpose of this program is to improve healthcare in the United States, foreign institutions may participate in projects as members of consortia or as subcontractors only. Applications submitted by for-profit organizations or foreign institutions will not be reviewed. Organizations described in section 501(c) 4 of the Internal Revenue Code that engage in lobbying are not eligible.
HHS grants policy requires that the grant recipient perform a substantive role in the conduct of the planned project or program activity and not merely serve as a conduit of funds to another party or parties. If consortium/contractual activities represent a significant portion of the overall project, the applicant must justify why the applicant organization, rather than the party(s) performing this portion of the overall project, should be the recipient and what substantive role the applicant organization will play. Justification can be provided in the Specific Aims or Research Strategy section of the PHS398 Research Plan Component sections of the SF424 (R&R) application. There is no budget allocation guideline for determining substantial involvement; determination of substantial involvement is based on a review of the primary project activities for which grant support is provided and the organization(s) that will be performing those activities.
Non-domestic (non-U.S.) Entities (Foreign Organizations) are not eligible to apply.
Non-domestic (non-U.S.) components of U.S. Organizations are not eligible to apply.
Applicant Organizations
Applicant organizations must complete and maintain the following registrations as described in the SF 424 (R&R) Application Guide to be eligible to apply for or receive an award. All registrations must be completed prior to the application being submitted. Registration can take 6 weeks or more, so applicants should begin the registration process as soon as possible.
- NATO Commercial and Government Entity (NCAGE) Code Foreign organizations must obtain an NCAGE code (in lieu of a CAGE code) in order to register in SAM.
- Unique Entity Identifier (UEI) - A UEI is issued as part of the SAM.gov registration process. The same UEI must be used for all registrations, as well as on the grant application.
- eRA Commons - Once the unique organization identifier is established, organizations can register with eRA Commons in tandem with completing their Grants.gov registration; all registrations must be in place by time of submission. eRA Commons requires organizations to identify at least one Signing Official (SO) and at least one Program Director/Principal Investigator (PD/PI) account in order to submit an application.
- Grants.gov Applicants must have an active SAM registration in order to complete the Grants.gov registration.
Program Directors/Principal Investigators (PD(s)/PI(s))
All PD(s)/PI(s) must have an eRA Commons account. PD(s)/PI(s) should work with their organizational officials to either create a new account or to affiliate their existing account with the applicant organization in eRA Commons. If the PD/PI is also the organizational Signing Official, they must have two distinct eRA Commons accounts, one for each role. Obtaining an eRA Commons account can take up to 2 weeks.
At least one PI, Co-PI, Senior and/or Key Person must be a licensed healthcare professional.
At least one PI, Co-PI, Senior and/or Key Person must have unique knowledge outside of traditional medical fields that will contribute to the research program.
Any individual(s) with the skills, knowledge, and resources necessary to carry out the proposed research as the Program Director(s)/Principal Investigator(s) (PD(s)/PI(s)) is invited to work with his/her organization to develop an application for support. Individuals from diverse backgrounds, including underrepresented racial and ethnic groups, individuals with disabilities, and women are always encouraged to apply for AHRQ support.
For institutions/organizations proposing multiple PDs/PIs, visit the Multiple Program Director/Principal Investigator Policy and submission details in the Senior/Key Person Profile (Expanded) Component of the SF424 (R&R) Application Guide. The AHRQ multiple PDs/PIs policy can be found at https://grants.nih.gov/grants/guide/notice-files/NOT-HS-16-018.html .
This NOFO does not require cost sharing.
While there is no cost sharing requirement included in this NOFO, AHRQ welcomes applicant institutions, including any collaborating institutions, to devote resources to this effort. An indication of institutional support from the applicant and its collaborators indicates a greater potential of success and sustainability of the project. Examples of institutional support would include: donated equipment and space, institutional funded staff time and effort, or other resource investments. Applicant institutions should indicate institutional support by outlining the specific contributions to the project and providing assurances that their organization and any collaborators are committed to providing these funds and resources to the project. This information can be included at the end of the budget justification section of the application, but institutional support dollars are not to be shown/included in the detailed budget request.
Applicant organizations may submit more than one application, provided that each application is scientifically distinct.
AHRQ will not accept duplicate or highly overlapping applications under review at the same time per NIH Grants Policy Statement 2.3.7.4 Submission of Resubmission Application . This means that the AHRQ will not accept:
- A new (A0) application that is submitted before issuance of the summary statement from the review of an overlapping new (A0) or resubmission (A1) application.
- A resubmission (A1) application that is submitted before issuance of the summary statement from the review of the previous new (A0) application.
The application forms package specific to this opportunity must be accessed through ASSIST, Grants.gov Workspace or an institutional system-to-system solution. Links to apply using ASSIST or Grants.gov Workspace are available in Part 1 of this NOFO. See your administrative office for instructions if you plan to use an institutional system-to-system solution.
It is critical that applicants follow the Research (R) Instructions in the How to Apply - Application Guide , except where instructed in this notice of funding opportunity to do otherwise. Conformance to the requirements in the Application Guide is required and strictly enforced. Applications that are out of compliance with these instructions may be delayed or not accepted for review.
Although a letter of intent is not required, is not binding, and does not enter into the review of a subsequent application, the information that it contains allows AHRQ staff to estimate the potential review workload and plan the review.
By the date listed in Part 1. Overview Information , prospective applicants are asked to submit a letter of intent that includes the following information:
- Descriptive title of proposed activity
- Name(s), address(es), and telephone number(s) of the PD(s)/PI(s)
- Names of other key personnel
- Participating institution(s)
- Number and title of this funding opportunity
The letter of intent should be emailed to:
Name: Ellen’s Deutsch, MD, MS O/C Division: General Patient Safety Telephone: 301-648-3028 Email: [email protected]
All page limitations described in the How to Apply Application Guide and the Table of Page Limits must be followed.
For this specific NOFO, the Research Strategy section is limited to 12 pages.
The following section supplements the instructions found in the How to Apply Application Guide and should be used for preparing an application to this NOFO.
All instructions in the SF424 (R&R) Application Guide must be followed.
Budget Component: Special Instructions for AHRQ applications
AHRQ is not using the Modular Grant Application and Award Process. Applicants applying for funding from AHRQ are to ignore application instructions concerning the Modular Grant Application and Award Process, and prepare applications using instructions for the Research and Related Budget Components of the SF 424 (R&R). Applications submitted in the Modular format will not be reviewed.
All instructions in the SF424 (R&R) Application Guide must be followed, with the following additional instructions:
Resource Sharing Plan :
Individuals are required to comply with the instructions for the Resource Sharing Plans as provided in the SF424 (R&R) Application Guide, with the following modification:
Submission of a data management plan is required. AHRQ applicants are reminded to refer to NOT-HS-20-011: The Agency for Healthcare Research and Quality Data Management Plan Policy (https://grants.nih.gov/grants/guide/notice-files/NOT-HS-20-011.html) for additional information on how to incorporate their data management plan into the resource sharing plan.
Do not use the Appendix to circumvent page limits. Follow all instructions for the Appendix as described in the SF424 (R&R) Application Guide.
When involving human subjects research, clinical research, and/or clinical trials (and when applicable, clinical trials research experience) follow all instructions for the PHS Human Subjects and Clinical Trials Information form in the SF424 (R&R) Application Guide, with the following additional instructions:
If you answered Yes to the question Are Human Subjects Involved? on the R&R Other Project Information form, you must include at least one human subjects study record using the Study Record: PHS Human Subjects and Clinical Trials Information form or Delayed Onset Study record.
Study Record: PHS Human Subjects and Clinical Trials Information
Delayed Onset Study
Note: Delayed onset does NOT apply to a study that can be described but will not start immediately (i.e., delayed start).
All instructions in the SF424 (R&R) Application Guide must be followed. For details regarding IRB approval, applicants may refer to the "AHRQ Revised Policy for Institutional Review Board (IRB) Review of Human Subjects Protocols in Grant Applications" ( https://grants.nih.gov/grants/guide/notice-files/not-hs-00-003.html ). Applicant should also be aware of the AHRQ policy for use of single IRB for cooperative research, 45 CFR 46.114 (b) https://grants.nih.gov/grants/guide/notice-files/NOT-HS-20-005.html .
See Part 1. Section III.1 for information regarding the requirement for obtaining a unique entity identifier and for completing and maintaining active registrations in System for Award Management (SAM), NATO Commercial and Government Entity (NCAGE) Code (if applicable), eRA Commons, and Grants.gov
Part I. Overview Information contains information about Key Dates and times. Applicants are encouraged to submit applications before the due date to ensure they have time to make any application corrections that might be necessary for successful submission. When a submission date falls on a weekend or Federal holiday , the application deadline is automatically extended to the next business day.
Organizations must submit applications to Grants.gov (the online portal to find and apply for grants across all Federal agencies). Applicants must then complete the submission process by tracking the status of the application in the eRA Commons , NIH’s electronic system for grants administration. NIH and Grants.gov systems check the application against many of the application instructions upon submission. Errors must be corrected and a changed/corrected application must be submitted to Grants.gov on or before the application due date and time. If a Changed/Corrected application is submitted after the deadline, the application will be considered late. Applications that miss the due date and time are subjected to the NIH Grants Policy Statement Section 2.3.9.2 Electronically Submitted Applications .
Applicants are responsible for viewing their application before the due date in the eRA Commons to ensure accurate and successful submission.
Information on the submission process and a definition of on-time submission are provided in the How to Apply Application Guide .
This initiative is not subject to intergovernmental review.
For efficient grant administration, AHRQ grant administration procedures will be used and conducted in accordance with the terms and conditions, cost principles, and other considerations described in the HHS Grants Policy Statement. The HHS Grants Policy Statement can be found at http://www.hhs.gov/sites/default/files/grants/grants/policies-regulations/hhsgps107.pdf .
Pre-award costs are allowable. A recipient may, at its own risk and without AHRQ prior approval, incur obligations and expenditures to cover costs up to 90 days before the beginning date of the initial budget period of a new award if such costs are necessary to conduct the project and would be allowable under the grant, if awarded, without AHRQ prior approval. If specific expenditures would otherwise require prior approval, the recipient must obtain AHRQ approval before incurring the cost. AHRQ prior approval is required for any costs to be incurred more than 90 days before the beginning date of the initial budget period of a new award.
The incurrence of pre-award costs in anticipation of a competing or non-competing award imposes no obligation on AHRQ either to make the award or to increase the amount of the approved budget if an award is made for less than the amount anticipated and is inadequate to cover the pre-award costs incurred. AHRQ expects the recipient to be fully aware that pre-award costs result in borrowing against future support and that such borrowing must not impair the recipient's ability to accomplish the project objectives in the approved time frame or in any way adversely affect the conduct of the project.
Applications must be submitted electronically following the instructions described in the SF424 (R&R) Application Guide. Paper applications will not be accepted.
Applicants must complete all required registrations before the application due date. Section III. Eligibility Information contains information about registration.
For assistance with your electronic application or for more information on the electronic submission process, visit How to Apply Application Guide .. If you encounter a system issue beyond your control that threatens your ability to complete the submission process on-time, you must follow the Dealing with System Issues guidance. For assistance with application submission, contact the Application Submission Contacts in Section VII .
Important reminders:
All PD(s)/PI(s) must include their eRA Commons ID in the Credential field of the Senior/Key Person Profile form . Failure to register in the Commons and to include a valid PD/PI Commons ID in the credential field will prevent the successful submission of an electronic application to AHRQ. See Section III of this NOFO for information on registration requirements.
The applicant organization must ensure that the unique entity identifier provided on the application is the same identifier used in the organization’s profile in the eRA Commons and for the System for Award Management. Additional information may be found in the SF424 (R&R) Application Guide.
See more tips for avoiding common errors.
The applicant should pay particular attention to the SF424 (R&R) application guide instructions https://grants.nih.gov/grants/how-to-apply-application-guide/forms-f/general-forms-f.pdf ) concerning letters of support. Letters of support, recommendation, or affirmation from any entity or individual not directly participating in the project should not be included.
Upon receipt, applications will be evaluated for completeness by the Center for Scientific Review, National Institutes of Health, and for responsiveness by AHRQ. Applications that are incomplete or non-compliant will not be reviewed.
Institutional Review Board (IRB) approval of human subjects is not required prior to peer review of an application (see https://grants.nih.gov/grants/guide/notice-files/NOT-HS-00-003.html ). However, initiation of IRB review, if necessary or applicable, is strongly encouraged to assure timely commencement of research.
Although there is no immediate acknowledgement of the receipt of an application, applicants are generally notified of the review and funding assignment within eight (8) weeks.
Please be sure that you observe the total cost, project period, and page number limitations specified above for this NOFO. Application processing may be delayed or the application may be rejected if it does not comply with these requirements
Applicants are required to follow the instructions for post-submission materials, as described in the policy . Any instructions provided here are in addition to the instructions in the policy.
Priority Populations
AHRQ is committed to the inclusion of priority populations in health services research. The overall portfolio of health services research that AHRQ conducts and supports shall include the populations specifically named in AHRQ’s authorizing legislation: inner city; rural; low income; minority; women; children; elderly; and those with special health care needs, including those who have disabilities, need chronic care, or need end-of-life health care. 42 U.S.C. 299(c)(1). AHRQ also includes in its definition of priority populations those groups identified in Section 2(a) of Executive Order 13985 as members of underserved communities: Black, Latino, and Indigenous and Native American persons, Asian Americans and Pacific Islanders and other persons of color; members of religious minorities; lesbian, gay, bisexual, transgender, and queer (LGBTQ+) persons; persons with disabilities; persons who live in rural areas; and persons otherwise adversely affected by persistent poverty or inequality.
AHRQ will broadly implement this inclusion policy across the research that AHRQ supports and conducts so that the portfolio of research is inclusive of all populations. AHRQ intends that these populations be included in studies such that the research design explicitly allows conduct of valid analyses. The policy applies to all grant applications. Investigators should review the document entitled, AHRQ Policy on the Inclusion of Priority Populations, which is available at https://grants.nih.gov/grants/guide/notice-files/NOT-HS-21-015.html . Applicants under this NOFO must consider and discuss including priority populations in research design as specified in this Notice.
Public Access to AHRQ-Funded Scientific Publications
Investigators should review the document titled AHRQ Announces new Policy for Public Access to AHRQ-Funded Scientific Publications , which is available at ( http://grants.nih.gov/grants/guide/notice-files/NOT-HS-16-008.html ). For all research arising from AHRQ support, this policy requires that AHRQ-funded authors submit an electronic version of the author’s final peer-reviewed accepted manuscript to the National Library of Medicine's PubMed Central (PMC) to be made publicly available within 12 months of the publisher’s date of publication.
AHRQ Data Management Plan Policy
Investigators should review the document titled AHRQ Data Management Plan (DMP) Policy, which is available at ( https://grants.nih.gov/grants/guide/notice-files/NOT-HS-20-011.html ). This policy requires applicants for AHRQ new/competing grants and research contracts to include a DMP for managing, storing and disseminating the primary data, samples, physical collections, and other supporting materials created or gathered in the course of research funded by AHRQ, or state why data management is not possible, as a component of their grant application or research contract proposal.
Plan for Sharing Research Data
The precise content of the data-sharing plan will vary, depending on the data being collected and how the investigator is planning to share the data. Applicants who are planning to share data should describe briefly the expected schedule for data sharing; the format of the final dataset; the documentation to be provided; whether or not any analytic tools also will be provided; whether or not a data-sharing agreement will be required and, if so, a brief description of such an agreement (including the criteria for deciding who can receive the data and whether or not any conditions will be placed on their use); and the mode of data sharing (e.g., under its own auspices by mailing a disk or posting data on its institutional or personal website or through a data archive or enclave). Investigators choosing to share under their own auspices may wish to enter into a data-sharing agreement. References to data sharing may also be appropriate in other sections of the application.
The reasonableness of the data sharing plan or the rationale for not sharing research data will be assessed by the reviewers. However, reviewers will not factor the proposed data sharing plan into the determination of scientific merit or the priority score.
Data Confidentiality
The AHRQ confidentiality statute, 42 USC 299c-3(c), requires that information that is obtained in the course of AHRQ supported activities and that identifies individuals or establishments be used only for the purpose for which it was supplied. Information that is obtained in the course of AHRQ-supported activities and that identifies an individual may be published or released only with the consent of the individual who supplied the information or is described in it. There are civil monetary penalties for violation of the confidentiality provision of the AHRQ statute. 42 USC 299c-3(d). In the Human Subjects section of the application, applicants must describe procedures for ensuring the confidentiality of the identifying information to be collected (see NOT-HS-18-012 : Confidentiality in AHRQ-Supported Research). The description of the procedures should include a discussion of who will be permitted access to this information, both raw data and machine-readable files, and how personal identifiers and other identifying or identifiable data will be restricted and safeguarded. Identifiable patient health information collected by recipient under this NOFO will also be obtained and managed in accordance with the HIPAA Privacy Rule, 45 CFR Parts 160 and 164.
The recipient should ensure that computer systems containing confidential data have a level and scope of security that equals or exceeds that established by the HIPAA Security Rules if applicable (see HIPAA website in prior paragraph) and that established by the Office of Management and Budget (OMB) in OMB Circular No. A-130, Appendix III - Security of Federal Automated Information Systems. The applicability and intended means of applying these confidentiality and security standards to subcontractors and vendors, if any, should be addressed in the application.
Sharing Research Resources: Rights in Data
Unless otherwise provided in grant awards, AHRQ recipients may copyright, or seek patents for, as appropriate, final and interim products and materials developed in whole or in part with AHRQ support, including, but not limited to, methodological tools, measures, software with documentation, literature searches, and analyses. Such copyrights and patents are subject to a royalty-free, non-exclusive, and irrevocable AHRQ license to reproduce, publish, use, or disseminate for any purpose consistent with AHRQ’s statutory responsibilities and to authorize others to do so for any purpose consistent with AHRQ’s statutory responsibilities. In accordance with its legislative dissemination mandate, AHRQ purposes may include, subject to statutory confidentiality protections, making project materials, databases, results, and algorithms available for verification or replication by other researchers. In addition, subject to AHRQ budget constraints, final products may be made available to the health care community and the public by AHRQ or its agents if such distribution would significantly increase access to a product and thereby produce substantial or valuable public health benefits. Ordinarily, to accomplish distribution, AHRQ publicizes research findings but relies on recipients to publish research results in peer-reviewed journals and to market grant-supported products. AHRQ requests that recipients notify the Office of Communications (OC) when an AHRQ-funded research article has been accepted for publication in a peer-reviewed journal. Researchers should submit manuscripts that have been accepted for publication in a peer-reviewed journal to [email protected] at least four to six weeks in advance of the journal’s expected publication date.
Regulations applicable to AHRQ recipients concerning intangible rights and copyright can be found at 45 CFR 75.322.
The mission of AHRQ is to produce evidence to make health care safer, higher quality, more accessible, equitable, and affordable, and to work with the U.S. Department of Health and Human Services (HHS) and other partners to make sure that the evidence is understood and used.
AHRQ's priority areas of focus are detailed here: https://www.ahrq.gov/funding/policies/nofoguidance/index.html .
As part of this mission, applications are submitted to AHRQ to support health services research which are evaluated for scientific and technical merit through the AHRQ peer review system.
Applications that are complete and responsive to the NOFO will be evaluated for scientific and technical merit by an appropriate objective group convened in accordance with standard AHRQ peer-review procedures that are described in 42 CFR Part 67, Subpart A. Incomplete and/or non-responsive applications or applications not following instructions given in this NOFO will not be reviewed. Only the review criteria described below will be considered in the review process.
Merit Review Criteria: Merit Review Criteria, as described below, will be considered in the review process.
Reviewers will provide an overall impact score to reflect their assessment of the likelihood for the project to exert a sustained, powerful influence on the research field(s) involved, in consideration of the following review criteria and additional review criteria (as applicable for the project proposed).
Reviewers will consider each of the review criteria below in the determination of scientific merit and give a separate score for each. An application does not need to be strong in all categories to be judged likely to have major scientific impact. For example, a project that by its nature is not innovative may be essential to advance a field.
Does the project address an important problem or a critical barrier to progress in the field? Is the prior research that serves as the key support for the proposed project rigorous? If the aims of the project are achieved, how will scientific knowledge, technical capability, and/or clinical practice be improved? How will successful completion of the aims change the concepts, methods, technologies, treatments, services, or preventative interventions that drive this field? How will the knowledge gained improve healthcare worker safety and well-being?
Are the PD(s)/PI(s), collaborators, and other researchers well suited to the project? If Early Stage Investigators or those in the early stages of independent careers, do they have appropriate experience and training? If established, have they demonstrated an ongoing record of accomplishments that have advanced their field(s)? If the project is collaborative or multi-PD/PI, do the investigators have complementary and integrated expertise; are their leadership approach, governance, and organizational structure appropriate for the project? Does the PI team have unique knowledge outside of traditional medical fields that will contribute to the research program? Are licensed healthcare professionals - such as physicians, nurses, pharmacists, social workers, therapists, or other healthcare professionals who provide direct patient care - participating as PIs, Co-PIs, Senior and/or Key Personnel?
Does the application challenge and seek to shift current research or clinical practice paradigms by utilizing novel theoretical concepts, approaches or methodologies, instrumentation, or interventions? Are the concepts, approaches or methodologies, instrumentation, or interventions novel to one field of research or novel in a broad sense? Is a refinement, improvement, or new application of theoretical concepts, approaches or methodologies, instrumentation, or interventions proposed?
Are the overall strategy, methodology, and analyses well-reasoned and appropriate to accomplish the specific aims of the project? Have the investigators included plans to address weaknesses in the rigor of prior research that serves as the key support for the proposed project? Have the investigators presented strategies to ensure a robust and unbiased approach, as appropriate for the work proposed? Are potential problems, alternative strategies, and benchmarks for success presented? If the project is in the early stages of development, will the strategy establish feasibility and will particularly risky aspects be managed? Have the investigators presented adequate plans to address relevant biological variables, such as sex, for studies in vertebrate animals or human subjects?
Do the investigators seek to improve healthcare delivery systems to improve healthcare worker safety and well-being? Have the investigators provided an operational definition of healthcare worker safety or well-being?
Will the scientific environment in which the work will be done contribute to the probability of success? Are the institutional support, equipment, and other physical resources available to the investigators adequate for the project proposed? Will the project benefit from unique features of the scientific environment, subject populations, or collaborative arrangements?
As applicable for the project proposed, reviewers will evaluate the following additional items while determining scientific and technical merit, and in providing an overall impact score, but will not give separate scores for these items.
Data Management Plan
The reviewers will comment on whether the Data Management Plan is reasonable.
If the project involves human subjects and/or clinical research, are there plans to address
1) the protection of human subjects from research risks, and
2) inclusion (or exclusion) of individuals on the basis of sex/gender, race, and ethnicity, as well as the inclusion or exclusion of individuals of all ages (including children and older adults), justified in terms of the scientific goals and research strategy proposed?
For research that involves human subjects but does not involve one of the categories of research that are exempt under 45 CFR Part 46, the committee will evaluate the justification for involvement of human subjects and the proposed protections from research risk relating to their participation according to the following five review criteria: 1) risk to subjects, 2) adequacy of protection against risks, 3) potential benefits to the subjects and others, 4) importance of the knowledge to be gained, and 5) data and safety monitoring for clinical trials.
For research that involves human subjects and meets the criteria for one or more of the categories of research that are exempt under 45 CFR Part 46, the committee will evaluate: 1) the justification for the exemption, 2) human subjects involvement and characteristics, and 3) sources of materials. For additional information on review of the Human Subjects section, please refer to the Guidelines for the Review of Human Subjects .
For details regarding IRB approval, applicants may refer to the "AHRQ Revised Policy for Institutional Review Board (IRB) Review of Human Subjects Protocols in Grant Applications" ( https://grants.nih.gov/grants/guide/notice-files/not-hs-00-003.html ).). Applicant should also be aware of the AHRQ policy for use of single IRB for cooperative research, 45 CFR 46.114 (b) https://grants.nih.gov/grants/guide/notice-files/NOT-HS-20-005.html .
Peer reviewers will assess the adequacy of plans to address the needs of AHRQ priority populations.
Peer reviewers must include their assessment of the proposed inclusion plan for priority populations in evaluating the overall scientific and technical merit of the application and assigning the impact score.
In evaluating the overall impact of the application, the review groups will:
- Evaluate the application for the presence or absence of the inclusion plan based on the proposed research objectives.
- Evaluate the adequacy of the proposed plan for the inclusion of priority populations.
- Evaluate the proposed justification for the exclusion of priority populations when a requirement for inclusion is described as inappropriate with respect to the purpose of the research.
- Evaluate the plans for outreach and recruitment of study participants, including priority populations, where appropriate.
- Evaluate the proposed plan for study design, execution, and outcome assessments so that study results will be relevant to one or more priority populations, where appropriate.
- Assess the plan as being acceptable or unacceptable with regard to the appropriateness of the inclusion or exclusion of priority populations in the proposed research.
Reviewers will assess how well the application addresses the purpose and objectives of this NOFO. How responsive is the application to the special eligibility criteria, including the project requirements, noted in the NOFO?
The committee will evaluate whether the proposed budget is reasonable, and whether the requested period of support is appropriate in relation to the proposed research.
For Resubmissions, the committee will evaluate the application as now presented, taking into consideration the responses to comments from the previous scientific review group and changes made to the project.
As applicable for the project proposed, reviewers will consider each of the following items, but will not give scores for these items, and should not consider them in providing an overall impact score.
Not Appliable
Applications that are complete and responsive to the NOFO will be evaluated for scientific and technical merit by an appropriate Scientific Review Group convened in accordance with standard AHRQ peer review procedures that are described in 42 CFR Part 67, Subpart A. Incomplete and/or non-responsive applications or applications not following instructions given in this NOFO will not be reviewed.
As part of the scientific peer review, all applications:
- May undergo a selection process in which only those applications deemed to have the highest scientific and technical merit (generally the top half of applications under review) will be discussed and assigned an overall impact score.
- Will receive a written critique.
The following will be considered in making funding decisions:
- Scientific and technical merit of the proposed project as determined by scientific peer review.
- Availability of funds.
- Responsiveness to goals and objectives of the NOFO.
- Relevance and fit within AHRQ research priorities, as well as overall programmatic and geographic balance of the proposed project to program priorities.
After the peer review of the application is completed, the PD/PI will be able to access their Summary Statement (written critique) via the eRA Commons . Refer to Part 1 for dates for peer review, advisory council review, and earliest start date.
Information regarding the disposition of applications is available in the HHS Grants Policy Statement (see https://www.ahrq.gov/funding/policies/hhspolicy/index.html ).
If the application is under consideration for funding, AHRQ Division of Grants Management staff will request "Just-In-Time" information from the applicant. Just-In-Time information generally consists of information on other support, any additional information necessary to address administrative and budgetary issues, and certification of IRB approval of the project's proposed use of human subjects. For details regarding IRB approval, applicants may refer to the "AHRQ Revised Policy for Institutional Review Board (IRB) Review of Human Subjects Protocols in Grant Applications" ( https://grants.nih.gov/grants/guide/notice-files/not-hs-00-003.html ). Applicant should also be aware of the AHRQ policy for use of single IRB for cooperative research, 45 CFR 46.114 (b) https://grants.nih.gov/grants/guide/notice-files/NOT-HS-20-005.html ".
If all administrative and programmatic issues are resolved, a formal notification in the form of a Notice of Award (NoA) will be provided to the applicant organization for successful applications. The NoA signed by the AHRQ grants management officer is the authorizing document and will be sent via email to the e-mail address designated by the recipient organization during the eRA Commons registration process.
Recipients must comply with any funding restrictions described in Section IV.6. Funding Restrictions.
Selection of an application for award is not an authorization to begin performance. Any costs incurred before receipt of the NoA are at the recipient's risk. These costs may be reimbursed only to the extent considered allowable pre-award costs.
Any application awarded in response to this NOFO will be subject to the UEI, SAM Registration, and Transparency Act requirements.
All AHRQ grant and cooperative agreement awards are subject to HHS’s Uniform Administrative Requirements, Cost Principles, and Audit Requirements for HHS Awards, ( http://www.ecfr.gov/cgi-bin/text-idx?node=pt45.1.75 , and the HHS Grants Policy Statement (see http://www.ahrq.gov/funding/policies/hhspolicy/index.html ), and the terms and conditions set forth in the Notice of Award.
All applicants will be subject to a term and condition that applies the terms of 48 CFR section 3.908 to the award, and requires that recipients inform their employees in writing of employee whistleblower rights and protections under 41 U.S.C. 4712 in the predominant native language of the workforce.
As necessary, additional Terms and Conditions will be incorporated into the NoA.
If you are successful and receive a Notice of Award, in accepting the award, you agree that the award and any activities thereunder are subject to all provisions of 45 CFR Part 75, currently in effect or implemented during the period of the award, other Department regulations and policies in effect at the time of the award, and applicable statutory provisions.
If you receive an award, HHS may terminate it if any of the conditions in 2 CFR 200.340(a)(1)-(4) are met. No other termination conditions apply.
There is a prohibition on certain telecommunications and video surveillance services or equipment that became effective on or after August 13, 2020.
If you receive an award, you must follow all applicable nondiscrimination laws. You agree to this when
you register in SAM.gov. You must also submit an Assurance of Compliance (HHS-690). To learn more, see the HHS Office for Civil Rights website.
- Health Literacy Below are available HHS resources.
- HHS Health.gov: Health Literate Care Model
- AHRQ: Health Literacy Universal Precautions Toolkit
For additional guidance regarding how the provisions apply to AHRQ grant programs, please contact the Scientific/Research Contact that is identified in Section VII under Agency Contacts of this NOFO.
In accordance with the statutory provisions contained in Section 872 of the Duncan Hunter National Defense Authorization Act of Fiscal Year 2009 (Public Law 110-417), AHRQ awards will be subject to the Federal Recipient Performance and Integrity Information System (FAPIIS) requirements. FAPIIS requires Federal award making officials to review and consider information about an applicant in the designated integrity and performance system (currently FAPIIS) prior to making an award. An applicant, at its option, may review information in the designated integrity and performance systems accessible through FAPIIS and comment on any information about itself that a Federal agency previously entered and is currently in FAPIIS. The Federal awarding agency will consider any comments by the applicant, in addition to other information in FAPIIS, in making a judgement about the applicant s integrity, business ethics, and record of performance under Federal awards when completing the review of risk posed by applicants as described in 45 CFR Part 75.205 Federal awarding agency review of risk posed by applicants. This provision will apply to all AHRQ grants and cooperative agreements except fellowships.
For details regarding annual progress report submission, refer to https://www.ahrq.gov/funding/grant-mgmt/noncomp.html . If instructions on the AHRQ website are different from the RPPR Instructions, follow the instructions on the AHRQ website. Annual progress reports are due 3 months before the start date of the next budget period of the award.
Recipients are required to submit expenditure data on the Federal Financial Report (FFR; SF 425) annually.
Expenditure data is to be reported on the Federal Financial Report (FFR; SF 425). AHRQ requires annual financial expenditure reports for ALL grant programs as described in the HHS Grants Policy Statement ( https://www.ahrq.gov/funding/policies/hhspolicy/index.html ). AHRQ implementation of the FFR retains a financial reporting period that coincides with the budget period of a particular project. However, the due date for annual FFRs is 90 days after the end of the calendar quarter in which the budget period ends. For example, if the budget period ends 4/30/2023, the annual FFR is due 9/30/2023 (90 days after the end of the calendar quarter of 6/30/2023).
A final Progress Report, final Federal Financial Report, and Final Invention Statement are required when an award ends. All final reports are due within 120 days of the project period end date. For further details regarding grant closeout requirements, refer to http://www.ahrq.gov/funding/grant-mgmt/index.html#Closeout.
The Federal Funding Accountability and Transparency Act of 2006 (Transparency Act), includes a requirement for recipients of Federal grants to report information about first-tier subawards and executive compensation under Federal assistance awards issued in FY2011 or later. All recipients of applicable AHRQ grants and cooperative agreements are required to report to the Federal Subaward Reporting System (FSRS) available at www.fsrs.gov on all subawards over the threshold. See the 2 CFR Part 170 ( https://www.ecfr.gov/current/title-2/subtitle-A/chapter-I/part-170 ) for additional information on this reporting requirement.
In accordance with the regulatory requirements provided at 45 CFR 75.113 and 2 CFR Part 200.113 and Appendix XII to 45 CFR Part 75 and 2 CFR Part 200, recipients that have currently active Federal grants, cooperative agreements, and procurement contracts from all Federal awarding agencies with a cumulative total value greater than $10,000,000 for any period of time during the period of performance of a Federal award, must report and maintain the currency of information reported in the System for Award Management (SAM) about civil, criminal, and administrative proceedings in connection with the award or performance of a Federal award that reached final disposition within the most recent five-year period. The recipient must also make semiannual disclosures regarding such proceedings. Proceedings information will be made publicly available in the designated integrity and performance system (currently FAPIIS). This is a statutory requirement under section 872 of Public Law 110-417, as amended (41 U.S.C. 2313). As required by section 3010 of Public Law 111-212, all information posted in the designated integrity and performance system on or after April 15, 2011, except past performance reviews required for Federal procurement contracts, will be publicly available. Full reporting requirements and procedures are found in Appendix XII to 45 CFR Part 75 and 2 CFR Part 200 Award Term and Condition for Recipient Integrity and Performance Matters.
Recipient performance will be measured based on success in the following Program goals:
Program Planning and Performance reporting requirements:
The overarching objectives of these projects should be addressed through this reporting including progress with respect to:
- Performance measures: Number or variety of practices, facilities, healthcare workers, and/or patients reached by the implementation.
- Timeline: reach target by year 4.
- Process: What processes were evaluated, implemented or improved as a result of this project?
- Performance measures: number, types, or magnitude of processes evaluated, implemented, or improved.
- Implementation of processes based in Safety-II principles.
- Healthcare worker participation in leadership in event analysis committees
- Performance measures: Impact of healthcare worker safety and well-being solution on select healthcare worker or patient safety process or outcome measures. Impact of healthcare worker safety or well-being solution on select healthcare worker or patient safety or well-being outcome measures. Increased satisfaction, capacity for resilience, efficiency, adaptability, margin, autonomy, agency, collaboration. Decreased brittleness, burnout, moral distress, administrative burden, stress, production pressure. Improved healthcare worker well-being, patient outcomes
- Timeline: report on impact target by year 4.
We encourage inquiries concerning this funding opportunity and welcome the opportunity to answer questions from potential applicants.
eRA Service Desk (Questions regarding ASSIST, eRA Commons, application errors and warnings, documenting system problems that threaten submission by the due date, and post-submission issues)
Finding Help Online: https://www.era.nih.gov/need-help (preferred method of contact) Telephone: 301-402-7469 or 866-504-9552 (Toll Free)
Grants.gov Customer Support (Questions regarding Grants.gov registration and Workspace) Contact Center Telephone: 800-518-4726 Email: [email protected]
Ellen’s Deutsch, MD, MS General Patient Safety Telephone: 301-648-3028 Email: [email protected]
Examine your eRA Commons account for review assignment and contact information (information appears two weeks after the submission due date).
Robyn Sagatov, PhD, MHS, RDN Division of Scientific Review Office of Extramural Research, Education, and Priority Populations Agency for Healthcare Research and Quality Email: [email protected]
Kathryn Woolridge Office of Management Services Division of Grants Management Agency for Healthcare Research and Quality Telephone: 301-427-1025 Email: [email protected]
Recently issued AHRQ policy notices may affect your application submission. A full list of policy notices published by AHRQ is provided in the NIH Guide for Grants and Contracts. Notices can also be found at AHRQ Grants Policy Notices ( https://www.ahrq.gov/funding/policies/hhspolicy/index.html) .
This program is described in the Assistance Listings (formerly called the Catalog of Federal Domestic Assistance) at https://sam.gov/content/home and is not subject to the intergovernmental review requirements of Executive Order 12372 or Health Systems Agency review.
Awards are made under the authority of 42 USC 299 et seq., and in accordance with 45 CFR Part 75 and other referenced applicable statutes and regulations. All awards are subject to the terms and conditions, cost principles, and other considerations described in the HHS Grants Policy Statement. The HHS Grants Policy Statement can be found at http://www.ahrq.gov/funding/policies/hhspolicy/index.html
Cookies on the NHS England website
We’ve put some small files called cookies on your device to make our site work.
We’d also like to use analytics cookies. These send information about how our site is used to a service called Google Analytics. We use this information to improve our site.
Let us know if this is OK. We’ll use a cookie to save your choice. You can read more about our cookies before you choose.
Change my preferences I'm OK with analytics cookies
Reviewing patient safety events and developing advice and guidance
The patient safety review and response and advice and guidance teams perform a core function within the National Patient Safety team, working to review and analyse sources of information on patient safety events to identify new or under recognised risks.
These teams then lead on the development of advice and guidance to support the NHS to address those risks, such as National Patient Safety Alerts .
The process is supported by a wide range of registered healthcare professionals representing a variety of clinical specialties, many of whom work on wider patient safety policy and projects.
Additionally, we access clinical and topic expertise both internally and externally when required.
How we review records of patient safety events
The role of the team starts with the clinicians in our National Patient Safety Team reviewing information from the National Reporting and Learning System (NRLS), the Strategic Executive Information System (StEIS), and also the new Learn from Patient Safety Events service (LFPSE) , as well as a range of other sources, to identify new or under-recognised issues that may need national action.
The team currently review around 32,000 records of patient safety events each year. Sources of this information include:
- Patient safety events recorded on our national systems as resulting in death or severe harm, or that meet other thresholds for clinical review
- Selected categories of Serious Incidents reported to StEIS
- Potential and confirmed Never Events reported to StEIS
- Patient safety events recorded on the NRLS by patients and the public (we review all of these whether they result in harm or not)
- Letters from coroners where they have identified a need for action to prevent further deaths (Regulation 28 letters)
Focused reviews
Where our review suggests there could be a new or under-recognised issue that requires national action, we explore further. Our decision to undertake a focused review can often triggered by a single patient safety incident, which we then explore to understand if there is a wider patient safety issue.
We do this by looking to identify any patterns in similar incidents previously reported, including no harm ‘near miss’ incidents – with a focus on what could go wrong in future. This focused review process sees the team analysing several thousand further records of lower harm patient safety events each year.
Deciding what action to take
Where new and under-recognised risks are identified, not all issues will require a National Patient Safety Alert.
As part of that decision process we:
- Check whose remit an issue falls under, as some aspects of patient safety are handled by other national organisations and we can pass to them for action.
- Look for up-to-date detail about the issue in the NRLS, research studies and other published material, and seek advice from specialists and frontline staff to help identify the likelihood of this happening again, and the potential for harm, including the risk of death or disability.
- Consider if the patient safety issue can be addressed at source – for example, by the manufacturer of a device – and if it can, whether this will happen rapidly enough for no other action to be required.
- Talk to experts, patients and frontline staff to identify if the patient safety issue is new or under-recognised; these groups may have different perspectives.
- If it is new or under-recognised, explore whether organisations can do something more constructive than simply raising awareness and warning people to be vigilant against error, and the options for these actions (including interim actions while more robust barriers to error are developed).
- If the patient safety issue is well known, including if it was the subject of an earlier Alert, we recognise that substantial efforts will already have been made to address it, and further improvements will need more support than can be provided by a National Patient Safety Alert alone. We will consider if there are new or under-recognised resources or interventions.
- Consider if a National Patient Safety Alert is the best route; if actions only require changes in practice by a professional specialty, rather than wider action by healthcare teams or organisations, they may be more effectively communicated by a professional society, such as a royal college.
You can also view our National Patient Safety Alert decision flowchart .
Who advises us?
Insight to help us understand each patient safety issue and to develop the required actions for our Alerts mainly comes from frontline staff, patient representatives, professional bodies and partner organisations on our ‘National Patient Safety Response Advisory Panel’.
This panel is made up of:
- 20% patient and public voice
- 40% frontline staff from providers and commissioners from all healthcare sectors
- 40% key national and professional stakeholders
As a member of the National Patient Safety Alerting Committee (NaPSAC) , we have developed and improved our processes for issuing National Patient Safety Alerts and were the first organisation to be accredited to issue this new national standardised format for alerts.
The standards set for National Patient Safety Alerts ensure that the safety-critical and mandatory actions an Alert requires organisations to take, are clear, feasible and effective .
Examples of how we have responded to the risks we’ve identified
- You can find details of all alerts we have issued, on Our National Patient Safety Alerts The page also provides further information about the alerting process and the criteria we use to decide when an alert should be issued.
- Our patient safety review and response case studies show where we’ve worked with other organisations to address issues through non-alert routes.
- You can also find more examples and information in the Patient safety review and response reports we previously published between 2016 – 2019.
Supporting the NHS’s response to specific healthcare issues
At particular times our patient safety review and response work is tailored to include surveillance of specific healthcare issues arising at a point in time, such as the impact on patient safety of supply chain challenges or emerging diseases, such as Covid-19.
Initiating specific surveillance at such times provides early signals of new and emerging risks to patient safety and supports our aim to provide rapid advice and guidance to the NHS.
Supporting ongoing improvement work
The information we collect through our national systems also supports ongoing improvement work to tackle the more common and well-known patient safety challenges, such as reducing diagnostic error, preventing self-harm, avoiding falls or managing long-term anticoagulation.
These issues have complex causes and no simple solutions, and are the focus of long-term improvement work, including the NHS England National Patient Safety Improvement Programmes .
We also have extensive arrangements for sharing records of patient safety events with other national bodies, royal colleges, professional societies, and researchers, so they can review and act on issues that relate to their specific areas of patient safety responsibilities or expertise.
Partnership learning from specialist review of patient safety event data
We regularly share data with a number of clinical and professional networks that review records of patient safety events and use their findings to support safety improvements in their specialty.
These include:
- the Royal College of Emergency Medicine, which shares its findings in safety flashes
- the Safer Anaesthesia Liaison Group, which shares its findings in quarterly patient safety updates and uses them to inform wider guideline development
- UK Health Security Agency (formally Public Health England), which shares its findings in Safer Radiotherapy reports
- the MHRA, which uses NRLS data to inform its regulatory functions for medication and medical device safety
- the UK Kidney Patient Safety Committee, which shares its findings in patient safety updates
- the Health Safety Investigations Branch (HSIB) , which uses NRLS and Serious Incident data to provide wider context to their specific investigations.
We also share data on patient safety events with organisations and researchers who are looking into a specific patient safety topic. Data on patient safety events also regularly appears in journal publications.
The impact of our review and response work
Our national review and response work is a key part of the NHS patient safety strategy , which estimate 160 lives and £13.5 million in treatment costs are saved every year from the resulting advice, guidance and other outputs. Other parts of the strategy are also interlinked and support this work such as the roll out of the new national patient safety event recording system, LFPSE, which will introduce machine learning to support and improve how we analyse records of patient safety events. Work to further develop a patient safety culture across the NHS also helps our efforts by staff being more open and honest when things go wrong to support safety improvement without fear of blame.
- Introduction
- Conclusions
- Article Information
PRO indicates patient-reported outcome.
a See eTable 1 in the Supplement for details regarding Delphi survey and consensus meeting participant characteristics.
b Six additional stakeholders provided comments prior to or after the meeting.
Acknowledgements
eFigure. Decision Tree for Inclusion/Exclusion of Candidate Items
eTable 1. Delphi Survey and Consensus Meeting Participants Characteristics
eTable 2. Consensus Meeting Notes and Outcomes of Voting for the PRO Ethics Guidelines
eTable 3a. Eight Items (Items 2-17) Which Fulfilled the Inclusion Criteria at the Delphi Exercise Stage
eTable 3b. Nine Items Proposed for Inclusion to Discuss at the Consensus Meeting
eTable 4a. Excluded Candidate Items at the Delphi Exercise Stage
eTable 4b. Excluded Candidate Items That Were Merged With Other Items and/or Were Covered by Existing Items
- Electronic Symptom Monitoring With Patient-Reported Surveys and Outcomes Among Patients With Metastatic Cancer JAMA Original Investigation June 28, 2022 This randomized clinical trial assesses the effect of using weekly electronic patient-reported outcome surveys to monitor symptoms, compared with usual care, on physical function, symptom control, and health-related quality of life among patients receiving treatment for metastatic cancer. Ethan Basch, MD, MSc; Deborah Schrag, MD, MPH; Sydney Henson, BS; Jennifer Jansen, MPH; Brenda Ginos, MS; Angela M. Stover, PhD; Philip Carr, MPH; Patricia A. Spears, BS; Mattias Jonsson, BA; Allison M. Deal, MS; Antonia V. Bennett, PhD; Gita Thanarajasingam, MD; Lauren J. Rogak, MA; Bryce B. Reeve, PhD; Claire Snyder, PhD; Deborah Bruner, PhD; David Cella, PhD; Lisa A. Kottschade, MSN; Jane Perlmutter, PhD; Cindy Geoghegan, MA; Cleo A. Samuel-Ryals, PhD; Barbara Given, PhD; Gina L. Mazza, PhD; Robert Miller, MD; Jon F. Strasser, MD; Dylan M. Zylla, MD; Anna Weiss, MD; Victoria S. Blinder, MD; Amylou C. Dueck, PhD
- Patient-Reported Outcome Measures in Clinical Research JAMA JAMA Guide to Statistics and Methods August 2, 2022 This Guide to Statistics and Methods provides an overview of patient-reported outcome measures for clinical research, emphasizes several important considerations when using them, and points out their limitations. Kevin P. Weinfurt, PhD; Bryce B. Reeve, PhD
- Stability of Changes in Health Status JAMA Viewpoint September 13, 2022 This Viewpoint explores how patient-reported response to treatment informs understanding of clinically meaningful outcomes to help establish patient expectations, inform clinical decision-making, and ensure treatment is sound. Muhammad Shahzeb Khan, MD, MSc; Javed Butler, MD, MPH, MBA
- Sharing of Individual Patient-Level Data by Trialists of Randomized Clinical Trials JAMA Research Letter May 16, 2023 This study describes access to individual patient-level data from randomized clinical trials during the COVID-19 pandemic to determine whether the intent to share what was reported in the registry, publication, or preprint was consistent with actual data access. Laura C. Esmail, PhD; Philipp Kapp, MSc; Rouba Assi, PharmD, MSc; Julie Wood, BA; Gabriela Regan, BA; Philippe Ravaud, MD, PhD; Isabelle Boutron, MD, PhD
See More About
Select your interests.
Customize your JAMA Network experience by selecting one or more topics from the list below.
- Academic Medicine
- Acid Base, Electrolytes, Fluids
- Allergy and Clinical Immunology
- American Indian or Alaska Natives
- Anesthesiology
- Anticoagulation
- Art and Images in Psychiatry
- Artificial Intelligence
- Assisted Reproduction
- Bleeding and Transfusion
- Caring for the Critically Ill Patient
- Challenges in Clinical Electrocardiography
- Climate and Health
- Climate Change
- Clinical Challenge
- Clinical Decision Support
- Clinical Implications of Basic Neuroscience
- Clinical Pharmacy and Pharmacology
- Complementary and Alternative Medicine
- Consensus Statements
- Coronavirus (COVID-19)
- Critical Care Medicine
- Cultural Competency
- Dental Medicine
- Dermatology
- Diabetes and Endocrinology
- Diagnostic Test Interpretation
- Drug Development
- Electronic Health Records
- Emergency Medicine
- End of Life, Hospice, Palliative Care
- Environmental Health
- Equity, Diversity, and Inclusion
- Facial Plastic Surgery
- Gastroenterology and Hepatology
- Genetics and Genomics
- Genomics and Precision Health
- Global Health
- Guide to Statistics and Methods
- Hair Disorders
- Health Care Delivery Models
- Health Care Economics, Insurance, Payment
- Health Care Quality
- Health Care Reform
- Health Care Safety
- Health Care Workforce
- Health Disparities
- Health Inequities
- Health Policy
- Health Systems Science
- History of Medicine
- Hypertension
- Images in Neurology
- Implementation Science
- Infectious Diseases
- Innovations in Health Care Delivery
- JAMA Infographic
- Law and Medicine
- Leading Change
- Less is More
- LGBTQIA Medicine
- Lifestyle Behaviors
- Medical Coding
- Medical Devices and Equipment
- Medical Education
- Medical Education and Training
- Medical Journals and Publishing
- Mobile Health and Telemedicine
- Narrative Medicine
- Neuroscience and Psychiatry
- Notable Notes
- Nutrition, Obesity, Exercise
- Obstetrics and Gynecology
- Occupational Health
- Ophthalmology
- Orthopedics
- Otolaryngology
- Pain Medicine
- Palliative Care
- Pathology and Laboratory Medicine
- Patient Care
- Patient Information
- Performance Improvement
- Performance Measures
- Perioperative Care and Consultation
- Pharmacoeconomics
- Pharmacoepidemiology
- Pharmacogenetics
- Pharmacy and Clinical Pharmacology
- Physical Medicine and Rehabilitation
- Physical Therapy
- Physician Leadership
- Population Health
- Primary Care
- Professional Well-being
- Professionalism
- Psychiatry and Behavioral Health
- Public Health
- Pulmonary Medicine
- Regulatory Agencies
- Reproductive Health
- Research, Methods, Statistics
- Resuscitation
- Rheumatology
- Risk Management
- Scientific Discovery and the Future of Medicine
- Shared Decision Making and Communication
- Sleep Medicine
- Sports Medicine
- Stem Cell Transplantation
- Substance Use and Addiction Medicine
- Surgical Innovation
- Surgical Pearls
- Teachable Moment
- Technology and Finance
- The Art of JAMA
- The Arts and Medicine
- The Rational Clinical Examination
- Tobacco and e-Cigarettes
- Translational Medicine
- Trauma and Injury
- Treatment Adherence
- Ultrasonography
- Users' Guide to the Medical Literature
- Vaccination
- Venous Thromboembolism
- Veterans Health
- Women's Health
- Workflow and Process
- Wound Care, Infection, Healing
Others Also Liked
- Download PDF
- X Facebook More LinkedIn
- CME & MOC
Cruz Rivera S , Aiyegbusi OL , Ives J, et al. Ethical Considerations for the Inclusion of Patient-Reported Outcomes in Clinical Research : The PRO Ethics Guidelines . JAMA. 2022;327(19):1910–1919. doi:10.1001/jama.2022.6421
Manage citations:
© 2024
Ethical Considerations for the Inclusion of Patient-Reported Outcomes in Clinical Research : The PRO Ethics Guidelines
- 1 Centre for Patient Reported Outcomes Research, Institute of Applied Health Research, University of Birmingham, Birmingham, United Kingdom
- 2 Birmingham Health Partners Centre for Regulatory Science and Innovation, University of Birmingham, Birmingham, United Kingdom
- 3 DEMAND Hub, University of Birmingham, Birmingham, United Kingdom
- 4 National Institute for Health and Care Research (NIHR) Applied Research Centre West Midlands, Birmingham, United Kingdom
- 5 Centre for Ethics in Medicine, Bristol Medical School, University of Bristol, Bristol, United Kingdom
- 6 Warwick Medical School, University of Warwick, Coventry, United Kingdom
- 7 NHMRC Clinical Trials Centre, Faculty of Medicine and Health, The University of Sydney, New South Wales, Australia
- 8 School of Population and Global Health, McGill University, Montreal, Quebec, Canada
- 9 A J Hunn Associates, London, United Kingdom
- 10 PRO Center of Excellence, Global Commercial Strategy Organization, Janssen Global Services, Warrington, United Kingdom
- 11 Division of Pediatric Haematology-Oncology, IWK Health Care Centre, Dalhousie University, Halifax, Nova Scotia, Canada
- 12 Observational and Pragmatic Research Institute, Midview City, Singapore
- 13 US Food and Drug Administration, Silver Spring, Maryland
- 14 European Organization for Research and Treatment of Cancer, Brussels, Belgium
- 15 Medicines and Healthcare Products Regulatory Agency, London, United Kingdom
- 16 Health Research Authority, London, United Kingdom
- 17 Patient partner, University of Birmingham, Birmingham, United Kingdom
- 18 Fast Track Research Ethics Committee, Health Research Authority, London, United Kingdom
- 19 JAMA Network, Chicago, Illinois
- 20 RTI Health Solutions, Research Triangle, North Carolina
- 21 Aparito Limited, Wrexham, Wales, United Kingdom
- 22 MRC-Versus Arthritis Centre for Musculoskeletal Ageing Research, Institute of Inflammation and Ageing, University of Birmingham, Birmingham, United Kingdom
- 23 NIHR Birmingham Biomedical Research Centre, University Hospital Birmingham and University of Birmingham, Birmingham, United Kingdom
- 24 NIHR Surgical Reconstruction and Microbiology Research Centre, University Hospital Birmingham and University of Birmingham, Birmingham, United Kingdom
- 25 Aetion, Washington, DC
- 26 Department of Clinical Trial Data Management, Graduate School of Medicine, The University of Tokyo, Bunkyo-ku, Tokyo, Japan
- 27 Global Patient-Centred Outcomes Research & Policy, UCB, Belgium, Brussels
- 28 Nature Medicine , New York, New York
- 29 Department of Cardiology, Odense University Hospital, Odense, Denmark
- 30 Clinical Institute, University of Southern Denmark, Odense, Denmark
- 31 Department of Medical Social Sciences, Northwestern University Feinberg School of Medicine, Chicago, Illinois
- 32 Menzies Institute for Medical Research, University of Tasmania, Tasmania, Australia
- 33 Department of Health Services, Policy and Practice, Brown University School of Public Health, Providence, Rhode Island
- 34 University of North Carolina at Chapel Hill
- 35 Consumer Forum, National Cancer Research Institute, London, United Kingdom
- 36 Patient Involvement Network, Health Research Authority, London, United Kingdom
- 37 Clinical Trials and Statistics Unit, The Institute of Cancer Research, London, United Kingdom
- 38 Health Data Research United Kingdom, London, United Kingdom
- 39 UK SPINE, University of Birmingham, Birmingham, United Kingdom
- Original Investigation Electronic Symptom Monitoring With Patient-Reported Surveys and Outcomes Among Patients With Metastatic Cancer Ethan Basch, MD, MSc; Deborah Schrag, MD, MPH; Sydney Henson, BS; Jennifer Jansen, MPH; Brenda Ginos, MS; Angela M. Stover, PhD; Philip Carr, MPH; Patricia A. Spears, BS; Mattias Jonsson, BA; Allison M. Deal, MS; Antonia V. Bennett, PhD; Gita Thanarajasingam, MD; Lauren J. Rogak, MA; Bryce B. Reeve, PhD; Claire Snyder, PhD; Deborah Bruner, PhD; David Cella, PhD; Lisa A. Kottschade, MSN; Jane Perlmutter, PhD; Cindy Geoghegan, MA; Cleo A. Samuel-Ryals, PhD; Barbara Given, PhD; Gina L. Mazza, PhD; Robert Miller, MD; Jon F. Strasser, MD; Dylan M. Zylla, MD; Anna Weiss, MD; Victoria S. Blinder, MD; Amylou C. Dueck, PhD JAMA
- JAMA Guide to Statistics and Methods Patient-Reported Outcome Measures in Clinical Research Kevin P. Weinfurt, PhD; Bryce B. Reeve, PhD JAMA
- Viewpoint Stability of Changes in Health Status Muhammad Shahzeb Khan, MD, MSc; Javed Butler, MD, MPH, MBA JAMA
- Research Letter Sharing of Individual Patient-Level Data by Trialists of Randomized Clinical Trials Laura C. Esmail, PhD; Philipp Kapp, MSc; Rouba Assi, PharmD, MSc; Julie Wood, BA; Gabriela Regan, BA; Philippe Ravaud, MD, PhD; Isabelle Boutron, MD, PhD JAMA
Question What ethical considerations should be considered by researchers, research ethics committees, and funders when conducting or reviewing patient-reported outcome (PRO) clinical research?
Findings An international consensus Delphi process was developed according to the Enhancing the Quality and Transparency of Health Research (EQUATOR) methodology; 14 items addressing ethical considerations were recommended for inclusion in the PRO ethics guidelines.
Meaning Addressing the items in the PRO ethics guidelines has the potential to improve the quality of PRO in clinical research while promoting and protecting participant autonomy and protecting participant and researcher welfare.
Importance Patient-reported outcomes (PROs) can inform health care decisions, regulatory decisions, and health care policy. They also can be used for audit/benchmarking and monitoring symptoms to provide timely care tailored to individual needs. However, several ethical issues have been raised in relation to PRO use.
Objective To develop international, consensus-based, PRO-specific ethical guidelines for clinical research.
Evidence Review The PRO ethics guidelines were developed following the Enhancing the Quality and Transparency of Health Research (EQUATOR) Network’s guideline development framework. This included a systematic review of the ethical implications of PROs in clinical research. The databases MEDLINE (Ovid), Embase, AMED, and CINAHL were searched from inception until March 2020. The keywords patient reported outcome* and ethic* were used to search the databases. Two reviewers independently conducted title and abstract screening before full-text screening to determine eligibility. The review was supplemented by the SPIRIT-PRO Extension recommendations for trial protocol. Subsequently, a 2-round international Delphi process (n = 96 participants; May and August 2021) and a consensus meeting (n = 25 international participants; October 2021) were held. Prior to voting, consensus meeting participants were provided with a summary of the Delphi process results and information on whether the items aligned with existing ethical guidance.
Findings Twenty-three items were considered in the first round of the Delphi process: 6 relevant candidate items from the systematic review and 17 additional items drawn from the SPIRIT-PRO Extension. Ninety-six international participants voted on the relevant importance of each item for inclusion in ethical guidelines and 12 additional items were recommended for inclusion in round 2 of the Delphi (35 items in total). Fourteen items were recommended for inclusion at the consensus meeting (n = 25 participants). The final wording of the PRO ethical guidelines was agreed on by consensus meeting participants with input from 6 additional individuals. Included items focused on PRO-specific ethical issues relating to research rationale, objectives, eligibility requirements, PRO concepts and domains, PRO assessment schedules, sample size, PRO data monitoring, barriers to PRO completion, participant acceptability and burden, administration of PRO questionnaires for participants who are unable to self-report PRO data, input on PRO strategy by patient partners or members of the public, avoiding missing data, and dissemination plans.
Conclusions and Relevance The PRO ethics guidelines provide recommendations for ethical issues that should be addressed in PRO clinical research. Addressing ethical issues of PRO clinical research has the potential to ensure high-quality PRO data while minimizing participant risk, burden, and harm and protecting participant and researcher welfare.
Patient-reported outcomes (PROs) are used in clinical research and routine care to provide information on the physical, functional, and psychological effects of disease and treatment from the patient perspective. 1 PRO data can inform health care decisions, regulatory decisions, health care policy, and cost-effectiveness analyses. PROs can also be used for audit/benchmarking and monitoring of symptoms to provide timely care tailored to individual needs. 1 , 2 Notwithstanding the potential benefits of incorporating PROs in research and routine practice, ethical considerations have been highlighted. 3 For example, the PRO content of clinical trial protocols and reporting of PRO results is commonly inadequate. A 2019 evaluation of 160 cancer trials showed nearly 50 000 participants were included in studies that failed to publish their PRO data. 4
The increasing use of PROs may lead to uncertainties for patients about why data are being collected and used. There is a lack of guidance on how research personnel should manage situations in which PRO data reveal concerning levels of psychological distress or physical symptoms. 5 If concerning data are not managed appropriately, those data could lead to suboptimal participant care or biased trial results. 6 In addition, PRO research may not reflect the perspectives of underserved groups such as older individuals, socioeconomically disadvantaged populations, and racial and ethnic minority groups, which could threaten the scientific validity of results. 3 , 7
Ethical issues should be resolved with justifications that use established principles, theories, and values, as well as consider individual and societal welfare. 3 In 2018, the SPIRIT (Standard Protocol Items: Recommendations for Interventional Trials)-PRO Extension was developed to provide PRO trial protocol guidance. 8 These guidelines were not, however, developed specifically for the use of research ethics committees (RECs) and limited attention has been given to the ethical dimensions of PROs in clinical research. 7 Thus, there is a need to develop ethical guidelines to address this. The aim of this international effort was to develop consensus-based guidelines for the specific use of PROs in clinical research.
The PRO ethics guidelines were developed through an international Delphi process following the Enhancing the Quality and Transparency of Health Research (EQUATOR) Network’s framework for guideline development ( Figure ). 9
The PRO Ethics Steering Group, formed by 11 international experts with patient and public involvement (Acknowledgements in the Supplement ), was established to oversee the design and conduct of the study.
Ethical approval was given by the University of Birmingham Ethical Review Board (ERN_21-0075).
Candidate items were identified by the steering group from the SPIRIT-PRO Extension 8 guidelines and Supplement 3 of the accompanying SPIRIT-PRO Extension article. 8 Explanation of the candidate items was derived from lay terminology of the SPIRIT-PRO Extension. 10 The candidate items were supplemented with items generated from a systematic review of articles describing the ethical implications of PROs in clinical research. The protocol for the systematic review was registered on PROSPERO (registration No. CRD42020176177). The databases MEDLINE (Ovid), EMBASE, Allied and Complementary Medicine Database (AMED), and CINAHL (Cumulative Index to Nursing and Allied Health Literature) Plus were searched from inception until March 2020 with the keywords patient reported outcome* and ethic* .
Publications were deemed eligible if they discussed ethical implications and/or guidance in the context of PRO clinical trials research, routine clinical practice, and broader PRO research. Two reviewers (SCR and OLA) independently conducted title and abstract screening before full-text screening to determine eligibility. Discrepancies were resolved through the involvement of a third reviewer (MJC). Text excerpts on ethical considerations of PRO research from the included studies were independently extracted by the 2 investigators (SCR and OLA) into a qualitative data analysis software package (NVivo 12; QSR International). Both reviewers independently generated categories and themes under the thematic analysis approach. The review identified 14 relevant articles, including qualitative reports, opinion and debate articles, and special communications that discussed the ethical implications of PRO research.
Based on the review, 6 candidate items were identified, and 17 items were drawn from the SPIRIT-PRO Extension guidelines 8 and Supplement 3 of that article.
In 2021, 201 international multidisciplinary individuals with interest in PRO research were invited to participate in the online Delphi process to vote on the candidate items and propose additional items. These participants comprised individuals responsible for developing PRO research submissions for ethical review, those undertaking ethical review, or using of data arising from PRO research. Potential participants were identified and contacted via the PRO Ethics Operations Group (S.C.R., M.J.C., O.L.A., A.P.D.) and the Health Research Authority (HRA). A snowballing technique and social media (LinkedIn and Twitter) were used to identify further participants. Participant characteristics are described in eTable 1 in the Supplement . DelphiManager software (version 5.0), developed and maintained by the COMET (Core Outcome Measures in Effectiveness Trials) Initiative, was used to undertake the 2 Delphi surveys. 11
Participants were provided with written information about the study prior to consenting to participate. Participants voted anonymously on a 9-point scale (1-3: not important; 4-6: important but not critical; and 7-9: important and critical) on the importance of the 23 items presented. Ninety-six responses were received for round 1 of the Delphi and 85 responses (89% of participants from round 1) were received for round 2. Participants were advised if they did not complete round 2, their round 1 responses would be retained. During round 1, participants had the option to suggest additional items. During round 2, 12 additional items were included. Anonymized item-level round 1 scores per participant group were presented to Delphi panelists for their consideration prior to round 2 voting.
The operations group mapped the 35 candidate PRO ethics items to existing HRA guidance from the UK, as an initial indicator of what may already be covered in existing ethics guidance, 12 removing duplicates and revising wording to aid clarification. The operations group presented the consensus delegates with recommendations for the inclusion or exclusion of items based on the decision tree (eFigure in the Supplement ). The COMET Initiative guidance informed the inclusion criteria (eMethods in the Supplement ). 13
An online consensus meeting took place in October 2021 hosted by the University of Birmingham, UK. Twenty-five international participants purposively selected from the Delphi survey attended the consensus meeting, comprising 7 clinical trialists/health academic researchers, 4 ethicists/members of an ethical review panel, 2 health care professionals, 3 PRO researchers from industry, 2 journal editors, 4 patients and members of the public, 1 policy maker, 1 regulator, and 1 bioethicist (eTable 1 in the Supplement ). Delegates were presented with candidate items and anonymously voted using the Zoom poll tool. Participants had the following voting options: include, exclude, or further discussion required (see the Participation in the Voting Process section, eMethods in the Supplement , for further details).
The aim of the meeting was to seek consensus on the content of the PRO ethics guidelines. Consensus panelists considered the focus of the guidelines and agreed that the guidelines covered ethical considerations when undertaking PRO clinical research. In addition, participants discussed the wording and explanatory text of each item. A threshold of 70% or more was prespecified to demonstrate consensus when voting on the items (see the Consensus Meeting section, eMethods in the Supplement , for further details). The items were presented alongside the overall Delphi score and the number of participant groups whereby 70% or more of respondents scored an item as important and critical.
Following the consensus meeting, attendees commented on the wording and agreed on the final version of the PRO ethics guidelines. Final edits were made to improve clarity and were approved by the steering group and patient partners. The eMethods section in the Supplement provides further information on methods.
The final PRO ethics guidelines identified 14 key questions that capture core ethical issues ( Table ). The items incorporated content from 14 of the 35 original candidate items, comprising 6 items that were merged during the consensus meeting and 8 items that were not modified (see eTables 2 and 3a and 3b in the Supplement ). Further details about the 21 excluded items are presented in eTables 4a and 4b in the Supplement . An explanation describing each item with supporting evidence is presented below. The items are presented in accordance with SPIRIT-PRO Extension subheadings and findings from the systematic review.
Explanation: Evidence suggests that many trials include PROs without specifying the PRO-specific research question and without a rationale or reference to PROs in related studies. 4 , 14 , 15 Researchers should carefully consider the PRO-specific research question to inform the selection of measures and methodological approach to help ensure results are meaningful. 8 In addition, patients and research personnel should understand why PRO data are being collected and how their data will be used, and this should be communicated effectively. 4 , 14 , 15 This can help build trust, particularly when participants may share potentially sensitive information. Why data are being collected and how these data will be used should be clearly explained in the information sheet, by research personnel, or both during the consent process.
Explanation: Clearly defined PRO objectives and hypotheses inform study design, including the selection of key PRO concepts and measures, time points for assessment, and analyses. 16 Poorly defined PRO objectives or hypotheses may affect the quality of research design and reporting. Poor science undermines participant consent (failing to respect autonomy) and exposes participants to unnecessary risk or burden because the results are ultimately not usable or not generalizable.
Explanation: Researchers should consider PRO-specific eligibility requirements at the design stage of the study and robustly justify excluding a subpopulation. It would undermine the principle of justice to exclude eligible people either directly or indirectly (eg, as a result of a failure to consider PRO accessibility or other equity, diversity, and inclusion issues). 17
Explanation: The PRO concept and analysis metric should be clearly outlined and aligned with the PRO objectives and hypothesis to ensure that they capture outcomes that matter to patients and other key interested groups, such as clinicians, regulators, and policy makers. Defining and justifying the selection of PRO instrument(s) are important aspects of ethical research. If possible, the PRO measure should be validated in the target population. The number of questionnaires used, acceptability of the questions, and participant burden should be considered carefully. PRO measures ideally should be used in accordance with existing user manuals to promote data quality and ensure standardized scoring. 8 When a PRO is being considered for a new population, representative patient input should be obtained about the suitability and appropriateness of the questions to determine whether the questions are relevant to the target population. 18
Explanation: Providing the schedule of PRO assessments in the study protocol and participant information sheet is the first step to ensuring potential participants understand the commitment and effort involved in taking part in the PRO study. A robust consent process includes information provision and checks on understanding. A poor process compromises respect for participant autonomy. 19 , 20
Explanation: Exposing participants to the risks and burdens of PRO research is only justifiable if these are outweighed by the potential value of the PRO data. A sample size that is too small may produce inconclusive and, therefore, not valuable results. A sample size that is too large will expose more participants than necessary to risks and burdens and incur unnecessary costs. 21 Item 14 of the SPIRIT-PRO Extension indicates that if PROs are the primary outcome of a study, a priori sample size calculation should be provided for that specific end point. If PROs are a secondary outcome, the sample size should provide enough power to test the principal PRO hypothesis. 8 This would not be required for exploratory PRO end points.
Explanation : Research personnel should understand how and where PRO data will be collected, and clear communication of this to potential participants is an essential component of a robust informed consent process. The mode(s) of administration should be influenced by the setting in which PRO data will be collected (eg, telephone or electronic completion may be more feasible from home) and the needs of the target population. 22 Ideally, participants from the target population would provide input on modes. Offering alternative modes of completion may help improve response rates and promote inclusivity and equity—all of which improve the quality of the results. 23 Item 18a(ii) of the SPIRIT-PRO Extension provides further information regarding the modes of PRO administration and setting for PRO randomized clinical trials. 8
Explanation: Responding to PRO alerts (concerning levels of psychological distress or physical symptoms that require timely response) 6 may protect the safety and welfare of participants, 17 which is an important ethical consideration. The research protocol should state whether, why, and by whom PRO data will be monitored during the study and this information should be shared with participants. 5 , 6 In low-risk studies in which alerts for concerning symptoms are not anticipated, PRO monitoring may not be necessary. Similarly, protocols should state whether research data will be shared with the patient’s care team or entered in the electronic medical record. Alternative support mechanisms (eg, 24-hour helpline) for participants should be outlined. All research personnel involved in the management of PRO alerts should receive appropriate training and have clear pathways for support. 24 , 25 Evidence suggests research personnel handle such data inconsistently, which may lead to inequitable patient care, cointervention bias, and confusion. 6 In addition, personnel in charge of collecting PRO data may feel emotional and/or ethical burden while dealing with concerning PRO data (eg, reports from trial participants of low self-esteem, depression, or risk of self-harm or suicide). 25
Explanation: PRO protocols should promote participant inclusivity while recruiting a diverse population that is representative of patients with the condition of interest. Barriers to participation, such as access to technology in rural areas, areas of socioeconomic disadvantage, or both, as well as disability, language, and cultural requirements, should be addressed to promote fairness and ensure results are as accurate and generalizable as possible. 26 For example, a clinical trial of adults receiving chemotherapy at 50 community cancer centers promoted inclusivity by offering internet and no-internet (automated telephone call) options to complete PROs remotely. Thirty-five percent of the participants chose the automated call (no-internet) option vs 65% who chose internet-based completion. 27 Without an alternative PRO mode, more than one-third of the vulnerable population may have been excluded.
Researchers may consider different modes of completion (item 7) to promote inclusivity and should be explicit about how the PRO strategy promotes or hinders the goal of recruiting a diverse sample representative of the target population. For instance, trials involving participants with different languages require the availability of validated language and culturally adapted PRO questionnaires, while some participants may need physical help or other types of assistance in responding (eg, turning pages, holding a pen, assistance with a telephone or computer keyboard). 8 , 16 , 24
Explanation: PROs should be acceptable to the population in which they will be administered, both in terms of the questions they ask and the overall burden to the patient (eg, is the completion time for the PRO measure acceptable). 28 The degree of participant burden depends on the frequency and timing of PRO assessments and on issues such as participant cognition, illness severity, treatment toxicity, and literacy. 16 Researchers should consider issues such as whether the questionnaire(s) capture important and relevant concepts to interested groups (such as overall health-related quality of life, specific domain or symptoms as described in item 4) and whether PROs include overlapping content and/or particularly sensitive questions. It is also important to consider the length, number of questionnaires, and end points, with respect to burden for subgroups of participants and if the mode of delivery (item 7) and schedule of assessments (item 5) are appropriate. If researchers demonstrate acceptable participant burden via robust involvement from representatives of the target patient population in the PRO selection process, RECs should not override the PRO strategy without strong ethical justification (eg, RECs should avoid automatically rejecting a proposal with a large number of PROs if justification is provided).
Short questionnaires minimize participant burden and assure greater completeness of PRO data while minimizing missing data. 29 However, patient input during the selection of PRO measures is key because participants may be willing to complete lengthy questionnaires if they understand the value of data collection and how the data will be used. 30 Thus, the views of the affected population are authoritative in this regard. Failure to seek participant input to core design issues, such as concepts to measure that matter most to patients, selection of questionnaires, time points, and mode of assessment, may lead to poor concordance, and therefore flawed results that cannot inform clinical practice. Poorly designed studies mislead participants who participate to help others and misuse research resources.
Explanation: It is well recognized in research governance that participants who lack capacity (eg, young children and adults who are cognitively impaired) are potentially vulnerable, and their interests in the context of research need to be protected. However, it is also important that such people are not unjustifiably excluded from relevant research. PRO research needs to meet the same well-defined standards.
These individuals may require a proxy: someone else to report the participant’s outcomes on their behalf. 8 This is different to assisting a participant to document their own answers (see item 9). 31 , 32 The correct administration of PRO tools when proxies need to be used contributes to the collection of robust and reliable data. The justification for including vulnerable participants in research is that it will either benefit them directly or it will benefit the population to which they belong. 33
In many research contexts, it is reasonable to anticipate the need for proxy response throughout all or some of the research (although the possibility can never be excluded) and this should be clearly documented in the research protocol. Researchers should be aware that proxy reporting is acceptable in some contexts and not in others. For example, the European Medicines Agency discourages proxy reporting because their data are often subject to biases and should only be used if it is the only effective means of obtaining vital information that might otherwise be lost. 28 The US Food and Drug Administration also discourages the use of proxy-reported outcomes to inform labeling claims, recommending observer reports for observable phenomenon only (eg, vomiting, but not nausea) instead. 16 However, in palliative care, collecting both proxy and observer measures is acceptable. 34
It is important to recognize that lack or loss of capacity to consent to research participation will not always be accompanied by an inability to self-complete PROs (with or without assistance), and appropriate support for such participants should be specified.
Explanation: Patient and public involvement refers to the partnership between patients, members of the public, and researchers in the codevelopment of research. 35 Patients and members of the public have unique insight derived from their lived experiences making research more relevant and enhancing the design, conduct, and quality of the research. 36 - 38 Incorporating these insights into research can make it prima facie more ethical in 2 ways: by democratizing the research agenda and/or helping to improve participant-facing documents and processes. 39
The inclusion of patient and/or public involvement should be considered best practice during the study design stage. Involvement of individuals with the disease can provide valuable insights into their lived experience and help ensure the research is relevant to their needs and acceptable, while public involvement may generate broader insights from a societal perspective. In addition, their inclusion should be integral to all the stages of research. The inclusion of patient involvement, public involvement, or both in the development of the PRO strategy may help to ensure that research measures what matters to patients, thereby maximizing its beneficial effect. It is also the best means of ensuring that PRO tools, and how they are administered, are acceptable (see item 10), and thereby may be influential in maximizing the response rate (see item 13). For example, recent patient involvement in the Therapies for Long COVID Study has led to the development of a new Symptom Burden Questionnaire because existing measures were felt to omit key symptoms experienced by those with the condition. 40
Explanation: Missing PRO data are a major problem in clinical research. 23 , 41 Missing data are normally caused by a combination of factors relating to methodology, logistic, administrative, and patient-related issues. 41 Protocols should describe how missing data will be minimized. Missing PRO data can complicate interpretation, lead to invalid conclusions, or may mean that the PRO data are not published. 4 , 42 , 43 When this occurs, it undermines the consent of participants who took part in the study and wastes research resources.
Although not all missing PRO data can be avoided, different strategies exist to mitigate this problem. 23 Specific recommendations related to data collection and management include using the minimum number of questionnaires appropriate to address the PRO research question, establishing standardized and documented PRO administration procedures, engaging and educating participants in the study by providing updates or incentives, using active quality assurance measures (such as monitoring of completion rates, reminders for upcoming or missed assessments), appointing a dedicated staff member responsible for PRO assessment at each center, training staff, and offering alternative modes of administration. 23 , 31 Reminders, notifications, or follow-up calls may be used to minimize missing data. Although different strategies exist to minimize avoidable PRO missing data, participants should be notified and provide consent, prior to accepting being part of the study, about the mechanisms the study will follow.
Explanation: The dissemination of PRO findings is essential to achieve beneficial outcomes. PRO data are, however, commonly omitted from primary and secondary publications. 4 Failing to report PRO data could limit the interpretation of the results and may hinder the translation of PRO findings into clinical practice, resulting in lost opportunities to benefit patients and the perpetuation of harmful practices. Failure to disseminate PRO findings is disrespectful of participants’ time, effort, and contribution to research. It may also undermine participants’ consent if they were misinformed about dissemination plans. 43 Sharing a summary of the PRO research results in accessible plain language for use by patients, participants, and members of the public promotes autonomy by empowering patients in shared decision-making around their care. 44
It is recommended that PRO findings should be incorporated into the main research publication or reported in a secondary publication providing a detailed explanation of the PRO data. 45 The CONSORT-PRO Extension guideline was developed to address the reporting of PRO trial data. The CONSORT-PRO provides evidence-based recommendations to improve completeness of reporting randomized clinical trials with either a primary or secondary PRO end point. 46
The Table shows an implementation tool for PRO researchers and RECs to be completed by research teams preparing PRO research or by reviewers.
The PRO ethics guidelines provide international consensus-based recommendations on questions that should be asked of a study’s design to facilitate the evaluation of its ethical acceptability. The guidelines highlight the ethical imperative to conduct robust science and the ethical issues to consider in the design and review of PRO clinical research. While a number of ethical issues identified are not unique to PROs and apply to research more widely, they raise particular challenges in the context of PROs, which is the focus of the work developed. The PRO ethics guidelines comprise 14 items to consider for use alongside the existing SPIRIT-PRO and CONSORT-PRO Extension guidelines 8 , 46 and other ethical recommendations relevant to the jurisdiction of interest. 12 , 47 , 48
The guidelines do not aim to mandate how ethical research should look, nor to mandate the correct response to the questions it asks. Instead, the guidelines aim to highlight issues that should be considered by research groups and ethics committees, including patients, research participants, and the public.
The recommendations within the PRO ethics guidelines reflect widely accepted ethical norms encapsulated in instruments such as the Declaration of Helsinki, 49 the Belmont Report, 50 and the Council for International Organizations of Medical Sciences guidelines. 51 The recommendations are in line with the 3 principles of respect of persons, concern for welfare, and justice outlined in the Tri-Council Policy Statement: Ethical Conduct for Research Involving Humans 48 and the widely used 4 principles of biomedical ethics: autonomy, justice, beneficence, and nonmaleficence. 19 As such, the guiding ethical questions presented here do not set out any new ethical ideas, but rather specify widely accepted norms in the context of PROs and frame them in a way that is accessible to PRO researchers and useful for reviewers of PRO research.
The use of the PRO ethics guidelines has the potential to reduce participant risk and burden. In addition, addressing the items of the PRO ethics guidelines may help promote and protect participant autonomy and the welfare of participants and researchers. Furthermore, it may promote inclusive, equitable PRO research; the sharing of PRO research findings with participants and patients; and minimization of research waste ( Box ).
Aims of the PRO Ethics Guidelines
Maximize beneficial effect from research resources
Promote and protect participant autonomy
Protect participant research welfare
Promote accessible research
Minimize participant burden and harm
Minimize participant risk
Promote high-quality research
Disseminate PRO research
The Table provides an implementation tool for PRO researchers to reflect how each item has been addressed prior to ethical submission and for RECs to make notes on the research submitted and discuss in detail any relevant points at the ethics meeting. This tool is a starting point and can be tailored according to the users’ needs. Collaborations with national and international networks are being planned to promote the implementation of the PRO ethics guidelines.
This study has several limitations. First, the review identified only limited literature on which to base items for inclusion in the Delphi. Therefore, some relevant candidate items may not have been included; however, additional items were proposed by the steering group, and further items were informed by the SPIRIT-PRO Extension work, based on an extensive review of PRO protocol guidance. Furthermore, participants had the opportunity to propose additional items during round 1 of the Delphi process. Second, only literature available until March 2020 was considered in development of the guidelines. However, an updated search was performed on March 23, 2022; an additional 569 articles were screened and no further relevant literature was identified. Third, because participants ranked items according to their general importance, it is possible that some items might be less relevant for certain types of trials.
The PRO ethics guidelines provide recommendations for ethical issues that should be addressed in PRO clinical research. Addressing ethical issues of PRO clinical research has the potential to ensure high-quality PRO data while minimizing participant risk, burden, and harm and protecting participant and researcher welfare.
Corresponding Author: Melanie J. Calvert, PhD, Centre for Patient Reported Outcome Research, Institute of Applied Health Research, University of Birmingham, Edgbaston, Birmingham B15 2TT, England ( [email protected] ).
Accepted for Publication: April 5, 2022.
Author Contributions: Drs Cruz Rivera and Calvert had full access to all of the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.
Concept and design: Cruz Rivera, Aiyegbusi, Ives, Mercieca-Bebber, Hunn, Bhatnagar, Bottomley, Campbell, Collis, Golub, von Hildebrand, Mahendraratnam, Wilson, Stover, Calvert.
Acquisition, analysis, or interpretation of data: Cruz Rivera, Aiyegbusi, Ives, Draper, Mercieca-Bebber, Ells, Hunn, Scott, Fernandez, Dickens, Anderson, Bottomley, Campbell, Collett, Craig, Davies, Gosden, Gnanasakthy, Haf Davies, Lord, Miyaji, Monteiro, Morel, Zwisler, Peipert, Roydhouse, Wilson, Yap, Calvert.
Drafting of the manuscript: Cruz Rivera, Draper, Hunn, Dickens, Anderson, Campbell, Collett, Collis, Davies, Gosden, Gnanasakthy, Haf Davies, Monteiro, Stover, Calvert.
Critical revision of the manuscript for important intellectual content: Aiyegbusi, Ives, Draper, Mercieca-Bebber, Ells, Hunn, Scott, Fernandez, Dickens, Anderson, Bhatnagar, Bottomley, Campbell, Craig, Golub, von Hildebrand, Lord, Mahendraratnam, Miyaji, Monteiro, Morel, Zwisler, Peipert, Roydhouse, Wilson, Stover, Yap, Calvert.
Statistical analysis: Bottomley, Haf Davies, Calvert.
Obtained funding: Mercieca-Bebber, Calvert.
Administrative, technical, or material support: Aiyegbusi, Dickens, Gosden, Haf Davies, von Hildebrand, Mahendraratnam.
Supervision: Calvert.
Other - patient partner: Collis.
Other - patient viewpoint and contributor: Wilson.
Other - statistical advice: Yap.
Other - consensus meeting participation: Roydhouse.
Other - Providing options and opinions: Haf Davies.
Other - helped with formulation of interpretation and analysis regarding ethical values/principles: Draper.
Other - Delphi panel member, reviewed and commented on the study documents and publications: Scott.
Conflict of Interest Disclosures: Dr Cruz Rivera reported receiving funding from UK SPINE and European Regional Development Fund–Demand Hub and personal fees from Merck. Dr Aiyegbusi reported receiving grants from the National Institute for Health and Care Research (NIHR) Birmingham Biomedical Research Centre, NIHR Applied Research Collaboration West Midlands, UK Research and Innovation (UKRI), Health Foundation, Janssen, Gilead, and GlaxoSmithKline and personal fees from Gilead Sciences Ltd, Merck, and GlaxoSmithKline outside the submitted work. Dr Draper reported receiving unrelated research funding from UK SPINE (UKRI), AHRC, and the University of Warwick and being a member of the Defence Medical Services ethics committee, Birmingham Women’s and Children’s NHS Foundation Trust clinical ethics committee, and NHS Blood and Transplant Deceased Donor Family Tissue Advisory Group. Dr Scott reported receiving a pension from Janssen and holding stock in Johnson & Johnson. Drs Ells and Fernandez are members of the Canadian Interagency Panel on Research Ethics, which is responsible for the interpretation and evolution of the Canadian Tri-Council Policy Statement: Ethical Conduct for Research Involving Humans . Ms Anderson reported receiving funding from the Health Education England/NIHR Integrated Clinical Academic Program Clinical Doctoral Research Fellowship. Dr Haf Davies reported owning an ePRO software platform called Atom5 through Aparito. Dr Lord reported being a member of the Nuffield Bioethics Working Group on the Future of Ageing. Dr Mahendraratnam reported owning stock options at Aetion Inc. Mr Miyaji reported grants (paid to the Department of Clinical Trial Data Management, Graduate School of Medicine, The University of Tokyo) from AC Medical, A2 Healthcare, New Age Trading, Japan Tobacco Inc, Japan Media Corp, Medidata Solutions, Ono Pharmaceutical, FMD K&L Japan, 3H Medi Solution, NOBORI, Medrio Inc, Welby Inc, Nipro Corp, and Intellim and personal fees from Pfizer Japan Inc, Takeda Pharmaceutical Co, Merck, Ayumi Pharmaceutical, and Welby Inc. Dr Morel reported owning shares of UCB Pharma. Dr Zwisler reported being chair of the national clinical coordinating group on PRO in cardiac diseases. Dr Peipert reported receiving unrelated research funding from the National Cancer Institute, the National Institutes of Health, the Food and Drug Administration, the ECOG-ACRIN Medical Research Foundation, the Peter G. Peterson Foundation, Veloxis Pharmaceuticals, Pfizer, and the Northwestern University George M. O’Brien Kidney Core Center. He has received unrelated personal fees from AstraZeneca, IMPAQ International, and FACIT.org; in addition, he is the International Society for Quality of Life Research’s psychometric special interest group chair. Through his institution, he is supported by unrelated grants and contracts from Bristol Myers Squibb, Clovis Oncology, Pfizer, and Veloxis Pharmaceuticals. Dr Roydhouse reported receiving unrelated personal fees in the last 24 months from Amgen. Through her institution, she is supported by an unrelated Select Foundation Fellowship and has received unrelated research funding from the Royal Hobart Hospital and the Food and Drug Administration. Dr Stover reported receiving unrelated consulting fees or speaking honoraria in the last 24 months from Navigating Cancer, Association of Community Cancer Centers, Genentech, Purchaser Business Group on Health, and Henry Ford Cancer Center and unrelated research funding from Sivan Innovation and UroGen Pharma Ltd. Dr Yap reported receiving unrelated consulting fees and speaking honoraria from Faron Pharmaceuticals and Celgene, respectively, and being an expert advisor for the Medicines and Healthcare products Regulatory Agency’s Clinical Trials, Biologicals and Vaccines Expert Advisory Group and a funding panel member for the Medical Research Council Experimental Medicine and Cancer Research UK Clinical Research Committee. Dr Calvert reported serving as director of the Birmingham Health Partners Centre for Regulatory Science and Innovation, director of the Centre for Patient Reported Outcome Research, and an NIHR senior investigator and receiving funding from the NIHR, UK Research and Innovation (UKRI), NIHR Birmingham Biomedical Research Centre, the NIHR Surgical Reconstruction and Microbiology Research Centre, NIHR ARC West Midlands, UK SPINE, European Regional Development Fund–Demand Hub and Health Data Research UK at the University of Birmingham and University Hospitals Birmingham NHS Foundation Trust, Innovate UK (part of UKRI), Macmillan Cancer Support, UCB Pharma, Janssen, GlaxoSmithKline, and Gilead. Dr Calvert has received personal fees from Astellas, Aparito Ltd, CIS Oncology, Takeda, Merck, Daiichi Sankyo, Glaukos, GlaxoSmithKline, and the Patient-Centered Outcomes Research Institute outside the submitted work. In addition, a family member owns shares in GlaxoSmithKline. No other disclosures were reported.
Funding/Support: This work was sponsored by the University of Birmingham, the NIHR Birmingham Biomedical Research Centre, UK Research and Innovation, UK SPINE, and the European Regional Development Fund.
Role of the Funder/Sponsor: The funders had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication. Several authors are employees of the University of Birmingham; however, beyond the declared author contributions, the sponsor had no additional role.
Disclaimer: The views expressed in this article are those of the author(s) and not necessarily those of national agencies (eg, the NIHR, Food and Drug Administration, Medicines and Healthcare products Regulatory Agency, Health Research Authority, Canadian Institutes of Healthcare Research, the Department of Health and Social Care, Canadian Interagency Panel on Research Ethics, or the Canadian Tri-Council Policy Statement 2. Dr Golub is Executive Deputy Editor of JAMA but was not involved in any of the decisions regarding review of the manuscript or its acceptance.
Additional Information: Dr Scott retired from Janssen Global Services in March 2021; however, she was still involved in the development of the guideline until its final stage. Coauthor Amanda Hunn, MA, died February 8, 2022.
- Register for email alerts with links to free full-text articles
- Access PDFs of free articles
- Manage your interests
- Save searches and receive search alerts
Log in using your username and password
- Search More Search for this keyword Advanced search
- Latest content
- Current issue
- BMJ Journals
You are here
- Volume 19, Issue 1
- Patient safety research: an overview of the global evidence
- Article Text
- Article info
- Citation Tools
- Rapid Responses
- Article metrics
- A K Jha 1 , 2 , 3 ,
- N Prasopa-Plaizier 4 ,
- I Larizgoitia 4 ,
- D W Bates 1 , 2
- 1 Department of Health Policy and Management, Harvard School of Public Health, Boston, MA, USA
- 2 Division of General Medicine, Brigham and Women's Hospital, Boston, MA, USA
- 3 The VA Boston Healthcare System, Boston, MA, USA
- 4 WHO World Alliance for Patient Safety, World Health Organization, Geneva, Switzerland
- Correspondence to Dr Ashish K Jha, Harvard School of Public Health, Boston, Massachusetts, USA; ajha{at}hsph.harvard.edu
Background Unsafe medical care may cause substantial morbidity and mortality globally, despite imprecise estimates of the magnitude of the problem. To better understand the extent and nature of the problem of unsafe care, the WHO World Alliance for Patient Safety commissioned an overview of the world's literature on patient safety research.
Methods Major patient safety topics were identified through a consultative and investigative process and were categorised into the framework of structure, process and outcomes of unsafe care. Lead experts examined current evidence and identified major knowledge gaps relating to topics in developing, transitional and developed nations. The report was reviewed by internal and external experts and underwent improvements based on the feedback.
Findings Twenty-three major patient safety topics were examined. Much of the evidence of the outcomes of unsafe care is from developed nations, where prevalence studies demonstrate that between 3% and 16% of hospitalised patients suffer harm from medical care. Data from transitional and developing countries also suggest substantial harm from medical care. However, considerable gaps in knowledge about the structural and process factors that underlie unsafe care globally make solutions difficult to identify, especially in resource-poor settings.
Interpretation Harm from medical care appears to pose a substantial burden to the world's population. However, much of the evidence base comes from developed nations. Understanding the scope of and solutions for unsafe care for the rest of the world is a critical component of delivering safe, effective care to all of the world's citizens.
- Patient safety
- adverse events
- unsafe care
https://doi.org/10.1136/qshc.2008.029165
Statistics from Altmetric.com
Request permissions.
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.
Despite the longstanding principle to “do no harm”, unsafe medical care appears to cause significant morbidity and mortality throughout the world. Although precise estimates are unavailable, prevalence studies from developed nations suggest that a substantial number of hospitalised patients are injured as a direct result of medical care. 1–4 The evidence for harm in the ambulatory setting is much less robust but is likely to be sizeable. 5 6 Generalising from these figures, tens of millions of people suffer injuries and millions likely die due to unsafe medical care. Injuries can occur in association with many medical interventions, from counterfeit or substandard drugs (due to regulatory and oversight failures) to healthcare-associated infections (due to unhygienic practices). Many of these injuries are preventable and, therefore, particularly troubling.
To better understand the causes and impact of the delivery of unsafe medical care from a global perspective, the World Health Organisation (WHO) Patient Safety team convened an ad hoc expert working group to establish priorities for research on patient safety. To help set priorities, the group commissioned a report on the current evidence available. This assessment was done by identifying topics in patient safety, examining related clinical and organisational issues and distinguishing gaps in current knowledge and directions for future research. This paper highlights the key points of the report. The full report, produced by the working group with the support of leading experts, is far more comprehensive 7 and available on the WHO World Alliance for Patient Safety website ( http://www.who.int/patientsafety/research/en/ ).
The group began by identifying the types and causes of adverse events that are particularly harmful to patients. Major patient safety issues were identified using a multi-faceted, iterative approach: we first began with a literature search. We identified the major causes of harm and their underlying causes. We then consulted with experts on the committee as well as with external sources, such as the National Patient Safety Foundation, reviews by the Agency for Healthcare Research and Quality 8 (which is the most comprehensive review of its kind) and epidemiologic studies from several nations including the US, 9 Canada 1 and Australia. 10 We went back to the experts on the committee with the preliminary list of harms and their underlying causes for further feedback. After finalising the list, we shared it with external experts to get any final feedback. With the list of harms and their underlying causes formalised ( table 1 ), we then sought out experts in each individual topic area to write a section of the report on their topic of expertise.
Major patient safety topics
While there is some debate about the relationship between quality and safety, the Institute of Medicine in the US suggests that safety is one critical component of the delivery of high-quality care. 11 Therefore, the committee chose to categorise the patient safety topics identified into structure, process and outcomes. 12 The Agency for Healthcare Research and Quality (AHRQ) defines “structure” as the resources and organisational arrangements in place to deliver care, “process” as the activities of providers for delivering care and “outcomes” as the consequences of clinical activities by providers. 13
Each identified topic was then reviewed in detail by lead experts who were asked to describe the basic epidemiology of the topic (eg, frequency, impact on care and preventability). These experts also identified major gaps in knowledge that need to be addressed through future research. Because healthcare systems, their focus and their impacts likely vary substantially based on the level of a nation's development, we asked experts to specify what is known about the individual topic in developing, transitional and developed nations. Nations were classified based on their category of national income (World Bank 2006 classification 14 ), and nations listed as “high income” were considered developed, “medium income” as transitional and “low income” as developing. Each section was reviewed by at least three other members of the committee for completeness and balance. Finally, the entire report underwent external review by seven international experts (names and affiliations listed in the Acknowledgement section).
In this manuscript, we synthesise the key points from the individual sections and present the major findings.
We identified 23 major patient safety topics for detailed examination (see table 1 ). Of these, eight were classified as structural, five as process and 10 as outcomes. While it was generally easier to separate the outcomes topics from others, the strong link between structural factors that underlie unsafe care and the unsafe processes they create necessarily lead to categorisation of some topics under one heading that could have arguably belonged in the other. The groupings were not meant to be definitive but created a practical framework for approaching the safety topics. Further, the topics covered activities and areas that could impact the health system at many different levels. For example, organisational accidents and failures obviously occur at the provider level and impact individual organisations such as hospitals or doctor practices. Counterfeit and substandard drugs, on the other hand, often represent failure to regulate and safeguard medications at the national level, impacting potentially all providers and patients in the country.
We found that unsafe patient care is common, that it is associated with significant morbidity and mortality throughout the world, and that much of the harm is likely amenable to intervention. Studies from the US suggest that approximately 3% to 4% of hospitalised patients suffer a serious adverse event, 2 3 while studies from other developed nations, using less restrictive definitions, found that between 8% and 16% of hospitalised patients suffer an adverse event. 1 4 A substantial proportion (between 30% and 50%) of adverse events are preventable, and most cause substantial harm, likely contributing to the deaths of tens of thousands of people in each of these nations. Although there have been few systematic prevalence studies in developing or transitional nations, existing evidence from these countries suggests that unsafe medical care is likely to be as—if not more—common 15 16 than in developed nations. For example, one study of Mexican hospitals suggested that nearly one in four hospitalised patients developed a nosocomial infection. 15
The data on structural and process factors that affect patient safety come almost exclusively from a small number of developed nations. This makes understanding the underlying causes of unsafe care or recommending solutions to improve safety in developing and transitional nations extremely difficult. Even in developed nations, there are still substantial gaps in knowledge about the structural and process factors that underlie unsafe care. We outline the key findings, beginning with these factors, and then the outcomes of such care.
Structural factors contributing to unsafe care
A major contributor to unsafe care is the breakdown of complex systems, which some have called “organisational accidents”. These breakdowns arise from combinations of factors originating at different levels of the system and can involve latent failures or poor oversight. 17 A key structural issue that impacts safety is the inadequate number of qualified healthcare providers worldwide. Globally, 57 countries have an estimated deficit of 2.4 million doctors, nurses and midwives 18 and thus face substantial challenges in meeting health-related Millennium Development Goals for improving the quality and safety of their healthcare systems. A closely related issue to inadequate staffing is production pressures, which refer to situations in which the optimal patient care capacity of a healthcare system or an individual healthcare provider has been exceeded. Providing for too many patients at one time distracts providers, forces greater reliability on memory to perform important actions and hinders effective communication among healthcare personnel. Each of these likely creates an environment for unsafe processes. For example, one analysis in 2005 identified communication problems as the single biggest cause of nearly 70% of sentinel events in the hospital setting in the US. 19 The roles of organisational structure, capacity and communication breakdowns in producing unsafe care in developing and transitional countries have not been adequately studied.
A related threat hindering the delivery of safe care is provider fatigue. Doctors-in-training who work traditional 24-h shifts make 36% more serious medical errors in the care of their patients than comparable doctors not doing extended shift work. Fatigued doctors make up to five times as many serious diagnostic errors, report making four times as many fatigue-related errors that lead to a patient's death and suffer many more occupational injuries themselves. 20–23 Although data on provider fatigue come primarily from developed nations, providers in developing and transitional nations are likely to be at least as susceptible to these threats.
We found that other key structural issues that may affect safety include the organisation's patient safety culture, which refers to shared attitudes, values and norms related to safety. A positive culture may result in improvements in safety practices through better communication, teamwork and knowledge sharing, although the evidence base underlying this assumption is weak. Further, there is scarce knowledge about how organisational factors combine with provider factors to affect patient safety culture. Another important topic includes the role that accreditation and regulations play in improving accountability and systems of care. While both are felt to likely impact safety, their actual level of influence on patient safety has not been empirically assessed and their roles are especially uncertain in developing and transitional countries.
Finally, human factors engineering (HFE) represents an important structural issue to understand the hazards of medical care and ways to minimise those risks. HFE techniques and heuristics can also assist in investigating adverse events 24 when they do occur. Their primary value has been in improving troublesome design issues involving architecture, devices and clinical procedures 25 (eg, anaesthesiology, 26 surgery 27 and nursing 28 ). By understanding how individuals actually interface with technology, for example, HFE can reduce adverse event rates by maximising the human ability to use technology effectively.
Processes that underlie unsafe care
Misdiagnosis is understudied but represents a major type of error in healthcare with widely ranging rates of delayed and erroneous diagnosis. Even in the most highly developed countries with sophisticated technology, at least 10% to 15% of diagnoses are incorrect. 29 The numbers from developing and transitional countries are surely higher and likely add substantial financial costs and create significant morbidity and mortality. For example, one review found high rates of overdiagnosis of malaria in developing nations with consequent underdiagnosis of pneumonia and other related disorders, leading to undertreatment and likely high rates of morbidity from the underlying condition. 30 Another important failure of process is the lack of adequate follow-up of important tests. Data from developed countries suggest that only about half of critically important laboratory results indicating potentially life-threatening conditions were followed up by appropriate treatment in a timely manner. 31 The rates of test follow-up in developing nations are also suboptimal and variable and cause serious lapses in patient care. 32
Counterfeit and substandard drugs, defined as those that are mislabelled, missing active ingredients or include wrong active ingredients, pose a major risk to patient safety. It was classified by the group under unsafe processes (due to poor regulatory oversight of medication safety) but could be considered a structural failure of the healthcare system. Repeated use of counterfeit or substandard medicines can result in therapeutic failure, drug resistance or even death. 33 34 Counterfeit drugs account for more than 10% of the global medicine market and up to 30% of medicines consumed in developing countries. 33 35 It is likely that hundreds of thousands of people, if not more, die each year due to consumption of substandard medications, 33 but the precise burden of the problem is unknown.
Unsafe injection practices also cause substantial morbidity and mortality in large parts of the world. In 2000, WHO estimated that some 16 billion injections were administered each year in transitional and developing nations and up to 40% of injections were given with syringes and needles reused without sterilisation; in some countries, this proportion was as high as 70%. 36 These obviously contribute to high rates of infections with hepatitis viruses, human immunodeficiency virus (HIV) and other transmittable disease.
Outcomes of unsafe care
Adverse events can occur as a result of nearly any interaction with the healthcare system. Estimates from developed nations suggest that between 7.5% and 10.4% of hospitalised patients experience injuries due to medications alone. 1 37 38 These adverse drug events (ADEs) cost tens of billions of dollars to healthcare systems around the world and have been estimated to contribute to 140 000 deaths each year in the US alone. 39 40 Best estimates suggest that 28% to 56% of ADEs are preventable. 41 The rates of ADEs from developing and transitional nations are largely unknown. Although the official rates of medication use are much lower in developing and transitional countries, the actual use of medications in these nations is hard to quantify. 42 Therefore, whether ADEs are less common in these nations is unknown but they likely represent a major source of patient harm and economic costs.
The best evidence suggests that medical devices can also cause substantial harm. Errors that underlie device-related injuries are often categorised into three groups: manufacturer-related errors, user errors and use or design errors. 43–50 In the US, more than 1 million adverse medical device events occur annually, at a rate of 6.3 events per 1000 patient days. 51 Studies by WHO suggest that adverse medical device events might be particularly problematic in developing countries, where medical equipment is often improperly maintained or replaced, placing patients at great risk. 52 53 One study from a transitional nation found that the rates of infection from medical devices alone were 34.2 per 1000 patient days in the hospital. 53
Surgery and anaesthesia also present substantial safety risks. In the US, estimates suggest that surgical adverse events account for 48% of all adverse events and are preventable 54% to 74% of the time. 2 54 The few available studies from the developing world have found surgical adverse event rates to be as much as fivefold to 10-fold higher. 55–60 Improving the use of evidence-based practices could potentially reduce these rates dramatically. 61
Nosocomial infections are reported to occur in approximately 5% to 10% of hospitalised patients in developed nations 15 and between 25% and 40% in developing nations. 15 16 One in four patients in intensive care may acquire an infection during a stay in hospital and one estimate suggests that these rates are twice as high in developing countries. 16 Common types of nosocomial infections include nosocomial pneumonia, catheter-related infections and surgical infections. These events are not only common but also highly preventable. They represent a major source of morbidity and mortality, as well as substantial associated financial costs to health systems.
Another important source of infections from medical care is the use of unsafe blood products. Recent estimates indicate that 5% to 15% of HIV infections in developing countries result from unsafe blood transfusion. 62 63 Unsafe blood poses a high risk for transmission of other blood-borne infections including hepatitis B, hepatitis C, syphilis, malaria, Chagas disease and West Nile fever. Studies demonstrate that nearly three in five countries lack an established quality system to screen collected blood for HIV and that 88.5% of blood units in sub-Saharan Africa are not screened for HIV in a reliable manner. 64 65 It is not known what fraction of blood products in the developing world is tainted. Safe blood products are a particular concern for women of childbearing age, for whom severe haemorrhaging is a leading cause of maternal mortality.
Outcomes of unsafe care in vulnerable populations
With an estimated 7.6 million infant deaths during the perinatal period each year and approximately 600 000 deaths in women due to pregnancy or childbirth (99% of which occur in developing countries), maternal and child health remains a major concern worldwide. 66–68 Although many such deaths result from lack of access to care, many are also due to unsafe care. No well-designed studies address what fraction of the morbidity and mortality of women and newborns is attributable to a lack of access to care or to the receipt of unsafe, poor-quality care. Given the importance of understanding the causes of high maternal mortality in developing nations, deciphering the role that unsafe medical care plays is critically important.
The older people are particularly vulnerable to adverse events. Falls, for example, represent the most common patient safety injury for the older patients in hospitals in developed nations. 69 Hip fractures remain common and only 14% to 21% of patients recover the ability to perform daily activities. 70 Decubitus ulcers are also widespread in this group, risk factors of which include immobility, friction, incontinence, cognitive impairment and poor nutritional status. 71–73 In the US between 1990 and 2001, decubitus ulcers were reported to be the cause of death for 114 380 persons (age-adjusted mortality rate, 3.79 per 100 000 population). 74 Additionally, rates of ADEs among older patients are much higher than in the general population. 75 76
We examined the available research on patient safety and focused specifically on 23 topics and have summarised the major consequences of unsafe care and its underlying causes. Several key findings emerged. First, the available data suggest that harm from medical care is widespread and likely imposes a substantial burden on the world's population. Second, most evidence about safety comes from developed nations, although there is growing epidemiological evidence of poor clinical outcomes due to unsafe medical care in developing and transitional countries. Finally, the data on structural and process factors that contribute to unsafe medical care are almost exclusively from a small number of developed nations. Although some solutions are readily apparent, large gaps in knowledge need to be filled before more comprehensive solutions can be developed, particularly for transitional and developing countries.
The nature and extent of unsafe care are still poorly understood in developing and transitional nations. For 5 of 10 “outcomes” of unsafe care (ADEs, adverse medical device events, surgical errors, falls and decubitus ulcer), there were few data points from developing or transitional nations. Data suggest that the burden of harm from unsafe care is sizeable in developed nations and likely to be comparable if not greater in transitional and developing nations. However, the lack of more reliable information underscores the need for high-quality epidemiological studies from these nations. Even among areas with a known level of harm, such as tainted blood products and nosocomial infections, research is needed to better understand the burden they pose to the population and national health systems and the efficacy of existing prevention and harm minimisation strategies.
In developed nations too, information is still lacking in many areas. We know relatively little about harm outside the hospital or who among the chronically ill is at particular risk for developing decubitus ulcers. We still need to develop many more effective intervention strategies. For all nations, the role of organisational structures and processes is poorly understood. Although inadequate numbers of high-quality staff affect the safety of care, optimal staffing levels and the appropriate mix of skills is not known. How organisational and provider factors combine to affect patient safety needs greater understanding.
To ensure comparability of data across the world, standardised tools, measures and definitions are needed. The WHO World Alliance for Patient Safety has focused on this area. 77 Beyond definitions, as practice patterns change, we will need robust mechanisms for ongoing data collection from a range of nations, especially developing and transitional ones. The potential benefits of improving safety are enormous. For example, studies suggest that establishing safe injection practices around the globe could prevent as many as 1.3 million early deaths each year and eliminate billions of dollars (US$) annually in direct medical costs. 78 The primary implications for funders of healthcare are that we still need both new evidence about the extent of harm that occurs from unsafe medical care in developing countries, the impact that harm has on patients' distrust of those systems, and we need solutions that are able to be implemented locally and are cost-effective.
This study has important limitations. First, although we attempted to account for the most important topics in patient safety, due to constraints, we addressed only those that account for the most harm and some topics that are also important could not be addressed. Although this report could not be wholly comprehensive and was not meant to be a series of systematic reviews of each topic, we do believe we captured most of the major issues in safety and identified the main literature in these areas. Another important limitation was a lack of reliable data in many of the areas we covered, especially for developing countries. Although the limited available information suggests that the burden of harm from unsafe care is very large, it will be essential to obtain more reliable data. Further, whether the key lessons learnt from the developed nations are applicable to the developing world is largely unknown. Next, as has been mentioned previously, many of the topics could have been categorised into multiple areas (ie, structure or process) and those categorisations were not meant to be definitive. Finally, in the interest of brevity, we necessarily omitted many details that are available in the larger report.
In conclusion, patients seek care to reduce their suffering. Based on research from the past two decades, we know that while the healthcare system cures disease and alleviates pain, it can also cause largely preventable harm and suffering. This evidence should not be interpreted as an acceptable cost of providing healthcare. Our review suggests that harm occurs too often and that much of it is avoidable. Reducing harm will require targeted, well-designed and appropriately managed research to gain greater understanding of its causes and contributing factors, especially in transitional and developing countries. The next generation of research should therefore focus both on better definitions of the problem and on effective solutions that reduce harm in medical care.
Acknowledgments
The contributors to the report: Linda Aiken, University of Pennsylvania; Benedetta Allegranzi, WHO World Alliance for Patient Safety; Roselie Bright, USA Food and Drug Administration; Eric Campbell, Massachusetts General Hospital; Richard Cooper, University of Pennsylvania; Neelam Dhingra-Kumar, World Health Organization; Björn Fahlgren, World Health Organization; Timothy Ferris, Massachusetts General Hospital and Harvard Medical School; Terry Field, Meyers Primary Care Institute and University of Massachusetts Medical School; John Gosbee, Red Forest Consulting and University of Michigan Health System; Daniel Grandt, Hospital of Saarbrücken; Jerry Gurwitz, Meyers Primary Care Institute and University of Massachusetts Medical School; Tom Isaac, Veterans Administration Boston Healthcare System; Allen Kachalia, Brigham and Women's Hospital; Selma Khamassi, World Health Organization; Barrett Kitch, Brigham and Women's Hospital and Harvard Medical School; Christopher Landrigan, Children's Hospital, Boston; Michael Matheny, Partners Healthcare System and Brigham and Women's Hospital; Saverio Maviglia, Partners Healthcare System; Mario Merialdi, World Health Organization; Harvey Murff, Veterans Administration Healthcare System and Vanderbilt Epidemiology Center; Eric Poon, Brigham and Women's Hospital; Jim Reason, University of Manchester; Gordon Schiff, Cook County Hospital; Ryan Sidorchuk, Winnipeg Regional Health Authority; Thomas Wuerz, New England Medical Centre.
The report reviewers: Ross Baker, University of Toronto; N K Ganguly, Indian Council of Medical Research; Tawfik Khoja, Health Ministers Council for Gulf Cooperation Council States; Niek Klazinga, University of Amsterdam; John Ovretveit, Karolinska Institutet; Susan Sheridan, Consumers Advancing Patient Safety; Lucian Leape, Harvard School of Public Health; Hilary Coates, Health Information and Quality Authority of Ireland; Martin Fletcher, National Patient Safety Agency of England and Wales; and the World Health Organization officers: Meena Nathan Cherian, Gerald Dziekan, Jan Fordham, Helen Hughes, Mathews Mathai and Ramesh Shademani.
The Research Priority Setting Working Group of the WHO World Alliance for Patient Safety include: Benedetta Allegranzi, World Health Organization; Peter Angood, Joint Commission on Accreditation of Health Care Organizations; Zulfiqar Bhutta, Aga Khan University; Peter Davis, University of Auckland; Daniel Grandt, Hospital of Saarbrücken; Maimunah Hamid, Institute for Health Systems Research of Thailand; Jorge Insua, Hospital Universitario Austral; Robinah Kaitiritimba, Uganda National Health Consumers Organization; Selma Khamassi, World Health Organization; Thandinkosi Madiba, University of Kwazulu-Natal; Takeshi Morimoto, Kyoto University; Douglas Noble, World Alliance for Patient Safety; Peter Norton, University of Calgary; Tikki Elka Pang, World Health Organization; Ryan Sidorchuk, Winnipeg Regional Health Authority; Anuwat Supachutikul, Institute of Hospital Quality Improvement and Accreditation of Thailand; Eric Thomas, University of Texas at Houston; and David Bates (Chair), Ashish Jha, Itziar Larizgoitia and Nittita Prasopa-Plaizier, World Alliance for Patient Safety, World Health Organization as secretariat.
We thank Ms Jordana Nunes Miranda and Ms Katthyana Aparicio for their assistance with this work.
The full report is available at http://www.who.int/patientsafety/research/en .
- Norton PG ,
- Flintoft V ,
- Brennan TA ,
- Thomas EJ ,
- Studdert DM ,
- Burstin HR ,
- Wilson RM ,
- Runciman WB ,
- Gibberd RW ,
- Gandhi TK ,
- Weingart SN ,
- Sequist TD ,
- Warren DK ,
- Cosgrove SE ,
- Diekema DJ ,
- Corrigan JM ,
- Donaldson MS ,
- Donabedian A
- ↵ Child Health Care Quality Toolbox : Measuring quality in children's health. Understanding quality measurement . Agency for Healthcare Research and Quality , 2004 . http://www.ahrq.gov/chtoolbx/understn.htm (accessed 10 August 2006).
- ↵ World Bank . Data and statistics: country classification . Washington, DC : World Bank , 2006 . http://web.worldbank.org/WBSITE/EXTERNAL/DATASTATISTICS/0,,contentMDK:20420458~menuPK:64133156~pagePK:64133150~piPK:64133175~theSitePK:239419,00.html (accessed 10 August 2006).
- ↵ Global patient safety challenge launched . http://www.findarticles.com/p/articles/mi_m0FSL/is_1_83/ai_n16100927/print (accessed 10 August 2006).
- Moreno CA ,
- Rosenthal VD ,
- ↵ WHO . Working together for health: the world health report 2006 . Geneva, Switzerland : WHO Press , 2006 .
- ↵ Sentinel Event Statistics . 2006 . http://www.jointcommission.org/SentinelEvents/Statistics (accessed 17 August 2006).
- Landrigan CP ,
- Rothschild JM ,
- Cronin JW ,
- Barger LK ,
- Lockley SW ,
- Johnson TR ,
- Graham MJ ,
- Kubose TK ,
- Carthey J ,
- de Leval MR ,
- Etchells E ,
- Graber ML ,
- Franklin N ,
- Tolhurst R ,
- Barnish G ,
- Gardner RM ,
- Shojania K ,
- McDonald K ,
- ↵ Substandard and counterfeit medicines , Fact sheet 275 . World Health Organization , 2003 . http://www.who.int/mediacentre/factsheets/2003/fs275/en/ . (accessed 10 August 2006).
- ↵ Counterfeit medicines : some frequently asked questions . World Health Organization , 2005 . Fact sheet. 2005. http://www.wpro.who.int/media_centre/fact_sheets/fs_20050506.htm (accessed 10 August 2006).
- ↵ WHO . Counterfeit medicines: an update on estimates . Geneva, Switzerland : World Health Organization , 2006 .
- Armstrong GL ,
- Miller GC ,
- Britth HC ,
- Cullen DJ ,
- Roughead EE ,
- Semple SJ ,
- Classen DC ,
- Pestotnik SL ,
- Raschke RA ,
- Gollihare B ,
- Wunderlich TA ,
- ↵ IMSHealth . Global pharmaceutical sales by region . 2005 . updated 2006 February 25. http://www.imshealth.com/ims/portal/front/articleC/0,00.html, 2777,6599_77478579_77479643 (accessed 17 August 2006).
- Vicente KJ ,
- van Gruting C
- Clarkson PJ
- Bright RA ,
- Donaldson L
- Yalcin AN ,
- Gawande AA ,
- Zinner MJ ,
- Hernandez K ,
- Boelaert M ,
- Bratzler DW ,
- Delgado-Rodriguez M ,
- Sillero-Arenas M ,
- Medina-Cuadros M ,
- Charlett A ,
- Koigi-Kamau R ,
- Kabare LW ,
- Wanyoike-Gichuhi J
- ↵ WHO . Second global patient safety challenge: safe surgery saves lives . 2008 . http://www.who.int/patientsafety/challenge/second_gpsc/en/index.html (accessed 14 August 2006).
- Lackritz EM
- ↵ Statement by Dr. Neelam Dhingra, Coordinator, Blood Transfusion Safety, World Health Organization : “Making Safe Blood Available in Africa” . Committee on International Relations Subcommittee on Africa, Global Human Rights, and International Operations, US House of Representatives 2006 .
- ↵ WHO . Global database on blood safety, 2001-2002 . Geneva (Switzerland) : World Health Organization , 2004 .
- Cousens S ,
- ↵ WHO . Reproductive health strategy . Geneva (Switzerland) : Reproductive Health and Research , 2004 .
- Schwendimann R ,
- De Geest S ,
- Zuckerman JD
- Berlowitz DR ,
- Brandeis GH ,
- Hossain M ,
- Allman RM ,
- Patrick MM ,
- Redelings MD ,
- Gurwitz JH ,
- Harrold LR ,
- Miller MA ,
Funding The project was funded by the WHO World Alliance for Patient Safety, World Health Organization, Geneva, Switzerland.
Competing interests None.
Linked Articles
- Quality lines Quality lines David P Stevens BMJ Quality & Safety 2010; 19 i-i Published Online First: 19 Feb 2010. doi: 10.1136/qshc.2010.040535
Read the full text or download the PDF:
- Systematic Review
- Open access
- Published: 26 April 2024
Systematic review on the frequency and quality of reporting patient and public involvement in patient safety research
- Sahar Hammoud ORCID: orcid.org/0000-0003-4682-9001 1 ,
- Laith Alsabek 1 , 2 ,
- Lisa Rogers 1 &
- Eilish McAuliffe 1
BMC Health Services Research volume 24 , Article number: 532 ( 2024 ) Cite this article
829 Accesses
1 Citations
1 Altmetric
Metrics details
In recent years, patient and public involvement (PPI) in research has significantly increased; however, the reporting of PPI remains poor. The Guidance for Reporting Involvement of Patients and the Public (GRIPP2) was developed to enhance the quality and consistency of PPI reporting. The objective of this systematic review is to identify the frequency and quality of PPI reporting in patient safety (PS) research using the GRIPP2 checklist.
Searches were performed in Ovid MEDLINE, EMBASE, PsycINFO, and CINAHL from 2018 to December, 2023. Studies on PPI in PS research were included. We included empirical qualitative, quantitative, mixed methods, and case studies. Only articles published in peer-reviewed journals in English were included. The quality of PPI reporting was assessed using the short form of the (GRIPP2-SF) checklist.
A total of 8561 studies were retrieved from database searches, updates, and reference checks, of which 82 met the eligibility criteria and were included in this review. Major PS topics were related to medication safety, general PS, and fall prevention. Patient representatives, advocates, patient advisory groups, patients, service users, and health consumers were the most involved. The main involvement across the studies was in commenting on or developing research materials. Only 6.1% ( n = 5) of the studies reported PPI as per the GRIPP2 checklist. Regarding the quality of reporting following the GRIPP2-SF criteria, our findings show sub-optimal reporting mainly due to failures in: critically reflecting on PPI in the study; reporting the aim of PPI in the study; and reporting the extent to which PPI influenced the study overall.
Conclusions
Our review shows a low frequency of PPI reporting in PS research using the GRIPP2 checklist. Furthermore, it reveals a sub-optimal quality in PPI reporting following GRIPP2-SF items. Researchers, funders, publishers, and journals need to promote consistent and transparent PPI reporting following internationally developed reporting guidelines such as the GRIPP2. Evidence-based guidelines for reporting PPI should be encouraged and supported as it helps future researchers to plan and report PPI more effectively.
Trial registration
The review protocol is registered with PROSPERO (CRD42023450715).
Peer Review reports
Patient safety (PS) is defined as “the absence of preventable harm to a patient and reduction of risk of unnecessary harm associated with healthcare to an acceptable minimum” [ 1 ]. It is estimated that one in 10 patients are harmed in healthcare settings due to unsafe care, resulting in over three million deaths annually [ 2 ]. More than 50% of adverse events are preventable, and half of these events are related to medications [ 3 , 4 ]. There are various types of adverse events that patients can experience such as medication errors, patient falls, healthcare-associated infections, diagnostic errors, pressure ulcers, unsafe surgical procedures, patient misidentification, and others [ 1 ].
Over the last few decades, the approach of PS management has shifted toward actively involving patients and their families in managing PS. This innovative approach has surpassed the traditional model where healthcare providers were the sole managers of PS [ 5 ]. Recent research has shown that patients have a vital role in promoting their safety and decreasing the occurrence of adverse events [ 6 ]. Hence, there is a growing recognition of patient and family involvement as a promising method to enhance PS [ 7 ]. This approach includes involving patients in PS policy development, research, and shared decision making [ 1 ].
In the last decade, research involving patients and the public has significantly increased. In the United Kingdom (U.K), the National Institute for Health Research (NIHR) has played a critical role in providing strategic and infrastructure support to integrate Public and Patient Involvement (PPI) throughout publicly funded research [ 8 ]. This has established a context where PPI is recognised as an essential element in research [ 9 ]. In Ireland, the national government agency responsible for the management and delivery of all public health and social services; the National Health Service Executive (HSE) emphasise the importance of PPI in research and provide guidance for researchers on how to involve patients and public in all parts of the research cycle and knowledge translation process [ 10 ]. Similar initiatives are also developing among other European countries, North America, and Australia. However, despite this significant expansion of PPI research, the reporting of PPI in research articles continues to be sub-optimal, inconsistent, and lacks essential information on the context, process, and impact of PPI [ 9 ]. To address this problem, the Guidance for Reporting Involvement of Patients and the Public (GRIPP) was developed in 2011 following the EQUATOR methodology to enhance the quality, consistency, and transparency of PPI reporting. Additionally, to provide guidance for researchers, patients, and the public to advance the quality of the international PPI evidence-base [ 11 ]. The first GRIPP checklist was a significant start in producing higher-quality PPI reporting; however, it was developed following a systematic review, and did not include any input from the international PPI research community. Given the importance of reaching consensus in generating current reporting guidelines, a second version of the GRIPP checklist (GRIPP2) was developed to tackle this problem by involving the international PPI community in its development [ 9 ]. There are two versions of the GRIPP2 checklist, a long form (GRIPP2-LF) for studies with PPI as the primary focus, and a short form (GRIPP2-SF) for studies with PPI as secondary or tertiary focus.
Since the publication of the GRIPP2 checklist, several systematic reviews have been conducted to assess the quality of PPI reporting on various topics. For instance, Bergin et al. in their review to investigate the nature and impact of PPI in cancer research, reported a sub-optimal quality of PPI reporting using the GRIPP2-SF, mainly due to failure to address PPI challenges [ 12 ]. Similarly, Owyang et al. in their systematic review to assess the prevalence, extent, and quality of PPI in orthopaedic practice, described a poor PPI reporting following the GRIPP2-SF checklist criteria [ 13 ]. While a few systematic reviews have been conducted to assess theories, strategies, types of interventions, and barriers and enablers of PPI in PS [ 5 , 14 , 15 , 16 ], no previous review has assessed the quality of PPI reporting in PS research. Thus, our systematic review aims to address this knowledge gap. The objective of this review is to identify the frequency PPI reporting in PS research using the GRIPP2 checklist from 2018 (the year after GRIPP2 was published) and the quality of reporting following the GRIPP2-SF. The GRIPP2 checklist was chosen as the benchmark as it is the first international, evidence-based, community consensus informed guideline for the reporting of PPI in research and more specifically in health and social care research [ 9 ]. Additionally, it is the most recent report-focused framework and the most recommended by several leading journals [ 17 ].
We followed the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines to plan and report this review [ 18 ]. The review protocol was published on PROSPERO the International Database of Prospectively Registered Systematic Reviews in August 2023 (CRD42023450715).
Search strategy
For this review, we used the PICo framework to define the key elements in our research. These included articles on patients and public (P-Population) involvement (I- phenomenon of Interest) in PS (C-context). Details are presented in Table 1 . Four databases were searched including Ovid MEDLINE, EMBASE, PsycINFO, and CINAHL to identify papers on PPI in PS research. A systematic search strategy was initially developed using MEDLINE. MeSH terms and keywords relevant to specific categories (e.g., patient safety) were combined using the “OR” Boolean term (i.e. patient safety OR adverse event OR medical error OR surgical error) and categories were then combined using the “AND” Boolean term. (i.e. “patient and public involvement” AND “patient safety”). The search strategy was adapted for the other three databases. Full search strategies are provided in Supplementary file 1 . The search was conducted on July 27th, 2023, and was limited to papers published from 2018. As the GRIPP2 tool was published in 2017, this limit ensured the retrieval of relevant studies. An alert system was set on the four databases to receive all new published studies until December 2023, prior to the final analysis. The search was conducted without restrictions on study type, research design, and language. To reduce selection bias, hand searching was carried out on the reference lists of all the eligible articles in the later stages of the review. This was done by the first author. The search strategy was developed by the first author and confirmed by the research team and a Librarian. The database search was conducted by the first author.
Inclusion and exclusion criteria
Studies on PPI in PS research with a focus on health/healthcare were included in this review. We defined PPI as active involvement which is in line with the NIHR INVOLVE definition as “research being carried out ‘with’ or ‘by’ members of the public rather than ‘to’, ‘about’ or ‘for’ them” [ 19 ]. This includes any PPI including, being a co-applicant on a research project or grant application, identifying research priorities, being a member of an advisory or steering group, participating in developing research materials or giving feedback on them, conducting interviews with study participants, participating in recruitment, data collection, data analysis, drafting manuscripts and/or dissemination of results. Accordingly, we excluded studies where patients or the public were only involved as research participants.
We defined patients and public to include patients, relatives, carers, caregivers and community, which is also in line with the NIHR PPI involvement in National Health Service [ 19 ].
Patient safety included topics on medication safety, adverse events, communication, safety culture, diagnostic errors, and others. A full list of the used terms for PPI and PS is provided in Supplementary file 1 . Regarding the research type and design, we included empirical qualitative, quantitative, mixed methods, and case studies. Only articles published in peer-reviewed journals and in English were included.
Any article that did not meet the inclusion criteria was excluded. Studies not reporting outcomes were excluded. Furthermore, review papers, conference abstracts, letters to editor, commentary, viewpoints, and short communications were excluded. Finally, papers published prior to 2018 were excluded.
Study selection
The selection of eligible studies was done by the first and the second authors independently, starting with title and abstracts screening to eliminate papers that failed to meet our inclusion criteria. Then, full text screening was conducted to decide on the final included papers in this review. Covidence, an online data management system supported the review process, ensuring reviewers were blinded to each other’s decisions. Disagreements between reviewers were discussed first, in cases where the disagreement was not resolved, the fourth author was consulted.
Data extraction and analysis
A data extraction sheet was developed using excel then piloted, discussed with the research team and modified as appropriate. The following data were extracted: citation and year of publication, objective of the study, country, PS topic, design, setting, PPI participants, PPI stages (identifying research priorities, being a member of an advisory or steering group, etc.…), frequency of PPI reporting as per the GRIPP2 checklist, and the availability of a plain language summary. Additionally, data against the five items of GRIPP2-SF (aim of PPI in the study, methods used for PPI, outcomes of PPI including the results and the extent to which PPI influenced the study overall, and reflections on PPI) were extracted. To avoid multiple publication bias and missing outcomes, data extraction was done by the first and the second authors independently and then compared. Disagreements between reviewers were first discussed, and then resolved by the third and fourth authors if needed.
Quality assessment
The quality of PPI reporting was assessed using GRIPP2-SF developed by Staniszewska et al. [ 9 ] as it is developed to improve the quality, consistency, and reporting of PPI in social and healthcare research. Additionally the GRIPP2-SF is suitable for all studies regardless of whether PPI is the primary, secondary, or tertiary focus, whereas the GRIPP2-LF is not suitable for studies where PPI serves as a secondary or tertiary focus. The checklist includes five items (mentioned above) that authors should include in their studies. It is important to mention that Staniszewska et al. noted that “while GRIPP2-SF aims to guide consistent reporting, it is not possible to be prescriptive about the exact content of each item, as the current evidence-base is not advanced enough to make this possible” ([ 9 ] p5). For that reason, we had to develop criteria for scoring the five reporting items. We used three scoring as Yes, No, and partial for each of the five items of the GRIPP2-SF. Yes, was given when authors presented PPI information on the item clearly in the paper. No, when no information was provided, and partial when the information partially met the item requirement. For example, as per GRIPP2-SF authors should provide a clear description of the methods used for PPI in the study. In the example given by Staniszewska et al., information on patient/public partners and how many of them were provided, as well as the stages of the study they were involved in (i.e. refining the focus of the research questions, developing the search strategy, interpreting results). Thus, in our evaluation of the included studies, we gave a yes if information on PPI participants (i.e. patient partners, community partners, or family members etc..) and how many of them were involved was provided, and information on the stages or actions of their involvement in the study was provided. However, we gave a “partial” if information was not fully provided (i.e. information on patient/public partners and how many were involved in the study without describing in what stages or actions they were involved, and vice versa), and a “No” if no information was presented at all.
The quality of PPI reporting was done by the first and the second authors independently and then compared. Disagreements between reviewers were first discussed, and then resolved by the third and fourth author when needed.
Assessing the quality or risk of bias of the included studies was omitted, as the focus in this review was on appraising the quality of PPI reporting rather than assessing the quality of each research article.
Data synthesis
After data extraction, a table summarising the included studies was developed. Studies were compared according to the main outcomes of the review; frequency of PPI reporting following the GRIPP2 checklist and the quality of reporting as per GRIPP2-SF five items, and the availability of a plain language summary.
Search results and study selection
The database searches yielded a total of 8491 studies. First, 2496 were removed as duplicates. Then, after title and abstract screening, 5785 articles were excluded leaving 210 articles eligible for the full text review. After a careful examination, 68 of these studies were included in this review. A further 38 studies were identified from the alert system that was set on the four databases and 32 studies from the reference check of the included studies. Of these 70 articles, 56 were further excluded and 14 were added to the previous 68 included studies. Thus, 82 studies met the inclusion criteria and were included in this review. A summary of the database search results and the study selection process are presented in Fig. 1 .
PRISMA flow diagram of the study selection process. The PRISMA flow diagram details the review search results and selection process
Overview of included studies
Details of the study characteristics including first author and year of publication, objective, country, study design, setting, PS topic, PPI participants and involvement stages are presented in Supplementary file 2 . The majority of the studies were conducted in the U.K ( n = 24) and the United States of America ( n = 18), with the remaining 39 conducted in other high income countries, the exception being one study in Haiti. A range of study designs were identified, the most common being qualitative ( n = 31), mixed methods ( n = 13), interventional ( n = 5), and quality improvement projects ( n = 4). Most PS topics concerned medication safety ( n = 17), PS in general (e.g., developing a PS survey or PS management application) ( n = 14), fall prevention ( n = 13), communication ( n = 11), and adverse events ( n = 10), with the remaining PS topics listed in Supplementary file 2 .
Patient representatives, advocates, and patient advisory groups ( n = 33) and patients, service users, and health consumers ( n = 32) were the main groups involved. The remaining, included community members/ organisations. Concerning PPI stages, the main involvement across the studies was in commenting on or developing research materials ( n = 74) including, patient leaflets, interventional tools, mobile applications, and survey instruments. Following this stage, involvement in data analysis, drafting manuscripts, and disseminating results ( n = 30), and being a member of a project advisory or steering group ( n = 18) were the most common PPI evident in included studies. Whereas the least involvement was in identifying research priorities ( n = 5), and being a co-applicant on a research project or grant application ( n = 6).
Regarding plain language summary, only one out of the 82 studies (1.22%) provided a plain language summary in their paper [ 20 ].
Frequency and quality of PPI reporting
The frequency of PPI reporting following the GRIPP2 checklist was 6.1%, where only five of the 82 included studies reported PPI in their papers following the GRIPP2 checklist. The quality of PPI reporting in those studies is presented in Table 2 . Of these five studies, one study (20%) did not report the aim of PPI in the study and one (20%) did not comment on the extent to which PPI influenced the study overall.
The quality of PPI reporting of the remaining 77 studies is presented in Table 3 . The aim of PPI in the study was reported in 62.3% of articles ( n = 48), while 3.9% ( n = 3) partially reported this. A clear description of the methods used for PPI in the study was reported in 79.2% of papers ( n = 61) and partially in 20.8% ( n = 16). Concerning the outcomes, 81.8% of papers ( n = 63) reported the results of PPI in the study, while 10.4% ( n = 8) partially did. Of the 77 studies, 68.8% ( n = 53) reported the extent to which PPI influenced the study overall and 3.9% ( n = 3) partially reported this. Finally, 57.1% ( n = 44) of papers critically reflected on the things that went well and those that did not and 2.6% ( n = 2) partially reflected on this.
Summary of main findings
This systematic review assessed the frequency of reporting PPI in PS research using the GRIPP2 checklist and quality of reporting using the GRIPP2-SF. In total, 82 studies were included in this review. Major PS topics were related to medication safety, general PS, and fall prevention. Patient representatives, advocates, patient advisory groups, patients, service users, and health consumers were the most involved. The main involvement across the studies was in commenting on or developing research materials such as educational and interventional tools, survey instruments, and applications while the least was in identifying research priorities and being a co-applicant on a research project or grant application. Thus, significant effort is still needed to involve patients and the public in the earlier stages of the research process given the fundamental impact of PS on their lives.
Overall completeness and applicability of evidence
A low frequency of reporting PPI in PS research following the GRIPP2 guidelines was revealed in this review, where only five of the 82 studies included mentioned that PPI was reported as per the GRIPP2 checklist. This is despite it being the most recent report-focused framework and the most recommended by several leading journals [ 17 ]. This was not surprising as similar results were reported in recent reviews in other healthcare topics. For instance, Musbahi et al. in their systematic review on PPI reporting in bariatric research reported that none of the 90 papers identified in their review mentioned or utilised the GRIPP2 checklist [ 102 ]. Similarly, a study on PPI in orthodontic research found that none of the 363 included articles reported PPI against the GRIPP2 checklist [ 103 ].
In relation to the quality of reporting following the GRIPP2-SF criteria, our findings show sub-optimal reporting within the 77 studies that did not use GRIPP2 as a guide/checklist to report their PPI. Similarly, Bergin et al. in their systematic review to investigate the nature and impact of PPI in cancer research concluded that substandard reporting was evident [ 12 ]. In our review, this was mainly due to failure to meet three criteria. First, the lowest percentage of reporting (57.1%, n = 44) was related to critical reflection on PPI in the study (i.e., what went well and what did not). In total, 31 studies (42.9%) did not provide any information on this, and two studies were scored as partial. The first study mentioned that only involving one patient was a limitation [ 27 ] and the other stated that including three patients in the design of the tool was a strength [ 83 ]. Both studies did not critically comment or reflect on these points so that future researchers are able to avoid such problems and enhance PPI opportunities. For instance, providing the reasons/challenges behind the exclusive inclusion of a single patient and explaining how this limits the study findings and conclusion would help future researchers to address these challenges. Likewise, commenting on why incorporating three patients in the design of the study tool could be seen as a strength would have been beneficial. This could be, fostering diverse perspectives and generating novel ideas for developing the tool. Similar to our findings, Bergin et al. in their systematic review reported that 40% of the studies failed to meet this criterion [ 12 ].
Second, only 48 out of 77 articles (62.3%) reported the aim of PPI in their study, which is unlike the results of Bergin et al. where most of the studies (93.1%) in their review met this criterion [ 12 ]. Of the 29 studies which did not meet this criterion in our review, few mentioned in their objective developing a consensus-based instrument [ 41 ], reaching a consensus on the patient-reported outcomes [ 32 ], obtaining international consensus on a set of core outcome measures [ 98 ], and facilitating a multi-stakeholder dialogue [ 71 ] yet, without indicating anything in relation to patients, patient representatives, community members, or any other PPI participants. Thus, the lack of reporting the aim of PPI was clearly evident in this review. Reporting the aim of PPI in the study is crucial for promoting transparency, methodological rigor, reproducibility, and impact assessment of the PPI.
Third, 68.8% ( n = 53) of the studies reported the extent to which PPI influenced the study overall including positive and negative effects if any. This was again similar to the findings of Bergin et al., where 38% of the studies did not meet this criterion mainly due to a failure to address PPI challenges in their respective studies [ 12 ]. Additionally, Owyang et al. in their review on the extent, and quality of PPI in orthopaedic practice, also described a poor reporting of PPI impact on research [ 13 ]. As per the GRIPP2 guidelines, both positive and negative effects of PPI on the study should be reported when applicable. Providing such information is essential as it enhances future research on PPI in terms of both practice and reporting.
Reporting a clear description of the methods used for PPI in the study was acceptable, with 79.2% of the papers meeting this criterion. Most studies provided information in the methods section of their papers on the PPI participants, their number, stages of their involvement and how they were involved. Providing clear information on the methods used for PPI is vital to give the reader a clear understanding of the steps taken to involve patients, and for other researchers to replicate these methods in future research. Additionally, reporting the results of PPI in the study was also acceptable with 81.8% of the papers reporting the outcomes of PPI in the results section. Reporting the results of PPI is important for enhancing methodological transparency, providing a more accurate interpretation for the study findings, contributing to the overall accountability and credibility of the research, and informing decision making.
Out of the 82 studies included in this review, only one study provided a plain language summary. We understand that PS research or health and medical research in general is difficult for patients and the public to understand given their diverse health literacy and educational backgrounds. However, if we expect patients and the public to be involved in research then, it is crucial to translate this research that has a huge impact on their lives into an easily accessible format. Failing to translate the benefits that such research may have on patient and public lives may result in them underestimating the value of this research and losing interest in being involved in the planning or implementation of future research [ 103 ]. Thus, providing a plain language summary for research is one way to tackle this problem. To our knowledge, only a few health and social care journals (i.e. Cochrane and BMC Research Involvement and Engagement) necessitate a plain language summary as a submission requirement. Having this as a requirement for submission is crucial in bringing the importance of this issue to researchers’ attention.
Research from recent years suggests that poor PPI reporting in articles relates to a lack of submission requirements for PPI reporting in journals and difficulties with word limits for submitted manuscripts [ 13 ]. Price et al. assessed the frequency of PPI reporting in published papers before and after the introduction of PPI reporting obligations by the British Medical Journal (BMJ) [ 104 ]. The authors identified an increase in PPI reporting in papers published by BMJ from 0.5% to 11% between the periods of 2013–2014 and 2015–2016. The study findings demonstrate the impact of journal guidelines in shaping higher quality research outputs [ 13 ]. In our review, we found a low frequency of PPI reporting in PS research using the GRIPP2 checklist, alongside sub-optimal quality of reporting following GRIPP2-SF. This could potentially be attributed to the absence of submission requirements for PPI reporting in journals following the GRIPP2 checklist, as well as challenges posed by word limits.
Strengths and limitations
This systematic review presents an overview on the frequency of PPI reporting in PS research using the GRIPP2 checklist, as well as an evaluation of the quality of reporting following the GRIPP2-SF. As the first review to focus on PS research, it provides useful knowledge on the status of PPI reporting in this field, and the extent to which researchers are adopting and adhering to PPI reporting guidelines. Despite these strengths, our review has some limitations that should be mentioned. First, only English language papers were included in this review due to being the main language of the researchers. Thus, there is a possibility that relevant articles on PPI in PS research may have been omitted. Another limitation is related to our search which was limited to papers published starting 2018 as the GRIPP2 guidelines were published in 2017. Thus it is probable that the protocols of some of these studies were developed earlier than the publication of the GRIPP2 checklist, meaning that PPI reporting following GRIPP2 was not common practice and thus not adopted by these studies. This might limit the conclusions we can draw from this review. Finally, the use of GRIPP2 to assess the quality of PPI reporting might be a limitation as usability testing has not yet been conducted to understand how the checklist works in practice with various types of research designs. However, the GRIPP2 is the first international, evidence-based, community consensus informed guideline for the reporting of PPI in health and social care research. Reflections and comments from researchers using the GRIPP2 will help improve its use in future studies.
Implications for research and practice
Lack of PPI reporting not only affects the quality of research but also implies that others cannot learn from previous research experience. Additionally, without consistent and transparent reporting it is difficult to evaluate the impact of various PPI in research [ 9 ]: “if it is not reported it cannot be assessed” ([ 105 ] p19). Enhanced PPI reporting will result in a wider range and richer high-quality evidence-based PPI research, leading to a better understanding of PPI use and effectiveness [ 103 ]. GRIPP2 reporting guidelines were developed to provide guidance for researchers, patients, and the public to enhance the quality of PPI reporting and improve the quality of the international PPI evidence-base. The guidance can be used prospectively to plan PPI or retrospectively to guide the structure or PPI reporting in research [ 9 ]. To enhance PPI reporting, we recommend the following;
Publishers and journals
First, we encourage publishers and journals to require researchers to report PPI following the GRIPP2 checklist. Utilising the short or the long version should depend on the primary focus of the study (i.e., if PPI is within the primary focus of the research then the GRIPP2-LF is recommended). Second, we recommend that journals and editorial members advise reviewers to evaluate PPI reporting within research articles following the GRIPP2 tool and make suggestions accordingly. Finally, we encourage journals to add a plain language summary as a submission requirement to increase research dissemination and improve the accessibility of research for patients and the public.
Researchers
Though there is greater evidence of PPI in research, it is still primarily the researchers that are setting the research agenda and deciding on the research questions to be addressed. Thus, significant effort is still needed to involve patients and the public in the earlier stages of the research process given the fundamental impact of PS on their lives. To enhance future PPI reporting, perhaps adding a criterion following the GRIPP2 tool to existing EQUATOR checklists for reporting research papers such as STROBE, PRISMA, CONSORT, may support higher quality research. Additionally, currently, there is no detailed explanation paper for the GRIPP2 where each criterion is explained in detail with examples. Addressing this gap would be of great benefit to guide the structure of PPI reporting and to explore the applicability of each criterion in relation to different stages of PPI in research. For instance, having a detailed explanation for each criterion across different research studies having various PPI stages would be of high value to improve future PPI reporting given the growing interest in PPI research in recent years and the relatively small PPI evidence base in health and medical research.
Funding bodies can also enhance PPI reporting by adding a requirement for researchers to report PPI following the GRIPP2 checklist. In Ireland, the National HSE has already initiated this by requiring all PPI in HSE research in Ireland to be reported following the GRIPP2 guidelines [ 10 ].
This study represents the first systematic review on the frequency and quality of PPI reporting in PS research using the GRIPP2 checklist. Most PS topics were related to medication safety, general PS, and fall prevention. The main involvement across the studies was in commenting on or developing research materials. Thus, efforts are still needed to involve patients and the public across all aspects of the research process, especially earlier stages of the research cycle. The frequency of PPI reporting following the GRIPP2 guidelines was low, and the quality of reporting following the GRIPP2-SF criteria was sub-optimal. The lowest percentages of reporting were on critically reflecting on PPI in the study so future research can learn from this experience and work to improve it, reporting the aim of the PPI in the study, and reporting the extent to which PPI influenced the study overall including positive and negative effects. Researchers, funders, publishers, journals, editorial members and reviewers have a responsibility to promote consistent and transparent PPI reporting following internationally developed reporting guidelines such as the GRIPP2. Evidence-based guidelines for reporting PPI should be supported to help future researchers plan and report PPI more effectively, which may ultimately improve the quality and relevance of research.
Availability of data and materials
All data generated or analysed during this study are included in this published article and its Supplementary information files.
Abbreviations
United Kingdom
National Institute for Health Research
Public and Patient Involvement
Health Service Executive
Guidance for Reporting Involvement of Patients and the Public
Second version of the GRIPP checklist
Long form of GRIPP2
Short form of GRIPP2
Preferred Reporting Items for Systematic Reviews and Meta-Analyses
The International Database of Prospectively Registered Systematic Reviews
British Medical Journal
Patient saftey: World Health Organisation. 2023. Available from: https://www.who.int/news-room/fact-sheets/detail/patient-safety . Updated 11 Sept 2023.
Slawomirski L, Klazinga N. The economics of patient safety: from analysis to action. Paris: Organisation for Economic Co-operation and Development; 2020.
Google Scholar
Panagioti M, Khan K, Keers RN, Abuzour A, Phipps D, Kontopantelis E, et al. Prevalence, severity, and nature of preventable patient harm across medical care settings: systematic review and meta-analysis. Bmj-Brit Med J. 2019;366:l4185.
Article Google Scholar
Hodkinson A, Tyler N, Ashcroft DM, Keers RN, Khan K, Phipps D, et al. Preventable medication harm across health care settings: a systematic review and meta-analysis. Bmc Medicine. 2020;18(1):313.
Article PubMed PubMed Central Google Scholar
Park M, Giap TTT. Patient and family engagement as a potential approach for improving patient safety: A systematic review. J Adv Nurs. 2020;76(1):62–80.
Article PubMed Google Scholar
Chegini Z, Janati A, Bababie J, Pouraghaei M. The role of patients in the delivery of safe care in hospital: Study protocol. J Adv Nurs. 2019;75(9):2015–23.
Chegini Z, Arab-Zozani M, Islam SMS, Tobiano G, Rahimi SA. Barriers and facilitators to patient engagement in patient safety from patients and healthcare professionals’ perspectives: A systematic review and meta-synthesis. Nurs Forum. 2021;56(4):938–49.
Going the extra mile: improving the nation’s health and wellbeing through public involvement in research. London: National Institute for Health; 2015.
Staniszewska S, Brett J, Simera I, Seers K, Mockford C, Goodlad S, et al. GRIPP2 reporting checklists: tools to improve reporting of patient and public involvement in research. Bmj-Brit Med J. 2017;358:j3453.
Article CAS Google Scholar
Minogue V. Knowledge translation, dissemination, and impact: a practical guide for researchers. Guide No 8: patient and public involvement in HSE research. Ireland: Health Service Executive Research and Development; 2021.
Staniszewska S, Brett J, Mockford C, Barber R. The GRIPP checklist: Strengthening the quality of patient and public involvement reporting in research. Int J Technol Assess Health Care. 2011;27(4):391–9.
Bergin RJ, Short CE, Davis N, Marker J, Dawson MT, Milton S, et al. The nature and impact of patient and public involvement in cancer prevention, screening and early detection research: A systematic review. Prev Med. 2023;167:107412.
Owyang D, Bakhsh A, Brewer D, Boughton OR, Cobb JP. Patient and public involvement within orthopaedic research a systematic review. J Bone Joint Surg Am. 2021;103(13):e51.
Busch IM, Saxena A, Wu AW. Putting the patient in patient safety investigations: barriers and strategies for involvement. J Patient Saf. 2021;17(5):358–62.
Lee M, Lee NJ, Seo HJ, Jang H, Kim SM. Interventions to engage patients and families in patient safety: a systematic review. West J Nurs Res. 2021;43(10):972–83.
Ocloo J, Garfield S, Franklin BD, Dawson S. Exploring the theory, barriers and enablers for patient and public involvement across health, social care and patient safety: a systematic review of reviews. Health Res Policy Syst. 2021;19(1):8.
Greenhalgh T, Hinton L, Finlay T, Macfarlane A, Fahy N, Clyde B, et al. Frameworks for supporting patient and public involvement in research: Systematic review and co-design pilot. Health Expect. 2019;22(4):785–801.
Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, et al. The PRISMA 2020 statement: An updated guideline for reporting systematic reviews. Plos Medicine. 2021;18(3):372.
INVOLVE. What is public involvement in research? NIHR; 2019. Available from: https://www.invo.org.uk/find-out-more/what-is-public-involvement-in-research-2/ .
Shahid A, Sept B, Kupsch S, Brundin-Mather R, Piskulic D, Soo A, et al. Development and pilot implementation of a patient-oriented discharge summary for critically Ill patients. World J Crit Care Med. 2022;11(4):255–68.
Bisset CN, Dames N, Oliphant R, Alasadi A, Anderson D, Parson S, et al. Exploring shared surgical decision-making from the patient’s perspective: is the personality of the surgeon important? Colorectal Dis. 2020;22(12):2214–21.
Article CAS PubMed Google Scholar
Morris RL, Ruddock A, Gallacher K, Rolfe C, Giles S, Campbell S. Developing a patient safety guide for primary care: A co-design approach involving patients, carers and clinicians. Health Expect. 2021;24(1):42–52.
Tobiano G, Marshall AP, Gardiner T, Jenkinson K, Shapiro M, Ireland M. Development and psychometric testing of the patient participation in bedside handover survey. Health Expect. 2022;25(5):2492–502.
Francis-Coad J, Farlie MK, Haines T, Black L, Weselman T, Cummings P, et al. Revising and evaluating falls prevention education for older adults in hospital. Health Educ J. 2023;82(8):878–91.
Troya MI, Chew-Graham CA, Babatunde O, Bartlam B, Higginbottom A, Dikomitis L. Patient and public involvement and engagement in a doctoral research project exploring self-harm in older adults. Health Expect. 2019;22(4):617–31.
Aharaz A, Kejser CL, Poulsen MW, Jeftic S, Ulstrup-Hansen AI, Jorgensen LM, et al. Optimization of the Danish National Electronic Prescribing System to improve patient safety: Development of a user-friendly prototype of the digital platform shared medication record. Pharmacy (Basel, Switzerland). 2023;11(2):41.
PubMed Google Scholar
Aho-Glele U, Bouabida K, Kooijman A, Popescu IC, Pomey MP, Hawthornthwaite L, et al. Developing the first pan-Canadian survey on patient engagement in patient safety. BMC Health Serv Res. 2021;21(1):1099.
Albutt A, O’Hara J, Conner M, Lawton R. Involving patients in recognising clinical deterioration in hospital using the patient wellness questionnaire: A mixed-methods study. J Res Nurs. 2020;25(1):68–86.
Bell SK, Bourgeois F, DesRoches CM, Dong J, Harcourt K, Liu SK, et al. Filling a gap in safety metrics: development of a patient-centred framework to identify and categorise patient-reported breakdowns related to the diagnostic process in ambulatory care. BMJ Qual Saf. 2022;31(7):526–40.
Boet S, Etherington N, Lam S, Lê M, Proulx L, Britton M, et al. Implementation of the Operating Room Black Box research program at the Ottawa Hospital through patient, clinical, and organizational engagement: Case study. J Med Internet Res. 2021;23(3):e15443.
Carter J, Tribe RM, Shennan AH, Sandall J. Threatened preterm labour: Women’s experiences of risk and care management: A qualitative study. Midwifery. 2018;64:85–92.
Da Silva Lopes AM, Colomer-Lahiguera S, Mederos Alfonso N, Aedo-Lopez V, Spurrier-Bernard G, Tolstrup LK, et al. Patient-reported outcomes for monitoring symptomatic toxicities in cancer patients treated with immune-checkpoint inhibitors: A Delphi study. Eur J Cancer. 2021;157:225–37.
de Jong LD, Lavender AP, Wortham C, Skelton DA, Haines TP, Hill AM. Exploring purpose-designed audio-visual falls prevention messages on older people’s capability and motivation to prevent falls. Health Soc Care Community. 2019;27(4):e471–82.
Doucette L, Kiely BT, Gierisch JM, Marion E, Nadler L, Heflin MT, et al. Participatory research to improve medication reconciliation for older adults in the community. J Am Geriatr Soc. 2023;71(2):620–31.
Elrod CS, Pappa ST, Heyn PC, Wong RA. Using an academic-community partnership model to deliver evidence-based falls prevention programs in a metropolitan setting: A community case study. Front Public Health. 2023;11:1073520.
Feldman E, Pos FJ, Smeenk RJ, van der Poel H, van Leeuwen P, de Feijter JM, et al. Selecting a PRO-CTCAE-based subset for patient-reported symptom monitoring in prostate cancer patients: a modified Delphi procedure. ESMO Open. 2023;8(1):100775.
Article CAS PubMed PubMed Central Google Scholar
Francis-Coad J, Watts T, Bulsara C, Hill A-M. Designing and evaluating falls prevention education with residents and staff in aged care homes: a feasibility study. Health Educ (0965-4283). 2022;122(5):546–63.
Fuller TE, Pong DD, Piniella N, Pardo M, Bessa N, Yoon C, et al. Interactive digital health tools to engage patients and caregivers in discharge preparation: implementation study. J Med Internet Res. 2020;22(4):e15573.
Gibson B, Butler J, Schnock K, Bates D, Classen D. Design of a safety dashboard for patients. Patient Educ Couns. 2020;103(4):741–7.
Giles SJ, Lewis PJ, Phipps DL, Mann F, Avery AJ, Ashcroft DM. Capturing patients’ perspectives on medication safety: the development of a patient-centered medication safety framework. J Patient Saf. 2020;16(4):e324–39.
Gnagi R, Zuniga F, Brunkert T, Meyer-Massetti C. Development of a medication literacy assessment instrument (MELIA) for older people receiving home care. J Adv Nurs. 2022;78(12):4210–20.
Goodsmith N, Zhang L, Ong MK, Ngo VK, Miranda J, Hirsch S, et al. Implementation of a community-partnered research suicide-risk management protocol: case study from community partners in care. Psychiatr Serv (Washington, DC). 2021;72(3):281–7.
Gorman LS, Littlewood DL, Quinlivan L, Monaghan E, Smith J, Barlow S, et al. Family involvement, patient safety and suicide prevention in mental healthcare: ethnographic study. BJPsych open. 2023;9(2):e54.
Green MM, Meyer C, Hutchinson AM, Sutherland F, Lowthian JA. Co‐designing being your best program—a holistic approach to frailty in older community dwelling australians. Health Soc Care Community. 2021;30(5):e2022–32.
Guo X, Wang Y, Wang L, Yang X, Yang W, Lu Z, et al. Effect of a fall prevention strategy for the older patients: A quasi-experimental study. Nurs Open. 2023;10(2):1116–24.
Hahn-Goldberg S, Chaput A, Rosenberg-Yunger Z, Lunsky Y, Okrainec K, Guilcher S, et al. Tool development to improve medication information transfer to patients during transitions of care: A participatory action research and design thinking methodology approach. Res Social Adm Pharm. 2022;18(1):2170–7.
Harrington A, Darke H, Ennis G, Sundram S. Evaluation of an alternative model for the management of clinical risk in an adult acute psychiatric inpatient unit. Int J Ment Health Nurs. 2019;28(5):1099–109.
Harris K, Softeland E, Moi AL, Harthug S, Ravnoy M, Storesund A, et al. Development and validation of patients’ surgical safety checklist. BMC Health Serv Res. 2022;22(1):259.
Hawley-Hague H, Tacconi C, Mellone S, Martinez E, Ford C, Chiari L, et al. Smartphone apps to support falls rehabilitation exercise: app development and usability and acceptability study. JMIR Mhealth Uhealth. 2020;8(9):e15460.
Holmqvist M, Ros A, Lindenfalk B, Thor J, Johansson L. How older persons and health care professionals co-designed a medication plan prototype remotely to promote patient safety: case study. JMIR aging. 2023;6:e41950.
Jayesinghe R, Moriarty F, Khatter A, Durbaba S, Ashworth M, Redmond P. Cost outcomes of potentially inappropriate prescribing in middle-aged adults: A Delphi consensus and cross-sectional study. Br J Clin Pharmacol. 2022;88(7):3404–20.
Johannessen T, Ree E, Stromme T, Aase I, Bal R, Wiig S. Designing and pilot testing of a leadership intervention to improve quality and safety in nursing homes and home care (the SAFE-LEAD intervention). BMJ Open. 2019;9(6):e027790.
Joseph K, Newman B, Manias E, Walpola R, Seale H, Walton M, et al. Engaging with ethnic minority consumers to improve safety in cancer services: A national stakeholder analysis. Patient Educ Couns. 2022;105(8):2778–84.
Khan A, Spector ND, Baird JD, Ashland M, Starmer AJ, Rosenbluth G, et al. Patient safety after implementation of a coproduced family centered communication programme: multicenter before and after intervention study. BMJ. 2018;363:k4764.
Khazen M, Mirica M, Carlile N, Groisser A, Schiff GD. Developing a framework and electronic tool for communicating diagnostic uncertainty in primary care: a qualitative study. JAMA Network Open. 2023;6(3):e232218-e.
Knight SW, Trinkle J, Tschannen D. Hospital-to-homecare videoconference handoff: improved communication, coordination of care, and patient/family engagement. Home Healthc Now. 2019;37(4):198–207.
Lawrence V, Kimona K, Howard RJ, Serfaty MA, Wetherell JL, Livingston G, et al. Optimising the acceptability and feasibility of acceptance and commitment therapy for treatment-resistant generalised anxiety disorder in older adults. Age Ageing. 2019;48(5):741–50.
Louch G, Reynolds C, Moore S, Marsh C, Heyhoe J, Albutt A, et al. Validation of revised patient measures of safety: PMOS-30 and PMOS-10. BMJ Open. 2019;9(11):e031355.
MacDonald T, Jackson S, Charles M-C, Periel M, Jean-Baptiste M-V, Salomon A, et al. The fourth delay and community-driven solutions to reduce maternal mortality in rural Haiti: a community-based action research study. BMC Pregnancy Childbirth. 2018;18(1):254.
Mackintosh N, Sandall J, Collison C, Carter W, Harris J. Employing the arts for knowledge production and translation: Visualizing new possibilities for women speaking up about safety concerns in maternity. Health Expect. 2018;21(3):647–58.
Marchand K, Turuba R, Katan C, Brasset C, Fogarty O, Tallon C, et al. Becoming our young people’s case managers: caregivers’ experiences, needs, and ideas for improving opioid use treatments for young people using opioids. Subst Abuse Treat Prev Policy. 2022;17(1):1–15.
Mazuz K, Biswas S. Co-designing technology and aging in a service setting: Developing an interpretive framework of how to interact with older age users. Gerontechnology. 2022;21(1):1–13.
McCahon D, Duncan P, Payne R, Horwood J. Patient perceptions and experiences of medication review: qualitative study in general practice. BMC Prim Care. 2022;23(1):293.
McMullen S, Panagioti M, Planner C, Giles S, Angelakis I, Keers RN, et al. Supporting carers to improve patient safety and maintain their well-being in transitions from mental health hospitals to the community: A prioritisation nominal group technique. Health Expect. 2023;26(5):2064–74.
Morris RL, Giles S, Campbell S. Involving patients and carers in patient safety in primary care: A qualitative study of a co-designed patient safety guide. Health Expect. 2023;26(2):630–9.
Morris RL, Stocks SJ, Alam R, Taylor S, Rolfe C, Glover SW, et al. Identifying primary care patient safety research priorities in the UK: a James Lind Alliance Priority Setting Partnership. BMJ Open. 2018;8(2):e020870.
Nether KG, Thomas EJ, Khan A, Ottosen MJ, Yager L. Implementing a robust process improvement program in the neonatal intensive care unit to reduce harm. J Healthc Qual. 2022;44(1):23–30.
Powell C, Ismail H, Cleverley R, Taylor A, Breen L, Fylan B, et al. Patients as qualitative data analysts: Developing a method for a process evaluation of the “Improving the Safety and Continuity of Medicines management at care Transitions” (ISCOMAT) cluster randomised control trial. Health Expect. 2021;24(4):1254–62.
Article PubMed Central Google Scholar
Powell C, Ismail H, Davis M, Taylor A, Breen L, Fylan B, et al. Experiences of patients with heart failure with medicines at transition intervention: Findings from the process evaluation of the Improving the Safety and Continuity of Medicines management at Transitions of care (ISCOMAT) programme. Health Expect. 2022;25(5):2503–14.
Radecki B, Keen A, Miller J, McClure JK, Kara A. Innovating fall safety: engaging patients as experts. J Nurs Care Qual. 2020;35(3):220–6.
Rosgen BK, Plotnikoff KM, Krewulak KD, Shahid A, Hernandez L, Sept BG, et al. Co-development of a transitions in care bundle for patient transitions from the intensive care unit: a mixed-methods analysis of a stakeholder consensus meeting. BMC Health Serv Res. 2022;22(1):10.
Schenk EC, Bryant RA, Van Son CR, Odom-Maryon T. Developing an intervention to reduce harm in hospitalized patients: patients and families in research. J Nurs Care Qual. 2019;34(3):273–8.
Spazzapan M, Vijayakumar B, Stewart CE. A bit about me: Bedside boards to create a culture of patient-centered care in pediatric intensive care units (PICUs). J Healthc Risk Manag. 2020;39(3):11–9.
Stoll JA, Ranahan M, Richbart MT, Brennan-Taylor MK, Taylor JS, Brady L, et al. Development of video animations to encourage patient-driven deprescribing: A team alice study. Patient Educ Couns. 2021;104(11):2716–23.
Subbe CP, Tomos H, Jones GM, Barach P. Express check-in: developing a personal health record for patients admitted to hospital with medical emergencies: a mixed-method feasibility study. Int J Qual Health Care. 2021;33(3):121.
Tai D, Li E, Liu-Ambrose T, Bansback N, Sadatsafavi M, Davis JC. Patient-Reported Outcome Measures (PROMs) to support adherence to falls prevention clinic recommendations: a qualitative study. Patient Prefer Adherence. 2020;14:2105–21.
Thakur T, Chewning B, Zetes N, Lee JTY. Involving caregivers in design and assessment of opioid risk and safety communication intervention in children. Patient Educ Couns. 2021;104(10):2432–6.
Thomas J, Dahm MR, Li J, Georgiou A. Can patients contribute to enhancing the safety and effectiveness of test-result follow-up? Qualitative outcomes from a health consumer workshop. Health Expect. 2021;24(2):222–33.
Tremblay MC, Bradette-Laplante M, Witteman HO, Dogba MJ, Breault P, Paquette JS, et al. Providing culturally safe care to indigenous people living with diabetes: Identifying barriers and enablers from different perspectives. Health Expect. 2021;24(2):296–306.
Troya MI, Dikomitis L, Babatunde OO, Bartlam B, Chew-Graham CA. Understanding self-harm in older adults: A qualitative study. EClinicalMedicine. 2019;12:52–61.
Tyler N, Giles S, Daker-White G, McManus BC, Panagioti M. A patient and public involvement workshop using visual art and priority setting to provide patients with a voice to describe quality and safety concerns: Vitamin B12 deficiency and pernicious anaemia. Health Expect. 2021;24(1):87–94.
Tyler N, Planner C, Shears B, Hernan A, Panagioti M, Giles S. Developing the Resident Measure of Safety in Care Homes (RMOS): A Delphi and think aloud study. Health Expect. 2023;26(3):1149–58.
Van den Bulck SA, Vankrunkelsven P, Goderis G, Van Pottelbergh G, Swerts J, Panis K, et al. Developing quality indicators for Chronic Kidney Disease in primary care, extractable from the Electronic Medical Record. A Rand-modified Delphi method. BMC Nephrol. 2020;21(1):161.
Van Strien-Knippenberg IS, Boshuizen MCS, Determann D, de Boer JH, Damman OC. Cocreation with Dutch patients of decision-relevant information to support shared decision-making about adjuvant treatment in breast cancer care. Health Expect. 2022;25(4):1664–77.
Wilson NA, Reich AJ, Graham J, Bhatt DL, Nguyen LL, Weissman JS. Patient perspectives on the need for implanted device information: Implications for a post-procedural communication framework. Health Expect. 2021;24(4):1391–402.
Winterberg AV, Lane B, Hill LM, Varughese AM. Optimizing Pediatric Induction Experiences Using Human-centered Design. J Perianesth Nurs. 2022;37(1):48–52.
Yang R, Donaldson GW, Edelman LS, Cloyes KG, Sanders NA, Pepper GA. Fear of older adult falling questionnaire for caregivers (FOAFQ-CG): Evidence from content validity and item-response theory graded-response modelling. J Adv Nurs. 2020;76(10):2768–80.
Young A, Menon D, Street J, Al-Hertani W, Stafinski T. A checklist for managed access programmes for reimbursement co-designed by Canadian patients and caregivers. Health Expect. 2018;21(6):973–80.
Yuen EYN, Street M, Abdelrazek M, Blencowe P, Etienne G, Liskaser R, et al. Evaluating the efficacy of a digital App to enhance patient-centred nursing handover: A simulation study. J Clin Nurs. 2023;32(19–20):7626–37.
Jo S, Nabatchi T. Coproducing healthcare: individual-level impacts of engaging citizens to develop recommendations for reducing diagnostic error. Public Manag Rev. 2019;21(3):354–75.
O’Hara JK, Reynolds C, Moore S, Armitage G, Sheard L, Marsh C, et al. What can patients tell us about the quality and safety of hospital care? Findings from a UK multicentre survey study. BMJ Qual Saf. 2018;27(9):673–82.
de Jong LD, Francis-Coad J, Wortham C, Haines TP, Skelton DA, Weselman T, et al. Evaluating audio-visual falls prevention messages with community-dwelling older people using a World Cafe forum approach. BMC Geriatrics. 2019;19(1):345.
O’Donnell D, Shé ÉN, McCarthy M, Thornton S, Doran T, Smith F, et al. Enabling public, patient and practitioner involvement in co-designing frailty pathways in the acute care setting. BMC Health Serv Res. 2019;19(1):797.
Russ S, Latif Z, Hazell A, Ogunmuyiwa H, Tapper J, Wachuku-King S, et al. A Smartphone app designed to empower patients to contribute toward safer surgical care: community-based evaluation using a participatory approach. Jmir Mhealth Uhealth. 2020;8(1):e12859.
Mazuz K, Biswas S, Lindner U. Developing self-management application of fall prevention among older adults: a content and usability evaluation. Front Digital Health. 2020;2:11.
Hjelmfors L, Strömberg A, Friedrichsen M, Sandgren A, Mårtensson J, Jaarsma T. Using co-design to develop an intervention to improve communication about the heart failure trajectory and end-of-life care. Bmc Palliat Care. 2018;17:17.
Horgan S, Hegarty J, Andrews E, Hooton C, Drennan J. Impact of a quality improvement intervention on the incidence of surgical site infection in patients undergoing colorectal surgery: Pre-test-post-test design. J Clin Nurs. 2023;32(15–16):4932–46.
Tyler N, Wright N, Grundy A, Waring J. Developing a core outcome set for interventions to improve discharge from mental health inpatient services: a survey, Delphi and consensus meeting with key stakeholder groups. BMJ Open. 2020;10(5):e034215.
Ward ME, De Brún A, Beirne D, Conway C, Cunningham U, English A, et al. Using Co-Design to Develop a Collective Leadership Intervention for Healthcare Teams to Improve Safety Culture. Int J Environ Res Public Health. 2018;15(6):1182.
Berthelsen DB, Simon LS, Ioannidis JPA, Voshaar M, Richards P, Goel N, et al. Stakeholder endorsement advancing the implementation of a patient-reported domain for harms in rheumatology clinical trials: outcome of the OMERACT safety working group. Semin Arthritis Rheum. 2023;63:152288.
Okkenhaug A, Tritter JQ, Landstad BJ. Developing a research tool to detect iatrogenic adverse events in psychiatric health care by involving service users and health professionals. J Psychiatr Ment Health Nurs. 2023;00:1–12.
Musbahi A, Clyde D, Small P, Courtney M, Mahawar K, Lamb PJ, et al. A systematic review of patient and public involvement (PPI) in bariatric research trials: the need for more work. Obes Surg. 2022;32(11):3740–51.
Patel VA, Shelswell J, Hillyard N, Pavitt S, Barber SK. A study of the reporting of patient and public involvement and engagement (PPIE) in orthodontic research. J Orthod. 2021;48(1):42–51.
Price A, Schroter S, Snow R, Hicks M, Harmston R, Staniszewska S, et al. Frequency of reporting on patient and public involvement (PPI) in research studies published in a general medical journal: a descriptive study. BMJ Open. 2018;8:e020452.
Amadea T, Anne-Marie B, Louise L. A researcher’s guide to patient and public involvement. 2017.
Download references
Acknowledgements
This research is funded as part of the Collective Leadership and Safety Cultures (Co-Lead) research programme which is funded by the Irish Health Research Board, grant reference number RL-2015–1588 and the Health Service Executive. The funders had no role in the study conceptualisation, design, data collection, analysis, decision to publish, or preparation of the manuscript.
Author information
Authors and affiliations.
UCD Centre for Interdisciplinary Research, Education and Innovation in Health Systems (UCD IRIS), School of Nursing, Midwifery and Health Systems, Health Sciences Centre, University College Dublin, Dublin, Ireland
Sahar Hammoud, Laith Alsabek, Lisa Rogers & Eilish McAuliffe
Department of Oral and Maxillofacial Surgery, University Hospital Galway, Galway, Ireland
Laith Alsabek
You can also search for this author in PubMed Google Scholar
Contributions
S.H and E.M.A designed the study. S.H developed the search strategies with feedback from L.A, L.R, and E.M.A. S.H conducted all searches. S.H and L.A screened the studies, extracted the data, and assessed the quality of PPI reporting. S.H analysed the data with feedback from E.M.A. S.H drafted the manuscript. All authors revised and approved the submitted manuscript. All authors agreed to be personally accountable for the author's own contributions and to ensure that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.
Authors’ information
Corresponding author.
Correspondence to Sahar Hammoud .
Ethics declarations
Ethics approval and consent to participate.
Not applicable.
Consent for publication
Competing interests.
The authors declare no competing interests.
Additional information
Publisher’s note.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Supplementary Information
Supplementary material 1., supplementary material 2., rights and permissions.
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.
Reprints and permissions
About this article
Cite this article.
Hammoud, S., Alsabek, L., Rogers, L. et al. Systematic review on the frequency and quality of reporting patient and public involvement in patient safety research. BMC Health Serv Res 24 , 532 (2024). https://doi.org/10.1186/s12913-024-11021-z
Download citation
Received : 10 January 2024
Accepted : 21 April 2024
Published : 26 April 2024
DOI : https://doi.org/10.1186/s12913-024-11021-z
Share this article
Anyone you share the following link with will be able to read this content:
Sorry, a shareable link is not currently available for this article.
Provided by the Springer Nature SharedIt content-sharing initiative
- Patient and public involvement
- Patient participation
- Research reporting
- Research involvement
BMC Health Services Research
ISSN: 1472-6963
Engaging Patients and Families in Safety: Recommendations, Resources, and Case Examples
Why it matters.
In September 2020, the Institute for Healthcare Improvement (IHI)-convened National Steering Committee for Patient Safety (NSC) announced the release of Safer Together: A National Action Plan to Advance Patient Safety . This report draws from evidence-based practices, widely known and effective interventions, exemplar case examples, and newer innovations.
The National Action Plan prioritizes the following four foundational — and interdependent — areas for developing a total systems approach to accelerate and sustain improvement in patient safety:
- Culture, Leadership, and Governance
- Patient and Family Engagement
- Workforce Safety
- Learning System
The following excerpt (adapted from the Implementation Resource Guide: A National Action Plan to Advance Patient Safety ) focuses on patient and family engagement.
To develop a total systems approach to advance patient safety, health care organizations must commit to the goal of fully engaging patients, families, and care partners in all aspects of care at all levels. To do this, the Safer Together: A National Action Plan to Advance Patient Safety makes the following recommendations:
- AHRQ Questions Are the Answer
- Choosing Wisely
- Prepare for Your Care
- The Conversation Project
- Engage patients, families, and care partners in the co-production of care . Health care leaders and health care professionals need to fully engage with patients, families, and care partners in ongoing co-design and co-production of their care.
- Include patients, families, and care partners in leadership, governance, and safety and improvement efforts . Health care leaders and governance bodies need to involve patients, families, and care partners from all backgrounds in health care oversight, design, and improvement, as well as harm reduction efforts.
- Ensure equitable engagement for all patients, families, and care partners . To ensure the ongoing engagement of patients, families, and care partners in safety, health care leaders must actively and equitably partner with all patients, families, care partners, and relevant community organizations.
- Promote a culture of trust and respect for patients, families, and care partners . Health care leaders must ensure that health care professionals and all personnel interact respectfully and transparently with patients, families, and care partners and with each other.
Consider these case examples as you strengthen your efforts to engage patients and families:
- OpenNotes facilitates open and transparent communication in health care by encouraging clinicians to share their visit notes with patients. Studies on sharing visit notes with patients and families indicate that this kind of transparency increases patient engagement, strengthens patient-provider relationships, and enhances safety. The OpenNotes academic research and advocacy group is based at Beth Israel Deaconess Medical Center and Harvard Medical School and works with collaborators nationally and internationally. The team is supported entirely by federal and philanthropic grants and does not develop software or products for sale.
- MedStar Health is a not-for-profit health care organization that oversees 120 sites, including ten hospitals in the Baltimore–Washington metropolitan area. To engage patients as partners, it has embedded Patient and Family Advisory Councils for Quality and Safety (PFACQS) at the system level, in every hospital, and in its medical groups. The PFACQS report to the Board of Directors Safety and Quality Committees in each hospital as well as at the system level.
- Emory Healthcare consists of 11 hospitals and offers health care services across more than 250 provider locations in Georgia, including primary care and urgent care . Emory leaders have established a robust Patient and Family Advisor (PFA) program that serves to guide decisions regarding a wide range of quality, safety, and operational improvement initiatives. Requests for partnerships are managed by using a database to match PFA’s lived experiences, skills, and interests to relevant improvement initiatives.
For more details, download the Safer Together: A National Action Plan to Advance Patient Safety , Self-Assessment Tool , and Implementation Resource Guide.
Related Insights
An official website of the United States government
The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.
The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.
- Publications
- Account settings
Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .
- Advanced Search
- Journal List
- Patient Prefer Adherence
Patient involvement in clinical research: why, when, and how
José a sacristán.
1 Medical Department, Lilly Spain, Madrid, Spain
Alfonso Aguarón
2 Myeloma Patients Europe, Madrid, Spain
Cristina Avendaño-Solá
3 Research Ethics Committee, University Hospital Puerta de Hierro, Madrid, Spain
Pilar Garrido
4 Oncology Department, Hospital Ramón y Cajal, Madrid, Spain
Juan Carrión
5 FEDER (Spanish Federation for Rare Diseases), Madrid, Spain
Alipio Gutiérrez
6 National Association of Health Journalists, Madrid, Spain
Robert Kroes
7 Clinical Open Innovation, Lilly Europe, Amsterdam, the Netherlands
Angeles Flores
The development of a patient-centered approach to medicine is gradually allowing more patients to be involved in their own medical decisions. However, this change is not happening at the same rate in clinical research, where research generally continues to be carried out on patients, but not with patients. This work describes the why, when, and how of more active patient participation in the research process. Specific measures are proposed to improve patient involvement in 1) setting priorities, 2) study leadership and design, 3) improved access to clinical trials, 4) preparation and oversight of the information provided to participants, 5) post-study evaluation of the patient experience, and 6) the dissemination and application of results. In order to achieve these aims, the relative emphases on the ethical principles underlying research need to be changed. The current model based on the principle of beneficence must be left behind, and one that upholds the ethical principles of autonomy and non maleficence should be embraced. There is a need to improve the level of information that patients and society as a whole have on research objectives and processes; the goal is to promote the gradual emergence of the expert patient.
Introduction
Patients are gaining a more active role in health care systems. The move toward patient-centered medicine aims to provide the best health care for each individual patient, taking his or her goals, preferences, and values into account. 1 Doctor–patient relationships are changing, and concepts such as shared decision making and patient empowerment 2 are becoming a reality.
Patient advocacy groups now claim that their opinions must have greater influence on the decisions that affect them, which is reflected in the phrase “nothing about me without me”. The development of new health care management models where patients become clients and the enormous expansion of information technology are additional factors that contribute to accelerate this change. 3 Patient-centered medicine cannot be practiced without patients participating in their own health care decisions and in the research that informs such decisions. 4
Although this cultural shift is beginning to change the way we understand health care, it is not having the same impact on the research process. 5 This may be because society does not see patient responsibility to participate in research as obvious as the responsibility to participate in their own medical care. 4 According to the predominant culture, research is performed on patients, not with patients. 6 Thus, patients continue to be regarded as a source of data and not as the true protagonists in the process. Initiatives have been developed in recent years to change this situation, but quite often these end up being merely tokenistic. 7 This paper analyzes the reasons why patients should be more actively involved in research and designates the phases in which they could participate ( Figure 1 ). Moreover, there are specific examples of how patients can achieve the level of true partnership in the research process.
How to engage patients in clinical research?
Identifying research priorities
The fact that the questions of clinical research are scientifically relevant does not necessarily mean that they are relevant from the patient’s perspective. 5 Although patients should play an active role in setting research priorities, such participation continues to be the exception rather than the rule. 8 , 9 Patient involvement is essential for achieving true translational research. 10 Nevertheless, most research questions are posed from a medical or regulatory perspective, and they are often based on what has been called the “culture of the laboratory”, which is excessively focused on basic science and often removed from the true needs of patients. 11 , 12
The Patient-Centered Outcomes Research Institute 13 in the USA and the James Lind Alliance ( www.lindalliance.org ) 14 in the UK have outlined the differences in research priorities between doctors and patients and instituted measures to ensure that patients participate in the process. The Patient-Centered Outcomes Research Institute has started to include patients in the review and prioritization of its research projects, 13 and the aims of the James Lind Alliance include the creation of a database of the principal uncertainties surrounding medical treatments (UK DUETs -database of uncertainties about the effects of treatments). These uncertainties are classified as known–unknown, in order to establish where new evidence would need to be generated, and which uncertainties already lend themselves to a systematic review of current evidence. 15 The Alliance studies have been based on systematic prioritization devised jointly by patients, caregivers, and clinicians. The results have made clear that aspects related to real-life management of disease matter most to patients. For example, in a study performed on patients with asthma, the top priorities were management of the adverse effects associated with bronchodilator treatment and management of asthma associated with other health conditions. 12 The identification and proper handling of patient needs in clinical research will yield advantages in terms of clinical and economic benefits. It has been estimated that up to 80% of the expense involved in gathering evidence in research could be avoided. 16 Considering patients’ priorities could contribute to more efficient clinical research.
Leading and designing research
Although it is not yet common practice, some patient associations have begun to lead research projects. 17 The increasing development of health social networks and crowd-sourced studies has enabled organizations such as PatientsLikeMe and 23andMe to promote research projects led by patient groups. 18 There are actually several examples of studies that use this method. 19 The proliferation of these networks, based on the altruistic transfer of personal data by patients, requires proper handling in terms of privacy and data protection.
Patient involvement is crucial for identifying the questions to ask and the outcomes to assess, 4 therefore it is increasingly common to involve patients or patient advocacy groups on study design. 20 – 22 This involvement could help to achieve more relevant results, while at the same time improving patients’ recruitment and retention. Conducting comparative effectiveness research 20 based on real-life data, long-term follow-up, comparisons with best current treatments, greater emphasis on individualized treatment, and, most of all, selection of variables relevant to patients or patient-reported outcomes (PROs) 21 should become a priority for patients engaged in study design. 22
It has been stated that >75% of patients with diabetes would prefer that clinical trials include variables that measure the impact of the disease on their quality of life, such as the onset of kidney failure and dialysis, or blindness, rather than surrogate variables such as HbA1c. 23 In oncology, there are examples that demonstrate the advantages of including patient-experience end points and not only the traditional survival-based end points. 24 There are similar examples for other diseases. 23 , 25 – 27 Worthy of special mention is the OMERACT (Outcome Measures in Rheumatoid Arthritis Clinical Trials) project, which has demonstrated the benefits of increasing involvement of patients with rheumatologic diseases in the design of clinical trials, particularly pertaining to identification of variables that matter to them. 27 It is encouraging that the regulatory agencies have already taken some steps to ensure the participation of patients in study design on the assumption that without this perspective the risk-benefit trade-off analysis for new pharmaceuticals would be incomplete.
Although an increasing number of clinical studies include them, the use of PROs, defined as “any report of the status of a patient’s health condition that comes directly from the patient, without interpretation of the patient’s response by a clinician or anyone else” 28 is still limited. 29 Only 18% of clinical trials in the area of diabetes 30 and 16% of trials in the cardiovascular field 31 include PROs. Likewise, fewer than 30% of drug data sheets include PROs. 32 , 33 The development of guidelines by regulatory agencies on the inclusion of PROs in the evaluation of new drugs 34 will probably increase their use in the coming years.
To be useful in clinical research, PROs must be relevant and adequately validated. 35 Unfortunately, this has not been the case with many of those used in research 36 , 37 and the scales continue putting too much emphasis on symptoms. Simple and easily measurable PROs based on objective metrics 38 that go beyond the mere symptoms of the disease must be developed and utilized. These must evaluate aspects such as quality of life and functional status, using global ratings provided by the patients themselves. 32 The development of computerized health applications (apps) can facilitate the use of PROs. However, it must be ensured that such applications actually measure what they are intended to measure 39 and that they are adequately regulated. 40
Improving access to clinical trials
If participation in clinical research is a societal obligation, 41 then the ability to do so should be a patient right. More than 70% of the general population believes that patients need more opportunities to participate in clinical trials. 42 In spite of this, only a small portion of patients who could participate in a clinical trial are able to do so. Participation in clinical trials typically does not exceed 5% of patients with cancer. 42 This is because only those patients whose doctors are also investigators participate in clinical trials. The final aim of everyone involved in the development of clinical trials and of health care systems that seek excellence should be to enable all patients who meet the selection criteria to participate in research, if they desire to do so.
Access for more patients, and for patients with different perspectives, would speed up obtaining relevant knowledge and results that are more applicable to heterogeneous groups of patients. Access to research is influenced by socioeconomic, racial, and ethnic factors and by the type of health insurance. 43 , 44 Programs to prevent such disparities should be set up to ensure that all patients who wish to participate in clinical trials can do so. It would be very beneficial for improving patients’ access to clinical research to enhance and systematize information about ongoing studies and about the sites participating in them. Patient associations should receive this information so they can pass it along with their associates. In addition, opportunities offered by new technologies should not be neglected. In this regard, clinical trial databases should include information about the characteristics of ongoing studies as well as details about the participating sites. There are already several drug agencies (eg, www.clinicaltrialsregister.eu in the European Union) that include such information. Recent initiatives, such as Trials4Me ( http://trials4me.lillycoi.com/ ) which uses Google Maps to locate sites participating in clinical trials from the Clinicaltrials.gov database, will greatly facilitate patient searches. Just as there is a unified reference database for clinical trials ( ClinicalTrials.gov ), a platform should be created with unified and standardized information directed to patients.
The growing development of learning health care systems should give rise to “point of care research”, where clinical research is embedded into clinical practice. 45 , 46 A large number of patients could have access to comparative effectiveness studies in which health interventions used in normal clinical practice are evaluated. This would contribute not only to the more efficient generation of new and relevant knowledge for patients but also to a speedier application of the results. The enormous development of electronic health records and information technologies could help in carrying out randomized database studies 47 (or randomized registry trials), a method recently termed “the next disruptive technology” in clinical research. 48 This method proposes the integration of research modules into usual clinical practice in order to combine the main strengths of randomized controlled trials (ie, initial randomization) and registries (ie, naturalistic follow-up). Obviously, in order for the development of big data to achieve its full potential, high-quality data with the end purpose of improving health care results in individual patients must be collected in medical practice and in research. 49
Adequate information about the study
It must be ensured that patients receive all the information they need to make a free and informed choice before consenting to participate in a study. The informed consent should constitute the means by which the principle of autonomy is applied. For the process to work properly, the information provided to patients must be complete, relevant, and easy to understand. Frequently, both the informed consent document (ICD) and the patient information sheet are excessively long documents that are difficult to read and understand. 50 , 51 Because of this, ~30% of participants are unaware of basic aspects of the research, such as the existence of random assignment, the possibility that they will be assigned to placebo, the fact that adverse effects could occur during the study, and even the availability of an insurance policy. 52 , 53
The ICD needs to be restored to its basic function, which is to ensure that patients have suitable information, not only about the overall aims of clinical research but also about the aims and characteristics of the study, its potential risks and benefits, and the option of withdrawing from it if they so desire. For this to happen, the informed consent process should no longer be viewed as an administrative requirement, which is intended to protect the researchers more than the patients themselves. 54 According to the new Regulation of the European Parliament on Clinical Trials on Medical Products for Human Use, the written information provided to the trial subjects or their legal representatives to obtain informed consent “shall be kept comprehensive, concise, clear, relevant, and understandable to a layperson”. 55
It is essential for participants to understand that the purpose of clinical research is to generate useful information for future patients and not necessarily to achieve a therapeutic benefit, since this cannot be guaranteed. Nevertheless, many patients participate in clinical trials because of the trust that they have in their doctors and the assumption that their participation will give them access to better treatments. This therapeutic misconception 56 leads many patients (as well as many researchers) to confuse research with medical care and to tend to overestimate the potential benefits of participating in clinical trials. Although there are times when participation in a clinical trial is the only way in which a patient who has failed with other alternatives can access a new, potentially more effective medication, 57 systematic reviews offer no evidence that clinical trial participants achieve better outcomes than nonparticipants. 58
There are few examples of studies where the opinion of health care system users has been considered to improve the design and readability of the patient information sheets included in clinical trials. 59 It would be highly advisable for patients to participate in ethical discussions related to the studies to make sure that the information provided is clear and accessible. As pointed out earlier, this participation could be achieved not only by having patients involved in study design but also through the participation of patient representatives in the Institutional Review Boards (IRBs). Although IRB members support the participation of patients in their membership lists, 60 very few IRBs actually put this into practice. Often, for practical reasons, the non-health care professionals who belong to IRBs are members of the hospital administration. 60 In this regard, the new European Regulation for clinical trials represents a positive step forward, as it 55 explicitly establishes that “Member States should ensure the involvement of laypersons, in particular patients or patients’ organizations”. Although in theory any citizen could represent the interests of patients, the fact that one is or has been a patient could be an additional advantage, provided of course that the individuals in question are highly motivated and have a sufficient educational level.
It has been suggested recently that, for comparative effectiveness studies on interventions in normal clinical practice, a simplified ICD could be used, similar to the one for medical care, or that the ICD could even be dispensed with. 61 In the case of these “low-intervention clinical trials”, the European Regulation recognizes the need to adapt the ICD, 55 so patients could play an important role in the review and evaluation of these simplified documents. They could also participate in the design of decision aids that would help to ensure that the consent granted by the patients is truly informed. 62 , 63
Assessing patients’ experience
Carrying out a clinical trial is an enormously complex process. There are often problems with recruiting or retaining patients, which can make the process even more drawn out and expensive. Once the study is complete, it would be of great value to know about the opinions and experiences of the participants, so future studies could be designed better and achieve greater acceptance by doctors and patients. 64 , 65
There have been several studies evaluating the experiences of patients who have participated in clinical trials. In a survey covering ∼5,000 patients who had taken part in clinical trials at 15 research sites supported by the National Institute of Health, 73% of the participants rated their experience with a score of 9 or 10 on a 10-point scale. In addition, 97% stated that they would definitely or probably recommend a clinical trial to a friend or relative. 66 These results are similar to those from another study of patients with depression, in which 87% of those interviewed were glad that they had participated in the clinical trial and 75% stated that they would be willing to participate again in the future. 64 Approximately 30% of the patients in the latter study stated that their participation had increased their trust in their doctors. These and other surveys indicated that one of the main motivations of patients in participating in clinical trials is their willingness to help future patients (with percentages higher than 90%). 64 , 67 This should be a powerful argument for overcoming the therapeutic misconception and the underlying hidden paternalism.
Despite these positive results, there are several areas that need improvement. In the previously mentioned study, 66 those surveyed indicated that the ICD did not contain enough details about the study (20%) or about the risks (19%). They said that it had been difficult for them to understand the content (22%), that they had not had enough time to think over their decision before signing the ICD (21%), or that they had received some form of pressure from the research staff to participate (6%). In another study, participants mentioned problems with describing their experiences in the questionnaires that were included in the clinical trial. 68 Most likely, the patient experiences with respect to participation in clinical trials may vary with factors such as educational level, age, the trial phase, or the disease under study. It would therefore be advisable to collect these experiences in a systematic way to identify the problems and propose solutions adapted to each circumstance.
Informing participants about study results
According to some surveys, ∼95% of patients 69 and members of IRBs 70 believe that patients should be informed of the results of the research study. Nevertheless, this is a fairly uncommon practice, and the usual situation is that after participating in a study, patients are not notified of the results. 71
The reasons offered for not informing patients have to do with their supposed desire to not know the results and with the possible anxiety that these results could cause. Then there is the difficulty, in practice, of communicating the results or ethical conflicts that a negative result could involve for the doctor in his dual role as physician and investigator. 71 However, it would seem to be an ethical standard, dictated by respect for the participants, to thank them for their participation and to provide them with the aggregate study results before they are made public to the scientific community. 72 Although informing patients of individual results is a matter of controversy, it could be justified by the same favorable arguments used to support disclosing aggregate results. 73 Fortunately, the 2013 update to the Declaration of Helsinki states, for the first time, that patients should have the option to receive information about the overall results of the study. 74
From a logistics standpoint, giving the participants information about the results is a relatively easy requirement to implement. It would involve merely including this option on the information sheet given to subjects when they are invited to participate in the study, and at the same time establishing how the results will be communicated at the end of the study to those participants who want them. 72 , 73 Patients would have to be offered a summary of the aggregate results in understandable language. This would detail the most important results, their implications, the possible long-term effects that are foreseen, and how the results will be communicated to the scientific community. Whether the results are positive or negative, it is important for patients to know if the study yielded new information and was useful for the advancement of knowledge.
Several studies have evaluated the satisfaction of patients with different ways of communicating results. In one of these, most of the patients (80%) chose to receive the information in a letter. 75 Positive experiences have also been reported with presentations made by the investigator to a group of patients. 76 It would seem that patients prefer to receive the information in a more personalized manner, for example, in a telephone call, rather than impersonally via the sponsor’s press release. 77 These differences suggest that, ideally, an attempt should be made to communicate the results in a way that fits the preferences of the participants.
Disseminating and applying research findings
The study results should be communicated not only to the study participants but also to the wider public. This obligation is stated in the previously mentioned European Regulations on clinical trials. In 2013, the European Agency upgraded the EudraCT database to enable sponsors to enter result-related information from clinical trials, in line with guidance from the European Commission. Since 2014, with the launch of a final iteration of EudraCT, it is mandatory for sponsors to make publicly available the details of the study, including a summary of the results. Such information should use the language understandable by the general public and be presented in simple and accessible formats. It would be highly advisable to have patients involved in the preparation of these documents.
There should also be mention of the initiatives by some of the scientific journals, such as the Annals of Internal Medicine and the Journal of the American Medical Association , which publish “summaries for patients” and brief and approachable reviews of common diseases and their treatments. An additional step has been taken by the British Medical Journal , which is heading a major campaign to promote and accelerate patient participation in the various areas of the health care field. For example, they have included patients in the peer review process for research articles. 78
Although the purpose of research is to generate new knowledge that is able to improve outcomes for future patients, many of the results of research are never put into practice or are not implemented for many years. It requires an estimated average of 17 years for only 14% of new discoveries to enter into daily clinical practice. 79 It is of little use, and indeed would be unethical, to involve patients in research and ask them about their experiences if these are not used to improve care. 80 The experiences of patients (eg, through PROs) should be included not only in regulatory documents (eg, drug data sheets) but also in clinical practice guidelines. Patient preferences are an essential component in the health care personalization process. There are already guidelines in which recommendations are established on the basis of such preferences. 81 Patient participation with the groups of experts who prepare these clinical guides would ensure that patient perspectives and preferences are considered.
To achieve greater transparency in the research process, some initiatives have been launched aimed at improving the dissemination of research results. In spite of the potential risks involved, 82 there are unquestionable advantages to the development of an open scientific approach in which “data sharing” is the norm. 83 From the standpoint of society, such an approach could generate greater confidence in the results of research, improving clinical trial participation and funding. 84 The Institute of Medicine has established recommendations for “guide sharing of clinical trial data”. 85 In addition, the European Medicines Agency 86 has published standards for promoting transparency and the dissemination of research results and allows different interest groups to request data from studies. Surprisingly, in the past 4 years, only 5.5%, 1.5%, and 0.5% of requests for access to European Medicines Agency documents were submitted by the general public, patient organizations, and nonprofit organizations, respectively. 86 It would not be surprising if the growing interest in research by patient organizations, many of which already have their own scientific advisors, were to generate an increase in requests for access to patient data.
Information and education: the expert patient
The earlier sections discussed the why, when, and how of active patient participation in research. However, there are two requirements to be met for all of these proposals to be implemented: 1) society must be much better informed on the aims and processes of clinical research and 2) the concept of the “expert patient” must be developed. Regarding the first requirement, there is a need to normalize society’s image of research. The general public’s scant familiarity with the basic aspects of research, and the fact that such familiarity tends to focus on the most negative aspects of research, contributes to it being perceived with a certain degree of suspicion and even fear.
Health care professionals, regulators, communication media, patient associations, and pharmaceutical companies need to collaborate to provide proper information about the purposes of research and the mechanisms available to protect participants. Society must realize that there is no progress without research, and that without patient participation in current studies, there can be no new knowledge to benefit future patients. For the same reason, we owe our current advantages of many medical advances to patients who participated in clinical trials in the past. The development and dissemination of campaigns in the media and educational materials can be of great help. The web pages of regulatory agencies such as the US Food and Drug Administration ( http://www.fda.gov/ForPatients/ClinicalTrials/default.htm ), national health institutes, such as the NIH in the USA ( http://www.nih.gov/health/clinicaltrials/index.htm ), and scientific associations such as the American Society of Clinical Oncology ( http://www.cancer.net/ ) include patient-directed information on the basic principles of clinical research. Similarly, some very interesting initiatives have emerged, such as Healthtalkonline ( http://healthtalkonline.org/ ), supported by Oxford University, that include videos in which actual patients relate their experiences with clinical trials and clarify certain basic aspects of the research process.
Second, many of the proposals made in this paper will not be feasible unless they include expert patients. 87 This concept, based on the idea that implementation of educational and self-care programs could help patients with chronic diseases who take an active role in managing their own conditions, 88 applies to the field of clinical research. Preparing patients to be experts in research will require an investment in their education. Of particular note here are the training courses offered by the European Patients’ Academy on Therapeutic Innovation (EUPATI), a consortium of 30 European organizations dedicated to providing information and resources on medical research to individuals affected by various diseases. The expert course has been designed to teach patients or their family members all about the clinical trial process, from preclinical research all the way through regulatory approval. The course consists of six training modules: 1) discovery of medicines and planning of medicine development; 2) nonclinical testing and pharmaceutical development; 3) exploratory and confirmatory clinical development; 4) clinical trials; 5) regulatory affairs, medicinal product safety, pharmacovigilance, and pharmacoepidemiology; and 6) health technology assessment principles and practice. The purpose is to empower these patients to participate actively in the research process, by direct collaboration with the industry, with regulatory agencies, or with patient associations. 89 The education of these expert patients could help to change the current situation, in which “experiential knowledge is often seen as less valuable than scientific knowledge”. 90
There are many opportunities for patients to participate more actively in the entire research process. As has been noted throughout this paper, some of the proposals are relatively easy to put into practice ( Table 1 ). The lack of implementation suggests that there are cultural barriers holding back the change process. The most urgent need is probably to change the relative emphases assigned to the ethical principles guiding the physician/investigator–patient relationship. The classic relational model, based on the principle of beneficence, which in turn is based on the authority of the physician, may be responsible for the fact that research is performed for patients, but not with them. The therapeutic misconception, the absence of patients on IRBs, the poor quality of information provided to participants, and the low levels of patient participation in establishing research priorities and study design are all elements indicating that the benevolence principle predominates in the research field.
Specific initiatives to engage patients in the research process
| Identifying research priorities | James Lind Alliance ( ) |
| PCORI ( ) |
| Leading and designing research | PatientsLikeMe ( ) |
| 23andME ( ) |
| OMERACT ( ) |
| Improving access to clinical trials | European Union ( ) |
| Trials 4 Me ( ) |
| NIH ( ) |
| Adequate information about the study | Readability of informed consent documents and patient information sheets |
| Assessing patients’ experience | Systematic collection of opinions and experiences of the participants |
| Informing participants about the study results | Process to communicate the results at the end of the study |
| Disseminating and applying research findings | Promoting transparency |
| Databases of randomized clinical trials |
| Information for patients in medical journals |
| Information and education | FDA ( ) |
| NIH ( ) |
| ASCO ( ) |
| Oxford University ( ) |
| EUPATI( ) |
Abbreviations: EUPATI, European Patients’ Academy on Therapeutic Innovation; FDA, Food and Drug Administration; NIH, National Institute of Health; PCORI, Patient-Centered Outcomes Research Institute; OMERACT, outcome measures in rheumatoid arthritis clinical trials; ASCO, American Society of Clinical Oncology.
For the changes described to become realities, physician/investigator–patient relationships should be based on the principles of autonomy and non-maleficence. Moreover, there can be no autonomy as long as patients lack the information they need. Providing the population with more information about the aims of research will help to normalize the process, generating trust and promoting participation in research. There must be improvement in the quality of information provided to participants, and promotion of the emergence of the expert patient: one who can become actively involved in all of the activities described in this paper and who is capable of properly representing the interests of other patients.
JAS, RK and AF are employees of Lilly Spain. The opinions and ideas expressed in this article are personal and do not necessarily represent those of the company. The authors report no other conflicts of interest in this work.
IMAGES
VIDEO
COMMENTS
FDA's Role in Reviewing Research Proposals for Patient Safety. FDA has a critical role in ensuring the safety of patients in clinical trials studying drugs or biological products for the ...
Background: Institutional review boards (IRBs), duly constituted under the Office of Human Research Protection, have the federally mandated responsibility of reviewing research involving human subjects to ensure that a proposed protocol meets the appropriate ethical guidelines before subjects may be enrolled in any study. The road leading to the current regulations and ethical considerations ...
In doing so, we have not just studied patient engagement in patient safety but produced measures, reporting systems, interventions and training that have the potential to reduce harm and improve the health of patients. We established comprehensive and effective systems for involving patients and the public in codesign and coproduction of applied health research and evaluated the impact of this ...
A multidisciplinary team developed a policy-based approach that provides guidance for using peer review protected information for safety research while maintaining peer review privilege. The approach includes project approval by an ad hoc review committee, signed confidentiality agreements by investigators and study staff, early removal of case identification numbers, standards for maintaining ...
The inaugural round of merit review for the Patient-Centered Outcomes Research Institute (PCORI) in November 2012 included patients and other stakeholders, as well as scientists. This article examines relationships among scores of the 3 reviewer types, changes in scoring after in-person discussion, …
This chapter explains why clinical practice guidelines are needed to improve patient safety and how further research into safety practices can successfully influence the guideline development process. There is a description of the structured process by which guidelines that aim to increase the likelihood of a higher score are created. Proposals are made relating to (a) the live updating of ...
Plain English summary Patient safety is a growing research area. However, although patients and the public are increasingly involved in clinical research, there is little guidance on how best to involve patients in patient safety research. Here we focus on how patients can contribute to the design of patient safety research. We conducted a workshop with patients as part of a project exploring ...
Background and Objectives for the Systematic Review The critical systematic review and analysis of Patient Safety Practices is an expansion of earlier evidence reports and current listing of Safe Practices for Better Healthcare 2010 Update by the National Quality Forum's (NQF)1. The analysis will assess the evidence of the effectiveness of new safe practices and the adoption of scientific ...
Patient safety research: an overview of the global evidence A K Jha,1,2,3 N Prasopa-Plaizier,4 I Larizgoitia,4 D W Bates,1,2 On Behalf of the Research Priority Setting Working Group of the WHO World Alliance for Patient Safety
Aim To focus on the "research" aspect in Patient Safety. Five important domains will be discussed in detail: 1) Measuring harm 2) Understanding causes
Background. The mission of the Agency for Healthcare Research and Quality (AHRQ) is to produce evidence to make health care safer, higher quality, and more accessible, equitable, and affordable, and to work within the U.S. Department of Health and Human Services and with other partners to make sure the evidence is understood and used.
The patient safety review and response and advice and guidance teams perform a core function within the National Patient Safety team, working to review and analyse sources of information on patient safety events to identify new or under recognised risks. These teams then lead on the development of ...
These international, consensus-based, patient-reported outcomes (PRO)-specific guidelines aim to lay out the ethical considerations that researchers, research ethics committees, and funders should take into account when conducting or reviewing PRO clinical research.
The Making Healthcare Safer (MHS) reports from the Agency for Healthcare Research and Quality (AHRQ) 1-3 have provided reliable information for improving the safety and quality of care for patients since 2001. The reports—providing an analysis of the evidence for various patient safety practices (PSPs)—have served as a consolidated and up-to-date source of information for multiple ...
Background Unsafe medical care may cause substantial morbidity and mortality globally, despite imprecise estimates of the magnitude of the problem. To better understand the extent and nature of the problem of unsafe care, the WHO World Alliance for Patient Safety commissioned an overview of the world's literature on patient safety research. Methods Major patient safety topics were identified ...
1 INTRODUCTION. Patient safety forms the key aspect of healthcare quality and is a serious global public health concern (Tevžič et al., 2021).The safety of patients is influenced by several factors and varies depending on the level of the healthcare system (Klemenc-Ketiš et al., 2020).As an entry point to the healthcare system, the emergency department serves a critical role and is an ...
Background In recent years, patient and public involvement (PPI) in research has significantly increased; however, the reporting of PPI remains poor. The Guidance for Reporting Involvement of Patients and the Public (GRIPP2) was developed to enhance the quality and consistency of PPI reporting. The objective of this systematic review is to identify the frequency and quality of PPI reporting in ...
Preliminary Prioritization of Patient Safety Practices (PSPs) Appropriateness and Regrouping. From the horizon scan and previous Making Healthcare Safer (MHS) reports, we identified a total of 136 PSPs that could be reviewed.
Why It MattersTo develop a total systems approach to advance patient safety, health care organizations must commit to fully engaging patients and families in every aspect of care at all levels. In September 2020, the Institute for Healthcare Improvement (IHI)-convened National Steering Committee for Patient Safety (NSC) announced the release of Safer Together: A National Action Plan to Advance ...
INTRODUCTION. The development of systematic reviews is considered a means of enabling clinicians to use evidence-based medicine (EBM) [], and the number of systematic reviews is growing quickly [].As literature searching forms the underlying basis of systematic reviews, the quality of the literature search is crucially important to the overall quality of the systematic review [].
Aim: To review available scientific evidence in the literature on theories that have been used to promote patient safety practices in the hospital environment.Methods: Design: A scoping review was ...
Introduction. Patients are gaining a more active role in health care systems. The move toward patient-centered medicine aims to provide the best health care for each individual patient, taking his or her goals, preferences, and values into account. 1 Doctor-patient relationships are changing, and concepts such as shared decision making and patient empowerment 2 are becoming a reality.
Abstract. Timely and effective prehospital care significantly impacts patient outcomes. Paramedics, as the frontline providers of emergency medical services, are entrusted with a range of critical responsibilities aimed at safeguarding the well-being of patients from the moment they initiate contact in the out-of-hospital environment to the time of handover at healthcare facilities.