Evaluating the Impact of the National Safety and Quality Health Service Standards

---

The National Safety and Quality Health Service (NSQHS) Standards provide a nationally consistent statement about the level of care consumers can expect from health service organisations and were introduced to protect patients from harm and drive quality improvement.

---

The Commission is currently conducting an evaluation of the impact of the NSQHS Standards.
What is the aim of the evaluation?
The evaluation aims to answer three questions:

• Have the NSQHS Standards made a difference to patient safety and the quality of care and what impact have they had?


• Have relevant objectives been achieved?


• How might the impact of the NSQHS Standards be measured into the future?

These questions are being considered at a macro level (national, state/territory or professional body), rather than at the individual health service level.
How is the evaluation being done?
Multiple sources and approaches are being used to identify changes in processes and outcomes that have occurred since the implementation of the NSQHS Standards.
For each of the 10 NSQHS Standards, key objectives were identified and from these relevant possible measures of processes and outcomes were identified. For example, NSQHS Standard 3: Preventing and controlling healthcare associated infections (NSQHS Standard 3) requires health service organisations to maintain hand hygiene programs and audit compliance (Figure 1). Uptake of hand hygiene activities and reported compliance rates since the implementation of the NSQHS Standards are process measures. Whether the objective of reducing infections has been achieved can only be measured by actual infection rates.
Figure 1. Evaluation framework applied to NSQHS Standard 3
This is a complex evaluation, in part because of the difficulties in retrospectively assessing national change in a heterogeneous health system with multiple factors contributing to change. Some aspects of the NSQHS Standards have been the focus of jurisdictional, clinical or other program initiatives prior to their introduction (for example, pressure injury and falls prevention programs). Nonetheless, the project is identifying useful sources of information that will help to track current and future outcomes in the areas covered by the NSQHS Standards.
Why look at the impact on processes?
While improvements in patient outcomes are the ultimate goal, and are being assessed for the evaluation, measuring the impact on processes is vital as it provides insight into whether implementation has been successful. Given the relatively early stage of implementation, measuring changes in process outcomes may also be useful where it is too early to expect to see changes in outcomes. Process evaluation can also help clarify whether or not it is reasonable to attribute any changes in various outcomes to implementation of the NSQHS Standards.
What data sources are being used?
The evaluation includes quantitative and qualitative approaches, and a range of data sources. As far as possible, existing routine data sources are being used, in order to limit the need for additional data collection.
Data sources include administrative hospital morbidity data, infections surveillance data, clinical incident systems, clinical registries and other existing data collections. Specific outcomes being considered include rates of healthcare associated infections, falls, pressure injuries, medication-related adverse events as well as any impact on cardiac arrest rates potentially resulting from better management of the acutely deteriorating patient.
In addition, some research is being conducted by the Commission to assess the impact of specific standards, including NSQHS Standards 1, 2, 3 and 9.
Table. Data sources for evaluation of the impact of the NSQHS Standards

---

 

• Interviews, data gathering and consultation with stakeholders

---

• Surveys of health professionals and governance bodies

---

• Interrogation of routine data collections

---

• Clinical quality improvement systems and registries

---

 
Next steps
The evaluation of the NSQHS Standards is due for completion in December 2015. The findings will be used to refine the NSQHS Standards and ensure they are achieving their objectives, together with the review being conducted by the Commission.

---

“Measuring the impact on processes is vital as it provides insight into whether implementation has been successful.”

---

For more information about the NSQHS Standards,
visit www.safetyandquality.gov.au/our-work/accreditation-and-the-nsqhs-standards/
You can also sign up to receive email newsletter updates or follow the Commission on Twitter @ACSQHC.