Associated material
Related literature
Other articles by authors
Related articles/pages
Download to ...
Share this article
Your browser does not support iframes
Email updates
Journal App
Explaining the effects of two different strategies for promoting hand hygiene in hospital nurses: a process evaluation alongside a cluster randomised controlled trial
Anita Huis*, Gerda Holleman, Theo van Achterberg, Richard Grol, Lisette Schoonhoven and Marlies Hulscher
Corresponding author:
Scientific Institute for Quality of Healthcare, Radboud University Nijmegen Medical Centre Nijmegen, Nijmegen, The Netherlands
Faculty of Health Sciences, University of Southampton, Southampton, UK
For all author emails, please .
Implementation Science 2013, 8:41&
doi:10.08-8-41
The electronic version of this article is the complete one and can be found online at:
Received:4 October 2012
Accepted:19 March 2013
Published:8 April 2013
& 2013 Huis et al.; licensee BioMed Central Ltd.
This is an Open Access article distributed under the terms of the Creative Commons Attribution License (), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
Background
There is only limited understanding of why hand hygiene improvement strategies are
successful or fail. It is therefore important to look inside the ‘black box’ of such
strategies, to ascertain which components of a strategy work well or less well. This
study examined which components of two hand hygiene improvement strategies were associated
with increased nurses’ hand hygiene compliance.
A process evaluation of a cluster randomised controlled trial was conducted in which
part of the nursing wards of three hospitals in the Netherlands received a state-of-the-art
strategy, including education, reminders, feedback, and optimising mate
another part received a team and leaders-directed strategy that included all elements
of the state-of-the-art strategy, supplemented with activities aimed at the social
and enhancing leadership. This process evaluation used four sets of measures: effects
on nurses’ hand hygiene compliance, adherence to the improvement strategies, contextual
factors, and nurses’ experiences with strategy components. Analyses of variance and
multiple regression analyses were used to explore changes in nurses’ hand hygiene
compliance and thereby better understand trial effects.
Both strategies were performed with good adherence to protocol. Two contextual factors
were associated with changes in hand hygiene compliance: a hospital effect in long
term (p & 0.05), and high hand hygiene baseline scores were associated with smaller
effects (p & 0.01). In short term, changes in nurses’ hand hygiene compliance were
positively correlated with experienced feedback about their hand hygiene performance
(p & 0.05). In the long run, several items of the components ‘social influence’ (i.e.,
addressing each other on undesirable hand hygiene behaviour p & 0.01), and ‘leadership’
(i.e., ward manager holds team members accountable for hand hygiene performance p
& 0.01) correlated positively with changes in nurses’ hand hygiene compliance.
Conclusion
This study illustrates the use of a process evaluation to uncover mechanisms underlying
change in hand hygiene improvement strategies. Our study results demonstrate the added
value of specific aspects of social influence and leadership in hand hygiene improvement
strategies, thus offering an interpretation of the trial effects.
Trial registration
The study is registered in ClinicalTrials.gov, dossier number: .
Keywords: P Q I H I Randomi LeadershipBackground
Strategies to improve adherence to practice guidelines are often multimodal and consist
of a number of potentially effective components and related improvement activities
[-] (See Table&). All of these components might influence effectiveness both independently and inter-dependently.
This poses challenges for strategy evaluation. A randomised controlled trial (RCT)
is the most rigorous way to evaluate the effectiveness of improvement strategies,
regardless of their complexity. However, published reports of RCTs mainly focus on
the outcomes, answering the question ‘Does it work?’ [,]. RCTs rarely answer the question of why an improvement strategy has been successful
or has failed []. Despite the CONSORT guidelines [], a detailed description of an improvement strategy — reporting on all components
and corresponding activities — and how well the strategy was performed is often lacking.
This equally applies to information on contextual aspects such as the environment
or setting, as well as factors that inhibited or promoted effectiveness [,]. Understanding RCT results is also complicated by the use of intention-to-treat analyses.
To provide unbiased comparisons among the treatment groups, individuals or clusters
are analysed according to the group (experimental or control) to which they were originally
allocated, regardless of whether they actually received the improvement strategy.
Therefore, it is necessary to combine the strength of an RCT with a well-designed
process evaluation [].
Explanation of terms used in this article
Process evaluations are important because they can clarify to what extent the improvement
strategy was performed in a uniform way, whether the target population actually received
the planned activities, what factors inhibited or promoted effectiveness, and what
the participants’ actual experiences with the executed strategy were [,-]. Process evaluations also provide information important to understanding and validating
theory-informed strategies. Identifying the mechanisms for how and why these strategies
produce successful change (or fail to produce change) is crucial to refining theory
and improving strategy effectiveness [].
Combined analysis of process and outcome data allows evaluations to explore associations
between strategy delivery and receipt, and outcomes on effectiveness []. In this way, insight is gained into the mechanisms responsible for the results,
which could improve the validity of the findings and help researchers understand the
potential generalizability of the improvement strategy [,,].
The case of hand hygiene: the HELPING HANDS study
Hospital acquired infections are the most common complications in hospital care, and
a major threat to patient safety [,]. Hand hygiene (HH) is considered the most important measure in the prevention of
hospital acquired infections [,,]. Unfortunately, compliance with HH recommendations is repeatedly found to be insufficient
[,,]. Many potentially effective strategies for improving HH compliance are described,
but most of the effects are small to moderate [,,]. Traditionally, strategies have concentrated on the healthcare professional or focused
on the introduction of new products and facilities [,]. However, often experienced barriers like negative role models, lack of management
involvement, and poor social culture are rarely addressed []. Using insights from the behavioural sciences and performing a strategy that also
targets social influence within teams and leadership could be a valuable addition
to HH implementation strategies [-].
In a recent study, we undertook a cluster randomised trial (the HELPING HANDS study)
at 67 nursing wards in three Dutch hospitals to compare the effectiveness of a state-of-the-art
strategy with a team and leaders-directed strategy for improving nurses’ compliance
with HH guidelines [,]. The state-of-the-art strategy was based on current evidence from literature on HH
compliance [,,]. This strategy targeted the individual and organizational level and included the
following components: education for improving relevant
for supporting the actual performance of HH; feedback as a means to provide insight
into current HH behaviour and to reinforc and screening for adequate
HH products and adequate facilities. The team and leaders-directed strategy was also
aimed at addressing barriers at team-level by focussing on social influence within
teams and strengthening leadership of the ward manager. The unique contribution of
this strategy was built upon the social learning theory [], social influence theory [], theory on team effectiveness [,], and leadership theory []. Together, these theories provide a coherent set of methods to target the social
context in which HH behaviour takes place. Table& provides an overview of our theory selecting process, including the characteristics
and key elements of the behaviour change theories. The identified key elements were
used to build our team and leaders-directed strategy that included all components
of the state-of-the-art strategy, supplemented with: gaining active commitment and
initiativ modelling by informal and setting
norms and targets within the team. Before the start of the intervention, all managers
participating in the team and leaders-directed group received a four-hour training
in coaching and motivating the nurses. During the intervention period, the ward manager
was assisted by an experienced coach in three team meetings. Also, two group sessions
were organised to support the ward managers and to discuss progress and difficulties.
Table& presents the content and related activities of both strategies.
Selected behaviour change theories matching barriers in performing HH
Description of the implementation strategies with the planned activities
Both strategies successfully improved hand hygiene compliance, but the team and leaders-directed
strategy showed better results []. The findings of this study indicated the added value of strategy components aimed
at social influence within teams and enhanced leadership of ward managers on nurses’
HH behaviour. However, these results provide no insight into the mechanisms of impact.
For instance, the extent to which nursing wards improved their HH compliance varied
considerably for both strategies, ranging from -2% to 70% improvement in the long
run. In addition, the effect size of the team and leaders-directed group was limited
by the intention-to-treat analysis, which is the main statistical approach for RCT
analyses. Wards were analysed according to the group state-of-the-art strategy or
team and leaders-directed strategy to which they were originally allocated. In the
HELPING HANDS study, thirty nursing wards and their managers were randomly assigned
to the team and leaders-directed group but ten wards declined to participate in the
team and leaders-directed strategy. Therefore, only twenty wards fully participated
in the team and leaders-directed group.
The current article expands on the findings of the HELPING HANDS study by linking
process and effectiveness evaluations. The aim of this paper is to ascertain which
components of the two HH improvement strategies can be particularly associated with
increased nurses’ HH compliance, as well as to explore other possible factors that
may be associated with changes in nurses’ HH compliance. We focused on three specific
questions:
1. What impact might variation in adherence to the improvement strategies as planned
have on changes in nurses’ HH compliance?
2. What impact might specific contextual factors as hospital and ward characteristics
have on changes in nurses’ HH compliance?
3. What impact might differences in nurses’ actual experiences with strategy components
have on changes in nurses’ HH compliance?
Setting and participants
The HELPING HANDS study was performed in three hospitals in the Netherlands: two general
hospitals and one university medical centre. Within the hospitals, all in-patient
nursing wards (n = 67) and all affiliated nurses participated in the study. We included
surgical wards (n = 21), internal medicine wards (n = 24), intensive care units (n
= 13), and paediatric wards (n = 9). Twenty wards received the team and leaders-directed
group, and 47 wards received the state-of-the-art group. Strategies were delivered
during a period of six months. Follow-up measurements took place directly after strategy
delivery (T2) and at six months after the end of strategy delivery (T3).
Measurements and data collection
Data were collected using a wide range of methods, including: student observations,
questionnaires to nurses, a ward structure survey, registration of website visitors,
structured logbooks of ward managers and coaches and researchers’ field notes of group
meetings. Using these data sources, we constructed four sets of measures.
Effect evaluation
Effects on nurses’ HH compliance
The primary outcome was the percentage of nurses’ actions in line with HH guidelines
in case of an opportunity to perform this action, according to the HH guidelines of
the World Health Organization [,]. We monitored nurses’ HH compliance unobtrusively during routine patient care before
and directly after strategy delivery, as well as six months later [].
Process evaluation
Adherence to the improvement strategies as planned
The measurement of adherence captures the following subcategories: content – whether
improvement activities were delivered as planned (yes/no); dosage – whether improvement
activities were delivered as often and long as planned (yes/no); coverage – the extent
to which the intended target group received the improvement activities [].
Education was assessed by monitoring the presence of instruction leaflets on the ward
and by measuring the number of nurses who completed the knowledge quiz. The use of
reminders was checked by measuring the presence of reminders (posters) at random moments
during the strategy delivery period. Feedback was assessed by checking the distribution
of performance feedback reports to ward managers and by a question from the study’s
survey asking if nurses had received performance feedback from the ward manager. In
addition, the extent to which products and facilities were available in each ward
was also explored by survey questions to ward managers and nurses. The attendance
of ward management and informal leaders to the training sessions and the support sessions
was derived from an attendance checklist. The use of coaching of ward management and
informal leaders was assessed by measuring the number of coaching sessions and the
total time spent on coaching. Additional details on coaching activities are available
from the authors on request. The use of organised team discussions for norm- and target-setting
was checked by measuring the number of team discussions performed, the number of nurses
attending per ward, the time investment per ward, and whether norms and targets were
established. Leadership was assessed by checking documented agreements on the following
points: whether the ward manager had discussed HH compliance rates during the team
whether the ward manager had prioritized good HH and whether
the ward manager had formulated specific activities to support the team members and
informal leaders. Finally, information on whether informal leaders served as role
models was derived from group discussion during the support sessions for ward managers
and informal leaders.
Contextual factors
We explored the influence of three contextual variables, namely: hospital, ward specialism
(e.g., general ward, surgical ward, paediatric ward or critical care ward) and the
HH compliance rate at baseline.
Nurses’ experiences with specific components of the improvement strategies
In order to explore the relationship between HH outcomes and nurses’ actual experiences
with different strategy components, we drew on the findings of a 7-subscale questionnaire
consisting of 24 items. Each item was a proposition on a specific component of the
improvement strategies. These components were education, reminders, feedback, facilities
and products, setting norms and targets, social influence and leadership. An example
of a proposition that explores nurses’ actual experiences with leadership is ‘my ward
manager holds team members accountable for HH performance.’ Nurses scored this proposition
on a 4-point Likert scale, ranging from strongly agree (4) to strongly disagree (1).
Negatively formulated propositions were recoded. Higher scores indicated more positive
experiences with respective components (Additional file ).
Additional file 1. Questionnaire on nurses’ experiences with strategy components.
Format: PDF
Size: 41KB This file can be viewed with:
Statistical analyses
In this study, our primary research goal was to understand the working mechanisms
of HH improvement strategies embedded in the relationship between strategy performance
and nurses’ HH compliance. Therefore, we combined data from the process evaluations
with data from the effect evaluation. To serve our research goal, we moved from the
original intention-to-treat analysis to an as-received basis, with 47 wards in the
state-of-the-art group and 20 wards in the team and leaders-directed group. Inputs
for the effect analysis, used in this paper, were based on the HH compliance findings
of the previously mentioned HELPING HANDS study. The effectiveness of the HELPING
HANDS study was examined using an ‘intention-to-treat’ analysis. However, 10 wards
declined to participate in the team and leaders-directed group and did not receive
any component of this strategy. We therefore explored whether the inclusion, in our
intention-to-treat analysis, of wards who did not receive the team and leaders-directed
strategy, might have resulted in different effects in changes in nurses’ HH compliance.
All data were analysed using SPSS version 19.0 (SPSS, Inc., Chicago, IL) and analyses
were performed at ward level.
Effect evaluation
Effects on nurses’ HH compliance: intention-to-treat versus as-received analysis
We compared the outcome data on changes in HH compliance of the intention-to-treat
analysis (37 wards in the state-of-the-art group and 30 wards in the team and leaders-directed
group) with the results of the as-received analysis (47 wards in the state-of-the-art
group and 20 wards in the team and leaders-directed group). We used descriptive statistics,
including mean and standard deviation, for the change in HH compliance between the
measurement points for each of the two strategies. One-way ANOVAs were used to test
whether there was a statistically significant difference between the group means for
both strategies. A p value of 0.05 or less was considered to indicate the statistical
significance of the difference between measurements at baseline (T1), directly after
strategy delivery (T2), and at six months after the end of strategy delivery (T3).
Process evaluations linked to effectiveness evaluations
Analysis of adherence to the improvement strategies and related changes in HH compliance
Frequencies and proportions were used to assess the adherence to the several components
of the improvement strategies. One-way ANOVAs were used to test the influence from
varying strategy components on HH compliance. If a strategy component was significant,
correlations between changes in nurses’ HH compliance and the significant term were
also examined within each strategy group using the Spearman correlation analysis.
Analysis of contextual factors and related changes in HH compliance
One-way ANOVAs were used to test the influence from the contextual factors hospital,
ward specialism, and the HH compliance rate at baseline. The correlation between nurses’
HH baseline scores and changes in nurses’ HH compliance was tested with the Pearson
correlation analysis. Next, we applied forced entry multiple regression analyses to
assess the impact of several potential explanatory variables on changes in HH compliance.
As an estimation for the explained variance of the model, an adjusted R-Squired was
determined.
Analysis of nurses’ actual experiences with specific components of the improvement
strategies and related changes in HH compliance
Descriptive statistics, including mean and standard deviation, were used to explore
differences in nurses’ actual experiences with specific strategy components between
nurses in the team and leaders-directed group, and in the state-of-the-art group.
Inclusion criteria for analysis were wards whose respondents returned ≥ 3 questionnaires.
One-way ANOVAs were used to test whether there was a statistically significant difference
between group means for both strategies. To determine whether differences in nurses’
actual experiences with strategy components predicted variation in HH compliance effects,
we tested non-parametric correlations with Spearman analyses between groups and within
Initially, 67 wards were included, 30 to the team and leaders-directed group, and
37 to the state-of-the-art group. Ten wards declined to participate in the team and
leaders-directed group because of a vacancy for the position of ward manager (2×),
reorganization of the ward (2×), workload of the ward manager ruled out other activities
(1×), inconvenient timing relating to the execution of the strategy (2×), or other
projects were given a higher priority (3×). Finally, 47 wards received only the state-of-the-art
strategy, and 20 wards received the team and leaders-directed strategy (Table&). At each point in time, 3,523 to 3,722 opportunities for HH were observed in 886
to 933 nurses. During the entire study, we obtained data on 10,785 opportunities for
HH in 2733 nurses.
Characteristics of the wards
Effect evaluation
Effects on nurses’ HH compliance: intention-to-treat versus as-received analysis
Table& displays the results of changes in nurses’ HH compliance derived from the intention-to-treat
analysis and the as-received analysis. In both analyses, the team and leaders-directed
group demonstrated better results on HH compliance than the state-of-the-art group.
The as-received analysis showed higher effect sizes for the team and leaders-directed
group than the intention-to-treat analysis. A statistically significant (p = 0.002)
increase in nurses’ HH compliance was observed in the long run (T3) in favour of the
team and leaders-directed strategy. The intention-to-treat analysis showed no significant
difference in nurses’ HH compliance between both strategies at T3.
Changes in HH compliance in participating hospitals during study period
Process evaluations linked to effectiveness evaluations
Adherence to the improvement strategies and related changes in HH compliance
Both improvement strategies were carried out with good adherence to protocol. Detailed
results on strategy adherence are described in Additional file .
Additional file 2. Adherence of nursing wards to strategy components.
Format: PDF
Size: 71KB This file can be viewed with:
Impact of variation in adherence to the components of the state-of-the-art strategy
In the adherence subcategory ‘content,’ we found that the main components of the state-of-the-art
strategy were generally delivered as planned. The ‘HH promotion event’ was not delivered
in one hospital. The infection control department of this particular hospital had
already organised an HH promotion event one year before the start of our study. Despite
the variation in delivering the ‘HH promotion event,’ no effect on changes in HH compliance
could be demonstrated (p = 0.384). The subcategory ‘coverage’ showed some variation
in the extent to which washstands were accessible. The analysis showed that variation
within these components had no effect on changes in HH compliance (p = 0.348).
The subcategory ‘coverage’ also demonstrated a significant difference between the
number of nurses from wards receiving the state-of-the-art strategy and the number
of nurses from wards receiving the team and leaders-directed strategy in completing
the knowledge quiz (13% and 37%; p = 0.001). This was positively correlated with changes
in HH compliance at both follow-up measurements (T1 to T2: p = 0.019; T1 to T3: p
= 0.016) However, completing the knowledge quiz did not predict variation in HH compliance
within groups of the state-of-the-art strategy (T1 to T2: p = 0.779; T1 to T3: p =
0.426) or within groups of the team and leaders-directed strategy (T1 to T2: p = 0.354;
T1 to T3: p = 0.452).
Impact of variation in adherence to the additional components of the team and leaders-directed
strategy (n = 20)
In the adherence subcategory ‘content,’ we found that all components of the team and
leaders-directed strategy were delivered as planned. Components that differed in adherence
across the wards concerned the subcategories ‘dose’ and ‘coverage.’ Five wards organised
only two team sessions instead of three team sessions. Thus, these wards did not receive
a full dose. However, this did not affect the course of nurses’ HH compliance (T1
to T2: p = 0.240; T1 to T3: p = 0.254). Full coverage was also not achieved for attending
two sessions in support of the role models and ward managers, but everyone took part
in at least one session. Variation in adherence within the component ‘support sessions’
had no effect on changes in HH compliance (ward managers T1 to T2: p = 0.262; T1 to
T3: p = 0.994; role models T1 to T2: p = 0.184; T1 to T3: p = 0.688). There was also
some variation in the average number of nurses that attended the team sessions, related
to total number of nurses employed. However, variation within this component had no
effect on changes in HH compliance (T1 to T2: p = 0.445; T1 to T3: p = 0.823). In
conclusion, the evaluation of strategy adherence did not provide any explanatory variables
associated with changes in nurses’ HH compliance.
Contextual factors and related changes in HH compliance
Our next step was to determine the impact of contextual factors on changes in nurses’
HH compliance. Two contextual factors were associated with changes in HH compliance:
type of hospital and HH performance at baseline. The ANOVA showed a hospital effect
on changes in HH compliance in long term (p = 0.036). HH compliance decreased in one
hospital in long term, while the HH compliance in the other two hospitals remained
stable or increased further. At baseline, the HH scores of all wards from the state-of-the-art
strategy and the wards that participated in the team and leaders-directed group were
comparable (p = 0.978). For both study groups, baseline HH scores were negatively
correlated with follow-up scores (r = -0.693; p = 0.000). Initially, short-term changes
in HH compliance (T1 to T2) revealed a specialism effect (p = 0.002). In particular,
the paediatric wards showed a smaller increase in HH compliance than the wards from
other specialisms. However, the baseline HH scores of the paediatric wards were significantly
higher than the baseline HH scores of other wards (p = 0.000). This alleged specialism
effect was, in reality, a baseline effect.
We then tested all significant variables in forced entry multiple regression analyses.
Table& presents the results from two multiple regression analyses. The basic model included
baseline HH compliance (covariate), hospital, specialism and strategy. The first model
analysed changes in HH scores from baseline (T1) to the first follow-up measurement,
directly after strategy delivery (T2). Baseline HH scores (p & 0.01) and hospital
(p & 0.05) contributed negatively to short-term changes in HH compliance. The team
and leaders-directed strategy contributed positively to short-term changes in HH compliance
(p & 0.01). The second model analysed changes in HH compliance from baseline (T1)
to the second follow-up measurement, six months after the end of strategy delivery
(T3). Baseline HH scores (p & 0.01) and hospital (p & 0.01) contributed negatively
to long-term changes in HH compliance. The team and leaders-directed strategy contributed
positively to long-term changes in HH compliance (p & 0.01). The adjusted R2 was 0.702 for the first model and 0.510 for the second model. This suggests that
70% and 51% of the variation in HH change scores could be explained by the regression
Summary of forced entry multiple regression analysis for variables predicting changes
in HH compliance in participating hospitals during study period
Nurses’ experiences with the improvement strategies and related changes in HH compliance
In this section, we explored differences in nurses’ actual experiences with strategy
components and how these differences affected changes in nurses’ HH compliance. A
total of 528 questionnaires out of 1,100 (369 questionnaires from the state-of-the-art
group and 159 from the team and leaders-directed group) were returned, giving a response
rate of 48%. Questionnaires of 515 nurses from 59 wards met the inclusion criteria
for analysis. Of these, 42 wards belonged to the state-of-the-art group (360 questionnaires),
and 17 wards to the team and leaders-directed group (155 questionnaires).
The ANOVA showed significant differences in actual experiences with several items
of the questionnaire between nurses from the state-of-the-art group and nurses from
the team and leaders-directed group. Nurses from the team and leaders-directed group,
who unlike the nurses from the state-of-the-art group were exposed to the strategy
components ‘setting norms and targets,’ ‘social influence’ and ‘leadership,’ experienced
more social support (p = 0.005), social influence (p = 0.046) and leadership (p =
0.011) with respect to HH performance. In addition, these nurses experienced more
priority for HH on their ward (p = 0.009) and experienced more feedback about their
HH performance (p = 0.000) than did nurses from the state-of-the-art group.
Table& displays nurses’ experiences with components of both improvement strategies and their
impact on changes in HH compliance. First, we examined the impact of strategy components
in both study groups (n = 67). In short term (T1 to T2) and in the long run (T1 to
T3), changes in nurses’ HH compliance were positively correlated with experienced
feedback about their HH performance (p & 0.05 and p & 0.01, respectively). In the
long run (T1 to T3), two items of the component ‘social influence’ correlated positively
with changes in nurses’ HH compliance: addressing each other on undesirable HH behaviour
(p & 0.01) and support from colleagues in performing HH (p & 0.01). Also in the long
run, five items of the component ‘leadership’ correlated positively with changes in
nurses’ HH compliance: regular attention to the adherence of HH guidelines (p & 0.05);
designation of HH as ward priority (p & 0.05); addressing barriers to enable HH as
recommended (p & 0.05); accountability for HH performance (p & 0.01); and encouraging
and motivating team members to perform HH (p & 0.01).
Nurses’ experiences with strategy components and correlations with changes in HH compliance
Within the state-of-the-art group (n = 47), we found a few correlations between nurses’
experiences with strategy components and changes in HH compliance. In short-term,
experienced knowledge of HH indications showed a negative correlation with HH change
scores (p & 0.05). In the long term, positive correlations with changes in HH compliance
could be demonstrated for one item of social influence, namely: addressing each other
on undesirable HH behaviour (p & 0.05). We also found positive correlations with changes
in HH compliance for two leadership items: accountability for HH performance (p &
0.01) and encouraging and motivating team members to perform HH (p & 0.05). We found
no significant correlations between scores on specific items and HH change scores
within the group of the team and leaders-directed strategy (n = 20).
Discussion
In this article, we examined which components of the HH improvement strategies were
particularly associated with increased nurses’ HH compliance, as well as other possible
factors that may have influenced nurses’ HH compliance. We therefore linked process
and effectiveness evaluations in the analysis of findings from the HELPING HANDS study
Effect evaluation: intention-to-treat versus as-received analysis
In this article, we have tried to explain the effects of two different HH improvement
strategies on changes in nurses’ HH. It is important to recognize that this research
goal requires a different view on the treatment effects compared to an evaluation
of effectiveness. The outcome suggests that the overall conclusions about the effectiveness
of the team and leaders-directed strategy arising from the original intention-to-treat
analysis may have underestimated the impact and strength of this strategy. The as-received
analysis showed higher effect sizes for the team and leaders-directed group than the
intention-to-treat analysis on both measurements points. In the long run, we now observed
a statistically significant (p = 0.002) increase in nurses’ HH compliance due to the
team and leaders-based strategy. This suggests that the team and leaders-directed
strategy might have had a more permanent impact on HH outcomes than shown by the intention-to-treat
analysis. This corresponds with the findings of Strange, et al. []. Their as-received analysis showed higher odds ratios in decreasing risky sexual
behaviour than the original intention-to-treat analysis, thereby suggesting that their
peer-led sex education program, if consistently implemented, probably had a greater
impact on study outcomes.
Effects of strategy adherence on nurses’ HH compliance
The evaluation of strategy adherence did not provide any explanatory variables associated
with changes in nurses’ HH compliance. Thus, variation in the HH outcomes across the
wards could not be explained by a so-called ‘failure of implementation’ []. Nevertheless, it is noteworthy that more nurses from the team and leaders-directed
group completed the knowledge quiz compared to nurses from the state-of-the-art group
(37% and 13%, p = 0.001). A possible explanation is that the team and
leaders-directed strategy positively influenced the adherence to specific components
of the state-of-the-art strategy.
Effects of contextual factors on nurses’ HH compliance
Hospital culture
The as-received analysis showed a hospital effect that was mainly due to one hospital.
Especially in the long run, HH compliance started to decrease in this particular hospital,
while HH compliance in the other two hospitals remained stable or increased further.
Little is known about how hospital cultural factors are associated with the implementation
of HH improvement strategies. The WHO [], Larson, et al. [] and Pittet [] emphasize the commitment of high-level administrators to create and support a culture
of safety and accountability. Culture manifests itself through the values, beliefs
and assumptions embedded in organizations and is reflected in ‘the way things are
done around here’ []. The two hospitals that showed sustainability in HH compliance designated HH as a
hospital-wide priority. The third hospital was less explicit and distinct in addressing
the goal of HH as an organizational priority. This raises the question of whether
the observed changes in HH compliance were affected by hospital culture.
Standard care activities
Although the average HH baseline scores of the wards were comparable between wards
from both groups, our analysis showed that a high baseline HH compliance was associated
with a smaller effect of both HH improvement strategies. High HH compliance at baseline
was particularly seen in the paediatric wards. Wagner and Kanouse [] have pointed out that standard care activities may affect adherence behaviours and
thus intervention outcomes. It is possible that certain components of our improvement
strategies are already part of daily practice in some wards and therefore leave less
room for improvement. Despite the influence of baseline scores and hospital effect,
the team and leaders-directed strategy significantly contributed to an additional
increase in nurses’ HH compliance, both short and long term.
Effects of experiences with the improvement strategies on nurses’ HH compliance
The exploration of the relation between determinants of success and HH compliance
provided empirical evidence for performance feedback, social influence and leadership
as important vehicles for changing HH behaviour. It seems likely that the mixture
of these strategy components affect the teams’ abilities to focus on achieving their
HH improvement goals. Our results have strengthened the theoretical underpinning of
the composition of our team and leaders-directed strategy by using a team approach
for changing individual behaviour. By setting clear norms and targets within the team,
individual team members are invited to support each other in achieving this goal.
The findings of our study also show that it is important to promote a team culture
that empowers team members to speak up when non-adherence is observed. In this finding,
we recognize key elements from the social influence theory [] (e.g., team members address each other in case of undesirable behaviour), and the
theory on team effectiveness [,] (e.g., participation safety and task orientation) (Table&). This is of particular interest because ‘speak up’ is positively correlated with
improved HH behaviour. During the team sessions, we taught the nurses to provide feedback
on the HH behaviour of their colleagues in a correct way. At the same time, we guided
the nurses to receive this feedback positively.
Active commitment and initiative from ward management
The results of our study show that specific components of leadership are positively
correlated with an improvement in nurses’ HH compliance. Thus, ward managers should
address barriers to enable HH as recommended, designate HH as a ward priority, motivate
and encourage team members to perform HH, and hold team members accountable for their
HH behaviour. This finding corresponds with the key elements from theory of leadership
[] as displayed in Table&.
Credits of our findings are not entirely due to the delivery of the team and leaders-directed
strategy. Nurses from the state-of-the-art group were not exposed to social influence
and leadership as a result of improvement activities from our study. A possible explanation
is that these wards, independent of our study activities, have given priority to HH
and were motivated and encouraged by their managers. This explanation is supported
by the results of a further analysis within the group of the state-of-the-art strategy.
We found a significant relation between changes in HH compliance and differences in
nurses’ experiences with social influence and leadership. Compared to the state-of-the-art
group, the analysis within the group of the team and leaders-directed strategy showed
less variation in changes of nurses’ HH compliance. Therefore, an association between
changes in HH compliance and differences in nurses’ perceptions of strategy components
within the team and leaders-directed group could not be demonstrated. We hypothesize
that the lack of variation in this group is due to the consistent implementation of
the team and leaders-directed strategy. As already shown by our evaluation of strategy
adherence, all nurses within the group of the team and leaders-directed strategy were
equally exposed to the main components of this strategy.
Strengths and limitations
The principal strength of our study was the comprehensive process evaluation within
the context of a pragmatic randomised controlled trial. Questions about variations
in the adherence to both HH strategies, and about factors contributing to the relationship
between the HH improvement strategies and nurses’ HH outcomes, would not have been
apparent as a result of only analysing the HH outcome data. Process evaluations are,
in this sense, part of a more theory-based approach to evaluation, responding to the
need to understand which theoretical constructs of an improvement strategy make a
difference []. By linking data of effectiveness to process data, a theoretical explanatory model
can be derived from the process evaluation itself [].
Some researchers encourage the simultaneous application of a process evaluation in
control groups [,]. By doing so, we discovered the impact of specific aspects of social influence and
leadership in the state-of-the-art group that served as a control group. This finding
has strengthened the theoretical underpinning of the composition of our team and leaders-directed
In combining process with outcome evaluations, we collected data using a wide range
of methods as recommended by several authors [,]. We developed a questionnaire, derived from the components of the improvement strategies.
We undertook extensive pilot work to ensure that all important components of the strategies
were adequately captured in questionnaire measures. We then pre-tested the questionnaire
among 90 nursing students.
An important issue concerns the use of ‘as-received’ analysis as distinct from the
conventional ‘intention-to-treat’ analysis used in the analysis of RCTs. These analyses
differ not only in terms of the estimation procedure, but also in terms of the underlying
research goal for a specific study. This study is an example of explanatory research,
and the as-received analysis was therefore appropriate. Our as-received analysis was
illuminating but also lost the benefits of the original random assignment, and therefore
the potential for bias exists. This should be considered when interpreting our results
A limitation of our study concerns the low questionnaire response rate of 48%. This
may be a potential source of bias. We didn’t test the psychometric properties of the
questionnaire. For these reasons, our findings from the nurses’ experiences analysis
need to be interpreted with caution.
Implications
This is the first prospective study that has assessed the working mechanisms of two
HH improvement strategies, demonstrating the added value of specific aspects of social
influence and leadership. This is an important finding for hospital administrators
and ward managers who want to improve nurses’ HH behaviour. Currently, most strategies
focus on the individual and the organization. Including activities aimed at social
influence and leadership could be a promising development. Our results point to: addressing
each other in case of undesirable behaviour, support from colleagues, accountability,
goal setting, and active commitment of the ward manager. The methodology of our team
and leaders-directed strategy can probably be used to improve team performance on
other patient safety issues as well.
Our study points to ways in which the design of process evaluations within randomised
controlled trials may be conducted. Our initial results require affirmation by further
process evaluations of HH improvement strategies. Further research is also needed
to examine the different aspects and impact of social influence and leadership. Finally,
future research should explore the influence of hospital culture.
Conclusion
In summary, with this study we were able to look inside the ‘black box’ of two HH
improvement strategies, to generate insights into which strategy components are effective.
Our results support the added value of social influence and enhanced leadership in
HH improvement strategies, thus offering an interpretation of the trial effects. Our
findings point to: addressing each other in case of undesirable HH behaviour, support
from colleagues, accountability, goal setting, and active commitment of the ward manager.
These results have strengthened the theoretical underpinning of the composition of
our team and leaders-directed strategy. Our study also points to ways in which the
design of process evaluations within randomised controlled trials may be conducted.
Ethical and legal aspects
The Medical Ethics Committee of district Arnhem – Nijmegen assessed the study and
concluded that our study was deemed exempt from their approval, as it did not include
collection of data at the level of patients.
Competing interests
All authors have completed the Unified Competing Interest form at
(available on request from the corresponding author) and declare: no support from
any organization fo no financial relationships with any organizations
that might have an interest in the submitted work in the
other relationships or activities that could appear to have influenced the submitted
Authors’ contributions
T van A, LS, MH and RG were responsible for the research question and the design of
the study. AH and GH designed and operationalized the team and leaders-directed strategy.
AH conducted the study, prepared and coordinated the field observations, and did the
data analysis. AH wrote the first draft of this manuscript and was responsible for
the revisions. T van A, LS, RG, GH, and MH contributed to the drafting of the manuscript.
T van A is the general supervisor of the study. All authors read and approved the
final version of the manuscript.
Source of funding
This study is funded by a research grant from ZonMw, dossier number: .
References
Grimshaw JM,
Thomas RE,
MacLennan G,
Ramsay CR,
Effectiveness and efficiency of guideline dissemination and implementation strategies. Health Technol Assess 2004,
Grimshaw J:
From best evidence to best practice: effective implementation of change in patients'
care. Lancet 2003,
World Health Organization: WHO Guidelines on Hand Hygiene in Health Care First Global Patient Safety Challenge.
Clean Care is Safer C
Glasziou P,
Chalmers I,
Altman DG,
Bastian H,
Boutron I,
Taking healthcare interventions from trial to practice. BMJ 2010,
341:c3852.
Strange V,
Stephenson J:
Process evaluation in randomised controlled trials of complex interventions. BMJ 2006,
332:413-416.
Strange V,
Johnson A,
Stephenson J:
Integrating process with outcome data in a randomised controlled trial of sex education. Evaluation 2006,
12:330-352.
Campbell MC,
Elbourne D,
CONSORT statement: extension to cluster randomised trials. BMJ 2004,
328:702-708.
Mihalic SF,
Argamaso S:
Implementing the LifeSkills Training drug prevention program: factors related to implementation
fidelity. Implement Sci 2008,
Wensing M, In Improving patient care: Effective implementation: A model.
London: Elsevier: Edited by Grol R, Wensing M, Eccles M;
Dusenbury L,
Brannigan R,
Hansen WB:
A review of research on fidelity of implementation: implications for drug abuse prevention
in school settings. Health Educ Res 2003,
18:237-256.
Hulscher ME,
Laurant MG,
Process evaluation on quality improvement interventions. Qual Saf Health Care 2003,
Freeman H,
Lipsey M: Evaluation. A systematic approach., seventh edition edn.
Thousand Oaks, CA: Sage Publications, I
Steckler A, In Process Evaluation in Public Health Interventions and Research: Process evaluation and public health interventions: An overview.
San Francisco: Jossey-Bass Publishers: Edited by Steckler A, Linnan L;
Hargreaves J,
Strange V,
Should structural interventions be evaluated using RCTs? The case of HIV prevention. Soc Sci Med 2006,
Systematic evaluation of implementation fidelity of complex interventions in health
and social care. Implement Sci 2010,
Hugonnet S,
Harbarth S,
Mourouga P,
Touveneau S:
Effectiveness of a hospital-wide programme to improve compliance with hand hygiene. Infection Control Programme. Lancet 2000,
Allegranzi B,
Role of hand hygiene in healthcare-associated infection prevention. J Hosp Infect 2009,
73:305-315.
Mourouga P,
Perneger TV:
Compliance with handwashing in a teaching hospital. Infection Control Program. Ann Intern Med 1999,
130:126-130.
Moralejo D,
Chudleigh JH:
Interventions to improve hand hygiene compliance in patient care. Cochrane Database Syst Rev 2010.,
Naikoba S,
Hayward A:
The effectiveness of interventions aimed at increasing handwashing in healthcare workers
- a systematic review. J Hosp Infect 2001,
47:173-180.
van Achterberg T:
Schoonhoven L, Grol R: Nursing implementation science: how evidence-based nursing
requires evidence-based implementation. J Nurs Scholarsh 2008,
40:302-310.
Hand hygiene: simple and complex. International Journal of Infectious Diseases 2005,
The Lowbury lecture: behaviour in infection control. J Hosp Infect 2004,
Schoonhoven L,
Donders R,
Hulscher M,
Impact of a team and leaders-directed strategy to improve nurses' adherence to hand
hygiene guidelines: A cluster randomised trial. Int J Nurs Stud 2012.
Schoonhoven L,
Hulscher M:
Helping hands: A cluster randomised trial to evaluate the effectiveness of two different
strategies for promoting hand hygiene in hospital nurses. Implement Sci 2011,
Bandura A: Social foundation and thought of action: a social cognitive theory.
New York: Prentice H
Mittman BS,
Jacobson PD:
Implementing clinical practice guidelines: social influence strategies and practitioner
behavior change. Qual Rev Bull 1992,
18:413-422.
Shortell SM,
Marsteller JA,
Pearson ML,
The role of perceived team effectiveness in improving chronic illness care. Med Care 2004,
West MA, In Innovation and creativity at work: Psychological and Organizational Strategies: The social psychology of innovation in groups.
John Wiley and Sons: Edited by West MA&FJl. C
OVretveit J: The Leaders' Role in Quality and Safety I a review of Re-search and G
the "Improving Improvement Action Evaluation Project.
Association of County Councils (Lanstingsforbundet): S
Werkgroep Infectiepreventie: Handhygi?ne medewerkers ziekenhuizen. 2007.
Carroll C,
Patterson M,
A conceptual framework for implementation fidelity. Implement Sci 2007,
Rychetnik L,
Frommer M,
Criteria for evaluating evidence on public health interventions. J Epidemiol Community Health 2002,
56:119-127.
Larson EL,
A multifaceted approach to changing handwashing behavior. Am J Infect Control 1997,
McCormack B,
Rycroft-Malone J,
Titchen A,
Getting evidence into practice: the meaning of 'context'. J Adv Nurs 2002,
38:94-104.
Wagner GJ,
Kanouse DE:
Assessing usual care in clinical trials of adherence interventions for highly active
antiretroviral therapy. J Acquir Immune Defic Syndr 2003,
33:276-277.
Rimer BK, In Health Behavior and Health Education: Theory, Research and Practice: Linking theory, research and practice.
San Francisco: Jossey-Bass: Edited by Glanz K, Lewis FM, Rimer BK;
Salyers MP,
Williams J,
Measurement of fidelity in psychiatric rehabilitation. Ment Health Serv Res 2000,
Pocock SJ,
Abdalla M:
The hope and the hazards of using compliance data in randomized controlled trials. Stat Med 1998,
17:303-317.
Sign up to receive new article alerts from Implementation Science