Tuesday, December 12, 2006

Wrong outcomes, timing, and measures: Social and behavioral sciences reveal flaws in current efforts to evaluate oral health promotion - Dalia Meisha

Oral health promotion seeks to achieve sustainable improvements in oral health and reduce inequalities through targeting the underlying determinants of oral health (Schou and Locker, 1997; Sheiham and Watt, 2003). Critical evaluation of oral health promotion is key for developing effective interventions, disseminating models of good practice, providing feedback to both participants and professionals, ensuring the appropriate use of limited resources and guaranteeing ethical principles are followed (Blinkhorn, 1993). A series of reviews evaluating the effectiveness of oral health promotion activities have been published (Brown, 1994; Schou and Locker, 1994; Kay and Locker, 1996; Sprod, 1996; Kay and Locker, 1998; Department of human services, 1999). All of these agreed that most oral health education and promotion interventions are poorly and inadequately evaluated. It is therefore difficult to determine the effectiveness of many interventions. It is also interesting to note that most investigations had developed their own evaluation measures and rarely used those developed by other researchers and practitioners (Kay and Locker, 1998). A major limitation of the evaluations undertaken was that the outcome measures used were of limited value, focused mostly on clinical or behavioral domains and were not comparable (Watt, 2006). A key recommendation of the effectiveness reviews was the need to improve the overall standard of oral health promotion evaluation, and in particular, to develop a broader range of quality outcome measures (Sprod, 1996; Kay and Locker, 1998). This lack of appropriate outcome measures is hampering the development of oral health promotion.

Failure or insufficient evaluation of oral health promotion outcomes may be attributed to the complex and difficult nature of the evaluation task which has been under-funded and generally neglected in the dental field. Moreover, oral health practitioners have often been given very limited support or training in evaluation methodology (Watt, 2006; Petersen, 2004). Though this may appear not directly related to social and behavioral sciences, having dental professionals exposed to, taught and trained in social and behavioral sciences will ultimately lead to improvements of oral health promotion evaluation through acknowledging the neglected sociobehavioural aspects. The aim of this paper is to criticize current oral health promotion evaluation measures from the social and behavioral sciences perspective.

One aspect why oral health promotion evaluation is falling short is that many oral health promotion activities measure changes in knowledge or knowledge and attitude as their only outcome measures. This is based on a fundamental error in many oral health education activities, which is the assumption that increasing person’s oral health knowledge will change their attitude and undoubtedly result in changing dental care behavior and consequently better oral health. This approach is based solely on the cognitive model assuming the knowledge, attitude, behavior sequence (Gluck and Morgansten, 2003). Study designs based on such weak theoretical model unsurprisingly result in flaws in evaluating their outcomes. As an example of such oral health promotion activities is the recent mass media campaign targeting periodontal disease. Martensson et al evaluated the effect of mass media campaign regarding periodontal disease that included brochures, newspapers, radio and TV. This was achieved by a pre- and post campaign questionnaire assessing knowledge only. It was concluded that mass media might increase knowledge about periodontitis as a health promotion strategy and consequently the campaign was considered successful from a public health standpoint (Martensson, 2006). The problem with this approach of assessing just knowledge change as an outcome measure of oral health promotion activity is the underlying presumption that increasing the level of dental knowledge would result in a behavioral change and thus improving the oral health status. A systematic review of the effectiveness of health promotion conducted by Kay and Locker has found that knowledge levels can almost always be improved by oral health promotion initiatives but whether these shifts in knowledge and attitudes can be causally related to changes in behavior or clinical indices of disease has also not been established (Kay and Locker, 1998). Wimbush et al. study also support the argument that improvement in knowledge is not necessarily concurrent with positive changes in behavior. They presented the results of the evaluation of a national mass media walking campaign in Scotland which involved a 40 television advertisement and a telephone helpline. The campaign impact was assessed in terms of awareness of the campaign and pre- and post-campaign changes in knowledge and beliefs about walking as a good form of exercise, in motivations and intentions regarding walking/exercise and in walking/exercise behavior. The evaluation involved two population surveys and baseline and follow-up surveys of the helpline callers. Awareness levels for the television advertisement peaked at 70% of the adult population during the first 4-week burst of advertising, falling to 54% during the non-broadcast period. The evaluation findings show that, at a population level, the campaign had a notable positive impact on knowledge about walking as a form of exercise but no impact on walking behavior (Wimbush, 1998). Although Wimbush et al. study dealt with walking/exercise behavior rather than oral health related behavior as in Martensson et al study, both evaluated a mass media campaign targeting changing health-related behavior. The distinctive difference between those two studies is that Wimbush study evaluated changes in knowledge, belief, motivation, intention, and behavior, whereas Martensson focused on changes in knowledge. Wimbush et al. study highlights the argument that positive changes in knowledge are not always predictive of positive changes in behavior as it is the assumption in many oral health education activities.
The knowledge- behavior gaps have been addressed by many studies to explore the reasons behind this gap. Rimal has analyzed 3 cross-sectional data and 2 longitudinal data from the Stanford Five-City Project to determine whether diet self-efficacy mediated the relation between diet knowledge and behavior. In the cross-sectional data, knowledge-behavior correlations were greater among those with high self-efficacy. In the longitudinal data, knowledge-behavior correlations increased among those who increased their self-efficacy and decreased among those who decreased their self-efficacy (Rimal, 2000). The Rimal study suggests self-efficacy as one explanation for the knowledge- behavior gap.

People may know what approach they should take to improve their health or other living conditions, but may not act on that knowledge. Possible reasons for that are the concepts of self efficacy, social norms, and social support. Bandura introduced the theory of self efficacy in which ones conviction that one can successfully execute the behavior is required to produce the outcomes. This perceived self efficacy influences all aspects of behavior, including the acquisition of new behaviors and the inhibition of existing behaviors (Bandura, 1982). Social norms theory states that much of people’s behavior is influenced by their perception of how other members of their social group behave. Social norm theory predicts that correcting misperceptions of the norm is likely to result in decreased problem or high risk behavior and an increase in healthy and responsible behaviors (Perkins, 1998). Social support from peers, family, society also is another factor that affects adoption of new healthier behaviors.

Another aspect to criticize in many oral health promotion evaluations is the inappropriateness to measure outcomes in the short-term only. Most oral health educational programs have been using short-term timescales to assess changes in behavior or its downstream consequences; therefore the long-term behavior are still questionable, as initial adoption of the behavior does not necessarily predict sustained behavior over the long-term. Biesbrock et al. examined the impact of an educational oral health program conducted within a Boys and Girls Club of America in Chicago, Illinois. The educational program focused on the gingival health (gingivitis and plaque) of the 6-10 years of age participating children. The multi-week program taught the participants the basics of oral biology and disease, as well as proper oral health prevention, including oral hygiene, dietary modification, and the importance of visiting the dentist. A calibrated examiner measured whole mouth gingival health and plaque at baseline (immediately prior to the initiation of the educational program) and four weeks later. In addition, subjects completed five questions at baseline and at four weeks to assess their oral health knowledge before and after exposure. There was a 24% reduction in Gingival Index score, 20% reduction in bleeding sites, and 3% reduction in Plaque Index score. The subject population was found to have statistically significantly greater knowledge following the program at week four, with 37% and 69% of subjects answering the questions correctly at baseline and week four, respectively. Collectively, they concluded that these data support the role of an educational program in promoting improved oral health in these children over a four-week period (Biesbrock, 2004). It is argued that measuring changes in gingival index and bleeding index over a period of four-weeks only is not adequate measure and might only reflect transient changes. A recent systematic review confirms this notion that it is possible to achieve short-term reductions in plaque and gingival bleeding by means of health education interventions. The importance of these reductions is questionable, however, from the point of view of both individual and public health. It is also unclear how well the beneficial changes are sustained in the long run: studies with long follow-up periods are needed to find out the long-term effects (Hausen, 2005). It is also argued that assessment of changes in knowledge in a 4-week timeframe is not a reliable measure, as it might be only short-lived change and that retention of knowledge is more important than short-term changes in knowledge. This is supported by the conclusion of a review of research in dental health education and health promotion, as it has stated that dental health education achieves only short-term gains in knowledge (Brown, 1994). It is suggested that 3-months be the least time interval for evaluating such oral health activity promoting plaque removal and knowledge change. The transtheoretical model developed by Proshaska and DiClemente constructs behavior change as a process involving progress through a series of five stages: precontemplation, contemplation, preparation, action, and maintenance (Prochaska & DiClemente, 1983). The recognition of the maintenance stage in this model supports the arguments that people can easily relapse and that initiation of behavioral change is not necessarily an indictor of a sustained change. This calls for long-term follow-up to assess behavioral changes after oral health promotion activities.

Although a significant number of evaluation outcome measures exist, their quality is highly variable with very few measures used in the assessment of policy and environmental change. A recent review of oral health promotion effectiveness have identified and assessed the quality of available outcome measures. The quality of oral health outcome evaluation measures were assessed in terms of content validity, applicability, efficiency, clarity, and sensitivity. A high proportion of the identified measures were classified as health literacy and healthy lifestyle outcomes, appropriate for the evaluation of oral health education activities. Only 1% of measures identified were classified in the healthy public policy category. When reviewed against the quality criteria, only 49% of the measures were considered satisfactory and it concluded that very few outcome measures exist for use in the evaluation of oral health policy and environmental interventions (Watt, 2006). Examples of public policies aiming to improve oral health are fluoridating water and monitoring that on regular basis, mandating dental examination before entrance to school and on yearly basis while in school, and mandating children in contact sports to wear mouth-guards (Allukian, 1990). Shortage of measures for assessing policy and environmental changes of oral health promotion interventions are noteworthy, especially that social and behavioral sciences support the notion that health promotion through targeting changes in policy or in environment have a profound effect on behavior. Many examples of the success of policy changes in achieving public health improvements have been documented as eliminating smoking in restaurants, increasing cigarette tax, and adopting mandatory seat belt law (Siegel, 2004). Without thoroughly evaluating such changes, the effectiveness of interventions targeting policy and environmental changes for better oral health will be overlooked.

It is essential that oral health promotion interventions are fully scrutinized to determine their impact and value. Evaluation of oral health promotion is a complex task and a key element is the use of appropriate study designs, theoretical framework, and outcome measures (Watt, 2001). Proper health evaluation measures are essential as means of developing good practice, to make best use of limited resources, to provide feedback to staff and participants and to inform policy development (Blinkhorn, 1993).Considerations of social and behavioral science concepts are one of the approaches to inform a better evaluation of oral health promotion activities.


Allukian M. Oral health policy issues for women and children. J Public Health Dentistry 1990; 50 (6):406-417.
Bandura A. Self efficacy mechanisms in human agency. American Psychology 1982; 37: 122-147.
Biesbrock AR, Walters PA, Bartizek RD. Short-term impact of a national dental education program on children's oral health and knowledge. J Clin Dent. 2004;15(4):93-7.
Blinkhorn A. Evaluation and planning of oral health promotion programmes. In: Schou L, Blinkhorn A, editors. Oral health promotion. Oxford: Oxford University Press; 1993. p. 249–70.
Brown L. Research in dental health education and health promotion: a review of the literature. Health Educ Q 1994;21: 83–102.
Department of Human Services. Promoting oral health 2000–2004: strategic directions and framework for action. Melbourne: Health Development Section; 1999.
Gluck MG, Morganstein WM. Jong’s Community Dental Health. Mosby; 2003.
Hausen H. Oral health promotion reduces plaque and gingival bleeding in the short term. Evid Based Dent 2005; 6: 31.
Kay L, Locker D. A systematic review of the effectiveness of health promotion aimed at promoting oral health. London: Health Education Authority; 1998.
Kay L, Locker D. Is dental health education effective? A systematic review of current evidence. Community Dent Oral Epidemiol 1996;24: 231–5.
Martensson C, Soderfeldt B, Halling A, Renvert S. Knowledge on periodontal disease before and after a mass media campaign. Swed Dent J. 2004;28(4):165-71.
Prochaska, J. O., & DiClemente, C. C. Stages and processes of self-change of smoking: Toward an integrative model of change. Journal of Consulting and Clinical Psychology 1983; 51, 390-395.
Rimal RN. Closing the knowledge-behavior gap in health promotion: the mediating role of self-efficacy. Health Commun. 2000;12(3):219-37.
Schou L, Locker D. Oral health: a review of the effectiveness of health education and health promotion. Amsterdam: Dutch Centre for Health Promotion and Health Education; 1994.
Schou L, Locker D. Principles of oral health promotion. In: Pine C, editor. Community oral health. Oxford: Wright; 1997. p. 177–87.
Sheiham A, Watt R. Oral health promotion and policy. In: Murray J, Nunn J, Steele J, editors. Prevention of oral disease. Oxford: Oxford University Press; 2003. p. 241–58.
Siegel M, Doner L. Marketing public health. Strategies to promote social change. Jones and Barlett Publishers; 2004. p. 129-130.
Sprod A, Anderson R, Treasure, E. Effective oral health promotion. Literature Review. Cardiff: Health Promotion Wales; 1996.
Watt RG, Harnett R, Daly B, Fuller SS, Kay E, Morgan A, Munday P, Nowjack-Raymer R, Treasure ET. Evaluating oral health promotion: need for quality outcome measures. Community Dent Oral Epidemiol. 2006 Feb;34(1):11-7.
Watt RG, Fuller S, Harnett R, Treasure ET, Stillman-Lowe C. Oral health promotion evaluation- time for development. Community Dent Oral Epidemiol. 2001;29:161-6.
Wimbush E, MacGregor A and Fraser E. Impacts of a national mass media campaign on walking in Scotland. Health Promotion International, 1998;13: 45-53.


Post a Comment

<< Home