Home > Grants Research DTCP
PRINTER FRIENDLY REPORT
Promoting Organ Donation through Public Education Campaigns: A Random-Effects Meta-Analysis
Thomas Hugh Feeley, Ph.D.
Shin-Il Moon, M.A.
Running Head: Public education campaigns in organ donation
Acknowledgement: This project was supported by a contract from the Division of Transplantation, Health Resources Services Administration, Health and Human Services (DoT/HRSA/HHS). The arguments and data presented are those of the authors and in no way represent the position of DoT/HRSA/HHS. Address all correspondence electronically to Thomas Feeley, firstname.lastname@example.org.
Objective: To estimate the combined effect of public education campaigns on individuals' attitudes and behaviors toward organ donation. Also considered are characteristics of studies that may influence the effect of public education campaigns on study outcomes.
Data Sources: Twenty-three comparisons from 16 reports are included in analyses. These comparisons were taken from published studies, convention papers and unpublished grant reports.
Study Selection: Public education campaigns designed to shape or reinforce pro-donation attitudes or behaviors in target sample were targeted. Study inclusion required studies to report appropriate statistics to allow comparison of campaign effects to baseline or control group.
Data Extraction: Effect sizes were converted to a common metric (correlation or r) and weighted by sample size and variance around mean. Outcome measures of effect included signing rates, discussion about donation, and attitudes toward donation; also examined was an average effect across measures.
Data Synthesis: Random-effects meta-analysis was conducted and weighted mean effect sizes, 95% confidence intervals and heterogeneity of variances were computed.
Conclusions: The overall effect of public education campaigns in promoting organ donation was a 5% gain in study outcomes. Several moderators including target group (general population vs. minority group), amount of campaign exposure, type of intervention, and publication bias were evaluated. Although there is overlap in 95% confidence intervals, campaigns that include an interpersonal communication outreach component achieve larger effect sizes in study outcomes.
Keywords: organ donation, meta-analysis, public health, education, media, persuasion.
One promising strategy to decrease the gap between the organ donation need and supply is to increase the consent and conversion rates for transplantable organs. Family consent rates tend to range between 54% and 62% nationally.1,2 Stated differently, if 100 individuals are approached by an OPO professional for consent, approximately 40 of these individuals decline. Consider that one eligible donor may donate up to 8 organs. A major obstacle often cited for the low consent rates is the lack of information or knowledge individuals possess about donation.3,4 Individuals may know very little about donation or worse yet, what little they know is inaccurate or based on fictional depictions of the transplantation process shown on prime-time television.5 Data clearly indicate the majority of Americans report positive attitudes toward organ donation6,7 and view donation as an altruistic and positive act.8
A commonly enacted method to educate or enlighten a target audience is to mass communicate the potential gains of compliance to a health behavior (e.g., applying sunscreen) or to communicate the potential risks of non-compliance (e.g., hazards of smoking). A recent meta-analysis by Snyder and colleagues9 found commencement campaigns (e.g., joining the state organ donation registry) to be more successful in changing behavior than cessation campaigns (e.g., quitting smoking). The authors found an overall campaign effect of r = .09 suggesting public health campaigns produce a 9% effect on study outcomes. It is interesting to note that effect sizes varied by campaign topic.
The goal of the current study is to summarize the population of studies that use campaigns to promote and to educate the population about organ donation. The current review benefits by using meta-analytic procedures10,11 to provide a quantitative estimate of the aggregate effect campaigns may have on desired persuasive outcomes (e.g., communicating one's intentions regarding organ donation) related to organ donation. The effect size provided through meta-analysis is of value to scholars, practitioners and policy-makers in the area of organ donation for two important reasons. First, knowing the aggregate effect a given type of media campaign may have on its intended audience would allow for a cost-benefit analysis when planning or designing public campaigns to increase donation or consent rates. The costs of a campaign may be financial and/or in terms of human capital expended. Second, modern meta-analytic procedures12 allow for analysis of moderators or factors that serve to increase or decrease the effect of a given intervention. Thus, one desired outcome of the current study is to understand factors that may increase the effects of media campaigns in the promotion of donation. Inclusion of study moderators in meta-analysis also allows for testing theoretical explanations of the relationship between factors and the potential to prescribe how one should conduct a public education campaign to maximize its impact on target audience.
It is predicted that greater exposure to campaign messages should moderate the size of the effect for mass media campaigns. If the proposition that increased education and knowledge about donation will increase commitment to donation then it stands to reason increased exposure to campaign messages (i.e., more education) should lead to greater persuasion. Attention to messages is a consistent factor cited in both social learning models13 and media campaign models14 as a necessary condition for success.
A second moderator important to consider is target population of media campaigns - the data clearly indicate minorities are less knowledgeable about donation, less committed to donation, less trustful of donation, and consent to donation at proportionately lower rates than do Caucasians.1,4,15 It is important to examine if campaigns specifically targeting minorities yield similar effect sizes compared to campaigns for the general population. One plausible argument predicts minority campaigns should be more successful as campaign designers may be more inclined to tailor the message to the specific minority audience. For example, one study16 focused its mass media efforts on television and radio stations recognized as important sources of health information for Hispanic Americans. Also, campaigns targeting minorities may have a wider audience to reach (i.e., fewer minorities are committed to donation) and thus more room for improvement.7
Many media campaigns in pro-social health domains are often accompanied by grass roots or interpersonal communication messages of influence. For example, a media launch might be complemented by a press release and several functions that serve to spread the message by word-of-mouth. Also common are educational fairs or functions that complement mass media campaigns with the hope the mediated message will be reinforced by these interpersonal communication efforts. Also, these efforts often provide a convenient method to secure commitment to a pro-social cause. In the case of organ donation, registry cards are made available and brochures disseminated that educate attendees about donation.16 It is expected that campaigns that use interpersonal communication efforts to bolster media campaigns will be more effective than media would alone.
Identification of Relevant Studies
In an effort to identify studies that test the effects of public education campaigns to promote organ donation, four methods were employed. First, a literature review was undertaken using several search engines including MEDLINE, PsychInfo, CIOS, SSCI and scholar.google.com. These reviews were conducted in November 2006 and again in August of 2007. The following search terms were used alone and in combination: "organ," "donation," "tissue," "campaign," "media," "intervention," "transplant," "mass media," "communication" and "interpersonal." Second, an ancestry search was conducted by examining the reference section of articles determined to be relevant to the study goal(s). Third, several prominent authors in the area of organ donation were contacted via telephone or email and surveyed about studies, perhaps unpublished, in media campaigns and organ donation. Fourth, all principal investigators of grants18 funded by Health Resources Services Administration, Division of Transplantation (HRSA/DoT) whose grant titles suggested the potential for public education campaigns were contacted. Over 50 emails were sent to investigators and authors in attempt to retrieve relevant studies. Retrieval efforts identified 23 campaigns from 16 studies.
Criteria for Study Inclusion
A study was included in the current analysis if it met the following criteria. First, the study or project was specifically designed to influence multiple individuals using a single or many communication channels. Second, a study must compare the intervention or influence message to a proper control group. The control group was not exposed to the messages and may include a baseline comparison (i.e., pre-post design), comparison to placebo group, or use a combined pre-post, control group design. Third, studies must include appropriate statistical information to allow for meta-analytic procedures (e.g., proportions, effect size, sample size, standard deviations around mean estimates). Studies that fail to report effect sizes (e.g., r, F value, Cohen's d, etc.) but presented descriptive statistics (e.g., means, proportions, standard error) were included and the effect size(s) were computed by the software. Fourth, studies must include one or more measures of effects related to organ donation, such as attitudes toward donation, intentions to donate, consent rates, family discussion rates, registry signing rates. Many studies included more than one such outcome measure and both an overall or average effect12 was computed and effects for individual outcome measures.
Studies that ran separate campaigns were treated independently when data were reported for each campaign. For example, Alvaro et al.16 conducted campaigns in two separate counties in Arizona and Feeley et al.19 conducted campaigns on two separate campuses in New York. For studies19,20 that report successive campaigns and evaluation and report independent groups evaluation design, data were averaged across the 2 or 3 years reported. Also, campaigns that ran different treatment conditions (e.g., Quinn et al.21) were treated as independent studies.
Each article was coded along several criteria and a second coder independently coded each article to insure reliability. All articles were coded and 100% agreement was achieved. First, articles were coded by type of campaign and one of three codes was assigned: media only, media & interpersonal, or interpersonal only. A media-only campaign (e.g., Frates et al.20) featured only use of traditional media (e.g., billboards, radio, TV) to promote donation. Media & interpersonal (e.g., Feeley et al.19) used both traditional media and face-to-face promotion methods such as tabling, presentations, organized functions, and luncheons with talks, to name four. Interpersonal only (e.g., Quinn et al.21) was included for comparison to media and these studies did not use traditional mass media to promote donation. Target audience was coded and articles were assigned one of the following codes: general population audience, general minority, African-American, Hispanic, or Asian.
Exposure to media campaigns was determined by respondents' self-reported exposure to campaign message. If average amount of exposure was less than 25% (e.g., Sanner et al.22) exposure was coded as low, 25–50% exposure is medium (e.g., UNYTS23), and if exposure was greater than 50%, exposure was high (e.g., Frates et al.20). In studies where exposure was not measured, exposure was labeled as unknown (e.g., Intermountain Donor Services24). Sanner et al.22 was conducted in Sweden and all other studies were conducted in the United States. To measure for potential publication bias10, studies were coded as published/in press or unpublished. Codes for each study are reported Table 1 along with sample size and overall effect size for each study across study outcomes.
The common effect size was either signing registry or family notification about one's intentions regarding donation. Also, attitudes toward donation were examined and the following outcomes were considered attitude measures: beliefs about donation, willingness to donate, and attitudes. One study24 measured actual consent rates for organs and/or tissue pre- and post-campaign. Separate analyses were conducted for each outcome measure and an aggregate analysis25–27 was conducted using the average of all study outcome measures for each campaign with more than one outcome measure.
Random-effects meta-analysis was conducted to aggregate study findings. Random-effects, compared to fixed-effects analysis,28 assumes the true effect of an intervention may vary from study to study and the studies included for analysis represent a random sample of observed effects from a larger population of samples. The practical difference is how each study is weighted during analysis. With fixed-effects analysis, each study is proportionately weighted by sample size; thus, a large study24,29 would be weighted proportionately higher while a small study30 could be almost ignored or trivialized. With random-effects analysis, the weights, while still proportional to sample size, are more balanced. Comprehensive Meta-Analysis software 2.031 was used for data analysis. CMA computes several effect sizes and the current study reports the effect size r throughout for ease of interpretation. Target audience, publication bias, exposure to campaign message, type of campaign and study design were all examined as potential moderators of effect size. Analyses weight effect sizes by the inverse of the variance of each study where the variance is a function of sample size and variance of effect estimate. Thus a study with a larger sample size24 and small confidence interval around the outcome measure is weighted more heavily than a study with a small sample.25
Overall Effect of Education Campaigns
Across the 23 campaigns (k = 23, N = 6,647,801) and 16 study reports, there was an overall significant effect for media campaigns in promoting donation. The average r = .05 (95% CI = .03, .07) and this represents an approximate 5% increase in campaign effects over control group using the average of all study outcomes. The Q statistic, which measures amount of heterogeneity in effect sizes, was statistically significant, Q (22) = 69.99, p < .001 indicating the variability in effect sizes is different than would be expected by chance. A second interpretation of the Q statistic suggests there are variables or moderators that may explain or account for the differing campaign effects across studies.
The effect size for campaigns on the outcome factor of signing the registry was r = .05 (.03, .07), k = 19, p < .001, Q (18) = 1834.08, p <.001 and r = .05 (.03, .08), k = 15, p < .001, Q (14) = 21.85, p = .08 for family discussion about donation. Campaigns to increase attitudes toward donation were not significantly greater than zero, r = .04 (-.00, .07), k = 11, p = .07, Q (10) = 45.99, p < .001.
Of the 23 campaigns, 11 reported data on respondents' campaign exposure with 6 studies reporting moderate to high message exposure, 5 with low exposure, and 12 studies that failed to report dosage. So this campaign received 43,090 views (at least 30 seconds). The campaign received a total of 48,475 clicks, as reported in your AdWords account. So you received 48,475 - 43,090 = 5,385 clicks to your website. Again, a lot of the calculations pull numbers from a few different places. Let me know if that number greatly exceeds your expectations. Table 1 reports codes for campaign exposure for each campaign. Thus, due to paucity of studies that measure dosage or campaign exposure, caution should be warranted when interpreting effect sizes. Studies reporting low exposure (fewer than 25% reporting exposure) have an insignificant campaign effect, r = .02 (-.04, .08), ns. Studies with moderate or high exposure yield significant aggregate effect size for campaigns, r = .07 (.04, .10), p < .001. For the 12 studies that fail to report campaign exposure, the overall effect was statistically significant, r = .05 (.02, .08), p < .001.
As expected, published studies yield greater effect sizes for campaigns, r = .07 (.04, .10), k = 9, p < .001, but the difference between published studies and unpublished studies was not statistically different at .05 alpha, as the confidence intervals overlap. Unpublished studies also indicate a significant effect for campaigns, r = .04 (.02, .07), k = 14, p < .01.
Nine studies report effects for campaigns specifically designed to impact minorities (see Table 1 for codings) and the aggregate or overall finding is significant, r = .07 (.05, .09), p < .001. Campaigns whose target population is the general population also yielded a significant overall average effect size estimate, r = .05 (.02, .07), k = 14, p < .01. The effect size for campaigns specifically targeting minorities was greater than the average effect size for the general population; however the difference between effect sizes was not statistically significant.
Type of Campaign
Campaigns that use a media-only message yielded a 4% increase in effect, r = .04 (.01, .06), k = 12, p < .01. Campaigns that used media + interpersonal messages had a 7% effect, r = .07 (.02, .12), k = 7, p < .01. Four studies used interpersonal-only messages to study campaign effects with a 9% effect, r = .09 (.03, .14), p < .01. Thus, it appears overall estimates of campaign effects increase as the amount of involvement or utilization of interpersonal message forms. However, due to possible the small number samples and/or heterogeneity in study effects, the differences in campaigns were not reliably (or statistically) different than one another.
The current study presents a meta-analytic analysis of studies testing the effects of public education campaigns on promotion of organ donation. The benefits of such an analysis are many. First, the current study provides a more reliable estimate of the overall media campaigns than any one study or even group of studies could provide. Thus, one planning an intervention might assume the "baseline" or expected campaign yield to be a 5% in registry signing rates or family notification rates and might attempt to improve upon these estimates.
More specific to a potential grant proposal in organ donation, a necessary input to computing power is the predicted effect size expected from a campaign.32,33 The current study suggests the generic input to this equation is .05 or 5%. It is important to note that the 5% gain reported in this paper is across differential study outcomes including signing, family notification, willingness to donate, beliefs about donation and so on. Thus, a proposed campaign may seek to improve upon the "typical" campaign effect and propose a 10% (or greater) effect in campaign outcomes. Modern meta-analytical techniques allow the investigator to combine and average across multiple effect sizes when a given study includes several outcome measures, such as registry signing, family notification, and/or willingness to donate.
Several moderators were tested and show effect sizes may be strengthened by the inclusion of several components into a campaign. It should be noted that the effect size differences reported did not reach statistical significance. For example, five studies that found audience members report low exposure to campaign messages indicate an overall effect of 2% while six studies that report moderate to high exposure indicate a 7% effect size. This difference was not statistically different but may be practically important for an investigator or media campaign designer. First, dosage did appear to matter and this is certainly what social scientists would predict. After all, if messages of influence matter at all it stands to reason they have a greater effect with greater exposure and/or multiple exposures to the message. Two campaigns16 reported an average effect size of r = .08 when comparing one's exposure to campaign messages to one's beliefs about donation.
Second, it is encouraging to note that campaigns targeting minorities (r = .07) were successful and appear to be somewhat more successful than campaigns targeting the general population (r = .05). Two interpretations are advanced for these results. One interpretation is that the greater effects are due to greater ceiling effects in obtaining gains with the general population. That is, minorities have lower signing and family notification rates and thus percentage gains are easier to accrue than with the general population where rates are higher. An alternative explanation is campaigns targeting a specific minority group, such as African-Americans34 or Hispanics13,20 may have designed messages better tailored to educate and influence individuals, perhaps through field testing or using focus groups.
The findings regarding type of campaign are perhaps the most intriguing results identified in the current meta-analysis. Simply stated, campaigns that included an interpersonal message component were more successful than campaigns that relied exclusively on the mass media.
These results would reinforce Katz and Lazarsfeld's35 two-step flow model of social influence for media campaigns in the context of promotion of organ donation. The two-step flow model predicts that mediated messages are more likely to influence what the authors call opinion leaders who, in turn, influence the general population. It is thought that opinion leaders are more likely to expose themselves to organ donation messages, more likely to internalize or cognitively process organ donation messages, and subsequently may communicate their own positive attitudes about donation to salient others. Feeley et al.19 found students who signed organ donation cards at University at Buffalo and University at Albany to report interpersonal communication with students in the campaign courses to be the primary motivation for their signing decision.
The preceding discussion prompts the question: How would one strategically take advantage of the two-step model to promote donation? One suggested solution is counterintuitive to most prescriptions regarding mass media campaigns. Campaigns should target the signed or those already committed to the topic of organ donation or those committed to pro-social health causes, such as blood donation or preventive medicine. Specifically, a campaign might target individuals who are known opinion leaders or zealots for donation who will be significantly more likely to tune into the message and discuss and spread the message through interpersonal communication. Personalized messages about donation are more likely, after all, to move individuals to action. For good reason, an organ procurement professional does not seek family consent over the phone, s/he goes to the hospital and approaches the family face-to-face. Thus, the trade-off is obvious: mass media has the potential to reach a wider and more difficult-to-reach audience with a small effect; however interpersonal messages, although more persuasive, only reach one person or group at a time.
The cumulative effect of public education campaigns in the promotion of donation is a 5% increase in outcomes compared to baseline or a control group. Attitudes toward donation, state registry signing rates, and family notification or discussion were the common outcome measures employed to test the campaign efforts. Future research efforts that allow more sophisticated analysis of campaign exposure in relation to campaign outcomes would go far in advancing the potential of public education campaigns in their goal to increase the national consent rates for solid organs.
Articles marked with an asterisk were included in meta-analysis