Influenza vaccination: policy versus evidence

suchandra · October 30, 2006

Shocking result by the BMJ, vaccination against influenza has no use!

British Medical Journal Analysis and comment

Public health

Influenza vaccination: policy versus evidence

Tom Jefferson, coordinator1

1 Cochrane Vaccines Field, Anguillara Sabazia, Roma 00061, Italy jefferson.tom@gmail.com........ type=text/javascript>.........>

Each year enormous effort goes into producing influenza vaccines for that specific year and delivering them to appropriate sections of the population. Is this effort justified?

Viral infections of the respiratory tract impose a high burden on society. In the last half of the 20th century, efforts to prevent or minimise their impact centred on the use of influenza vaccines. Each year enormous effort goes into producing that year's vaccine and delivering it to appropriate sections of the population. Here, I will discuss policies on the use of inactivated vaccines for seasonal influenza; the evidence for their efficacy, effectiveness, and safety ("effects"); and possible reasons for the gap between policy and evidence.

Policies

Every vaccination campaign has stated aims against which its effects must be measured. The US Advisory Committee on Immunisation Practices produces a regularly updated rationale for vaccination against influenza.1 The current version identifies 11 categories of patients at high risk of complications from influenza (box).

The rationale rests on the heavy burden that influenza imposes on the population and the benefits of vaccination. For example, reductions in cases, admissions to hospital, mortality of elderly people in families with children, contacts with healthcare professionals, antibiotic prescriptions, and absenteeism for children and household contacts are the main arguments for extending vaccination to healthy children aged 6-23 months in the United States.2 Canada introduced a similar policy in 2004.3 Less comprehensive policies recommending vaccination for all people aged 60 or 65 and over are in place in 40 of 51 developed or rapidly developing countries.4 On the basis of single studies, the World Health Organization estimates that "vaccination of the elderly reduces the risk of serious complications or of death by 70-85%."5 Given the global nature of these recommendations, what type of evidence should we expect to support them and what does available evidence tell us?4

Which evidence?

When considering the best evidence for vaccination we must take into account the unique epidemiological features of influenza viruses and the rationale for immunisation. The incidence and circulation of seasonal influenza and other respiratory viruses vary greatly each year, each season, and even in each setting. A systematic review of the incidence of influenza in people up to 19 years' old reported a seasonal variability of 0-46%; during a five year period the average incidence was 4.6% in this age group. During a period of 25 years the incidence was 9.5% in children under 5.6 Because of this variability and lack of carryover protection from one year's vaccine to the next,7 especially if the virus changes its antigenic configuration, single studies reporting data from one or two seasons are difficult to interpret. Single studies are also not reliable sources for generalising and forecasting the effects of vaccines, especially when numbers are small. They introduce further instability into already problematic forecasting. Additional limitations to our forecasting ability are imposed by our use (and misuse) of studies assessing the effects of influenza vaccines. Although the effect assessed depends on the aims of the particular campaign, most concentrate on serious effects (such as pneumonia or death) and person to person transmission (table 1). Field efficacy studies are only relevant when viral circulation is high, but no one can forecast with precision the impact on next year's influenza.

Studies of the effects on influenza-like illness and its complications most closely replicate real life conditions because no one knows what agent (if any) causes this disease. Influenza-like illness is an acute respiratory disease caused by many different viruses (including influenza A and B), which presents with symptoms and signs that cannot be distinguished from those of influenza. Influenza-like illness does not have documented laboratory isolation of the causative agent and is the syndrome that most commonly presents to doctors ("the flu").

In general the most powerful and reliable studies are those that "average" out several years and perform subanalyses by setting, population, viral circulation, and viral-vaccine antigenic match—variables that affect interpretation of the effects of a vaccine. Systematic reviews are the best way to perform such analyses, and provide powerful evidence weighted by the methodological quality of the studies involved. Large datasets containing several decades of observations help us to assess the performance of vaccines more accurately.

<CENTER><TABLE cellSpacing=0 cellPadding=0 width="95%" border=1><TBODY><TR bgColor=#e1e1e1><TD><TABLE cellSpacing=2 cellPadding=2 width="100%"><TBODY><TR bgColor=#e1e1e1><TD vAlign=top align=left bgColor=#ffffff>People for whom vaccination is recommended in the United States1

People aged 65 or more

Patients in institutions who have chronic medical conditions

Adults and children with chronic disorders of the cardiovascular and respiratory systems (including asthma but excluding hypertension)

Adults and children who have been treated in hospital in the preceding 12 months for a range of conditions (for example, diabetes or haemoglobinopathy) Adults and children with conditions that compromise respiratory function or handling of infected secretions Children aged 6 months to 18 years being treated with aspirin

Women who are pregnant during the influenza "season"

Children aged 6-59 months

Adults aged 50-64 years

Carers and household contacts (including children) of those in the above risk categories and of children aged 0-59 months

Healthcare workers

</TD></TR></TBODY></TABLE></TD></TR></TBODY></TABLE></CENTER>

The evidence

I searched for relevant systematic reviews when updating and expanding the Clinical Evidence chapter on influenza (see bmj.com)—evidence was plentiful. The examples in table 2 show the strength of the evidence and the contradictions in relation to the stated aims of the vaccination campaign. Whenever possible, I chose evidence gathered in the optimal circumstances (for inactivated vaccines)—high viral circulation and a good match between the viral antigen and the vaccine.

<CENTER><TABLE cellSpacing=0 cellPadding=0 width="95%"><TBODY><TR bgColor=#e1e1e1><TD><TABLE cellSpacing=2 cellPadding=2><TBODY><TR bgColor=#e1e1e1><TD vAlign=top align=middle bgColor=#ffffff>View this table:

[in this window]

[in a new window]

</TD><TD vAlign=top align=left>Table 2 Examples of evidence from systematic reviews comparing inactivated influenza vaccines with placebo or no intervention

</TD></TR></TBODY></TABLE></TD></TR></TBODY></TABLE></CENTER>

Three problems are immediately apparent. The first is heavy reliance on non-randomised studies (chiefly cohort studies), especially in the elderly. This makes assessment of methodological quality an important part of data interpretation. For example, of 40 datasets assessing the effects of influenza vaccines in elderly people in institutions, only 26 reported data on viral types in circulation and only 21 gave information on vaccine content. Insufficient data were available in 11 of 17 retrospective studies of elderly people in institutions to allow reviewers to assess the authors' claim of "high" or "epidemic" viral circulation.11 14 A metaanalysis of inactivated vaccines in elderly people showed a gradient from no effect against influenza or influenza-like illness to a large effect (up to 60%) in preventing all-cause mortality. These findings are both counterintuitive and implausible, as other causes of death are far more prevalent in elderly people even in the winter months.15 16 It is impossible for a vaccine that does not prevent influenza to prevent its complications, including admission to hospital.

A more likely explanation for such a finding is selection bias, where one half of the study population (hemi-cohort) systematically differs from the other in one or more key characteristics.14-16 In this case, the vaccinated hemi-cohort may have been more mobile, healthy, and wealthy than the control hemi-cohort, thus explaining the differences in all-cause mortality.11 14 The same effect is seen in stronger study designs (such as cluster randomised trials) that are badly executed, which introduces bias.10 Its presence seems to be a marker of confounders that persist even after adjusting for known ones, and it makes accurate interpretation of the data difficult. Caution in interpretation should thus be the rule, not the exception. This problem (in the opposite direction—with frailer people more likely to be vaccinated) has been identified before but not heeded.17 The only way that all known and unknown confounders can be adequately controlled for is by randomisation.

The influence of poor study quality is also seen in the outcome of a review of evidence supporting the vaccination of all children to minimise transmission to family contacts.18 Five randomised studies and five non-randomised studies were reviewed, but although data were suggestive of protection, its extent was impossible to measure because of the weak methods used in the primary studies.18

The second problem is either the absence of evidence or the absence of convincing evidence on most of the effects at the centre of campaign objectives (table 2). In children under 2 years inactivated vaccines had the same field efficacy as placebo,8 and in healthy people under 65 vaccination did not affect hospital stay, time off work, or death from influenza and its complications.9 Reviews found no evidence of an effect in patients with asthma or cystic fibrosis, but inactivated vaccines reduced the incidence of exacerbations after three to four weeks by 39% in those with chronic obstructive pulmonary disease.12 13 19 All reviewers reported small data sets (such as 180 people with chronic obstructive pulmonary disease13), which may explain the lack of demonstrable effect.

The third problem is the small and heterogeneous dataset on the safety of inactivated vaccines, which is surprising given their longstanding and widespread use. A Cochrane Database Systematic Review found only one old trial with data from 35 participants aged 12-28 months.8 In the general population of elderly people, despite a dataset of several million observations, safety was only reported in five randomised controlled trials (2963 observations in total) on local and systemic adverse events seen within a week of giving parenteral inactivated vaccine.11 Although there appears to be no evidence that annual revaccination is harmful, such a lack of knowledge is surprising.

Gap between policy and evidence

The large gap between policy and what the data tell us (when rigorously assembled and evaluated) is surprising. The reasons for this situation are not clear and may be complex. The starting point is the potential confusion between influenza and influenza-like illness, when any case of illness resembling influenza is seen as real influenza, especially during peak periods of activity. Some surveillance systems report cases of influenza-like illness as influenza without further explanation. This confusion leads to a gross overestimation of the impact of influenza, unrealistic expectations of the performance of vaccines, and spurious certainty of our ability to predict viral circulation and impact. The consequences are seen in the impractical advice given by public bodies on thresholds of the incidence of influenza-like illness at which influenza specific interventions (antivirals) should be used.20

The confusion between influenza and influenza-like illness is compounded by the lack of accurate and fast surveillance systems that can tell what viruses are circulating in a setting or community within a short time frame, and after the "season" is finished give an accurate picture of what went on to enable better forecasting of future trends.21 Accurate surveillance must be based on a properly worked out sampling system for cases of influenza-like illness that meet set criteria, with accurate and quick feedback of a presumptive microbiological diagnosis. Without this, we cannot generalise from random sampling.

Another reason may be "availability creep." In their efforts to deal with, or be seen to deal with, policy makers favour intervention with what is available—registered influenza vaccines. A similar philosophy is the "we have to make decisions and cannot wait to have perfect data" approach. This attitude may have an altruistic basis but has two important consequences. Firstly, it uses up resources that could be invested in a proper evaluation of influenza vaccines or on other health interventions of proven effectiveness. Secondly, the inception of a vaccination campaign seems to preclude the assessment of a vaccine through placebo controlled randomised trials on ethical grounds. Far from being unethical, however, such trials are desperately needed and we should invest in them without delay. A further consequence is reliance on non-randomised studies once the campaign is under way. It is debatable whether these can contribute to our understanding of the effectiveness of vaccines. Ultimately non-randomised designs cannot answer questions on the effects of influenza vaccines.

<CENTER><TABLE cellSpacing=0 cellPadding=0 width="95%" border=1><TBODY><TR bgColor=#e1e1e1><TD><TABLE cellSpacing=2 cellPadding=2 width="100%"><TBODY><TR bgColor=#e1e1e1><TD vAlign=top align=left bgColor=#ffffff>Summary points

Public policy worldwide recommends the use of inactivated influenza vaccines to prevent seasonal outbreaks

Because viral circulation and antigenic match vary each year and non-randomised studies predominate, systematic reviews of large datasets from several decades provide the best information on vaccine performance

Evidence from systematic reviews shows that inactivated vaccines have little or no effect on the effects measured

Most studies are of poor methodological quality and the impact of confounders is high

Little comparative evidence exists on the safety of these vaccines

Reasons for the current gap between policy and evidence are unclear, but given the huge resources involved, a re-evaluation should be urgently undertaken

</TD></TR></TBODY></TABLE></TD></TR></TBODY></TABLE></CENTER>

The optimistic and confident tone of some predictions of viral circulation and of the impact of inactivated vaccines, which are at odds with the evidence, is striking. The reasons are probably complex and may involve "a messy blend of truth conflicts and conflicts of interest making it difficult to separate factual disputes from value disputes"22 or a manifestation of optimism bias (an unwarranted belief in the efficacy of interventions).23

Whatever the reasons, it is a sobering thought that Archie Cochrane's 1972 statement that we should use what has been tested and found to reach its objectives is as revolutionary now as it was then.

webplus.f2.gif Details of the search strategy are on bmj.com

Contributor: TJ designed and wrote the paper and is the sole contributor and guarantor.

Competing interests: TJ owned shares in Glaxo SmithKline and received consultancy fees from Sanofi-Synthelabo (2002) and Roche (1997-9).

References

<LI value=1>Centers for Disease Control and Prevention. Prevention and control of influenza: recommendations of the Advisory Committee on Immunization Practices (ACIP). Morbid Mortal Wkly Rep 2006;55: 1-41. <LI value=2>American Academy of Pediatrics Committee on Infectious Diseases. Recommendations for influenza immunization of children. Pediatrics 2004;113: 1441-7.[Abstract/Free Full Text] <LI value=3>Orr P. An advisory committee statement (ACS). National Advisory Committee on Immunization (NACI). Statement on influenza vaccination for the 2004-2005 season. Can Commun Dis Rep 2004;30: 1-32.[Medline] <LI value=4>Van Essen GA, Palache AM, Forleo E, Fedson DS. Influenza vaccination in 2000: recommendations and vaccine use in 50 developed and rapidly developing countries. Vaccine 2003;21: 1780-5.[CrossRef][iSI][Medline] <LI value=5>World Health Organization. Influenza vaccines. WHO position paper. Wkly Epidemiol Rec 2002;77: 230-40.[Medline] <LI value=6>Bueving HJ, van der Wouden JC, Berger MY, Thomas S. Incidence of influenza and associated illness in children aged 0-19 years: a systematic review. Rev Med Virol 2005;15: 383-91.[CrossRef][iSI][Medline] <LI value=7>Beyer WE, de Bruijn IA, Palache AM, Westendorp RG, Osterhaus AD. Protection against influenza after annually repeated vaccination: a metaanalysis of serologic and field studies. Arch Intern Med 1999;159: 182-8.[Abstract/Free Full Text] <LI value=8>Smith S, Demicheli V, Di Pietrantonj C, Harnden AR, Jefferson T, Matheson NJ, et al. Vaccines for preventing influenza in healthy children. Cochrane Database Syst Rev 2006;(1):CD004879. <LI value=9>Demicheli V, Rivetti D, Deeks JJ, Jefferson TO. Vaccines for preventing influenza in healthy adults. Cochrane Database Syst Rev 2004;(3):CD001269. <LI value=10>Thomas RE, Jefferson T, Demicheli V, Rivetti D. Influenza vaccination for healthcare workers who work with the elderly. Cochrane Database Syst Rev 2006;(3):CD005187. <LI value=11>Rivetti D, Demicheli V, Di Pietrantonj C, Jefferson TO, Thomas R. Vaccines for preventing influenza in the elderly. Cochrane Database Syst Rev 2006;(3):CD004876. <LI value=12>Cates CJ, Jefferson TO, Bara AL, Rowe BH. Vaccines for preventing influenza in people with asthma. Cochrane Database Syst Rev 2003;(4):CD000364. <LI value=13>Poole PJ, Chacko E, Wood-Baker RWB, Cates CJ. Influenza vaccine for patients with chronic obstructive pulmonary disease. Cochrane Database Syst Rev 2006;(1):CD002733. <LI value=14>Jefferson T, Rivetti D, Rivetti A, Rudin M, Di Pietrantonj C, Demicheli V. Efficacy and effectiveness of influenza vaccines in elderly people: a systematic review. Lancet 2005;366: 1165-74.[CrossRef][iSI][Medline] <LI value=15>Simonsen L, Reichert TA, Viboud C, Blackwelder WC, Taylor RJ, Miller MA. Impact of influenza vaccination on seasonal mortality in the US elderly population. Arch Intern Med 2005;165: 265-72.[Abstract/Free Full Text] <LI value=16>Simonsen L, Viboud C, Taylor R. Influenza vaccination in elderly people. Lancet 2005;366: 2086.[iSI][Medline] <LI value=17>Hak E, Verheij TJ, Grobbee DE, Nichol KL, Hoes AW. Confounding by indication in non-experimental evaluation of vaccine effectiveness: the example of prevention of influenza complications. J Epidemiol Community Health 2002;56: 951-5.[Abstract/Free Full Text] <LI value=18>Jordan R, Connock M, Albon E, Fry-Smith A, Olowokure B, Hawker J, et al. Universal vaccination of children against influenza: are there indirect benefits to the community? A systematic review of the evidence. Vaccine 2006;24: 1047-62.[CrossRef][iSI][Medline] <LI value=19>Bhalla P, Tan A, Smyth R. Vaccines for preventing influenza in people with cystic fibrosis. Cochrane Database Syst Rev 2000;(1):CD001753. <LI value=20>Harling R, Hayward A, Watson JM. Implications of the incidence of influenza-like illness in nursing homes for influenza chemoprophylaxis: descriptive study [see comments]. BMJ 2004;329: 663-4.[Free Full Text] <LI value=21>Carman WF, Wallace LA, Walker J. Rapid virological surveillance of community influenza infection in general practice. BMJ 2000;321: 736-7.[Free Full Text] <LI value=22>MacCoun RJ. Biases in the interpretation and use of research results. Annu Rev Psychol 1998;49: 259-87.[CrossRef][iSI][Medline]
Chalmers I, Matthews R. What are the implications of optimism bias in clinical research? Lancet 2006;367: 449-50.[CrossRef][iSI][Medline]

(Accepted 21 September 2006)

Related Articles

<DL><DT>Of measles and flu <DD>Fiona Godlee

BMJ 2006 333: 0. [Full Text]

</DD></DL><DL><DT>Implications of the incidence of influenza-like illness in nursing homes for influenza chemoprophylaxis: descriptive study <DD>Richard Harling, Andrew Hayward, and John M Watson

BMJ 2004 329: 663-664. [Full Text]

</DD></DL><DL><DT>Rapid virological surveillance of community influenza infection in general practice <DD>William F Carman, Lesley A Wallace, Jacqueline Walker, Sheena McIntyre, Ahilya Noone, Peter Christie, James Millar, and James D Douglas

BMJ 2000 321: 736-737. [Full Text]

</DD></DL>

Rapid Responses:

Read all Rapid Responses <DL><DT>Emperor's clothes exposed <DD>Nick Hardwick <DD>bmj.com, 28 Oct 2006 [Full text] <DT>Vested interests will always trump evidence <DD>GH Hall <DD>bmj.com, 28 Oct 2006 [Full text] <DT>Policy versus evidence: policy in the lead <DD>Peter Doshi <DD>bmj.com, 29 Oct 2006 [Full text] <DT>Influenza vaccination: what evidence can we rely on? <DD>Stuart J cornell <DD>bmj.com, 29 Oct 2006 [Full text] </DD></DL>

December 12, 2006

PAMHO

I'm 63 years of age, follow a Vedic diet, have NEVER gotten a flu vaccine, and do not INTEND to ever get one. The last time I got anything resembling the flu was once in 1993, and another time in 2004; a day or so of bed-rest, lots of liquids, and I was back rockin' & rollin'! Many of my friends at work (before I became semi-retired) used to kid me and call me the "alfalfa sprout"; after seeing how I got thru flu season after flu season and came to work while most of the staff were down-for-the-count, they started asking me to give them some of my vegetarian recipes. "Raj", one of my co-workers exclaimed, "You're gonna live to be 200!"

yus,

Bhakta Raj Prabhu das