If accurate estimation of causal relations is the goal of epidemiologic studies, then success has been attained when the measure of effect accurately quantifies the causal impact of the exposure on disease in the population under investigation. Even if this goal is completely fulfilled, more is needed. Public health practice and policy requires extrapolation of the findings to other defined populations or to people in general. Such generalization does not result from a single study, nor is it a matter of statistical sampling and inference (Rothman, 1986). Only a series of internally valid studies can result in a body of evidence to help judge (not prove) whether some more universal causal relation is operating that would apply to populations not yet studied.
Extrapolation of findings to previously unstudied populations, by definition, goes beyond the available data, and is thus vulnerable to error in addition to whatever error is contained in the studies that provide the basis for the extrapolation. Universal causal relations (Smoking causes lung cancer.) reflect the ultimate extrapolation, synthesizing a series of individual studies into the untestable assertion about what exposure would do to disease risk in all possible past, present, and future populations. Nonetheless, when we use epidemiologic evidence to guide decisions about individual behavior and public policy, we are implicitly extrapolating a set of research observations to just such new and previously untested situations and populations. Causality is assessed based on judgments about the validity of individual studies, the accumulation of those studies, and extrapolation of the results beyond the study populations that generated the findings.
Application of epidemiologic evidence to other populations, to individual decision-making, or to public health policy requires caution. There are concentric layers of application for epidemiologic evidence. A narrow one might be the use of the data to estimate, within the study population, the quantitative effect of exposure on the occurrence of disease. Somewhat broader would be the use of that evidence to estimate the effect of exposure on disease for a broader population outside the study but otherwise socially and demographically similar to the study population, perhaps to help formulate policy. Assuming that the policy experts are well informed, they will be able to accurately evaluate the strength and clarity of the epidemiologic evidence.
As the information reaches clinicians or the public at large, the questions may go well beyond what can be gleaned directly from epidemiologic data, for example, asking whether a change in clinical practice or individual behavior is warranted. It is thus important for epidemiology to examine the full spectrum of potential consequences of a change in policy or practice on the public's health, identifying unanticipated consequences of altered exposure as well as desired outcomes. These are often the most important considerations in setting policy, and epidemiology has a unique role to fulfill in generating critical information.
Beyond the scope of epidemiology comes the feasibility and costs of actually modifying exposure through behavior change, clinical guidelines, or regulation. As the goal is expanded, moving from a characterization of the risks and benefits of alternative courses of action to the question of what should be done and how to achieve the desired end, the sufficiency of even impeccable epidemio-logic information diminishes and considerations outside of epidemiology often become increasingly prominent. There may be tension between the cautiousness of researchers who wish to ask narrow, modest questions of the data (for which it may be well-suited) and the public who wish to ask the broadest possible questions of ultimate societal interest (for which the data are often deficient).
Even among researchers, different questions can be asked of the same data, and the quality of a given body of data for one type of application may well differ from the same data used for other purposes. Tabulations of cervical cancer mortality in relation to women's occupation (Savitz et al., 1995) have several possible applications, for example. We might ask what guidance such data can provide for cervical cancer screening, making no assumptions whatsoever regarding why some occupational groups have higher risk than others, accepting at face value the observation that they have such elevated risk. If women who work in the manufacturing industry show increased rates of cervical cancer, worksite screening programs in the manufacturing sector might be encouraged. A different application of this observation would be to address the etiologic basis for why some occupational groups show higher risks than others. If we are concerned with the effect of workplace chemical exposures in the etiology of cervical cancer, the exact same data documenting differential risk by employment sector are far less effective since we are lacking needed information on actual workplace exposure. The very same study may answer some questions very effectively, e.g., "Which work sectors are at higher risk?" and others poorly or not at all, e.g., "Do workplace chemicals cause cervical cancer?" Thus, the study's value must be defined relative to a specific application.
The distinctions between the goals of the data generator and data interpreter are especially apparent for descriptive epidemiology, such as demographic patterns, time trends in disease, and to some extent, patterns across groups defined by such broad attributes as gender, social class, and occupation. Such data are often generated for administrative purposes or perhaps to stimulate new ideas about why risks vary across time and populations. Nevertheless, clever interpreters often bring such data to bear in evaluation of causal hypotheses, such as the effect of the introduction or removal of potential causes of disease on time trends in disease occurrence, or the effectiveness of a newly introduced therapy in reducing mortality from a given disease. Even technically accurate data do not guarantee that the inferences that rely on those data are free of error. It depends on the match between the information and the use to which is it put.
The diverse interests of those who evaluate epidemiologic data, including those who generate it, serve as a useful reminder that the data are the object of inquiry rather than the character or intelligence of those who generate it. Individuals of the highest intelligence and moral character can generate flawed information, and those of limited talent can stumble into important and trustworthy findings. The elusive search for objectivity in generating and interpreting epidemiologic evidence is well served by a single-minded focus on the product and application of the information rather than the people who generate or use that product. Efforts to judge evidence based on the track record or mental processes of the investigator can only be a distraction. Although it may sound obvious that it is only the quality of the data that counts, this issue arises in considerations of disclosure of financial support for research that that may bias the investigator (Davidoff et al., 2001), the interpretation of data based on the intent or preconceptions of the investigator (Savitz & Olshan, 1995), and most insidiously, when research is judged based on the track record of those who generated it. As we make use of epi-demiologic data to draw inferences, it is necessary to step back not only from the investigators as human beings but even from the original study goals to ask how effectively the information answers a specific question and contributes to a specific inference.
Was this article helpful?