Instead of data pooling, which is logistically difficult due to the need to obtain raw data and conduct new analyses, epidemiologists have increasingly turned to

Finland-I

Winnipeg

Missouri

100 200 300 400 Radon concentration (Bq/m3)

Finland-ll

100 200 300 400 Radon concentration (Bq/m3)

Finland-I

Winnipeg

Missouri

Finland-ll s r

Shenyang

Figure 11.2. Relative risks (RRs) for radon concentration categories and fitted exposure-response models for each case-control study. Fitted lines are adjusted to pass through the quantitative value for the baseline category. Models fit to the logarithm of the RRs are linear with respect to radon. There was a significant departure from linearity in the Finland-I data, and also shown is the model which is linear and quadratic with respect to radon (Lubin & Boice, 1997, page 53).

0 100 200 300 400

Sweden

100 200 300 400 Radon concentration (Bq/m3)

Sweden

100 200 300 400 Radon concentration (Bq/m3)

STUDY |
RR* |
95% CI |
REPORTED IN ORIGINAL PAPERt |

Finland-Ii |
1.30 |
1.09-1.55 |
NA |

Finland-II |
1.01 |
0.94-1.09 |
1.02 |

New Jersey |
1.83 |
1.15-2.90 |
1.77 |

Shenyang |
0.84 |
0.78-0.91 |
0.92§ |

Winnipeg |
0.96 |
0.86-1.08 |
0.97 |

Stockholm |
1.83 |
1.34-2.50 |
1.79 |

Sweden |
1.20 |
1.13-1.27 |
1.15 |

Missouri |
1.12 |
0.92-1.36 |
NA |

Combined^ |
1.14 |
1.01-1.30 |

*Values shown are estimated RR at 150 Bq/m3, i.e., exp(b X 150), where b was the estimate of ¡3

obtained from a weighted linear regression fitting the model log(RR) = ¡(x — xo), where xo is the quantitative value for the lowest radon category and x is the category-specific radon level.

fRR at 150 Bq/m3, based on or computed from exposure-response relationship provided in original reports. Exposure response was not available (NA) in Finland-I and Missouri studies.

¿For Finland-I, there was a significant departure from linearity (P = 0.03). The estimated RR for

150 Bq/m3 under a linear-quadratic model was 1.71.

§Taken from results in pooled analysis (18).

^Combined estimate and CI based on a random effects model. Fixed effects estimate was 1.11 (95% CI = 1.07-1.15).

RR, relative risk; CI, confidence interval. Lubin & Boice, 1997.

*Values shown are estimated RR at 150 Bq/m3, i.e., exp(b X 150), where b was the estimate of ¡3

obtained from a weighted linear regression fitting the model log(RR) = ¡(x — xo), where xo is the quantitative value for the lowest radon category and x is the category-specific radon level.

fRR at 150 Bq/m3, based on or computed from exposure-response relationship provided in original reports. Exposure response was not available (NA) in Finland-I and Missouri studies.

¿For Finland-I, there was a significant departure from linearity (P = 0.03). The estimated RR for

150 Bq/m3 under a linear-quadratic model was 1.71.

§Taken from results in pooled analysis (18).

^Combined estimate and CI based on a random effects model. Fixed effects estimate was 1.11 (95% CI = 1.07-1.15).

RR, relative risk; CI, confidence interval. Lubin & Boice, 1997.

meta-analysis as a quantitative approach to integrating information from a set of published studies on the same topic. In meta-analysis, the unit of observation is the individual study and statistical tools are applied to the array of study results in order to extract the most objective, useful information possible about the phenomenon of interest. The methods and results of the included studies are analyzed to draw inferences from the body of relevant literature. The tools of metaanalysis have been described in detail elsewhere (Petitti, 1994; Greenland, 1998).

In synthetic meta-analysis, the goals are comparable to those of data pooling. The large number of observations acquired by integrating multiple studies are used to derive a more precise measure of effect than could be obtained from any of the individual studies. The goal is to generate a precision-weighted average result from across the series of studies, as illustrated in Table 11.2 for the estimated relative risk of 1.14 (95% confidence interval = 1.01-1.30) for lung cancer from exposure to 150 Bq/m3 in the meta-analysis of Lubin and Boice (1997). Statistical methods are available that take the variability among studies into account and derive a common point and interval estimate. Obviously, by taking into account all the data that comprise the individual studies, the overall estimate is markedly more precise than any of the original studies taken in isolation. Cynics suggest that the main goal of meta-analysis is to take a series of studies that demonstrate an effect that is not statistically significant and combine the studies to derive a summary estimate that is statistically significant, or worse yet, to take a series of imprecise and invalid results and generate a highly precise invalid result.

One of the challenges in conducting such meta-analyses is to ensure that the studies that are included are sufficiently compatible methodologically to make the exercise of synthesizing a common estimate an informative process. The algebraic technology will appear to work even when the studies being combined are fundamentally incompatible with respect to the methods used to generate the data. In practice, studies always differ from one another in potentially important features such as study locale, response proportion, and control of confounding, so that the decision to derive a summary estimate should be viewed at best as an exercise, in the same spirit as a sensitivity analyses. The question that is addressed by a meta-analysis is as follows: If none of the differing features of study methods affected the results, what would be the best estimate of effect from this set of studies? The value and credibility of that answer depends largely on the credibility of the premises.

An alternative approach to synthetic meta-analysis focused on the derivation of a single, pooled estimate, is to apply the statistical tools of meta-analysis to examine and better understand the sources of heterogeneity across the component studies. By focusing on the variability in results as the object of study, we can identify and quantify the influences of study methods and potential biases on study findings, rather than assume that such methodologic features are unimportant. The variability in study features, which are viewed as a nuisance when seeking a summary estimate, is the raw material for exploratory meta-analysis (Greenland, 1998).

In exploratory meta-analysis, the structural features of the study, such as location, time period, population source, and the measures of study conduct such as response proportion, masking of interviewers, and amount of missing information, are treated as potential determinants (independent variables) of the study results. Through exploratory meta-analysis, the manner in which study methods influence study results can be quantified, perhaps the most important goal in evaluating a body of epidemiologic literature. In parallel with the approach to examining methods and results within a single study, the focus of previous chapters, the same rationale applies to the examination of methods and results across studies. Just as the insights from such analyses of potential biases within a study help to assess the credibility of its findings, the pattern of results across a series of studies helps to more fully understand the constellation of findings and its meaning.

Sometimes, systematic examination of the pattern of results across studies yields a clear pattern in which methodologic quality is predictive of the results.

That is, studies that are better on average tend to show stronger (or weaker) measures of association, suggesting where the truth may lie among existing results or what might be expected by extrapolating to studies that are even better than the studies conducted thus far. For example, if higher response proportions were independently predictive of stronger associations, one would infer, all other things being equal, that a stronger association would be expected if non-response could be eliminated altogether. The studies with the higher response proportion are presumably yielding more valid results, all other things equal, and thus the observation that these studies yield stronger associations supports an association being truly present and stronger in magnitude than was observed even in the study with the best response thus far. The opposite pattern, higher response proportions predicting weaker association, would suggest that no association or only a weak one is present. Heterogeneity of results across studies is being explained in a manner that indicates both which results among completed studies are more likely to be valid and the basis for projecting what would be found if the methodologic limitation could be circumvented altogether. In meta-regression, such independent effects of predictors can be examined with adjustment for other features of the study that might be correlated with response proportions. With a sufficient number of observations, multiple influences on study findings can be isolated from one another.

Interpretation of the patterns revealed by exploratory meta-analysis is not always straightforward, of course, just as the corresponding relation between methods and results is not simple within individual studies. For example, one might observe that studies conducted in Europe tend to yield different results (stronger or weaker associations) than those conducted in North America. Neither group is necessarily more valid, but this pattern would encourage closer scrutiny of issues such as the methods of diagnosis, available tools for selecting controls in case-control studies, cultural attitudes toward the exposure of interest, or even the very nature of the exposure, which may well differ by geographic region. Sometimes there are time trends in which results of studies differ systematically as a function of the calendar period of study conduct, again subject to a variety of possible explanations. Even when features of the studies that do not correspond directly to indices of quality are predictive of results, much progress has been made beyond simply noting that the studies are inconsistent. The product of examining these attributes is refinement of the hypotheses that might explain inconsistent results in the literature.

The requirements for the application of exploratory meta-analysis are substantial, and often not met for topics of interest. The key feature is having a sufficient number of studies to conduct regression analyses that can examine and isolate multiple determinants of interest. The number of available studies determines in part the feasibility of conducting meta-regression with multiple predictors, just as the number of individual subjects does so in regression analyses of individual studies (Greenland, 1998). A second requirement is sufficient variability in potential influences on study results for informative evaluation, as in any regression analysis. If all studies of a given topic use a population-based case-control design, the influence of design on results cannot be examined. The alternative to exploratory meta-analysis, which is not without its advocates, is a careful narrative review and description of the relationship between study methods and results, without a formal statistical analysis of that pattern. The traditional detailed review of the literature without quantitative analysis may lend itself to closer scrutiny of individual study methods and their results. In addition, narrative reviews avoid the potential for the appearance of exaggerated certainty resulting from the meta-analysis. Ostensibly precise, quantitative information can be misleading if the assumptions that went into its generation are not kept firmly in mind. On the other hand, without statistical tools, it is difficult if not impossible to isolate multiple determinants from one another and discern patterns clearly. The reasons for a multivariate approach to examining influences on results across studies is identical to the rationale for multivariate analysis in studies of individuals: without it, there is no way to understand how multiple influences operate, independent of and in relation to one another.

Was this article helpful?

Your first day without cigarettes can be difficult, but having a plan will make it easier! Learn what steps to take on the day you quit smoking.

## Post a comment