Collecting Data and Experimental design

Eat Stop Eat

Best Weight Loss Programs That Work

Get Instant Access

Before we discuss any type of data summary and statistical analysis, it is important to recognize that the value of any statistical analysis is only as good as the data collected. Because we are using data or samples to draw conclusions about entire populations or processes, it is critical that the data collected (or samples collected) are representative of the larger, underlying population. In other words, if we are trying to determine whether men between the ages of 20 and 50 years respond positively to a drug that reduces cholesterol level, we need to carefully select the population of subjects for whom we administer the drug and take measurements. In other words, we have to have enough samples to represent the variability of the underlying population. There is a great deal of variety in the weight, height, genetic makeup, diet, exercise habits, and drug use in all men ages 20 to 50 years who may also have high cholesterol. If we are to test the effectiveness of a new drug in lowering cholesterol, we must collect enough data or samples to capture the variability of biological makeup and environment of the population that we are interested in treating with the new drug. Capturing this variability is often the greatest challenge that biomedical engineers face in collecting data and using statistics to draw meaningful conclusions. The experimentalist must ask questions such as the following:

• What type of person, object, or phenomenon do I sample?

• What variables that impact the measure or data can I control?

• How many samples do I require to capture the population variability to apply the appropriate statistics and draw meaningful conclusions?

• How do I avoid biasing the data with the experimental design?

Experimental design, although not the primary focus of this book, is the most critical step to support the statistical analysis that will lead to meaningful conclusions and hence sound decisions.

One of the most fundamental questions asked by biomedical researchers is, "What size sample do I need?" or "How many subjects will I need to make decisions with any level of confidence?"

We will address these important questions at the end of this book when concepts such as variability, probability models, and hypothesis testing have already been covered. For example, power tests will be described as a means for predicting the sample size required to detect significant differences in two population means using a t test.

Two elements of experimental design that are critical to prevent biasing the data or selecting samples that do not fairly represent the underlying population are randomization and blocking.

Randomization refers to the process by which we randomly select samples or experimental units from the larger underlying population such that we maximize our chance of capturing the variability in the underlying population. In other words, we do not limit our samples such that only a fraction of the characteristics or behaviors of the underlying population are captured in the samples. More importantly, we do not bias the results by artificially limiting the variability in the samples such that we alter the probability model of the sample population with respect to the probability model of the underlying population.

In addition to randomizing our selection of experimental units from which to take samples, we might also randomize our assignment of treatments to our experimental units. Or, we may randomize the order in which we take data from the experimental units. For example, if we are testing the effectiveness of two different medical imaging methods in detecting brain tumor, we will randomly assign all subjects suspect of having brain tumor to one of the two imaging methods. Thus, if we have a mix of sex, age, and type of brain tumor participating in the study, we reduce the chance of having all one sex or one age group assigned to one imaging method and a very different type of population assigned to the second imaging method. If a difference is noted in the outcome of the two imaging methods, we will not artificially introduce sex or age as a factor influencing the imaging results.

As another example, if one are testing the strength of three different materials for use in hip implants using several strength measures from a materials testing machine, one might randomize the order in which samples of the three different test materials are submitted to the machine. Machine performance can vary with time because of wear, temperature, humidity, deformation, stress, and user characteristics. If the biomedical engineer were asked to find the strongest material for an artificial hip using specific strength criteria, he or she may conduct an experiment. Let us assume that the engineer is given three boxes, with each box containing five artificial hip implants made from one of three materials: titanium, steel, and plastic. For any one box, all five implant samples are made from the same material. To test the 15 different implants for material strength, the engineer might randomize the order in which each of the 15 implants is tested in the materials testing machine so that time-dependent changes in machine performance or machine-material interactions or time-varying environmental condition do not bias the results for one or more of the materials. Thus, to fully randomize the implant testing, an engineer may literally place the numbers 1-15 in a hat and also assign the numbers 1-15 to each of the implants to be tested. The engineer will then blindly draw one of the 15 numbers from a hat and test the implant that corresponds to collecting data and experimental DESIGN 7

that number. This way the engineer is not testing all of one material in any particular order, and we avoid introducing order effects into the data.

The second aspect of experimental design is blocking. In many experiments, we are interested in one or two specific factors or variables that may impact our measure or sample. However, there may be other factors that also influence our measure and confound our statistics. In good experimental design, we try to collect samples such that different treatments within the factor of interest are not biased by the differing values of the confounding factors. In other words, we should be certain that every treatment within our factor of interest is tested within each value of the confounding factor. We refer to this design as blocking by the confounding factor. For example, we may want to study weight loss as a function of three different diet pills. One confounding factor may be a person's starting weight. Thus, in testing the effectiveness of the three pills in reducing weight, we may want to block the subjects by starting weight. Thus, we may first group the subjects by their starting weight and then test each of the diet pills within each group of starting weights.

In biomedical research, we often block by experimental unit. When this type of blocking is part of the experimental design, the experimentalist collects multiple samples of data, with each sample representing different experimental conditions, from each of the experimental units. Figure 2.1 provides a diagram of an experiment in which data are collected before and after patients receives therapy, and the experimental design uses blocking (left) or no blocking (right) by experimental unit. In the case of blocking, data are collected before and after therapy from the same set of human subjects. Thus, within an individual, the same biological factors that influence the biological response to the therapy are present before and after therapy. Each subject serves as his or her own control for factors that may randomly vary from subject to subject both before and after therapy. In essence, with blocking, we are eliminating biases in the differences between the two populations

Block (Re

peated Measures)

No Block

(No repeated measures)


Measure before treatment

Measure after treatment


Measure before treatment


Measure after treatment





























FIGuRE 2.1: Samples are drawn from two populations (before and after treatment), and the experimental design uses block (left) or no block (right). In this case, the block is the experimental unit (subject) from which the measures are made.

(before and after) that may result because we are using two different sets of experimental units. For example, if we used one set of subjects before therapy and then an entirely different set of subjects after therapy (Figure 2.1, right), there is a chance that the two sets of subjects may vary enough in sex, age, weight, race, or genetic makeup, which would lead to a difference in response to the therapy that has little to do with the underlying therapy. In other words, there may be confounding factors that contribute to the difference in the experimental outcome before and after therapy that are not only a factor of the therapy but really an artifact of differences in the distributions of the two different groups of subjects from which the two samples sets were chosen. Blocking will help to eliminate the effect of intersubject variability.

However, blocking is not always possible, given the nature of some biomedical research studies. For example, if one wanted to study the effectiveness of two different chemotherapy drugs in reducing tumor size, it is impractical to test both drugs on the same tumor mass. Thus, the two drugs are tested on different groups of individuals. The same type of design would be necessary for testing the effectiveness of weight-loss regimens.

Thus, some important concepts and definitions to keep in mind when designing experiments include the following:

• experimental unit: the item, object, or subject to which we apply the treatment and from which we take sample measurements;

• randomization: allocate the treatments randomly to the experimental units;

• blocking: assigning all treatments within a factor to every level of the blocking factor. Often, the blocking factor is the experimental unit. Note that in using blocking, we still randomize the order in which treatments are applied to each experimental unit to avoid ordering bias.

Finally, the experimentalist must always think about how representative the sample population is with respect to the greater underlying population. Because it is virtually impossible to test every member of a population or every product rolling down an assembly line, especially when destructive testing methods are used, the biomedical engineer must often collect data from a much smaller sample drawn from the larger population. It is important, if the statistics are going to lead to useful conclusions, that the sample population captures the variability of the underlying population. What is even more challenging is that we often do not have a good grasp of the variability of the underlying population, and because of expense and respect for life, we are typically limited in the number of samples we may collect in biomedical research and manufacturing. These limitations are not easy to address and require that the engineer always consider how fair the sample and data analysis is and how well it represents the underlying population(s) from which the samples are drawn.

Was this article helpful?

0 0
Weight Loss Enigma

Weight Loss Enigma

Finally Revealed The Revolutionary & Miraculous Weight Loss Secrets! Discover How to Command Those Unwanted Pounds To Take A Hike, So That You Can Get Into Shape & Lose Weight Easily Just Like You Dream Of In Just A Matter Of Weeks! You're About to Discover The Insider's Fat Burning Diet Tips to Easily Shed Off Those Extra Pounds And Obtain a Lovable Shape In No Time Flat!

Get My Free Ebook


  • Eoin
    How to collect data for experimental designs?
    8 years ago
  • zahra temesgen
    How to collect data for experiment design?
    8 years ago
  • alfreda
    Is collecting data an experimental design?
    8 years ago
    How to collect experiemental data?
    7 years ago
  • benigna de luca
    How to collect data in an experimental design?
    7 years ago

Post a comment