1
Foundation of Experiments

1.1 Uncertainties in Evidences

A research investigation can be exploratory or confirmatory. Exploratory research is generally performed with observational studies or experiments to formulate new hypotheses, tested with a confirmatory investigation [Jaeger and Halliday, 1998]. Confirmatory work by performing experiments involve testing hypotheses generated from an earlier exploratory analysis. The uncertainties in an experiment are controllable in part, but the remaining vast majority remain uncontrollable.

R.A. Fisher wrote in Chapter 1: Introduction, of his book The Design of Experiments [Fisher, 1935]:

"When any scientific conclusions is supposed to be proved on experimental evidence, critics who still refuse to accept the conclusions are accustomed to take one of two lines of attack. They may claim that the interpretation of the experiment is faulty, that the results reported are not in fact those which should have been expected had the conclusion drawn been justified, or that they might equally well have arisen had the conclusion drawn been false. Such criticisms of interpretation are usually treated as falling within the domain of statistics . The other type of criticism to which experimental results are exposed is that the experiment itself was ill designed, or, of course, badly executed. If we suppose that the experimenter what intended to do, both of these points come down to the question of design, or the logical structure of the experiment."

D.R. Cox wrote in Chapter 1: Preliminaries, of his book Planning of Experiments [Cox, 1958]:

"This book is about the planning of experiments in which effects under investigation tend to be masked by fluctuations outside the experimenter's control. Large uncontrolled variations are common in technological experiments and in many types of work in biological sciences, and it is in these fields that the methods to be described are most used."

1.2 Examples

1.2.1 The Louis Pasteur Anthrax Vaccination Experiment

The famous French microbiologist and chemist, Louis Pasteur, performed a dramatic public trial from 5 May to 2 June in 1881, known as "The Anthrax Vaccination Experiment," at the small French village of Pouilly-le-Fort near Melun [Pasteur, 2002/1881]. The goal of his experiment was to demonstrate the efficacy of anthrax vaccination on animals (2002: Translated). Anthrax was a leading cause of the death of a very large number of animals at that time in France and became a major burden on the economy of France.

The 60 animals were divided into two groups:

1. Group 1: 24 sheep, 1 goat, and 6 cows, in total 31 animals;
2. Group 2: 24 sheep, 1 goat, and 4 cows, in total 29 animals.

On 5 May 1881, all the animals in Group 1 were inoculated with five drops of attenuated anthrax. Attenuation makes anthrax agent harmless or less virulent. On 17 May, all the animals in Group 1 were re-vaccinated by more attenuated anthrax that were more virulent than the anthrax used in the previous vaccination. The Group 1 animals were thus vaccinated on 5 May and 17 May differently, but Group 2 animals were not at all vaccinated up to that point of time. On 31 May, all the animals in both Group 1 and Group 2 were inoculated with the strong virulent anthrax.

On 2 June, 48?hours after the inoculations with the strong virulent anthrax, all the animals in Group 1 were found healthy. Among 29 animals in Group 2, 21 sheep, and 1 goat were found already dead at the time of viewing, 2 sheep died while viewing, and 1 sheep died at the end of the day. In total 25 out of 29 animals in Group 2 were dead on 2 June. All four cows were alive as cows were less prone than sheep and goat to die of anthrax. Louis Pasteur was successful in clearly demonstrating the efficacy of vaccination in his public trial. This was one of the many experiments performed by Louis Pasteur. We are now blessed in having the vaccines available to save all living beings from this deadly disease.

1.2.2 The Lanarkshire Milk Experiment: Milk Tests in Lanarkshire Schools

Pasteurization of milk is the process of heating milk to a very high temperature and cooling it again to kill all pathogenic or disease-causing microbes. Louis Pasteur developed pasteurization in 1864. A very large-scale three-arm nutritional experiment was performed in 1930 for about four months by the Department of Health in Scotland with the investigators Dr Gerald Leighton, Medical Officer (Foods), and Dr Peter L. McKinlay, Medical Officer (Statistics) [Leighton and McKinlay, 1930]. Twenty thousand Lanarkshire (an historic county of Scotland) schoolchildren in seven age groups with ages between 5 to 12 years, and the same number of male and female students in each age group were divided into three experimental groups:

1. Group 1: Five thousand children who received a daily ration (three-quarters pint) of Grade A (Tuberculin tested) raw milk;
2. Group 2: Five thousand children who received a daily ration (three-quarters pint) of Grade A (Tuberculin tested) pasteurized milk;
3. Group 3: Ten thousand children who were control subjects.

The 67 schools were located in the densely populated industrial part of the county with no information on the economic status of the localities chosen. Only one-third of the children had estimated complete or partial unemployment at their homes. Each school provided between 200 and 400 students for the experiment: half of them were assigned to Group 3 and the other half were assigned to one of Groups 1 and 2. Children from a school were assigned to either Groups 1 and 3 or Groups 2 and 3 but not Groups 1, 2, and 3. The weight and height of children were recorded at the beginning and end of the experiment. Complete records were available for only 17?159 children. Children in Groups 1 and 2 were "feeders" and in Group 3 were "controls." The conclusions drawn from the experiment with respect to the rate of growth in weight and height (Student 1931) are the following:

1. Children in Groups 1 and 2 demonstrated a definite increase over Children in Group 3;
2. No obvious or constant difference found between the genders. Little evidence of definite relation found between the age of the children and the amount of improvement. Younger children showed no higher increase than older children in Groups 1 and 2 not supporting the popular belief that the younger children gain more than the older children;
3. Children in Group 1 demonstrated the same with children in Group 2.

R.A. Fisher and S. Bartlett [1931] as well as Student [1931], the pseudonym of W.S. Gosset expressed their concerns about the comparability of students in Group 1 and Group 2 and consequently, about Conclusion 3 drawn on the contrast of the effects of pasteurized vs. raw milk. With 14 groups from 7 age groups and 2 gender groups, Fisher and Bartlett pointed out that pasteurized milk gave a greater increase in height in only 2 groups, the increases were equal in 1 group, and the raw milk gave the greater increase in 11 groups. Student [1931] commented that the same school students were assigned to either Group 1 or Group 2 but not both for administering the experiment conveniently. Student remarked that the teachers presumably assigned more "ill-nourished" students among the "feeders" and too few among the "controls" making the assignment "non-random" and thereby introduced "selection bias" in the experiment. A possible source of "measurement bias" identified by Student was in the difference in winter clothing between "ill-nourished" and "well-fed" students affecting the weight measurements. Student concluded: "I would be very chary of drawing any conclusion from these small biased differences. That is not to say that there is no difference between the effect of raw and pasteurized milk?-?personally I believe that there is and that it is in favour of raw milk?-?but that this experiment, in spite of all the good work which was put into it, just lacked the essential condition of randomness which would have enabled us to prove the fact."

The large-scale Lanarkshire milk experiment was performed to establish the benefit of drinking milk and comparing pasteurized against the raw milk. Two lines of concerns were expressed on Conclusions 1-3: The experiment was not properly "designed" or "planned" and the interpretation of the experiment was "faulty."

1.3 Replication, Randomization, Blocking, and Blinding

1.3.1 Replication

Out of 20?000 schoolchildren in the Lanarkshire milk experiment, 5000 children received raw milk in Group 1, another 5000 children received pasteurized milk in Group 2, and another 10?000 children received no milk in Group 3. The replication in Group 1 is 5000 which is also the replication in Group 2. So the replication in Group 1 and Group 2 together representing children who received milk raw or pasteurized is 10?000 which is also the replication in Group 3 representing children who received no milk. In the Student's proposed milk experiment, the replication of children who...