Research Methods lecture 3

This lecture will be a continuation to the last two lectures. The lecture starts from the slide entitled Measurement Bias:

Last time we stopped at the bias and we said that we have different types of bias so we have Selection bias, Measurement bias and Confounding bias. But first the doctor has received a very good question from one of our colleagues and it was about how to distinguish between Selection bias and Confounding bias. And the doctor will explain the difference in an example, the example was about the mine workers; as they inhale and breathe from the chemicals so it is possible that these people develop lung cancer, also we have the effect of smoking as some of the workers smoke, so how can we study the effect of the chemicals on the workers without considering the effect of smoking on lung cancer. So can we consider both smoking and chemicals Selection bias? We said no, if you want to consider it as confounding bias the confounder affects the relationship between the independent variable and dependent variable, so the independent variable is working at a mine and the dependent variable is developing lung cancer, smoking comes and affects this relationship. But in Selection bias we dont have a factor that affects a relationship, we have two things that I should select and separate but I didnt. So an example about Selection bias>> I want to examine dental caries, to see dental caries among different age groups, this is a Selection bias because normally I know that the rate of caries is different among age groups; children have a high rate of caries but as we grow in age caries become less and less and old people have small or low rate of caries, so it is not possible to do a research
Research Method Page 1

where I dont define my age group so this is a Selection bias, I cant say that this affects a relationship. So in Selection bias I have a general group but I didnt select properly from that group, I should define my sample, so if a type of bias results when I fail to define my sample. But the confounding bias we have a common relationship between a factor and another, and then another factor comes in and affects this common relationship. Example on Confounding bias>> tri-glyceride and cholesterol level are associated with the risk of a coronary heart disease; we cant take only the effect of triglyceride because we also know that cholesterol also affects the relationship between tri-glyceride in developing heart disease. So it similar to the example mentioned earlier about the mine workers. But if I want to control the confounder (back to the example of the mine workers) we might want to study miners who do not smoke, so smoking will not affect my results. Another example is the relation between taking folic acid and developing colon cancer, it is a good research but people who take folic acid and vitamins are concerned about their health so they may not develop colon cancer not because of taking folic acid but because they take good care of themselves, so this is confounding factor that we want to consider. A variable is not confounded if it is directly along the path from cause to effect. A confounding variable is not necessarily a cause itself. To understand the last sentence lets go back to the taking folic acid and colon cancer developing example, so we have a relationship, but we didnt consider the confounding factor that healthy people may also have a lower rate of colon cancer, so in this case the confounder is not the cause by itself because health isnt related to lowering colon cancer. But when taking the example of the miners both smoking and working at a mine are causes of lung cancer. Selection bias is an issue in patients selection for observation, so it is important in the design of study. Selection bias always comes when you do not pay attention to selecting or defining your sample, so you can avoid that before doing the study.
Research Method Page 2

Confounding bias is an issue in analysis data, once the observations have been made. All the time Confounding bias after you do the study you have to consider those who were smokers among miners so you have to exclude them or you have to do the study without them, so thats why it doesnt come in the selection of the sample but this bias can be avoided while you are analyzing your data. Bias often at in the same study more than one bias operates. It is not necessarily that one bias operates sometimes we have a Selection bias, a Measurement bias and a Confounding bias at the same time. A distinction must be made between the potential for a bias and the actual presence of a bias a particular study. Sometimes there is a possibility of a bias to operate; sometimes the bias is actually present. So we have to understand the point where the bias actually exists or the bias may be present, so we want to be able to identify the difference between them. How we deal with bias? First we have to identify the bias because if we dont identify it we might miss it and it will appear in the study and we might not pay attention. Measuring the potential effect of bias is to measure the potential effect for bias, sometimes we can modify the research design when the potential effect on the result is big. If we think that the effect is big then we have to modify the study. Lets go back again to the workers at a mine whom might develop lung cancer, I have to modify my study, how? I have to study who are non-smokers, so I have to modify my study to avoid Confounding bias. Sometimes changing the conclusion in a clinically meaningful way when the effect is not big enough. When the effect is small like when I say among those who work in a mine 2% are smokers (which is not true, as the usually have a high percentage of smokers) and because I see that it is a small percentage I continue my study but I have to provide good explanation while I am writing the report, so in the conclusion I have given a meaningful explanation so the reader can see that you did a good study. Bias usually leads to systematic error in general. Example: I want to know the average blood pressure in the Jordanian population; we know the average is 120/80. If I do a study and the average is 105/75 this is a
Research Method Page 3

systematic error so we are sure that there is a bias because the truth is not in its actual position. The truth has departed either a lower or higher. But in chance it is different; unbiased may misrepresent the population because of chance. For chance to happen the sample should be unbiased. Chance is the divergence of an observation on a sample from the true population value. But be careful it is the divergence. So there is divergence in a direction or another. It is very important to know that the effect of chance can happen in both directions around the truth. Again we know that the average blood pressure is 120/80 and did a study and got results like 115/80 or 125/80, so it is possible to be higher or lower. And the percentage of those higher should be equal to those who are lower, but in bias the observation should be away from the actual truth, either sides but in chance for the data to be higher or lower and the distribution is equal around the truth. Chance is called random variation, example tossing a coin 100 times, so when you toss a coin 10 times can you guarantee that you will get 5 times coin and 5 tails? The answer is no, you cant guarantee that. Maybe 7 to 3 or 6 to 4. But what happens if you do it 100 times? Can you get something like 70 to 30? It is not a big possibility; you might get something like 55 to 45. That means as you increase your sample the effect of chance decreases. So what if you do it 1000 times? You might get 520 to 480. So when the sample is increased the effect of chance decreases. The chance is the factor that makes the results of tossing not equal 50:50 heads and tails. Chance can be decreased by increasing the sample, but bias cant be avoided by increasing the sample size, bias has nothing to do with sample size, so increasing or decreasing the sample has no effect on bias. Now an explanatory example: If we want to know who is taller boys or girls, and take 20 students, that means we cant allow to have more than one girl to be taller than the boys because this the 5%. So if I only take 20 students I might have 2 or 3 girls taller so here the effect of chance has exceeded 5%, so why did this happen? Because the sample is very small. But what happens when I take the whole hall (maybe 300 people), the females who are taller than males might not exceed 5%, what if I take the whole Jordanian population? I will get something
Research Method Page 4

more truly. So all the time think about increasing the sample. So the always say that the minimum of a sample is 30, this number is a critical number to get good results, but it also depends on the type of research you are trying to do. Bias distorts the situation in one direction or another. But it cant go in both directions, the bias takes the data to the right or the left but doesnt take the data to the right and left in the same percentage. In chance random variation results in an observation above the true value as likely as 1 below the true value, so the percentage of observations above the true value should equal to the percentage below the true value. The mean of many unbiased observations of a sample approximates the true observation of the population; this is a very important point, because we said: in chance there is a possibility for some people to have higher data as likely as some people having lower data, and because of that we cancel them out they cancel each other out, but in bias the result depart systematically from the true value. In small samples this may not be close to the true observation of the population, as we said: when the sample is very small you cant guarantee that those above the true value should be equal to those below it, but as you enlarge it, it is called the normal variation of people. Just like tossing a coin, when you toss it 10,000 times, of course you may have occasions where the heads are more than tails and vice versa, so they cancel each other out, but when you toss it only 10 times, you cant guarantee that.

Bias vs. Chance

Bias can be prevented by proper conduction of clinical investigations (this is selection and measurement bias), and it can be corrected through proper data analysis (this is confounding bias). Chance cannot be eliminated (you can eliminate bias but not chance imp), you cant kill all females that are taller than males, we cant eliminate that, it is in our observation, but we can reduce the effect of it by taking a large sample.
Research Method Page 5

 Its influence can be reduced by proper research design. Statistics can be used to estimate the probability of chance or random variation. Because chance can be measured, we have statistical tests that can be employed to estimate the probability of chance; it is called the P-Value (Probability Value) and it is the estimation of the effect of chance, all the time it shouldnt exceed 5%. Sometimes in your research you can define your P-Value to 1%, example: in case of studying a drug that causes very bad side effects, so we dont want the 5%, we only allow 1%. In order for me to say that males are taller than females, the percentage of females that are taller than males shouldnt exceed 5%, if they do, then Im not allowed to say that males are taller than females. If I did a study and I discovered that females who are taller than males make up 10%, in this case I say: there was no statistically significant difference in height between males and females, but if I found out that females who are taller than males are less than 5%, I can confidently say: there was statistically significant difference in height between males and females, and because of that, males are taller than females. Relationship between bias & chance
True BP Intra-arterial canula BP measurement Sphygmomanometer

chance

bias

80
Research Method

90
Page 6

This is a very nice example about Blood Pressure; the true BP is measured directly by inserting an Intra-arterial canula, but it is not really possible all the time, so we use the more common way of measuring BP via Sphygmomanometer, because of not selecting the proper way of measuring BP, we ended up with what we call measurement bias, now the true BP should be 80, and because of measurement bias it departed systematically and became 90, so because it became so, it goes to one side this is bias, but the effect of chance is really different, it is now 90 and we have data above it and below it, and the percentage of those data should be equal all the time when your sample is very big, and you cant guarantee that if your sample is small. So the effect of chance is on either side of the value, but bias departs the reading to one side. Systematic error vs. random error Now systematic error results because of bias, while random error results because of chance. All the time we want random error, but we dont want systematic error, if you have a big systematic error in your research, it is considered as rubbish and you have to re-do it, but if you have a random error, thats fine, we accept it in researches, because it is a result of chance, we cant avoid it but we can reduce it. Systematic error: biases pushing the values of separate measurements away from the true value, and remains systematically different no matter how many times the measurement is repeated. Now you did a study on dental caries, and you didnt define age groups, this is systematic error selection bias. If you repeat the study again, you will have the same error again if you dont modify the design of your research. Random error: even distribution about the true value. If you have a high percentage of random error, you can repeat your study again and you may have something different.
Research Method Page 7

For example: you have 20 people to measure who is taller, males or females, and I had a result saying that the percentage of females that are taller than males equals 10%, it is not accepted Im not convinced, I can re-do it by taking another 20 people, and I might have a percentage that is less than 5%. So if you repeat the study you can have different data different random error, and when you enlarge the sample the effect of those are above and below the reading so they cancel each other out and you can get the actual reading.

Now we will talk about Validity

Its defined as the truth; your results in any study should be valid true. Validity is correspondence to the true value measured or searched for. For an observation to be valid it must be neither biased nor incorrect due to chance, no bias and the effect of chance is not that big. Types of validity: Internal validity: the degree to which the results of a study are correct for the patients being studied. Example: you take one sample, and you want to measure the height of people in that sample, assuming that the actual height is 1.65 cm, but you get something like 1.75 cm, this is a problem in internal validity, because your reading is not very similar to the actual reading for that sample. It is internal because it applies to the conditions of the particular group of people being observed and not to others, it doesnt apply to another sample, so you want your data to be true for the sample you studied. Is determined by how well the design, data collection and analyses are conducted and threatened by biases and random variation, we dont want for this threatening to be big the enemies of any researcher are: bias and chance, bias can be avoided, chance can be reduced. Necessary but not sufficient by itself, it is not enough, Im not only interested in having the results true in the group Im studying, I also want this result to be true if I studied another group as well.
Research Method Page 8

 External validity(generalizability): Is the degree to which the results of an observation hold true in other setting I want my results to be generalized, which means if I take another sample; the results should be the same. It is the answer of: Assuming that the results are true in other settings; do they apply to my patients as well? I dont want those results to be true only on people who are studied; I want it to be true on my patients too applied to another sample. Generalizability assumes that patients in a study are similar to other patients. A study with high internal validity may be misleading if its results are generalized to the wrong patients. In the figure below, it is a summary of what we mean by internal and external validity. The violet color (on right) is your sample, you have conclusion. This is actually the internal validity, because the conclusions are true for your sample, but its not enough, you want your conclusions to be true on another sample, another group of patients and the whole population Generalizability. If you take a sample, you may have a selection bias, because you didnt include both groups didnt define your sample from the beginning, and after doing the measurement you might have measurement bias or confounding bias. The effect of chance is also important so you have to think about it.

Research Method

Page 9

All patients with condition of interest

Internal validity

sample

sample

patients

Selection A B bias
Measurement & confounding bias

population
?

chance

External validity Generalizability

Conclusion

Research Method

Page 10