Assignment: Discuss The Validity of Survey Data

Assignment: Discuss The Validity of Survey Data

Collecting survey data is hard work. It means constantly seeking subjects and dealing with lots of extraneous sources of variance that are difficult to control. It is somewhat of a surprise, however, how relatively easy it is to establish the validity of such data. For example, one way to establish the validity of the data gained from an interview is to seek an alternative source for confirmation. Public records are easy to check to confirm such facts as age and party affiliation. Respondents can even be interviewed again to confirm the veracity of what they said the first time. There is no reason why people could not lie twice, but a good researcher is aware of that possibility and tries to confirm factual information that might be important to the study’s purpose.

Table 9.1 An example of how data can be collected and scored in a survey setting

Physical Punishment Is Cruel and IneffectivePhysical Punishment Is Harsh and UnnecessaryPhysical Punishment Can Work Under Certain ConditionsPhysical Punishment Is a Useful Deterrent for Poor BehaviorPhysical Punishment Is the Most Effective Method for Dealing with Poor BehaviorParents Who Use Punishment1214152332Parents Who Don’t Use Punishment461314 7 6

Evaluating Survey Research

Like all other research methods, survey research has its ups and downs. Here are some ups. First, survey research allows the researcher to get a very broad picture of whatever is being studied. If sampling is done properly, it is not hard to generalize to millions of people, as is done on a regular basis with campaign polling and such. Along with such powers to generalize comes a big savings in money and time.Second, survey research is efficient in that the data collection part of the study is finished after one contact is made with respondents and the information is collected. Also, minimal facilities are required. In some cases, just a clipboard and a questionnaire is enough to collect data.Third, if done properly and with minimal sampling error, surveys can yield remarkably accurate results.The downs can be serious. Most important are sources of bias which can arise during interviews and questionnaires. Interviewer bias occurs when the interviewer subtly biases the respondent to respond in one way or another. This bias might take place, for example, if the interviewer encourages (even in the most inadvertent fashion) approval or disapproval of a response by a smile, a frown, looking away, or some other action.On the other hand, the interviewee might respond with a bias because he or she may not want to give anything other than socially acceptable responses. After all, how many people would respond with a definite “yes!” to the question, “Do you beat your spouse?”These threats of bias must be guarded against by carefully training interviewers to be objective and by ensuring that the questions neither lead nor put respondents in a position where few alternatives are open.Another problem with survey research is that people may not respond, as in the case of a mail survey. Is this a big deal? It sure can be. Nonresponders might constitute a qualitatively distinct group from responders. Therefore, findings based on nonresponders will be different than if the entire group had been considered. The rule? Go back and try to get those who didn’t respond the first time.

TEST YOURSELF

You read about ethics and some guidelines in Chapter 3B. What might be some conflicts that can arise with those ethical principles and the use of the various survey methods we discussed earlier?

Correlational Research

Correlational research describes the linear relationship between two or more variables without any hint of attributing the effect of one variable on another. As a descriptive technique, it is very powerful because this method indicates whether variables (such as number of hours of studying and test score) share something in common with each other. If they do, the two are correlated (or co-related) with one another.In Chapter 5, the correlation coefficient was used to estimate the reliability of a test. The same statistic is used here, again in a descriptive role. For example, correlations are used as the standard measure to assess the relationship between degree of family relatedness (e.g., twins, cousins, unrelated) and similarity of intelligence test scores. The higher the correlation, the higher the degree of relatedness. In such a case, you would expect that twins who are raised in the same home would have more similar IQ scores (they share more in common) than twins raised in different homes. And they do! Twins reared apart share only the same genetic endowment, whereas twins (whether monozygotic [one egg] or dizygotic [two eggs]) reared in the same home share both hereditary and environmental backgrounds.

The Relationship Between Variables

The most frequent measure used to assess degree of relatedness is the correlation coefficient, which is a numerical index that reflects the relationship between two variables. It is expressed as a number between 21.00 and 11.00, and it increases in strength as the amount of variance that one variable shares with another increases. That is, the more two things have in common (like identical twins), the more strongly related they will be to each other (which only makes sense). If you share common interests with someone, it is more likely that your activities will be related than if you compared yourself with someone with whom you have nothing in common.For example, you are more likely to find a stronger relationship between scores on a manual dexterity test and a test of eye–hand coordination than between a manual dexterity test and a person’s height. Similarly, you would expect the correlation between reading and mathematics scores to be stronger than that between reading and physical strength. This is because performances on reading and math tests share something in common with each other (intellectual and problem-solving skills, for example) than a reading test and, say, weight-lifting performance.Correlations can be direct or positive, meaning that as one variable changes in value, the other changes in the same direction, such as the relationship between the number of hours you study and your grade on an exam. Generally, the more you study, the better your grade will be. Likewise, the less you study, the worse your grade will be. Notice that the word “positive” is sometimes interpreted as being synonymous with “good.” Not so here. For example, there is a negative correlation between the amount of time parents spend with their children and the child’s level of involvement with juvenile authorities. Bad? Not at all.Positive correlations are not “good” and negative ones are not “bad.” Positive and negative have to do with the direction of the relationship and nothing else.Correlations can also reflect an indirect or negative relationship, meaning that as one variable changes in value in one direction, the other changes in the opposite direction, such as the relationship between the speed at which you go through multiple-choice items and your score on the test. Generally, the faster you go, the lower your score; the slower you go, the higher your score. Do not interpret this to mean that if you slow down, you will be smarter. Things do not work like that, which further exemplifies why correlations are not causal. What it means is that, for a specific set of students, there is a negative correlation between test-taking time and total score. Because it is a group statistic, it is difficult to conclude anything about individual performance and impossible to attribute causality.The two types of correlations we just discussed are summarized in Table 9.2.Interestingly, the important quality of a correlation coefficient is not its sign, but its absolute value. A correlation of 2.78 is stronger than a correlation of 1.68, just as a correlation of 1.56 is weaker than a correlation of 2.60.