Font Size: 

Numerous studies in mass communication research rely on data provided by student convenience samples. Especially when conducting experiments, researchers rely on the appropriateness of convenience sampling due to the focus on the high internal validity of the experimental setting. Although the decision whether further research is done or not is often based on studies using convenient samples, their appropriateness seems to be discussed only marginally. The paper on hand focuses on this problem. It is shown that the experimental design itself may endanger its internal validity. Participants’ motivational states as well as their perceived knowledge of the research purpose influence their answering behavior. As these factors can sparsely be controlled in advance, it depends on the structure of the convenience sample whether it is likely to confirm a research hypothesis or not.
We conducted a 2x2x2 experiment using a cover-survey on the subject of cultivation research. Hereby we tested the influence of college education level, knowledge in communication science and specific awareness of the research hypothesis on motivational states as well as the answering behavior of participants with regard to the surveys internal validity.
The data suggests that students from different terms and different fields of study do not differ in their motivational states but show noticeable differences in their answering behavior. Especially students with expertise in communication research show a significant deviant answering pattern from the other subjects and the least cultivation effects overall. Concerning hypothesis awareness, all subject groups were found to correct themselves towards a direction against the cultivation hypothesis.
The findings support that the structure of a student convenience sample has no influence on participants’ motivational states. But indeed it makes a difference in answering behavior. Although freshmen students showed greater cultivation effects, it seems questionable to recruit only this student group for experimental research. Experimental designs do not reach to carry the output of the experiment on the total population but to find proof for a causal relationship. Even more, the pertinence of this special kind of sample should not only be measured by the successful causal proof. This study shows that concentration on student convenience samples does not guarantee high internal validity. Instead, researchers have to further increase internal validity by enhancing the homogeneity of their sample as well as they have to conduct follow-up studies – regardless of their original study’s result.