Survey Experiment: a promising methodological approach

The survey experiment is a very promising methodological approach in scientific research. In this article, which will probably be divided into two parts, I want to summarize in a very objective way what this methodological approach is and why it is so important in the field of experiments. In a second step, I intend to present some concrete examples and the possibilities in the field of ethics, demonstrating how this approach can assist the researcher in his experiments.

According to researchers Yanna Krupnikov and Blake Findley, the survey experiment is under the umbrella of experimental research as well as other methodological approaches.

The survey experiment incorporates basic aspects of a “common research” with experimental interventions that together enable researchers to both generalize a survey and not lose control of an experiment. Thus, they combine survey and experimental research, thus becoming a true ally for the researcher.

The general idea is to assign respondents to a control group and a treatment group at random, manipulating some aspects of the research protocol. Thus, what is sought is to determine if there was any variation in the stimulus proposed in the treatment group.

To get a better idea, researchers Steven Nock and Thomas Guterbock describe that some familiar research experiments study variations in the formulation of questions, in the order of questions or in the response scales. Survey experiment can also involve intentional variation in sampling methods, recruitment tools, incentives to participate, modes of data collection, interviewer actions or even aspects of the research context, including interview location or interviewer characteristics.

Note that several experiments using research questionnaires may not be survey experiment. We can think of a questionnaire that measures the traffic attitudes of a specific intervention group and another control group. Then, educational materials related to the reduction of traffic accidents are exposed to the intervention group to reduce negative attitudes in traffic. In this case, what happens is that research is only a measurement tool and not experimental manipulation.

The survey experiment can be divided and used in two ways, namely basic methodological research, and substantive research. Basic methodological research according to Steven Nock and Thomas Guterbock is carried out to advance general theories about respondents’ cognitions or behaviors. Some substantive research is applied, for example, to survey experiments on how best to write public messages for a defense group or political campaign.

Unlike the most common experiment approach that includes pre-test and post-test measurements, the survey experiment does not include pre-test measurement. What is more, you also do not need a large probability-based sample. Steven Nock and Thomas Guterbock describe that survey experiment that use better samples will be more generalizable, however a sample for convenience is entirely feasible.

One of the techniques most used in the research experiment is known as vignettes, or even factorial vignettes when manipulated and analyzed with the statistical method of factor analysis. In this case, the subjects evaluate several vignettes, but the set of vignettes differ between the subjects. These experiments can be conducted both in laboratories and outdoors. When we do experiments in laboratories the costs can be better, in addition to having better control. However, field research (external environments) generates more representativeness and realism.

The possibilities and motivations for conducting research with this methodological approach are indeed promising. More substantive research benefits greatly from this approach, in addition to the statistical power of using vignettes and factor analysis, of course.

The idea here, as mentioned at the beginning of the article, was to bring some information about this methodological approach that has been widely used and shown to be promising in the field of research. I hope in a second moment to bring practical examples of research already carried out and the possibilities in the field of ethics.

References:

NOCK, S. L.; GUTERBOCK, T. M.. “Survey experiments.” In Handbook of Survey Research, eds. Marsden, P. V., and Wright, J. D.. Emerald, UK, 837–64, 2010.78

KRUPNIKOV, Y.; FINDLEY, B. Survey Experiments: Managing the Methodological Costs and Benefits. Rae Atkeson, L. and Alvarez, RM (edt.): The Oxford Handbook of Polling and Survey Methods. Oxford University Press, 2016.

Back To Top