Stefan Stieger
The effectiveness of the High Hurdle Technique: Do artificially increased loading times enhance data quality?
Authors
Stefan Stieger (Medical University of Vienna (A))
Anja Göritz (University of Erlangen-Nürnberg (D))
Two Web-based experiments examined the usefulness of artificially delaying the loading of the first study page. The hope attached to this technique is to filter out less-motivated respondents through a higher respondent burden in the form of waiting time. Participants who remain in the study despite having had to wait for the first study page to appear on the screen are expected to be more highly motivated and thus to produce data of higher quality.
In both experiments, as expected, the longer the loading time, the lower the likelihood of people responding to the study. However, contrary to expectation, the dropout rate and quality of data were independent of the loading time. Therefore, artificially delaying the loading of the first study page is counterproductive.
|
|
|
Olivier Meyenhofer
Contamination in Web surveys: How response options can introduce subjective biases
Authors
Olivier Meyenhofer (University of Zurich)
Ulf-Dietrich Reips (University of Zurich)
In the following study two experiments are presented that examine context effects in Internet-based surveys. The two context effects under examination are the frequency scale effect, that is the effect response alternatives may have on respondents’ behavioral reports and the anchor effect, that is the effect previous questions and the answers given to them may have on subsequent judgments.
In the first experiment the two context effects were found with a sample of 352 ski tourers. An interaction effect of two frequency scale effects could not be confirmed.
The second experiment was conducted with 730 rock climbers and revealed a significant increase of the anchor effect when questions are presented on one single screen as opposed to the presentation of one question per page. Other format variations such as drop-down-menus versus radio buttons showed no influence on either of the two context effects.
Implications are discussed and recommendations for gathering self-reports in Internet-based surveys are given.
|
|
|
Mirta Galesic
Response time measurements in the lab and on the Web: A comparison
Authors
Mirta Galesic (Max Planck Institute, Berlin (D))
Ulf-Dietrich Reips (University of Zurich)
Lars Kaczmirek (Center for Survey Research and Methodology, Mannheim (D))
More and more psychologists use the Internet to conduct their studies. Oftentimes, they adjust the experiments and questionnaires traditionally applied in the lab setting for online use. A frequently used dependent variable in many areas of psychology is response time. A natural question is to what extent we can use the Internet to collect such data. Although most researchers agree that the error rate is higher on the Internet than in the lab, there is not much data on exactly how much higher the error is.
An especially popular way of collecting response time data online is by implementing javascript in the experiment’s Web page. Javascript is easy to implement and works on most computers without special software or adjustments. In this study, we are comparing response time measurements produced by the javascript, both on the Internet and in the lab, with the reaction times from a widely used software for laboratory psychological experiments, E-Prime. The study should help the psychological community in understanding which studies are feasible for implementation online, and how would the timing results compare with findings from their labs.
|
|
|
Frederik Funke
New types of measurement in Internet-based research
Authors
Frederik Funke (University of Zurich)
Ulf-Dietrich Reips (University of Zurich)
Two different kinds of Internet-based research are presented: The utilization of the established but rarely used visual analogue scale (VAS) for surveying large samples and the application of collecting data with dynamic answer formats.
VAS are widely used in assessing subjective phenomena. In a paper and pencil environment, this way of collecting data takes up a great deal of time as values have to be read out manually - time that is saved in computer-based administration of VAS. We conducted two Web experiments. The combination of both findings: There is strong evidence that data collected with VAS are equidistant and on the level of an interval scale, while categorical scales produce ordinal data only.
But Internet-based research can be more than just a carbon copy of paper and pencil surveys. A new technical approach (dynamic forms) offers the possibility of reacting to the respondent’s choice immediately. In experiment 3 we managed to reduce the complexity of open-ended please-specify-if-yes-questions by dynamically showing text fields for specifying answers only if the person questioned answered with yes. In experiment 4 we could show that breaking down the answer process by gradually offering chunks of information lead to better data quality than traditional ways (i.e. either showing all possible answers or applying filters). Impact on response times is discussed.
|
|
|
Ulf-Dietrich Reips
Metadata from research sites and their implications for psychological research
Authors
Ulf-Dietrich Reips (University of Zurich)
The Internet provides Psychology with a unique opportunity for easy investigation of behavior by mining data from online interactions, in the traditions of archival research. On a higher level of abstraction, via the same means it is now possible to easily conduct research on the scientific processes in psychological research conducted via the Internet.
The recent increase in use of Internet-based research methods is reflected in the strong use of Web services that assist in scientific activities. For example, WEXTOR (http://psych-wextor.unizh.ch/wextor/en/) is an experiment generator that is used by about 1400 researchers and teachers of experimental design. Data from this tool provide us, for example, with a frequency distribution of experimental design types and with information about the use of research techniques. The web experiment list at http://genpsylab-wexlist.unizh.ch/ is a recruitment portal for participants and lists more than 700 Internet-based studies. Results from a survey among Web experimenters are compared with data from the web experiment list to answer questions like „Which areas of Psychology most frequently use Internet-based research methods? Do we see a change in distribution over time?“
|