Laboratory v. field experiments The tradeoff between internal and

Laboratory v. field experiments The tradeoff between internal and external validity

Natural sciences In natural sciences, most experiments have been undertaken in carefully controlled laboratory conditions – Seeking laws of physics, etc. Inanimate objects are unlikely to react differently in controlled and uncontrolled conditions The main goal is to avoid contaminating the experiment via the introduction of third variables Far more exacting and finer measurement is possible under controlled conditions

Laboratory experiments in social science Closely controlled social science experiments do have the advantage of limiting the impact of third variables, but the unnatural situation presents problems for applying the findings outside the laboratory – “Artificiality” of laboratory setting

Human behavior is sensitive to the environment within which it occurs – People act differently in a laboratory than in the natural world Several characteristics of laboratories are thought to influence behavior The very third variables controlled in the lab may be the ones that determine behavior in the real world So, findings from laboratory experiments may only be valid for laboratory environments

An example New commercials are tested in controlled conditions – Eye tracking – Liking for commercials – Influence on purchase interest May try to provide less artificial conditions for study – Simulated living room Commercials that test high in lab experiments often do not work very well when used in real marketing campaigns

So, experiments move out of the lab: Researchers want to retain some of the advantages of the experiment: – Ability to manipulate/introduce the independent variable and to control how much of it is presented – Time order—which comes first While sacrificing some of their ability to control third variables The goal is to improve our ability to generalize our findings to the real world

The field experiment One way to do so is to carry out a ‘field experiment’ – The researcher still manipulates the independent variable, but she does so within the natural world – For example, police try randomly stopping motorists and testing for alcohol to see if alcoholrelated crashes decline

Information Resources, Inc. BehaviorScan system

What problems do we encounter? Greatly reduced ability to prevent third variable contamination of results – The crusading doctor in MHHP – History effects Cycles What can you do? – Measure/monitor likely alternative explanatory variables – Question subjects about sources of influence – Multiple manipulations of the independent variable over time – Multiple measurement waves – Multiple dependent measures None will be perfect

Expense – Tradeoff between extensive and intensive study – Budget constraints on number of sites, etc. Access/permission – Some research may present concerns to authorities, citizens, etc. Gain authorization/support prior to entering the field Maintain good relationships with community leaders, etc. throughout the intervention/research

Capturing ‘natural experiments’ Sometimes unusual or unique events occur – ALAR scare – Service disruptions – Political campaigns – School shooting – Institution of Wi-Fi in an area

“Natural experiments” Because most such events are unplanned, the ability to prepare for them is limited – May keep a research group, materials and resources ready for certain types of events Must engage in ‘firehouse research’ gathering as much data as possible in a short time – Inefficient, and may miss important data – However, real-world events, etc. may provide very valuable data—may have both internal and external validity

Use statistics, research analysis models that are appropriate to experimental studies