Conducting Online Research on Amazon Mechanical Turk and Beyond

Chapter 1: Introduction A Scientific Revolution in the Making A Brief History of Online Research in the Social and Behavioral Sciences: From HTML 2.0 to Mechanical Turk The Use of Online Samples in Applied Behavioral Research Amazon Mechanical Turk Chapter 2: The Mechanical Turk Ecosystem - Leib Litman, Cheskie Rosenzweig, Jonathan Robinson How Quality is Maintained Reputation Mechanism Selectively Recruiting Specific Workers Protections for Workers Communicating with Workers A Worker's Perspective Worker Communities Chapter 3: Conducting a Study on Mechanical Turk Sample Project Setting up a Requester Account on Mechanical Turk Creating a HIT The 'Design Layout' tab Monitoring Progress on the Requester's Dashboard When a Worker Runs Out of Time Sample Study Results Conducting Follow-up Studies Using Requester-Issued Qualifications Appendix A: Checklist for best practices of setting up a Mechanical Turk HIT Chapter 4: API and Third Party Apps Third Party API-based Platforms Common Uses for API Scripts and Third Party API-based Apps TurkPrime Chapter 5: Data Quality Issues on MTurk - Jesse Chandler, Gabriele Paolacci, David Hauser Defining and Measuring Data Quality Measuring Individual Participant Data Quality Causes of and Cures for Poor Data Quality Concluding Thoughts Chapter 6: Who are the Mechanical Turk Workers? Sources of Data Location of Workers in the US Demographics of Mechanical Turk Chapter 7: Sampling Mechanical Turk Workers: Problems and Solutions Sampling on Mechanical Turk Sources of Sampling Bias The Problem of Superworkers Time-of-day Effects Pay Rate Dropout Sampling Best Practices Chapter 8: Data Representativeness of Mechanical Turk Samples Representativeness, surveys, and survey sampling The methodology of survey sampling Mechanical Turk as a sampling frame The fit-for-purpose framework Chapter Overview: Chapter 9: Conducting Longitudinal Research on Amazon Mechanical Turk Why Longitudinal Research? Retention, Longitudinal Research, and MTurk Case Studies Best practices for longitudinal research Chapter 10: Beyond Mechanical Turk: Using Online Market Research Platforms Limitations of MTurk Online probability-based panels Online market research platforms Overall comparisons between Mechanical Turk and market research platforms Chapter 11: Conducting Ethical Online Research: A Data-Driven Approach Historical background Risk of harm in online research Research on sensitive topics A deeper dive into controversial and complex issues Economics of Mechanical Turk: Considerations for setting wages Setting wages: Considerations of ethics and methodology Considerations for rejecting, blocking and disqualifying workers. Practical advice for requester/worker interactions Anonymity Appendix A