An undergraduate training in statistics tells us that sequential testing with optional stopping inflates the false positive rate, but exactly how much it inflates it is less intuitive. Sometimes, the metaphor of rolling a 20-sided dice every time a statistical test is run (assuming alpha = 0.05). However, whereas each roll of the dice is independent, running a new instance of a test on a slightly data set after adding a number of participants is not fully independent.

Ed Hagan recently wrote a piece titled “Academic success is either a crapshoot or a scam” in which he uses some simple maths about about the rate at which research is done to argue that “empirical social scientists with lots of pubs in prestigious journals are either very lucky, or they are p-hacking.” That is, regardless of intentions, a situation where research practices are inflating the false-discovery rate. I have no doubt that p-hacking occurs and is actively rewarded by our current incentive structures, but I’d nonetheless like to argue for an additional possibility: fabricated interest.

Most experiments using reaction time data separate scoring and analysis. That is, the study captures a large number of reaction times for each participant, then reduces this down to a smaller number of metrics (e.g., mean reaction time, D scores, etc), and then analyses these metrics. In doing so, we can throw away the variance associated with each participant’s performance and unnecessarily decrease the power of our analysis. Mixed effects modelling provides an accessible alternative that does away with data scoring.

Research questions What age would people chose to be if given the choice? How old is “old”? Do opinions about ‘how old is old’ change with one’s own age? Open data This data was collected on Project Implicit and is freely available under a CCO license on the Open Science Framework. Specifically, I took the data from the Age related Implicit Association Test collected in 2015. # dependencies ------------------------------------------------------------ library(tidyverse) library(psych) library(knitr) library(broom) data_trimmed <- read_rds("data/data_trimmed.

Perhaps it’s just my own learning curve, but it feels like internet resources are making it easier and easier to conduct and communicate research. LimeSurvey does away with the need for paper questionnaires, PsychoPy allows me to write training and testing procedures in a matter of hours, Google docs allows me to write collaboratively, Zotero references the manuscripts for me, Recite checks them for errors, ResearchGate takes care of distribution, and Dropbox covers my ass incase anything goes wrong.

Lorem ipsum dolor sit amet, consectetur adipiscing elit. Duis posuere tellus ac convallis placerat. Proin tincidunt magna sed ex sollicitudin condimentum.

Lorem ipsum dolor sit amet, consectetur adipiscing elit. Duis posuere tellus ac convallis placerat. Proin tincidunt magna sed ex sollicitudin condimentum.

Lorem ipsum dolor sit amet, consectetur adipiscing elit. Duis posuere tellus ac convallis placerat. Proin tincidunt magna sed ex sollicitudin condimentum.