You really have been learning with portals.
Val Shute studies the psychology of education at Florida State University. She also loves Portal 2 – a mind-bending, physics-based adventure from Valve Corporation, the creators of Half-Life, Counter-Strike, and Left 4 Dead. So it was natural for her to want to design a study featuring the game. But to see if the problem solving in Portal 2 was really paying off, she would need a comparison. Shute and her team chose Lumosity, a popular “brain-training†software.
“Portal 2 kicks Lumosity’s ass,” Shute told Popular Science.
Reporting in a study in the journal Computers & Education late last month, Shute and her colleagues pitted Lumosity against Portal 2 in a study designed to find out whether extended periods of play of either game would produce cognitive benefits. More specifically, Shute wanted to see if test results on problem solving, spatial skill, and “persistence†questions would increase after gaming in either situation, and see if those results were significantly different from each other.
77 college undergraduates were recruited into the study and were randomly assigned to either play a total of eight hours of Portal 2 or Lumosity (spread across four sessions). To compare the results, Shute had the students complete the problem solving, spatial skills, and persistence questionnaires both before and after gaming.
In every way, Portal 2 players improved more on the cognitive tests than Lumosity players. Those that completed the brain training games even did slightly worse on the cognitive ability questions than they did before trying Lumosity.
I think we can put our differences behind us… for science… you monster.
The Game Is A Lie
Before you pull your kids out of school to try and figure out those darn light bridge levels with you, there are some caveats to Shute’s study. First, the sample size was small. For the relatively minor increases in cognitive performance the study found, it’s hard to put real weight behind those conclusions with only 77 people tested. Second, the study session wasn’t very long. It’s possible that the supposed benefits of Lumosity come after eight hours of play, while you could finish all of Portal 2 in that time.
And I wonder if the low results from Lumosity were from the other student participants knowing that they weren’t getting to play one of the best games of the last decade.
But Shute’s research corroborates what we’ve already found in gaming-related science in two important ways: video games do seem to have positive effects on the brain, and so-called “brain training†games don’t work.
For all the flashy marketing, the claims of brain training regimens like Lumosity have not lived up to the hype. The research that we have doesn’t back up brain training in general and Lumosity in particular. The big business of brain games is quickly bounding over and beyond what the science says.
“Contrary to media claims that Lumosity improves specific problem solving skills,†Shute and colleagues conclude, “[the results] show that while there were increases in scores from pretest to posttest for those in the Portal 2 group, all three of the problem solving test scores slightly decreased for the Lumosity group, from pretest to posttest.â€
Aside from providing another nail for brain training’s coffin, Shute’s Portal 2 study does point out something interesting – if playing a problem-solving game actually can increase your cognitive performance in some way, what are brain training games missing, and what is Portal 2 getting so right? Is it a combination of puzzles with humor and gorgeous visuals? Is it immersion or extended periods of play? Those are questions for another study to answer.
I’m just GLaD I can make a note here: Huge success.
—
Kyle Hill is the Chief Science Officer of the Nerdist enterprise. Follow on Twitter @Sci_Phile.
IMAGES: Screenshot from the game Portal 2; Valve Corporation
Only thing Portal 2 doesn’t do: raise your self esteem.
We want want we don’t have. It makes us feel like we aren’t good enough, until you win, and laugh at petty GlaDos, the one who called you stupid. If it was nice, it wouldn’t be so addictive. Your drive would run out.
It’s 4 o’clock in Florida now shouldn’t they all be eating dinner right now? Then of course … nap time!
Study done in Florida? enough said.
Next up Antichamber vs lumosity
I think they should give them the test again, like, two days later, for a total of three tests. Perhaps the luminosity players we’re just tired of tests, and mentally exhausted after eight hours of luminosity? And, perhaps Portal give you a short-term increase in cognitive ability, which wanes over time? Not saying that these things are true, just that I feel they should be part of the experiment.
So, do it again, with a larger sample size, with the three test system. I’d be very interested to see the results.
In this article it says that 77 is not a large enough sample size. However, the journal article does state that most of its results were significant (in the statistical sense of the term). All a larger sample size would do is increase that significance.
Well I was only able to read the abstract but it said nothing about the confidence of the results. Increasing sample size would increase your confidence that these two samples are different (significant) from one another, but would not necessarily make the results any more significant. You would simply approach the true population value as sample size increased.
I’ve played both. Portal 2 does address problem-solving, but does nothig in other areas like math skill building. More research is needed. Where is my controller?
Programs like Lumosity build cognitive skills over the course of months, not hours. This study is flawed.
They acknowledge that the study was too short
“Iām just GLaD I can make a note here: Huge success.” That pun is the worst I’ve heard in a long while. So bad that if I saw you on the street I would high-five the crap out of you for it sir, lol.
The first thing I’d like to know is if this study is peer-reviewed or not.
After that, there are some major caveats to even this article’s caveats…
Lumosity doesn’t purport to be a short-term boost. It was designed to be a long-term cognitive maintenance and development program. So the study’s choice of Lumosity is misguided. There are plenty of other “brain boost” programs out there.
There’s also a boredom factor at play here… Portal 2 is a fully-developed video game, designed to be played for hours on end. Lumosity is a small collection of different minigames that take about one-two minutes to play apiece. That would bore anybody to tears.
If this is a pilot study, then okay, let’s see something more comprehensive. Mission accomplished. But if it purports to be entirely conclusive (I’m not paying $25 to read it), then this isn’t really viable science.
Lumosity, not Luminosity. š
Lumosity, not Luminosity.