Imagine for a moment, for the sake of science, that I am standing in front of you with Mike Tyson, the legendary boxer, next to me.
The next thing you know, me and Tyson are taking turns punching you, again, for the sake of science. After the two of us punch you one time each, you are asked to rate the pain you feel on a scale of 1-10, 1 being no pain and 10 being the most intense pain imaginable.
You rate my punch as a 3 and Tyson’s punch as a 9. Fair enough, you have gone through enough for the sake of science, but what do these numbers mean?
It means that you found Tyson’s punch more painful than mine but would it be fair to say that Tyson’s punch is precisely three times more painful than mine? Would it make sense if I said that three punches from me will hurt the same as one punch from Tyson?
That said, would you rather face three punches from me or one from Tyson?
Is pain really additive? Can I add the pain of spraining my ankle and the pain of being hit by a hammer to come to a collective pain score?
Psychology, especially psychometrics, works on the assumption that human experiences can be codified into distinct variables such as pain, shame, fear, guilt, and wellness and that these variables can be expressed in numbers.
But can they? Let’s discuss
The Problem of Child Birth
In the example above I asked you to compare the pain you felt after being punched to the ‘most intense pain imaginable’ but what does that standard mean? We don’t all have a universal pain imaginable or experienced do we? And even if we did, do you think imagining some pain is the same as experiencing it?
In a study by Bartoshuk (2014), the researcher found that women who had been through childbirth rated the same painful stimulus as being 20% less painful than men. So the same amount of external pain was perceived as being less painful in people who had been through something much more painful.
There is a very basic assumption in quantitative psychology that the experiences of one person can be compared to those of another and this difference can be understood using numbers.
But the most important question is, “Are psychological phenomena really quantifiable?”
Orders Don’t Mean Intervals
When you were punched by Mike Tyson and me a few moments back, it was clear to you that one punch hurt more than the other but does that also mean that this order can be converted into meaningful intervals?
You must have filled a 5-point Likert Scale to help a friend with their practical or dissertation. The Likert scale gives you two extreme options and intervals in the middle to place your own experience on that scale.
For example, I enjoy spending time on Instagram
Not at all——somewhat——neutral——Most of the time——All of the time
Most of the time, we just select whichever option makes sense to us at the moment and get done with it but that is not how these scales are supposed to be filled. These scales ideally require deep introspection from the participant before filling them.
“What does being neutral about enjoying something really mean?”
“Is my enjoyment always there or does it come rarely?”
“Do I ‘enjoy’ my time there, or do I merely ‘spend’ it?”
We don’t give these scales the time they deserve and end up making some really big claims from the data collected using these scales.
Psychology is full of studies which claim quite huge things (racial differences in intelligence, gender differences in multitasking tasks etc) because they perform statistical analysis and get a result where the p < 0.05 but is that enough to make a phenomenon real? What if it were at p = 0.051. Does that mean the phenomenon is so close to being real/statistically significant but is not?
Can we really use scales to understand the deep-seated mind and its intricacies? I don’t think so.
Why Do We Like Numbers?
Psychologists, me included, have always had a fascination with numbers. We think it makes the world simpler and easier to analyse and science easier to perform.
We are, obviously, secretly jealous of physics as well because of how well it works with numbers and how broad its applicability is.
Imagine throwing a ball on the floor and waiting for it to bounce back to your hand.
Numbers can be associated with every single aspect of the ball. The radius of the ball, its velocity when it drops, its velocity when it reaches the floor, how long it takes to get there, how long it takes to get back up and at what speed it comes back up. All these questions can be answered with neat, mathematical laws.
But humans don’t work like that. No matter how much we want them to.
Statistical analyses may make sense when it comes to involuntary situations or actions that humans do, such as eye movement or pupil dilation but to think that the attitudes of people can be codified into numbers and then compared is deeply flawed.
Does that mean quantitative psychology is useless?
Well of course not. I have been a big fan of psychometrics ever since I was introduced to it. I even wanted to become one in my undergraduate days.
What I am trying to say is that, much like neuropsychology, it is not the be-all, end-all of psychology. The human experience, the human mind and the phenomenon of consciousness cannot be limited to some numbers, especially if the numbers are to be used for comparison among people.
We can all be better by being humble about our ignorance :)
Bonus: The Spectre of Stats
I want to take a moment to shift from the talk on quant psychology to focus on Indian quant psychology.
If there is one thing I learned in my UG days, it’s that most of my classmates hate statistics. Some of my teachers hated it too. They didn’t even know how to teach it because they hadn’t been taught properly either.
This has led to generations of psychologists who have absolutely no idea how to use statistics and avoid them at all costs. Yet, when it comes to publishing, quantitative studies are likelier to get published. This pushes these untrained students into areas that are uncomfortable for them for the sake of publication; and the result? Low-quality research.
Students get their publications, journals claim to have found ‘empirical’ proof of something even if the methodology used is completely flawed.
Most psychology graduates in India are scared of statistical analysis and are not trained in qualitative research well enough. How then, do we expect them to become good researchers?
Post continued below
Question of The Week
What do you think about quantitative psychology? Is it simply misunderstood, or is it overrated as a method of science in the modern world?
You can let me know by replying to this email or in the comments below. I would appreciate any and all responses.
Until next time :)
Arjun