4 Surprising Lessons from Running a Giant Study on IQ

Key Takeaways

• High IQ doesn't equal perfectionism. Despite cultural tropes like the “genius perfectionist,” the correlation we found between IQ and perfectionism was effectively zero.

• IQ relates more to high school performance than college. IQ scores were moderately correlated with high-school GPA but barely correlated with college GPA.

• People with higher IQ scores are less confident in solving big problems. Higher IQ scores were negatively correlated with self-reported confidence in solving societal issues.

• IQ scores are just part of the story. Knowing someone’s IQ score can warrant adjusting your expectations in a variety of ways (some small, some moderate), but it can’t justify definitive judgments - for that, you need to get to know the person

• Our Cognitive Assessment offers a deeper view than traditional IQ tests. Built from data on thousands of participants, it measures seven key areas of intelligence, giving you a more complete picture of your cognitive strengths and areas of improvement.

What do you think your IQ score says about you? 

Looking around the internet, or through the horrifying history of how IQ has been used in the past, it’s easy to see a wide range of answers. At the extreme ends, some claim that IQ scores are nonsense and mean nothing at all, while others claim your score can determine your very worth as a human being.

Fortunately, modern researchers are generally much less interested in using IQ to make moral claims (and we too certainly do not endorse any moralizing of IQ scores). Instead, today’s researchers spend more time wondering to what extent IQ scores are predictive of things in the real world, like:

1. Perfectionism

2. College or high-school GPA

3. Usage of difficult words

4. Reaction time

And many, many more.

We wanted to put to the test a wide array of claims about IQ made by modern academic researchers as well as by the general public, and in the usefulness (or not) of the concept of IQ. So we ran a giant study on 3691 participants, to investigate 40 such claims. The full write-up of our findings, including details of the design of our study can be  found here. 

But in this article we’re going to walk you through four of the more surprising results, and then conclude by drawing some lessons from them. If you’re curious about how IQ relates to perfectionism, GPA, vocabulary, or reaction time, we’ll cover those (and many more) below. However, if you're highly skeptical of IQ as a concept, or interested in whether it's meaningless or not, you may want to check out our previous investigation on that topic.

For the full list of 40 claims we tested, the results for each, and details about how we tested them, go to the report, here. Or read on to learn more about what surprised us in this research.

Things to keep in mind, about these results

1) Given our large sample sizes, in many cases even quite low correlations are statistically significant by conventional criteria (i.e., achieving p<0.05), meaning that they are large enough to be unlikely to occur due to random chance alone. 

2) Many of the correlations shown in this article are small or only modestly sized. When referring to the size of the correlations, we are relying on an expanded version of Cohen’s criteria for small, medium, and large effect sizes, which provides rules of thumb, but keep in mind that whether a correlation of a given size is useful depends on context. Specifically, here is the nomenclature we are using:

3) Like all IQ tests, our test has measurement noise, and so gives approximate estimates for what’s known as 'g' (i.e., ‘general intelligence’) (which is a technical term that is linked to, but not the same thing as, the everyday notion of ‘intelligence’). This means that any correlations reported in this article are around 20% smaller, on average, than they would be if we had a flawless measure. (We explain how we calculated that number in the full report.)

Now, let’s explore some of the questions and their answers that we think you might find surprising!

1. Do people with higher IQ scores tend to be more perfectionistic than average? 

When he was filming The Shining, Stanley Kubrick was so focused on getting things perfect that he required 148 takes for one close-up shot. This broke the world record for number of takes in a scene with dialogue - a record that still stands today. And when he received a letter from a filmmaking hero of his (Akira Kurosawa), he spent so long trying to craft the perfect reply - writing many, many drafts - that Kurosawa died before ever getting a response.

Kubrick is often invoked as an example of a trope you have probably encountered many times: the brilliant genius, plagued by perfectionism. Someone whose towering intellect leads them to have incredible creative vision and also an insatiable desire for perfection. It’s a common cultural archetype, and that’s what might make it surprising that perfectionism has approximately no relationship with IQ score (technically, we found a "negligibly small" negative relationship in our study).

Given our sample size for this test (n = 477, which is smaller than the full study, because we measured perfectionism only for a random sample of participants) this result isn't even statistically significantly different from 0.

Does this mean that the mythology of the genius perfectionist is nonsense? Not necessarily. Let’s use this to reflect on an important lesson about correlations: They can miss meaningful patterns in small subgroups.

Almost all elite basketball players are tall. Very tall. But they make up such a small percentage of very tall people that there’s virtually no correlation between being very tall and being a professional basketball player (in fact, the correlation is roughly zero). Let’s say that again because it's so counterintuitive: despite the fact that being an elite basketball player almost requires being very tall (save for a few truly exceptional shorter athletes), there is virtually no correlation between height and whether or not you're an elite basketball player across the whole population. All because the signal (the real relationship) is hard to find since there is such an enormous number of tall people who are not basketball players. 

In fact, this is a more general interesting fact about correlations: the correlation between any two traits is almost always very low if one of the traits is very rare (in this case, being a professional basketball player).

So, no. We haven’t completely debunked the idea of the genius perfectionist. Kubrick’s fans can rest easy. It’s possible that even if IQ score is strongly related to being a genius, and even if being a genius entails being a perfectionist (which is surely too strong a claim), there are so many non-genius perfectionists in the population that no correlation between the two traits can easily be detected. But we have at least given strong evidence that perfectionism is not a sign of having a high IQ score.

2. Does IQ matter more for performance in college than for performance in high-school?

We asked participants to report three things:

1. Their highest level of education

2. Their high-school GPA

3. Their college GPA

We found that IQ score was positively correlated with all three, but much less for college GPA than high-school GPA.

IQ score correlation with level of education: r = 0.19 (n = 3688) 

IQ score correlation with high-school GPA: r = 0.21 (n = 3173)

IQ score correlation with college GPA: r = 0.05 (n = 2386)

The correlation with high school GPA is moderate, but the correlation with college GPA is negligible. That raises the question: Why would this be the case? Why would a relationship between GPA and IQ score shrink so much when studying becomes more demanding? 

Data like these can be used to tell lots of different stories, and a key part of critical thinking is being able to spot when there are alternative explanations that fit the data. If someone (including us!) is telling you that their data support some claim or another, and it's a topic that's important to you, it’s worth asking yourself whether you can poke holes in that support. Then you can ask the Question of Evidence: “How much more likely would I be to see this result if this hypothesis were true, compared to if it were false?” The answer tells you how strongly that evidence supports the hypothesis (in fact, it's the only question that properly measures the strength of evidence). Here are some (but not all) of the stories that could be told about these results:

Story 1: IQ plays a role in early academic success, but less so in later success 

This is the interpretation of the data that most takes them at face value. On this interpretation, the stronger correlation with high school GPA suggests that IQ contributes meaningfully to performance in high school but once students enter college, other factors become more important, causing the relationship between IQ score and GPA to shrink.

Maybe the fact that college usually requires more of non-cognitive skills like self-regulation, time management, and motivation than school to succeed means that the role that IQ plays in GPA is massively reduced. If so, then this raises questions like:

• Are college assessments measuring what they should be?

• What does that mean for students who have higher IQ scores but lower amounts of support with things like structure and or self-regulation?

• Are students going from one uneven playing field to another? I.e., from a place where those who are lucky enough to have higher IQ scores will do better to a place where those who are lucky enough to have more support structures and training in life skills will do better.

Story 2: College GPA just isn’t a very reliable or standardized indicator of academic performance.

High school curricula are much more standardized than college curricula. Most school students take a relatively similar set of core classes (math, science, English, history, etc), which have more similar difficulty levels and grading conventions than college classes are likely to have. So, because school students are navigating more or less the same academic environment, differences in GPA are more likely to reflect underlying individual traits.

But college is much more variable: Students choose very different majors (in part, based on what they believe they will succeed at), each with their own grading norms (e.g., grading on a curve or inflating grades), classes, and different amounts of overlap with the thing measured by IQ scores. This means that the weak correlation between IQ and college GPA might just be a product of the inconsistency in what college GPA tracks across different contexts.

Story 3: Selection effects reduce the predictive power of IQ

Maybe looking at the set of results from college students introduces a selection effect that hides the importance of IQ scores. Here are two ways that could happen:

Range restriction: Suppose IQ scores matter for getting into college (which is why we see a positive correlation between IQ and highest level of education) but the IQ scores of people who go to college are much more similar to each other than the scores of the whole population are (college-goers and non-college-goers included). In this case, the distribution of IQ scores in the sample of college students would be less spread out (the range is restricted) and this would therefore make it harder for IQ scores in that group to correlate with anything. 

To understand intuitively why this would matter, consider the link between age (of adults) and top running speed. We know that as people age they will, on average, be able to run less quickly, so age and top running speed are negatively correlated among all adults. But suppose we ran a study measuring this correlation only on people in the age range of 50 to 51. In this case, the relationship between age and running speed will be negligible, since 51 year olds are only going to be, on average, very slightly slower than 50 year olds.  As such, we would measure a low correlation between top running speed and age in that range-restricted group, despite there being a much larger correlation across all adults because the amount that age matters for predicting running speed (relative to the size of other effects that impact running speed) is much less in the 50 to 51 year old group than it is in the whole adult population.

If something similar is happening with college IQ scores, then the negligible correlation between IQ scores and college GPA might look like a sign that IQ score has little relevance, but it could instead simply reflect the fact that IQ score has already had its influence; in affecting who got into college, rather than how they perform when they’re there, so that people within a given college don't differ that much in IQ scores. And indeed, there is some evidence for this: standardized tests used for college admission (like the SAT and ACT) are known to be quite highly correlated with IQ scores.

This teaches us an important lesson about correlations: If you’re looking at a sample that has quite a restricted range of some trait X (like age or IQ scores) then you should not expect to find strong correlations between X and any other traits.

Collider bias: Now suppose that having a higher IQ score really is one of the factors that helps with getting into college. But also assume that colleges are savvy and are trying to pick the best candidates using a variety of metrics, IQ score just being one of them. If this is so, then the group of college students who have lower IQ scores must, on average, have some other traits that compensate. For example, they might have especially strong work ethics or have great time management. So, within this group, students with higher and lower IQ scores might end up with similar GPAs not because IQ score doesn’t matter for grades, but because being at this college means you likely are the sort of person to have some other trait(s) that compensate for the effect that a higher IQ score would have. But, in the wider population, it needn’t be true that those with lower IQ scores will tend to have such traits. Thus, the fact that being in the group of college students means you must (according to this potential explanation) either have a higher IQ score or some traits that compensate ultimately hides the effect that IQ scores would have shown in a wider population.

In statistical terms, being in college is what’s called a ‘collider’ variable. It is influenced by IQ scores and other traits that contribute to higher GPAs. 

These two selection effects illustrate another important lesson about correlations: Real correlations might not show up in your data just due to the way the data points are selected. This could be by restricting the range of your sample, or because you’ve introduced collider bias. 

This is something to keep in mind whenever you encounter a weak or nonexistent correlation. Before concluding that there’s no relationship, you can ask whether selection effects might be hiding it.

Maybe all of these potential stories (above) contain some truth. Maybe none of them do. We’re curious: What do you think is the best explanation for finding that high school GPA is so much more correlated with IQ score than college GPA is?

3. Does having a higher IQ score mean you think you could solve society’s problems?

We asked participants (n = 692) about 35 diverse behaviors that popular culture or ‘common sense’ associates with having a higher IQ score. Here are the results. We show the correlations (together with their 95% confidence intervals) between IQ score and each of these self-reported behaviors in the forest plot below:

Each black circle shows the respective measured correlation and the black bar extending out either side of the circle shows the 95% confidence interval. If the 0.0 point lies within the 95% confidence interval, then our results show no statistically significant correlation between that behavior and IQ scores (at the 0.05 threshold for statistical significance).

There are lots of results here that could be called surprising. Given that these were all behaviors we think people could reasonably expect to be positively correlated with IQ score, any null result (where the 95% confidence interval contains the 0.0 point) is potentially surprising. Of course, it could just be that our study was not able to detect a real effect, for some reason, or that people don't self-report accurately for some of these behaviors - for instance, they think they are a fast learner but be wrong.

Even more surprising are the results that show a statistically significant negative correlation between IQ score and some behavior. Let’s focus on just one that we think is especially interesting: Believing that he/she could figure out solutions to societal problems.

We found an r=-0.15 correlation between participants’ IQ scores and the amount they reported believing that they could figure out solutions to societal problems (r = -0.146, p = 0.0004, 95% CI [-0.218, -0.072]). This surprised us because we’d expect that there is some correlation (however small) between IQ score and actually being able to figure out solutions to societal problems.

In his book The Republic, Plato argued that only people who are trained in and guided by reason, truth, and virtue are able to govern society in a way that solves its problems and leads to harmony. Funnily enough (given that he was a philosopher), he thought this meant societies would flourish best if they rejected democracy and had philosophers for kings. Although the idea of IQ wasn't developed until far after Plato's time, its relationship with reasoning skills may make one wonder whether he’d have included having a higher IQ score in his list of attributes.

So, what explains our finding that lower (not higher) IQ scores are associated with believing you could fix society's problems? One possibility is that it may simply be that those who have higher IQ scores are (on average) able to recognize more of the complexity of society’s problems and therefore the difficulty that anyone would have in attempting to solve them. Or perhaps such people have (on average) a higher bar for what counts as solving an issue. You might think this sounds like a straightforward example of what has been dubbed ‘the Dunning-Kruger effect’, which is often characterized in terms similar to the ones that comedian John Cleese uses here:

Maybe something along those lines does explain this result, but we advise caution with explanations like this: As we explored in depth in a prior article it can be tough to distinguish a genuine Dunning-Kruger effect from a statistical artifact (not a real effect), or the result not of stupidity but of people rationally assuming that, when they are uncertain about their skill, they may be near the middle.

Whatever the correct explanation is, this surprising finding reminds us to recognize the difference between confidence and competence.

4: How much does an IQ score tell you about a person?

For the last surprising finding, let’s zoom out and think about all of our results as a whole. This will lead us to some kind of answer to the question: “How much can an IQ score tell you about a person?”

Each participant’s IQ score was derived from their performance on 6 or 7 different ‘intelligence tasks’ from a pool of 62. Something that makes IQ scores informative is that they correlate strongly with performance on these tasks even when those tasks weren't used to compute the score. This suggests that IQ isn't just a summary of the specific tasks it includes, but captures something broader and more general. For example, we found that IQ score had the following correlations with the following tasks (for each of these, IQ score is calculated without using results from the respective task):

Probabilistic reasoning: r = 0.53, n = 423

Reading comprehension: r = 0.50, n = 397

Mental rotation of shapes: r = 0.36, n = 389

Going beyond the tasks used to calculate IQ scores, we found correlations with other traits, skills, and behaviors (some of which are discussed above, others of which can be found in our full write up). For example, one of the largest we found was the correlation with actively open-minded thinking (AOT) (r = 0.43, n = 670). This is the cognitive disposition to consider different perspectives, evidence, and possibilities before forming judgments or decisions. Individuals high in AOT tend to be more willing to revise their beliefs in light of new evidence and are less prone to cognitive biases or rigid thinking. We measured this disposition by asking participants how much they agreed with statements such as “It is important for me to be ‘open-minded’, even with regards to topics that challenge my deeply held beliefs.” or “I think that people should stick to their important beliefs even in the face of contradictory information.” (reverse-coded). Below is the scatter plot for that correlation - what do you think it tells you?

The spread of data points is broad, but has a clear pattern. For any amount of AOT that we measured, there is a range of different IQ scores that we saw in our population (and vice versa), but the top-left and bottom-right regions are clearly much emptier than the others. This shows that, if all you knew about a person was their IQ score, it’s unlikely that you’d be able to make any certain, definitive judgments about how much actively open-minded thinking they engage in (for example, there are some people in our data set with IQ scores around 60 who have higher levels of AOT than some of the people with IQ scores above 115) but, that being said, having an IQ score towards the upper end of the scale clearly makes it more likely that you have a higher amount of AOT, and a lower IQ score makes less AOT more likely. So, learning someone’s IQ score can warrant ‘updating your priors’ (which is to say, adjusting your expectations); it can change what you think is most likely - particularly if they are at the upper or lower end of the range. 

When it comes to smaller correlations, the same is true again. But this time, the amount you can adjust your expectations is lower. Consider this scatter plot for the correlation between IQ scores and household income (r = 0.15, n = 3688)

In this case, the spread of data points is even more broad. There are plenty of people who have high household income and high IQ scores, as well as plenty for whom both are low or one is high and the other low. This shows that, if you’re using someone’s IQ score as the basis for inferring something about their household income (or vice versa), you may still be warranted in adjusting your expectations slightly, but you should do so much, much less.

Some of the higher correlations we found included things like:

• IQ score and ‘pathological’ celebrity worship: r = -0.42, n = 681

• Verbal IQ score and facial recognition ability: r = 0.41, n = 294

• IQ score and self-reported job performance (if IQ score is <92): r = 0.46, n = 175

While the lower (but not negligible) correlations included things like:

• IQ score and having enough food in childhood (controlling for childhood poverty): 0.16, n = 499

• IQ score and being read to as a child: r = 0.17, n = 652

• IQ score and being breastfed in early childhood: r = 0.12, n = 499

• IQ and being politically conservative: r = -0.18, n = 3688

• IQ score and grit: r = 0.10, n = 686

These scatter plots do a good job of pointing out that:

1. IQ scores aren’t destiny: any given score is compatible with a wide range of values of all sorts of skills, behaviors, traits, and life outcomes. This means that an IQ score won’t help you learn anything definitive about a person; it can only help you adjust your expectations - sometimes a moderate amount, sometimes much less. For people who treat IQ as a deeply meaningful feature that determines a great deal about a person, this could be surprising.

2. That said, IQ scores clearly don’t tell you nothing. Learning a person’s IQ score really can warrant updating your expectations (at least a little) in a wide variety of ways, about a very wide range of things. For people who treat IQ as meaningless nonsense, this could be surprising.

As is usually the case, the truth resists simplicity. To avoid the pitfalls of thinking about IQ scores in unjustified ways, here is some advice (similar to advice we have given elsewhere):

1️⃣ Adjust your expectations when doing so is warranted, but avoid pre-judging people based on their IQ score – learn about people as individuals before coming to judgments about them. A small or medium difference in group averages is almost never a good justification for assuming those differences apply to individuals. Doing so is, in many cases, an example of the logical error known as the ecological fallacy.

2️⃣ Avoid language like “people with high IQs are like this, people with low IQs are like that” so that you don’t oversimplify the truth. If what you're actually trying to say is that one group has an average of some trait that's a little higher, then state that in all its nuance - do your best to avoid language that could lead to misinterpretation.

3️⃣ Avoid denying that average differences exist when they do. Denying the truth can cause a lot of harms: it can undermine your credibility, it can spread misinformation, and it can lead you to support ineffective policies.

4️⃣ When relevant, remind others that small average differences are not a good basis for judging individuals (epistemically or morally), and point out that the distribution of any given trait between those with higher and lower IQ scores is heavily overlapping (when it is) to combat people using differences in the average as a justification for stereotyping.

Conclusion

By conducting a giant study on IQ, we learned some potentially surprising things: 

1. Perfectionism is not a sign of having a higher IQ score

2. IQ score might matter more for high school than for college

3. Having a higher IQ score means being less likely to think you could solve society’s problems

4. Knowing someone’s IQ score can warrant adjusting your expectations in a variety of ways (some small, some moderate), but it can’t justify definitive judgments - for that, you need to get to know the person

Through exploring these surprising results, we encountered some general insights about correlations:

• They can miss meaningful patterns in small subgroups, when the signal is hidden by the noise of all the data points not in that subgroup.

• Real correlations might not show up in your data, if you pick your sample in a way that introduces a selection effect, such as range restriction or collider bias. 

These are both reasons to stop and think whenever someone tells you there’s no correlation between two things.

And remember: If someone is telling you that their data tell a specific story, an important part of critical thinking is to take a moment and ask whether there are other explanations available. If the person talking is making bold claims with few hedges, it is much more likely that they’re not giving you the only possible explanation.

Go Beyond IQ Scores!

Why not go beyond simplistic IQ scores and try our more thorough Cognitive Assessment?

We designed this tool based on the data from our giant IQ study, and it will break down your cognitive abilities in 7 key areas: logical thinking, conceptual knowledge, vocabulary depth, pattern recognition, numerical reasoning, spatial visualization, and linguistic learning.

Note: Unlike our other tools, this one costs $35, which goes to support our mission.