Tuesday, May 25, 2010

Moore's Law vs The Flynn Effect

It's instructive, I think, to compare the rate of progress of machine computation with the rate of progress of human intelligence.  Above is the data on Moore's Law (taken from Gordon Moore's paper Moore's Law at Forty, p76 of Understanding Moore's Law).  It shows the number of individual logic components per single die (the unit of manufacturing in the semiconductor industry).  Note that the y-axis is logarithmic, so a straight line on this graph would be an exponential on a linear graph.  You can see that the "1975 projection" was slightly optimistic, but not too shabby at all.

This next graph (same reference, p69) shows the total number of transistors being shipped by the entire global industry.  Again, it's a logarithmic scale.

If we look at the 1988-2004 segment, it's gone from a shade under 1015 to a shade over 1018 so a factor of 2000 or so in 16 years, which is a doubling time of a little under 18 months.  If you thought human population was a problem, doubling every few decades, it's worth thinking about the implications of the transistor population doubling every 18 months.

But, the humans are not standing still!  It turns out that the psychologists who study IQ have noticed that it's been gradually increasing, something known as the Flynn effect.  All the money we spend on child social services, child development research, better nutrition, etc, has some effect.  Here, for example, are some data on an IQ measure for Norwegian conscripts from Sundet et al.

(The effect has been documented in most advanced countries, but I like this paper because it has annual trend data).  The paper is titled "The End of the Flynn Effect?" because they are concerned that the 1994 point represented some kind of "Peak IQ".  Personally my guess would be a little more optimistic - I think knowledge of child cognitive development has continued to improve and I would hope for further improvements in this curve over time.

Still, it's striking that if we rather optimistically take the rate of improvement from 100 in 1954 to 110 in 2002, that's an improvement of 10 points in 48 years.  That's 0.2%/year.

That's a doubling time of about 350 years.


marku said...

Interesting. I didn't know that Moore's law was per-device, I thought it was # devices/unit area. Since die sizes have been increasing right along with decreasing device size, it convolves the improvement in device size. I do remember early in my engineering career that a 10um device (usually measured as gate length of a MOSFET transistor) was state of the art, now the comparable number is around 45nm. so that is a factor of 220 decrease.

I was surprised to see that there was no rolloff on the 2010 data. ML seems to be hitting a wall (with silicon, at least) as now MOSFET gate oxide layers (which are shrinking along with gate length) are now only several atoms thick, and leakage current management has become a large problem. Obviously gate oxides thinner than one atom are, shall we say, unlikely?

But the end of ML has been called many times before, and always progress has gotten back on track.

As I said, Interesting.

Simon Tegg said...

Malcolm Gladwell covered the Flynn effect.
http://www.newyorker.com/arts/critics/books/2007/12/17/071217crbo_books_gladwell?printable=true#ixzz0p0E9yqepThe key point is here

"If the everyday world is your cognitive home, it is not natural to detach abstractions and logic and the hypothetical from their concrete referents,” Flynn writes. Our great-grandparents may have been perfectly intelligent. But they would have done poorly on I.Q. tests because they did not participate in the twentieth century’s great cognitive revolution, in which we learned to sort experience according to a new set of abstract categories. In Flynn’s phrase, we have now had to put on “scientific spectacles,” which enable us to make sense of the WISC questions about similarities. To say that Dutch I.Q. scores rose substantially between 1952 and 1982 was another way of saying that the Netherlands in 1982 was, in at least certain respects, much more cognitively demanding than the Netherlands in 1952. An I.Q., in other words, measures not so much how smart we are as how modern we are.""

IQ measures how well one manipulates abstract symbols (maths and language). Some people, usually people who are good at manipulating abstract symbols, call this intelligence. I would agreee that abstract thought is useful. But rising IQ's could just be a sign that abstract thought has become more valued over practical intelligence during the 20th century.


H/T Ran Prieur http://ranprieur.com/archives/015.html

Stuart Staniford said...


I would certainly agree this far: IQ is a cultural product of western culture, and is an attempt to measure traits of value in a culture built on abstraction and reason. At the same time, abstraction and reason are fundamentally what separates us from all other species, and in particular, seem to be basic attributes of civilization (all of which, to my knowledge, have developed concepts of number and at least some kind of proto-writing or pictographic system).

Simon Tegg said...

I agree with your statements here, but you seem to imply that abstract reasoning is superior practical knowledge. Therefore, the rise in IQ's is a sign of human progress.
My postition is that a balance between abstract thinking and practical thinking is desirable. Unbalanced abstract thinking can lead to the fallacy of misplaced concreteness. For example, Herman Daly attributes much of the errors of mainstream economics to this fallacy.

CanadianFox said...

There is an inverse correlation of childhood infectious disease and IQ that was reported here:

It might explain this Flynn effect as well, although I have not heard of anyone else making that claim.

However given the introduction of childhood vaccination and antibiotics in the 20th century and the apparent flattening of the Flynn Effect in the later half of the curve, where these things are virtually universal in N. America, this is a candidate worth investigating in my opinion.


DavidLJ said...

Three things that have had massively upward effects on IQ:

The spread of iodine. Inland areas far from the sea have long had huge populations with goiters and idiocy purely from this nutritional deficiency. Our blood plasma is roughly the sea water of some hundreds of millions of years ago, less saline than today's seas, but just as varied in its mineral make-up. Without all those nutrients the human grows up inadequate.

The coming of the railroad. Isolated villages everywhere were inbred from their formation, as contractors to run the cities' outsourced gardens, agriculture, until the railroads arrived. Then, suddenly, people stopped marrying their cousin next door, and everybody had a stranger in the family. Idiocy, cleft palates, and crossed eyes became less common. IQs leapt.

Wars. In countries where the language of the capital has become, France, Britain, Germany, or is becoming, China, Russia, the language of the whole country, war is a major factor. There's a feedback with railroads here. Modern war needs the railway. Steel-based society needs to work out new power relations, and so on. But the emergence of national languages is the emergence of new and larger gene pools - and mongrels are always and everywhere, tougher, healthier and brighter than poor benighted purebreds.

Stuart Staniford said...

David - fascinating perspective, makes sense.