RIQ - the principles of combining many results

RIQ = Real IQ.

How Real IQ is presented in the user's portal.


The first RIQ concept was released as a normal mean calculation from all tests made by the user, this was not accepted by the tests takers at IQexams, they wanted something better like the Ferguson's formula. See https://megasociety.org/noesis/141/ferguson.html .

The problem with Ferguson's formula is that we need correlations between all 100 tests, maintaining correlation matrices with 100 x 100 tests were too overwhelming. So we decided to introduce Rarity Mean, a concept suggested by John Culkin , this was an interesting approach, a kind of built-in weighting towards the higher scores. Even though it looked promising and was accepted by the users, we had problems with outliers, since it emphasized single outlying high scores in some cases.

RIQ4, the introduction of Ferguson formula, using one "fine-tuned" correlation constant "k" in the figure below. By using a common constant that was fine-tuned for best correlation to proctored tests, we were able to change to Ferguson formula, instead of Rarity mean. N is the number of tests.

k = 0,84 is the figure that has been set after fine-tuning, this is the best compromise.
A too low value (k=0.6) will inflate high RIQ4 results.
A too high value (k=0.9) is meaningless and unrealistic and does not improve (reduce) inflated RIQ4.

RIQ5 - the current version. It is basically RIQ4 with weighted Quality from the tests performed by the user. Z the constant in the picture above, is the sum of any suitable Quality indicator from all the tests. This can for instance be the Cronbach's Alpha (which is probably the best quality indicator). You can combine Alpha with other quality indicators as well.

RIQ - The versions
RIQ : Arithmetic mean from all tests
RIQ2 : Rarity mean from all tests, excluding 30% of the lowest scores
RIQ3 : Rarity mean, the users could cherry-pick the test they wanted, 2:nd attempts -3% also ok.
RIQ4 : Ferguson's formula with fine-tuned common correlation constant replaces Rarity mean.
RIQ5 : Ferguson's formula with Quality Weighting of the scores.

Problems with RIQ3, RIQ4 and RIQ5. Understandable the users love to cherry-pick the tests they performed best on, some do it in an honest way, others do it only to maximize the high score and still looking as good as possible. This have caused some inflation on the RIQ figures compared with proctored results, we calculate the current RIQ5 mean to be inflated by +5 IQ points, in the way it is used by the rest takers. A re-introduction of a completely automatic RIQ5 calculation would deal with this fairly, but it will not be popular. Hence RIQ6 is planned to use both automatic and user selected figures.