Although saying that, it certainly has better results than doing nothing at all. A few false positives are bound to slip through.

You are both right, it does that. It only catches the majority of cases; People stretching a single letter to express hesitation, disbelief, sarcasm or shouting. In other cases it just replaces one misspelling with another.

My stance on handling bad spelling is that that is the job of commercial spell checkers that I could some day hook up to my program. I only use these minimal-effort methods to reduce the amount of junk in my program’s vocabulary. That it would momentarily help against last year’s judge is just a convenient coincidence, but as it shows, even his dumb methods can be outsmarted.

I found some questions that are tough as nails: A test of verbal reasoning for primary school children. It contains word/letter/number games, analogies and text comprehension. Most of these require pattern recognition and deciphering skills as they are often found in IQ tests. Ironically the most human answer to most of these questions would be to get it wrong or not have a clue, but there are some like analogies and number sequences that might make likely Turing Test questions. For your inspiration:

http://www.elevenplusexams.co.uk/assets/189/CGP11+VerbalReasoningSampleTest.pdf

http://www.dailymail.co.uk/news/article-2690082/Cheer-children-seven-eight-parents-fail-11-Plus-Survey-reveals-huge-support-grammar-schools-one-small-problem.html

I just tried some of those and struggled myself.

However, like the ASCII art questions, I would enjoy watching a Turing Test judge trying to enter something like:

Ken, Artur, Hada, Fiona and Louise do a treasure hunt with five clues to find. Ken got all the clues right but finished last. Louise got back second, but got two clues wrong. Hada got more questions right than Louise, but finished after her. Artur wasn’t faster than Louise. Fiona was faster than Hada, and only got one question wrong. If these statements are true, only one of the sentences below cannot be true. Which one?

A - Fiona got back first.
B - Hada was faster than Ken.
C - Fiona was faster than Artur.
D - Fiona came third.
E - Louise scored the least points

It would certainly eat up the time allowance.

That sounds like some of the logic problems I used to enjoy as a kid (I still enjoy them, but just don’t have that sort of time anymore. ).

I always had more trouble understanding the complicated instructions than figuring out the puzzles themselves.
I’ve been tinkering on numerical sequence questions “2 4 6 ...” a bit, also featured in that test. I’ve heard some people say they would be “impossible” for a chatbot to get, yet I’ve seen Merlin and Wouter Smet handle them. Rather than for silly tests, I figured one could apply the same math to predict object movement or to deduce time intervals of (user) events, so it just might be useful. I had hoped a formula like y = ax2 + bx + c would help me out, but I find myself programming separate patterns for linear and exponential increase, and then some, I suspect.

I would list use case examples of real IQ test samples. Find examples from IQ tests.

3 9 81 ?

2 3 5 ?

Then, figure out the formula yourself:

x, x^2, . . . etc.

Use your solution list like pattern matching, with brute force check the sample question against the list of sequence solutions.

I wouldn’t look for a universal equation. Also some tests show 4 or 5 members of the sequence, so your examples need to be robust enough to handle sequences of varying length.

Robby.

Thanks for your input Robby. You have actually convinced me to go with equations for e.g. “1, ..., ..., 4, 5”. I always try to avoid manual work as much as possible, and using a known equation like y = ax +b covers all linear patterns. Perhaps there are equations that cover even more ground, otherwise the rest of the procedure should be as you’ve described.
This is all related to the purpose of course. For a Turing Test, three simple patterns (+x, *x, ^x) should cover the imagination of a judge under pressure. But equations such as I mentioned can also be applied to trains leaving stations at irregular times and travelling at particular speeds, which gets close to being useful. Don’t tell my maths professor I said that though.

Good luck!

http://oeis.org/ is a good site for finding out numerical sequences. With a bit of coding you might be able to pass some numbers to it to find the sequence.

For example: 2,4,6,8,10
http://oeis.org/search?q=2,4,6,8,10&language=english&go=Search

The answer comes back: “The even numbers” and shows you a good sized extract of the sequence to work out which is next/previous.

That site is easily confused by simply using two sequences and alternating between them. For example, 1, 0, 2, 2, 3, 4, 4, 6, 5, 8. It really just alternates between two very simple sequences: 1, 2, 3, 4, 5… and 0, 2, 4, 6, 8…

True but it’s better than nothing. I would probably struggle to see what came next with 1, 0, 2, 2, 3, 4, 4, 6, 5, 8 had you not posted the answer.

In principle, it’s possible to perform arbitrary pattern finding by searching for computer programs (defined as finite strings of instructions in the machine code of some arbitrary universal turing machine) that generate the pattern*. I did something like that once and applied it to predicting number sequences, but it took an inordinate amount of time to handle any but the very simplest patterns (and would also tend to crash due to running out of memory, but that could have been easily fixed by generating the candidate programs one at a time rather than many at once). I do wonder how far this approach could be extended by techniques such as:

a) Carefully choosing the universal turing machine based specifically on the task of predicting number sequences (I just used a really crappy virtual machine I’d programmed years earlier primarily as an educational exercise, and not for the purpose of predicting number sequences).
b) Using techniques other than exhaustive search (random walk, genetic algorithms, simulated annealing, etc.).
c) Generating candidate solutions in a very specific order based on various heuristics.

I’m not sure how efficient such an approach could be made, but to that extent it would be truly general (no sequence would ever be in principle beyond its grasp).

*The pure form of this approach eventually runs afoul of the halting problem, but it’s easy to write approximations of this approach which provably converge on the true result in the limit.

Oeis.org also has the issue that it does not seem to sort results by simplicity. The top result it returns for ‘1,2,3’ is the Fibonacci sequence.

EDIT: Another sequence based on the same theme as before that the site can’t solve, but this time one that a human can very easily handle: 0,0,3,3,6,6,9,9

EDIT2: signed:1,2,0,3,-1,4,-2,5

(t(0) = 1, t(n + 1) = t(n) + (-1)^(n + 1) * n)
It doesn’t solve it.

The site is an interesting resource. It appears not to calculate anything but rather pattern-matches the sequence with the sequences in its database, much like a chatbot.

Are two mixed patterns still a pattern? I had a hard time recognising the alternating patterns even after explanation. A real-world equivalent would be to hear the footfalls of two people walking out of sync; it would sound like chaos. When programming an NLP AI for this sort of thing, one should also maintain the possibility that the user is stating their lottery ticket numbers, or nonsense.
(The solution to your examples is to completely ignore every alternate number)

While this shows that there are more number games imaginable than a narrow AI could answer, in a Turing Test it would also be reasonable not to know the correct answer to advanced questions, from ASCII to number sequences to really bad spelling. Turing Tests then become more a matter of detecting the type of question in order to make the appropriate excuse. From this it would appear that the more extreme the question, the more easily it can be answered “human-like”.