Artificial Intelligence - Evaluating Progress

Evaluating Progress

In 1950, Alan Turing proposed a general procedure to test the intelligence of an agent now known as the Turing test. This procedure allows almost all the major problems of artificial intelligence to be tested. However, it is a very difficult challenge and at present all agents fail.

Artificial intelligence can also be evaluated on specific problems such as small problems in chemistry, hand-writing recognition and game-playing. Such tests have been termed subject matter expert Turing tests. Smaller problems provide more achievable goals and there are an ever-increasing number of positive results.

One classification for outcomes of an AI test is:

  1. Optimal: it is not possible to perform better.
  2. Strong super-human: performs better than all humans.
  3. Super-human: performs better than most humans.
  4. Sub-human: performs worse than most humans.

For example, performance at draughts is optimal, performance at chess is super-human and nearing strong super-human (see computer chess: computers versus human) and performance at many everyday tasks (such as recognizing a face or crossing a room without bumping into something) is sub-human.

A quite different approach measures machine intelligence through tests which are developed from mathematical definitions of intelligence. Examples of these kinds of tests start in the late nineties devising intelligence tests using notions from Kolmogorov complexity and data compression. Two major advantages of mathematical definitions are their applicability to nonhuman intelligences and their absence of a requirement for human testers.

Read more about this topic:  Artificial Intelligence

Other articles related to "evaluating progress":

The Theory Of Artificial Intelligence - Evaluating Progress
... Artificial intelligence can also be evaluated on specific problems such as small problems in chemistry, hand-writing recognition and game-playing ... Such tests have been termed subject matter expert Turing tests ...

Famous quotes containing the words progress and/or evaluating:

    This pond never breaks up so soon as the others in this neighborhood, on account both of its greater depth and its having no stream passing through it to melt or wear away the ice.... It indicates better than any water hereabouts the absolute progress of the season, being least affected by transient changes of temperature. A severe cold of a few days’ duration in March may very much retard the opening of the former ponds, while the temperature of Walden increases almost uninterruptedly.
    Henry David Thoreau (1817–1862)

    Evaluation is creation: hear it, you creators! Evaluating is itself the most valuable treasure of all that we value. It is only through evaluation that value exists: and without evaluation the nut of existence would be hollow. Hear it, you creators!
    Friedrich Nietzsche (1844–1900)