Cleverbot is a corpus based chatbot capable of producing some natural conversations by using responses from humans.

As you can see it carries on just fine and can fool a casual observer. But the longer you carry on a conversation with it the more apparent that Cleverbot is frustrating to talk to, not so much that it isn’t human — after all, all of the responses are taken from human sources. If it weren’t so good at emulating a human from which you expected more, you wouldn’t be frustrated.

Cleverbot is frustrating in two ways:

• it isn’t interesting
• it doesn’t make sense

In other words, it lacks deepness, like a shallow human. Why?

Some say that Cleverbot isn’t an AI because it’s just humans talking to humans via the intermediary of a selection algorithm. One is tempted to be dismissive like this but it’s too convenient. It places too much weight on the role of the corpus and too little on that of learning and interaction. Cleverbot clearly captured some aspect of the human intellectual process that raised it to the level of a shallow conversation partner. What it neglected though will be enlightening to the question of what is deepness.

The most obvious way in which Cleverbot is shallow is in novelty. This is a result of mimicry of others. Mimicry is of course fundamentally human because we learn by mimicking. It doesn’t preclude novelty of some sort. In fact, although it isn’t clear at what level Cleverbot replicates from the corpus, whether in phrases or entire sentences, we are forced to accept that some novelty is injected into the selection process, as responses don’t come out the same every time. However it’s not enough to make Cleverbot consistently interesting, probably because Cleverbot doesn’t synthesize new ideas, merely passing on received wisdoms at the appropriate moments. It seems deep novelty requires more atomicity, and more computation.

The more subtle way in which Cleverbot is shallow is in contextual memory. Cleverbot, probably also for computational reasons, just makes remarks in response to the most recent context. As a result, it changes topics frequently, and doesn’t pursue a line of inquiry to its satisfactory conclusion because it doesn’t know what a line of inquiry is. It can be described even as evasive or flighty. While each response usually makes sense, Cleverbot doesn’t make sense over the span of a conversation, which requires more persistence.

It seems deepness is related to computational resources. And if we believe more intelligent machines — those that exhibit deepness and frustrate us less — to be more “human,” then we must expect a high level of computational competence as a human trait. Deep conversation is a measure of that. Disturbingly, the kind of conversation that Cleverbot can make is rather the more commonplace case among humans, yet it borders on meaningless drivel because it produces nothing new. Even more disturbingly, this is despite that Cleverbot can be quite clever sometimes, being stimulated to produce some of its store of wisecracks after elicitation by a motivated human counterpart who steered it to them. It shows that it doesn’t take much to carry on one half of a conversation, and deceptively so especially if one of the two is playing the role of an intelligent “human.”

Maybe being “human” isn’t that hard. Maybe the binary identification of machine/human should be considered a necessary quantification artifact, and the Turing test really measures a gradation of deepness instead (it is a probabilistic test after all, and of course subjective). If so, let’s run the Turing test on people in a double-blind way.