Hi,

My name is Martin and I work with art, mostly digital and computerized.

I am hoping to find out if it would be possible (any since in trying) to train a chatbot by parsing literature, lyrics, or existing chatlogs.

So basically what I asking is : Is it possible to exclusively train a chat using static “untreated” data, which is following any flowcharts or rules (except for the English language grammar itself).

I am saying exclusively, since I would like to “capture” the essence of the writer.

PS.
I do understand that this would not be the best way to train a chatbot, but what I would like to find out, if this is completely naive?

Martin,
It is not currently possible to create a chatbot by exclusively using unformatted literature or lyrics. There have been explorations by “botmasters” (advanced chtbot desiners) that have created bots by reading input and then massaging the resulting data. “Fake Kirk” might be the best example:

Pandorabots Spellbinder scanned publicly available fan-compiled transcripts of 72 episodes of the original Star Trek TV series. In those 72 episodes, Captain Kirk has about 9000 lines of dialog. Pandorabots Spellbinder read these lines and created a Fake Captain Kirk chat bot with about 2000 unique categories and 6000 responses (a category is the basic unit of knowledge). The AI Foundation has published this parody characters through this link:
http://alicebot.org/kirkbot

It is possible to create a bot by reading transcripts and editing the resulting data, but building it is a non-trivial process.

Ok, I see

Thank you for your answer.
I guess it will be a massive challenge to achieve this as I suspected.
=/

Regards

What you’re asking is a bit beyond available technology. I’m sure you could load all phrases of lyrics into a chatbot and have it quote at random (or further train it manually), and some AI are capable of extracting knowledge from text using sophisticated parsing, but neither will extract the essence or style of a writer. Only one project I know of comes close to that, but it doesn’t chat:
Robot that writes like Shakespeare

Have s look here.

http://pandorabots.com/static/html/About/SpellBinderPR.html

Revisiting the topic title, I find it a fascinating idea to be able to “upload” biographical texts and bring historical figures back to life.
Some time ago I happened to upload facts (no dialogue) to my AI-in-progress by having it process wikipedia text, with results like these when I changed its identity to the name “Shakespeare”:

Text fed to the AI: “William Shakespeare was an English author who wrote 38 plays and also many poems. He was a writer”
User question: “Tell me something about yourself.”
AI output: “I am Shakespeare, William. I am an author and a writer, and I write a lot.”

Given that I’m an amateur, an IBM-grade version of this functionality could deliver us an interactive biography that could talk about Shakespeare’s children like they were its own. Fascinating prospects.
However, it still leaves us to find a way to capture and implement the character’s personality profile, and that’s at least a decade of research away. So in that regard, harnessing and re-using existing dialogue is still your best bet.

That’s petty cool Don.

Techniques may be used with lyrics and literature to train a chatbot. One is called a Marchov chain. As for, “existing chatlogs”, the answer is yes.  Merlin gave a good example.

@Pla*Net

The correct name is Makov chain, and it is a stochastic finite states-model where you only have memory of the last N states and the condicional probability to ‘jump’ int the next state, given any random stimulus.

I doubt this could yield a good predictor, it used for POS tagging (detecton of specialized chunks) and many other things, there is a more complex model called Hidden Markov Chains, where the states are hidden and you get only an observation and the probability to see it, but is esentially the same.

Conditional Random Fields is the ‘new’ generalization of this model, and might contain any-length (N>1) chains, as precondition, but its very expensive on training.

They are all used as classificators but with a reduced number of states (usually <100)

But both are unlikely to be used for this, because in linguistics the number of symbols (as words) is too large to handle. not the POS tags, but for lyrics you need some kind of broad semmantic classification, havin thousand of “states”.

cheers

Oh, thank you Andres,

(I must have had the “ch” from “chain” stuck in my head. )

ACM Student Lecture Series: Beyond ELIZA: Using Markov chains to build ‘intelligent’ chatbots

Abstract

“ELIZA was one of the first, and is certainly the most famous chatbot designed to explore this question of what it means to be intelligent. Much has changed in the 46 years since ELIZA was released, but the goals remain the same: Produce text that looks like it was written by a human. Using Python, we will learn how to use Markov models to create a conversational bot that produces surprisingly humanlike results.”

http://www.cs.uoregon.edu/Activities/Public_Talks/20120521-Ross.php