To understand the work which I am doing this semester, a broader look at the world of the chatbot is needed. With the following readings, students will dive into the potential uses of this virtual conversational agent, and thus my motivations behind pursuing this work. Students will also be introduced the technology and techniques I plan on using. These techniques will be compared with existing techniques which have been used to create virtual conversational agents in the past. Students will interact with multiple existing virtual conversational agents to get a firsthand look at what works and doesn’t work when it comes to interactions with a virtual agent. Students will also get a direct look at the work which I have already completed with regards to this work.
Students will get a look at both the creative and more technical aspects of this project. Students will be assigned tasks which may be more suited for a creative writing class, but also a linguistics class. This project brings multiple areas of study together – thus it is imperative that students understand the relevant areas of each of these fields.
Week 1 – Introductions and Potential
“Everything” you need to know about chatbots
This recent article is an easy introduction to the world of chatbots, and even provides links to resources for creating your very own! To understand what a chatbot is at a basic, conceptual level, read this article to get an explanation. The article also provides a link to many chatbots which you can interact with via Facebook Messenger. Students should choose three random chatbots from this list and interact with them. Students can find chatbots for other platforms via the same site, and I encourage you to interact with as many as you can until you get a sense of how natural or unnatural they feel to interact with in general. Try your best to pinpoint what is natural or unnatural about speaking with them.
This is an excellent overview of different dialogue systems present in video games. An overview is necessary because one of the intended uses of this project is to insert a character capable of human-like casual conversation into a game or virtual setting. Pay attention to the Parser-Driven Dialogues section and the following quote:
“Parser-Driven Dialogues are rare in modern games for two reasons. The first is that the freedom they provide is extremely time-consuming to produce. The system needs hundreds of potential responses to accurately simulate a single, short conversation.”
One aim of this project is to make Parser-Driven Dialogues not so rare. The conversation may also be longer, and repeat conversations may take place with the same character, in which case the character needs to be able to remember information from the initial conversation.
This is one example of an application of a chatbot in a healthcare related setting. This chatbot is integrated into a wheelchair which the user can speak to and tell in plain language where to go, and the system will understand the user. One way the chatbot I am working on can help with this specific project is that the chatbot will be capable of casual conversation, which can enhance the ease of conversing with the wheelchair system.
Week 2 – Current Project Status and Basic Considerations
FLoReS: A Forward Looking, Reward Seeking, Dialogue Manager
Don’t be scared by the academic nature of this paper! This is the backbone of the system which we have decided to use thus far. In this paper, various examples of existing virtual humans which use this system behind-the-scenes are given. The message of this paper is that the system relies on a “reward-seeking” method in which weights are given to certain dialogue acts which the system can take. The system can predict all possible ways a conversation may go and will pursue the dialogue acts which follow a “path” towards the highest available rewards. For instance, if I wanted the system to ask a question as a means of telling a story related to the question later in the conversation, I would assign a high weight to the telling-of-story dialogue act, and the system would choose to ask the question if that were the only means of telling the story and thus obtaining the “reward.” This method allows for an internal representation of the many different paths a conversation might follow, and thus various ways to handle each different response a user might give. Other methods do not predict what a user might say, whereas with FLoReS, the user responses can be predicted and classified accordingly.
This is (a portion of) a presentation I gave at the end of my internship in which I discuss this project. I give an overview of the project, going over areas of consideration I had while building the chatbot. I talk about knowledge representation and provide an example ontology of family relations in order to demonstrate what the system should ideally be able to “understand” using inferential logic. I also run through a “demo” of a conversation with the chatbot, during which I detail exactly what is happening within the system as the conversation progresses.
Introduction to Natural Language Processing (NLP) 2016
This blog post is an excellent overview of the field of Natural Language Processing, which is the central body of knowledge utilized for this project. NLP is becoming commonplace in the technology we use in our everyday lives, and regardless of this project, an understanding of how the many apps we use work with language understanding is vital for any student. The central problem in NLP is that human language is difficult to parse by a computer, as the irregularities in speech from person to person, let alone the ungrammatical sentences we use in speech along with the subtle messages we send implicit in our words make computer understanding of human language a challenge. Machine learning is identified as a central part of the field.
Week 3 – Behavioral Considerations
“I Like You Shirt” – Dialogue Acts for Enabling Social Talk in Conversational Agents
This paper introduces the concept of dialogue acts and introduces their usage in casual conversation with a virtual conversational agent. It is important to understand social dialogue acts in relation to this project, as the design behind each individual utterance the system outputs, there is a dialogue act (AKA a conversational behavior) associated with that utterance. This paper bases the cooperative social dialogue acts it presents on the concept of “face,” which is “the perception of the self of both interactors.”
Activity Planning for Long-Term Relationships
This paper takes a stab at the problem of rapport-building with a user. Ideally, when a system interacts with a user, it will be able to give relevant information and not say something out of place. In the case of this paper, the authors attempted to use rapport to allow a system to know when something was appropriate or inappropriate to say or do. In a casual conversation, it might be inappropriate for the system to ask about traumatic childhood events if this is its first conversation with a given user, let alone in any conversation. Being able to use a measurement of rapport in the system will allow or disallow appropriate and inappropriate dialogue acts and utterances.
Towards A Multi-Dimensional Taxonomy Of Stories In Dialogue
This paper examines the very fabric of what a story is, and presents a story taxonomy through which we can identify different aspects of a story. Integrating a capability of being able to parse stories in the chatbot would be a very useful endeavor as it would allow the chatbot to better understand the user and better utilize the information which the user is conveying. The more the chatbot can understand from the user, the better.
Week 4 – Conversational Behaviors & Realistic Dialogue
Dunbar, R. M., Marriott, A., & Duncan, N. C. (1997). Human conversational behavior. Human Nature, 8(3), 231-246. doi:10.1007/BF02912493
Sharing stories is one means of exchanging social information, which this work proposes is the is the main purpose of conversation amongst acquaintances. While this probably seems obvious, this paper allows us a formal basis from which to base our work in terms of justifying what this chatbot should be talking about. It also forces us to consider the question of gender – do we have to gender our chatbot? It is also worth examining the criteria by which the authors classify the difference in conversation between gender and age. We can then design a character based on these criteria and assign the character gender and age should we choose.
16 Observations About Real Dialogue
While this article was written in the context of writing realistic dialogue in concert with a story’s plot, the observations are still useful to us. For example, with regards to item 1, keeping in mind that real people “circle around to the important parts” – if the chatbot and a conversational partner are talking about a topic in depth and get off track, our behavioral goal values should be set in such a way that even if the conversation gets off topic the chatbot is capable of circling back to the original conversation. Or with regards to item 6 – we already have implemented a timer of sorts which considers whether the conversational partner replies so that our chatbot isn’t silent until eternity.
Assignment: Eavesdrop on conversations
The best way to know what real human dialogue sounds like is to hear others speak it. The second-best way is to observe yourself speaking. The former takes a lot more effort though, and for that reason it is important to set aside the time to go into a coffee shop, park, or other area where people have casual conversation and go listen to them. If you may legally tape record, that could be helpful as well. The point is that obtaining real dialogue will aid in knowing what real dialogue looks like and could help us in writing our chatbot’s lines of dialogue for use in the system. Aim to do this exercise for at least two hours of actual listening and recording.
Week 5 – Machine Learning
This introduction to machine learning is beginner friendly and will teach students about the differences in supervised learning vs unsupervised learning. It also provides a practical example with which the student can work through along with the author. For something more visual, this link is a visual introduction to machine learning. Machine learning is important to the field of NLP and Artificial Intelligence as well as Data Science. It can make the job of coming up with rules which match conversational patterns a lot easier. But students should ask themselves: can machine learning do everything? What are the limitations? Spoiler: we don’t know what exactly the machine is learning, in that the rules it can internally establish are not made explicit in human-understandable terms (though they can be made explicit in mathematical terms). Machine learning has been the go-to solution for many chatbot systems.
Wonder is a bot that will remember anything for you
This chatbot, Wonder, can store both information you send it and the way which you refer to the information you send it. You can text Wonder information you’d like it to store, and then text it again to get the information sent back to you. “Memory” is an important consideration in chatbots, and the one being built for this project is no different. In Wonder’s case, it gets better at recognizing user requests over time, as it uses machine learning. Looking at Wonder is important for thinking about what information is important to store and it’s a good example of a chatbot with “memory.”
Twitter taught Microsoft’s AI chatbot to be a racist asshole in less than a day
Tay.ai is a perfect example of where machine learning can go wrong. In this case, Tay was given bad data – that data being the utterances of users across the internet, who purposefully sent Tay disgusting and racist messages. Tay was built to learn from these utterances and essentially mirror what real people said to it. This case goes to show a couple things: that the output of a machine learning algorithm is only as good as its input, and that not all chatbots were meant to be created. Tay was a “fun” experiment by Microsoft gone awry.
Another example of the ethics of machine learning, this time in the form of fiction. The plot of Her revolves around Theodore and a personal assistant AI, Samantha. The two grow closer than is expected of machine and man. This story raises questions of whether a system can be too intelligent, and more closely related to this project, if a human can feel too close to a system.
Week 6 – Human-Computer Interaction
This writeup gives a broad look at the field of Human Computer Interaction, which is important to understand as we the advent of virtual conversational agents and dialogue systems is a game-changer in the field. Keep in mind that the means of interacting with the computer is altered based on the purpose of computer use. Hypothetically, how does the use of a computer change if it is only to be spoken to? This reading will give us another perspective on what we are working towards.
An example of how Siri became a companion to a boy with autism. Even though the ability to maintain a casual conversation is not one of Siri’s primary functions, Apple put sufficient conversational ability in its flagship virtual conversational agent to be able to appropriately respond to a user’s words, whatever they may be. The situation discussed in this piece exemplifies how important the “conversation” aspect of virtual conversational agents is, and demonstrates that a sense of companionship is valuable.
What’s notable from this study is that human-chatbot interaction is different in terms of length, level of vocabulary, and level of profanity that human-human interaction in casual conversation. We must ask ourselves whether we want the human to know that they are speaking with a chatbot or whether they assume the conversational partner is human.
Week 7 – Gaming and Interactive Storytelling
Making a chatbot that drives a narrative in sci-fi exploration game Event[0]
The chatbot being talked about is a big step forward for the upsurge of chatbot systems in video games, but differs from this project in a few key ways. For one, the chatbot is presented to the gamer as flawed, which excuses mishaps in conversation. Also, the conversations are a lot more like those one would have with Siri than with another person – the player gives commands to the chatbot, asking the occasional question. This project aims to create a chatbot which can better handle casual conversation, not domain-specific conversation. The integration of this game’s chatbot into the gameplay is a great design choice which should be followed in future games. Check out this video and watch for a few minutes to get a sense of how the interaction with this chatbot works (and please excuse the immaturity/profanity of the player).
Chris Crawford on Interactive Storytelling by Chris Crawford
It’s important to consider what has been done in the past with regards to telling stories interactively, and if we consider our chatbot as a possible outlet for creators to tell stories, then an understanding of what has worked and what has failed in interactive storytelling is vital. Chris Crawford’s work on this topic covers the human side of storytelling while teaching some of his own techniques for creating interactive works. He covers his Storytron technology, which notably was a failure (and is “currently in a medically-induced coma while [they] redesign the technology”). http://www.storytron.com/
Digital Storytelling by Carolyn Handler Miller
While at first glance, this work may seem tangential to this project, a background in the various means of telling stories (especially in new media) will allow us to better shape virtual conversational agent to better entertain and tell stories, should we aim to insert this into a game or similar platform. This book covers a lot of different platforms, which is useful when we consider the product and the platform on which we share it. Does our chatbot live in a kiosk, where interactions are limited and in public? Or does it live in a mobile app, close to just one person? Having an understanding of these possibilities will better inform us as to where best to implement our creation.
Week 8 – Knowledge/Topic Representation: Ontologies
Assignment: Create your own ontology
An ontology is defined as a “a set of topics connected with different types of relations” (http://ontogen.ijs.si/). Your task is to find one specific domain (family, food, video games) and create an ontology of that one domain (it could be family relationships, food groups, video game genres, but not limited to just those areas for each of the examples). Focus on the relations, and be as expansive as possible.
FrameNet seeks to build a “lexical database of English that is both human- and machine-readable,” with one of the applications being information extraction. The “frame elements” described in the About section are of interest to us for use in deciphering what a user is saying into a usable format with regards to topic recognition. FrameNet may be our main resource for topic recognition, so it would benefit you to become familiar with this writeup and perhaps even poke around other areas of the website.
Billed as a “semi-automatic ontology editor,” this tool makes it easier for us to edit our ontologies, and is especially useful because we have no specialist knowledge in all the areas of casual conversation (which are numerous, possibly limitless) while the tool itself is user-friendly. It can automatically suggest relationships between entities in the ontology. Also read this paper (http://ailab.ijs.si/dunja/SiKDD2006/Papers/Fortuna.pdf) on the work by the creators of the software.
