Simulation of personal emotion experience

Reeves & NAS (1996) showed that humans like to communicate with computers as they do with people. Software applications which include models of emotional processes are needed to model the social and emotional aspects of human-machine interaction. Extending classic AAA and logic by adding simulated emotions can be useful to improve the user’s experience in many ways. This chapter will provide a brief overview of existing solutions and models used for artificial emotions (AWE) and present a novel model of emotion simulation (SIMPLEX).

Empirical data will be reported on its performance, especially the occurrence of emotions, in a gameenvironment. This chapter concludes with a comment on the usefulness of separating AAA and AWE engendering recent advances in cognitive neuroscience. 2. Models for artificial emotions 2. 1 Historical roots The ass saw what might have been the first debate about emotions andartificial intelligence. The main and – as we know now – most important point was that purely cognitive systems lacked emotions, which strongly influence human thought processes.. Two of the models that emerged at that time will be described here.

Simony’s interrupt system Herbert Simon was the first to propose that emotions should be part of a model of cognitive processes (Simon, 1967). His intention was to provide a theoretical inundation for a system incorporating emotions and multiplegoals. Within this system, important processes could be interrupted so that more attention went into satisfying important needs (e. G. Hunger, safety). Herbert Simon imagined two parallel systems, one designed to achieve goals (cognition, planning) and one observing the environment for events that require immediate attention (emotions).

Indeed, the possibility of interrupting current cognitive processes is 2 Name of the book (Header position 1, 5) vital for survival, as it makes it possible to react to threats, but also to pay more attention to one’s surroundings when a threat is expected. Today’s Fungus Eater Another step towards a theory for the computer modeling of emotions was made by the psychologist Mason Toad (Toad, 1982) between 1961 and 1980, with a model called the Fungus Eater. This model resulted in the design of an autonomous robot system and partial implementations.

At first, Toad only wanted to create a scenario for a cognitive system that would require concentrating on multiple issues at the same time. In this scenario, the task was collecting as much ore as possible with the help of a mining robot. Operating his robot required energy that could only be gained by collecting a special fungus. Additionally, different Fungus Eaters were competing for the same resources, thus making the scenario more complicated. Toad came to the conclusion that in order to survive on their own, these Fungus Eaters would need to have emotions and to be partially controlled by them.

However, Toad named them ” urges” instead of emotions and on closer examination, it is apparent that some of these are actual emotions like joy or anger, while others are needs, goals or motives (e. G. Hunger). 2. 2 Theoretical approach and recent models There are roughly three areas where emotion models are applied. Artificial emotions (AWE) can be used to improve problem-solving in complex environments, as in the early approaches mentioned above. Emotion models can also be used to test psychological emotion theories in experiments using controlled scenarios.

Finally, emotions are essential to make computer characters more believable. Emotion models which synthesize and express emotions are necessary to make AAA characters more human- like. These models will be the focus of the next sections as they have inspired our own emotional model. The most influential theoretical approach, COCO, will be presented in detail, as it is the basis of many computational models of emotion. Then, three interesting recent models are briefly described.

COCO – a theoretical approach to simulate emotions The COCO model by Retort, Color and Collins is an emotion theory based on appraisal which was explicitly developed to offer a foundation for artificial emotion systems (Retort, Color, & Collins, 1988). Its authors succeeded as it inspired many modern models and approaches to artificial emotions. The basis of the model is that emotions are reactions to the attributes of objects, to vents or to actions. Note that internal events (like bodily sensations ormemories) which are a part of most modern emotion theories are neglected in the COCO approach.

Objects, events and actions are evaluated in an appraisal process based on specific criteria, and result in multiple emotions of different intensities. Figure 1 gives an overview of the COCO approach. Appraising the aspects of objects requires the agent to have attitudes (tastes or preferences) in order to decide whether the object is appealing or not. This appraisal process results in either love or hate. Chapter Title (Header position 1, 5) 3 Fig. 1. The COCO model Events, or rather consequences of events, are appraised by analyzing their impact on the agent’s goals. This determines the desirability of events.

The degree of desirability depends on how much closer to or further away from achieving the goal the agent will be after the event. The emotions of Joy and distress are direct results of desirable and undesirable events, considering the consequences they have for the agent himself. Some emotions, like for example pity, are triggered when processing events that have consequences for other agents. An open issue is whether this appraisal should be based upon the agent’s own goals or rather the other agent’s goals. How much should an agent be empathic if another one looses something that is not important to the first agent?

In an attempt to solve this issue, abstract goals were introduced (such as for example, not losing property). It eventually became clear that it is very important to keep the goals general and abstract, to avoid having to define too many specific goals. The emotions triggered by reacting to other agents’ good or bad fortune depend on how well-liked they are. Another agent’s bad fortune can trigger pity or gloating, while happy events can result in either feeling ofhappinessor of resentment, depending on the relationship between the agents.

Appraising an event also means evaluating its prospects – hoping or fearing that something will or will not occur. Prospect-based emotions include disappointment and relief. The intensity of these emotions is usually based on the intensity of the preceding hope or fear. The criterion used to appraise the actions of agents is their praiseworthiness, which is based on the agent’s standards. Generally, praiseworthy actions cause pride and blameworthy actions cause shame, if the agent himself is the one acting.

When the actions of other agents are 4 appraised, the emotions triggered are admiration or reproach. Standards can be as complex as attitudes (aspects of objects) and goals (consequences of events), and are almost as subjective and individual. Again, the problem of listing them was solved by describing actions in an abstract way. An interesting phenomenon is the ability of feeling proud or ashamed of someone else’s actions. Simply put, the closer an agent feels related to the acting agent(s), the more he will identify with him in appraising is actions.

Examples of this phenomenon (called the strength of the cognitive unit) can range from parents being proud of their child tosoccerfans being ashamed of their team’s performance. One of the many practical implementations of COCO is the model by Stapler & PETA (1999). They constructed a virtual agent which emotion architecture links discrete emotions categories to 14 action response categories, comprising a large range of individual actions. The COCO emotion model is also partly congruent with Nice Fried’s renewed theory of emotions (Afraid, 1986). For more details on emotion theory, see Trace & Kessler (2003).

Artificial Emotion Engine The aim of the Emotion Engine (E) is to control the behavior of an artificial agent in complex scenarios. It is made of three layers- emotions, mood andpersonality(Wilson, 2000). If an emotion is triggered, the actions will be based on this emotion. When emotions are not triggered, the engine bases its actions on the current mood; when no mood is activated, then personality serves as a basis for behavior. The emotion engine is based on the FEE model, which is a three-dimensional space, describing personality traits in terms of Extroversion, Fear and Aggression.

Within this space, an area around the point representing an artificial agent’s personality is determined and all traits located inside this area are considered to be available to the specific character. For Wilson, the FEE is congruent with the three central systems of the human brain which according to Gray (Gray & McLaughlin, 1996) determine behavior: the Approach system, the Behavior Inhibition system and the Fight/Flight system. These three basic dimensions are intuitive, which makes programming easy. Different personalities trigger some moods more frequently than others: extroversion s linked to good moods, and fear to negative moods.

Aggression affects the speed of mood changes. Reward and punishment signals work as the main inputs, and this is comparable with the desirability of events in COCO. Inputs are adjusted based on personality, but also on how often this input occurred before. An agent can get used to a certain input, and this lowers the impact it will eventually have (habituation). On the contrary, a rare or unprecedented input will have more effect (novelty). Needs are organized hierarchically. Physiological needs, such as hunger, thirst, and the need for warmth and energy are the most important.

Each of these needs can become a priority, as when for example a very hungry agent will consider eating as his most important goal. Safety, affiliation and esteem needs are the remaining layers. While physiological needs are the most important, the order of the other layers can vary, depending on what is more important to the agent. Memory is very limited; an agent only remembers how much he likes the other agents. In the same way, in COCO, sympathy is used to cause different emotions for liked and disliked entities. Only the six basic emotions of fear, anger, Joy, sadness, disgust and surprise can be triggered.

This might appear like a limited selection compared to the 24 emotions of COCO, but given the reactive nature of emotions in this model (working without inner events and 5 triggers) and since some emotion theorists consider the broad spectrum of emotions as mixtures of these basic emotions, this is quite a sensible choice. Personality is used to adjust the intensity or the frequency of the occurrence of emotions, so that a character with personality that is ” low in Fear” will simply not experience as much fear as others.

FLAME The Fuzzy Logic Adaptive Model of Emotion (FLAME) is partially based on COCO, but hat differentiates FLAME from other models is the use of fuzzy logic. This results in a relatively simple appraisal process. FLAME can integrate multiple emotions at the same time (in a process called emotional filtering), as emotions at times inhibit one another. For example, imagine an agent feeling Joy and pride because he Just obtained a new position, but who at the same time feels anger, because a relative of the boss of the company was given a higher position than himself. At this point, his anger may prevent him from feeling joy any longer.

When opposite emotions occur, FLAME lets the stronger emotion inhibit the weaker one(s), giving a slightly stronger weight to negative emotions. Another way to handle conflicting emotions is through mood, which is determined by comparing the intensities of positive and negative emotions over the last few steps. If the summed up intensities of positive emotions are higher than that of the negative emotions, then the mood will be positive. If a positive and a negative emotion of comparable intensities occur at the same time, the mood determines which of these emotions will inhibit the other one.

As there is little research about the decay of emotions, FLAME uses a simple constant cay, though positive emotions decay faster than negative emotions. FLAME does not make it possible to implement an agent’s personality; instead, differences in behavior are created through learning. For example, an agent may learn that reacting in an angry way will enable him to reach his goals, thus enticing him to be more choleric. FLAME implements multiple types of learning, such as classical conditioning (associating expectations with objects) which occurs in many situations, triggering fear or hope.

Another type of learning is learning about consequences of actions or events. This is simple whenever an action directly causes a result. For example, learning that eating will result in feeling less hungry is rather trivial. In the case of more complex causal relations over time, FLAME is using Q-learning, a form of reinforcement learning. Another form of learning, quite similar to model learning, is the ability to recognize patterns in the behavior of a user by observing sequences of actions. For this type of learning, FLAME simply counts the occurrences of sequences.

The last type of learning in FLAME, but one of the most important, is learning about the value of actions. Remember that COCO relies on the praiseworthiness of actions, which is based on the agent’s standards. In FLAME, these standards are not predefined knowledge, but they are learned from the interaction between users. Using learning instead of predefined knowledge seems like a very sensible way to avoid most of the troubling issues that come with using COCO. Additionally, learning allows agents to adjust, which makes them all the more believable.

ALMA The intention in designing A Layered Model of Affect (ALMA) was to control agents in conversational scenarios. In interactive game or learning environments, the artificial harassers display facial expressions of emotions and moods through their postures to 6 appear more believable. Emotions, moods and personalities are implemented and interact with each other. Events and actions are described in terms of abstract tags which are then evaluated during the appraisal process and describe things like for example the expressed emotion or gesture accompanying an action or simply if something is a good or bad event.

As ALMA is aimed at conversations, an action is often a statement. Hence, there are tags to describe the kind of statement, for example if it was an insult or a compliment. In addition, ALMA requires defining personality profiles for each agent. Essentially, these profiles already contain the desirability and praiseworthiness the agent assigns to certain tags. Since our own emotion model shares some features with ALMA (see below) a key difference should be pointed out. In SIMPLEX we considered it impractical to explicitly specify this information, as this would have limited the model to a small number of agents.

So instead of using tags, our model requires to specify goals and their priorities for an agent, where generic goals can be used for all agents. Events still need to be scribed in a special way, but this is reduced too relatively objective list of which agents goals are affected and in which way. All other information like praiseworthiness is automatically derived from this and the agent’s personality. Although this approach is providing less control over an agent’s appraisal process, it is better suited for a generic system meant to be used with minimal extra effort. 3. SIMPLEX – Simulation of Personal Emotion Experience 3. Overview SIMPLEX is a context-independent module to create emotions as a result of primary application (environment) events. Goals, emotions, mood-states, personality, memory and relationships between agents have been modeled so they could interact as in real life. Figure 2 shows an overview of the model. SIMPLEX is based on the COCO model by Retort, Color and Collins (1988) in that it creates discrete emotions by appraising events based on the desirability of their consequences and the praiseworthiness of the actions of agents. The appraisal process was modified by including the personality of virtual agents.

The personality component is based on the Five Factor Model (FM) introduced by psychologists McCrae & Costa (1987), which includes extroversion, conscientiousness, agreeableness, neurotics and openness. The personality module influences the emotion module on multiple levels during appraisal processes and in the development of mood-states. Other important aspects of the model are mood-states and relationships. Mood- states are represented in a three-dimensional space which dimensions are pleasure, arousal and dominance (Bradley & Lang, 1994), and they are based on active or recently experienced emotions (implemented by pull-functions).

In the absence of motions, a mood state will slowly gravitate back to a default mood-state based on the agent’s personality. A mood-state also functions as a threshold to determine whether an emotion is strong enough to become active at a given time. Relationships are handled as if they were mood-states towards other agents (for instance a player in a game scenario): they are based on emotions caused by other agents and they can be considered as a simplified way to store memories of experiences with these agents.

They are used as thresholds as well; for example, an agent will be more likely to become angry at another agent towards when their legislation is in the range of negative valence. 7 Fig. 2. The emotion module SIMPLEX Personality (long-term), mood-state (mid-term) and emotions (short-term) thus represent three levels of the emotion module that interact with each other in order to create believable agents. Events from the scenario serve as the model’s inputs. They are appraised according to the COCO algorithm (see figure 1).

This appraisal is influenced by the agent’s goals, his personality and his relationships with other agents. At the end of an appraisal one or several discrete emotions are generated. These emotions and the current mood-state are represented in the same three- dimensional PAD space: on the one hand, the emotion(s) serve(s) as an attractor for the recent mood-state position (pull function). On the other hand, the closer an emotion is located to the current mood-state, the more probable it will be that the emotion will be activated.

The speed at which the mood-state changes, is influenced by the agent’s neurotics (a personality variable). Additionally, emotions that are caused by other agents will influence another mood-state representation (stored on another PAD space) representing the relationship with that agent. Thus, every agent has specific relationships with other agents, which influences his behavior towards others. Emotions, mood-states and relationships with other agents are the outputs of the model and can be used by the AAA application.

Originally, the PAD space was designed to represent emotions in a dimensional rather than a discrete way (Russell, 1978). In our model, PAD is used as a common space where three different constructs (discrete COCO emotions, continuous mood- states and personality), are represented in order to be handled together by the SIMPLEX algorithm. An agent’s current mood-state is thus the result of a mathematical function which takes into account the default mood (defined by personality), the pulling behavior of COCO emotion(s) triggered by appraisals, and weighed factors influencing movement speed (see equation 1).

Mood-state = f(PADDED, Paternosters, Filter) 8 3. 2 Basic components Mood-state represented in the PAD-Space (Pleasure-Arousal-Dominance) Beyond discrete emotions, which are typically short-term, mood-states are a powerful way to model emotional shifts and explain affective influences over longer periods of time. To implement mood-states in our model, we chose to use Russell three-dimensional space to describe emotions (Russell, 1978) and Meridian’s concept of how emotions are linked to personality traits (Meridian, 1996).

The dimension of Pleasure encompasses valence ranging from very positive to very negative. Arousal is an indicator of how intensely something is perceived, or of how much it affects the organism. Dominance is a measure of experienced control over the situation. For example, a different degree of dominance can make the difference between fear and anger. Both of these emotions are states of negative valence and sigh arousal, but not feeling in control is what differentiates fear from anger. When an agent is angry, it is because he believes he can have a potential influence.

Although emotions are triggered by COCO appraisals and are therefore discrete, they are handled in a continuous three-dimensional space by SIMPLEX. The advantage of treating emotions in this way and not Just as a fixed set of possible emotions is that it makes it possible to represent emotions that do not even have a name. It also creates the possibility to combine emotions, mood-state and personality in one space. First, a ordinate in PAD space can obviously represent an agent’s mood-state. But emotions and personalities can also be described in terms of Pleasure, Arousal and Dominance values.

For example, the value of arousal can be not only the degree of arousal associated with a specific emotion, but also the restorability of a person. Meridian (1996) gives specific names to the resulting different octants in PAD-space and describes the diagonally opposite octants as Exuberant/Bored, Dependent/ Disdainful, Relaxed/Anxious, Docile/Hostile. Thus mood-states are not points but octants of the Bedspread. However, positioning a personality (based on FM) within a PAD-space could have been a rather difficult task, since there is no mathematically- correct way to make the conversion.

Luckily, this transformation can be based upon empirical data. Meridian provided such a conversion table from FM to PAD after correlation analyses of questionnaires measuring both constructs in healthy subjects (Meridian, 1996). Five Factor Model of Personality (FM) The implementation of personality is a key factor when creating believable agents that differ from each other. COCO already offers a few possibilities: different goals, tankards and attitudes automatically result in differences during the appraisal process.

However, since personality goes beyond preferences, it was necessary to find a model of personality that made it possible to adjust the appraisal process, to shift the agent’s perception and to influence mood-states. The model chosen for SIMPLEX was the Five Factor Model (McCrae & Costa, 1987). After years of research, an agreement emerged that five groups of traits are sufficient to describe a personality. Using self-report questionnaires with multiple items, a personality profile can be provided for each individual scoring high or low in each of he five factors (this approach is called ” dimensional”).

In the case of our model, the value for each factor can be typed in when defining the artificial agent. 9 Agreeableness refers to a tendency to cooperate and to compromise, in order to interact with others in an agreeable way. High agreeableness often means having a positive outlook on human nature, assuming people to be good rather than bad. Low agreeableness is essentially selfishness, putting your own needs above the needs of others and not caring about the consequences your actions might have for others. Conscientiousness is usually high in people who plan a lot, who think everything through, and who are very tidy or achievers.

Extreme cases can appear to be compulsive or pedantic. The opposite personality trait includes sloppiness or ignoring one’s duties. Extroversion can be a measure of how much people experience positive emotions. An enthusiastic and active person that enjoys company and attention is extroverted, while a quiet individual who needs to spend more time alone is introverted. Neurotics is partly an opposite of Extroversion in being a tendency to experience negative emotions. However, being neurotic also means being more sensitive in general, and reacting emotionally to unimportant events that wouldn’t usually trigger a response.

Neurotics can be prone to mood swings and tend to be more negative in their interpretation of situations. Low neurotics means high emotional stability and describes calm people who are not easily upset. Finally, those scoring high on Openness to Experience are creative and curious individuals, interested in art and more in touch with their own emotions than others. Those scoring low on that dimension are conservative persons with few interests, hey prefer straight and simple things rather than fancy ones, and they do not care about art orscience.

It is suspected that Openness can be influenced byeducation. 3. 3 Technical implementation The appraisal process and the generation of emotions There are three categories of inputs to the appraisal process of the emotion model: consequences of events, actions of agents and objects (see the COCO model in figure 1). The following section will describe the respective mechanisms applied when mapping each type of input to emotions. Each event handled by a character is first adjusted according to the agent’s rationality.

First, the consequences are adjusted based on the agent’s neurotics. As neurotic people tend to see things more negatively, consequences are rated worse than what they actually are. The factor by which neurotics can reduce the desirability of events is adjustable. Note that all personality traits are in the range [-1; 1], so that negative neurotics actually makes consequences more positive. In real life, positive people could think ” it could have been worse”. The desirability of events is determined by (predefined) goals during the event appraisal.

A goal consists of two aspects: relevance [O; 1] and state of realization [O; 1], which means to which percentage the goal is already achieved. Afterwards, the praiseworthiness of actions is determined. Basically, the more positive consequences an action has, the more praiseworthy it is considered to be. Sympathy plays a role in this process, as it is added to positive values and subtracted from negative ones. Consequences for self are considered to be more important than consequences for others, which are currently factored in at 50% of their value. 0 After the adjusted values for all consequences have been summed up, unconsciousness is used to obtain the final result, by being scaled and subtracted. Thus the more conscientious an agent is, the harder it will be to commit an action positive enough to be deemed praiseworthy. This applies to both actions of other agents and actions of the agent himself. Agreeableness works the opposite way, but only for the actions of others. This is based on the psychological notion that agreeable people tend to be more forgiving in order to get along with others.

Apart from having a different weight, factoring in agreeableness has the same results as active conscientiousness. The remaining factors serving as parameters for the action (responsibility, unexpectedness, publicizes) are averaged and used to scale the result of the above calculations. Finally, as cost is attempted to be derived from consequences for self, it is subtracted, before the calculated praiseworthiness is averaged over the number of consequences or rather the number of affected agents. The resulting value of praiseworthiness is used as the intensity for admiration or reproach, depending on whether it is positive or negative.

If the agent is appraising his own actions, the motions are pride or shame instead of admiration and reproach. Once the praiseworthiness has been calculated, a search is conducted through the list of prospects for all the ones that are active and that match the name of the event. For each, the prospect appraisal function is called, which determines the net desirability by multiplying it with the affected goal’s relevance. This value will be compared to the expected desirability for this event. The simplest situation is when a positive consequence was expected but a negative one occurs.

This would obviously cause disappointment. However, this is also the case if a very high desirability was hoped for and the actual consequences are less positive, but still not negative. Having a hope fulfilled results in satisfaction. If an event has exactly the expected consequences, it results in the full intensity for the emotion. The intensity of emotions is the product of the determined quality of the event and of the intensity of the prospects. For example, if there was very little hope, there cannot be strong satisfaction. Which emotion is created depends on the kind of prospect and on the sign of the quality value.

Hope and positive quality result in satisfaction, hope and negative quality in disappointment, fear and positive quality in fears-confirmed and fear and negative quality in relief. After the prospect appraisal is done, short term or one-shot prospects (only valid for one round) are removed. Appraisal concerning Joy and distress is done for each consequence affecting the agent himself, while appraisal for pity/gloating and happy-for/resentment is done for the remaining consequences.