Exploring the emotional expression of AMRs

Exploring the emotional expression of AMRs

In this study, I explored methods for emotional expression by a non-humanoid mobility robot and designed various emotional expressions based on the dimensional theory of emotion. I presented these designed emotional expressions to individuals and examined the differences in their characteristics. Additionally, we investigated the variations resulting from different combinations of emotional expressions and explored the correlation between the application context of the robot and emotional expression

In this study, I explored methods for emotional expression by a non-humanoid mobility robot and designed various emotional expressions based on the dimensional theory of emotion. I presented these designed emotional expressions to individuals and examined the differences in their characteristics. Additionally, we investigated the variations resulting from different combinations of emotional expressions and explored the correlation between the application context of the robot and emotional expression

In this study, I explored methods for emotional expression by a non-humanoid mobility robot and designed various emotional expressions based on the dimensional theory of emotion. I presented these designed emotional expressions to individuals and examined the differences in their characteristics. Additionally, we investigated the variations resulting from different combinations of emotional expressions and explored the correlation between the application context of the robot and emotional expression

Group / Individual

Group / Individual

Group / Individual

Individual Project

Individual Project

Individual Project

Location

Location

Location

Koreatech (Cheonan, South Korea)

Koreatech (Cheonan, South Korea)

Koreatech (Cheonan, South Korea)

Course

Course

Course

Master Thesis (Supervisor : Joo Young Jung)

Master Thesis (Supervisor : Joo Young Jung)

Master Thesis (Supervisor : Joo Young Jung)

Duration

Duration

Duration

2022. 1 ~ 2023. 6 (18 months)

2022. 1 ~ 2023. 6 (18 months)

2022. 1 ~ 2023. 6 (18 months)

Research keywords

Research keywords

Research

keywords

In Human-Robot Interaction research, emotion models are utilized as criteria for categorizing or quantifying the emotions intended to be expressed. Among them, dimensional emotion models are often used, asserting that emotions are not independent but rather composed of combinations of various dimensions or components. One of the dimensional emotion models, the circumplex model (Russell, 1980), is convenient for quantifying the perceived intensity of emotions in individuals.


In the circumplex model, emotions are composed of two dimensions: Valence and Activation.

Valence:  The degree of negativity (negative, unpleasant) and positivity (positive, pleasant)

Activation: analyzed in energy, tension, activity, and arousal

In Human-Robot Interaction research, emotion models are utilized as criteria for categorizing or quantifying the emotions intended to be expressed. Among them, dimensional emotion models are often used, asserting that emotions are not independent but rather composed of combinations of various dimensions or components. One of the dimensional emotion models, the circumplex model (Russell, 1980), is convenient for quantifying the perceived intensity of emotions in individuals.


In the circumplex model, emotions are composed of two dimensions: Valence and Activation.

Valence:  The degree of negativity (negative, unpleasant) and positivity (positive, pleasant)

Activation: analyzed in energy, tension, activity, and arousal

Research questions

Research questions

Research

questions

Upon reviewing various studies in the field of Human-Robot Interaction (HRI), it was found that the majority of research has been actively conducted on robots with human-like characteristics such as facial expressions and arms. However, research on robots lacking human characteristics has been significantly lacking. Moving beyond human characteristics provides designers with more freedom, allowing for greater diversification in robot design. Consequently, in this study, five research questions were selected to explore emotion expression in robots devoid of human characteristics

Upon reviewing various studies in the field of Human-Robot Interaction (HRI), it was found that the majority of research has been actively conducted on robots with human-like characteristics such as facial expressions and arms. However, research on robots lacking human characteristics has been significantly lacking. Moving beyond human characteristics provides designers with more freedom, allowing for greater diversification in robot design. Consequently, in this study, five research questions were selected to explore emotion expression in robots devoid of human characteristics

Surveying literatures

Surveying literatures

Surveying

literatures

Research in the field of Human-Robot Interaction (HRI) has spanned across various domains. To investigate the signal methods utilized in electronic devices and robots, a review of papers in the fields of HRI and HCI was conducted. Additionally, to acquire foundational knowledge on human emotion recognition, papers in psychology, cognitive engineering, emotion theory, and color research were reviewed.

Research in the field of Human-Robot Interaction (HRI) has spanned across various domains. To investigate the signal methods utilized in electronic devices and robots, a review of papers in the fields of HRI and HCI was conducted. Additionally, to acquire foundational knowledge on human emotion recognition, papers in psychology, cognitive engineering, emotion theory, and color research were reviewed.

Emotional expressions applicable to

robots devoid of human characteristics

Emotional expressions applicable to robots devoid of human characteristics

When summarizing the results of the literature review, emotion expressions applicable to robots devoid of human characteristics were identified as movement, light, sound, and vibration. However, conveying diverse emotion expressions efficiently through vibration proves challenging unless the robot involves direct physical contact. Therefore, excluding vibration, the focus of the research was placed on the remaining three elements

When summarizing the results of the literature review, emotion expressions applicable to robots devoid of human characteristics were identified as movement, light, sound, and vibration. However, conveying diverse emotion expressions efficiently through vibration proves challenging unless the robot involves direct physical contact. Therefore, excluding vibration, the focus of the research was placed on the remaining three elements

Exploring methods of emotional expression

applicable to robots devoid of human

characteristics : Movement

Exploring methods of emotional expression applicable to robots devoid of human characteristics

Movement

Exploring methods of emotional expression

applicable to robots devoid of human

characteristics : Movement

As a result of synthesizing literature, it has been confirmed that expressing positive emotions can be achieved through forward or rotating movements towards the user, while negative emotions can be conveyed through backward or outward rotations. There is an exception where forward movements can occasionally express the negative emotion of 'anger.' Furthermore, the speed of movement determines the activation level of emotions.

As a result of synthesizing literature, it has been confirmed that expressing positive emotions can be achieved through forward or rotating movements towards the user, while negative emotions can be conveyed through backward or outward rotations. There is an exception where forward movements can occasionally express the negative emotion of 'anger.' Furthermore, the speed of movement determines the activation level of emotions.

A total of 24 movement patterns were designed for evaluation purposes. Following Butler and Agah(2001), where the robot's speed causing discomfort to humans is identified as 1 m/s, the fastest speed was set at 1 m/s, considering concerns about the potential impact on the intensity of negative and positive emotions when moving faster than 1 m/s. Three speeds, occurring at one-second intervals, were also set. Additionally, for the rotation method, angular differences were incorporated, setting rotations at 90 degrees and 180 degrees

A total of 24 movement patterns were designed for evaluation purposes. Following Butler and Agah(2001), where the robot's speed causing discomfort to humans is identified as 1 m/s, the fastest speed was set at 1 m/s, considering concerns about the potential impact on the intensity of negative and positive emotions when moving faster than 1 m/s. Three speeds, occurring at one-second intervals, were also set. Additionally, for the rotation method, angular differences were incorporated, setting rotations at 90 degrees and 180 degrees

Exploring methods of emotional expression

applicable to robots devoid of human

characteristics : Light

Exploring methods of emotional expression

applicable to robots devoid of human

characteristics

Light

Exploring methods of emotional expression

applicable to robots devoid of human

characteristics : Light

Color is composed of Hue, Lightness, and Chroma (Fairchild, 2013), and these elements influence human psychological functions (Camgoz et al., 2004). The applied LED in the product is equipped with numerous pixels, allowing the expression of various light patterns such as blinking or breathing. In this study, we focused on pre-existing research on Hue and light patterns, which users can relatively easily perceive changes. Drawing on papers from color studies by Goldstein (1942), Nakashian (1964), among others, we investigated the emotional vocabulary associated with each color. Additionally, based on Terada et al. (2012), we discovered the impact of temporal changes in light, such as blinking or breathing, on differences in emotion recognition

Color is composed of Hue, Lightness, and Chroma (Fairchild, 2013), and these elements influence human psychological functions (Camgoz et al., 2004). The applied LED in the product is equipped with numerous pixels, allowing the expression of various light patterns such as blinking or breathing. In this study, we focused on pre-existing research on Hue and light patterns, which users can relatively easily perceive changes. Drawing on papers from color studies by Goldstein (1942), Nakashian (1964), among others, we investigated the emotional vocabulary associated with each color. Additionally, based on Terada et al. (2012), we discovered the impact of temporal changes in light, such as blinking or breathing, on differences in emotion recognition

The image on the left provides an example of brightness variation graphs for

the blinking (top) and breathing (bottom) patterns,

altering the light intensity at a 500ms interval

The image on the left provides an example of brightness variation graphs for the blinking (top) and breathing (bottom) patterns, altering the light intensity at a 500ms interval

The image on the left provides an example of brightness variation graphs for the blinking (top) and breathing (bottom) patterns, altering the light intensity at a 500ms interval

Emotional expression through light utilized a total of 48 expressions,

incorporating 7 colors, 2 patterns (blinking, breathing),

and 4 durations of brightness variations

Emotional expression through light utilized a total of 48 expressions, incorporating 7 colors, 2 patterns (blinking, breathing), and 4 durations of brightness variations

Emotional expression through light utilized a total of 48 expressions, incorporating 7 colors, 2 patterns (blinking, breathing), and 4 durations of brightness variations

Exploring methods of emotional expression

applicable to robots devoid of human

characteristics : Sound

Exploring methods of emotional expression

applicable to robots devoid of human

characteristics

Sound

Exploring methods of emotional expression

applicable to robots devoid of human

characteristics : Sound

As the robot utilized in the study has a simple form, it was deemed suitable to employ straightforward sounds with minimal musical elements. Therefore, only pitch and duration were utilized. Considering that a mismatch between the robot's form and user expectations can lead to discomfort and unnatural emotional conveyance (Read and Belpaeme, 2016), the design was inspired by Komatsu (2005), adjusting pitch and duration. To simulate a simple beep-like sound, the timbre was set as a sine wave, and the sounds were produced using the audio editing program Audacity

Making the robot for evaluation

Making the robot for evaluation

Making the robot for evaluation

Utilizing Arduino and Appinventor, a robot for evaluation, capable of expressing the previously designed movements, light, and sound, was created

Environment & Process of evaluation

Environment &

Process of evaluation

Environment & Process of evaluation

To create an evaluation environment where participants can observe emotion expressions up close, the robot was placed on a large desk measuring 75cm. Except for forward and backward movements, the robot displayed emotion expressions at a distance of 50cm. Forward and backward movements were designed to cover a distance of 1m

To create an evaluation environment where participants can observe emotion expressions up close, the robot was placed on a large desk measuring 75cm. Except for forward and backward movements, the robot displayed emotion expressions at a distance of 50cm. Forward and backward movements were designed to cover a distance of 1m

To create an evaluation environment where participants can observe emotion expressions up close, the robot was placed on a large desk measuring 75cm. Except for forward and backward movements, the robot displayed emotion expressions at a distance of 50cm. Forward and backward movements were designed to cover a distance of 1m

Interview questions

Interview questions

To complement the emotion recognition data collected based on the circumplex model, additional interviews were conducted for comparison with the participants' responses

To complement the emotion recognition data collected based on the circumplex model, additional interviews were conducted for comparison with the participants' responses

Analysis method for evaluation results

Analysis method for evaluation results

Analysis method for evaluation results

  1. Examining trends in valence and activation level differences:

    • Placing the average responses of valence and activation levels on a graph with X-axis(valence) and Y-axis(activation)

  1. Examining trends in valence and activation level differences:

    • Placing the average responses of valence and activation levels on a graph with X-axis(valence) and Y-axis(activation)

  1. Examining trends in valence and activation level differences:

    • Placing the average responses of valence and activation levels on a graph with X-axis(valence) and Y-axis(activation)

  1. Checking for differences in valence and activation levels among different expression methods:

    • Performing one-way ANOVA and Tukey post-hoc tests

    • Paired-sample T-tests (comparing blinking and breathing light)

  1. Checking for differences in valence and activation levels among different expression methods:

    • Performing one-way ANOVA and Tukey post-hoc tests

    • Paired-sample T-tests (comparing blinking and breathing light)

  1. Checking for differences in valence and activation levels among different expression methods:

    • Performing one-way ANOVA and Tukey post-hoc tests

    • Paired-sample T-tests (comparing blinking and breathing light)

  1. Examining the correlation between variables constituting emotional expressions and valence, activation levels:

    • Conducting Pearson correlation analysis

      1. Movement : Speed of movement and valence, activation

      2. Light : Brightness changing speed and valence, activation

      3. Sound : Duration of sound and valence, activation

  1. Examining the correlation between variables constituting emotional expressions and valence, activation levels:

    • Conducting Pearson correlation analysis

      1. Movement : Speed of movement and valence, activation

      2. Light : Brightness changing speed and valence, activation

      3. Sound : Duration of sound and valence, activation

  1. Examining the correlation between variables constituting emotional expressions and valence, activation levels:

    • Conducting Pearson correlation analysis

      1. Movement : Speed of movement and valence, activation

      2. Light : Brightness changing speed and valence, activation

      3. Sound : Duration of sound and valence, activation

  1. Analysis of interview responses:

    • Grouping similar responses and calculating the frequency of each group

  1. Analysis of interview responses:

    • Grouping similar responses and calculating the frequency of each group

  1. Analysis of interview responses:

    • Grouping similar responses and calculating the frequency of each group

  1. Deriving the top 3 emotional vocabulary terms closest to emotional expressions:

  • Calculating the Euclidean distance between the average response coordinates for emotion word and emotional expressions

  1. Deriving the top 3 emotional vocabulary terms closest to emotional expressions:

  • Calculating the Euclidean distance between the average response coordinates for emotion word and emotional expressions

  1. Deriving the top 3 emotional vocabulary terms closest to emotional expressions:

  • Calculating the Euclidean distance between the average response coordinates for emotion word and emotional expressions

Response results for

emotional vocabulary

(Average of 30 participants)

Response results for

emotional vocabulary

(Average of 30 participants)

Response results for emotional vocabulary

(Average of 30 participants)

Analysis results of responses : Movement

Analysis results of responses : Movement

Analysis results of responses : Movement

  • Forward and rotational movements toward the user are perceived positively, with a higher degree of valence

  • Backward and rotational movements away from the user are perceived negatively, with a lower degree of valence

  • Correlation analysis between movement time and activation indicates a positive correlation between movement speed and activation level

  • Approximately 30% of participants (9 individuals) perceived the forward movement (1s) negatively.

    1. The emotion of 'anger' is sometimes linked to the motivation for approach (Carver, 2004; Harmon-Jones, 2003)

    2. The speed causing discomfort to humans in a robot is 1 m/s (Butler and Agah).

    3. If the robot moves forward faster than 1 m/s, there is a high likelihood that users will perceive it negatively

  • The forward and backward movements showed higher activation than rotational movements.

    1. 8 participants responded that the forward and backward movements were more activating and evoked stronger emotions compared to rotation

    2. Participants felt more activated and intense emotions in the order of forward movement, 180-degree rotation, and 90-degree rotation.

    3. Some participants expressed that a 90-degree rotation could be calming if angry, but a 180-degree rotation seemed to induce intense anger

    4. Lee et al.(2007) argue that the activation dimension of emotion is not only related to the speed of movement but also correlated with its size

  • There is a positive correlation between the valence levels and the speed of backward movement and 180-degree rotation (the faster, the more negatively perceived).

    1. The speed of movement may influence the pleasure-displeasure levels when expressed negatively

  • Forward and rotational movements toward the user are perceived positively, with a higher degree of valence

  • Backward and rotational movements away from the user are perceived negatively, with a lower degree of valence

  • Correlation analysis between movement time and activation indicates a positive correlation between movement speed and activation level

  • Approximately 30% of participants (9 individuals) perceived the forward movement (1s) negatively.

    1. The emotion of 'anger' is sometimes linked to the motivation for approach (Carver, 2004; Harmon-Jones, 2003)

    2. The speed causing discomfort to humans in a robot is 1 m/s (Butler and Agah).

    3. If the robot moves forward faster than 1 m/s, there is a high likelihood that users will perceive it negatively

  • The forward and backward movements showed higher activation than rotational movements.

    1. 8 participants responded that the forward and backward movements were more activating and evoked stronger emotions compared to rotation

    2. Participants felt more activated and intense emotions in the order of forward movement, 180-degree rotation, and 90-degree rotation.

    3. Some participants expressed that a 90-degree rotation could be calming if angry, but a 180-degree rotation seemed to induce intense anger

    4. Lee et al.(2007) argue that the activation dimension of emotion is not only related to the speed of movement but also correlated with its size

  • There is a positive correlation between the valence levels and the speed of backward movement and 180-degree rotation (the faster, the more negatively perceived).

    1. The speed of movement may influence the pleasure-displeasure levels when expressed negatively

  • Forward and rotational movements toward the user are perceived positively, with a higher degree of valence

  • Backward and rotational movements away from the user are perceived negatively, with a lower degree of valence

  • Correlation analysis between movement time and activation indicates a positive correlation between movement speed and activation level

  • Approximately 30% of participants (9 individuals) perceived the forward movement (1s) negatively.

    1. The emotion of 'anger' is sometimes linked to the motivation for approach (Carver, 2004; Harmon-Jones, 2003)

    2. The speed causing discomfort to humans in a robot is 1 m/s (Butler and Agah).

    3. If the robot moves forward faster than 1 m/s, there is a high likelihood that users will perceive it negatively

  • The forward and backward movements showed higher activation than rotational movements.

    1. 8 participants responded that the forward and backward movements were more activating and evoked stronger emotions compared to rotation

    2. Participants felt more activated and intense emotions in the order of forward movement, 180-degree rotation, and 90-degree rotation.

    3. Some participants expressed that a 90-degree rotation could be calming if angry, but a 180-degree rotation seemed to induce intense anger

    4. Lee et al.(2007) argue that the activation dimension of emotion is not only related to the speed of movement but also correlated with its size

  • There is a positive correlation between the valence levels and the speed of backward movement and 180-degree rotation (the faster, the more negatively perceived).

    1. The speed of movement may influence the pleasure-displeasure levels when expressed negatively

Analysis results of responses : Light

Analysis results of responses : Light

Analysis results of responses : Light

Red

  • Due to the universal association of red with negativity, participants intuitively perceived it negatively (16 out of 30)​​​​​​​

  • Red is considered suitable for expressing definite negative emotional expressions

Orange

  • Responses indicated that the color was mostly perceived as less negative than red, often associated with a mild and non-urgent warning.

  • Some participants felt positive emotions, such as warmth and excitement, due to its warm color

  • Opinions varied, with some finding it calm during slow brightness changes and alerting during rapid changes​​​​​​​

Yellow

  • The Valence levels generally exhibited a coexistence of positive and negative tendencies

  • When perceived negatively, participants felt a mild alertness compared to orange and associated warm colors with negativity

  • Positive responses were linked to words like spring, flowers, brightness, liveliness, chicks, cuteness, and excitement

Green

  • Most participants associated the color with the meaning of 'go' in traffic lights (8 out of 30).

  • All collected keywords were positive, suggesting green is suitable for positive emotional expressions

Blue

  • The most common reason for positively responding to blue was personal preference for the color (6 participants)

  • Unlike green, two participants associated negative keywords like 'depression' with blue

Pink

  • Similar to yellow, both negative and positive responses coexisted for blue, with more diverse opinions than yellow

  • Participants who felt negatively could not provide specific explanations

  • Positive responses varied widely, with individual preferences for gaming, clubs, love, etc

Brightness changing speed & Light pattern

  • As the brightness change speed increased, the activation level of emotions also increased.

  • Participants responded that blinking is stronger, more mechanical, and suitable for expressing urgency

  • While breathing is more natural, stable, and relaxing compared to blinking

Red

  • Due to the universal association of red with negativity, participants intuitively perceived it negatively (16 out of 30)​​​​​​​

  • Red is considered suitable for expressing definite negative emotional expressions

Orange

  • Responses indicated that the color was mostly perceived as less negative than red, often associated with a mild and non-urgent warning.

  • Some participants felt positive emotions, such as warmth and excitement, due to its warm color

  • Opinions varied, with some finding it calm during slow brightness changes and alerting during rapid changes​​​​​​​

Yellow

  • The Valence levels generally exhibited a coexistence of positive and negative tendencies

  • When perceived negatively, participants felt a mild alertness compared to orange and associated warm colors with negativity

  • Positive responses were linked to words like spring, flowers, brightness, liveliness, chicks, cuteness, and excitement

Green

  • Most participants associated the color with the meaning of 'go' in traffic lights (8 out of 30).

  • All collected keywords were positive, suggesting green is suitable for positive emotional expressions

Blue

  • The most common reason for positively responding to blue was personal preference for the color (6 participants)

  • Unlike green, two participants associated negative keywords like 'depression' with blue

Pink

  • Similar to yellow, both negative and positive responses coexisted for blue, with more diverse opinions than yellow

  • Participants who felt negatively could not provide specific explanations

  • Positive responses varied widely, with individual preferences for gaming, clubs, love, etc

Brightness changing speed & Light pattern

  • As the brightness change speed increased, the activation level of emotions also increased.

  • Participants responded that blinking is stronger, more mechanical, and suitable for expressing urgency

  • While breathing is more natural, stable, and relaxing compared to blinking

Red

  • Due to the universal association of red with negativity, participants intuitively perceived it negatively (16 out of 30)​​​​​​​

  • Red is considered suitable for expressing definite negative emotional expressions

Orange

  • Responses indicated that the color was mostly perceived as less negative than red, often associated with a mild and non-urgent warning.

  • Some participants felt positive emotions, such as warmth and excitement, due to its warm color

  • Opinions varied, with some finding it calm during slow brightness changes and alerting during rapid changes​​​​​​​

Yellow

  • The Valence levels generally exhibited a coexistence of positive and negative tendencies

  • When perceived negatively, participants felt a mild alertness compared to orange and associated warm colors with negativity

  • Positive responses were linked to words like spring, flowers, brightness, liveliness, chicks, cuteness, and excitement

Green

  • Most participants associated the color with the meaning of 'go' in traffic lights (8 out of 30).

  • All collected keywords were positive, suggesting green is suitable for positive emotional expressions

Blue

  • The most common reason for positively responding to blue was personal preference for the color (6 participants)

  • Unlike green, two participants associated negative keywords like 'depression' with blue

Pink

  • Similar to yellow, both negative and positive responses coexisted for blue, with more diverse opinions than yellow

  • Participants who felt negatively could not provide specific explanations

  • Positive responses varied widely, with individual preferences for gaming, clubs, love, etc

Brightness changing speed & Light pattern

  • As the brightness change speed increased, the activation level of emotions also increased.

  • Participants responded that blinking is stronger, more mechanical, and suitable for expressing urgency

  • While breathing is more natural, stable, and relaxing compared to blinking

Analysis results of responses : Sound

Analysis results of responses : Sound

Analysis results of responses : Sound

Pitch variation

  • Increasing pitch was perceived as positive, decreasing pitch and steady tones were perceived as negative.

  • Increasing sound was associated with uplifted mood, excitement, positive curiosity, rising intonation at the end of sentences(when in a good mood), and positive reactions.

  • Decreasing sound was linked to disappointment, gloominess, and sighing.

  • Steady tones were described as mechanical, warning sounds, cardiac arrest, failure-related sounds, and emotionless

Duration

  • The correlation analysis results indicated a positive correlation between increasing duration and activation levels, but participants could not provide specific reasons for this trend.

  • For rising tones, there was a positive correlation between longer duration and increased valence levels.

  • Conversely, for falling tones and steady tones, there was a negative correlation between longer duration and decreased valence levels

Pitch variation

  • Increasing pitch was perceived as positive, decreasing pitch and steady tones were perceived as negative.

  • Increasing sound was associated with uplifted mood, excitement, positive curiosity, rising intonation at the end of sentences(when in a good mood), and positive reactions.

  • Decreasing sound was linked to disappointment, gloominess, and sighing.

  • Steady tones were described as mechanical, warning sounds, cardiac arrest, failure-related sounds, and emotionless

Duration

  • The correlation analysis results indicated a positive correlation between increasing duration and activation levels, but participants could not provide specific reasons for this trend.

  • For rising tones, there was a positive correlation between longer duration and increased valence levels.

  • Conversely, for falling tones and steady tones, there was a negative correlation between longer duration and decreased valence levels

Pitch variation

  • Increasing pitch was perceived as positive, decreasing pitch and steady tones were perceived as negative.

  • Increasing sound was associated with uplifted mood, excitement, positive curiosity, rising intonation at the end of sentences(when in a good mood), and positive reactions.

  • Decreasing sound was linked to disappointment, gloominess, and sighing.

  • Steady tones were described as mechanical, warning sounds, cardiac arrest, failure-related sounds, and emotionless

Duration

  • The correlation analysis results indicated a positive correlation between increasing duration and activation levels, but participants could not provide specific reasons for this trend.

  • For rising tones, there was a positive correlation between longer duration and increased valence levels.

  • Conversely, for falling tones and steady tones, there was a negative correlation between longer duration and decreased valence levels

Designing the multimodal

emotional expressions

(Preparation for evaluations

regarding research questions 3, 4 and 5)

Designing the multimodal

emotional expressions

(Preparation for evaluations

regarding research questions 3, 4 and 5)

Designing the multimodal emotional expressions

(Preparation for evaluations regarding research questions 3, 4 and 5)

After completing evaluations related to research questions 1 and 2, we designed multimodal emotional expressions combining various emotional displays to proceed with assessments related to research questions 3, 4, and 5

After completing evaluations related to research questions 1 and 2, we designed multimodal emotional expressions combining various emotional displays to proceed with assessments related to research questions 3, 4, and 5

Selection of evaluation variables related to personalities and spaces

Selection of evaluation variables related to personalities and spaces

Selection of evaluation variables related to personalities and spaces

To examine the correlation between personality and the level of valence and activation a survey tool based on the Likert 5-point scale assessing introverted-extroverted personality traits was utilized

To explore which types of emotional expressions are suitable for different environments, nine spaces were selected. These spaces are composed of environments that people can easily imagine based on their activity levels and noise levels as criteria for categorization

The spaces were initially categorized by the researcher based on the level of human activity and noise. However, to determine a series of rankings for each space regarding their levels of activity and noise, an Analytic Hierarchy Process (AHP) was employed. Through this process, relative weights for activity and noise in each space were clearly defined and compared. These weights were later utilized to analyze the correlation with the activation level of emotional expressions in the study

Evaluation Process

Evaluation Process

Evaluation Process

The same 30 individuals participated in the previous evaluation participated in the assessment 3 weeks later. Directly responding to a questionnaire after observing the robot's display of multimodal emotional expressions

Analysis results of responses:

Comparing the multimodal emotional expressions

Analysis results of responses:

Comparing the multimodal emotional expressions

Analysis results of responses:

Comparing the multimodal emotional expressions

  • The combination of movement and sound shows a lower level of valence compared to other three

  • The combination of light and sound exhibits lower activation levels compared to other three

  • The combination of movement, light, and sound demonstrates higher activation levels than combination of movement and sound

The relationship between the robot's personalities,

activation level and the spaces

Relationship between robot's personalities, activation level and the spaces

The relationship between the robot's personalities, activation level and the spaces

According to the Pearson correlation analysis results, there is a positive correlation between activation level, robot personality, and space(activity and noise level)

According to the Pearson correlation analysis results, there is a positive correlation between activation level, robot personality, and space(activity and noise level)

According to the Pearson correlation analysis results, there is a positive correlation between activation level, robot personality, and space(activity and noise level)

Frequency analysis of responses regarding the space (1)

Frequency analysis of responses regarding the space (1)

Frequency analysis of responses regarding the space (1)

  • The combination of light and sound in the library had the highest response rate compared to other emotional expressions (31 out of 69 responses, approximately 45%)

  • Positive expressions were more frequent in responses from restaurants and amusement parks (61% for amusement parks, 65% for restaurants).

  • Negative expressions dominated responses on the streets (60% expressed negativity).

  • The combination of light and sound in the library had the highest response rate compared to other emotional expressions (31 out of 69 responses, approximately 45%)

  • Positive expressions were more frequent in responses from restaurants and amusement parks (61% for amusement parks, 65% for restaurants).

  • Negative expressions dominated responses on the streets (60% expressed negativity).

  • The combination of light and sound in the library had the highest response rate compared to other emotional expressions (31 out of 69 responses, approximately 45%)

  • Positive expressions were more frequent in responses from restaurants and amusement parks (61% for amusement parks, 65% for restaurants).

  • Negative expressions dominated responses on the streets (60% expressed negativity).

Frequency analysis of responses regarding the space (2)

Frequency analysis of responses regarding the space (2)

Frequency analysis of responses regarding the space (2)

  • In amusement park, the response rate for methods associated with high activation levels was 82%

  • Street showed relatively high activation responses

  • The library showed a relatively high number of low activation responses

  • In amusement park, the response rate for methods associated with high activation levels was 82%

  • Street showed relatively high activation responses

  • The library showed a relatively high number of low activation responses