Learning and Comfort in Human–Robot Interaction: A Review

Abstract

Collaborative robots provide prospective and great solutions to human–robot cooperative tasks. In this paper, we present a comprehensive review for two significant topics in human–robot interaction: robots learning from demonstrations and human comfort. The collaboration quality between the human and the robot has been improved largely by taking advantage of robots learning from demonstrations. Human teaching and robot learning approaches with their corresponding applications are investigated in this review. We also discuss several important issues that need to be paid attention to and addressed in the human–robot teaching–learning process. After that, the factors that may affect human comfort in human–robot interaction are described and discussed. Moreover, the measures utilized to improve human acceptance of robots and human comfort in human–robot interaction are also presented and discussed.

1. Introduction

Collaborative robots provide prospective and great solutions to complex hybrid assembly tasks, especially in smart manufacturing contexts [1,2]. Through human–robot interaction, the tasks can be split between humans and robots based upon their capabilities to leverage their unique advantages. For example, in human–robot collaborative assembly tasks, robots can execute tedious and strength-based sub-tasks, while humans can conduct brainwork-based sub-tasks [3,4]. In general, collaborative robots are mainly programmed and controlled by well-trained experts through offline coding devices, such as teaching pendants [5], which usually cost humans significant time and effort. However, modern smart manufacturing is experiencing quick product upgrades with more customization and shorter life cycles to meet the ever-changing market needs. In order to shorten this gap between research and application, one of the up-to-the-moment topics—robot learning from human demonstrations—is proposed and has been studied by both academia and industry in recent years [6,7]. Via this approach, humans can transfer knowledge to robots through demonstrated actions without needing considerable coding skills to have robots understand how to accomplish tasks [8,9].

Apart from the understanding of human actions and intentions in human–robot interaction [10,11], human comfort, which has a direct and immediate influence on the collaboration quality between the robot and its human partner, is also a significant factor for the robot to be aware of [12]. For instance, in human–robot collaborative tasks, technical safety (keeping a required physical distance or developing a safety interlock system between the human and the robot) does not necessarily mean human perceived safety and comfort since the perceived feelings are mostly subjective [13]. In addition, the same performance of the robot in the same task may lead to different humans having diverse comfort levels. For example, a slow robot speed usually makes some people feel safe in a human–robot interaction. However, some others may consider it to be less effective and feel uncomfortable.

Therefore, it is in the human–robot interaction context that we will introduce and discuss two significant topics—learning and comfort—in this review. The rest of the review is organized as follows. Section 2 provides the topic of teaching and learning in human–robot interaction, which contains discussion of robots learning from demonstration, human teaching approaches, and robot learning approaches. The conceptual details and corresponding applications of each sub-topic are presented. Several important issues, including extraction, real-time, correspondence, execution, and safety in the human–robot teaching–learning process are discussed in Section 3. Section 4 describes the details about what affects human comfort in human–robot interaction. Different kinds of factors that may influence human comfort are investigated and discussed with related studies. Section 5 describes how to improve human comfort in human–robot interaction, where the three sub-topics of human comfort measurement, measures to improve human acceptance of robots, and measures to improve human comfort are included and discussed. Finally, conclusions are presented in Section 6.

2. Teaching and Learning in Human–Robot Interaction

2.1. Robot Learning from Demonstration

The robot programming approach has gone through three distinct reforms in the past 60 years. As shown in Figure 1, these robot programming approaches include: teach pendant-based programming, computer-aided design (CAD)-based programing, and robot learning from human demonstrations.

In the first approach, teach pendants are handheld devices that can be used by humans to program robots directly with predefined tasks via different programming languages, such as RAPID, KRL, and VAL3. Afterwards, the robots are controlled to plan motions step-by-step to accomplish the fixed workflows [14]. However, the teach pendant-based programming approach usually requires the human mastering skillful technical expertise. Therefore, it is time-consuming and not cost-effective, especially for large-scale manufacturing tasks.

In order to assist users with programming robots in an intuitive way, the CAD-based robot programming approach was developed that allows humans to program robots in 3D manufacturing environments with basic CAD skills. Using the CAD-based programming approach, the robot programs are generated offline and then converted into robot commands for corresponding tasks [15]. This approach improves the efficiency of robot programming to some extent. However, in this approach, the users are still required to master a certain level of programming skills and expertise.

Robot learning from demonstration (LfD), which is also known as robot programming by demonstration (PbD), imitation learning, or apprenticeship learning, is a paradigm for enabling robots to autonomously perform new tasks [16]. The study of robot learning from demonstration started about 30 years ago. This approach has grown significantly and has become a center topic of robotics, especially in the area of human–robot interaction [8]. Via the approach of robot learning from demonstration, the human partners can program robots easily and greatly extend the robots’ capabilities for different tasks without programming expertise [7]. The process of robot learning from human demonstrations is basically divided into two steps: human teaching and robot learning. Numerous methods and technologies, such as force-sensor-based teaching [17], vision-system-based teaching [18], and natural-language-based teaching [19], have been developed and implemented in the human teaching process. Additionally, multiple learning algorithms have also been designed and developed for the robots to extract, learn, and build task strategies from human demonstrations [9]. In the human–robot teaching–learning process, the robot usually learns from the human in a direct or indirect manner. The former is human intuitive teaching toward the robot (e.g., kinesthetic-based teaching) and the latter is the human teaching using some external devices (e.g., via a vision system). Approaches of human teaching and robot learning are detailed as follows.

2.2. Human Teaching Approaches

2.2.1. Kinesthetic-Based Teaching

In this approach, the robot is physically guided through the task by the human, where its passive joints are moved through desired motions. Typically, the human teacher operates the robot learner, whose sensors record the execution [20]. The robot gets referee signals by pairing kinesthetic teachings with the human teacher executions being recorded. The human teacher demonstrates several actions for the robot for the same task and changes the location of each step in between to allow the robot to generalize exactly. After that, the robot can infer the relative positions of the objects by observing the demonstrations [16].

This approach provides a natural teaching interface for the robot towards learning the correct required motions. However, one of the drawbacks of the kinesthetic teaching approach is that the human usually uses more of their own degrees of freedom to guide the robot than the number of degrees of freedom they are trying to control. For instance, the human has to use two arms to move one robot manipulator or use both hands to move a few robot fingers in the juice making task [16].

2.2.2. Joystick-Based Teaching

In this approach, the human teacher controls the demonstration and transmits the information regarding the actions to the robot’s controller using a wireless or wired joystick. It is a low-level form of demonstration teaching. This approach has been developed for a variety of applications, such as robot polishing tasks [21], robot soccer tasks [22], and robot welding tasks [23]. In Katagami and Yamada [22], the mobile robot is taught by a human teacher in a soccer game. The operator observes the robot from a viewpoint of overlooking the environment, considers the next operation for teaching, and demonstrates the expected movements using a joystick. However, not all systems are suitable for this technique since the operated robot must be manageable without a high degree of freedom or a complicated structure.

2.2.3. Immersive Teleoperation Scenarios Teaching

In this approach, the human teacher is limited to using the robot’s own sensors and effectors to perform the task [24]. Compared to joystick teaching, this approach underlines utilizing the robot’s own body by the human teacher rather than external devices. In the immersive teleoperation scenarios teaching approach, teleoperation itself can be done using haptic devices, which allow the teacher to teach tasks that require the precise control of forces [24].

Since this approach not only solves the correspondence problem, which is one of the significant issues in human–robot teams (see Section 3), but it also allows human teachers to train robots from a distance, and is well utilized for robot locomotion tasks. In Peternel and Babič [25], a system with a capacity for teaching humanoid robots balancing skills in real-time is proposed. This system employs the human sensorimotor learning ability in which a human demonstrator learns how to operate a robot in the same manner that the human adapts to various everyday tasks. The robot learns the task while the human is operating the robot.

2.2.4. Wearable-Sensor-Based Teaching

Human teachers use their own bodies to perform example executions by wearing sensors, which are able to record the teachers’ states and actions [10,26]. For example, the force-sensing glove can be used for acquiring data from pressure sensors. The collected data can be mapped to manipulation actions, which will in turn be used to interpret physical human–robot interaction [27]. In addition, manipulation tasks can also be demonstrated in a virtual reality environment using a data glove and a motion tracker, by which the specific parts of the objects where grasping occurs are learned and encoded in the task description for the robot.

For other program-robot-by-demonstration systems, for example, the virtual environment is built upon the Virtual Hand Toolkit library provided by the Immersion Corporation, where the human hand is drawn in the virtual scene and driven based on the data captured with the virtual reality (VR) glove [28]. Human teachers also use wearable motion sensors attached on the arm to incrementally teach human gestures to a humanoid robot [29].

2.2.5. Natural-Language-Based Teaching

In this approach, the human teacher can present the demonstrations to the robot through a natural language, by which the teacher specifically tells the robot what actions to execute [9]. For instance, the human teacher can use natural language to teach a vision-based robot how to navigate itself in a miniature town, where the robot is provided with a set of primitive procedures derived from a corpus of route instructions in order to enable unconstrained speech [30]. In the recent work [31], the human teacher can propose a representation of high-level actions that only consists of the desired goal states rather than step-by-step operations in the robot teaching based on the traditional planning framework. For another case, human teachers collect a dataset of task descriptions in a free-form natural language and the corresponding grounded task-logs of the tasks performed in an online robot simulator. After that, they build a library of verb environment instructions that represent the possible instructions for each verb in the working context [32].

2.2.6. Vision-Based Teaching

This approach is based on the external observation teaching manner, where the executions information is recorded using vision devices [9], such as stereo-vision cameras, which can be located externally to the executing platform. In this approach, human teachers integrate visual tracking and shape reconstruction with a physical modeling of the materials and their deformations, as well as action learning techniques, and all these sub-modules are integrated into a demonstration platform [33]. Another way is based on a luminous marker built with high-intensity light-emitting diodes (LEDs) that can be captured by a set of industrial cameras, where the marker supplies a six degree of freedoms (DoFs) human wrist tracking with both position and orientation data using stereoscopy. Then, the robot is automatically programmed from the demonstrated task [18]. This approach is mainly based on the image processing technology and is different from the VR glove method in which the wearable sensing technology is mainly employed.

2.3. Robot Learning Approaches

2.3.1. Kinesthetic-Based Learning

In this approach, the robot is handled and controlled by a human teacher, where it directly records the states and actions experienced by the sensors distributed on its body during the executions and then completes the explicit task. For example, in Kormushev et al. [34], a robotic manipulator learns to perform tasks that require exerting forces on external objects by interacting with a human operator in an unstructured environment. The robot learns from human teacher demonstrations based on positional and force profiles, by which the action skills are reproduced through an active control strategy based on task space control with a variable stiffness. Additionally, this approach can be combined with multiclass classification methods to realize robot learning from the human teacher. The goal parameters of linear attractor movement primitives can be learned from manually segmented and labeled demonstrations. Moreover, the observed movement primitive order can help to improve the movement reproduction for the robot in the learning process [35].

2.3.2. One-Shot Learning

Rather than teach the robot at once with all the demonstrations like the kinesthetic-based learning method, in the one-shot learning approach, the human teacher provides one or more examples of each sub motion apart from the others and robot learns from the observation of a single instance of the motion [16]. The robot can use this algorithm to select a previously observed path demonstrated by a human and generates a path in a novel situation based on pairwise mapping of invariant feature locations, which is then used in both the demonstrated and the new scenes using a combination of minimum distortion and minimum energy strategies [36].

2.3.3. Multi Shot Learning

Multi shot learning can be performed in batches after several demonstrations are recorded. Robot learning processes are usually inferred from statistical analysis of the data across demonstrations [16]. For instance, in Lee and Ott [37], a humanoid robot starts with observational learning and applies iterative kinesthetic motion refinement using a forgetting factor on the basis of a multi shot learning approach. The kinesthetic teaching is handled using a customized impedance controller, which combines tracking the performance with a compliant physical interaction on the real-time control level.

2.3.4. Vision-Based Learning

This approach corresponds to vision-based teaching, where the robot encodes the information recorded by the vision devices and maps it to its actions. The vision system acts as the robot’s eyes in the system for tracking the human teacher’s actions. In recent works, researchers employed a 7-DoF robot manipulator and a Kinect sensor to train the robot to learn from humans [38]. The skill learning approach is based on symbolic encoding other than trajectory encoding such that it offers a more concise representation of a skill, which is easily transferable to different forms of embodiments. This learning approach is also employed in the object affordances, where researchers extract a descriptive labeling of the sequence of sub-activities performed by a human describing the interactions with the objects in the form of associated affordances. Through a red green blue-depth (RGB-D) video, researchers can model the human activities and object affordances as a Markov random field, where the nodes represent objects and sub-activities. Then the robot learns from a human teacher by means of a structural support vector machine (SSVM) [39].

2.3.5. Reinforcement-Learning-Based Approach

In this approach, the robot learns through trial and error to maximize a reward such that it allows the robot to discover new control policies through free exploration of the state–action space. This approach assumes that there is a known set of necessary primitive actions for the robot to imitate to improve its behavior. Reinforcement learning is widely used in many applications, such as aerial vehicles, autonomous vehicles, robotic arms, and humanoid robots. In Bagnell and Schneider [40], an autonomous helicopter leverages a model-based policy search approach to learn a robust flight controller. In Ghadirzadeh et al. [41], a data-efficient reinforcement learning framework is proposed to enable a robot to learn how to collaborate with a human partner. The robot learns the task from its own sensorimotor experiences in an unsupervised manner. On the basis of reinforcement learning, a distributed and asynchronous policy learning is presented as a means to achieve generalization and improve training times in real-world manipulation tasks [42].

2.3.6. Inverse-Reinforcement-Learning-Based Approach

Inverse reinforcement learning is different from the reinforcement learning. This approach offers a framework to automatically determine the reward and infer the optimal control strategy when the robot learns from human demonstrations [16]. For instance, by means of inverse reinforcement learning, the robot can learn acceptability-dependent behavioral models from human demonstrations and build its own task policies to assist its human partner in collaborative assembly tasks [7]. Multi-robot inverse reinforcement learning is also proposed, where the behaviors of multiple robots execute fixed trajectories and interact with each other from passive observations [43]. Additionally, the inverse reinforcement learning can be leveraged to combine with a neural network to deal with large-scale and high-dimensional state spaces. In this case, the expert’s behaviors to unvisited regions of the state space can be generalized and the expert’s explicit or stochastic task policy representation can also be easily expressed [44].

This approach also utilizes the human teacher’s failed demonstrations as examples to train the robot. This method deliberately makes the robot avoid repeating the teacher’s mistakes rather than maximize the similarity to the demonstrations. Purposely generating failed demonstrations is easier than generating successful ones in some cases. For example, in Shiarlis et al. [45], the approach called inverse reinforcement learning from failure (IRLF), which converges faster and generalizes better than other methods, is proposed to exploit both successful and failed demonstrations.

2.3.7. Skill-Tree-Construction-Based Approach

Skill tree construction is an online algorithm from demonstration trajectories. In this approach, the demonstration trajectory is segmented into a chain of component skills in which each skill has a goal and is assigned a suitable abstraction from an abstraction library. This approach is able to segment multiple demonstrations and merge them into one skill tree [46,47].

2.3.8. Syntactics-Based Approach

In this approach, some significant task structures conducted by humans can be captured in the form of probabilistic activity grammars from a reasonably small number of samples even under noisy conditions. After that, the learned grammars are employed to facilitate the recognition of more complicated and unforeseen tasks that share underlying structures [48].

2.3.9. Semantic-Networks-Based Learning

This approach adopts semantic networks to make the robot gain the ability to model the world with concepts and relate them to low-level sensory-motor states [49]. By means of this approach, the robot can learn from its human teacher on the basis of hierarchical types of knowledge using the robot’s senses, actions, and spatial environment. The learning algorithm derives from a computational model of the human cognitive map that exploits the distinction between procedural, topological, and metrical knowledge of large-scale space. Moreover, the semantic hierarchy approach has been extended to continuous sensorimotor interaction with a continuous environment [50]. In Li et al. [51], a semantic-network-based learning approach is used to combine with a wearable sensor based method to obtain semantic information efficiently and link it to a metric map. In its applications, an intelligent mobile robot platform can create a 2D metric map, and meanwhile, human activity can be recognized via motion data from wearable motion sensors that are mounted on the human teacher.

2.3.10. Neural-Models-Based Learning

This approach models a mirror neuron system for the robot to learn from its human teacher, where the neural model is the basis of recognition and produces basic grasping motions. Meanwhile, it proves the existence of forward models for guiding these motions [52,53]. For instance, a robot control model is presented to integrate multimodal information, make decisions, and replicate a stimulus–response compatibility task in Sauser and Billard [54]. The model contains a neural network on the basis of a dynamic field approach, the natural ability of which is famously employed for stimulus enhancement and cooperative–competitive interactions within and across sensorimotor representations.

2.3.11. Procedural-Memory-Based Learning

In this approach, the robot’s procedural memory is developed based on an adaptive resonance system. In Yoo and Kim [55], the robot learns the knowledge from human demonstrations just like a child learns through interactions with parents and teachers to build its knowledge system. In this case, human demonstrations are captured using an RGB-D camera, from which the robot segments each execution with the acquired continuous streams. The robot’s procedural memory is developed based upon an adaptive resonance system. using this procedural-memory-based learning approach, the robot is able to perform full sequences of tasks with only partial information of the tasks to be carried out.

2.4. Comparison and Discussion of Different Approaches in Human–Robot Teaching–Learning Processes

In this section, we include comparisons of the differences, strengths, and weaknesses of the reviewed approaches above. As shown in

Table 1. Comparisons of different human teaching approaches.

Approach	Category	Human-Robot Interface	Cost	Features
Kinesthetic-Based Teaching	Physical touch	Robot links and force sensors	Low	Cost-competitive Intuitive operation Significant human efforts required
Joystick-Based Teaching	Physical touch	Joystick	Low	Cost-competitive Intuitive operation Not suitable for high degree of freedoms robot systems
Immersive Teleoperation Scenarios Teaching	Physical touch	Robot force sensors and end effector	Low	No external devices required Precise robot control Professional expertise required
Wearable Sensor-Based Teaching	Physical touch	Force-sensing glove and VR glove	High	Intuitive operation No professional expertise required High cost
Natural Language-Based Teaching	Non-physical touch	Natural language	Low	Cost-competitive Constrained by speech recognition technology
Vision-Based Teaching	Non-physical touch	Vision sensors/systems	High	No professional expertise required Intuitive operation Easy to be influenced by environment

, during human teaching processes, according to the employed teaching equipment and demonstration manners, human teaching approaches can be categorized into physical-touch approaches and non-physical-touch approaches. We also summarize the human–robot interfaces when humans teach robots in diverse kinds of tasks. The costs of these different teaching techniques are relatively divided into low and high costs. In robot learning processes, as presented in

Table 2. Comparisons of different robot learning approaches.

Approach	Category	Algorithm	Features
Kinesthetic-Based Learning	Low-level learning	Task space control	Non-complex computation Not suitable for complex tasks
One-Shot Learning	Low-level learning	Pairwise mapping	Intuitive logics and non-complex computation Perform tasks in sub-motion and one single instance for each learning
Multi Shot Learning	Low-level learning	Iterative kinesthetic motion refinement	Perform tasks in a batch Not suitable for large-scale tasks
Vision-Based Learning	High-level learning	Symbolic encoding, Structural support vector machine	Intuitive representation for tasks Not suitable for tasks with a complex background
Reinforcement Learning-Based Approach	High-level learning	Reinforcement learning algorithm	Increases behavior and maximizes performance for the robot Risk of overload of states may decrease the results
Inverse Reinforcement Learning-Based Approach	High-level learning	Inverse reinforcement learning algorithm	Learning rewards instead of learning policy to adapt dynamic environment Need to repeatedly solve the Markov Decision Process in the learning
Skill Trees Construction-Based Approach	High-level learning	Skill trees algorithm	Can segment multiple demonstrations and merge them into on High computational complexity for complex tasks
Syntactics-Based Approach	High-level learning	Probabilistic activity grammars	Learn from a small number of samples for complicated tasks Tasks are constrained by specific structures
Semantic Networks-Based Learning	High-level learning	Semantic hierarchy algorithm	Intuitive knowledge representation for tasks Not suitable for large-scale tasks
Neural Models-Based Learning	High-level learning	Mirror neuron model	Not only produce robot basic motions, but also guide these motions High computational complexity for complex tasks
Procedural Memory-Based Learning	High-level learning	Adaptive resonance model	Perform full sequences of tasks with only partial task information High computational complexity for complex tasks

, the learning methods can be classified by low-level learning approaches and high-level learning approaches. For the low-level approaches, the collected demonstration information of the human teacher is usually directly used for pairing robot goal states and current states, then the robot is controlled by the robot controller. However, in the high-level approaches, more flexible and complex methods (e.g., machine learning algorithms) are employed for the robot to infer action policies from human demonstrations, and even predict unknown or unlearned actions in human–robot collaborative tasks. Therefore, these approaches make robots more intelligent than low-level learning approaches. From Table 1, Table 2, it can be concluded that different human teaching and robot learning approaches have their own features in diverse kinds of human–robot interactive contexts. Therefore, for the different levels of difficulty of tasks, the selection of the human teaching approach or robot learning approach should correspond to its features to properly solve the issues (see Section 3) in human–robot teaching–learning processes.

3. Several Issues in Human–Robot Teaching–Learning Processes

In the human–robot teaching–learning process, there are several issues that should be paid attention to and addressed. These issues include extraction, real-time, correspondence, execution, and safety issues. Each issue is discussed in the following.

3.1. Extraction

Extraction refers to whether the human teacher’s behavior states or actions are absolutely and correctly extracted into the dataset that will be used by the learning approaches to teach robots. For instance, in the vision-based teaching–learning systems, a teacher displays his hand in the camera’s view, so the dataset should record his palm and five figures, rather than three figures or no palm.

3.2. Real-Time

Real-time issues exist in most electrical device signal processing. In the robot system, we should ensure that the demonstration acquiring, information record, and execution embodiment are synchronous at per state and action level, or ensure that the time delay is within the margin of error. For example, when the teacher completes the entire screw pick–transport–place movements in a human–robot co-assembly task, the robot should not just start the picking action.

3.3. Correspondence

The issue of correspondence is very important in human–robot interaction, where it refers to the identification of a mapping between the human teacher and the robot learner that allows for the transfer of information in the human–robot team [9]. In a short and vivid statement, it concerns: “whether the robot could identify the information transferred by its human teacher.” Correspondence contains two sub-issues: the perceptual equivalence and the physical equivalence. In the perceptual equivalence, for the same scene in human–robot interaction, it may present differently because of the differences between human and robot sensory capabilities. For example, the robot can utilize depth cameras to observe human hands, while the human may recognize them from light. In the physical equivalence, the human and the robot may take a different action to complete the same physical effect because of the differences between human and robot embodiments [16].

3.4. Execution

This issue refers to whether robot should physically execute the human teacher’s behaviors completely and accurately. After a movement from a human teacher is acquired, recorded, and embodied, the robot could accept and transfer it to its effector and actuator to reproduce the same action.

3.5. Safety

Safety should always be kept in mind. The issue of safety includes two aspects: one is that the robot must be friendly to his teacher and must not inflict injury on or bring potential danger to the human demonstrator, and the other point is that the robot must have security for its tasks and will not damage other machines or products around its workspace. Thereby, safety measures, including interlock chains or emergency stop strategies, must be considered when designing the robot and its teaching–learning mechanism.

4. What Affects Human Comfort in Human–Robot Interaction?

The general factors that affect human comfort in human–robot interaction have been primarily studied from different perspectives. These factors mainly include the robot response speed, the robot movement trajectory, the human–robot proximity, the robot object-manipulating fluency, human coding efforts, the robot sociability, and factors outside human–robot teams. Each corresponding factor is discussed below.

4.1. Robot Response Speed

The speed of the robot response usually has a direct influence on human feelings in human–robot interaction. In Dautenhahn [56], the researchers conducted a study to investigate how a robot should best approach and place itself relative to a seated human subject by trying different approach directions and speeds. They discuss the results of the user studies in the context of developing a path-planning system for a mobile robot. Considering the robot speed as one of factors that may have an impact on human comfort in human–robot collaboration, Mitsunaga et al. developed an adaptation mechanism based on reinforcement learning to read subconscious body signals from humans and utilize this information to adjust robot actions [57]. In order to provide safe and socially acceptable robot paths for humans to make them comfortable in human–robot collaboration, Sisbot et al. designed a human-aware motion planner for the robot to adapt its speed by inferring human accessibility and preferences [58].

4.2. Robot Movement Trajectory

As shown in Figure 2, in a human–robot co-assembly task, the robot starts from point A, picks the part at point B, and deliveries the part to its human partner at point C. However, different robot movement trajectories (one is close to the human and the other is far away from human) may induce different psychological feelings in the human. For instance, through the comparisons of functional motion, legible motion, and predictable motion, Dragan et al. investigate the positive and negative impacts of different planning motions on human comfort and the success of physical human–robot collaboration [59].

4.3. Human–Robot Proximity

Human–robot proximity is the distance between the robot and its human partner in their collaboration. Mumm explores whether the human–human proxemics models can also explain how people physically and psychologically distanced themselves from robots. By conducting a controlled laboratory experiment in human–robot interaction, they conclude that humans who like/dislike the robot show different behaviors in physical and psychological distancing [60]. Walters et al. investigate human–robot and robot–human approach distances by testing two hypotheses. One is that approach distances preferred by humans in human–robot interaction will be comparable to those preferred in human–human interaction. The other is that common personality factors can be employed to predict humans’ likely approach distance preferences. They confirm these two hypotheses via human–robot interactive experiments in a conference room [61]. Takayama et al. explore issues regarding human personal space around robots by testing several research hypotheses in a controlled experiment. They discuss the factors that influence human–robot proximity and human comfort in human-approaching-robot and robot-approaching-human contexts [62]. In human–robot collaborative tasks, Stark et al. conducted a study to evaluate how comfort changes when the robot reaches into the human’s personal space at different distances and urgency levels [63].

4.4. Robot Object-Manipulating Fluency

Human–robot collaborative fluency can be regarded as a high level of coordination that gives rise to a well-synchronized meshing of the actions of humans and robots. The fluency in human–robot collaboration is also considered to be a quality factor that can be positively assessed and recognized when compared to a non-fluent scenario, which has an impact on task efficiency and human comfort [64]. Lasota et al. conducted an experiment in which the participants worked with an adaptive robot incorporating human-aware motion planning to perform a collaborative task. They evaluated the human–robot team fluency via a set of quantitative metrics, and further analyzed human satisfaction and comfort in human–robot collaboration [65]. Through the spatial (formed by distinct hand-over poses) and temporal contrast (formed by unambiguous transitions to the hand-over pose), Cakmak and colleagues improved the human–robot collaborative fluency that makes humans feel comfortable in human–robot teams [66].

4.5. Human Coding Efforts

Most robots are traditionally programmed using offline devices, such as a teaching pendant, which is tedious and time-consuming and makes the human expert feel uncomfortable in some situations. Therefore, novel and more effective approaches need to be developed for humans to interact and program robots easier and more intuitively. Neto et al. developed a CAD-based approach allowing the users to program robots with basic CAD skills to generate robot programs without inputting too much human coding effort comparing to the robot teaching pendant [15]. In order to facilitate human–robot collaborative tasks, Wang et al. proposed a teaching–learning collaboration model where the robot learned from human assembly demonstrations and actively collaborated with the human in collaborative tasks through a natural language, which can largely improve human–robot efficiency and human comfort [7].

4.6. Robot Sociability

Robot sociability has also been identified as a factor influencing human–robot interaction [61], where robots are regarded as “social entities” to interact with humans socially while they work together. From the social science perspective, the amount of robot smiling, eye contact, and the appearance of robots usually result in different feelings, including happiness, sadness, fatigue, nervousness, and worry, of human partners about the robots [67].

4.7. Factors Outside Human–Robot Teams

The factors outside the teamwork, including task types, working surroundings, and mission contexts [68], have also been investigated in human–robot collaboration [69]. These external factors generally affect the development and implementation of a human–robot team to accomplish the collaborative task successfully and ergonomically.

5. How to Improve Human Comfort in Human–Robot Interaction?

5.1. Human Comfort Measurement

Before getting familiar with how to improve human acceptance of robots, it is necessary to know how to measure human comfort. For human comfort measurement, there are two main widely used approaches: the self-evaluation approach and the physiological approach.

5.1.1. Self-Evaluation Approach

For the self-evaluation approach, many studies have been conducted to measure human comfort in various situations [70,71] using different kinds of questionnaires [72,73,74]. Several typical designs, such as human self-reports [75], frequency of human interventions [76], self-assessment manikin [77], and compliance with robot suggestions [68], have been utilized for assessing comfort in human–machine interactions. Some other approaches enable humans to rate their comfort levels using a Likert scale [78] in real time during user studies. In these approaches, online devices, such as smartphones with designed applications (APPs) [79,80,81], are adopted where the participants can change their comfort levels by either intuitively sliding comfort bars or speaking.

5.1.2. Physiological Approach

For the physiological approaches, many studies have shown that various physiological signals can be used to recognize the internal states of human subjects, such as emotions and feelings. In human–robot interaction contexts, it has been found that some affective states of humans, such as excitement, fatigue, engagement, and distractions (and likely comfort as well) [82,83], are correlated to their physiological signals including those measured using electrodermal activity (EDA), photoplethysmography (PPG), skin temperature (ST), eye tracking (ET), and electroencephalography (EEG). EDA, also known as galvanic skin response, is the measurement of conductance or resistance across the surface of one’s skin, which continuously varies as one responds to various stimuli [84]. PPG is a measurement of the changes in light absorption of the skin, which is measured using a pulse oximeter; it is a periodic signal that measures cardio performance and can be used to evaluate a participant’s level of arousal [85]. ST is measured with a temperature sensor and has been used to measure stress [86]. ET measures eye motion relative to the participant’s head in addition to pupil diameter via eye-tracking glasses. Studies have shown that pupil diameter is strongly related with the emotional arousal and autonomic activation [87]. EEG is a measure of the brain’s electrical activity and can be used to evaluate stress, excitement, focus, interest, relaxation, and engagement of the human in human–robot interaction [88].

5.2. Measures to Improve Human Acceptance of Robots

User acceptance of a robot has a direct impact on the quality of shared tasks when a robot works with its human partner. Many studies have been performed on the improvement of user acceptance in human–robot interaction [89,90,91]. Generally, improving the robot performance can result in a positive impact on user acceptance, where the proposed solutions in the previous studies include developing the robot with a friendly and intuitively human–robot interface [92,93,94], designing different kinds of robots for diverse age groups [95,96], improving the robot response to human needs [97,98], etc. Additionally, user acceptance will be improved if a high collaboration efficiency exists in the human–robot team. Several measures have been taken such as reducing human idle time [99,100] and facilitating human–robot collaboration fluency [101,102,103]. Making the robot easy to use can also improve user acceptance of the robot; for example, programing robots using human demonstrations to assist humans to accomplish collaborative tasks [7,19,39]. Moreover, the idea of human acceptance of new things, especially new robots, should also be improved. There are several methods that can be employed, such as professional user training [104,105,106] and popularizing robots with the public [107,108,109].

5.3. Measures to Improve Human Comfort

Several studies have been conducted on improving human comfort in human–robot interaction. In general, accommodating the robot actions to different individuals by considering their working preferences can improve human comfort. The proposed approaches include adjusting human–robot proximity in human personal space [65,110], designing multiple robot motion trajectories [59,111], controlling the robot with diverse velocities [58,112,113], and planning the robot with different manipulation orientations [114]. In addition, a fluent interaction [64] between the human and the robot can also improve human comfort. Some typical solutions have been developed, such as human intention anticipation for robot action selection [10,103,115], human-inspired plan execution system [99], and perceptual symbol practice [116]. Moreover, from the perspective of robot sociability, developing friendly appearances for robots in human–robot interaction, especially for assistive robots in home settings [117,118,119,120] and entertainment robots in public places [92,121,122,123], play a significant role in improving human comfort.

6. Conclusions

In this review paper, we have presented and discussed two significant topics in human–robot interaction: learning and comfort. The collaboration quality between the human and the robot has been improved largely by taking advantage of robots learning from demonstrations. Human teaching and robot learning approaches with their corresponding applications have been investigated in this work. We have presented and discussed several important issues that need to be paid attention to and addressed in the human–robot teaching–learning process. After that, the factors that may affect human comfort in human–robot interaction have been described and discussed. We have also presented and discussed the measures to improve human acceptance of robots and human comfort in human–robot interaction.