JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2015 1

New Methods for Stress Assessment andMonitoring at the Workplace

Davide Carneiro, Paulo Novais, Juan Carlos Augusto, and Nicola Payne

Abstract—The topic of stress is nowadays a very important one, not only in research but on social life in general. People areincreasingly aware of this problem and its consequences at several levels: health, social life, work, quality of life, etc. This resulted in asignificant increase in the search for devices and applications to measure and manage stress in real-time. Recent technological andscientific evolution fosters this interest with the development of new methods and approaches. In this paper we survey these newmethods for stress assessment, focusing especially on those that are suited for the workplace: one of today’s major sources of stress.We contrast them with more traditional methods and compare them between themselves, evaluating nine characteristics. Given thediversity of methods that exist nowadays, this work facilitates the stakeholders’ decision towards which one to use, based on how muchtheir organization values aspects such as privacy, accuracy, cost-effectiveness or intrusiveness.

Index Terms—Stress, Human-Computer Interaction, Survey, Ambient Intelligence.

F

1 INTRODUCTION

THE topic of stress currently attracts significant attention,not only in research but on social life in general. The

public is aware of this phenomena and of its consequences atmany levels (e.g. psychological, physical, social, well-being).On the other hand, researchers in many different fields workto find new ways to assess, monitor and reduce stress, thatcan not only answer the interest of the public but also allowa better understanding of the phenomenon.

Of all the important perspectives on stress, a particularlyinteresting one concerns occupational stress. While occupa-tional stress affects individuals at a personal level, there isa special interest in the effects at the organizational level,mainly its economic impact. There is a broad consensusthat job stress has a significant economic impact, amountingto billions of dollars each year in the United States alone[1]. These losses are due to the increased cost of medicalinsurance, excess of pressure on medical facilities and pro-fessionals, lower productivity, human error, absenteeism,and so forth [2].

This calls for the development and implementation ofinitiatives for stress management that can not only reducethese costs but, at the same time, improve well-being, work-place quality, among other indicators.

• D. Carneiro is with the CIICESI/ESTG, Polytechnic Institute of Porto,Portugal and the Algoritmi Centre/Department of Informatics, Universityof Minho,Braga, Portugal.E-mail: [email protected]

• P. Novais is with the Algoritmi Centre/Department of Informatics, Uni-versity of Minho,Braga, Portugal.E-mail: [email protected]

• J. C. Augusto is with the Research Group on Development of IntelligentEnvironments, Department of Computer Science, Faculty of Science andTechnology, Middlesex University, London, United Kingdom.E-mail: [email protected]

• N. Payne is with the Department of Psychology, School of Science andTechnology, Middlesex University, London, United Kingdom.E-mail: [email protected]

Manuscript received ***; revised ***.

The main aim of this paper is thus to survey existingmethods for stress assessment and monitoring in Humans.Specifically, we focus on methods that can be used continu-ously throughout the day in milieus such as the workplace.We seek ways to measure stress over long periods of time,that do not influence the workers’ routines [3].

Although we address methods that can be deemed asmore traditional (e.g. physiological sensors, questionnaires)in the sense that they have been in use for decades, we focusespecially on novel methods that, for their characteristics,raise significant interest. Moreover, given their novelty, thesemethods pose new challenges at several levels (e.g. techno-logical, ethical [4]) and caution is advised when using them.

Specifically, we focus on methods that can be used inline with Ambient Intelligence systems, allowing a con-tinuous monitoring of the users while they perform theirdaily activities, without interference [5]. In this sense, itis important to start by clarifying two concepts that areoften found in research in this field that, although different,are frequently used interchangeably: invasive and intrusive.In a physiological sense, an action is called invasive if itinfiltrates, cuts or destroys healthy tissue, namely the skin.An intrusive action, on the other hand, is one that intrudesor interferes in one’s space, resulting in (often unwanted)changes in routines.

Consequently, a non-invasive approach is one in whichthere is no invasion of the user’s body. This includes mostof the sensors currently used for stress monitoring (e.g. skintemperature, heart rate). A non-intrusive approach, on theother hand, must meet more strict criteria. Specifically, itcannot change, in any way, the routine of the user. Thismeans that users must be able to carry out their daily activ-ities as if they were not being monitored. This includes, bydefinition, approaches based on computer vision or speechanalysis, for example. Nonetheless, other types of intrusionmay be present, as will be addressed later (e.g. the use ofa video camera may be seen as a privacy intrusion). Theseapproaches will thus be compared, namely in terms of their

JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2015 2

degree of intrusion, so that researchers or practitioners candecide on the best to use for each domain of application.

The paper is organized as follows. Section 2 addressesthe fields of Ambient Intelligence and Ambient AssistedLiving, describing their main characteristics and aims. Sec-tion 3 describes stress, its origin, its effects at several levelsand its importance, especially in the workplace. Section4 addresses traditional approaches for stress assessment,namely those based on physiological sensors and question-naires. Section 5 contains the core of the paper, detailingseveral new methods for stress assessment that presentsignificant advantages when compared to more traditionalones, making them more suitable to be used in the work-place. These methods are critically analyzed and comparedin Section 6, allowing practitioners to decide on the bestmethod to use in each specific domain. Finally, Section 7presents a discussion of the main conclusions of this workand points out future research trends and directions.

2 AMBIENT INTELLIGENCE AND AMBIENT AS-SISTED LIVING

Computer Science has been fast evolving in the last fewdecades pushed by technological advances. Innovation inelectronics had created the need for the field of comput-ing in general to get more involved with decentralizedcomputation. Nowadays it is possible to embed reasonablyuseful computational capabilities in a small space and at anaffordable cost.

Mobile phones, tablets, PCs, smart watches which cantrack several health parameters, smart toys with ingeniousindependent behaviour, houses which can control comfortand safety aspects of our life, smart classrooms, smartoffices, automated farming, autonomous cars, autonomousairplanes, and a myriad of other advances have broughtcomputing very close to our daily lives in a way mostdid not anticipated just a couple of decades ago and itwas the topic of science fiction films. This transformationwas not simultaneous: the most influential work in thisdirection is acknowledged to have started within the areasof “Ubiquitous Computing” and “Pervasive computing”[6]. It then progressed into other concepts with “AmbientIntelligence” [7] and “Intelligent Environments” [8].

All these can be described as attempts to create “…digitalenvironments that proactively, but sensibly, support peoplein their daily lives.” [9]. Some of these areas put more em-phasis in different aspects of the system as developers weregaining experience and understanding of the most chal-lenging aspects of these multi-disciplinary systems whichbrought together sensing, networking, human-computer in-teraction, artificial intelligence, and software engineering, tomention some of the most relevant disciplines.

Key to the success of all these systems is that to pleasethe intended users, the system has to have an understandingof the context where services have to be delivered. Thesubtler this understanding is, the more informed the systemto satisfy a given user. This includes understanding personalthings, like the preferences or the emotional state of a user.Say I like to be greeted with some ambient music when Iarrive home. The system needs to know that I prefer musicfrom J.S. Bach to music from Iron Maiden, but not any music

from J.S. Bach will do every day, so if some days I am inneed of more cheerful music then perhaps a cantata maybe a good choice and if one day I am in need of morerelaxing music I may prefer some pieces from the Well-Tempered Clavier. Some days I may not want any musicat all. How is the system going to know that? See section on“Mindreading” in [8].

The example above may not seem too important as itis related to leisure. However, one of the most importantpossible applications and one of the most widely researchedand tried benefit expected from this area is what is oftenreferred to as Ambient Assisted Living (AAL). “AAL refersto intelligent systems of assistance for a better, healthier andsafer life in the preferred living environment and coversconcepts, products and services that interlink and improvenew technologies and the social environment.” [10]. Thereare several definitions of AAL. However, most of them putemphasis on the safety, health, and well-being of individ-uals. Although these type of benefits are usually placedin the home environment, AAL system do not have to berestricted to houses and can actually be delivered in otherplaces such as the work place, where many people spend aconsiderable part of their lives. AAL services are also mostoften associated with older people and in particular withsenior citizens experiencing some category of dementia.Although it is true those are the type of applications whichhave most funding so far, hence more interest, it is clearAAL can help citizens with other conditions, Parkinson’sdisease, Down’s syndrome, autism, etc. From this we canalso state that AAL benefits are not only for senior citizensbut it is a type of service with the capacity to improve thequality of life of all citizens.

Having introduced AAL as a kind of specific branch ofAmbient Intelligence with specific interest in the welfare ofcitizens, it is clear that a system given such a responsibilityhas to have substantial capabilities to understand what aperson is going through at a given time as well as powerfuldecision-making. For a system to be capable of lookingafter the welfare of an individual, it has to understand thatindividual deeply. It is not only a matter of knowing aboutthe preferences of that individual and how those preferencesare linked to different situations but it also implies beingcapable to understand how a user feels ‘now’. We can revisitthe ambient music scenario, but now imagine the person inquestion is depressed. If the ambient music is the wrong onefor the mood of the user, it may have a detrimental effect,increasing the levels of anxiety, depression or stress of theindividual in question. Stress may lead to wrong decisions,which in turn can have undesired consequences resulting inmore stress [11].

If understanding a specific mental state of an individuallike feeling stressed is so important for the success of AAL,how can we do it? There are different approaches, some ofthem more behavioural and others more biological. By thesewe mean that some answers to the challenge try to under-stand how the individual is behaving, e.g., body language,whilst the latter approach relies more on measuring specificpersonal body parameters which can provide an indicator,e.g. high blood pressure as a potential indicator of stress.The next sections of this paper provide a more specific

JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2015 3

account of these approaches and highlight the challengesbehind each of these options.

3 STRESS AS A BROAD COGNITIVE PROCESS

Stress and related concepts can be traced as far back aswritten science and medicine [12]. Likewise, its influenceat both an organizational and individual level is nowadaysunquestionable [13].

3.1 Fundamental Concepts

In modern science, stress started to be studied at a physi-ological level, in the decade of 1950. This resulted in a setof reliable physiological indicators for the study of stress,that supported the development of the bio-feedback unitsavailable nowadays. In the 70’s researchers started studyingthe somatic disorders resulting from these biologic aspects[14]. At the same time, Hans Selye provided an accurate andsimultaneously accessible definition of stress [15], puttingforward the notion of stressor and addressing the hormonalchanges caused by stress.

Although such views have changed throughout history,there is an agreement that responses to stress are coordi-nated by a so-called stress system, whose composition isnowadays well studied and known to include as maincomponents the corticotropin-releasing hormone and locusceruleus-norepinephrine/autonomic systems and their pe-ripheral effectors [12]. Moreover, the effects of stress atdifferent levels (e.g. behavioral, peripheral, physiological[16], cognitive) are nowadays becoming known. As a con-clusion, an up-to-date view of stress looks at it as a physic-physiologic arousal response occurring in the body as resultof stimuli.

A single-modality approach for measuring the effects ofstress would thus not be suited, as some experimental re-sults demonstrate [17]. In fact, for a sufficiently precise andaccurate measurement of stress, a multi-modal approachmust be considered. The diagram depicted in Figure 1 repre-sents a simplified multi-modal view on stress as consideredin this paper. This diagram is composed of two main parts:the upper part concerns the predictive aspects of stresswhile the lower part concerns the diagnostic aspects.

The Predictive part of the model considers the followingaspects: Context, Profile, Goal and Trait.

Context includes meaningful information to describe thedifferent dimensions of the individual, including the histor-ical, economic, social or geographical contexts. Numerousstudies exist that map such information to a base levelof stress: the effect of socioeconomic status [18], social orgeographical context [19], [20], [21] or individual economicsituation [22], just to name a few.

The Profile of the individual includes personal informa-tion and characteristics that have an ongoing influence onthe level of stress. These include age, gender, marital status,number of dependents [23], type (or lack) of employment[24], job category, among others.

The Goal of the individual at a given moment in timeor, likewise, the objectives, aspirations or ambitions alsohave a significant influence on the level of stress. Namely,individuals with higher ambitions are generally known to

be under increased stress, resulting from the continuedeffort of trying to achieve above average standards [25].

Finally, Trait is related to the personality of each individ-ual, i.e., habitual patterns of behavior, thought or emotion.Some traits are more generally associated with stress thanothers [26]. As an example, an impulsive individual isgenerally a more stressed one, with stress driving his hastydecisions.

In the diagnostic part of the model, a larger numberof components could be included. Namely those orientedtowards psychological or psychosomatic diagnostics, i.e.,subjective self-report mechanisms such as surveys or ques-tionnaires. We however focus on objective measures ratherthan the subjective ones, especially those that can be usedto provide real-time feedback. Thus, the Diagnostic compo-nents of the model include Physical, Physiological, Behav-ioral and Performance aspects.

Physical aspects include, in a general way, body move-ments or postures that may have some particular meaningin terms of stress assessment. Especially interesting areaspects such as eyelid movement, facial expressions, bodymovements (e.g. specific gestures, head movements, repeti-tive movement patterns) or pupil movement and dilatation

Physiological diagnosis aspects are those that providethe most reliable diagnose of stress. In fact, many ap-proaches exist nowadays that can evaluate the level of stressof an individual from physiological indicators with signifi-cant precision, as will be addressed in detail in Section 4.

On the other hand, the behavior of an individual canbe seen as the visible end of his inner self. In that sense,aside from other aspects, behaviors (and especially changesin behaviors) may also be a good indicator of stress effects.Given the scope of this paper, particular attention will bededicated to behaviors when interacting with technologicaldevices or behaviors that can be acquired within technolog-ical environments, non-intrusively.

Finally, the Performance of an individual is significantlyaffected by stress. The optimum level of stress will maximizeperformance. A higher level of stress may increase perfor-mance temporarily but will soon wear the the individual. Alower level of stress will decrease productivity and lead toincreasing lethargy. Thus, tests that evaluate performance ingiven tasks, for which standard performance measurementsare known, can be a good indicator of the effects of stress onthe individual.

From a high-level point of view, two different typesof stress can also be identified: acute and chronic stress.Acute stress comes from recently acknowledged demandsand pressures and from anticipated demands in the nearfuture. On the other hand, chronic stress is long-term, dueto social or health conditions, dysfunctional families, amongmany other issues. This type of stress will have nefariouseffects on the body and mind of the individual, slowlywearing him away day after day. Acute stress, because it isshort-term, won’t do the extensive damage associated withchronic stress, although overtime frequent acute stress maycontribute to the development of chronic stress. Neverthe-less, it will instantaneously influence the performance of theactions being carried out.

Given the broadness of the field, in this paper we clearlyfocus on acute cognitive stress. Indeed, most if not all

JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2015 4

non-intrusive and non-invasive current methods for stressassessment are based on the observation of changes on theindividual (as detailed in Section 5). When considering acutestress, these changes are easily observed as they constitutesignificant deviations from an otherwise regular behavior orstate [27]. Chronic stress, on the other hand, is more difficultto detect using these means as the individual is constantlyexperiencing the effects of stress, thus no abrupt changesare observed [28]. We do not mean to imply that it would beimpossible to accomplish. However, given the characteris-tics of chronic stress, its detection using the means exploredin this paper would require more extensive data collectionabout each individual, spanning longer time-frames.

3.2 Stress in the WorkplaceAs already addressed, stress has effects at many differentlevels and in many different spheres of our daily lives.Nonetheless, the workplace can be pointed out as a gen-erally stressful environment, especially given today’s de-mands for productivity, competitiveness and performance.For this reason, in this section we analyze the specific char-acteristics of stress and stressors in the workplace, especiallyfocusing on the causes and outcomes.

3.2.1 CausesIn a general way, a stressor can take many different formsincluding a chemical or biological agent, environmentalcondition, external stimulus or any event that forces anorganism to adapt to new conditions. Human stressors, inparticular, may include environmental factors such as noiseor over-illumination, daily stress events such as traffic orlack of physical activity, dramatic life changes such as thedeath of a relative or a divorce, workplace stressors suchas job demands or unrealistic objectives, chemical stressorssuch as alcohol or drugs consumption, or social stressorssuch as society’s demands/expectations.

When considering the specific issue of stress in theworkplace, many theories have been proposed to examinepossible causes. Two examples are described below:

• Job demands-control model – this is one of the mostwidely accepted models to study occupational stress[29]. It considers two main causes for stress (Fig-ure 2): (1) psychological demand of the task and(2) worker’s degree of decision/control. (1) includesworking pace, difficulty of the task or conflicts atwork. (2) includes the possibility to be creative andthe autonomy to take decisions about the work andabout the work pace. In the 1980s social support atwork was added [30]. This deals with the amountand quality of the social relationships at work andtheir degree of support. The most stressful milieusare, naturally, those with low control, high demandand weak social suppotr, known as high iso-strainjobs [31], [32];

• Job demands-resources model – alternatively, thismodel regards occupational stress as the result ofan imbalance between two main aspects [33]: (1) jobdemands on the individuals (e.g. physical, psycho-logical, social or organizational aspects of the jobthat require sustained effort/skills) and (2) resourcesthey have to deal with those demands (aspects thatfacilitate the achievement of work goals or reducethe cost of job demands, including opportunity forpersonal development, career opportunities or au-tonomy). Instead of focusing on the negative out-comes of stress alone, this model considers positiveindicators of employee well-being as well (Figure 3).

Less obvious stressors have also been identified andstudied by researchers in the last years. A study conductedin 1995 by researchers of the State University of New Jersey,analyzed the impact of electronic performance monitoringand its social context on the productivity and level of stressof employees [34].

Electronic Performance Monitoring (EPM) systems areone of the many technological developments employees facein today’s workplaces. These systems provide managersa wide range of information about employees’ routinesincluding real-time information such as the pace of work,

Fig. 1: A possible stress recognition model as viewed in this paper. It includes two main groups of aspects: predictiveand diagnostic. Predictive aspects are the ones that can be estimated from the background or context of the individual.Diagnostic aspects are the ones that can be observed and measured and have a relation-ship with the level of stress.

JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2015 5

Fig. 2: The job demands-control model [29].

Fig. 3: The job demands-resource model [33].

degree of accuracy, log-in and log-out times, and eventhe amount of time spent on bathroom breaks. This studyexamined how productivity and subjective experiences areaffected by EPM systems and how the social context of theworkplace moderates that influence. In a survey involvingthe monitored workers, 81% of the respondents declaredthat electronic observation made their jobs more stressful[35]. Another study compared the behavior of monitoredand non-monitored workers who performed similar jobs,and found that monitored workers felt more stressful [36].The introduction of EPM systems can transform ordinaryjobs into high-stress jobs. It can also reduce the opportu-nities for employees to socialize with each other at work,leading to a loss of social support, partially responsible forthe stress associated with EPM [37], [38].

Researchers from the School of Psychology of the Uni-versity of Liverpool have also analyzed the relationshipbetween stress and productivity in the workplace. The re-searchers investigated the predictors of productivity usingthe ”A Shortened Stress Evaluation tool” (ASSET) [13].

The economic and social effects of the existence of stres-sors have also been studied, with results estimating thateach worker experienced an average monthly productivityloss of approximately $200 to $400 due to depression [39].Similarly, another study estimated that the loss of productiv-ity due to depression has cost American corporations $12.1billion in 1990 alone [40].

3.2.2 OutcomesThe research community clearly acknowledges the existenceof a relationship between workspace stressors and mentaland physical health outcomes, which is out of our focus.A smaller amount of research is devoted to the effects ofstressors on workplace indicators such as productivity.

Albeit sparse, there is some evidences that establishes anassociation between stress and productivity. Yeh, Lester, andTauber performed a study on real estate agents that revealeda negative relationship between stress and productivity[41]. Jamal and Baba, using data collected from blue-collar,managerial and nursing employees showed a direct, linearand negative stress-productivity relationship: the greater thestress was, the less productive the workforce was [42].

Several things thus result clearly from this brief analysis:stress in the workplace exists, is increasing with EPM initia-tives and other factors, and has negative effects at severallevels. Interestingly enough, and given the proven effectson productivity, companies should be one of the interestedparties in reducing stress, along with employees who worryabout their own health and well-being.

From the analysis of the literature, several causes forcompanies not implementing active stress management ini-tiatives can be pointed out, namely: (1) their cost; (2) theneed for human experts (namely psychologists); (3) possiblechanges in established work routines; (4) unwillingness ofemployees to participate (namely when they must talk toother people about their stress-related issues). Many of theseissues, if not all, exist due to the characteristics of traditionalstress-measuring approaches, which are, as described in thefollowing section, impractical for the workplace.

There is thus the need to study and develop new waysto assess and manage stress that can be effectively used inthe workplace, that minimize the disadvantages associatedwith these traditional approaches. These new methods aresurveyed in Section 5.

4 TRADITIONAL APPROACHES

Current stress management techniques in organizationshave as a main goal the evaluation of the employees’ stateso as to implement approaches that allow them to cope withthe negative effects of this phenomena [43]. In a general way,these techniques aim to answer two main questions: (1) towhich extent is a given event affecting an employee? and (2)which are the more suited methods to help a collaboratordeal with these effects?. The work described in this paperfocuses on the first question.

After a state of stress has been identified, different ap-proaches can be followed for its management, includingpersonalized training, support or counseling, group therapy,breathing and relaxing exercises or socialization games.Here, the challenge lies in the choice of the appropriatetiming and approach(es). In this process, both the identifica-tion of the occurrence of stress as well as the definition andimplementation of coping strategies are the responsibility ofthe organization.

Currently, the use of Human experts in this task repre-sents a cost that often prevents organizations from adheringto these initiatives [44]. Moreover, there are also issues re-lated to limited availability (of both parts), the eventual need

JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2015 6

for personnel displacements, and the frequent reluctance ofemployees to discuss their issues in the workplace [45]. Anautomatized approach, or at least partly automatized, coulddefinitely constitute an important step towards the increasein the adherence to these initiatives.

Traditionally, two main approaches can be followed toquantify the effects of stress: (1) questionnaires or surveys,used mostly by psychology and (2) physiological sensors,used mostly by medical approaches. Each of these ap-proaches has advantages and disadvantages of its own,when considered to be used in a workplace.

Questionnaires, as other self-reporting mechanisms, areseen as an inexpensive approach to collect vast amountsof information. They do not represent a very significanteffort for the researcher, who also benefits from the easinessin compiling data, which results from a set of predefinedanswers [46]. These instruments are eminently practicaland can be administered either by the researcher or byanyone else, possible remotely, without affecting validity orreliability.

They have, however, a number of disadvantages that gobeyond traditional problems related to the definition andformulation of questions [47]. They are based on individualperceptions of rather subjective concepts such as good, poor,big or low. It is also easy for a participant to (unconsciously)hide information, voluntarily lie or depreciate/over-valuatecertain indicators [48]. This type of behavior is virtuallyunidentifiable by the researcher. Finally, at the moment ofdeveloping the questionnaire, researchers take their owndecisions and assumptions concerning what is or is notimportant. Consequently, even if an individual considers acertain issue as being very important, there is no efficientway to express this if no specific questions regarding thisissue exist in the questionnaire.

Nonetheless, many questionnaires and other instru-ments exist for stress assessment in many different domains(e.g. trauma, family, occupational), which have been vali-dated and used thoroughly over the last decades. Specific in-struments exist for the workplace and similar milieus. Somewidely used instruments include the 30-question ”PerceivedStress Questionnaire” [49], the ”NIOSH Generic Job StressQuestionnaire”, including psychosocial measures such asmental demands, perceived control, workload or job ambi-guity [50] or the ”Copenhagen Psychosocial Questionnaire”(COPSOQ), a comprehensive instrument for the assessmentof psychosocial work load and strain [51]

Technological advances and medical research lead to amore accurate approach to the problem, based on a range ofsensors that measure physiological or neurological effects ofprocesses such as stress, fatigue or emotions on the humanbody. Please note that in this section we are analyzingthe more traditional applications of physiological sensors,tendentiously in the medical context. For more recent appli-cations of these kind of technologies please refer to Section 5.

In this field, one of the most precise indicators is theadrenocorticotropic hormone (ACTH): a hormone producedby the pituitary gland, located below the brain. ACTHactivates glands on the kidneys (adrenal glands) to makecortisol. Cortisol has many functions: it helps the bodyuse sugar (glucose) and fat for energy (metabolism), and ithelps the body manage stress. Cortisol levels can be affected

by many conditions, such as physical or emotional stress,strenuous activity, infection, or injury. When cortisol levelsin the blood rise, the ACTH levels in a healthy personnormally fall in response.

Cortisol can be measured in saliva, hairs or blood [52].Assessment of cortisol in saliva is an especially widely ac-cepted and frequently employed method in psychoneuroen-docrinology [53], due to several advantages over other anal-yses (e.g., stress-free sampling, laboratory independence,lower costs) [54].

Nonetheless, other sensors or combinations of sensors,measuring other physiological manifestations, can be usedfor similar purposes. The rationale behind the analysis ofphysiological signals to study inner states of an individualis based on well-known associations between two maindivisions (parasympathetic and sympathetic) of the Auto-nomic Nervous System (ANS) and numerous physiologicalprocesses around the body [55]. The ANS influences thecardiovascular, respiratory, digestive, urinary and reproduc-tive functions. The parasympathetic division of the ANSstimulates visceral activity and promotes a state of rest in theorganism. In contrast, the sympathetic division of the ANSprepares the body for heightened levels of somatic activitythat may be necessary to implement a reaction to stimulithat disrupt this state of rest. The sympathetic division isthus responsible for the well-known flight-or-fight response,which prepares the body for a scenario that may requiresudden, intense physical activity [55].

Skin conductivity, for instance, measures the skin re-sistance to electric current, which varies according to thelevel of perspiration. Given that sudoriparous glands arecontrolled by the sympathetic nervous system, they unveilmental states associated with psychological or physiologicalarousal, which take place during peaks of stress. Likewise,boredom states can also be detected. Skin temperature, heartrate or respiratory rate are also well-known indicators forthe study of stress, emotions or fatigue [55], [56]. Heartrate variability, defined as the variation of the time betweenheartbeats, has been increasingly used to study stress [57],showing that both are closely connected [58].

The steep growing of biofeedback tools in the last yearsis also worthy of note. These tools combine feedback frommultiple bodily functions, using instruments that analyzeindicators such as brain waves, muscular response, skinconductivity, heart rate, pain perception, among others [59].The study of brain waves is particularly interesting sinceit provides clues about aspects such as fatigue, levels ofstress, arousal or emotional state in a very thorough way,also allowing to compare, at the same time, other related(or not) phenomena. Biofeedback tools can also be used toimprove certain aspects such as daily habits or behaviors,since they provide real-time feedback to the user about theconsequences of their attitudes, decisions or behaviors [60].

In a general way, approaches based on physiologicalsensors can be seen as very precise and are used not onlyto evaluate the state of an individual but also as a basis formedical treatments and intervention. Their use, validity andutility are nowadays acknowledged by research initiativesand medical applications alike [55], [56], [61], [62]. However,in the context of this work, both approaches (physiologicalsensors and questionnaires) are looked at considering their

JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2015 7

use in a real workplace. In that sense, it becomes necessaryto ascertain the extent to which these approaches are suit-able to evaluate the state of an individual in these milieus.Our conviction, based on the rationale detailed below, is thatsuch approaches are not suitable.

When people use a questionnaire to describe themselvesor some of their behaviors, it may happen that their viewsor opinions do not exactly fit the possible answers. To dealwith this issue, individuals often decide not to answer or usethe option that, in their view, more closely relates. Doubtsabout the quantification of the answers are another frequentproblem. While some of the frequently used concepts, suchas never or always, are easy to define, others such as frequentlyor occasionally are less clear. When these questionnairesconcern the behavior of the individual in a given situation(e.g. ”How would you react if, feeling incredibly tired, youwere given a task to complete in a short time frame?”), thereis no guarantee that the behavior of the user in the actual sit-uation would match the answer. That is, individuals answerhow they believe they would or do behave. Nonetheless,stress is partly a subjective psychological experience, i.e., itdepends on how each individual interprets and copes withstressors. From this point of view, self-report mechanismsare still interesting.

When, alternatively, physiological sensors are used, themain and most immediate drawback is that the individ-ual may feel uncomfortable (Figure 4). This may result inrefusal to participate, especially in cases in which the useof sensors involves wires and other hardware that maylimit movement. All these factors make it more difficultto collect the data. On the other hand, there may be anundesirable effect on the variables under study caused bythe monitoring itself: the simple fact that the individualis connected to sensors may increase stress, consequentlyaffecting the results.

These problems can be briefly analyzed through somespecific examples. [63] present an approach based on fourdifferent sensors to detect stress in a non-invasive way.Nonetheless, the method is highly intrusive as the partici-pant cannot move the left hand. The work described in [55]uses, in addition to physiological sensors, a video-based eyetracking gazing system, which poses an additional draw-back concerning privacy. Many other examples can be foundwith very different physiological markers (e.g. galvanic skinresponse in both feet and hands, heart rate variability, elec-trocardiograph signal, electromyography signal) [56], [61],[64] and in different domains of application (e.g. stress instudents during exams, stress in drivers) [65], [66]. Theproblems do however remain: these approaches, althoughundoubtedly accurate and usable in real-time, have manyand significant effects on the routines of the individuals.None of these so-called traditional approaches can be used,in a realistic manner, to quantify the level of stress of anindividual in a workplace. At least, they cannot be usedwithout changing established work routines or withoutinterfering with the individual.

Given this, the following sections detail a new paradigmfor the acquisition of valuable information for stress as-sessment in which the focus is on the behaviors of theindividuals, looked at as mirrors of their inner states. Infact, processes such as stress, fatigue or emotions have

measurable effects not only on our physiology but also onour observable behavior: a healthy individual is able to lookat someone and identify these signs, in an innate way. If weprovide computer systems with the ability to identify andquantify such behaviors and, if a relationship is establishedbetween these behaviors and certain mental states, the dooris open to the development of non-intrusive methods forthe classification of the state of individuals based on theobservation of behavioral indicators. This is the goal thatmotivates the work described in this document.

5 NEW METHODS FOR STRESS ASSESSMENT

New methods for stress assessment were developed inthe last years as a result of an unprecedented evolutionin consumer electronics and miniaturization. Others weremade possible from a better understanding of stress and itseffects on the Human being at several levels: physiological,behavioral or physical. The diversity of alternatives, asshown in this section, allows for solutions to be used inspecific scenarios with increased accuracy and commodity(e.g. driving vehicles, working at the computer). In thissection we analyze in detail the characteristics of each ofthese new methods and in Section 6 we provide a criticalanalysis and a comparison between them.

5.1 WearablesOne of the latest trends in stress management is beingfostered by wearable devices. Indeed, in the last years therewas a major development in consumer electronics, withdevices being used for acquiring physiological signs. They

Fig. 4: Several traditional approaches for stress monitoring:(a) electromiography, (b) salivary cortisol, (c) heart ratevariability and (d) electroencephalogram.

JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2015 8

constitute more comfortable approaches than the traditionalsensors mentioned in Section 4 as they can be worn, as aregular fashion accessory or clothing (Figure 5). They have,thus, some advantages.

Chan et al. provide a thorough overview of the extensiveefforts made in both academia and industry in the researchand development of smart wearable systems for healthmonitoring [67], their driving forces and their future im-pact in healthcare industry. Choi and Gutierrez-Osuna [68]address this issue in a general way, describing the develop-ment of a wearable sensor platform to monitor physiologicalcorrelates of mental stress for ambulatory stress monitoring.The work relies on a Wireless Body Sensor network withspectral features that estimate the balance of the autonomicnervous system by combining information from the powerspectral density of respiration and heart rate variability.

While the two previously mentioned works are rathergeneric, specific examples of application can also be found.In [69], the authors analyze the discriminative power ofelectrodermal activity in distinguishing stress from cogni-tive load, using a wrist-worn device, with sensors placedin strips attached to two fingers. In [62] the authors alsopresent a wearable system for assessing stress, sensible tothe task being carried out by the user so that the user doesnot need to necessarily sit in a chair, as usual. Similarly, [70]presents AutoSense: a wireless sensor suite that collects andprocesses cardiovascular, respiratory, and thermoregularitymeasurements that can inform about the general stress stateof test subjects in their natural environment.

Finally, the work detailed in [71] presents a stress man-agement biofeedback mobile service for everyday use, aid-ing users to reflect on both positive and negative patternsin their behavior. To accomplish this the authors also devel-oped a wearable set of sensors that facilitate data acquisitionand analysis. The main difference from the previous worksis the development of the biofeedback mobile service that,through a set of intuitive interfaces, aids the users in per-ceiving the effects of stress on their daily lives.

Fig. 5: New approaches for taking stress-related measures,based on wearables: (a) a chest belt, (b) a onesie for trackingbabies’ vitals and (c) a wrist band.

5.2 SmartphonesThe evolution witnessed in the field of smartphones in thelast years also led to the emergence of a new paradigm:wellness mobiles. Technological developments make it pos-sible for health-care professionals to have access to com-prehensive real-time patient data. Likewise, users can alsocontinuously track their health on the go, build a compre-hensive history and receive real-time advice or warning [72].Indeed, mobile phones have a growing number and varietyof sensors that can nowadays be leveraged to produce, inthe near future, what can be called as personal wellnessdashboards: devices with the ability to measure our heartrate or body temperature and quickly analyze our state ofhealth. This may make personal health care cost-effective,decreasing the use of emergency care [72].

Some mobile apps take advantage, to some extent, ofthe sensors currently present in smartphones (Figure 6).Although, in many cases, some of these apps lack provenscientific validity, their low cost and their availability makesthem easily reach a significant number of users.

The majority of existing apps use the smartphones’ built-in sensors. Azumio’s Stress Check uses the camera and lightfeatures of the smartphone to measure heart rate. A similarapproach is followed by other apps (e.g. StressViewer).There is also a significant amount of apps dedicated notto measuring stress but to decreasing or coping with it,namely through breathing exercises, with visual or soundaids. Stress Releaser is one such app. Another example isDeStressify, that is based on music and specific exercises.

There are also apps that use specific hardware, such asPIP Relax and Race, which is based on an electrodermalactivity sensor. In this specific app, the user takes part ina race where victory is achieved only by out-relaxing theopponents. A generally competitive activity is thus changedinto a relaxing one, with real-time biofeedback. Similar appsexist for this specific hardware. DroidJacket [73] requires theuse of VitalJacket – a shirt that embeds an electrocardiogramsensor, allowing a continuous monitoring of the patient. Thework described in [74] also uses a specific sensor platform(Personal Biomonitoring System), in parallel with the smart-phone, to monitor the level of stress of the smartphone user.

Other smartphone-based approaches are based on thechanges in the speech production process, that happenduring stress. To this end, these applications use the mi-crophones embedded in the mobile phones. StressSense[75] is one of such applications, based on a classifier thatcan robustly identify stress across multiple individuals indiverse acoustic environments.

There are also authors who look at the behavior ofsmartphone users for stress indicators. Although not ina conclusive manner, in [76] the authors find significantdifferences in location traces, visible bluethooth devices andphone call patterns when comparing stressful with stressfree periods.

5.3 Computer VisionMany different image sources can be used to monitor stress,the most frequently used being the Human face. Althoughcultural differences can intensify facial expression of emo-tions, there is considerable scientific evidence that emotions

JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2015 9

are communicated in distinct facial displays across cultures,age and gender [77]. These approaches can be classifiedas two-dimensional or three-dimensional. Their main dif-ference is that the first tries to recognize features directlyfrom a two-dimensional decomposition/transformation ofthe image, and is generally not sensible to rotations andtranslations of the face.

In [77], the authors apply optical computer recognitionalgorithms to detect facial changes due to low and high-stressor performance demands, with the aim to develop anapproach suitable to be used by astronauts. This approachtakes as input images from the whole face. On a similarapproach but on a different field of application, Gao et al.present a system for detecting stress from facial expressionsin car drivers [78]. Barreto et al., on the other hand, consideronly pupil diameter (together with physiological signals),to assess stress [55], [63]. To this end, they make use of aspecific camera-based eye-tracking system.

Other authors simultaneously look at groups of featuresextracted from the face. In [79] an approach is presentedbased on what the authors call ”physical appearance”: facialexpression, eye movements and head movements. Thesefeatures are used together with physiological signs and be-havioral data to assess the level of stress of a computer user.Thermal imaging can also be used [80], namely to measureblood perfusion in the orbital muscles, which correlates to

Fig. 6: Android apps for measuring stress, with and withoutadditional hardware: (a) Stress Check, (b) Droid Jacked [73]and (c) Stress Tracker [74].

stress. These approaches are however prone to error whencertain types of light or heating systems are used.

There are also authors who look at sequences of imagesto search for signs of stress. Giakoumis et al. analyze videoand accelerometer information to extract activity-related be-havioral features and perceive signs of stress [81]. Sharma etal. also consider video analysis, using both temporal thermalspectrum and visible spectrum video features, which theymake available as a database – ANUStressDB [82].

5.4 Speech and Other Linguistic Features

This section describes approaches for stress assessmentbased on vocal cues such as speed, rhythm or intonation.Interestingly, the variability introduced by stress or emotioncan severely reduce speech recognition accuracy. Thus theimportance of techniques for detecting or assessing thepresence of stress to improve the robustness of speechrecognition systems [84].

In [85], the authors present a hierarchical framework,which consists of three layers of classifiers, for automaticstress detection in English speech utterances: a linguisticclassifier, an acoustic classifier and an AdaBoost classifier.The paper presents accuracy rates higher than 90%.

In a related approach, Imoto et al. address sentence-levelstress detection of English for Computer-Assisted LanguageLearning by Japanese students. Stress models are set up byconsidering syllable structure and position of the syllable ina phrase, providing diagnostic information for students [86].

There are also approaches based on prosodic or acousticfeatures. Xie et al. present an approach for the automatic

Fig. 7: Approaches based totally or partially on computervision: (a) detecting emotional stress on drivers [78], (b)stress detection in Human-Computer Interaction throughgaze detection [83] and (c) stress detection using a visualsensor and an emotional mouse [79].

JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2015 10

detection of rhythmic stress in spoken English, based onspeaker independent prosodic features and vowel qualityfeatures as terminals to classify each vowel segment asstressed or unstressed [87]. Similarly, in [88] an Englishlexical stress detection approach using acoustic features isproposed. The feature set includes the semitone, the dura-tion, the loudness and the emphasis features.

5.5 Computer MouseIn this section we analyze a group of approaches thatcan be deemed as non-intrusive as they rely on the usageof the mouse, requiring no specific additional hardware.This approach can thus be included in the so-called MouseDynamics field.

Different mouse-based approaches can be implemented.On the one hand, it is possible to build sensors into themouse, which collect physiological signs when the useris in contact with the mouse, as proposed in [79]. Photo,pressure, temperature and galvanic skin response sensorswere integrated in a mouse, allowing them to determinewhen the user is using the mouse and acquire physiologicalvariables that correlate to stress. On a similar approach, theauthors of [89] developed a capacitive mouse that measuresthe amount of hand contact with the mouse, allowing theauthors to estimate the pressure exerted on the mouse. Theauthors conclude that mouse contact is significantly higherwhen participants are under stress.

Other authors have looked at the mouse and its usagein search for different stress correlates. Namely, [90] look athow the users move the mouse and propose a simple modelof arm-hand dynamics that captures muscle stiffness duringmouse movement. Other authors have also extracted behav-ioral features from mouse usage, such as in [91], extractingfeatures such as mouse speed, inactivity, or mouse click rate.Finally, features extracted from the mouse have also beenshown to be related to stress measures during exams [92].

5.6 Computer KeyboardThe keyboard of the computer is also being researched as away to assess the effects of stress. One of the most frequentapproaches relies on Keyboard Dynamics, which looks atthe behavior of the user while typing through features suchas key latency or writing speed [93].

Such an approach is followed in [91], in which the au-thors consider three keyboard features: average key latency,average typing speed and occurrence of error keys. In [94]the authors also use seven behavioral features extractedfrom the keyboard, but rely on a Case-based Reasoningsystem for stress classification.

Specially designed keyboards have also been developedto extract additional features that regular keyboards do notprovide. In [89], the authors present a pressure-sensitivekeyboard that provides, for each keystroke, a value of pres-sure between 0 (no pressure) and 255 (maximum pressure)(Figure 8). The authors conclude that the pressure on thekeyboard is significantly higher under stress.

Finally, there are also keyboard-based approaches thatincorporate the linguistic features of the written text. In[83], the authors combine keystroke and linguistic featuresof spontaneous generated text, measuring physical and cog-nitive stress.

Fig. 8: A regular mouse and keyboard can be used to assessstress through behavior. This hardware can also be modifiedto provide additional features. The image depicts a pressure-sensitive keyboard and a capacitive mouse [89].

6 CRITICAL ANALYSIS

6.1 MethodsSection 5 showed that there are many novel methods toassess stress. However, all these different methods havecharacteristics of their own, as well as unique advantagesand disadvantages.

Wearable devices incorporate physiological sensors inclothing or accessories, constituting a very convenient al-ternative to traditional physiological sensors, although theperson still has to ’wear’ the device. The battery life isnowadays relatively large, often allowing the device to beworn for several days. One of the key advantages of theseapproaches is that they can integrate physiological signswhose relationship with stress is nowadays well-known andthoroughly studied (e.g. heart beat, respiratory rate, bodytemperature). This means that very accurate approaches canbe implemented. The main drawback of these approachesis their price, since these devices tend to have a significantcost.

Most of the existing smartphone-based approaches forstress assessment rely on the use of the integrated flashto measure heart rate and provide a quantification of thelevel of stress. This approach has a lower accuracy thanwearable devices, in part because it is based on a singlephysiological sign. Moreover, in order to continuously col-lect data over long periods of time, the user would haveto constantly touch the light of the smartphone, which isimpracticable. This type of solution is thus more suited toperiodic analyses. On the other hand, the main advantageof these approaches is that they can be used by anyone whoalready owns a smartphone, generally by just installing asimple app that also makes the logging, visualization andsharing of information very easy. Nonetheless, the evolutionof smartphones, namely through the inclusion of additionalsensors, may open the door to more accurate approaches.Smartphones have, undoubtedly, a significant role to playin the future of personal healthcare.

Some of the most well-studied approaches to stress de-tection are based on video-cameras. Many algorithms areknown nowadays which not only detect or quantify stressbut also assess emotional state and other cues. The costof these approaches can vary significantly, depending onthe quality of the video camera used. Moreover, they tendto only be accurate when the user stares frontally at thecamera and with proper lightning conditions. Under theseconditions, video-cameras can prove useful and comfortablein assessing stress, since the user does not need to be

JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2015 11

connected to sensors or do any specific task. However,the user does need to remain relatively still and face thecamera directly. For this reason, these approaches are mostlydirected at tasks with these characteristics, such as workingwith a computer or driving a vehicle. Nonetheless, themost negative aspect of this approach concerns privacy.Indeed, people often dislike being monitored, especially inthe workplace and in such a direct way, which may, by itself,influence stress levels.

Speech- and linguistic-based approaches face a similardrawback as users may look at the monitoring of theirspeech or their words as an invasion of their privacy. To copewith this, people often change their normal behaviors (e.g.avoiding conversations that they would usually have), thusundermining the process itself. While some features do takeinto account specific words and may pose these problems,others are based on how the person talks rather than on whatthe person talks (e.g. speech rhythm). These features shouldthus be preferred and the users should be made aware oftheir characteristics in order to increase acceptance. Otherthan that, these approaches are generally inexpensive asthey are based either on written text or on speech acquiredthough a microphone, which can be embedded in existingdevices, such as in the case of smartphones. They do, how-ever, require that the person speaks or types text in orderto produce a result. For this reason, they tend to be moresuited to specific domains (e.g. call centers).

Finally, approaches based on computer peripherals alsoconstitute inexpensive and interesting ways of assessingstress. The main drawback is, evidently, that they can onlybe used in domains in which people interact with a com-puter. They are thus directed at environments such as labo-ratories, workplaces or academia. People also often expressconcerns with these kind of logger applications that registerall that is done with the mouse and the keyboard. The mostimportant step to take regarding this concern is to focuson features that (as with speech-based approaches) do notconsider what is written but how it is written. Similarly, whenconsidering the mouse, features should focus on how peopleclick or move the mouse rather than where people clicked ormoved to. Fortunately, if some of the linguistic features areleft aside, most of the features extracted from the mouse andthe keyboard pose no concerns in this regard. The strongestaspects of these approaches are: (1) a very low cost sincethey are generally based on existing and inexpensive hard-ware; and (2) the diversity of features that can be extractedwhich, depending on minor hardware modifications, mayinclude physical, behavioral and physiological measures.

Table 1 presents a summary of the characteristics consid-ered to assess each stress assessment method and a score (1- lowest, 5 – highest) that allows for an intuitive comparison.The following characteristics were taken into consideration:

• Versatility – Quantifies how fit the method is to beused in different domains;

• Cost-effectiveness – The cost-effectiveness of themethod (e.g. additional hardware, cost of associ-ated software). The value 1 denotes a low cost-effectiveness, i.e., a more expensive method;

• Intrusiveness – Quantifies the extent to which theroutine of the individual is affected by the stress

Vers

atili

ty

Cos

t-ef

fect

iven

ess

Intr

usiv

enes

s

Feat

ure

Div

ersi

ty

Spec

ific

Har

dwar

e

Ava

ilabi

lity

Priv

acy

Ric

hnes

s

Acc

urac

y

Wearables 5 1 4 5 1 3 5 3 5Smartphones 3 2 3 5 3 5 5 5 3

Computer Vision 2 1 5 3 2 3 1 2 5Speech & Linguistic 2 4 5 3 4 5 2 2 4

Mouse 2 5 5 4 5 5 5 3 3Keyboard 2 5 5 4 5 5 5 4 4

TABLE 1: Comparison of the different methods studied forstress assessment.

assessment method. The value 1 denotes that themethod is very intrusive while the value 5 denotesthat it is completely transparent to the user. For ex-ample video cameras or keyboard are not consideredintrusive since their use for the purpose of stressmonitoring does not have an effect on work routines;

• Feature Diversity – While some methods provide asmall number of features or features from a reducednumber of modalities (e.g. physiological, behavioral,physical), others give access to a larger number andvariety. Multi-modal approaches generally hold po-tential for increased performance;

• Specific Hardware – Quantifies the degree to whichspecific additional hardware is required for themethod to assess stress. The value 1 denotes thata significant amount of additional hardware is re-quired for the method to be used while the value 5denotes that no additional hardware besides what isavailable in a general scenario is necessary;

• Availability – Determines to which extent the methodis easily available, from the point of view of the user.As an example, simply downloading an applicationis very convenient;

• Privacy – Quantifies the extent to which a givenmethod can constitute a potential threat to pri-vacy. The value 1 denotes a potentially threateningmethod;

• Richness – This characteristic compares the methodsin terms of the richness of analysis that can be com-bined in a single device. For example, a smarphoneallows the acquisition of behavioral, physical andphysiological features;

• Accuracy – Denotes, in a general way, the accuracy ofthe approaches concerning stress classification.

The data displayed in Table 1 can be graphically sum-marized to allow a more intuitive interpretation. Figure 9shows how each of the methods studied scores in each ofthe 9 categories evaluated. Through this graphical repre-sentation, the methods can be compared using the area ofthe radar plot. Thus, in the overall, the best method forstress assessment in the workplace, according to the analysiscarried out, is the keyboard, with a score of 54.32 (out of amaximum value of 72.48). Likewise, the worst methods forthis purpose are those based on computer vision, mostly dueto their privacy concerns, cost and requirements in terms ofhardware. Figure 10 details and compares the score of eachmethod as well as the maximum possible score.

JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2015 12

6.2 Characteristics

It is likewise interesting to conduct this analysis from thepoint of view of each of the characteristics studied (espe-cially in cases in which one or several characteristics aremore important than others). Figure 11 supports this kindof analysis.

In terms of versatility, methods based on smartphonesand wearables are those that score higher, especially in thecase of wearables. Wearables can provide a wide range ofphysiological measures, as well as other features extractedfrom hardware such as accelerometers. In the case of smart-phones, their versatility comes from the ability to developspecific and custom applications, that use not only thebuilt-in sensors but also external hardware (e.g. heart ratemonitors).

In what concerns cost-effectiveness, methods based onthe keyboard and mouse appear as the best, followed closelyby speech and linguistic. Indeed, these computer periph-erals are common in modern workplaces, especially thoselinked to the so-called ’white-colar’ jobs, and their cost isnowadays very low, making them a cost-effective approachfor continuous and non-intrusive stress assessment.

Of all the characteristics considered, intrusiveness is theone that achieves a higher score, i.e., the one that is morebroadly contained in all methods. Apart from smartphonesand wearables (in which people need to perform a specificaction or wear a specific fabric), stress is assessed from the

regular actions of the individual with the devices in theenvironment. This makes these methods highly transparentand unobtrusive.

Feature diversity also scores relatively high, especiallydue to the contribution of smartphones and wearables,which can provide a rich set of features for stress assess-ment.

Considering the need for additional/specific hardware,the best methods are, once again, those based on keyboard,mouse or speech. Indeed, when considering modern work-places, the mouse and the keyboard are nowadays common.The same is almost as true for microphones since most of ourlaptops or smartphones have embedded microphones thatcan be used for stress assessment. For these reasons, thischaracteristic has a very similar score to cost-effectiveness.On the other hand, methods based on computer visionrequire video cameras, smartphones may require additionalsensors or hardware and methods based on wearables nec-essarily require specific hardware, contributing to worsescores.

In terms of availability, the highest scores belong tomethods based on the smartphone, keyboard, mouse andspeech. This is due to the fact that these devices are nowa-days easily available and that it is only necessary to installa specific software to start assessing stress.

Concerning privacy, the highest scores are attributedto smartphones, wearables and computer peripherals. In

Fig. 9: The score of each method in each of the 9 characteristics evaluated.

JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2015 13

Fig. 10: Score of each method studied, as well as the maxi-mum score.

the case of the first two, these devices are tendentiouslypersonal, which gives the user increased confidence on theiruse. In the case of computer peripherals, they can be usedto assess stress in a safe way, i.e., without actually knowingwhat the person is typing or where the person is clicking.Speech- and video-based approaches score lower in terms ofprivacy. The former mostly because the user needs to speakout in order for the stress assessment to take place. The lattermostly because of all the privacy-related issues associated tothe use of video cameras, especially in the workplace.

In terms of the richness of the analysis that each methodallows, the methods based on smartphones and on periph-erals are the ones that achieve highest scores. The score isespecially high since it may provide access to features of dif-ferent modalities, including behavioral (namely applicationusage), physical and physiological. Similar statements canbe made about the mouse and the keyboard.

Finally, in terms of accuracy, the methods that achievethe best scores are those based on computer vision and onwearable devices, especially the ones incorporating phys-iological sensors, which are among the most accurate ap-proaches to assess stress.

Pointing out an ”absolute best” method is rather impos-sible as this always depends on the characteristics and con-straints of each specific setting. However, and focusing onthe specific problem of stress assessment in the workplace,the methods that achieve generally best results are thosebased on mouse and keyboard, also achieving good scoreson characteristics such as cost-effectiveness, intrusiveness,need for specific hardware, availability and privacy. Theseconclusions, however, apply mostly to office-type jobs, inwhich people sit at the computer for a significant partof their workday. When considering other jobs, alternativemethods could be more appropriate. As an example, smart-phones or wearables are a more suited method to assessstress in health professionals.

6.3 Stress Modalities

Alternatively, these methods can also be analyzed from theirmultimodality, i.e., their ability to capture stress effects intheir different modalities. The underlying assumption is thata method that can measure effects from multiple modalitiesis, expectedly, more accurate than a method based on asingle modality. The modalities considered are those put

Phys

ical

Phys

iolo

gica

l

Beha

vior

al

Perf

orm

ance

Wearables 2 5 2 1Smartphones 2 3 4 4

Computer Vision 4 2 2 1Speech & Linguistic 1 1 4 1

Mouse 2 2 5 5Keyboard 2 1 5 5

TABLE 2: Score of multimodality.

forward in the diagnostic part of the model presented in Fig-ure 1: physical, physiological, behavioral and performance.We score each method (1 – lowest, 5 – highest) accordingto their ability to produce features from each of the fourmodalities put forward in the model (Table 2).

Wearables score the highest in the physiological modal-ity. Indeed, these methods are mostly based on physiologicalsensors that can be placed directly on the body of the indi-vidual, ensuring a continuous monitoring of physiologicalsigns. These methods can also provide some informationregarding physical/behavioral modalities, mostly throughaccelerometers, allowing the acquisition of features such asmovement patterns, activity levels or activity classification.

Smartphones stand out especially in the behavioral andperformance modalities. Behavioral features are acquiredfrom aspects such as application usage patterns, user lo-cation or from the accelerometer or gyroscope of the de-vice. Concerning performance, smartphones can providefeatures such as accuracy measures or response times,namely through the use of specific or modified applica-tions. Physiological measures can also be acquired from asmartphone, either from the built-in camera or from addi-tional sensors. Finally, smartphones can also provide somephysical features such as the intensity of the touch on thescreen or the acceleration measured on the device when theuser is interacting with it. All this makes smartphone-basedmethods one of the most multimodal ones.

Methods based on computer vision are essentially di-rected at physical measures, including eyelid movement,facial expressions, head movement, or pupil dilatation.Nonetheless, some features from physiological and behav-ioral modalities can also be extracted. As an example of thephysiological modality, a group of researchers from MIT’sComputer Science and Artificial Intelligence Lab (CSAIL)showed that it is possible to measure human heart rate andheart rate variability in ordinary video footage. In what con-cerns the behavioral modality, methods based on computervision allow the analysis of aspects such as changes in focus(attentive behavior) or gaze detection.

Methods based on speech and linguistic features areamong the more limited ones in terms of the diversity ofmodalities. Indeed, these methods provide access to mostlybehavioral features.

Finally, there are the two groups of methods based onthe computer peripherals (mouse and keyboard), whichare among the most multimodal ones and achieve similarscores. The highest scores of these methods are achieved inthe behavioral and performance modalities. To some extentit is also possible to extract physical features from thesedevices, especially in what concerns physical fatigue from

JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2015 14

long periods of use. Moreover, some of the features that weassociate here with behavior are, in part, also influencedby physical aspects of the individual (e.g. developmentof certain muscles, elasticity of certain tendons). Finally,we also surveyed some works in which researchers mod-ified the standard keyboard and mouse to provide physi-cal/physiological features (e.g. pressure sensitive keyboard,capacitive mouse).

The score of multimodality of each method can thus becomputed, as done before, by calculating the area of theradar chart for each method, as shown in Figure 12.

7 DISCUSSION AND FUTURE TRENDS

From the papers surveyed in this document, one first con-clusion is evident: there is nowadays an unprecedenteddiversity in methods for stress assessment. Until a fewyears ago, stress assessment would be carried out through

questionnaires or through invasive or even intrusive ap-proaches. While these are suited for medical interventionsor treatment, they are not adequate for more modern appli-cations, in line with Ambient Intelligence and the paradigmof personal healthcare. Especially, and if we keep in mindthe scope of this research line, they are not suited to beused in the workplace, in which the aim is to monitorcontinuously and in real time, without interfering with theworker’s routine.

Another conclusion is that existing non-intrusive/non-invasive methods are more suited to measure acute stressthan chronic stress, as pointed out at the end of Section 3.It is also a fact that it is really the chronic stress that costscompanies billions of dollars and hurts workers the most[28]. This could result, at a first glance, in the deemingof the surveyed methods as inadequate or insufficient tosolve the addressed problem. However, it must be kept in

Fig. 11: The extent to which each of the characteristics studied is contained in each method.

JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2015 15

mind that chronic stress occurs, among other issues suchas exposure to traumatic events, in response to frequenteverday stressors that are ignored or poorly managed. Thus,it is our conviction that the surveyed methods may have animportant role in the process: helping to prevent the actualoccurrence of chronic stress by supporting the identificationand management of everyday peaks of acute stress.

On the other hand, consumers are nowadays very keenon inexpensive methods to monitor and log their health andwell being. These methods can be implemented with thesupport of smartphones and other similar devices which, bythemselves or with the support of additional hardware, cancollect information about the user. This may, however, fallshort. While smartphone-based approaches are still limited,namely in terms of the features that they collect, they requireusers to stop what they are doing in order to take a measure-ment (e.g. placing a finger on the light of the smartphoneto measure heart rate). Moreover, they require the user toconsciously engage in the process.

Other very different approaches can also be imple-mented nowadays, namely based on video cameras, mi-crophones or wearable devices. These approaches are, un-doubtedly, suited to assess stress in certain domains. Video-based approaches are especially suited to vehicle drivingas they can accurately point out driver stress or fatigue,mainly because the driver must look ahead at all times,thus constantly facing a frontally placed camera. However,such approaches are not suited to the common workplace,for two main reasons: (1) they require at least one cameraper person, which makes them expensive; (2) they areseen as a privacy-threatening and may acquire images even

Fig. 12: Score of multimodality of each method.

from employees who did not agree to be monitored. Ap-proaches based on speech are significantly less expensive.They do, however, suffer a similar problem in the sensethat the speech of the employees must be acquired and pro-cessed. This is especially worrying when linguistic featuresbased on the type of words used are employed. Moreover,microphone-based approaches are prone to errors in noisyenvironments.

Approaches based on wearable devices, which incorpo-rate physiological sensors, are generally more accurate andallow the user to move freely around the environment. Theydo, however, require users to constantly wear one or morepieces of clothing or accessories. Moreover, they also tendto be expensive, especially if the aim is to monitor groupsof people in an organization. This can constitute an obstacleto the implementation of initiatives for monitoring stress orwell-being in the workplace.

When this is the aim, i.e., to monitor stress in mod-ern workplaces, a special interest must be placed in theparadigm of Ambient Intelligence, in the sense that thesetechnologically empowered environments can simultane-ously be sensitive and transparent. That is, in AmI systemsthe user is constantly being monitored in a way that iscompletely non-intrusive and transparent. Ultimately, theuser forgets about the monitoring and notices only theenvironment’s contextualized actions.

This new view on the problem can be made possiblethrough behavioral analysis. Under this approach, every-thing the user does (e.g. interactions with devices, move-ment patterns, interactions with other users) can be used asa potential input. Moreover, one can consider not only whatthe user does but how the user does it.

In fact, our behaviors are commonly associated with ourinner states. We look at someone who is restless, biting thenails or fiddling and we instantly know that the person isnervous or stressed. We look at someone who is movingslowly, whose eyes are half closed and who gets distractedeasily and we know that the person is tired. The fact isthat, in an interaction, our behaviors often give away moreinformation than the words we use. And we, as humans,have evolved to collect this information to, even in anunconscious way, better understand the state of the otherindividual. This information is actually paramount for theefficiency of the communication process.

The challenge thus lies in developing ways to acquirethis information and use it as a way to perceive the user’sinner state. Indeed, many of our behaviors can be usedas input to classify our state. Namely, the way we type,the way we move the mouse, the way we hold or touchour smartphone, the way we talk or even the way we sit.While one of these features may not be enough to accuratelydescribe the user’s state, their combined use may constitutea reliable source of information.

The main advantage of this approach is, undoubtedly,that it can be used continuously throughout the day, withoutinterfering with the users’ routines. It is transparent, non-intrusive and pervasive. It allows for behavioral modelsto be trained in short time-frames that allow us to knowone’s frequent behaviors when in neutral states as well asin specific states. These models can be dependent on many

JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2015 16

variables (that can also be acquired by the environment)including geographical, social or historic context.

Given this, the most suited approaches to assess stress inthe workplace are, in our opinion, the behavioral ones [95].This is especially so in workplaces in which employees inter-act with a computer for long periods, such as the so-calledwhite-collar jobs. In these scenarios, approaches based onkeyboard and mouse dynamics are especially suited. Specif-ically: (1) they are not intrusive as they are based on theinteraction of the individual with the computer; (2) theyare not expensive as they are based on the mouse and thekeyboard; (3) they safeguard privacy as it is not necessaryto determine what the employees are doing, only how theyare doing it; (4) they allow the training and developmentof fairly simple models, based on features that are easilyacquired and processed; (5) they can be used continuouslythroughout the day, providing real-time feedback about thestate of the individual; and (6) they can provide informationabout other important aspects, namely the level of mentalfatigue [96].

One key conclusion of this survey is thus that if weconsider modern workplaces, the best suited approach forcontinuously assessing stress is a behavioral one, basedon the analysis of the interaction patterns of the workerswith the computer. Moreover, stress management tecniquesshould be considered as stress may have positive effectseven in the workplace [97], especially for small periods oftime.

Nonetheless, such approaches should always take intoconsideration individual characteristics (i.e. some people arenaturally more stressed than others) and contextual factors(e.g. in some scenarios, such as brainstorming, an increasedlevel of stress is more acceptable than in others). This canonly be achieved through the development of individual-ized models, that shape each individual’s characteristics.

In order for this field to continue to develop towardshigh reliability and acceptance, we believe that future effortsshould be guided by the following main lines:

• Personalized models should be trained to shape eachindividual’s reaction to stress, also considering otherimportant aspects such as workload, context, taskdifficulty, etc.;

• A framework for stress must be defined that identi-fies and incorporates the key stressors in the work-place and their effect on the level of stress;

• Stress management/reduction techniques must takeinto consideration the fact that not all individualsand situations are alike.

In terms of the ”goodness” of each method, it is impos-sible to point out, in absolute terms, the best method. Ina general way, the methods that achieve the highest scoreare those based on the mouse and the keyboard. Thesemethods also achieve high scores in characteristics such ascost-effectiveness, low intrusiveness or privacy. However, ifthe focus of the organization is on accuracy, methods such ascomputer vision or wearables should be considered instead(or in addition). These methods are, however, also those thatrepresent the higher cost.

The best method or group of methods for assessing stressin a given environment will always depend on the charac-

teristics of the environment (e.g. are there video camerasavailable?, Do the individuals interact with the mouse andthe keyboard?), on the constraints (e.g. is the cost of themethod a limitation?, Is it possible to record speech?) andon the weight of each characteristic for the organization (e.g.privacy, accuracy, versatility).

ACKNOWLEDGMENT

This work has been supported by COMPETE: POCI-01-0145-FEDER-007043 and FCT – Fundação para a Ciência eTecnologia within the Project Scope: UID/CEC/00319/2013.

REFERENCES

[1] T. A. Beehr and R. S. Bhagat, Human stress and cognition in organi-zations: An integrated perspective. Wiley-Interscience, 1985.

[2] T. A. Beehr, Psychological Stress in the Workplace (Psychology Re-vivals). Routledge, 2014.

[3] D. Carneiro, P. Novais, R. Costa, P. Gomes, and J. Neves, “Emon:embodied monitorization,” in European Conference on Ambient In-telligence. Springer, 2009, pp. 133–142.

[4] S. Jones, S. Hara, and J. C. Augusto, “efriend: an ethical frameworkfor intelligent environments development,” Ethics and InformationTechnology, vol. 17, no. 1, pp. 11–25, 2015.

[5] D. Carneiro, A. Pimenta, S. Gonçalves, J. Neves, and P. Novais,“Monitoring and improving performance in human–computerinteraction,” Concurrency and Computation: Practice and Experience,2015.

[6] M. Weiser, “The computer for the 21st century,” SIGMOBILE Mob.Comput. Commun. Rev., vol. 3, no. 3, pp. 3–11, Jul. 1999. [Online].Available: http://doi.acm.org/10.1145/329124.329126

[7] E. Aarts, R. Harwig, and S. M, Ambient Intelligence. McGraw-Hill,Inc., 2001.

[8] J. C. Augusto, V. Callaghan, D. Cook, A. Kameas, and I. Satoh,“Intelligent environments: a manifesto,” Human-Centric Computingand Information Sciences, vol. 3, no. 1, pp. 1–18, 2013.

[9] J. C. Augusto, “Ambient intelligence: Basic concepts and applica-tions,” in Software and Data Technologies. Springer, 2006, pp. 16–26.

[10] G. van den Broek, F. Cavallo, and C. Wehrmann, AALIANCEambient assisted living roadmap. IOS press, 2010, vol. 6.

[11] J. M. Fernandez, J. C. Augusto, R. Seepold, and N. M. Madrid,“A sensor technology survey for a stress-aware trading process,”Systems, Man, and Cybernetics, Part C: Applications and Reviews,IEEE Transactions on, vol. 42, no. 6, pp. 809–824, 2012.

[12] G. P. Chrousos and P. W. Gold, “The concepts of stress and stresssystem disorders: overview of physical and behavioral homeosta-sis,” Jama, vol. 267, no. 9, pp. 1244–1252, 1992.

[13] I. Donald, P. Taylor, S. Johnson, C. Cooper, S. Cartwright, andS. Robertson, “Work environments, stress, and productivity: Anexamination using asset.” International Journal of Stress Manage-ment, vol. 12, no. 4, p. 409, 2005.

[14] J. M. Weiss, “Somatic effects of predictable and unpredictableshock.” Psychosomatic Medicine, vol. 32, no. 4, pp. 397–408, 1970.

[15] H. Selye, “The stress of life. edição revista,” 1978.[16] J. E. Dimsdale and J. Moss, “Plasma catecholamines in stress and

exercise,” Jama, vol. 243, no. 4, pp. 340–342, 1980.[17] W. Liao, W. Zhang, Z. Zhu, Q. Ji, and W. D. Gray, “Toward a

decision-theoretic framework for affect recognition and user as-sistance,” International Journal of Human-Computer Studies, vol. 64,no. 9, pp. 847–873, 2006.

[18] A. Soylu, S. Kavukçu, M. Türkmen, N. Çabuk, and M. Duman,“Effect of socioeconomic status on the blood pressure in childrenliving in a developing country,” Pediatrics International, vol. 42,no. 1, pp. 37–42, 2000.

[19] G. W. Evans, M. Bullinger, and S. Hygge, “Chronic noise exposureand physiological response: A prospective study of children livingunder environmental stress,” Psychological science, vol. 9, no. 1, pp.75–77, 1998.

[20] S. E. Stockdale, K. B. Wells, L. Tang, T. R. Belin, L. Zhang, andC. D. Sherbourne, “The importance of social context: neighbor-hood stressors, stress-buffering mechanisms, and alcohol, drug,and mental health disorders,” Social science & medicine, vol. 65,no. 9, pp. 1867–1881, 2007.

JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2015 17

[21] I. H. Meyer, “Prejudice, social stress, and mental health in lesbian,gay, and bisexual populations: conceptual issues and researchevidence.” Psychological bulletin, vol. 129, no. 5, p. 674, 2003.

[22] G. H. Elder and A. Caspi, “Economic stress in lives: Developmen-tal perspectives,” Journal of Social Issues, vol. 44, no. 4, pp. 25–45,1988.

[23] J. O. Berry and W. H. Jones, “The parental stress scale: Initialpsychometric evidence,” Journal of Social and Personal Relationships,vol. 12, no. 3, pp. 463–472, 1995.

[24] L. Fagin, “Stress and unemployment,” Stress medicine, vol. 1, no. 1,pp. 27–36, 1985.

[25] S. T. Tidd and R. A. Friedman, “Conflict style and coping with roleconflict: An extension of the uncertainty model of work stress,”International Journal of Conflict Management, vol. 13, no. 3, pp. 236–257, 2002.

[26] M. Vollrath, “Personality and stress,” Scandinavian Journal of Psy-chology, vol. 42, no. 4, pp. 335–347, 2001.

[27] T. W. Colligan and E. M. Higgins, “Workplace stress: Etiology andconsequences,” Journal of workplace behavioral health, vol. 21, no. 2,pp. 89–97, 2006.

[28] B. H. Gottlieb, Coping with chronic stress. Springer Science &Business Media, 2013.

[29] T. Theorell and R. A. Karasek, “Current issues relating to psy-chosocial job strain and cardiovascular disease research.” Journalof occupational health psychology, vol. 1, no. 1, p. 9, 1996.

[30] J. V. Johnson and E. M. Hall, “Job strain, work place social support,and cardiovascular disease: a cross-sectional study of a randomsample of the swedish working population.” American journal ofpublic health, vol. 78, no. 10, pp. 1336–1342, 1988.

[31] N. Krause, “Social support, stress, and well-being among olderadults,” Journal of gerontology, vol. 41, no. 4, pp. 512–519, 1986.

[32] K. Daniels and A. Guppy, “Occupational stress, social support, jobcontrol, and psychological well-being,” Human Relations, vol. 47,no. 12, pp. 1523–1544, 1994.

[33] A. B. Bakker and E. Demerouti, “The job demands-resourcesmodel: State of the art,” Journal of managerial psychology, vol. 22,no. 3, pp. 309–328, 2007.

[34] J. R. Aiello and K. J. Kolb, “Electronic performance monitoringand social context: impact on productivity and stress.” Journal ofApplied Psychology, vol. 80, no. 3, p. 339, 1995.

[35] L. Gallatin, Electronic Monitoring in the Workplace: Supervision OrSurveillance? CNOT (241 St. Botolph St., Boston, MA 02115), 1989.

[36] R. Irving, C. A. Higgins, and F. R. Safayeni, “Computerized per-formance monitoring systems: Use and abuse,” Communications ofthe ACM, vol. 29, no. 8, pp. 794–801, 1986.

[37] M. J. Smith, P. Carayon, K. J. Sanders, S.-Y. Lim, and D. LeGrande,“Employee stress and health complaints in jobs with and withoutelectronic performance monitoring,” Applied Ergonomics, vol. 23,no. 1, pp. 17–27, 1992.

[38] B. C. Amick and M. J. Smith, “Stress, computer-based workmonitoring and measurement systems: A conceptual overview,”Applied Ergonomics, vol. 23, no. 1, pp. 6–16, 1992.

[39] R. C. Kessler, C. Barber, H. G. Birnbaum, R. G. Frank, P. E.Greenberg, R. M. Rose, G. E. Simon, and P. Wang, “Depressionin the workplace: effects on short-term disability,” Health affairs,vol. 18, no. 5, pp. 163–171, 1999.

[40] P. E. Greenberg, L. E. Stiglin, S. N. Finkelstein, and E. R. Berndt,“The economic burden of depression in 1990.” Journal of ClinicalPsychiatry, 1993.

[41] B. Yang, D. Lester, and S. WACHTER, “Subjective stress andproductivity in real estate sales people: a replication,” Psychologicalreports, vol. 67, no. 3, pp. 738–738, 1990.

[42] M. Jamal and V. Baba, “Stressful jobs and employee productiv-ity: Results from studies on managers, blue-collar workers andnurses,” International Journal of Management, vol. 9, no. 1, pp. 62–67, 1992.

[43] D. L. Jones, T. Tanigawa, and S. M. Weiss, “Stress managementand workplace disability in the us, europe and japan.” Journal ofoccupational health, vol. 45, no. 1, pp. 1–7, 2003.

[44] C. I. of Personnel and Development. (2008) Building thebusiness case for managing stress in the workplace. [On-line]. Available: https://www.cipd.co.uk/hr-resources/guides/building-business-case-managing-stress-workplace.aspx

[45] K. P. Eisen, G. J. Allen, M. Bollash, and L. S. Pescatello, “Stressmanagement in the workplace: A comparison of a computer-basedand an in-person stress-management intervention,” Computers inHuman Behavior, vol. 24, no. 2, pp. 486–496, 2008.

[46] S. Ackroyd, Data collection in context. Longman Group UnitedKingdom, 1992.

[47] K. Popper, The logic of scientific discovery. Routledge, 2005.[48] J. Milne, “Questionnaires: advantages and disadvantages,” Evalu-

ation cookbook, 1999.[49] S. Levenstein, C. Prantera, V. Varvo, M. L. Scribano, E. Berto,

C. Luzi, and A. Andreoli, “Development of the perceived stressquestionnaire: a new tool for psychosomatic research,” Journal ofpsychosomatic research, vol. 37, no. 1, pp. 19–32, 1993.

[50] T. Haratani, N. Kawakami, and S. Araki, “Reliability and validityof the japanese version of niosh generic job stress questionnaire,”Jpn J Ind Health, vol. 35, p. S214, 1993.

[51] M. Nübling, U. Stößel, H.-M. Hasselhorn, M. Michaelis, andF. Hofmann, “Measuring psychological stress and strain at work-evaluation of the copsoq questionnaire in germany,” GMS Psycho-Social Medicine, vol. 3, 2006.

[52] S. M. Staufenbiel, B. W. Penninx, A. T. Spijker, B. M. Elzinga,and E. F. van Rossum, “Hair cortisol, stress exposure, and mentalhealth in humans: a systematic review,” Psychoneuroendocrinology,vol. 38, no. 8, pp. 1220–1235, 2013.

[53] C. Kirschbaum and D. H. Hellhammer, “Salivary cortisol in psy-choneuroendocrine research: recent developments and applica-tions,” Psychoneuroendocrinology, vol. 19, no. 4, pp. 313–333, 1994.

[54] D. H. Hellhammer, S. Wüst, and B. M. Kudielka, “Salivary cor-tisol as a biomarker in stress research,” Psychoneuroendocrinology,vol. 34, no. 2, pp. 163–171, 2009.

[55] A. Barreto, J. Zhai, and M. Adjouadi, “Non-intrusive physiologicalmonitoring for automated stress detection in human-computerinteraction,” in Human–Computer Interaction. Springer, 2007, pp.29–38.

[56] J. Healey, R. W. Picard et al., “Detecting stress during real-worlddriving tasks using physiological sensors,” Intelligent Transporta-tion Systems, IEEE Transactions on, vol. 6, no. 2, pp. 156–166, 2005.

[57] C. Brüser, K. Stadlthanner, S. De Waele, and S. Leonhardt, “Adap-tive beat-to-beat heart rate estimation in ballistocardiograms,”Information Technology in Biomedicine, IEEE Transactions on, vol. 15,no. 5, pp. 778–786, 2011.

[58] L. Bernardi, J. Wdowczyk-Szulc, C. Valenti, S. Castoldi, C. Passino,G. Spadacini, and P. Sleight, “Effects of controlled breathing,mental activity and mental stress with or without verbalization onheart rate variability,” Journal of the American College of Cardiology,vol. 35, no. 6, pp. 1462–1469, 2000.

[59] M. S. Schwartz and F. E. Andrasik, Biofeedback: A practitioner’s guide. Guilford Press, 2003.

[60] J. F. Lubar, “Discourse on the development of eeg diagnostics andbiofeedback for attention-deficit/hyperactivity disorders,” Biofeed-back and Self-regulation, vol. 16, no. 3, pp. 201–225, 1991.

[61] A. de Santos Sierra, C. S. Ávila, J. G. Casanova, and G. B. D.Pozo, “A stress-detection system based on physiological signalsand fuzzy logic,” Industrial Electronics, IEEE Transactions on, vol. 58,no. 10, pp. 4857–4865, 2011.

[62] F.-T. Sun, C. Kuo, H.-T. Cheng, S. Buthpitiya, P. Collins, andM. Griss, “Activity-aware mental stress detection using physi-ological sensors,” in Mobile computing, applications, and services.Springer, 2010, pp. 211–230.

[63] J. Zhai and A. Barreto, “Stress detection in computer users throughnon-invasive monitoring of physiological signals,” Blood, vol. 5,p. 0, 2008.

[64] M. Jiang and Z. Wang, “A method for stress detection based onfcm algorithm,” in Image and Signal Processing, 2009. CISP’09. 2ndInternational Congress on. IEEE, 2009, pp. 1–5.

[65] J. Bakker, M. Pechenizkiy, and N. Sidorova, “What’s your currentstress level? detection of stress patterns from gsr sensor data,”in Data Mining Workshops (ICDMW), 2011 IEEE 11th InternationalConference on. IEEE, 2011, pp. 573–580.

[66] P. Melillo, M. Bracale, L. Pecchia et al., “Nonlinear heart rate vari-ability features for real-life stress detection. case study: studentsunder stress due to university examination,” Biomed Eng Online,vol. 10, no. 1, p. 96, 2011.

[67] M. Chan, D. Estève, J.-Y. Fourniols, C. Escriba, and E. Campo,“Smart wearable systems: Current status and future challenges,”Artificial intelligence in medicine, vol. 56, no. 3, pp. 137–156, 2012.

[68] J. Choi, B. Ahmed, and R. Gutierrez-Osuna, “Development andevaluation of an ambulatory stress monitor based on wearablesensors,” Information Technology in Biomedicine, IEEE Transactionson, vol. 16, no. 2, pp. 279–286, 2012.

JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2015 18

[69] C. Setz, B. Arnrich, J. Schumm, R. La Marca, G. Troster, andU. Ehlert, “Discriminating stress from cognitive load using awearable eda device,” Information Technology in Biomedicine, IEEETransactions on, vol. 14, no. 2, pp. 410–417, 2010.

[70] E. Ertin, N. Stohs, S. Kumar, A. Raij, M. al’Absi, and S. Shah,“Autosense: unobtrusively wearable sensor suite for inferringthe onset, causality, and consequences of stress in the field,” inProceedings of the 9th ACM Conference on Embedded Networked SensorSystems. ACM, 2011, pp. 274–287.

[71] P. Sanches, K. Höök, E. Vaara, C. Weymann, M. Bylund, P. Ferreira,N. Peira, and M. Sjölinder, “Mind the body!: designing a mobilestress management application encouraging personal reflection,”in Proceedings of the 8th ACM conference on designing interactivesystems. ACM, 2010, pp. 47–56.

[72] A. Kailas, C.-C. Chong, and F. Watanabe, “From mobile phonesto personal wellness dashboards,” Pulse, IEEE, vol. 1, no. 1, pp.57–63, 2010.

[73] M. F. Colunas, J. M. A. Fernandes, I. C. Oliveira, and J. P. S.Cunha, “Droid jacket: Using an android based smartphone forteam monitoring,” in Wireless Communications and Mobile Comput-ing Conference (IWCMC), 2011 7th International. IEEE, 2011, pp.2157–2161.

[74] A. Gaggioli, G. Pioggia, G. Tartarisco, G. Baldus, M. Ferro, P. Ci-presso, S. Serino, A. Popleteev, S. Gabrielli, R. Maimone et al., “Asystem for automatic detection of momentary stress in naturalisticsettings,” Stud Health Technol Inform, vol. 181, pp. 182–186, 2012.

[75] H. Lu, D. Frauendorfer, M. Rabbi, M. S. Mast, G. T. Chittaranjan,A. T. Campbell, D. Gatica-Perez, and T. Choudhury, “Stresssense:Detecting stress in unconstrained acoustic environments usingsmartphones,” in Proceedings of the 2012 ACM Conference on Ubiq-uitous Computing. ACM, 2012, pp. 351–360.

[76] G. Bauer and P. Lukowicz, “Can smartphones detect stress-relatedchanges in the behaviour of individuals?” in Pervasive Computingand Communications Workshops (PERCOM Workshops), 2012 IEEEInternational Conference on. IEEE, 2012, pp. 423–426.

[77] D. F. Dinges, R. L. Rider, J. Dorrian, E. L. McGlinchey, N. L. Rogers,Z. Cizman, S. K. Goldenstein, C. Vogler, S. Venkataraman, andD. N. Metaxas, “Optical computer recognition of facial expres-sions associated with stress induced by performance demands,”Aviation, space, and environmental medicine, vol. 76, no. Supplement1, pp. B172–B182, 2005.

[78] H. Gao, A. Yuce, and J.-P. Thiran, “Detecting emotional stress fromfacial expressions for driving safety,” in Image Processing (ICIP),2014 IEEE International Conference on. IEEE, 2014, pp. 5961–5965.

[79] W. Liao, W. Zhang, Z. Zhu, and Q. Ji, “A real-time human stressmonitoring system using dynamic bayesian network,” in ComputerVision and Pattern Recognition-Workshops, 2005. CVPR Workshops.IEEE Computer Society Conference on. IEEE, 2005, pp. 70–70.

[80] A. Derakhshan, M. Mikaeili, M. A. Khalilzadeh, and A. Moham-madian, “Preliminary study on facial thermal imaging for stressrecognition.” in Intelligent Environments (Workshops), 2014, pp. 66–73.

[81] D. Giakoumis, A. Drosou, P. Cipresso, D. Tzovaras, G. Hassapis,A. Gaggioli, and G. Riva, “Using activity-related behaviouralfeatures towards more effective automatic stress detection,” PloSone, vol. 7, no. 9, p. e43571, 2012.

[82] N. Sharma, A. Dhall, T. Gedeon, and R. Goecke, “Thermal spatio-temporal data for stress recognition,” EURASIP Journal on Imageand Video Processing, vol. 2014, no. 1, pp. 1–12, 2014.

[83] L. M. Vizer, L. Zhou, and A. Sears, “Automated stress detectionusing keystroke and linguistic features: An exploratory study,”International Journal of Human-Computer Studies, vol. 67, no. 10, pp.870–886, 2009.

[84] G. Zhou, J. H. Hansen, and J. F. Kaiser, “Nonlinear feature basedclassification of speech under stress,” Speech and Audio Processing,IEEE Transactions on, vol. 9, no. 3, pp. 201–216, 2001.

[85] M. Lai, Y. Chen, M. Chu, Y. Zhao, and F. Hu, “A hierarchicalapproach to automatic stress detection in english sentences,” inAcoustics, Speech and Signal Processing, 2006. ICASSP 2006 Proceed-ings. 2006 IEEE International Conference on, vol. 1. IEEE, 2006, pp.I–I.

[86] K. Imoto, Y. Tsubota, T. Kawahara, and M. Dantsuji, “Modelingand automatic detection of english sentence stress for computer-assisted english prosody learning system,” Acoustical science andtechnology, vol. 24, no. 3, pp. 159–160, 2003.

[87] H. Xie, M. Zhang, and P. Andreae, “Genetic programming forautomatic stress detection in spoken english,” in Applications ofEvolutionary Computing. Springer, 2006, pp. 460–471.

[88] J. Zhao, H. Yuan, J. Liu, and S. Xia, “Automatic lexical stressdetection using acoustic features for computer assisted languagelearning,” Proc. APSIPA ASC, pp. 247–251, 2011.

[89] J. Hernandez, P. Paredes, A. Roseway, and M. Czerwinski, “Underpressure: sensing stress of computer users,” in Proceedings of theSIGCHI conference on Human factors in computing systems. ACM,2014, pp. 51–60.

[90] D. Sun, P. Paredes, and J. Canny, “Moustress: detecting stress frommouse motion,” in Proceedings of the SIGCHI conference on Humanfactors in computing systems. ACM, 2014, pp. 61–70.

[91] Y. M. Lim, A. Ayesh, and M. Stacey, “Detecting cognitive stressfrom keyboard and mouse dynamics during mental arithmetic,”in Science and Information Conference (SAI), 2014. IEEE, 2014, pp.146–152.

[92] D. Carneiro, P. Novais, J. M. Pêgo, N. Sousa, and J. Neves, “Usingmouse dynamics to assess stress during online exams,” in HybridArtificial Intelligent Systems. Springer, 2015, pp. 345–356.

[93] M. Rodrigues, S. Gonçalves, D. Carneiro, P. Novais, and F. Fdez-Riverola, “Keystrokes and clicks: Measuring stress on e-learningstudents,” in Management Intelligent Systems. Springer, 2013, pp.119–126.

[94] J. Andrén and P. Funk, “A case-based approach using behaviouralbiometrics to determine a user’s stress level.” in ICCBR Workshops,2005, pp. 9–17.

[95] D. Carneiro, J. C. Castillo, P. Novais, A. Fernández-Caballero, andJ. Neves, “Multimodal behavioral analysis for non-invasive stressdetection,” Expert Systems with Applications, vol. 39, no. 18, pp.13 376–13 389, 2012.

[96] A. Pimenta, D. Carneiro, P. Novais, and J. Neves, “Monitoringmental fatigue through the analysis of keyboard and mouse inter-action patterns,” in Hybrid Artificial Intelligent Systems. Springer,2013, pp. 222–231.

[97] M. B. Hargrove, D. L. Nelson, and C. L. Cooper, “Generating eu-stress by challenging employees,” Organizational Dynamics, vol. 42,no. 1, pp. 61–69, 2013.

Davide Carneiro is an Invited Professor at theDepartment of Informatics of the University ofMinho and at the Polytechnic Institute of Porto.He is also a researcher at the Algoritmi center,in the Department of Informatics, University ofMinho, Braga, Portugal. He holds a PhD withEuropean Honors attributed under a joint Doc-toral Programme in Computer Science that ag-gregates three top Portuguese Universities. Heauthored one book, edited another, and is theauthor of papers in 12 international journals. He

is also the author of more than 50 book chapters and papers in confer-ence proceedings. He is interested in developing Human-aware Intelli-gent Systems, focusing especially on Human-Computer Interaction.

Paulo Novais is an Associate Professor withHabilitation of Computer Science at the Depart-ment of Informatics, in the School of Engineeringof the University of Minho (Portugal) and a re-searcher at the ALGORITMI Centre in which heis the coordinator of the research group Intelli-gent Systems Lab, and the coordinator of the re-search line in “Ambient intelligence for well-beingand Health Applications”. His main research aimis to make systems a little more smart, intelligentand also reliable. He is the co-author of over 230

book chapters, journal papers, conference and workshop papers andbooks. He is the president of APPIA (the Portuguese Association for Arti-ficial Intelligence) for 2016/2017 and member of the executive committeeof the IBERAMIA (IberoAmerican Society of Artificial Intelligence).

JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2015 19

Juan Carlos Augusto (Licenciate in ComputerScience – 1992, Ph.D. in Artificial Intelligence- 1998, M.Sc. in Higher Education – 2009) isProfessor of Computer Science at MiddlesexUniversity, London, UK. He is the Head of theResearch Group on Development of IntelligentEnvironments. His personal research is focusedon the improvement of the intelligence, inter-faces and development process of Intelligent En-vironments. He has contributed to the researchcommunity with more than 220 publications, is

co-editor in chief of JAISE and JoRIE journals. He has provided invitedkeynotes for several scientific events and is currently the General Chairof IE’16. He has been a reviewer for the EU and several NationalFunding organizations in other countries. He is also an IEEE, ACM,AAAI and BCS member

Nicola Payne is a Health Psychologist and As-sociate Professor in Psychology at MiddlesexUniversity, London, where she has run the MScHealth Psychology at Middlesex for more thanten years. Her primary research interests arewithin the field of occupational health psychol-ogy. These include work–life balance, occupa-tional stress, the impact of work on health be-haviours and behaviour change interventions,and issues related to combining work and fertilitytreatment.