Which of the following is the best example of use of human factors engineering?

Design for cognitive support

L.C. Schubel, ... K.M. Miller, in Design for Health, 2020

Design guidelines

The following HFE strategies can help clinicians combat human error attributed to perception and attention:

Learning to recognize cognitive errors and biases increases one’s ability to create active change, enabling clinicians to intentionally change their emotions and behaviors.

Providing contrast is a crucial component of clear and correct perception. Poor contrast, such as light lettering on a light background or dark lettering on a dark background, can create confusion and lead to errors. Using high contrast for displays, labeling, and handwriting can reveal significant improvement.

Ensuring discriminability, how well items can be differentiated, is vitally important. The existence of confusingly similar drug names is one of the most common causes of medication error and is of concern worldwide (Davis, 1999; Lambert, 1997). Whenever possible, facilities should eliminate “look-alike” and “sound-alike” drugs or develop systems processes to combat confusion. Process solutions include utilizing computerized physician order entry when available, or developing protocols that require clinicians to check the purpose of the medication on the prescription, and an active diagnosis that matches that purpose, prior to administering the medication.

HFE strategies can also address risk of human error due to working memory limitations:

Strategies to support working memory include chunking and allowing for frequent offloading of working memory after rehearsal. Chunking is a strategy involving splitting concepts into small pieces or “chunks” of information for retention (e.g., it is easier to remember 123, 456 rather than 1, 2, 3, 4, 5, and 6) (Miller, 1956). A second technique is supporting memory or perceptual judgments through technology recognition rather than recall (Horsky, Kaufman, Oppenheim, & Patel, 2003), for example, an EHR displaying test results alongside the screen for sending a message to the patient about those test results (Nielsen, 1994).

General techniques to maximize vigilance detection include shorter vigilance periods with frequent rest breaks, strategic use of caffeine, and enhanced signal visibility (Alves & Kelsey, 2010; Ariga & Lleras, 2011; Young, Robinson, & Alberts, 2009). Well-designed alarms that differentiate between a routine alert and a potential emergency also decrease the need for continuous vigilance (e.g., a disconnected ventilator needing an immediate response vs an alarm notifying a nurse that an intravenous infusion needs to be adjusted) (Barnsteiner, 2011; Weinger et al., 1993).

Strategies for interruptions in the clinical setting include establishing a no-interruption or “quiet” zone (OR Manager, 2013), creating checklists to working memory support (Reason, 1990), and utilizing awareness displays (Dabbish & Kraut, 2004).

Clinical decision-making benefits from the following HFE strategies:

Decision support techniques and technologies should account for a diagnosis’ occurrence, likelihood, and case-specific factors with the intention of guarding against the effects of anchoring, priming, recency bias, or availability heuristics. If a provider is inclined to diagnose a second case of adolescent congestive heart failure that week, technologies should evaluate the probabilistic likelihood of the occurrence against the potential of distorted influence from the availability heuristic (Kahneman & Tversky, 1973).

Cognitive support technologies and clinician education should account for and train medical personnel to police for unconscious stereotyping by race, ethnicity, or gender. Literature has documented variable prescribing practices as they relate to patients’ demographics (Groenewald, Rabbitts, Hansen, & Palermo, 2018; Smith, Dolk, Smieszek, Robotham, & Pouwels, 2018). Physicians and their support should thus counterintuitively seek reasons to challenge initial decisions to ensure optimal patient care.

If a practice offers CDS systems, clinicians should pay special attention to alerts and take advantage of existing datasets and protocols. Well-designed CDS tools are intended to facilitate the care process from diagnosis to treatment protocols and can even alert clinicians when incorrect and impractical information is entered into a patient’s chart (such as a 2 m tall toddler).

Read full chapter

URL: https://www.sciencedirect.com/science/article/pii/B9780128164273000129

Work Physiology Model

Donald W. Boyd, in Systems Analysis and Modeling, 2001

9.1 INTRODUCTION

Ergonomics and human factors engineering have contributed to remarkable increases in productivity that improve the health, safety, and well-being of workers and end-users. Driven by management's concern for improvement in terms of work productivity, absenteeism, workman's compensation costs, and worker morale, human resources will continue to be a critical component of industrial economics. Improvement involves eliminating hazards, reducing risk of injury, and making tasks more comfortable. Better human resource management requires tools to assist managers and engineers in making better matches between workers and tasks. This chapter draws from a master's thesis by Robert R. Wehrman [55] in which he applied systems engineering to human physiology to synthesize a work physiology model that describes a worker's physiological response to a given task. Dr. Robert J. Marley, Wehrman's major professor, served as domain expert.

Physiological models are primarily applied to repetitive lifting where it is presumed that the lifting load is well within the physical strength of the worker. The respiratory (oxygen transport) system is the key limiting factor impacting these lifting tasks. Knowledge of physiology, understanding the human body, is critical to modern work design. At best, it has been difficult to quantify physiological demands on the human body. Although it is possible to measure some physical parameters, such as oxygen uptake of a worker while performing a task, costs restrict such practice to research laboratories. Given a known workload and physiological capacity of a worker, an effective method for worker screening is sought by researchers. Most commonly, the approach has been to relate motion analysis to oxygen uptake by the worker.

Many researchers have modeled oxygen uptake as a function of energy expenditure, but the majority of these models have been static regression models developed from empirical observations. Hagan et al. [22] conducted a comparative study of 12 regression models for steady-state oxygen uptake and energy expenditure during horizontal treadmill tests and found their predictions to be similar. Disadvantageously, these models were applicable only to specific segments of the population and for horizontal running.

Garg et al. [20] have derived a series of static regression models that expand applicability to numerous, common, manual materials handling jobs. Although robust in applying to jobs without previous experimental measure, their industrial application was limited. Furthermore, these models were static, applying only to steady-state workloads.

Not all industrial work achieves steady state. The body's response to workload increase is transient. Stored energy supplies are consumed to meet short-term demand, whereas oxygen uptake increases to a level required to meet the new demand. A similar transient response occurs when the workload is decreased. Researchers have defined models, such as that by Morton [35], that account for rate of change of oxygen uptake with respect to time. However, these models are still static, applying only to engaging or disengaging constant workloads.

Ergonomists are continually looking for better ways to determine the best combination of job parameters for reducing physical stress. Specific individuals or population segments are tested against computer simulated tasks to quantify the risk of injury. These data from task simulation coupled with systems analysis provide adequate knowledge-based support for modeling the human at work. A work physiology model provides a mechanism for quantifying actual workload during simulation of a specific task by a worker. That is, given the workload in energy units, the model predicts oxygen uptake; or, given oxygen uptake, the model derives the energy-equivalent workload. For cost effectiveness and accuracy, a macro systems model can be used in a variety of applications.

Wellman, a mechanical engineer, explored the use of systems analysis for modeling physiological response to changing workloads for tasks involving large, rhythmic muscle movements for both transient and steady-state responses. He observed that mathematical formulation and iterative solution of Mtm models are in some ways analogous to finite element analysis (FEA), commonly employed by mechanical engineers. In FEA, each finite element is similar to a subsystem. FEA is typically applied to mechanical systems having geometric properties, heat distributions, stress distributions, and so on, that are too complicated to describe and solve by traditional differential equation techniques. Finite elements are established as linear forms for parameters such as heat and stress. Thus, the system is described by a balanced set of equations and unknowns. However, Mtm and FEA differ in application. In FEA, algorithms automatically generate finite elements to fully circumscribe the system. Computational resources are the limiting factor in the number of elements that can be handled. In Mtm, no generic algorithm can automatically divide a system into subsystems. The modeler must explicitly relate each variable component of the system to a specific subsystem, relying on knowledge of the system. The distinguishing feature of an FEA model is its many generic finite elements as opposed to a few unique and specific subsystems in an Mtm model.

Read full chapter

URL: https://www.sciencedirect.com/science/article/pii/B9780121218515500095

Background: Introduction

Rex Hartson, Pardha Pyla, in The UX Book (Second Edition), 2019

6.3.1 Engineering Paradigm

With its roots in software and human factors engineering, the engineering paradigm in HCI prescribed starting with an inventory of the functionality envisioned for a new system and proceeding to build those items with the best quality possible given available resources. The engineering focus is on functionality, reliability, user performance, and avoiding errors. Recognizing that user interaction deserved attention on its own, usability engineering emerged as a practical approach to usability with a focus on improving user performance, mainly through evaluation and iteration.

The engineering paradigm also had strong roots in human factors, where "work" was studied, deconstructed, and modeled. An example is the study of an assembly line where each action required to do work was carefully described. It was a purely utilitarian and requirements-driven approach. Alternative methods and designs were compared and success was measured by how much the user could accomplish.

Read full chapter

URL: https://www.sciencedirect.com/science/article/pii/B9780128053423000060

Introduction to HCI research

Jonathan Lazar, ... Harry Hochheiser, in Research Methods in Human Computer Interaction (Second Edition), 2017

1.2 Types of HCI Research Contributions

The field of HCI draws on many different disciplines, including computer science, sociology, psychology, communication, human factors engineering, industrial engineering, rehabilitation engineering, and many others. The research methods may have originated in these other disciplines. However, they are modified for use in HCI. For instance, techniques such as experimental design and observation from psychology, have been modified for use in HCI research. Because HCI draws on the work in so many different disciplines, people often ask “what is considered HCI research? What types of effort are considered research contributions?” In a recent article that we believe will become a classic read, Wobbrock and Kientz (2016) discuss seven types of research contributions:

Empirical contributions—data (qualitative or quantitative) collected through any of the methods described in this book: experimental design, surveys, focus groups, time diaries, sensors and other automated means, ethnography, and other methods.

Artifact contributions—the design and development of new artifacts, including interfaces, toolkits, and architectures, mock-ups, and “envisionments.” These artifacts, are often accompanied by empirical data about feedback or usage. This type of contribution is often known as HCI systems research, HCI interaction techniques, or HCI design prototypes.

Methodological contributions—new approaches that influence processes in research or practice, such as a new method, new application of a method, modification of a method, or a new metric or instrument for measurement.

Theoretical contributions—concepts and models which are vehicles for thought, which may be predictive or descriptive, such as a framework, a design space, or a conceptual model.

Dataset contributions—a contribution which provides a corpus for the benefit of the research community, including a repository, benchmark tasks, and actual data.

Survey contributions—a review and synthesis of work done in a specific area, to help identify trends and specific topics that need more work. This type of contribution can only occur after research in a certain area has existed for a few years so that there is sufficient work to analyze.

Opinion contributions—writings which seek to persuade the readers to change their minds, often utilizing portions of the other contributions listed above, not simply to inform, but to persuade.

The majority of HCI research falls into either empirical research or artifact contributions, and this book specifically addresses empirical research using all of the potential data collection methods utilized in empirical research. In their analysis of research papers submitted to the CHI 2016 conference, Wobbrock and Kientz found that paper authors indicated in the submission form that over 70% of the papers submitted were either empirical studies of system use or empirical studies of people, and 28.4% were artifact/system papers (it is important to note that authors could select more than one category, so percentages can add up to more than 100%). There were a fair number of papers submitted on methodological contributions, but submissions in all of the other categories of contributions were rare (Wobbrock and Kientz, 2016). This provides some empirical data for what we (as book authors) have observed, that most HCI research is either empirical or systems research (or sometimes, a combination of both, such as when you develop a prototype and have users evaluate it).

Read full chapter

URL: https://www.sciencedirect.com/science/article/pii/B9780128053904000017

Introduction: Toward a Multidisciplinary Science of Human-Computer Interaction

John M. Carroll, in HCI Models, Theories, and Frameworks, 2003

1.1 THE GOLDEN AGE

With respect to traditional concerns and subdisciplines of computer science, HCI was originally a joining of software engineering and human-factors engineering. It integrated concerns about tools and methods for software development with concerns about verifying the usability of the software produced. This integration offered solutions to critical problems in both software engineering and human factors.

In the 1970s, software engineering faced a crisis over the so-called waterfall development method, a linear organization of software development activities, each producing interim results—for example, the functional specification document—that are handed off to subsequent stages. The waterfall development method was slow and unreliable; important requirements often emerged only after initial implementation, wasting effort and forcing costly reworking of software. Software human factors also faced a crisis; it was positioned at the end of the waterfall, and thus it became involved only after fundamental design decisions had been taken. It was positioned too far downstream to make more than cosmetic difference in software products. These crises coincided with the dawn of the personal computer, creating a whole new set of challenges for software development, many pertaining to user interfaces and end-user applications. This amplified the sense that computing was in crisis.

Towards the end of the 1970s, cognitive science had coalesced as a multidisciplinary project encompassing linguistics, anthropology, philosophy, psychology, and computer science. One principle of cognitive science was the representational theory of mind, the thesis that human behavior and experience can be explained by explicit mental structures and operations. A second principle was that an effective multidisciplinary science should be capable of supporting and benefitting from application to real problems. Many domains were investigated, including mechanics, radiology, and algebra. HCI became one the first cognitive-science domains.

The initial vision of HCI as an applied science was to bring cognitive-science methods and theories to bear on software development. Most ambitiously, it was hoped that cognitive-science theory could provide substantive guidance at very early stages of the software-development process. This guidance would come from general principles of perception and motor activity, problem solving and language, communication and group behavior, and so on. It would also include developing a domain theory, or theories, of HCI. This first decade of HCI was a golden age of science in the sense that there was wide tacit agreement as to the overarching research paradigm. And a lot got done.

For example, Card, Moran and Newell (1983) developed the Goals, Operators, Methods and Selection rules (GOMS) model for analyzing routine human-computer interactions. This was an advance on prior human-factors modeling, which did not address the cognitive structures underlying manifest behavior. It was a fundamental advance on the cognitive psychology of the time: It explicitly integrated many components of skilled performance to produce predictions about real tasks. There was a relatively broad range of such work. Malone (1981) developed analyses of fun and of the role of intrinsic motivation in learning based on studies of computer-game software. Carroll (1985) developed a psycholinguistic theory of names based on studies of filenames and computer commands. This body of work was a beginning of a science of HCI, but it also contributed to the cognitive-science foundation upon which it drew fundamental concepts. Indeed, much of it was published in cognitive-science journals.

In the mid‐1980s, HCI saw itself as an emerging scientific discipline. A vivid indication is Allen Newell's 1985 opening plenary address at the ACM CHI Conference, the major technical conference in HCI (Newell & Card, 1985). In the talk, Newell presented a technical vision of a psychology of HCI. It is striking that the conference organizers had recruited a plenary address describing a program for scientific research. And the effects of the talk were also striking. Newell's talk provoked controversy and new research. It led to alternate proposals, modified proposals, replies, and rejoinders (Carroll & Campbell, 1986; Newell & Card, 1986). It helped to heighten interest in theory and science for at least five years.

Read full chapter

URL: https://www.sciencedirect.com/science/article/pii/B9781558608085500010

What sample sizes do we need? Part 2: formative studies

Jeff Sauro, James R. Lewis, in Quantifying the User Experience (Second Edition), 2016

Some history—the 1980s

Although strongly associated with Jakob Nielsen (e.g., Nielsen, 2000), the idea of running formative user studies with small-sample iterations goes back much further—to one of the fathers of modern human factors engineering, Alphonse Chapanis. In an award-winning paper for the IEEE Transactions on Professional Communication about developing tutorials for first-time computer users, Al-Awar et al. (1981, p. 34) wrote:

Having collected data from a few test subjects – and initially a few are all you need – you are ready for a revision of the text. Revisions may involve nothing more than changing a word or a punctuation mark. On the other hand, they may require the insertion of new examples and the rewriting, or reformatting, of an entire frame. This cycle of test, evaluate, rewrite is repeated as often as is necessary.

Any iterative method must include a stopping rule to prevent infinite iterations. In the real world, resource constraints and deadlines often dictate the stopping rule. In the study by Al-Awar et al. (1981), their stopping rule was an iteration in which 95% of participants completed the tutorial without any serious problems.

Al-Awar et al. (1981) did not specify their sample sizes, but did refer to collecting data from “a few test subjects.” The usual definition of “few” is a number that is greater than one, but indefinitely small. When there are two objects of interest, the typical expression is “a couple.” When there are six, it’s common to refer to “a half dozen.” From this, it’s reasonable to infer that the per-iteration sample sizes of Al-Awar et al. (1981) were in the range of three to five—at least, not dramatically larger than that.

The publication and promotion of this method by Chapanis and his students had an almost immediate influence on product development practices at IBM (Kennedy, 1982; Lewis, 1982) and other companies, notably Xerox (Smith et al., 1982) and Apple (Williams, 1983). Shortly thereafter, John Gould and his associates at the IBM T. J. Watson Research Center began publishing influential papers on usability testing and iterative design (Gould, 1988; Gould and Boies, 1983; Gould et al., 1987; Gould and Lewis, 1984), as did Whiteside et al. (1988) at DEC (Baecker, 2008; Dumas, 2007; Lewis, 2012).

Read full chapter

URL: https://www.sciencedirect.com/science/article/pii/B9780128023082000072

What Sample Sizes Do We Need?

Jeff Sauro, James R. Lewis, in Quantifying the User Experience, 2012

Some History: The 1980s

Although strongly associated with Jakob Nielsen (see, for example, Nielsen, 2000), the idea of running formative user studies with small sample iterations goes back much further—to one of the fathers of modern human factors engineering, Alphonse Chapanis. In an award-winning paper for the IEEE Transactions on Professional Communication about developing tutorials for first-time computer users, Al-Awar et al. (1981, p. 34) wrote:

Having collected data from a few test subjects—and initially a few are all you need—you are ready for a revision of the text. Revisions may involve nothing more than changing a word or a punctuation mark. On the other hand, they may require the insertion of new examples and the rewriting, or reformatting, of an entire frame. This cycle of test, evaluate, rewrite is repeated as often as is necessary.

Any iterative method must include a stopping rule to prevent infinite iterations. In the real world, resource constraints and deadlines often dictate the stopping rule. In the study by Al-Awar et al. (1981), their stopping rule was an iteration in which 95% of participants completed the tutorial without any serious problems.

Al-Awar et al. (1981) did not specify their sample sizes, but did refer to collecting data from “a few test subjects.” The usual definition of “few” is a number that is greater than one, but indefinitely small. When there are two objects of interest, the typical expression is “a couple.” When there are six, it's common to refer to “a half dozen.” From this, it's reasonable to infer that the per-iteration sample sizes of Al-Awar et al. (1981) were in the range of three to five—at least, not dramatically larger than that.

The publication and promotion of this method by Chapanis and his students had an almost immediate influence on product development practices at IBM (Kennedy, 1982; Lewis, 1982) and other companies, notably Xerox (Smith et al., 1982) and Apple (Williams, 1983). Shortly thereafter, John Gould and his associates at the IBM T. J. Watson Research Center began publishing influential papers on usability testing and iterative design (Gould, 1988; Gould and Boies, 1983; Gould et al., 1987; Gould and Lewis, 1984), as did Whiteside et al. (1988) at DEC (Baecker, 2008; Dumas, 2007; Lewis, 2012).

Read full chapter

URL: https://www.sciencedirect.com/science/article/pii/B9780123849687000072

Machine Design

Gerard Voland, ... Uffe Hindhede, in Encyclopedia of Physical Science and Technology (Third Edition), 2003

III.I Design for Human Factors

Consideration of human factors is especially important in the design of frames and housings. In the automotive industry, for example, frames must be designed to accommodate people as operators, passengers, and maintenance personnel.

The goal of human factors engineering (or “ergonomics”) is to insure that people can interact with machines safely, comfortably, and efficiently. This means that designers must place controls where they will be accessible to the operator and comfortable to use. Hand tools should be light, free from excessive vibration, and safe to use. Concern for ergonomics is clearly evident in the design of space vehicles, which are literally built around the astronauts.

Read full chapter

URL: https://www.sciencedirect.com/science/article/pii/B0122274105003914

Introduction

Rex Hartson, Partha S. Pyla, in The UX Book, 2012

1.6.3 Psychology and Cognitive Science

In addition to the major influence of human factors and engineering, HCI experienced a second wave of formative influence (Tatar, Harrison, & Sengers, 2007) from a special brand of cognitive science, beginning with Card, Moran, and Newell (1983), offering the first theory within HCI.

Like human factors engineering, cognitive psychology has many connections to the design for, and evaluation of, human performance, including cognition, memory, perception, attention, sense and decision making, and human behavioral characteristics and limitations, elements that clearly have a lot to do with user experience. One difference is that psychology is more about the human per se, whereas human factors engineering looks at the human as a component in a larger system for which performance is to be optimized. However, because of the influence of psychology on human factors and the fact that most human factors practitioners then were trained in psychology, the field was known at least for a while as occupational psychology.

Because the field of human factors is based on a foundation in psychology, so are HCI and user experience. Perhaps the most fundamental contribution of psychology to human–computer interaction is the standard bearer, Card, Moran, and Newell (1983), which is still today an important foundational reference.

The empiricism involved in statistical testing in human factors and HCI has especially apparent common roots in psychology; see, for example, Reisner (1977). Hammond, Gardiner, and Christie (1987) describe the role of cognitive psychology in HCI to include observing human behavior, building models of human information processing, inferring understanding of the same, and scientific, or empirical, study of human acquisition, storage, and use of knowledge/information. Cognitive psychology shares with human factors engineering the goal of system operability and, when connected to HCI, computer-based system operability.

Perhaps the most important application of psychology to HCI has been in the area of modeling users as human information processors (Moran, 1981b; Williges, 1982). Most human performance prediction models stem from Card, Moran, and Newell's Model Human Processor (1983), including the keystroke level model (Card, Moran, & Newell, 1980), the command language grammar (Moran, 1981a), the Goals, Operators, Methods, and Selections (GOMS) family of models (Card, Moran, & Newell, 1983), cognitive complexity theory of Kieras and Polson (1985), and programmable user models (Young, Green, & Simon, 1989). In the earliest books, before “usability” was a common term, “software psychology” was used to connect human factors and computers (Shneiderman, 1980).

Carroll (1990) contributed significantly to the application of psychology to HCI in fruitful ways. Carroll says, “… applied psychology in HCI has characteristically been defined in terms of the methods and concepts basic psychology can provide. This has not worked well.” He goes on to explain that too much of the focus was on psychology and not enough on what it was being applied to. He provides a framework for understanding the application of psychology in the HCI domain.

As an interesting aside to the role of cognitive psychology in HCI, Digital Equipment Corporation researchers (Whiteside et al., 1985; Whiteside & Wixon, 1985) made the case for developmental psychology as a more appropriate model for interaction design than behavioral psychology and as a framework for studying human–computer interaction. The behavioral model, which stresses behavior modification by learning from stimulus–response feedback, leads to a view in which the user adapts to the user interface. Training is invoked as intervention to shape the user's behavior. The user with “wrong” behavior is importuned with error messages. Simply put, user behavior is driven by the interaction design.

In contrast, developmental psychology stresses that surprisingly complex user behavior springs from the person, not the design. The developmental view studies “wrong” user behavior with an eye to adapting the design to prevent errors. Differences between system operation and user expectations are opportunities to improve the system. “User behavior is not wrong; rather it is a source of information about the system's deficiencies (Whiteside & Wixon, 1985, p. 38).”

Finally, as even more of an aside, Killam (1991) proffers the idea that humanistic psychology, especially the work of Carl Rogers, Rogerian psychology as it is called, is an area of psychology that has been applied unknowingly, if not directly, to HCI. A client-centered approach to therapy, Rogerian psychology, as in the developmental approach, avoided the normative, directive style of prescribing “fixes” for the patient to adopt, instead listening to the patient's needs that must be met to affect healing.

The tenets of Rogerian psychology translate to some of our most well-known guidelines for interaction design, including positive feedback to encourage, especially at times when the user might be hesitant or unsure, and keeping the locus of control with the user, for example, not having the system try to second-guess the user's intentions. In sum, the Rogerian approach leads to an interaction design that provides an environment for users to find their own way through the interaction rather than having to remember the “right way.”

As in the case of human factors engineering, many people moved into HCI from psychology, especially cognitive psychology, as a natural extension of their own field.

Read full chapter

URL: https://www.sciencedirect.com/science/article/pii/B9780123852410000014

Background: UX Evaluation

Rex Hartson, Pardha Pyla, in The UX Book (Second Edition), 2019

28.2 The Dangers of Trying to (or Even Appearing to) do FORMAL Summative Evaluation in UX Practice

28.2.1 Engineering Versus Science

It's all very well in practice but it will never work in theory.

French management saying

Sometimes, empirical lab-based UX testing that includes informal quantitative metrics is the source of controversy with respect to “validity.” Sometimes we hear, “Because your informal summative evaluation was not controlled testing, why should we not dismiss your results as too ‘soft’?” “Your informal studies aren’t good science. You can’t draw any conclusions.”

These questions ignore the fundamental difference between formal and informal summative evaluation and the fact that they have completely different goals and methods. This may be due, in part, to the fact that the fields of HCI and UX were formed as a melting pot of people from widely varying backgrounds. From their own far-flung cultures in psychology, human factors engineering, systems engineering, software engineering, marketing, and management, they arrived at the docks of HCI with their baggage containing their own perspectives and mindsets.

Formal Summative Evaluation

A formal, statistically rigorous summative (quantitative) empirical UX evaluation that produces statistically significant results (Section 21.1.5.1).

Thus, it is known that formal summative evaluations are judged on a number of rigorous criteria, such as validity. But informal summative evaluation may be less known as an important engineering tool in the HCI bag, and that the only criterion for judging this kind of summative evaluation method is effectiveness within an engineering process.

28.2.2 What Happens in Engineering Stays in Engineering

Because informal summative evaluation is engineering, it comes with some very strict limitations, particularly on sharing informal summative results.

Informal summative evaluation results are only for internal use as engineering tools to do an engineering job by the project team and shouldn’t be shared outside the team. Because of the lack of statistical rigor, these results especially can’t be used to make any claims inside or outside the team. To make claims about UX levels achieved from informal summative results, for example, would be a violation of professional ethics.

We read of a case in which a high-level manager in a company got a UX report from a project team, but discounted the results because they were not statistically significant. This problem could have been avoided by following our simple rules and not distributing formative evaluation reports outside the team or by writing the report for a management audience with careful caveats.

Read full chapter

URL: https://www.sciencedirect.com/science/article/pii/B978012805342300028X