03 January 2006

Trip Report

Department of Defense

Human Factors Engineering Technical Advisory Group

(DOD HFE TAG) Meeting #54

07-10 November 2005

 

The 54th meeting of the DoD HFE TAG was held in Baltimore, MD. The meeting was chaired by Ms. Maureen Bergondy-Wilhelm, Naval Air Systems Command, Orlando, FL. The theme of the meeting was Human Factors Engineering in Network Centric Warfare. Approximately 125 people attended the meeting, representing the US Army, US Navy, US Air Force, DARPA, US Coast Guard, NASA, FAA, several human factors-related technical societies and industry associations. Personnel representing industry and academia attended as invited speakers.

 

Four items are attached:

·      Background of the DoD HFE TAG, attachment (1)

·      Meeting schedule, attachment (2)

·      DoD HFE TAG Operating Board, attachment (3)

·      TAG attendees, attachment (4)

·      DoD HFE TAG Policies, attachment (5)

 

Plenary Session Presentations

 

The DoD HFE TAG Chair for the 54th meeting, Maureen Bergondy-Wilhelm, welcomed attendees summarized results of the morning’s Executive Committee meeting:

 

Army Host Welcome: Dr. Robin Keesee (Deputy to the Commander US Army Research, Development and Engineering Command) welcomed TAG attendees to the 54th semi-annual meeting. He noted that today’s Soldiers have 100+ Lbs of load to carry, tanks are 70+ tons each, and that battle command tasks must be conducted while stationary. The future force will reduce the Soldier’s load to 40 Lbs, reduce tank weight to 20 tons (much more deployable) and conduct Battle Command tasks on-the-move. Future Combat Systems program Network Centric operations concepts will not only apply to future systems, they will be “spun out” to current systems also.

 

Research Development & Engineering Command (RDECOM) is the technology developer for the Army. It is staffed with 12,500 civilian workers. Some of the more important technology development programs at RDECOM include long-life batteries, C4ISR, inexpensive and reliable position tracking and navigation to enable Situational Awareness (SA) and control of robotic vehicles. Human Factors tools must link to performance, cost and schedule to be relevant. The rigor of human factors analysis techniques must be raised. Results must be relevant to current and future programs.

 

Human Factors Challenges of Technology-Driven Network-Centric Warfare: Dr. Alfred Kaufman, Institute for Defense Analysis, spoke on Network Centric Operations (NCO) doctrine. Many references were made to Carl von Clausewitz book, On War. NCO doctrine does not really accommodate human capabilities and constraints. Man cannot be an afterthought to NCO technology; NCO needs to be human centered warfare, instead. Many people refer to NCO as a revolution; however, it is recommended that we look to the past first regarding information in war. Most information during wartime is false, contradictory or vague => unreliable. NCO doctrine tends to ignore the human in the process. For example, getting familiar with a battle space object is important but not included.

 

Data can be shared easily but information understanding cannot be shared easily. To be perceived as reliable, data/information must be interpreted the same by all members of the team. But, there is no room for human nature in NCO….. Seeing so much data on a screen may tend to cause operators to put their judgment to sleep. NCO doctrine minimizes the need for and the importance of leaders.

 

Perspectives on HSI from DoD Chief Technology Officer’s Office: Dr. Robert Foster, Director Bio-Systems, OUSD (S&T). Two to three years ago, Dr. Foster had asked the TAG to develop guidelines for how to insert the human into C4ISR. Some recent examples underscoring the impact of humans on C4ISR include:

 

HFE specialists need to read the following documents and put them into context:

 

Some trends where there is continuing investment:

 

Some areas where there is increasing investment:

 

Some emerging systems/concepts Demanding HSI:

 

Issues:

 

What can the DoD HFE TAG do?

 

Last note: Dr. Foster indicated that his office is responsible for conducting Technology Readiness Reviews. He indicated that if HSI is not up to par, mitigation plans will be required.

 

Understanding and Leading the Human Cognitive Network in Net-Centric Warfare. LTCOL John M. Graham (Deputy Director, Department of Behavioral Sciences and Leadership, US Military Academy) presented views of the human in net-centric warfare. He advocates more study of the human factors in network centric ops.

 

NCW as the technologist sees it:                    

 

NCW as the Behavioral scientist sees it:

 

Other observations include:

 

Empowering Warriors: Harness NCW to Maximize Human Capability. LTCOL Steven Waller (Office of the Secretary of the Air Force, Warfighting Integration, SAF/XC Program Manager DARPA IXO) spoke on the importance of connectivity to Warfighting ability. First, you need the “wires” to get and stay connected. This is imperative to stay fully engaged with what the warrior wants and needs. Second, you need knowledge (red and blue knowledge, dynamic combat plan, collaboration, deconfliction, etc. In NCW, the Warfighter needs simplification – fewer applications, ease of use, chat, more integrated GUI, cognitive assistance. NCW exists to optimize human performance but many of its features tend to work against this objective.

 

The New HSIIAC (Human Systems Integration Information Analysis Center). Dr. Kristen Liggett, Ph.D. (HSIIAC Technical Director, WPAFB) provided a short briefing on the new HSIIAC, which was re-established in September 2005. Dr. Liggett described the HSIIAC mission to support DoD and Service RDT&E, assist in the dissemination of HSI information across the defense community, and to promote and service a strong HSI community of practice across DoD, Federal agencies and industry. She also described the HSIIAC free market model, which is based on providing market advantage worth investing in. Northrop-Grumman and Alion have teamed to manage the IAC. Dr. Lawrence Wolpert is the IAC Director. Alion already manages four DTIC IACs, including Modeling and Simulation which is also a non-DTIC IAC. Many of the products of the old IAC will be maintained (e.g., Gateway, SOARs) and new products will be added.

 

Defense Safety Oversight Council (DSOC) Safety Improvement Task Force, Human Factors Working Group. LTCOL Donald J. White, USAF (Deputy Chief, Safety Issues Division & Chief, Human Factors and Operational Safety, HQ Air Force Safety) discussed some of the outgrowths from the 19 May 03 SECDEF Memo that called for a 50-% reduction in mishaps and accident rates within the next two years. IN November 2003 the Joint safety chiefs directed establishment of a Human Factors Working Group to work with Defense Safety Oversight Council Aviation Safety Improvement Task Force. The working group was tasked to create a joint services Human Factors analysis taxonomy to aid in their investigations (see Figure 1, below). There were four recommended study areas:

 

 

International Test and Evaluation Association (ITEA). Ms. Reta Reynolds, reta.reynolds@atc.army.mil (US Army, Aberdeen Test Center, Future Force, Transformation Division) spoke about ITEA; she is president of the ITEA Francis Scott Key Chapter. ITEA is a not-for-profit educational organization to further the exchange and advancement of technical information in the field of test and evaluation. ITEA has seven chapters in the US and internationally. There are regular, student and corporate memberships available. ITEA sponsors educational events, conferences, workshops and luncheons. It publishes a quarterly journal. Email: itea@itea.org Web: www.itea.org

 

 

Sub-Group Meetings Attended at the DOD HFE TAG:

       

Human Factors Standardization (HFS) Mr. Alan Poston, FAA, chaired the meeting (alan.poston@faa.gov). The new chair/chair select for the Human Factors Standardization SubTAG will be Mr. Dave Britton.

The HFS website is: http://dtica.dtic.mil/hftag/hfs.html

MIL-STD-1472F. The US Army Missile Command (MICOM) is not able to maintain the human factors (HFAC) standardization documents (no budget). A letter was sent to Dr. Foster on 20 September 2005 seeking $100k. If the TAG were to propose revisions to MIL-STD-1472, it is unknown what would happen. MIL-STD-1472 is becoming an “overage” document in that the last major technical update was made in 1989 and technology has continued to develop and evolve.

GEIA & HEB-1: An update was provided by Mr. Louis Adams, (Lockheed-Martin, lou.adams@lmco.com) who is the GEIA G-45 Human Factors Committee chair. Acquisition reform was fully supported by the G-45 committee of GEIA. Now, companies are authoring their own standards and requirements. HEB-1 has been modified to deal with accidents and aborts-saying that HFE must be involved in mishap investigations. A draft of the updated standard is now available. The annual GEIA G-45 Human Factors Committee meeting was held in conjunction with the Human Factors and Ergonomics society meeting in San Francisco. Teresa Alley (DTIC, MATRIS) will post the draft on the HFS Sub TAG website.

MIL-STD-1474: Dr. Joel T. Kalb, at the Army Research Laboratory (Human Research and Engineering Directorate) proposed an ear model for use in MIL-STD-1474 Revision E. The purpose of the update is to minimize noise-induced hearing damage and permit acceptable speech communications in noisy environments. Impulse noise-induced damage is the 5th most prevalent disability in Iraq. Current limits are based on peak pressure level B duration and the number of levels of hearing protection worn. Proposed changes are to base limits on recorded waveforms measured either in free-field or under the protection on a human or an acoustical mannequin. Allowed number of rounds would be determined by computation using a model of the ear and hearing protection worn. The current method is wrong…and too conservative, with hearing protection described by a single value for single and double protection. Advantages of the proposed update are that the model is based on anatomy of the human ear; the model accounts for peak-clipping and aural reflex as well as material fatigue in the inner ear. This new approach has been tested extensively on animal and human ears with all characteristic blast and other responses. The proposed update to MIL-STD-1474 will go to Tom Cook at MICOM in mid November 2005, and it will then ho to proponent agencies for review.

MIL-STD-1787D: Jim Kinzig (937-255-7138, james.kinzig@wpafb.af.mil) provided an update on MIL-STD-1787. This document standardizes the meanings and behaviors of symbols used in military aircraft. It provides criteria for adequate (safe) primary flight reference displays. The standard allows deviations if:

·      There is improved system performance

·      Life cycle cost is reduced

·      Development costs are reduced

·      Imposition of standard requirements would result in compromise

The standard now includes a rotary wing section, updated definitions, new missile launch zone and attack steering (includes principles and mathematical principles behind medium range missile symbology), unmanned air vehicle symbols (remotely operated). Some work remains to make the rotary wing section consistent with other sections of the document. Future revisions will address unmanned rotary wing and V-22 types of vehicles, off-boresight HMS/D symbology, more threat symbols, wide body/heavy symbols (to differentiate them from fighters), tapes versus counter-pointer displays, and flight director steering bar movement limits. The appendix will also be updated. The schedule for remaining activities is as follows:

·    November 05 – Clean up document and resolve text issues

·    December 05 – Submit to technical editing

·    February 06 – Submit to tri-services and industry for coordination

·    April 06 – Resolve comments

·    May 06 – Publish updated standard

           DIDs: Mr. Dick Armstrong noted that a draft HSI Plan DID has been developed.

           Joint Services Specification Guide (JSSG): Dave Britton (ASC/ENFC, david.britton@wpafb.af.mil) indicated that no progress had been made on updating this guide. All work lately has been on JSSG-516 air worthiness standards.

           FAA Human Factors Design Standard: Ms. Vicky Ahlstrom (FAATC, Atlantic City, Vicki.ahlstrom@faa.gov) indicated that HF-STD-001 is placing a high level of emphasis on human computer interface. They are currently determining who is using the document, purpose, feedback from users, sources of requirements, clarity of requirements, update for new technologies and improved access to information. Sections on keyboards and input devices were recently updated; rationale, references and sources have been added. They have also created an electronic, annotated bibliography for keyboards, input devices, touch screens, etc.

           NASA MSIS: Ms. Faith Chandler (NASA HQ, Faith.T.Chandler@nasa.gov) reported that since the last major space shuttle accident, there is now a human factors technical authority. It is expected that MSIS-3000 will be updated next in 2009.

           Electronic Industries Association HEB-1: GEIA has placed update of HEB-1 into the ballet. The first revision expands the scope from military only to military and commercial. Dr. Mark Brauer has authored an introduction. Votes are due to the committee by 27 May 2005. The update would then be distributed for publication (expected by July 2005). EIA 632 “Systems Engineering” is also in the revision cycle. Lou Adams plans to distribute it for comment shortly.

         Human Factors and Ergonomics Society (HFES)/ISO/TC159: Updates were provided in a handout for activities in the four TC159 subcommittees (SC-1, SC-3, SC-4 and SC-5).

         Index of Government and Non-Government Standards: Mr. Poston indicated these documents have been placed into DTIC. Their accession numbers are as follows:

·  Government Standards: ADA436638

·  Non-Government Standards: ADA436635

         Human Factors Design Digest: No input.      

 

 

Human Factors in Training. Not attended.

 

System Safety/Health Hazards/Survivability. Co-chairs for the meeting were George Murnyak (USACHPPM, george.murnyak@amedd.army.mil) and Michael Fineberg, Ph.D. (SAIC, finebergm@saic.com). The first presenter was Mr. Michael Leggieri (Anteon Corporation, US Army Medical Research and Materiel Command -CHPPM) who spoke on the “Materiel Command, Military Operational Medicine Research Program (MOMRP) Research Supporting Survivability.” The U.S. Army Medical Research and Materiel Command supports the Army Health Hazard Assessment (HHA) and Soldier Survivability (SSv) Programs with biomedical research products developed by the Military Operational Medicine Research Program (MOMRP). The MOMRP provides biomedical solutions to protect and enhance soldier performance in multi-stressor operational and training environments. The MOMRP is organized into three medical research areas: neuropsychology and performance, energetics and environmental medicine, and injury sciences. Major research projects in the energetics and environmental medicine, and the injury sciences areas produce health risk criteria and health risk assessment methods for the HHA Program. The MOMRP research also provides soldier survivability assessment tools for the Army Research Laboratory’s Survivability/Lethality Analysis Directorate (SLAD), and biomedically-valid design criteria for materiel developers. The MOMRP's current research program includes projects to develop HHA and SSv assessment methods for heat-related injuries, neck injury from head-supported devices, injuries from exposures to repeated jolt in ground vehicles, and incapacitation and injury from inhaled toxic fire gases.

The second presenter was Mr. Mark Geiger (Navy CNO, Occupational Safety and Health) who spoke on “Avoiding Injury Through Human- Capable Design In DoD.” The integration of human capable design concepts can enhance the DOD acquisition process and improve the system safety and occupational health programs. Human capable design is creating products that reduce user exposure to mechanical stress. The advantages include decreasing the risk of operator injury, increasing operator performance, and allowing operators to safely and comfortably interact with products. Ergonomic injuries and illnesses represent the single largest source of injury claims and costs to the Navy. Other services are estimated to be similarly affected. Specific examples of common DOD system design errors and their potential impacts were discussed. Recommendations were made for enhanced coordination between practitioners in the areas of ergonomics, cognitive sciences and system safety.

 

The next presenter was Mr. Chuck Jokel (US Army Center for Health Promotion and Preventive Medicine (CHPPM), Hearing Conservation Program) who spoke on “Update on Impulse Noise Hazard Assessments.” The current damage risk criteria (DRC) or assessment of impulse noise is a direct adoption of MIL-STD-1474 Design Criteria. This DRC has been unchanged for almost 40 years, except for addition of free-field criteria. The DRC is known to be over-protective at high end. The Army Research Laboratory (ARL) researchers have proposed a biomechanical model of the ear as a replacement. This model is called the Auditory Hazard Assessment Algorithm for Humans (AHAAH). The Center for Health Promotion and Preventive Medicine (CHPPM) and ARL are validating the version of AHAAH that includes hearing protection, and have recommended to the Office of The Surgeon General to approve the unprotected version of the model as a DRC. The ARL has already proposed adopting the entire model as a Design Criterion for use in MIL-STD 1474.
 

The next presenter was Mr. John Cambre (US Army Center for Health Promotion and Preventive Medicine-CHPPM) who spoke on “Soldier Occupational Health Assessment Program.” Soldier Occupational Hazard Assessment (SOHA), Stryker Brigade Combat Team (SBCT) Project, National Training Center, Fort Irwin, CA Oct 03. The study is a comprehensive Industrial Hygiene sampling survey, to assess soldier occupational exposures, to potential chemical and physical agents that could impact health and performance during operation of this new military vehicle. The sampling survey will be conducted through a collaborative venture with US Army Center for Health Promotion and Preventive Medicine (CHPPM) Headquarters, Aberdeen Proving Ground, MD and USACHPPM-West, Fort Lewis, WA.

 

Due to the urgent need to field a light armored vehicle, Army officials placed the type-classification and procurement of Stryker vehicles on the fast track.  Normally concept, development and procurement of fielded equipment takes years and follows procurement procedures that include health evaluation under AR 40-10, The Army Health Hazard Assessment Program. Stryker fast-track procurement has not allowed for the normal health hazard assessment process of industrial hygiene data collection and evaluation so the SOHA is concentrating on gathering and assessing this data.  The Stryker Project is designed to look at the potential health hazards and health effects associated with the new Stryker vehicle and its variants being fielded by the Army. These goals will be accomplished through general area, and personal sampling (collected on the soldiers), during field deployment and testing of the Stryker Vehicles during the NTC. The study findings and sample results will be presented in a format that is intended to help identify design problems that may pose a health and safety hazard to our soldiers and implement Force Health Protection by including personal sampling data into the soldier’s medical records.

 

The last presenter was LTCOL Donald White (Edp. Chief Safety Issues & Chief HF and Op Safety, HQ/USAF, Donald.white@pentagon.af.mil) who spoke on “DSOC Aviation Safety Improvements Task Force, Human Factors Working Group.” The SECDEF Memo of 19 May 03 charged DoD stated that “World-class organizations do not tolerate preventable accidents. Our accident rates have increased recently, and we need to turn this situation around. I challenge all of you to reduce the number of mishaps and accident rates by at least 50% in the next two years.” Based on an analysis of accident statistics the recommended study areas were:

 

The suggested approach was to examine these leading mishap categories and identify and mitigate hazards using common mishap accounting methods – meaningful metrics, common taxonomy/lexicon, data-driven processes, and reliable and valid data collection tools. To do this DoD Components were directed to adopt the accident categories, classification criteria, a human error categorization system, and reporting formats and procedures and to establish procedures to provide for the cross-feed of human error data using a common human error categorization system that involves a human factors taxonomy accepted among the DoD Components and U.S. Coast Guard.

 

This presentation described the Human Factors Working Group and its efforts in support of the DoD charge. Thus the HFWG has help to develop a common Human Factors Analysis and Classification System for DoD-wide implementation, recommended standardization of Human Factor and Human Performance terminology, promoted standardization of Human Factor data collection for DoD and provide Human Factors subject matter experts to all ASI TF working groups, JSAT, JSIT.

 

 

Technical Society/Industry. The Technical Society/Industry (TS/I) Sub TAG met twice during the TAG meeting on Tuesday morning and afternoon. Mr. Bill Lytle (William.b.lytle@lmco.com, (303) 971-8972) chaired the two meetings. In the morning session, the industry role in HSI was discussed, in light of the services’ definitions of HSI/MANPRINT varying significantly. Issues such as HSI products, plans, databases and reports were discussed.

 

Ms. Cynthia Nelson (Boeing) briefly discussed the SAE G-13 modeling and simulation committee of which she is a member. Mr. Baruch Sachs discussed the Usability Professionals Association (UPA). UPA is a large organization with an interesting website, http://www.upassoc.org/, many chapters and members. The UPA recently adopted a code of ethics. Mr. Lou Adams (Lockheed-Martin) provided a brief overview of the Government Electronics and Information Technology Association and the G-45 Human Factors Committee, of which he is the current chairman.

 

In the afternoon session, the T/SI group discussed Dr. Foster’s remarks about HSI guidelines for industry. The TS/I SubTAG will support any TAG efforts on this topic, including review and comment on any documents or ideas. An issue was raised pertaining to T/SI membership. Currently each technical professional society or industry group may have a primary representative and an alternate or two. Several large societies (e.g., APA, SAE) have many divisions and/or standing committees. In order to allow representation from multiple society sub-organizations, the TS/I charter would need to be amended and TAG approval secured.

 

Modeling and Simulation. Lt. Jeff Grubb (NAVAIR, jeff.grubb@navy.mil) is the new Modeling and Simulation SubTAG chair. The first presenter was Elizabeth Bowman, Ph.D. (US Army Research Lab (SLAD, ebowman@arl.army.mil) spoke on “Effects-Based Operations in Two Dissimilar Coalition Headquarters.” This presentation was based on Multi-National Experiment 4 (MNE4) to be held in February 2006. This experiment will investigate Stability Operations in Afghanistan with a distributed coalition of eight nations. Critical issues to be investigated include:

 

In the previous experiment, NME3, held in February 04, C3TRACE was used to model tasks, communications, and tasks assigned to individuals and groups. All communications were defined and characterized in terms of who, what, when, why, etc. /Communications were rated as good or degraded. Outputs of the study included utilization rates, high workload/task shedding events, and information quality. “Effects synchronization tasks” had, by far, the greatest task times. Moderate task times included war gaming, mission analysis, effects assessment and action assessments. Lowest task times included such tasks as determining priority lists, effects matrices and coalition process requests. Staff utilization rates were 98% for planners, 50% for operations 58% for information specialists, 20% for knowledge management, 13% for command staff. Lessons learned from NME4 included the following:

·      Significant communications delays

·      Distributed organizations are difficult to replicate and require careful communications

·      Distributed structures slowed tasks

·      Communications difficult to verify due to large nature of groups

·      Perceptions of information quality generated by groups were surprisingly low

·      Tasks tended to be centralized in collocated areas, rather than dispersed as planned.

IN NME4, Coalition Task Force (CTF) behavior will be compared to National Task Force (NTF) behavior. It is expected that CTF will initially have longer task times but they will improve.

 

The next speaker was Mark Livingson, Ph.D. (Naval Research Laboratory, mark.livingston@nrl.navy.mil) who spoke on “Situational Awareness for Teams of Dismounted Warfighters and Unmanned Vehicles.” A variety of studies are being conducted in this area, which is characterized as urban, quick-reaction forces, high pressure, increased use of unmanned ground vehicles. The goal of the NRL studies is to provide information superiority. A candidate solution being pursued is an Augmented Reality System – an overlay of graphics on real world imagery for dismounted Soldiers. So far, a number of domain analyses have been conducted. Based on initial investigations, it is believed that wearable systems hold promise.

 

The next speaker was Judith Orasanu, Ph.D. (NASA Ames Research Center, judity.orasanu@nasa.gov) who spoke on “What Needs to be Shared in Shared Situation Models?” This area of investigation looks into how people differ in their use of information. Shared data doesn’t guarantee shared understanding. The objective should be to focus decision making in dynamic environments. Challenges include situation assessment under uncertainty and projecting outcomes under uncertainty. Dr Orasanu reported on an investigation of differences between commercial and General Aviation (GA) pilots in how they respond to risk. Commercial pilots have a more complex understanding of risk (time pressure, risk management). GA pilots are more uni-dimensional in their assessment of level of risk. In looking at Air Traffic Controllers versus pilots, controllers focus on proximity and climb/descend behavior. Pilots focus on traffic density. Factors affecting perception of risk include personnel vulnerability, level of experience, control over the situation, timeframe, culture and personality. Conflicts based on different understanding of situation tended not to be resolved. Conflicts based on different information tended to get resolved much more quickly.

 

The last speaker was Gina Thomas-Meyers, Ph.D. who spoke on “Human Behavior Representation (HBR) in Constructive Simulation.” Dr. Thomas-Meyers was reporting on a NATO Research Study Group (RSG) of the NATO Research Technology Organization - Human Factors and Medicine Panel. The goal was to develop a generic structure of Human Factors for implementation of constructive models of various purposes, with emphasis on the balance between complexity, lack of detail and accuracy. The RSG is looking at mismatches between models and case demands – low cost-effectiveness and lack of re-use, insufficient validation and limited basis for establishing realism. The end product of this RSG is a Practitioner’s Guide.

 

Controls and Displays. Not attended.

 

Human Factors Engineering/Human Systems Integration: Management and Applications. Ms. Katrina Baker (US Army, Aberdeen Test Center, Katrina.baker@atc.army.mil) and Mr. Corry Cook (corey.cook@navy.mil) co-chaired the HSI/HFE session. The first presenter was Mr. James Fielder (US Army, Aberdeen Test Center, james.fielder@atc.army.mil) who spoke on “Joint Distributed Free-play Event (JOFE): The Future of Human Performance Testing.” The Aberdeen Test Center has over 70,000 acres for testing of all US Army systems. An HSI team was established in the spring of 2005. JDFE is a proof-of-concept event to demonstrate augmented T&E capabilities in a distributed operational military environment. Participants include active duty military, Army National Guard, and civilians. The JDFE timeframe was 22 August 05 through 1 September 05, with training occurring for the three days preceding the start. A pilot-down rescue mission was conducted. Questionnaires included MARS (SA plus workload), SABARS (SA), generic task questionnaires, comfort questionnaire, JTF questionnaire, data collector observations, and weather. Instrumentation included:

 

Since this event was just recently concluded, complete results are not available. However there are some emerging results. On the MARS questionnaire’s first four questions (on SA) there was an increase from the first to the second week. Workload decreased from the first to the second week. SABARS results were consistent with MARS. OP-TES indicated that Blue force kills (good kills bad) increased in the second week while OPFOR kills (bad kills good) decreased in the second week. The comfort questionnaire (30 questions asked every day) indicated that the impact of equipment on the Soldiers improved (they got used to it). PADMAS (temperature, heart rate and EEG) data are still being analyzed. New test instrumentation is desired, including a voice recorder, respiration measurement device, pill-type core temperature monitor, indoor tracking systems, “simunitions” (simulated munitions), and a ballistic impact vest (to simulate being hit).

 

The next presentation was by Ms. Debbie Patton (US Army Research Lab, HRED, dpatton@arl.army.mil) who spoke on “Effects of Stress on Cognitive Performance in Military Environments.” Her focus is on metrics and methods for cognitive readiness. They have developed a comprehensive psychological stress assessment battery, salivary amylase field assay kit, and several cognitive readiness measures, including:

o      Cognitive performance assessment for stress/endurance

o      Army cognitive readiness assessment

o      Resiliency MAP

o      CPASE (verbal memory recall, addition, logical reasoning and spatial manipulation)

 

Some initial findings include determining there is an 18% degradation in logical reasoning and a 13% reduction in verbal memory while wearing MOPP IV gear. Wearing of encapsulating gear raises feelings of failure and frustration; it also lowers feelings of well-being. To help assess battlefield cognitive effects, CASEL (Cognitive Assessment Engineering Laboratory) has been developed to determine impacts of communications on performance, impact of stress on team performance, effects of human-robot interaction and several other impact areas.

 

The next speaker was Mr. Steve Merriman (Boeing, stephen.c.merriman@boeing.com) who presented his observations on “System-of-Systems MANPRINT/HSI.” Mr. Merriman is manager of MANPRINT activities on the Army Future Combat Systems program. His presentation described some of the differences between implementing HSI or MANPRINT on a single program or platform (such as F-22 or Stryker) or a System-of-Systems such as FCS. Some of the major areas discussed were:

 

The next presenter was Mr. Terry Stanard, Ph.D. (USAF Research Laboratory, WPAFB, terry.stanard@wpafb.af.mil) who spoke on “Human Computer Interface Design Patterns for Cognitive and Collaborative Tasks within Command and Control.” Cognitive systems engineering is critical. Re-usable design patterns save costs. A comparison between using design patterns and custom designs follows:

 

         Design Pattern                Custom Designs                  Off-the-Shelf

Practical HCI design           Reflects current practice          Reflects the majority of work

Tailored to fit work             Perfect fit to work                   Clumsy fit to work

Lowered expenses              Very expensive                       Cheap

Lower risk                          Higher risk                              Lowest risk

 

An architecture student named Christopher Alexander developed a computer program for his doctoral dissertation that attempted to analyze and create new environments based on logical programmatic analysis. Disenchanted with computer-driven design, but more than ever interested in what made certain places work both spatially and psychologically, Alexander developed a theory of "fit" in terms of what he called "patterns". This theory suggested a means for creating successful places that blended application of logic with collective experience. This interest in creating new environments would mark all of his future works. HCI design patterns identify patterns around interactions with a “work field.” HUD layouts for example share many similarities. The vision is to have a design tool like an excel “chart wizard” where the user could select needed functions and the system would propose HCI approaches to implementation. The Air Force is working now with Navy SPAWAR Systems Center in San Diego; they may jointly host a workshop in the near future on HCI design patterns.

The next presenter was Dr. Lesial Crumpton-Young (Professor and Chair of the Department of Industrial Engineering and Management Systems, University of Central Florida), who spoke on “Quantifying Total Body Fatigue: An Approach for Estimating the Impact of Task Demands on Human Performance.” There are five faculty members assigned to the ergonomics/human factors team at UCF. They are integrating research on human cognitive and physical capabilities; they are:

o      Developing quantitative metrics

o      Quantifying factors to predict performance

o      Develop new methodologies, robust models

o      Develop innovative tools, methods and computer models

 

The research areas being pursued include fatigue, virtual reality, human reliability, cumulative trauma disorders, information security and advanced computer models & techniques. The UCF team vision is to understand the impact of total body fatigue on human performance and identify interventions for enhancing performance. Their objective is to quantify how much total body fatigue is present, including:

o      Mental components

o      Physical components

o      Development of mathematical models to integrate results from various assessments (they will construct performance capability models based on total body fatigue and then integrate them into one model). The approach is to use linguistic modeling and fuzzy set theory.

o      Validation of models

 

Future activities include focusing on:

o      The relationship between time-stress and fatigue

o      The relationship between stress, workload and fatigue

o      The relationship between work demands and fatigue

 

User-Computer Interface. LT Nausheen Momen (US Military Academy, Department of Behavioral Sciences and Leadership, nmomen@kenbt.edu) and Stephen C. Merriman (Boeing, stephen.c.merriman@boeing.com) co-chaired the meeting.

 

The first speaker was Elizabeth Bowman, Ph.D. (US Army Research Laboratory, Human Research and Engineering Directorate, ebowman@arl.army.mil) who spoke on “Human Factors of Communications Capabilities in Platoon Field Tests.” Dr. Bowman reported on experimental results involving the tryout of a C4ISR on-the-Move test bed vehicle at Ft. Dix. The goal was to determine how communications could be used to increase SA. Basically in the experiment, a base case (traditional) was compared against the use of unmanned ground and air vehicles as well ass unmanned sensors. Participants all had on average about 9 years of experience. The personnel knew very little if anything about the Future Combat Systems program. ASA TLX was used four times during and after experimentation to assess workload. Physical and cognitive workload was higher in the baseline case but frustration was higher with the experimental case. The workload results indicate that Training, Tactics and Procedures are needed to take full advantage of the high technology systems. In the baseline case, Soldiers had no knowledge of where Blue forces were; in the experimental case there was much better SA on friendlies. UGV and UAV info quality scores were very low. Unmanned sensors couldn’t tell Soldiers if Red forces were advancing or retreating. FBCB2 in the vehicle increased the platoon leader’s SA. Dismounted use of FBCB2 was fraught with problems – power, glare, dysfunctional mouse, no indication that messages had failed, etc.). SAGAT was used to assess SA. Very little activity was done on the move. Relative scoring on representative on-the-move tasks were

 

In actuality, Soldiers spent most of their time in vehicles either eating or sleeping. FBCB2 got high marks. Imagery from UAV and UGV were rated as good.

 

The second speaker was Ms. Cynthia Nelson (Boeing, Cynthia.a.nelson@boeing.com) who spoke on “User Interface Considerations for Improving Situational Awareness.” Ms. Nelson is currently investigating how to improve SA and reduce workload. She explained many of the issues that must be taken into consideration when developing or upgrading systems.

 

The last speaker was Ms. Janet Perron (Mitre Corporation, jperron@mitre.org) who spoke on “Interaction Challenges for Pen-based Input at the Front Line.” A current pen-based user interface (“TACTER” was evaluated in a Joint, Close Air Support (CAS) environment. An Air Force Tactical Air Coordination Party (TACP) has the job of coordinating aircraft fire power against enemy ground forces; this can be either via pre-planned (e.g. Air Tasking Order) or emergency divert/reassignment. TACPs are co-located with Army units. TACPs use five classes of equipment: radio, laser rangefinder, ruggedized PC (TACTER) a variety of software, and multi-band multimode vehicle radio systems. The TACTER has lengthy set-up procedures, limited connectivity (must be stationary), slow down-load of shared data and limited battery life. The software includes a map with own position and location of other ‘friendlies.’ TACTER uses the laser rangefinder (LRF) to plot target location. TACTER also enables communications between the TACP, command center and aircraft. The task was to perform a human factors evaluation of TACTER; this was done via heuristic evaluations and observation of exercises.

 

TACPs are a high stress, mission critical operation. TACTER has a small display that is hard to see. It must be NVG/Laser protection capable and compatible with CBR gear. The software is not optimized for pen-based input or expected environmental conditions (traditional windows interface). Screens are complex and densely populated. TACTER employs pull down menus with more than 7 items, does not incorporate traditional right-click pop-up menu commands or short-cuts. “Targets” pull-down menu arrows, buttons, etc.) are too small to be used with a pen. Lighting on the unit was designed for office use, not the outdoors, in a laser, CBR environment. TACTER formats do not match each other or follow any sort of convention; they don’t match the tasks well either. Frequently used controls are not placed consistently – for example, “OK” appears in at least three different locations. “Required” fields are scattered among “optional” fields, making them hard to find. Terminology is inconsistent across screens. Specific findings regarding the map are as follows:

o Screen is too small

o Detailed map is very slow to update (processor limited)

o Filtering controls are not suited to the information environment (not easy to use)

o Map manipulations (e.g., zoom, pan) are too difficult to use

o Intermittent connectivity forces loss of SA

o User is not told when the map has been updated

 

Ongoing activities include development of new, ruggedized tablets. Mitre is working with TACP personnel to understand tasks. Mitre is investigating adaptive interface technologies - that is, software that will run on PDAs, PCs, TACTER, tablets, etc. and adjust to the hardware. They are also investigating how to optimize pen-based manipulations and improve filtering controls for high density target environments. At the conclusion of the presentation Ms. Perron was asked if this Air Force developed system had any HFE requirements levied in the contract; Ms. Perron did not know if there were or not.

 

Human Factors Test and Evaluation. This Sub TAG meeting was co-chaired by Mr. John Rice and Mr. Darren Cole. The first presenter was Mr. John Winters, BCI, who spoke on “An Operation Assessment of USS Ronald Reagan”. Data were collected on six events, three with HFE riders. The events were highly rehearsed, pre-disclosed events with SME prompting. They involved single ships only (no data link or distributed data distribution). Ninety percent of ratings were very low for workload; performance ratings were much higher. A modified Cooper-Harper rating scale was used and raters found the wording non-intuitive….it wasn’t in “sailor-speak.” A different workload scale may be more sensitive. It was difficult to select the right events to monitor so criteria were authored for choosing events; realistic scenario, representative watch standers, software and hardware presents a coherent tactical picture, degree to which results can impact on design. Future plans are to resolve how and when to best do HSI evaluations, refine methodology (closer tie of workload measures to system performance measures), define how results can be tied to test requirements called out in certification policy.

 

The next presenter was Linda Torsani Fatkin (ASRL HRED, lfatkin@arl.army.mil) and Dr. Michael Paley (Aptima, Inc.) who spoke on “CogFIT.” The Cognitive Fightability Index Tool (CogFIT). CogFIT metrics are defined to assess cognitive capabilities. Four hundred tests for cognitive capabilities were reviewed. Sixteen cognitive attributes were assessed (e.g., sustained attention, working memory, decision making, task multiplexing, genitive flexibility, concept formulation). CogFIT may be used to predict effects of technology on operational performance or estimate performance of technology under novel conditions. The CogFIT model is shown in Figure 2.

 

 

 

The next speaker was Pamela Savage-Knepshield, Ph.D. (US Army ARL-HRED, psavageknepshield@arl.army.mil), who spoke on “Quantifying the Usability and Utility of the Warfighter-Machine Interface.” US Army ARL-HRED performed an after-the-fact assessment of ease of use and survivability on the “Plugger GPS” system. Although GPS is now available to Soldiers, maps and compass are still the primary land navigation tools. Sixteen Soldiers were interviewed at Ft. Gordon and Ft. Leonard Wood. Evaluations were conducted under both day and night conditions. Soldiers were first trained to criterion on four tasks. Then, they performed the tasks and answered post-test questionnaires. Measures of success on usability were:

·  Success rate on tasks

·  Ease of use using a 5-point rating scale

·  Cognitive workload using a modified Cooper-Harper scale (1-3 =OK, 4--10=unacceptable)

Goals for the study were an 85% success rate for non-critical tasks and 100% for critical tasks. Most tasks (basically, looking for things) went well. Hierarchical SA loss problems were noted. Other findings included BIT buried in lower level menu (hard to find), unobtrusive hazard alert notification. A critical fault was discovered; during a cal for fire, 35% of Soldiers mistakenly called for fire on their own positions. A warning page was added (there are now over 100 warning screens!). Some of the Soldier feedback included statements as “We’re Soldiers, not engineers.” Some of the lessons learned: must include Soldiers during product design, and re-testing must be conducted to ensure that mitigations are effective.

 

The next speaker was Mr. Jim Cotton (Navy Human Performance Center (JHT, Inc., james.cotton.ctr@navy.mil), who spoke on the “Navy Shared Performance Metrics Repository.” Navy HPC current barriers to effective measurement were briefly described – such as methodological barriers (no baselines, lack of knowledge of psychometrics). HPC wants to improve human performance measurement by implementing standardized tools and methods, common vision and action. A human performance repository with web-based access is being developed. Wide use is expected; access by industry is a possibility but not primary. Some high emphasis areas are:

·  New technologies, such as NCW, require new methods (team performance measurement in NCW – who is the team?)

·  Human reliability assessment

 

Personnel Selection and Classification. Not attended.

 

Sustained/Continuous Operations (SUSOPS/CONOPS). Not attended.

 

Workload and Stress. Not attended.

 

DOD HFE TAG Operating Board Meeting:

 

1.Sub TAG Minutes: Sub TAG chairs are required to submit minutes to the TAG coordinator within 10 days of the completion of the TAG meeting.

2. Hot Issues Document: An ad hoc group will be formed to clarify what the Hot Issues document should consist of. The ad hoc group will be chaired by Dawn Woods and members will include Steve Merriman, Teresa Alley, Adrian Salinas, John Rice, Faith Chandler and Al Poston.

3.Revamping of the TAG Web site: Redesign might be competed at a Service academy or other military school. Maureen Burgondy-Wilhelm will contact Dr. Nita Lewis at Navy PGS.

4. New TAG Brochure: A new brochure will be ready for the next TAG meeting in Las Vegas NV.

5.Contractor Badge Policy: Some attendees are uncomfortable that they cannon readily determine who are contractors at TAG meetings. It was suggested that the TAG go back to different colored badges to designate contractors. Two other suggestions were put forward: either mark contractor badges with “Invited Guest” or place a US Flag sticker on DoD personnel badges. Sub TAG chairs were requested to solicit feedback from their groups.

6. Caucus Reports:

·      Air Force: The USAF hosted TAG-55 theme will be “enabling Decision Superiority with HSI.” There were 11 attendees at the caucus.

·      Army: There were 11 attendees at the caucus. Many army personnel did not attend.

·      FAA: No report

·      NASA: No report.

7. HSI Questions Raised by Dr. Foster (DDR&E): It was decided that the Service Representatives (plus FAA and NASA) would determine what is currently underway in terms of providing HSI guidance to industry and to identify “holes.” Feedback was requested from service representatives at the Operating Board meeting.

·      Army: It was indicated that the Army used to have HSI personnel at requirement generations sites, but this is no longer the case. HSI training is poor. Some documentation is in place but rarely used. There is no lessons-learned repository.

·      Air Force: Training exists for government and government contractor personnel. Requirements handbooks are needed. There is a crew systems integrity program. There may be a workshop on HSI in the future.

·      Navy: (Response to be provided)

8. TAG Meeting Scheduling: It was suggested that the Operating Board members meet briefly with the Executive Committee prior to the plenary session for a pre-brief on what should be done during Sub TAG meetings.

9. HSI Workshop: It was suggested that 20 or fewer TAG personnel meet early in calendar year 2006 to work on the HSI issues raised by Dr. Foster. Orlando FL was suggested as a good place for this workshop. Steve Merriman was requested to prepare a “one-pager” for Maureen Burgondy-Wilhelm on this workshop.

 

 

ATTACHMENT (1)

 

DOD HFE TAG Background

 

The DoD HFE TAG was begun via memorandum of agreement signed by the Service Secretaries in November 1976. Goals of the TAG were established as follows:

 

• Provide a mechanism for exchange of technical information in the development and

   application of human factors engineering.

• Enhance working level coordination among Government agencies involved in HFE

        technology research, development and application.

• Identify human factors engineering technical issues and technology gaps.

• Encourage and sponsor in-depth technical interaction, including subgroups as required in selected topical areas.

• Assist as required in the preparation and coordination of Tri-Service documents such as           Technology Coordinating Papers and Topical Reviews.

 

The TAG addresses research and technologies designed to impact man-machine system development and operation throughout the complete system life cycle. Topics include:

 

• Procedures for use by HFE specialists, system analysts and design engineers in providing HFE support during system development and modification

• Methodologies to identify and solve operator/maintainer problems related to equipment design, operation and cost/effectiveness

• Mechanisms for applying HFE technologies, including formal and informal approaches to validation and implementation, and the determination of time windows for application.

 

The TAG comprises technical representatives from Government agencies with research and development responsibilities in the topical areas mentioned above. Additional representatives from activities with allied interests affiliate with the TAG as appropriate. Technical experts in special topic areas may augment attendance at specific meetings. Also participating in the TAG are official representatives of technical societies (e.g., Human Factors and Ergonomics Society, SAFE Association) and industrial associations (e.g., Electronics Industry Alliance) with a stated interest in HFE. These representatives may attend subgroup and general plenary sessions and they must be credentialed by the TAG prior to attending any meetings.