AFCAPS-FR-2011-0006Knowledge, Skills, Abilities, andOther Characteristics forRemotely Piloted Aircraft Pilotsand OperatorsWilliam R. HowseDamos Aviation Services, Inc.Sponsored byHQ AFPC/DSYX & HQ AF/A1PFKenneth L. SchwartzStrategic Research and AssessmentBranchOctober 19, 2011Air Force Personnel CenterStrategic Research and AssessmentHQ AFPC/DSYX550 C Street West, Ste 45Randolph AFB TX 78150-4747Approved for Public Release. Distribution UnlimitedUNCLASSIFIED

NOTICExxxx xxxxxxxPA Review: Cleared for Public ReleaseAir Force Personnel Center; Office of Public AffairsPA: Ms Paige HughesWhen Government drawings, specifications, or other data are used for anypurpose other than in connection with a definitely Government-relatedprocurement, the United States Government incurs no responsibility or anyobligation whatsoever. The fact that the Government may have formulated or inany way supplied the said drawings, specifications, or other data, is not to beregarded by implication, or otherwise in any manner construed, as licensing theholder, or any other person or corporation; or as conveying any rights orpermission to manufacture, use, or sell any patented invention that may in anyway be related thereto.This report was cleared for release by HQ AFPC/DSYX Strategic Research andAssessment Branch and is releasable to the Defense Technical InformationCenter. The Public Affairs Office has reviewed this paper, and it is releasable tothe National Technical Information Service, where it will be available to thegeneral public, including foreign nationals.This report is published as received with minor grammatical corrections. Theviews expressed are those of the authors and not necessarily those of the UnitedStates Government, the United States Department of Defense, or the UnitedStates Air Force. In the interest of expediting publication of impartial statisticalanalysis of Air Force tests SRAB does not edit nor revise Contractorassessments appropriate to the private sector which do not apply within militarycontext.Federal Government agencies and their contractors registered with DefenseTechnical Information Center should direct request for copies of this report to:Defense Technical Information Center - for public release. Distribution Unlimited. Please contact AFPC/DSYXStrategic Research and Assessment with any questions or concerns with thereport.This paper has been reviewed by the Air Force Center for Applied PersonnelStudies (AFCAPS) and is approved for publication. AFCAPS members include:Senior Editor Dr. Thomas Carretta AFMC 711 HPW/RHCI, Associate Editor Dr.Gregory Manley HQ AFPC/DSYX, Dr. Lisa Mills AF/A1PF, Dr. Paul DiTullio HQAf/A1PFA, Kenneth Schwartz HQ AFPC/DSYX, Johnny Weissmuller HQAFPC/DSYX, Dr. Laura Barron HQ AFPC/DSYX, Dr. Mark Rose HQAFPC/DSYX, and Brian Chasse HQ AFPC/DSYX.

Form ApprovedREPORT DOCUMENTATION PAGEOMB No. 0704-0188Public reporting burden for this collection of information is estimated to average 1 hour per response, including the time for reviewing instructions, searching data sources,gathering and maintaining the data needed, and completing and reviewing the collection of information. Send comments regarding this burden estimate or any other aspect of this collectionof information, including suggestions for reducing this burden to Washington Headquarters Service, Directorate for Information Operations and Reports,1215 Jefferson Davis Highway, Suite 1204, Arlington, VA 22202-4302, and to the Office of Management and Budget,Paperwork Reduction Project (0704-0188) Washington, DC 20503.PLEASE DO NOT RETURN YOUR FORM TO THE ABOVE ADDRESS.1. REPORT DATE (DD-MM-YYYY)2. REPORT TYPE19-10-20113. DATES COVERED (From - To)TechnicalAugust 2009 – September 20114. TITLE AND SUBTITLEKnowledge, Skills, Abilities, and Other Characteristics forRemotely Piloted Aircraft Pilots Operators5a. CONTRACT NUMBERW911NF-07-D-00015b. GRANT NUMBER5c. PROGRAM ELEMENT NUMBER6. AUTHOR(S)5d. PROJECT NUMBERHowse, William R.5e. TASK NUMBERTCN-09-02165f. WORK UNIT NUMBER7. PERFORMING ORGANIZATION NAME(S) AND ADDRESS(ES)8.Damos Aviation Services, Inc.36303 N. Old Woods TrailGurnee, IL 60031-1683PERFORMING ORGANIZATIONREPORT NUMBERDAS 2011-049. SPONSORING/MONITORING AGENCY NAME(S) AND ADDRESS(ES)10. SPONSOR/MONITOR'S ACRONYM(S)Air Force Personnel CenterStrategic Research and Assessment BranchRandolph AFB TX 78150HQ AFPC/DSYX11. SPONSORING/MONITORINGAGENCY REPORT NUMBERAFCAPS-FR-2011-000612. DISTRIBUTION AVAILABILITY STATEMENTApproved for Public Release, Distribution Unlimited13. SUPPLEMENTARY NOTES14. ABSTRACTThis report presents the results from a review of a literature base of more than 200 publications in which eightwere found to contain lists of Knowledge, Skills, Abilities, and Other Characteristics (KSAOs) for remotelypiloted aircraft (RPA) crew positions, either for the purpose of system design specification or for personnelselection. The report compares KSAO lists across these studies and presents a cross reference table for them.The report also surveys past efforts to establish KSAOs in anticipation of related developments and discussesthe major likely sources and nature of future changes in KSAO demand for RPA operators.15. SUBJECT TERMSAviator selection, classification, pilot selection, KSAO assessment, UAV, unmanned, UAS, RPA, remotelypiloted, sensor operator, drone17. LIMITATION OFABSTRACT16. SECURITY CLASSIFICATION OF:Unclassifieda. REPORTUb. ABSTRACTUc. THIS PAGEU18. NUMBEROF PAGES51U19a. NAME OF RESPONSIBLE PERSONKenneth L. Schwartz19b. TELEPONE NUMBER (Include area code)210-565-3139i

ACKNOWLEDGMENTThis work was supported by the AIR FORCE PERSONNEL CENTER (BRIAN G CHASSE)under the auspices of the U.S. Army Research Office Scientific Services Programadministered by Battelle (Contract No. W911NF-07-D-0001, Delivery Order 0906).ii


ABSTRACTThis report presents the results from a review of a literature base of more than 200 publicationsin which eight were found to contain lists of Knowledge, Skills, Abilities, and OtherCharacteristics (KSAOs) for remotely piloted aircraft (RPA) crew positions, either for thepurpose of system design specification or for personnel selection. The report compares KSAOlists across these studies and presents a cross reference table for them. The report also surveyspast efforts to establish KSAOs in anticipation of related developments and discusses the majorlikely sources and nature of future changes in KSAO demand for RPA operators.iv

KNOWLEDGE, SKILLS, ABILITIES, AND OTHER CHARACTERISTICSFOR REMOTELY PILOTED AIRCRAFT PILOTS AND OPERATORSIntroductionIn August 2009 Damos Aviation Services (DAS) accepted Subcontract TCN 09216 from BattelleColumbus Operations under the U.S. Army Research Office Scientific Services Program,Contract W911NF-07-D-0001 / Delivery Order 0906. This report responds to the requirement tocompare and contrast current and future Knowledge, Skills, Abilities, and Other Characteristics(KSAOs) for Remotely Piloted Aircraft operators. In pursuit of this requirement relevant workswere extracted from more than 40 sources beginning with the bibliographic database previouslyproduced under this Delivery Order (Howse & Damos, 2011) and expanding with search criteriatailored to remotely piloted systems. More than 200 works were reviewed and only eight werefound that are relevant to identification and evaluation of KSAOs for remotely piloted aircraftpersonnel and that at least elicited information from practitioners and/or subject matter expertsthat contributed to the identification of KSAOs.Terminology varies considerably across services and over time within services. Even the phrasesused to refer to the systems of interest are different. The phrases found in common use are:Unmanned Aerial Vehicle (UAV), Unmanned Aerial System (UAS), Unmanned Combat AirVehicle or Uninhabited Combat Aerial Vehicle (UCAV), Remotely Piloted Vehicle (RPV),Tactical Unmanned Aerial Vehicle (TUAV), Unmanned Combat Armed Rotorcraft (UCAR),Vertical Take-off UAV (VTUAV), Ground Control System (GCS), and Common GroundStation (CGS). Because the U.S. Air Force has transitioned to universally adopting the term,Remotely Piloted Aircraft, the default term for air vehicles for this report is RPA.There are a number of crew positions that are part of RPA systems. Not all of these have beentargets of research efforts to identify or develop KSAOs. Again, there are differences interminology across services. The crew positions that are the subject of one or more efforts citedin this paper are listed with their relevant Military Occupational Specialties (MOS), Ratings, andAir Force Specialty Codes (AFSC) are listed in Table 1.1

Table 1. Duty positions and their associated MOS, Rating, AFSC.Personnel Duty PositionsAir Vehicle Operator (AVO)ServiceUSAMOS,Rating,AFSC96UUSA15WRPA operators are integral to providing Army personnelwith information about enemy forces and potential battleareas. RPA operators are remote pilots of unmannedobservation aircraft, who gather and study informationthat's required to design operational plans and tactics. TheRPA operator supervises or operates the RPA, such as theArmy's Shadow RPA, to include mission planning, missionsensor/payload operations, launching, remotely piloting,and recovering the aerial vehicle.USNNEC 8363USMC7413Operates and navigates RPA during the enroute, mission,and return phase of flight.External RPA operators execute the initial takeoff and finallanding phases of RPA operations. They are also anintegral part of all mission planning, takeoff and landingsequences, and crew coordination aspects of RPA flight.USAF18XUSNNEC 8362USMC7413USA96UUSA15WUSNNEC 8364Basic Position DescriptionThe tactical unmanned aerial vehicle (TUAV) operatorsupervises or operates the TUAV, to include missionplanning, mission sensor/payload operations, launching,remotely piloting, and recovering the aerial vehicle.Air Force RPA Pilots plan and prepare for missions. Theyreview mission taskings, intelligence, and weatherinformation, supervise mission planning, equipmentconfiguration, and crew briefings. Also, they ensure groundstation and aircraft are preflighted, inspected, loaded, andequipped for mission. RPA pilots operate aircraft andcommand crew, operate aircraft controls and equipment,perform, supervise, or direct navigation, surveillance,reconnaissance, and weapons employment operations.Further, they conduct or supervise training ofcrewmembers, ensure operational readiness of crew byconducting or supervising mission specific training. RPAPilots also develop plans and policies, monitor operations,advise commanders, assist commanders and perform stafffunctions related to this specialty.Internal Pilot (IP) - See AVO aboveExternal Pilot (EP)Mission Package Operator (MPO)Performs organizational level maintenance on aircraftsystems. RPA External Pilot directly controls the flight ofthe RPA during launch and recovery operations by visualreference to the RPA.Operates the EO/IR RPA sensor during all phases of flight.2

USAF1UOX1USNNEC 8364Common Ground Station Operator(GSO)USA96HThe common ground station (CGS) operator supervises orparticipates in detecting, locating and tracking groundtargets and rotary wing and slow moving fixed wingaircraft. The GSO receives Joint Surveillance TargetAttack Radar System (JSTARS) near-real time radarimagery data, RPA imagery, Commander’s TacticalTerminal/Joint Tactical Terminal (CTT/JTT) SignalsIntelligence (SIGINT) data, and Secondary ImageryDissemination (SIDS) products which are transmitted tothe Common Ground Station (CGS) to provide situationaldevelopment, battle management, and targetinginformation and imagery intelligence of value to thecommander.Intelligence AnalystUSA96BThe intelligence analyst supervises, performs, orcoordinates, collection management, analysis, processing,and dissemination of strategic and tactical intelligence.Imagery Analyst (Enlisted)USA96DThe imagery analyst supervises or analyzes aerial andground permanent record imagery developed byphotographic and electronic means. Plans andrecommends the use of imaging sensors forreconnaissance and surveillance missions.Sensor Operator (SO) - See MPOaboveMission Commander (MC)RPA Sensor Operators perform duties as a mission crewmember on unmanned aerospace systems. They employairborne sensors in manual or computer-assisted modes toactively and/or passively acquire, track, and monitorairborne, maritime and ground objects. Qualified personnelconduct operations and procedures in accordance withSpecial Instructions (SPINS), Air Tasking Orders (ATO)and Rules of Engagement (ROE). Crewmembers assistUAS pilots (who are commissioned officers) through allphases of employment to include mission planning, flightoperations, and debriefings. Sensor Operators continuallymonitor aircraft and weapons systems status to ensurelethal and non-lethal application of airpower. At present,Air Force 1UOX1 specialists perform their duties on theMQ-1 Predator and the MQ-9 Reaper remotely pilotedaircraft (RPA).3

Imagery Analyst (Warrant Officer)USA350DProvides technical expertise and manages activitiesengaged in imagery interpretation activities. Acts as thechief of a platoon, section, detachment, or team performingimagery interpretation. Identifies changes of terrain,equipment locations, troop movements, or otherinformation that contributes to intelligence. Identifiesequipment by nomenclature and location to developassessments of possible threat to U.S. forces. Developssummaries and prepares reports on imagery interpretationfindings. Establishes and maintains files on imageryinterpretation data, findings, records, and reports.Develops map overlays which reflect changed tacticalinformation. Conducts intelligence briefings based oninformation obtained.4

Historical BackgroundThe following is an extremely brief timeline of the development and acquisition of remotelypiloted systems by the U.S. military. It is included to give the reader a sense of the time-span ofRPA development and the rapidity, in recent years, with which system capabilities haveexpanded. It will also give the reader a sense of position in the timeline for the RPA systems thatwere evaluated in studies cited below. A far more complete history of RPAs has been written byL.R. Newcome (Newcome, 2004). Descriptions of current and emerging RPA systems can befound in the Department of Defense’s Unmanned Systems Integrated Roadmap for the years2009 to 2034 (Department of Defense, 2009).The first documented application of remote piloting in aviation was contained in an operatingmodel dirigible balloon exhibited by A.J. Roberts of Australia in 1912. This balloon was radiocontrolled (based on designs of Nikola Tesla). The model was about 15 feet long and could becontrolled at a maximum range of about 500 feet.At the beginning of World War I Professor A.M. Low of England, whose prior efforts wererelated to radio range finding, developed a remotely piloted airplane for the Royal Flying Corps.He successfully developed a radio control system. However, the engine of the prototype aircraftcreated too much radio interference for it to function properly. The first successful flight of apractical remotely piloted dirigible balloon took place in 1924 when the Royal AircraftEstablishment’s Target flew for 12 minutes after launch from the deck of the HMS Stronghold.This project led to the design of the Larynx, a monoplane designed as an unmanned aerial bomb.It first flew in 1927 but there is no record of application in combat. The DeHavilland TigerMoth, a biplane, served as a trainer from 1932 to 1947 in England. Several hundred of theseaircraft were modified as remotely piloted target aircraft (designated Queen Bee). A few of thesewere employed in coastal reconnaissance missions during World War II.In the early 1950s the Northrop/Radioplane Company developed the YQ-1B high altitude targetdrone into an unmanned reconnaissance vehicle, the B-67 Crossbow. The YQ-1B was equippedwith a warhead and a radiation seeker for attacking air defense systems. The acquisitionprogram was cancelled in 1957 before it became operational. Another early remotely piloted airvehicle was the AQM-34A Firebee I. This system was a modified target drone developed byTeledyne Ryan for the U.S. Air Force. It first flew in 1960. It could autonomously fly preprogrammed reconnaissance routes and employ a variety of mission equipment modulesincluding photographic and infrared imaging and electronic countermeasures. Firebee I wasused during the Vietnam war. No records have been located regarding selection practices forsystem operators.In the 1970s the Army developed the MQM-105 Aquila, with Lockheed Missile and SpaceCompany as the major contractor. It was designed for multiple battlefield roles concentrating ontarget acquisition, designation and aerial reconnaissance. The prototype first flew in 1975 andthe first full scale development air vehicle flew in 1982. It had an endurance of 3.5 hrs andcarried a daylight television camera and a laser rangefinder/designator with autotrackingcapability. Its flight path could be preprogrammed by altering waypoints stored in the flightcontrol system. Aquila was never considered a successful system and was cancelled in 1987.5

The RQ-2 Pioneer was procured first by the Navy in 1985 from Israeli Aircraft Industries (IAI)as a naval gunnery spotting system. It has been used by the Army, Navy and Marine Corps. Ithas an approximate 5 hour endurance carrying electro-optical and infrared sensors.Another system originally developed by IAI is the RQ-5 Hunter, originally procured from TRWin 1993 and, starting in 2003, from Northrop Grumman as the MQ-5B. Hunter systemacquisition ended in 1996 but existing units continue to be employed and retrofitted. Thesesystems have an endurance of 12-18 hours. In 1999 some were equipped with a laser designatorand in 2003 some were modified to carry a variant of the Brilliant Anti-Tank (BAT) guidedmunition. They can carry a variety of munitions as wing stores and a variety of sensor packageswith electro-optical and infrared capabilities. It also can carry a radio relay package to provideextended communications ranges.The Army began procurement of the RQ-7 Shadow in 1999. It has an on-station endurance ofapproximately 6 hours and carries a variety of electro-optical and infrared sensor packages andcan be equipped with a laser designator.The MQ-1 Predator was developed by General Atomics for the Air Force, though Army andNavy also use them, and has been in service since 1995. This is a long-range system capable ofcovering very long distances to an area of operations and loitering there for as much as 14 hoursbefore returning. Whereas the preceding systems are controlled by line-of-sight radio link,Predator uses satellite links, allowing operators to be located nearly anywhere on the globe. Itcan carry a variety of sensor packages in daylight and infrared modes plus a laser designator andsynthetic aperture radar. It can carry various wing-mounted stores including weapons, especiallymodified Hellfire missiles. Variants include the MQ-1C Extended Range/ Multipurpose(ER/MP) UAS with greater range and payload, and the MQ-9 Grey Eagle.Identifying RPA Operator KSAOsIn 1979 an effort was made by the U.S. Army Research Institute for the Behavioral and SocialSciences (ARI) to extract from the small existing cadre of remotely piloted air vehicle operatorsand sensor operators a set of attributes that could be used to inform the development of selectionand training systems (Crumley & Bailey, 1979). The approach was to administer two surveyinstruments, one to air vehicle operators and the other to sensor operators, and use theirresponses to guide structured group interviews. The surveys were made up of 27 multiple choiceitems, mostly having a Likert-type format, and one open-ended item soliciting written comments.Participants in the interviews were six airframe and sensor operators and nine support personneland supervisors. These people were serving in the development group for the Aquila system.The survey items were derived from system manuals and contractor job descriptions. The groupinterview was guided toward consensus. The responses to the survey instruments were neithertabulated nor reported.The authors suggested there were two personality types within their subject sample. This isbased on the existence of two comparatively distinct attitudes toward continued work as systemoperators. Personnel who preferred the air vehicle operator role were described as having anaffinity for planning and logic, those who preferred the sensor operator role were described ashaving an affinity for uncertainty. All operators disliked having close supervision during6

missions but agreed that a third (artillery trained) person was needed in the Ground ControlStation (GCS).The authors inferred from their data that eye/hand coordination was an important component ofjob performance but that physical strength, endurance, hearing acuity and color vision were notrequired at greater than average levels. Other attributes included patience and logic. Skill inverbal communication was also indicated as important. Two specific knowledge componentswere identified: map reading and photo interpretation.After a 19 year absence of any apparent attempts to identify KSAOs for unmanned aerial systemoperators, Biggerstaff et al. (Biggerstaff, Blower, Portman, & Chapman, 1998) conducted a moreformal project to predict training performance albeit their scope was limited to the role of theRPA external pilot (EP). This study had the expressed purpose of establishing selection criteriafor entrance into training for Pioneer RPA operators. The attributes identified were also used toassist in making recommendations for medical screening of applicants for EP and internal pilots(IP); however, this study did not establish any training or job performance predictors for anycrew position other than that of EP. The EP for the Navy version of Pioneer has a moreextensive role in system operation than in larger RPAs. For Pioneer the EP is in directinteractive control of the airframe for takeoffs, landings, and while the airframe is in visual rangeof the crew. That is, the EP operates the airframe by manipulation of a set of flight controlssimilar to those of a recreational radio controlled aircraft. Takeoff and landing may beaccomplished on a short airstrip or by jet-assisted launch from a pedestal and landing by capturenet (shipboard operations).The authors approached the operator tasks through task analyses, field observations, interviews(structured and unstructured), and test battery trials. An initial set of critical tasks was derivedfrom observations and interviews of EPs, IPs, mission payload operators (MPO) and otherrelated personnel. These tasks provided the source for personal attributes posited to affectperformance. A single test battery, the Computer-Based Performance Test (CBPT) was selectedto measure predictor variables (Delaney, 1992). A criterion measure was formed from anunspecified composite of instructor evaluations and flight grades, on a 0-100 scale.The derived list of attributes, according to the authors, consisted of skills only; no specificknowledge, abilities, or other attributes were included. The derived skills were likely referred toas such because their source was a sample of fully trained, successful practitioners, rather thancandidates. Therefore, personal attributes imparted by prior selection and training processes areassumed to be present. These skills were: Mental reversals/rotation, estimation of time tocontact, eye-hand coordination, selective auditory attention, and multitasking (psychomotor visual). It was determined that six subtests of the CBPT would adequately measure these skills:Psychomotor (PMT), Dichotic Listening (DLT), Horizontal Tracking (HT), Digit Cancellation(DC), Manikin, and Time Estimation (TET).It appears to be the case that performance on the test battery components operationally definedthe skill set. These test battery components are neither simple nor independent. They aredesigned to be administered in combinations and in specific orders progressing in complexity.PMT is combined with DLT, HT with DC, and Manikin with TET. The test components were7

associated with specific abilities: PMT with multilimb coordination, DLT with divided attention,HT with perceptual motor tracking, DC with reaction time and short-term memory, Manikin withmental rotation and short-term memory, and TET with perceptual tracking. Note that theabilities associated with DC do not directly correspond to any of the identified skills but maycontribute indirectly to their measurement. The authors state that DC was used as a distracter,although no rationale for this is given. It may be that without an interfering secondary task HTwas insufficiently difficult to produce measurement variance. The prototype battery wasadministered to eight students in EP training and six practicing EPs. It is not specified whetherdata from all 14 subjects were used to factor analyze eight performance scores derived from thebattery. A single factor, consisting of a linear combination of the eight derived performancescores, was determined and used to predict the criterion measure using linear regression. Theauthors recognized that eight subjects (for whom the criterion measure was available) made for arather small sample. Their solution was to insert 3 fictitious data points, one at the extreme lowend of both predictor and criterion score, one at the extreme high end of both scores, and one atthe approximate midpoints of both scores. The fictitious data points placed at the extremes wereplaced not at the extremes of observed scores on the predictor but rather, near the extremes ofpossible scores on the predictor. It is not unreasonable to expect that such manipulation wouldhave a strong effect on the outcome of the regression. A review of the plot of factor scoresversus criterion scores for 11 data points indicates that if the fictitious points were removed theslope of the regression would be similar but the dispersion far greater. The actual utility of thereported adjusted R2 of .86 is questionable.A job analysis intended for RPA future requirements was conducted by the Army ResearchLaboratory Human Research and Engineering Directorate (Barnes, Knapp, Tillman, Walters, &Velicki, 2000) that used the Job Assessment Software System (JASS) to explore manpoweralternatives. The primary issues considered were the need for rated aviators and for imagery andintelligence specialists in RPA crews. This study was not intended to produce any informationregarding selection of personnel for training. JASS was used to elicit relative importance ratingsof two reduced sets of skills and abilities for RPA operators. The reduced skill and ability setswere extracted, presumably by the authors, from the 50 that comprise the full set in the JASSapplication. The 50 attributes in JASS are based on 52 abilities in Fleishman’s Manual forAbility Requirements Scales (Fleishman & Quaintance, 1984). The reduced attribute sets werecognitive skills and abilities judged to be related to flight and navigation tasks for air vehicleoperators (AVO) and to takeoff and landing tasks for EPs. The ratings were taken from a mix of30 soldiers and contractors who were practicing as Hunter AVOs, MPOs and EPs. In addition,ratings o

Oct 19, 2011 · analysis of Air Force tests SRAB does not edit nor revise Contractor . the RPA during launch and recovery operations by visual reference to the RPA. USMC 7413 Mission Package Operator (MPO) USA 96U . MQ-1 Predator and the MQ-9 Reaper remote