Transcription
Korn Ferry’sFour DimensionalExecutive AssessmentTechnicalbrief
Korn Ferry’s Four Dimensional Executive AssessmentTechnical brief Korn Ferry 2016. All rights reserved.No part of this work may be copied or transferred to any otherexpression or form without a license from Korn Ferry.For the sake of linguistic simplicity in this product, where themasculine form is used, the feminine form should always beunderstood to be included.www.kornferry.comKorn Ferry’s Four Dimensional Executive AssessmentTechnical briefVersion 15.1a—10/2016
Korn Ferry’sFour DimensionalExecutive AssessmentTechnicalbrief
Korn Ferry’s Four Dimensional Executive Assessment Technical briefTable of contentsKorn Ferry’s Four Dimensional Executive Assessment. 3Person measures. 5Technical features.8Context measures.11Organizational culture .11Role variability.12Predicting outcomes.13Fairness and group differences.17Intended use. 18References. 19Contributors. 20 Korn Ferry 2016. All rights reserved.1
Korn Ferry’s Four Dimensional Executive Assessment Technical briefKorn Ferry’s Four Dimensional Executive AssessmentA context driven assessmentThe process of fitting a candidate to a leadership role according to the results of assessments canbe highly complex. On the one hand, much research suggests that certain person characteristics doseem to have a non-context and non-role-specific effect on job performance. For example, individualswith higher general cognitive ability (e.g., IQ) tend to perform better in most professional vocationswith few exceptions. Yet a one-size-fits-all approach to the predictive utility of many potentially usefulpsychological constructs is likely to lead human resources professionals and decision makers astray ina considerable number of cases.We know that no job is the same. The nature of job roles, organizational contexts, organizationalculture, and other variables are all likely to moderate the desirability of a given response profile on anysingle measure or group of measures. For example, highly successful individuals in vocations requiringa high degree of expert orientation often have and likely require quite different social behavior andproblem-solving tendencies compared to highly successful individuals in people management and/or executive strategy and decision-making vocations. Individuals who are well-adjusted sociallyand emotionally tend to perform better in most jobs, but the impact and importance of emotionalintelligence on job performance and related outcomes is more salient for some job roles, such as thosethat require a greater degree of effectively motivating and leveraging the skills and abilities of others.Interactions between person characteristics and context are often critical. A specific profile in a givenjob role can be desirable in one industry, company type, company size, or organizational culture, butmuch less desirable or undesirable in others. In short, some measures yield a single desirable scoreor score profile that can be expected to predict success or indicate potential for success for nearlyall respondents across roles and contexts, while the desirability of scores or score profiles on othermeasures are subject to the specifics of job and context. Korn Ferry 2016. All rights reserved.3
Korn Ferry’s Four Dimensional Executive Assessment Technical briefKorn Ferry’s Four Dimensional Executive Assessment (KF4D-Exec) uses organizational and roleContext Measures provided by the hiring organization along with Korn Ferry’s extensive database todescribe a Unique Client Profile (UCP) for a job that will optimize the prediction of Job Performanceby the KF4D Person Measures. Figure 1 below illustrates the process.Figure 1. KF4D-Exec Conceptual FrameworkOrganizational and Role Context MeasuresJob factors, Key competencies, Culture,Function, Industry, LevelUNIQUECLIENTPROFILEWork engagementOrganizational commitmentCompetencies,Job performanceTraits, DriversCareer successKF4DPERSONMEASURESFITJOBOUTCOMESAs we increase the fit of the KF4D person measures to the UCP for a job, we increase desirable joboutcomes. Whether higher KF4D person measures scores are desirable in every search situation orwhether they depend on context measures is a reflection of the way in which the person measuresimpact is moderated by organizational and role context measures—and the nature and size of themoderation can vary. In some cases, elevated scores on a measure might always be predictive ofincreased (or decreased) success, but the magnitude of its predictive coefficient(s) might vary acrossjob roles and organizational contexts. Here, we have moderated magnitude, which can indicate thedegree of salience for a context. On the other hand, elevated levels on some variables may sometimesbe positively associated with desired outcomes and other times negatively associated. Here, we havemoderated sign, which will indicate whether an elevated score is desirable or undesirable. Clearly,magnitude and sign moderation are not mutually exclusive, although elevated levels of variableshaving only the former will help forecast success in all or most cases, regardless of contexts or thenature of job roles. Identifying not only specific moderator variables but also gauging whether, how,and the degree to which a variable’s impact is moderated has much potential to optimize prediction ofperformance for each job.In the following sections of this brief, we begin by describing our KF4D person measures, whichinclude personality measures (Traits), skill and behavioral measures (Competencies), and motives/values measures (Drivers). We then briefly discuss the nature of executive job roles and organizationalcontexts, with particular attention to identifying key organizational and role context measures in theareas which interact with and moderate the desirability of specific profiles in a way that optimizescandidates’ fit for particular roles.4 Korn Ferry 2016. All rights reserved.
Korn Ferry’s Four Dimensional Executive Assessment Technical briefPerson measuresTraitsTraits are personality characteristics that exert a notable influence on behavior. Examples includeoptimism, introversion, and other natural leanings, such as social astuteness. In organizationalpsychology, traits may be more or less crucial for success depending on job roles and contexts. InKF4D-Exec, we measure 14 traits and 3 higher-order trait factors, which are described in Table TRDEF.Table TRDEF. KF4D-Exec traits, higher-order factors, and definitionsFACTORTRAITAgilityDEFINITIONCapacity for complexity, novelty, adaptability, cognitive flexibility, risk,ambiguity, and change; tendency to eschew dogmatism and rigidity;high value on learning and growing from experiences, including failures.Risk-takingA willingness to make decisions based on limited information or to takea stand.AdaptabilityComfort with unanticipated changes of direction or approach.Tolerance of ambiguityComfort with uncertain, vague, or contradictory information thatprevents a clear understanding or direction.CuriosityThe extent to which people are likely to tackle problems in a novel way,see patterns in complex information, and pursue deep understanding.FocusPreference for organization, procedure, and exactitude.Social leadershipCapacity for composure, self-awareness, empathy, affiliation, sociability,and relating socially in ways that motivate and facilitate the success ofothers in terms of work-related activities.ComposureThe ability to stay calm and poised in stressful, difficult, or ambiguoussituations.Situational self-awarenessMaintaining broad, receptive, and non-judgmental attention to presentexperience.SociabilityThe degree to which people enjoy interacting with others.InfluenceThe degree to which people motivate and persuade others.EmpathyBeing attuned to others’ feelings, motivations, and concerns.AffiliationThe degree to which people are inclined to align with a larger team ororganization toward a common goal.EnergyCapacity for tenacity, achievement orientation, and leadershipascendance despite obstacles and difficulties.Need for achievementA tendency to work intensely to achieve and exceed difficult standards.PersistenceA tendency toward passionate and steadfast pursuit of personallyvalued long-term or lifetime goals, despite obstacles, discouragement,or distraction.AssertivenessThe degree to which people are inclined to assume wide responsibility,take charge, and lead others. Korn Ferry 2016. All rights reserved.5
Korn Ferry’s Four Dimensional Executive Assessment Technical briefDriversWork motivation has been a central focus of organizational research for many years. This high levelof interest can be attributed to the long-held belief that individual behavior and performance areprimarily influenced by knowledge and skills, motivation, and situational constraints/facilitators.In KF4D-Exec, we assess enduring, distal motivations that can predict and explain an individual’srelatively stable behavioral patterns. These “drivers” are deep internal values, motivations, andaspirations that shape a person’s choices across situations and relatively long time frames. Wemeasure 6 drivers, as shown in Table DRDEF.Table DRDEF. KF4D-Exec driver names and definitions6DRIVERDEFINITIONBalanceThe degree to which individuals are motivated by achieving a balance between work and personallife. High scorers prefer work-related flexibility, broadly defined self-development, and preferto avoid high stress life-defining job roles. Low scorers place career as a top life-priority and aprimary component of identity.CollaborationThe degree to which individuals prefer work-related interdependence, group decision making,group-based goal setting and pursuit. High scorers prefer to be part of teams, build consensus,share responsibility, and rely on social behavior for work-related success. Low scorers prefer workcharacterized by limited reliance on social behavior, independence, and being primarily responsiblefor their own work and decisions.PowerThe degree to which individuals are motivated by work-related status, influence, and the ability tomake an impact on the organization. High scorers seek to climb to higher levels of visibility andresponsibility within an organization and to acquire a high degree of influence. Low scorers aredriven by intrinsic interest in one’s work and prefer to avoid high-visibility and high-influence jobroles.ChallengeThe degree to which individuals are motivated by achievement in the face of tough obstacles.High scorers prefer challenging and competitive work assignments and environments thatoften preclude operating comfortably and in familiar ways. Low scorers prefer non-competitiveenvironments and work that allows them to stick to their strengths.IndependenceThe degree to which an individual prefers independence and an entrepreneurial approach towork activities. High scorers prefer freedom from organizational constraints, setting and pursuingtheir own vision, and value employability more than job security. Low scorers prefer pursuinggroup-defined goals, structured organizations, and prefer to identify strongly with a particularorganization and its collective vision.StructureThe degree to which individuals prefer work-related stability, predictability, and structure. Highscorers seek job security, known problems and solutions, and jobs that more often require depthand specialized knowledge/skill. Low scorers prefer work characterized by meritocracy, breadth,ambiguity, variety, and unpredictability. Korn Ferry 2016. All rights reserved.
Korn Ferry’s Four Dimensional Executive Assessment Technical briefCompetenciesCompetencies are the skills and behaviors required for success (Lombardo & Eichinger, 2009).We conceptualize and design our self-ratings of competencies as measures of self-efficacy for theperformance of competencies. Self-efficacy is among the more widely investigated and well-knowntheoretical constructs derived from social-cognitive psychology and refers to an individual’s state ofmind concerning their capacity to execute upon certain behaviors and/or to attain certain outcomesrelated to specific skills or behaviors. Because competencies are behaviors and skills, they are wellsuited to be conceptualized and measured according to a self-efficacy framework. Table COMDEFshows the 15 competency areas we measure in KF4D-Exec.Table COMDEF. Executive search competency names and es stakeholdersAnticipating and balancing the needs of multiple stakeholders.Cultivates innovationCreating new and better ways for the organization to be successful.Global perspectiveTaking a broad view when approaching issues, using a global lens.Strategic visionSeeing ahead to future possibilities and translating them intobreakthrough strategies.Aligns executionPlanning and prioritizing work to meet commitments aligned withorganizational goals.Ensures accountabilityHolding self and others accountable for meeting commitments.Develops talentDeveloping people to meet both their career goals and theorganization’s goals.Engages and inspiresCreating a climate in which people are motivated to do their best tohelp the organization achieve its objectives.Manages conflictHandling conflict situations effectively, with a minimum of noise.Navigates networksEffectively building formal and informal relationships inside andoutside the organization.PersuadesUsing compelling arguments to gain the support and commitment ofothers.CourageStepping up to address difficult issues, saying what needs to be said.Manages ambiguityOperating effectively, even when things are not certain or the wayforward is not clear.Nimble learningActively learning through experimentation when tackling newproblems, using both successes and failures as learning fodder.Situational adaptabilityAdapting approach and demeanor in real time to match shiftingdemands of different situations.ResultsPeopleSelf Korn Ferry 2016. All rights reserved.7
Korn Ferry’s Four Dimensional Executive Assessment Technical briefTechnical featuresAddressing the problem of faking. Respondents have increasingly become aware that, in manyor most applications, extreme high (or low) scores are desirable. When this knowledge is coupledwith a general understanding of desired responses to particular content, the potential for faking intests using Classical Test Theory and related conventional response formats becomes particularlyproblematic and disconcerting.As an alternative, Brown and Maydeu-Olivares (2011) developed a structured multidimensional forcedchoice response model that addresses problems associated with faking, response bias, and forcedchoice ipsativity. The model rearranges forced-choice responses into a series of exhaustive binarycomparisons, thereby allowing for components of non-ipsative trait measures to drive parameterestimation, scoring, and interpretation of normative person-scores. The model is novel in that itcreates a relative independence among otherwise predictably auto-correlated forced-choice basedconstruct scores. It is flexible in terms of forced-choice block sizes, number of items per construct, andnumber of constructs. We utilize this model in the KF4D-Exec assessment.Administration. Our measures of traits, drivers, and competencies are all administered in forcedchoice response format in order to decrease potential problems associated with faking and responsebias. Each construct type is grouped together in its own test form. Traits are measured with traits,drivers with drivers, and competencies with competencies. Construct scores are estimated using amodification (Zes, Lewis, & Landis, 2015) of the Brown and Maydeu-Olivares (2011) Forced-ChoiceItem Response Theory (FCIRT) model to address ipsativity and arrive at construct estimates whosecorrelations are based on the nature of the constructs and not forced-choice item response formatartifacts.Eight items were designed to tap each trait, and trait response blocks contain four items each. Eachcompetency and each driver are measured using ten items, and response blocks contain seven and sixitems each, respectively.Reliabilities. Reliabilities for each of our person measures can be examined in Tables RCOMP, RTRAIT,and RDRIVE below and show acceptable test reliability for each of our KF4D-Exec measures (r'tt .72in every case). All empirical data collection and analysis was done between 2013 and 2015, as notedin the more comprehensive Korn Ferry’s Four Dimensional Executive Assessment Research Guide andTechnical Manual (Lewis, Goff, Jones, & Hezlett et al., 2015).8 Korn Ferry 2016. All rights reserved.
Korn Ferry’s Four Dimensional Executive Assessment Technical briefTable RCOMP. Composite reliabilities for competenciesFACTORCOMPETENCYRELIABILITY ESTIMATEThoughtBalances stakeholders0.87Cultivates innovation0.78Global perspective0.82Strategic vision0.84Ensures accountability0.84Aligns execution0.87Navigates networks0.87Engages and inspires0.85Develops talent0.81Manages conflict0.84Persuades0.84Courage0.88Manages ambiguity0.86Nimble learning0.84Situational adaptability0.77ResultsPeopleSelfNote. N 1001. Reliabilities are average trait range reliabilities from estimated IRT scores. Korn Ferry 2016. All rights reserved.9
Korn Ferry’s Four Dimensional Executive Assessment Technical briefTable RTRAIT. Composite reliabilities for traitsFACTORTRAITRELIABILITY .80Risk-taking0.83Tolerance of 78Influence0.83Situational Need for achievement0.81Persistence0.84Agility0.89Social leadership0.87Energy0.81Social leadershipEnergyHigher-order compositesNote. N 2022. Sub-domain reliabilities are average trait range reliabilities from estimated IRT scores. Composite scorereliabilities are Mosier (1943) reliabilities.Table RDRIVE. Composite reliabilities for driversDRIVERRELIABILITY enge0.85Independence0.83Structure0.75Note. N 2001. Reliabilities are average trait range reliabilities from estimated IRT scores.10 Korn Ferry 2016. All rights reserved.
Korn Ferry’s Four Dimensional Executive Assessment Technical briefContext measuresContext measures are provided by client subject-matter experts as part of the Unique Client Profileprocess. These measures moderate and determine the nature and magnitude of KF4D-Exec personmeasures’ utility and, as such, help to increase the possibility of an optimal match between personsand particular job roles. As previously discussed, context measures are linked to the salience of apredictor across contexts and help determine whether a measure is positively or negatively predictiveof outcomes for a particular leadership role.Organizational cultureIndividual-organization fit is particularly vital at the executive level because senior leaders havegreater potential impact on organizational cultures in terms of affecting culture and/or fitting in agiven culture. Organizational culture—a set of shared values, beliefs, and norms that are reflected inpractices and behaviors—is the “glue” that holds an organization together. Assessing and selectingnew executives who match the ideal culture could become an important step toward successfulorganizational transformation.Our executive search cultural assessment is very brief, administered only to client representativesclose to a specific search engagement. It is based, in part, on Cameron and Quinn’s (2006) competingvalues culture model, which specifies two dimensions—internal vs. external and stable vs. flexible.The two dimensions create four prototypical organizational cultures. In our executive assessment,we largely adopt the Cameron and Quinn (2006) four types, and have client representatives rankrelated descriptions of each from most to least like their own culture or their ideal culture. Each type isdescribed below using our own terms.Regulatory organizations are characterized by a primary need for accountability, efficiency, andadhering to standards. They tend to be improvement and stability oriented, with an emphasis oncreating efficient and reliable systems and processes.Collaborative organizations tend toward a long-term focus on building and maintaining cohesion,community, belonging, and empowerment among its members. They are people oriented andemphasize continuous development and training, particularly among internal members andstakeholders.Competitive organizations tend toward long- and short-term focus on profitability and earnings.They are customer and market oriented and emphasize goal setting, goal achievement, and drivingfor results.Innovative organizations focus on change, expansion, creating the new and different, and marketdisruption. Korn Ferry 2016. All rights reserved.11
Korn Ferry’s Four Dimensional Executive Assessment Technical briefRole variabilityMany upper-level managerial and executive leadership roles are often similar. They typically involvehigh-stakes decision making, a large scope of responsibility, and high-profile accountability forcompany and/or business unit outcomes. Nonetheless, executive roles and role contexts also vary inimportant ways. This variability can sometimes impact the desirability of scores or score profiles onassessments.Traditional job analysis is often atheoretical and can create difficult challenges to generalizabilityacross jobs and organizations. More recent thinking characterizes traditional job analysis asincreasingly obsolete, perhaps especially for upper-level managers and executives. In light of emergingresearch on job analysis and our focus on upper-level management and executive leaders, we chooseto make a distinction between functional/legacy vocational job analyses and what high-profileresearchers have otherwise referred to as trait- or values-based job analysis, asserting that personvariables (competencies, traits, and drivers) are important to the extent that contexts provide cuesand needs that allow for or require their expression. We employ a six-factor model of executive andupper-level manager role variability that is expected to interact with assessment profiles to informperson-role fit. The six work-analysis areas are described below.Strategic vs. Tactical. The former involves long-term goal setting and vision, while the latterinvolves driving execution and carrying out strategic initiatives that have been prescribed.Ambiguity vs. Clarity in goals and solutions. The former is characterized by uncertainty and riskin goals and methods by which goals are achieved. The latter primarily involves executing onknown objectives by using or identifying best-practice processes and ensuring accountability andcompliance.Matrixed/Lateral Influence vs. Top-Down. The former emphasizes influencing without authorityand nurturing wide, self-determined buy-in among organizational members. The latter emphasizesformal decision-making authority and siloed organizational structure that render formalized rankand prescriptive management relatively important and effective.Change Agents vs. Stability Managers. The former oversees and facilitates change, innovation,and adaptation of internal structures and processes. The latter involves a greater emphasison overseeing and facilitating maintenance and ensuring the flow and continuity of existingoperations, organizational processes, and organizational structures.Deep Experts vs. Broad/Fast Learners. Some upper-level managerial roles require individuals whoare depth oriented, high experts in some area(s), and/or relatively technical and/or specialized.Other roles require professionals who are breadth oriented, more focused on fast and widelearning, and likely rely more on people management, talent deployment, and social behavior astools for success.Stable vs. Volatile Objectives. Upper-level managerial roles may primarily involve legacy and/ornarrowly defined goods and services and related processes. Other roles involve identifying andmaintaining poise for effectively reacting to quick-changing and/or novel objectives characterizedby market responsiveness, continually emerging opportunities, and/or change.12 Korn Ferry 2016. All rights reserved.
Korn Ferry’s Four Dimensional Executive Assessment Technical briefPredicting outcomesKF4D-Exec person measures have been validated against a variety of outcome variables, includingorganizational commitment and career success. It also shows expected and significant relationshipswith management level, managerial scope, and work-analysis variables. Work engagement has alsobeen central to our calibration efforts.Work engagement reflects the extent to which professionals are satisfied with and emotionallyinvested in their jobs and whether they will expend discretionary effort for their organizations. Itis often of particular interest to HR professionals and organizational scientists and is known to bemarkedly predictive of both organizational and person-level outcomes including service, sales, quality,retention, profits, shareholder returns, turnover, customer service, productivity, job performance, andothers. Many also link collective worker engagement to industry and even national outcomes. In theexamples below, findings were calibrated according to outcome variables including work engagementand career success, as well as other key correlates like management level and our six work-analysisvariables.Prototypic roles based on context measuresWe emphasize that our context measures yield numeric information and allow for a near-infinitenumber of customized role/context characterizations. Yet, within both the psychological researchliterature and in our own calibration data, certain managerial role types are found repeatedly andhave important implications and utility for demonstrating how roles and contexts moderate therelationships between KF4D-Exec measures and outcome variables.For example, using contextual variables, the commonly seen Architect role (e.g., Tropman & Wooten,2013) is characterized by high ambiguity, volatile objectives, matrixed/lateral influence, high strategicand change orientation, and low depth/expertise in favor of breadth and fast/wide learning orientationto management. The research literature and the patterns in our own data suggest that this type bestrepresents today’s prototypical upper-level or C-level executive job role. The Builder or maintainer roleis also a known and commonly observed archetype and is typically characterized, according to ourwork-analysis variables, by a tactical and maintenance orientation, clarity, deference to top-down rankbased influence, as well as relatively high depth and expertise orientation.Ideal scores vary by roleFigure ST1 shows model-implied typical social leadership sub-domain scores for high and low success/engagement persons in the Architect role. The typical high success/high engagement Architectsemphasize Affiliation and Influence in leadership, while also being notably above C-level averages inboth Empathy and Sociability. The least effective Architects, on the other hand, lead with Composureand Empathy, while typically having scores on all other Social leadership measures that are belowC-level averages, most notably Affiliation and Sociability. In short, the strengths of the most effectiveArchitects are Affiliation (79th percentile) and Influence (73rd percentile) and the strengths of theleast effective are Composure and Empathy. Korn Ferry 2016. All rights reserved.13
Korn Ferry’s Four Dimensional Executive Assessment Technical briefFigure ST1. Model-implied Social leadership sub-domain scores across engagement/successlevels for Architect roles0.85C-Level sureEmpathyInfluenceHigh Success/High EngagementSociabilityS. Self AwareLow Success/Low EngagementIn contrast, the most effective Builders lead with
Korn Ferry’s Four Dimensional Executive Assessment Technical brief Korn Ferry’s Four Dimensional Executive Assessment A context driven assessment The process of fitting a candidate to a leadership role according to the results of assessments can be highly complex. On the one hand, much
