Common Assessment Mistakes and How to Avoid Them

People want easy solutions, especially to important and complex problems. So we look for the one exercise that will significantly change our bodies or the miracle diet that will take off that excess weight.  Easy solutions, like “planking” or “South Beach diet”, are fine as long as the problem isn’t too complex.

Very often, executive pre-hire assessment is approached with this understandable desire for simplicity. People are looking for the one test that will identify star employees whose talent will propel the organization to greatness.  This singular focus on finding the one test leads to mistakes that can be avoided.  Here are some of the more common mistakes:

ASSESSMENT MISTAKE #1: Looking for THE one best test – I have heard many clients talk about being a fan of one test or another, “We are a big believer in assessment, everyone has to take the (fill in name of preferred test) before being hired.”  I suppose that might work if everyone was doing the same job.  That is the equivalent of someone saying, “I am a big vitamin C fan, I use it to treat all health issues.” Psychological tests measure specific traits or abilities.  No one test measures all traits and abilities.  Each job requires a unique mix of traits and abilities.  Therefore, no one test is appropriate for every job.

ASSESSMENT MISTAKE #2: Lack of job clarity – If you have three people in a car and they haven’t discussed and agreed upon their intended destination, then you are likely to have at least two disappointed passengers when the car stops.  I once was hired by a client to assess candidates for a Head of Risk role.  The COO wanted someone who could revamp the risk function’s archaic systems and processes.  The CEO wanted someone who could effectively manage relationships with the board and external regulators.  The Chief Investment Officer wanted someone who could consult with portfolio managers and help them manage risk through scenario testing and better reporting.  Three different views requiring three different sets of skills.  This highlights the often overlooked point that assessment starts by defining the job and what knowledge, skills and abilities are critical for the role.

ASSESSMENT MISTAKE #3: Over-weighting some abilities at the expense of others – I have done several projects looking at the position of “quants” or “strats.”  These are the uber-brainy researchers who work for investment firms and look for patterns in data that can lead to returns.  Often these individuals are Ph.D. level scientists with degrees in math, physics or computer science.  The top organizations screen these candidates heavily on quality of past research and intelligence. Strats often fail because of deficiencies in their communication or organizational skills, rather than technical deficiencies, but little or no time is focused on assessing these areas.  Why?  One potential contributing factor is the “similar to me” bias.  Hiring managers look for people with the skills they value in themselves.  This is the reason why so many former division I & II lacrosse players are employed on securities trading floors.  Another potential issue is that individuals are poor judges of skills and abilities they are weaker on.  So a very poor communicator is unlikely to be able to properly gauge the communication demands of roles reporting to him/her and also would unlikely to be able to properly recognize gradations of the skill.

ASSESSMENT MISTAKE #4: More is always better: While a trait or an ability may be important for a given job, it is not always true that more of it is better than less of it.  For example, height is undeniably correlated with basketball performance.  Yet if you always picked the tallest available players, you may overlook Michael Jordan at 6”6 (which is not tall by NBA standards).   That is because any job is an interplay of several different skills and abilities.  On some dimensions, there is a threshold that once you hit, it may cease to be a differentiator.  For example, many general executive jobs require high intelligence, but once you hit a certain level, other abilities become more important to job success.  On other dimensions you have a “cologne” effect.  A certain amount is good but a lot can be really bad.  Aggressiveness and goal orientation are good examples of that effect.  So higher test scores on a single dimension are not always better.

OK let us stop here for now. I am going to write a separate piece on operational mistakes that decrease the effectiveness of the assessment process.

The solution to all four mistakes listed above is to spend time defining the job deliverables, and requisite skills and abilities, prior to choosing your assessment methods. While there is a bit of art to job analysis that comes with experience, here is the general approach:

• Have all stakeholders agree on the top 1-3 job deliverables – Best case is for this to be a discussion where deliverables are expressed in 1-3 bullets that all agree to. Easier said than done in some cases, but there is no use in starting a selection process before you get agreement.
• Based on the deliverables, identify critical skills and knowledge – Note which ones are needed day 1 and which can potentially be developed on the job. For those skills and abilities you are selecting for, gauge the desired proficiency level needed. Using the prior example of the Head of Risk, the candidate needed to have expert level knowledge of financial products and markets, but only needed strong enough communication skills to be credible for the board.

Job analysis has a long history and there are many tools out there that can help.  At the heart of it, job analysis involves taking enough time to agree upon and understand what the individual will need in order to be successful.  The time devoted to this step is the foundation for the assessment process.

Feel free to reach out to me with your thoughts and ideas.

A Four-Dimension Model for Group Leadership Assessment & Due Diligence

When asked what I do for a living, after resisting the urge to adopt an outlandishly exciting persona, I say that I help people and groups.  While I love my individual coaching work, I always get very excited when a client asks for help evaluating and addressing group-level issues.  It is fun and interesting work.  When learning to do this type of work early in my career, I was struck by the complex models and methodology that were used.  What was especially puzzling was that often the actual issues identified and their solutions were straightforward.  Since then, I have developed and used the following simple model for a variety of group assessment and development projects.

The model posits that all groups require leadership in the following four areas:

Missing photo

The model can be used for the following purposes:

Defining a group’s Leadership needs – While every group needs leadership resources in each of these four areas, the required mix may differ depending on function and mission.  For example, a sales organization may have heavy Client Leadership and People Leadership needs but have comparatively less need for Operational Leadership.

Determining effectiveness of Leadership – For each of the areas I assess a group as “strong”, “adequate” or “weak” (I have specific assessment criteria, if you are interested please contact me).   There are two overall themes regarding effectiveness that I see consistently:

• Inadequate resourcing – Groups may not allocate enough leadership resources to a given category. For example, I have seen Investment Banking groups which had many individuals focused on Client Leadership and Thought Leadership but not enough senior leader labor hours allocated to People Leadership tasks, such as like hiring or development.
• Skill and experience deficits – Individuals responsible for given leadership tasks may not have the requisite skills and experience. For example, a former line level manager is put into a COO role without having the needed Operational Leadership experience in technology or process engineering.

Assessing risk levels – I also assign risk ratings to describe the degree of exposure that leadership resources may be inadequate to meet future group needs.  There are two themes I consistently see around risk:

• High levels of key man risk – A concentration of any of the areas of leadership responsibility placed on one individual.
• Lack of identified successors – Unfortunately, this generally goes hand in hand with key man risk, such that the same leaders who have concentrated power are also not focused on developing successors.

Results can be summarized in a score card.

Missing photo

In this example there were two main areas to address:

• Inadequate People Leadership
• Risk stemming from the senior leader owning key relationships

Based on the analysis, the group structure was changed to create a Deputy Head role responsible for improving People Leadership on an ongoing basis. This included creation and implementation of plans to improve recruiting and development.

Additionally, high potential employees were identified, provided with mentors outside of the group and given expanded People and Client Leadership responsibilities (depending on their skill levels in these areas). All key accounts were now required to have director level account managers (in addition to the key account manager). These director level account managers were expected to develop independent relationships at corresponding levels in the client organization.

Additional actions were taken but I hope this gives you a feel for how the model can help to stimulate change.

I hope you find this model useful. It is simple and clients like that. If you do use it, please feel free to reach out and let me know. I would love to get your feedback.

Avi Shatzkes
914 714 0776
avi@adstalentconsulting.com