Cupids Health

What’s All the Noise About?


The topic of ‘bias’ in decision making has received a lot of attention in recent years. Bias can manifest in various forms, including confirmation bias, affinity bias and loss aversion bias. In contrast to noise, decisions heavily influenced by bias can appear highly consistent, and invite causal explanations as a result. The goal of Kahneman et al. (2021) is to draw attention to the distinct and equally potent issues arising from noise; the implications of failing to do this are immense.

Noise, argue the authors, is everywhere. It plagues professions including criminal justice, insurance underwriting, forensic science, futurism, medicine, and human resources. Kahneman et al. (2021) document in detail the large and growing body of literature demonstrating how people working side-by-side in the same job can make widely varied judgements about similar cases that often prove costly, life changing, and in some industries, fatal.

Minimising ‘noise’ has been a key objective in the application of psychometrics. The challenges in evaluating individual differences in abilities, competencies and personalities and their significance for personal, team and organisational success are considerable. Subjectivity, inconsistency, and bias are major contributors to the noise in personnel decision making. Psychometric assessment’s aim for consistency in process, methodology and measurement criteria parallel the aim of ‘noise’ reduction. In the context of psychometrics, the parallel term is ‘reliability’. “An indicator of the consistency which a test or procedure provides. It is possible to quantify reliability to indicate the extent to which a measure is free from error” Arnold et al. (2020).

Noise and bias are inevitable, but efforts should still be made to reduce these distortions by maximising reliability and validity. There are several sub-categories of reliability that psychometrics must consider, including test retest, internal consistency, and split half reliability. The more ‘noisy’ a test is, the greater amount of unwanted variance its results will generate. An example of this in test-retest terms would be the desire to minimise differences in results between two test administrations for the same participant. A more thorough breakdown of these reliability sub-categories can be found in a PCL blog piece here.

 

The most relevant area to our psychometric assessment industry that Kahneman et al. (2021) focus on is selection. Unsurprisingly, unstructured interviews come in for heavy criticism. Using competency-based interviews with multiple assessors can significantly boost quality, but a degree of noise and bias will always be present. A meta-analysis of employment interview reliability found an average inter-assessor correlation of .74, suggesting that even when two interviewers observe the same two candidates, they will still disagree about who was stronger about one-quarter of the time (Huffcutt et al., 2013).

One specific example resulted from a candidate who was asked in two separate interviews by two separate assessors why he had left his short-lived former CFO role. His response was that he had a “strategic disagreement with the CEO”. When the two assessors met to compare notes, the first had perceived the candidate’s decision as a sign of integrity and courage, yet the second had construed the response as a sign of inflexibility and potential immaturity. Even when stimuli are consistent, prior attitudes influence our interpretation of facts.

Kahneman et al. (2021) also discuss how prior attitudes can influence whether salient facts emerge to begin with. Research has indicated the striking impact first impressions have on interviewers’ perceptions of candidates and may dictate whether certain evidence is sought. Even if competencies around team working are identified as predictors of role performance, interviewers may be less inclined to ask tougher questions in pursuit of this information if they perceive the candidate to be cheerful and gregarious in the opening informal exchanges.

These examples illustrate that, whilst structured competency-based interviews improve the accuracy of selection decisions, it is advisable to apply ‘decision hygiene’ by supplementing them with additional methods. Psychometrics are used to improve selection processes, not least because they can reduce noise. If two or more candidates submit the same responses to a set of psychometric items, they are guaranteed to receive identical outputs regardless of the gender, age, or ethnicity of the candidate or the assessor.

 

So far, we have established that selection judgements will be more accurate when several sources of relevant information are considered and aggregated, preferably by multiple assessors. But when considering multiple metrics, how they are aggregated is key.

Kahneman et al. (2021) differentiate between ‘clinical’ and ‘mechanical’ aggregation and demonstrate this distinction by presenting the reader with ratings received by two subsequently hired candidates (see Table 1. below). Two years later, the reader is asked to predict the stronger performer.



Source link

8 thoughts on “What’s All the Noise About?

  1. Thanks in support of sharing such a fastidious thought, article is pleasant, thats why i have read it fully

  2. Write more, thats all I have to say. Literally, it seems as though you relied
    on the video to make your point. You obviously know what youre talking about, why waste your
    intelligence on just posting videos to your weblog when you could be giving us something
    informative to read?

  3. I think the admin of this web page is truly working
    hard in favor of his web site, because here every data is
    quality based stuff.

  4. Have you ever thought about adding a little bit more than just your articles?
    I mean, what you say is valuable and all. But think of if you added some great graphics or
    video clips to give your posts more, “pop”! Your content
    is excellent but with pics and video clips, this blog could certainly be one of the greatest in its
    niche. Excellent blog!

  5. Hello! I know this is kind of off topic but I was wondering which blog platform
    are you using for this site? I’m getting fed up of WordPress
    because I’ve had issues with hackers and I’m looking at alternatives for another platform.
    I would be fantastic if you could point me in the
    direction of a good platform.

    Also visit my web site เล่นหวย

  6. Office.com/setup or Setup.office.com is the website/webpage for the activation of Microsoft Office & Microsoft 365.
    Microsoft Office gives limitless advantages with the best per-user monthly subscription Plan. visit our website for
    activation here https://offficesetup.com/

  7. I’m not that much of a online reader to be
    honst but your blogs really nice, keep it up! I’ll go ahead and bookmark your website tto come back in the
    future.All the best

Leave a Reply

Your email address will not be published. Required fields are marked *

close

Be The First In The Know - Health Alerts

Get new posts by email:
RSS52.1k
Follow by Email77.5k
Twitter