Applying Bayesian Hierarchical Probit Model to Interview Grade Evaluation

Abstract

Job interviews are a fundamental activity for most corporations to acquire potential candidates, and for job seekers to get well-rewarded and fulfilling career opportunities. In many cases, interviews are conducted in multiple processes such as telephone interviews and several face-to-face interviews. At each stage, candidates are evaluated in various aspects. Among them, grade evaluation, such as a rating on a 1-4 scale, might be used as a reasonable method to evaluate candidates. However, because each evaluation is based on a subjective judgment of interviewers, the aggregated evaluations can be biased because the difference in toughness of interviewers is not examined. Additionally, it is noteworthy that the toughness of interviewers might vary depending on the interview round. As described herein, we propose an analytical framework of simultaneous estimation for both the true potential of candidates and toughness of interviewers' judgment considering job interview rounds, with algorithms to extract unseen knowledge of the true potential of candidates and toughness of interviewers as latent variables through analyzing grade data of job interviews. We apply a Bayesian Hierarchical Ordered Probit Model to the grade data from HRMOS, a cloud-based Applicant Tracking System (ATS) operated by BizReach, Inc., an IT start-up particularly addressing human-resource needs in Japan. Our model successfully quantifies the candidate potential and the interviewers' toughness. An interpretation and applications of the model are given along with a discussion of its place within hiring processes in real-world settings. The parameters are estimated by Markov Chain Monte Carlo (MCMC). A discussion of uncertainty, which is given by the posterior distribution of the parameters, is also provided along with the analysis.