• News |  
• Articles |  
• TYDKYDK that AI knows about you

How well can an AI chatbot measure your personality?

A new study on Artificial Intelligence, or AI, led by Psychological Sciences Professor Jinyan Fan finds that an AI chatbot can infer personality traits as well as or better than traditional self-report personality measures. In “How well can an AI chatbot infer personality? Examining psychometric properties of machine inferred personality scores,” published in the “Journal of Applied Psychology,” Fan and researchers from four other universities used an AI firm’s cognitive AI chatbot to infer personality and examine psychometric properties—reliability and validity—of machine-inferred personality scores.  

What did this study find? Why is it important?

The overall conclusion is an AI chatbot can infer personality pretty well based on a series of evidence. For instance, to verify or to validate a psychological assessment, we look closely at the nature of psychometric properties of machine scores. All that evidence suggests yes, the machine score seems to demonstrate reasonably good reliability and validity, based on well-established criteria in psychometrics.

The big takeaway is the traditional assessment has some downsides. For instance, it's easy to fake. If you're applying for jobs and your potential employer asks you to complete a personality test as part of the application process, because your motivation to get a job offer is very high, you try to put the best foot forward. If you do, that damages the validity of traditional assessment. The AI approach is not easy to fake because it’s less transparent than the traditional approach. You don’t know which features are important, so it’s very difficult for lay people to figure that out.

In the AI approach, you just go through a virtual interview. So, we have some confidence it’s more difficult to fake than traditional assessment. It’s a two-way communication back and forth, the chatbot asks you a few questions and may even make small talk and show certain levels of active listening. It feels more like talking to somebody naturally through two-way communication, so it’s a better experience.

What are the real-world implications of these findings?

I can only speculate that a little bit because I'm in the field of human resource management. I study psychology, how to apply knowledge of psychology to the workplace. One such area is human resource management or talent assessment. You can select people based on their machine scores instead of having them going through the traditional assessment. Or, you can have a group of current employees chat with the AI chatbot, figure out their personality scores, which areas they need to improve on, and you can then send them to different training programs. That's one major area of application.

Another application is in marketing. If the chatbot can chat with a client for a few minutes, maybe the algorithm can infer that specific client's personality and then the chatbot can have a better way of dealing with clients’ needs, and that will make the marketing effort more efficient. These are just a couple of examples.

How will this change future research on personality?

I think you will see more and more research focusing on this machine learning-based approach to personality assessment. In other words, my prediction is over time, AI-based personality assessment will dominate the traditional assessment, particularly if we can further establish the psychometric property of the machine scores. In other words, if machine scores can replace self-reported personality scores, then suddenly, the major implication is we don't need traditional assessment anymore. That's a dangerous idea. It's not here yet but moving forward, that could be a real possibility.

What does this study implicate about the future of AI?

It's just another application of AI in a different field. You can see AI is being applied to many, many different fields, even biology, sports games, et cetera. And now it comes to human resource management. I would predict you will see more and more AI in our daily life, and it will replace some of the traditional working methods or working people, but also it will create new opportunities for people who specialize in data science and things like that. So, I can see that coming.

About Jinyan Fan:

Jinyan Fan is a professor in the Department of Psychological Sciences in the College of Liberal Arts at Auburn University. His research expertise includes artificial intelligence, newcomer orientation and socialization, cross-cultural adjustment and training, personnel selection and adult social self-efficacy. Fan’s work has appeared in the “Journal of Applied Psychology,” “Journal of Management,” “Journal of Organizational Behavior” and more.

Tags: Psychological Sciences Research