By: Jonathan Winkle, Manager of Behavioral Sciences, LenddoEFL
The last post showed how our psychometric content reveals people’s personality traits, but our assessment also captures an abundance of metadata. Metadata is information about how people process the questions and exercises they complete. Here are some examples.
How long did an applicant take to answer a question compared to their average response time?
How many times did an applicant change their mind and switch their response before submitting their answer?
Is the applicant’s information consistent with their written request to the financial institution? (e.g., requested loan amount)
By measuring metadata, LenddoEFL’s approach goes beyond what is possible in traditional credit applications to reveal more information about applicants. Consider the following question from our test:
Which BLUE person is most like you?
For this question, we consider how long it took the applicant to slide to one answer or another and whether they changed their opinions in the middle. Someone who is confident that they are an organized person should move the slider in only one direction and relatively quickly. Quick, smooth answers belie confidence, whereas slow, wavering responses demonstrate uncertainty.
The relationship between response time and default rate can be complex. Consider another psychometric exercise:
In this case response time was a non-linear predictor of default, where both slow and fast response times were associated with a greater credit risk!
There are many ways to interpret response time metadata. If an applicant answers a question quickly, are they confident or are they cheating? If they are taking a long time to respond, are they having difficulty understanding the question or putting extra effort into getting their answer right? By collecting metadata across all questions, we can compare a single response time to the applicant’s overall response time distribution to differentiate things like confidence and cheating (see graph below).
An example distribution of response times generated from artificial data
Conclusion
Metadata reveals another layer of behavior on top of the personality traits we target and can be used to identify features such as confidence, cheating, and confusion. These behavioral traits can be used for predicting default and ensuring that we are collecting high quality data for our models.