jump to navigation

Using Prosper and Mechanical Turk to figure out if people who are shifty look shifty March 19, 2009

Posted by jeremyliew in user generated content, web 2.0.

The Economist has another fascinating article about face. Specifically, about physiognomy – the idea that the way you look is a reflection of your character.

In particular, it describes research done by at Rice University to see if people could identify people who were bad credit risks by the way they look. They looked at 6,821 loan applications on Prosper. They asked 25 Mechanical Turk workers to assess each of the potential borrowers’s likelihood to repay a $100 loan. Here is what they found:

Their first finding was that the assessments of trustworthiness, and of likelihood to repay a loan, that were made by Mechanical Turk workers did indeed correlate with potential borrowers’ credit ratings based on their credit history. That continued to be so when the other variables, from beauty to race to obesity, were controlled for statistically. Shifty physiognomy, it seems, is independent of these things.

That shiftiness was also recognised by those whose money was actually at stake. People flagged as untrustworthy by the Mechanical Turks were less likely than others to be offered a loan at all. To have the same chance of getting one as those deemed most trustworthy they were required to pay an interest rate that was, on average, 1.82 percentage points higher, even when the effects of historical creditworthiness were statistically eliminated.

So it takes two web 2.0 services to tell you that many people who look shifty are indeed shifty.


1. Greg - March 19, 2009

There is a possible alternative explanation. It is possible that the fact that someone looks shifty (due only to genetic lottery) get screwed financially throughout life and as a result are more likely to be unable to pay back loans. So maybe looking shifty makes one shifty.

Not saying it is likely though, just a fun idea.

2. scott - March 19, 2009

Lol — does that apply to finding shifty venture capitalists, too? I’ve met with ones that have looked worse than some of the kiva.org pics O.o

3. Jeff Barr - March 19, 2009

The research was actually done at the University of Washington. There are some other clarifications at http://www.econsteve.com/?p=166 .

4. Anon - March 19, 2009

Correlation does not imply causation.

5. AnotherAnon - March 28, 2009

I’ve spent a bit of time with the Prosper data set, as of late 2007 and early 2008 (when they were still actively originating loans), in an attempt to build a statistical credit model using logistic regressions. As a prospective lender, I hoped to exploit what I saw as a general mis-pricing of credit risk on Prosper.

From my initial modeling, it appeared that that, in general, people who uploaded any photos were, at the time, far worse credit risks, as were people who wrote at length about their needs for a loan. The working thesis for this positive correlation was that these were “desperation” markers–one is more likely to tug at heartstrings if one really needs the money. People who really need the money might not, on average, be as good at managing their finances as people who don’t find themselves in such dire straights.

This finding may also represent a systematic bias in the credit decisions made by Prosper lenders, of the type that I hoped to exploit. As detailed in Michael Lewis’ book Moneyball, some players “look” like stars, though the stats tell a different story. The statistical models don’t care. Did the researchers relate the attractiveness rating to the actual defaults? From the Economist article, it sounds like they looked at likelihood of receiving a loan and loan interest rate, after controlling for credit rating. But that may be a symptom of a the type of mis-pricing I was hoping to exploit: perhaps those “shifty” individuals weren’t actually less likely to repay their loans than more attractive individuals, and, all other things being equal, I’d rather lend to them.

While it was a fun exercise (and taught me both how to parse extremely large XML files in a memory efficient way and to love Excel 2007), it ended up not being worth investing in, especially after Prosper’s sudden demise.

J. - July 7, 2009

Demise? Please elaborate… as far as I know they shut down loans while looking for sec regulatory compliance, and are running a “university” on the side…

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: