Lending Club will end screen-scraping p2p investing

Peer to peer lending has spawned a diverse ecosystem. A growing number of investment advisers, managed funds, and websites have formed that focus exclusively on investing through Lending Club and Prosper. Part of this ecosystem has been the rise of third-party statistics and auto-investing sites. In today’s post I have a prediction to share: third-party auto-investing with Lending Club, as it stands today, is going to be cut off.

The Pain-Points of Investing Through LendingClub.com

This collection of third-party sites has risen up in the past few years to provide a very important service. People like myself, having spent a lot of time on Lending Club’s platform, have come to realize that not all the loans are the same. For instance, the graphic below shows the current number of Lending Club’s C-grade loans available for investment:

Obviously, there are some slight differences within this group of loans. Some are C4 loans with a 15.3% interest rate while others are C5 loans with a 15.6% interest rate. But if we drill down into each loan’s specifics, there are dramatically large differences between them, particularly in the credit history of each borrower.

Savvy investors have used tools like NickelSteamroller to discover which of these differences matter. For example, loans from borrowers who have had zero credit inquiries in the past six months are statistically less likely to default; they are more likely to pay their loans back in full (see this video). As a result, loans with no inquiries are in higher demand than those with one or two inquiries, so many investors do their best to buy loans immediately when Lending Club adds new volume to the platform (6am, 10am, 2pm, & 6pm PST). If they wait until later in the day, the higher-quality loans have already been snatched up.

Furthermore, Lending Club itself has made it a bit of a task to filter their own platform’s available loans. Despite years of investors requesting otherwise, the small section of their website dedicated to filtering their available loans remains overly simple. My favorite example of this problem is on the right: their filter for a borrower’s Earliest Credit Line. There is a dramatic difference between borrowers with five versus nine years of credit history. Loans with nine years are often more valuable, yet Lending Club has kept these borrowers lumped together for as long as I can remember.

As a result, investors have had the chore of examining the credit details of each prospective loan one by one, a lot of work for those who invest regularly throughout their week.

The Launch of Interest Radar, et al.

This pain-point was addressed when third-party tool sites like Rev’s Interest Radar arrived on the scene in 2012. They were quickly celebrated by the wider self-directed lending community for the ease they brought to peer to peer lending (see the Interest Radar forum). Particularly, these tools did two things better than Lending Club’s site:

1. They allowed detailed filtering of the Lending Club platform. No longer were investors constrained to Lending Club’s own loose filtering. Investors could find loans that matched incredibly precise filtering criteria.
2. They allowed auto-investing when new loans were added to the platform. No longer did regular investors keep the tedious habit of setting hourly alarms throughout the day. They could simply save their highly-tuned filters and set these filters to automatically invest in additional loans. Lending Club eventually released their own auto-investing tool (PRIME), but this tool did not invest during the peak times as mentioned above, so the need for a 3rd party tool remained.

Screen-Scraping is Not Best-Practice

Unfortunately, the medium with which these sites operated was by screen-scraping Lending Club’s site. In short, third-party auto-invest tools used a program to simulate each investor actually logging into Lending Club’s website, artificially moving a mouse and clicking Place Order, investing in loans as if the person themselves was logged in.

This method is still being used today, but it is rife with long-term problems. First, it conflicts with Lending Club’s own Terms of Use. See below:

Secondly, this method uses Lending Club’s website in a way it was not designed, so it has the added risk of being error-prone. For example, if Lending Club was to change the layout of their website without notification, these tools could inadvertently invest in loans their users did not want. This danger is further exacerbated by the screen-scraping tools that offer to auto-sell notes. While mispurchased notes could at least be sold at cost, wrongfully sold notes are permanent.

To boot, these tool sites require investors to share their Lending Club username and password. And despite these sites having good security features in place, the real danger remains that these site’s databases could be compromised and their Lending Club accounts then plundered by hackers. Such a scenario, either a large-scale screen-scraping error or a compromised credential database, would be a small disaster for Lending Club and peer to peer lending as a whole. It could erode the public perception of peer to peer lending that has only recently begun to feel secure.

Screen-Scraping Slows Lending Club’s Website

Also, it seems that a large number of investors are using screen-scraping to query the list of available loans from Lending Club, then placing orders for these loans through API. These queries, often thousands per second, put tremendous strain on Lending Club’s system, slowing the investor experience for those trying to invest normally through the website.

As a result, it seems in Lending Club’s best interest to end the practice of screen-scraping and force investors to use their platform in a way that is both safe and consistent. I spoke with Lending Club’s COO Scott Sanborn recently, and he seemed to affirm that screen scraping will eventually be cut off, focusing on the issue of server-load:

“Screen scraping is not allowed under our terms of use, but it is important to understand that we put this policy in place specifically to address activity that slows the platform and degrades the user experience for other investors, and currently that is our emphasis in enforcing it (i.e. we monitor the platform, and if we see something happening that has an impact on the platform’s performance, we take action).”

Why We Love Third-Party Tool Sites

Even though I have never used his service, I actually have a lot of respect for people like Rev at Interest Radar. Having personally invested manually at Lending Club for my first year or more, I can attest that filtering their loans one by one is a ton of work. And setting hourly alarms to get access to the best loans feels a bit ridiculous. Wealthy institutions may have the money to set up custom auto-investing systems, but us regular folks have only talented indie-programmers like Rev to turn to.

I am grateful for the beer-stained coders who stepped into the gap and worked long hours to provide the services that self-directed investors needed. And it is heavy news that the current functionality of their auto-invest tools needs to be severed if Lending Club wants to enter its future with security and stability. But screen scraping has to go. As an advocate of peer to peer lending as a whole, I feel that an en masse screen-scraping error or a compromised credential database would be a terrible headline for everyone involved, not just Lending Club.

A Functional API Mostly Helps Institutional Investors

But here is the tension: if it is cut off, the only approved way to auto-invest in loans has been to use their API. For those who are unaware, investing via Lending Club’s API means communicating with their platform in pure code. It is an incredibly stable and secure way to invest by sending strings of console text to their server. Unfortunately, there are some major problems with forcing third-party tools to use the API:

1. The current API is not fully functional. For instance, there is no API command to query Lending Club for your available cash balance. As a result, investors have to track how much cash they have available in some less-consistent way. Some API investors get around this by screen-scraping their account’s cash balance before then placing orders through the API.
2. A fully functional API could kill small-dollar retail investing. Despite finally providing sites like Interest Radar with approved automation, a polished Lending Club API would mostly be a boon for big-dollar hedge funds and other financial institutions. The best C-grade loans, as mentioned above, would go to whoever had the fastest server – the technical elite, if you will. In short, a wide-open API would mean better loan access for rich investors.

Self-Directed Investors Between a Rock and a Hard Place

This leaves us with a tough situation today. As screen-scraping is cut off, the API promised to replace it is not totally functional, and when it does eventually gain full functionality, it may actually do more harm than good since the best loans may simply go to those with the fastest servers.

Dear Interest Radar & Friends

A new era has arrived, and screen scraping is getting left behind. Any service out there needs to be ready for the hammer to fall, because peer to peer lending has entered the big leagues (Economist). As Lending Club prepares for this year’s IPO, you can bet that they will begin to tie up any loose ends that have been put on the backburner. My wager is that Lending Club will end screen-scraping by the year’s end.

Dear Lending Club

We love peer to peer lending. Not only do we enjoy the justice of how it cuts out the banks, but also for the way its medium democratizes third-party services. Like you, we believe innovation happens on the small-scale. We have tasted the first-fruits of a self-directed investor ecosystem, want it to continue, and want to invite you to guide its best practices.

Mr. Laplanche, what about an API that keeps the playing field level while also promoting a creative ecosystem of third-party tools?

[image credit: Steven Depolo “Harmony Brewing Glass of Beer”
Josep Ma. Rosell “Cut my breath”
CC-BY 2.0]