Crowdfunding 201: Kicktraq Predicts the Future

Here at the headquarters of “Crowdfunding 201” (which is really just wherever I’ve set up my laptop on a given day) we spend a lot of time looking at campaigns and crunching numbers for campaigns, so it was a pretty exciting day when someone pointed out the existence of Kicktraq. What started as something of a curiosity has become an essential tool, really the essential tool for digging deeper into a campaign at a glance.

How so? Let’s dig.

Kicktraq is the brainchild of Adam Clark, a self-described “jack of all trades”, who got introduced to Kickstarter via the Cards Against Humanity campaign and, well, got hooked. I reached out to him via email:

“I’m a data nut, so it wasn’t long before I started keeping track of campaigns I pledged on within a spreadsheet,” Clark wrote. “This morphed into trying to see if I could find trending patterns to see if project A would fund as well (and have a chance at over-funding goals) as project B. So, my spreadsheets and charts morphed into a web app where I could input daily project data and play with the data in a more robust way. This continued to evolve until I built a browser extension for Firefox to make automating the process of adding projects easier. I shared it with a couple friends to help me gather some data from projects they were watching, and started building some of the first charts people use today.”

Basically what Kicktraq does is a form of data scraping, where every hour it goes through all the active links on file, pulls updated data, and dumps it into a database. The output looks something like this:

The campaign we’re looking at, by the way, is finishing funds for the documentary Voices and Faces: Cairo. Go, check it out.

That chart is the main view, but it also breaks down like this:

What you’re looking for here is spikes and outliers. Your standard campaign is going to run on something of an inverted bell-curve. It’ll start fast, level off in the middle, and then take off near the end until it hits the goal, and then level off again. But sometimes, things change. Take, for example, the campaign for This Last Lonely Place. It hit 121% of the goal. But, then check it out on Kicktraq.

Clearly, something happened around August 6th. Well, Humphrey Bogart happened. And suddenly, the campaign makes a lot of sense.

Or, you can see the Reddit spike for Fat Kid Rules The World when it suddenly brought in 431 backers on a random day in the middle of the campaign.

It’s insanely valuable for getting a picture of a campaign in process.

But let’s go back to our example. Kicktraq’s trending line has the campaign trending toward $11K. The Projection tab puts the high range at just shy of $14K (these update daily). [Ed. note: The live chart-- embedded here-- has since updated and shows a new projection based on a recent spike. This paragraph was written over the weekend.]

Obviously, if you’re running a campaign (or have a vested interest in a campaign), you’ll see that projection range and get worried. That’s really far from the goal, and it remains to be seen if this campaign has enough juice to get to the goal. It certainly isn’t out of the realm of possibility.

Personally, I don’t put a lot of stock in any of the projection numbers.

The trending line is pretty easy to figure out. It takes the previous X days of the campaign, looks for a trendline, and then extrapolates it out over the course of the campaign. So, if you’re in the trough, the trendline will assume you’re more or less going to stay there. If the campaign is really accelerating, it’ll also assume that’s going to keep happening. Neither is realistic.

In the Kicktraq blog, Adam describes projections as “a sort of windsock for the project”. From what I can tell, the system does best with a campaign that steadily marches on toward the goal. But give it a rally, like Republique, and it’s caught completely off-guard. Really, any sort of rally shocks it.

And, honestly, an informal survey of 5 Kickstarter veterans can get a more accurate projection than Kicktraq’s system. Why is that? Does Kicktraq not have enough data? In the interest of transparency, I’m just going to reprint that portion of my discussion with Adam.

McNelly:You mention in your blog the weights you give different categories for the projections. Can you talk a little bit about that? What goes into figuring out those weights?

Kicktraq: The projection system currently works as a sort of feedback loop. I’ve done a little work with machine learning and genetic algorithms so I thought this sort of problem would work best with a similar approach. I wanted to try and build a feedback system based on topical similarity where every few weeks I feed the latest projects into it and have it see how the last round of weighting worked out overall for a specific category. If not, it churns through and tries to pick a new weighted curve that works for the historical projects as well as the latest round of input.

It’s all a learning process for me as well, so I’m constantly tweaking around with it when I get some free time. It might end up not being the best approach, but with crowdfunding campaigns, there are so many variables and external influences, there might not be any golden solution. Right now, I’m happy with best guess, and it’s seemed to work out well for me so far.

McNelly: Have you given thought to fitting the trendline to a previously established pattern, kind of like how Nate Silver built PECOTA?

Clark: Somewhat, though the more data I seem to gather the more I find that there is no real established pattern that fits overall across campaigns that I can see, and that continues to pose a challenge. As I note in my last round of projection updates, sometimes a single tweet can throw off projections dramatically especially because there is an intrinsic function through any given window to soften any sudden spike, however if that spike continues to accelerate a campaign, there can be a significant lag for the projection to catch up and again to come back down if the spike in pledges is sustained for any length of time.

McNelly: Is there a category that does better or worse in your projections? I know, for example, that a lot of film campaigns pretty reliably rally in the final 72-96 hours, which is something Kicktraq seems to not anticipate (you mention Hybrid Vigor in your blog). Or: is there a type of curve that does better?

Clark: I think some categories like Film are unique in that they tend to strive specifically to meet the goal and then flatten out once it’s met, especially documentaries. A lot of films tend to struggle their entire campaign just to reach their goals, while other categories focused on consumer goods tend to fund earlier and chase overfunding goals.

Again, this is somewhat why I took a category-specific approach, but it still poses a challenge to find a model that fits both of these scenarios well.

Allow me to editorialize a little here (shocking, right?). I think what Kicktraq does in terms of data collection is insanely important. It’s probably the most valuable tool we’ve got at the moment. It saves me more time than you can imagine.

The projections, however, are pretty lacking. And I’m even sure if that’s something that’s the system’s “fault” or owes more to a lack of data being available. Kickstarter does a pretty mediocre job of supplying analytics (although it has gotten better lately). It may just be that we don’t yet know what to look for, or it may be that we don’t yet have a big enough sample size to draw from. But, there should be a way to predict a rally, especially in a category like film where it happens a lot.

Imagine, and I might be dreaming here, a model where you could pick from a couple of different possible tracks and see how that path would affect that campaign. Or imagine we could pull from a variety of other data–data from Twitter and Facebook, for example–to project whether or not a campaign has the juice for a rally. What separates Hybrid Vigor from a campaign that sort of tops out at 20%? I’m not sure we know this yet. But I’m pretty sure there’s a scenario by which we could.

Which is not so much a criticism as it is a hope. Kicktraq is a pretty fantastic tool that’s very much in a stage of infancy. It can only get better.

Extra Credit

The blogger behind My Silent Half is raising funds to chronicle the process of her coming out of the closet.….The documentary Mile…Mile & A Half hit their goal….You know how we keep saying that dollars raised do not indicate the quality of the campaign? Hotel Noir is the latest example.

Lucas McNelly is the filmmaker behind A YEAR WITHOUT RENT, UP COUNTRY, BLANC DE BLANC, and GRAVIDA. He consults on Kickstarter campaigns for a living. He hasn’t lived anywhere in a long time.