Project Report: Movie Hoard

Movie Hoard is another old one, this one dating from June of 2006. It started out as the Streamload project and then was later renamed Syndicator. Streamload was a web-storage service that offered great rates on large datasets, so we got together and wrote a program to let us use it as an off-site source for all our media files. We wanted to keep things legal, though, while still having easy access, so we needed to encrypt everything we uploaded to keep the RIAA and MPAA off our asses, as it were. Once the files were up on the Streamload site pretty much anybody could download them, but we were the only ones with the decryption key necessary to turn them back into media files.

That probably sounds like a lot of hassle to watch a backup of The Waterboy while you're on travel for work, and it is. To clean things up, we wrote a massive program that would handle everything -- it tracked a database of everything we had uploaded, including extra information such as Artist and Album for songs, or a list of Actors and a Plot Synopsis for TV shows and movies, it scanned our computers for media that needed to be uploaded but wasn't yet, it handled the uploading, encrypting the files before sending, and then handled the downloading, decrypting them as it went. And, of course, we had to build a GUI so we could browse our database, pick the file we wanted, and then download it from Streamload.

Unfortunately, while we were busy buliding the most reliable and useful software project any of us had ever worked on, the Streamload service itself was busy undergoing a total breakdown. We got a good year out of it, but in the end it failed and we all stopped using the program.

Then, a couple months ago, K-- brought up the spectre of the old beast by commenting on what a useful tool the indexer was -- that's the bit that scans our computers, finds media files, and downloads all the extra info from a handful of internet sources like IMDB and TV.com. Even after Streamload died, K-- was still using our program just to index the media files on his own computer, and keep track of where everything was.

We spent an evening discussing it, I think in December, and decided it wouldn't be all that hard to prune out all the uploading and downloading and encryption stuff, and just make it a local media indexer. As part of that discussion (and the email threads that followed it in the next few days), I mentioned that nearly all modern media player software includes some type of indexer and lookup, and I was wondering if we could find one (whether it was mPlayer or Windows Media Player or even iTunes) with a plugin language we liked and learn how to just plug our data into the existing media player.

And then, like a bolt from the blue, I rediscovered XBMC. I went over this in a Journal post a day or two ago, but XBox Media Center was a homebrew application for modded XBoxes that allowed them to play pretty much any media format, and we've all been using it for years. Actually, most of the time we would download files using our Streamload client (back when it worked), then play them over the network on our modded XBoxen.

In the last four months, though, all my XBoxen have died or been stolen. I was busy lamenting that fact one day when I decided to check on the status of an old project to port XBox to Windows. They brought it to Linux first, and then Mac, but lo and behold, these days there's a functional (if not entirely stable) build for Windows. Not only that, but it now supports "Plugins" in addtion to the "Scripts" that I used to write.

Plugins, in this case, are Python script-driven menus within the main XBMC client. What that means is I can use Python to read our Streamload media database, use Python to format all of that into a list of movie files or songs on a local server, whatever, and then provide that list to XBMC to show just like it would a list of files if it were browsing my hard drive. The big benefit there is that I don't have to build a GUI. Building GUIs is ass, plain and simple. In this case, I just format my data however I want, then hand it off to XBMC to display.

While I was busy making that discovery, K-- and I were both trying to figure out how to get the old software working, because it was built on a Python GUI package (PyQT) that was obsolete and unavailable now, and none of us were still running machines with it installed. So, even though pruning out the downloading and encryption stuff was supposed to be a simple project, we had no way to actually show any of our results until we built a whole new GUI (which, as I just told you, is ass).

So the two discoveries merged, and Streamload became the Movie Hoard plugin for XBMC. So I started working through some tutorials to figure out how to format the plugin, and K-- started querying Google to figure out how to access a SQL database in XBMC, and as of last weekend, we have a plugin that will read our old Streamload database, display results based on whatever search parameters we want to use, give us access to all that additional data we have stored, and do it all with the same simplicity of use as browsing a local folder directory in XBMC (something we've all done thousands of times).

We haven't plugged in the last step yet (where pushing "OK" actually launches the file), but that should be a relatively trivial task. We also need to write some extra stuff to handle filters and sorting, but that's mostly a port of code we wrote for the old Python client. Then I have to undergo the major task of actually indexing all my data, because K-- has been doing it for over a year now, but I haven't been running the software at all, so it'll be no small feat to get my library imported into the database.

Still, it's all good. Every roadblock we've encountered so far, we've found an answer to. This, frankly, isn't my absolute top priority, or it would probably already be done. The next month will probably see some major progress, and I'll keep you posted.