On not hosting everything

I’ve been moving all my photos to Flickr lately. It’s been a long process, one complicated by the fact that it seems silly to run my own WordPress installation (and things like ArchivesWiki) if I’m not going to bother hosting everything myself. Of course, that’s not really very logical, and so I’ve decided that it’s perfectly okay to host photos on Flickr, videos on YouTube, and all the text (and miscellaneous) stuff here on my own server.

rsync.net

I signed up for an rsync.net account a bit over a month ago. They’re a reasonably-priced off-site filesystem provider, seemingly run by people who care about security and doing things normally. By ‘normally’, I mean rsync for starters (oddly enough, given their name) but also the whole gammut of *nix-y ways of doing things; one can interact with them with the usual tools. So they provide a proper, old-fashioned filesystem, and protect it well (there’s even a warrent canary). There’s a choice of datacentre — I chose the Zurich one — and plans ranging from 7GB (80c/GB/month) to 10TB (8c/GB/month). They even correspond via email, of all things! It really is odd that a company that behaves so normally is so uncommon…. I don’t care about pretty graphics, boring and unused extra features, or ‘enterprise-readiness’ (whatever the flip that is), I just want a share of some disk in a big strong building somewhere, one that’s going to be protected and maintained properly and simply. All I can say so far is three cheers for rsync.net. (I’ll be sure to report back if my opinion changes.)

So that’s all well and good, and I’ve got my big disk in the sky, but how am I going to use it? I am going to host a Subversion repository there, to serve as an everything-bucket. That is all. How well will svn handle a huge (multi-gigabyte) repository? I’ve heard varying reports, but most seem to think it’ll be fine. Certainly it’s data-copying system will work well, as far as resuming aborted connections goes (it’ll only copy what’s not yet been copied; much as rsync.net does (although I don’t think it does it at any smaller unit than that of the whole file)). Questions remain about how much overhead diskspace I’ll waste by doing this, but as most of the binary files will only be modified at most once or twice, and generally not at all once they’re checked-in, I don’t think it’ll matter too much.

I’ll see how things go.

Brewster Kahle on the Internet Archive

We offered unlimited storage, unlimited bandwidth, for ever, for free — to anybody who has something to share that belongs in a library.

Brewster Kahle, Entertainment Gathering Conference 2007 (republished as a TED Talk). The above quote is at 14:19.

The crux of it is of course “something that belongs in a library”. If one has something that could conceivably be held in a library, then there should be a library in which it can be held; the Internet Archive is one possibility.