BOINC: five weeks in

I’m five weeks into my BOINC adventure and I wanted to report some results and talk about how it’s all been going. The major breakthrough - as you saw in the last blog - was when I discovered I could use the free trial offers for various cloud server providers to run BOINC projects on. This was huge because it vastly increased the amount of processing power available to me and hence my chances of crunching through a considerable amount of data. The Pi just wasn't cutting it in terms of speed of calculations - which was entirely expected. But I got greedy.

I was able to sign up for a frankly ridiculous 9 (!) cloud server instances:

• 6 from Google (3 year-long trials of an n1-standard server and 3 always free - forever - f1-micro instances.)
• 2 Microsoft Azure instances, both year long trials of B1-small platforms
• 1 Amazon AWS t2.micro instance for a year

Utterly ridiculous. And what's more I have been totally above board with all of these accounts. They all existed already for use for different things and weren't just my own sockpuppet accounts. Is it against the ToS of those providers? For Google the answer was made clear when someone got their account banned for “mining cryptocurrency” - subsequently reversed on appeal when the user explained that he was running BOINC projects. The others? I’ve not seen anything, but considering how compartmentalised these instances are I’d be surprised if it was causing any problems.

My Pi has been going all of this time of course and has been hardly troubling the scoresheet. Since day one on the 6th May it has been plodding along nicely and has done a fair amount of work. I can’t compare it to the cloud servers properly as they have been running for different amounts of time, but it’s pretty slow. It can take days to complete a task that would take mere hours for one of my cloud instances can complete. Those cloud servers, obviously, have been absolutely flying along. They all have different speeds as they differ in specification.

It’s easy to see the point where I got the cloud servers up and running from this graph:

Lessons learned

One aspect to look out for when using cloud servers is the seeming inability for the BOINC client to understand when planning the amount of work to download that it’s using fractions of a CPU (these are shared resources). Sometimes tasks are reported to take much longer when running than they were estimated to take when downloaded. This leads to you hitting the task deadlines for some projects whose work is complex to run. I’ve had to abort huge swathes of jobs when I realised there was no chance they’d complete. I don’t think there’s a penalty other than wasted time but try and keep an eye on it and adjust your future work level preferences accordingly. You can see this below: estimates of 4h for a whole task but reporting 3h left at 80% done.

Another tip: give your instances hostnames! They’ll report to BAM and projects with those hostnames and two of my Google instances had the same default hostname - I forgot to change them. So BAM both sees all 7 hosts connected now but reports two of them as overlapping. My overall stats aren’t affected but for breaking it down it makes it harder.

And of course you’ll need to keep an eye on your billing. By starting single free instances on accounts you should never run out of credits or be charged. But if you’re worried then you can configure some alarms for your cloud server accounts on each provider’s admin pages. Usually that will be a case of “if charge > $0.00 then email the hell out of me”.

A few final things to note about accounts and billing: On Azure: you’ll need to convert your free account to a paid account to continue to use your free credit for the other 11 months of the year. You’ll get reminders for this. With Google they won’t charge you anything over your free year without asking first. On Amazon there seem to be no such guards.

The future...

I did all of the setup of these servers manually. That took a while. I’m planning to make a GitHub repo with an automatic script to help with this process - that project is here for now. I’ve also got some idiotic idea to make a nice project website to show you how to get things going on each provider and try and publicise it. Get as many people as possible to sign up and contribute. I realise this is a lofty goal, but if you’re interested it’s here.

There is the chance that if lots of people sign up then those providers will just change the rules. But I hardly think they will: science crunching looks a lot better than crypt currencies, given the issues that sometimes come with that. So, probably worth a go. If I could make the whole thing a “turnkey” (ugh) solution then it’s be easier but I’m pretty sure there’s no chance to make the process of registering accounts & creating and launching instances automatic in any way.