Raspberry PI 2 Cluster Case pt1

This is the first post of a series about Raspberry PI 2 BigData cluster case.

  1. Raspberry PI 2 Cluster Case pt1
  2. Raspberry PI 2 Cluster Case pt2
  3. Raspberry PI 2 Cluster Assembly Tutorial
  4. Build Hadoop Cluster with 5 clicks

Once we have bunch of Raspberry PI 2s (RPI2 from now on) and an Apache Spark/Hadoop image file, there is one subtle issue we need to take care of; the case for cluster.

RPI2 Spark cluster running
RPI2 Spark cluster running

Look at my cluster setup. White & black cables tangled up all around. It takes up a lot of space to lay down. Even if RPI2 cluster runs BigData software very decently, I cannot easily recommend you put this mess to on your desk. Something’s got to be done.

First step I took was to test if my current cases were somehow stackable.

Plastic RPI 2 Case
Plastic RPI 2 Cases

It is doable without much hassle, but, as you can see, it is an effective cage to harness the generated heat. If I am to run a cluster in this case for an extended period, chip soldering joints might go wrong permanently damaging a RPI2, not to mention very unstable operating system.

RPI case #2
RPI case #2

So, I bought a second case. It had a unique capacity to keep the lid open to evaporate the heat. Sadly, I could not pile them up.

No way to pile up
No way to pile up

What a mess. 😦  I have to clarify that it is none of case makers’ fault. Raspberry PI is designed as an educational platform with the ease of replacement. You pick *ONE* for your kids to play with. Something goes wrong? Buy another. That’s what 35 USD price tag and credit-card-sized form factor represent from the first place. It’s no wonder all the cases out there are designed for single Raspberry PI.

Then I found this beauty. It was from modmypi.com and looked great. It is very easy to make RPI2 stack to save space, and opened at all sides to let go of heat.

Stackable RPI case (ModMyPi.com)

Just before purchasing the stacker in mass, however, I imagined stacking up six RPI2s in one direction, and wondered how stable that could be. On top of that, if the stacker was that simple, I thought there should be a way to improve it for my particular situation.

Further, I used a USB hub to power up the cluster, but it created more problems than it solved; most USB hubs are underpowered (as low as total 12.5W) for number of RPI2s, it is hard to arrange cables in clean manner. I needed something compact and powerful.

A couple hours of Amazon search found me these babies.

Photive 6 port USB charger
Anker 6-Port USB Charger

They output 50~60W of 5V power, and all have 6 ports. Price ranges from 25 to 35 USD. They are 4 inches (~10 cm) in height, 2.8 inches (~7cm) in width, and 1 inch (~2.5cm) in thickness. They weight 7 ounces (~200g) in average. I just cannot ask more.

I am going to draw up stackable plates, and mill them in CNC for prototyping. We’ll see how things go next time.

E-mail Subscribtion
Subscribe for upcoming posts!
Join Slack
Join my channel!
We are a participant in the Amazon Services LLC Associates Program, an affiliate advertising program designed to provide a means for us to earn fees by linking to Amazon.com and affiliated sites.

 

Apache Spark 1.4.0 on Raspberry PI 2 Cluster

This is the second post of a series about Raspberry PI 2 BigData Cluster for OSX.

  1. Apache Spark on Raspberry Pi 2
  2. Apache Spark 1.4.0 on Raspberry PI 2 Cluster
  3. One Step Spark/Hadoop Installer for OSX v0.1.0
  4. Build Hadoop Cluster with 5 clicks

 

In the previous post, I’ve shown you a RPI2 cluster with Apache Spark 1.1.1 that has run for three months. Since Apache Spark 1.4.0 is out a few days ago, I’ve just upgraded the cluster.

Apache Spark 1.4.0 comes with SparkR finally. R has such a strong position in DataScience field that it is no surprise R and Spark merge into one. Among the many benefits this integration brings, DataFrame, the primary data structure for data processing in R, is ranked on the top.  This is such a great news that one can expect higher level of R algorithms eventually appear in SparkR. You can read more technical detail in AMPlab’s post

There are also other features and improvements coming in together such as early result of project Tungsten, prettier job monitoring, and numerous bug fixes.

Spark Web Console
Spark 1.4.0

Since it takes time to collect all the bits from various places, I’ve compiled an RPI image for you below. Also, this image comes with extra goodies; Numpy and Scikit-Learn. The two giant pillars in Python DataScience land, and they usually take a few hours each to compile into RPI2. Here, all compiled and cleanly installed.

Following is the summary of installed items

  1. Scala 2.11.6
  2. Hadoop 2.6.0
  3. Spark 1.4.0
  4. Numpy 1.9.2
  5. Scipy 0.15.1
  6. Scikit-Learn 0.16

Download RPI2 node image : 2015-06-21-rpi-spark140.img.7z

*  [2015-11-08] A new raspberry image will be uploaded. old image is removed for now.

 

E-mail Subscribtion
Subscribe for upcoming posts!
Join Slack
Join my channel!