Weekly Roundup – Jan. 10, 2016

This week’s BigData frameworks, tools, and examples roundup!

Example

Chess

A MapReduce job to explore blunders in chess games.

Time Series DCOS Demo

In this demo we take the crime dataset from the City of Chicago, turn it into a streaming data source and process the data in two paths.

 

Frameworks

Oryx

orxy

Oryx 2 is a realization of the lambda architecture built on Apache Spark and Apache Kafka, but with specialization for real-time large scale machine learning.

Pachyderm

Pachyderm is a complete data analytics solution that lets you efficiently store and analyze your data using containers. We offer the scalability and broad functionality of Hadoop, with the ease of use of Docker.

Terrapin

Serving system for Hadoop generated data

Presto

Presto is a distributed SQL query engine for big data.

 

Toolsets

Pinball

Pinball is a scalable workflow manager.

Searchkit

Searchkit is a suite of UI components built in react. The aim is rapidly create beautiful search applications using declarative components, and without being an ElasticSearch expert.

 

Libraries

librdkafka

Apache Kafka C/C++ client library

Fenzo

Fenzo is a scheduler Java library for Apache Mesos frameworks that supports plugins for scheduling optimizations and facilitates cluster autoscaling.

Thunder

Large-scale image and time series analysis with Spark.

Tormenta

Scala extensions for the Storm distributed computation system. Tormenta adds a type-safe wrapper over Storm’s Kafka and Kestrel spouts.

Storehaus

Storehaus is a library that makes it easy to work with asynchronous key value stores. Storehaus is built on top of Twitter’s Future.


You can find a lot more tools, frameworks and libraries at PocketCluster Index. Go check it out! Looking into adding your repo? tweet to @stkim1!

E-mail Subscribtion
Subscribe for upcoming posts!
Join Slack
Join the channel!

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s