Upcoming talks and demos:

Jupyter Con - New York 23-25 Aug

View Natalino Busa's profile on LinkedIn

Principal Data Scientist, Director for Data Science, AI, Big Data Technologies. O’Reilly author on distributed computing and machine learning.

Natalino leads the definition, design and implementation of data-driven financial and telecom applications. He has previously served as Enterprise Data Architect at ING in the Netherlands, focusing on fraud prevention/detection, SoC, cybersecurity, customer experience, and core banking processes.

​Prior to that, he had worked as senior researcher at Philips Research Laboratories in the Netherlands, on the topics of system-on-a-chip architectures, distributed computing and compilers. All-round Technology Manager, Product Developer, and Innovator with 15+ years track record in research, development and management of distributed architectures, scalable services and data-driven applications.

Monday, February 10, 2014

Preparing breakfast with scala, akka, and spray (part 1)

How to prepare breakfast in a reactive, asynchronous way in scala


Preparing breakfast is an excellent exercise of practical real-life programming. Breakfast making has some interesting properties.

1. Can be (partially) parallelized, for instance by cooking eggs and toasting bread at the same time.
2. It's constrained by the available resources, think of preparing two breakfasts with a single pan.
3. A breakfast is a recipe. Which means that it's compositional by nature.
   It should be easy to adjust the recipe in order to serve a different breakfast.
4. A breakfast can go wrong. For instance, your can burn your toast. What about error recovery and termination?

Preparing breakfast allows also to reason about the entities which take part of the process.
Think of the ingredients (eggs, bacon), the cooked artifacts (scrambled eggs, fried bacon), the actors (the cook), and the utensils/functionals (e.g. the pan)

Also since this is a breakfast web API, this example allows you to reason about the concept of http routing, json (un-)marshalling, performance, monitoring. etc.
Enough motivations, let's prepare some breakfast!

The project is on github: https://github.com/natalinobusa/breakfast


1. It's an API call
2. Inputs:
nr. of eggs, bacon stripes, toast slices, coffee mugs, and glasses of juices
3. Output:
Json of the assemble breakfast with main, side dishes and drinks
4. Timeout:
1 second to complete the breakfast.
Partially prepared breakfasts are not delivered.
5. Composable
Easy to maintain and modify the recipe.
6. Reactive and lazy
Make use of the resources as soon as possible, and only when necessary.
7. Scalable (In, Up, and Out)
Providing more resources, should (linearly) scale the system (increase throuput)
Providing more resources, should parallelize the breakfast making algorithm (descrease latency)

Reference Request/Response



First implementation

The first implementation consists of just four files (less than 100 lines of code in total!). Two files define the objects of the breakfast (Elements.scala and JsonImplicit.scala), one defines the API and the last define the main http listener service.

The elements are described in two files:
The first snippet is the classes definitions code. By using case classes in facts you have dramatically reduce the definition of your json compared to thepojo way. Since print methods setters and getters are generated as sugar syntax and don't pollute the code. Understanding how a breakfast is build up from the basic components is now a very ease task.

Scala favours decoration rather than inheritance. This is a powerful concept when you are developing functions which are auxiliary to the original object. By using implicits you can extend the class functionality without interfering with the original definition of the class/object.

The service route:
The following code shows how compact and powerful a json http api can be described in scala and spray

The main is quit straightforward, the akka actor system is defined. The service is instantiated and thereafter the service is attached to the IO http listener.