Upcoming talks and demos:

Jupyter Con - New York 23-25 Aug

View Natalino Busa's profile on LinkedIn

Principal Data Scientist, Director for Data Science, AI, Big Data Technologies. O’Reilly author on distributed computing and machine learning.

Natalino leads the definition, design and implementation of data-driven financial and telecom applications. He has previously served as Enterprise Data Architect at ING in the Netherlands, focusing on fraud prevention/detection, SoC, cybersecurity, customer experience, and core banking processes.

​Prior to that, he had worked as senior researcher at Philips Research Laboratories in the Netherlands, on the topics of system-on-a-chip architectures, distributed computing and compilers. All-round Technology Manager, Product Developer, and Innovator with 15+ years track record in research, development and management of distributed architectures, scalable services and data-driven applications.

Sunday, April 22, 2012

Co-routines vs Sub-routines

from wikipedia:

When subroutines are invoked, execution begins at the beginning and once a subroutine exits, it is finished; an instance of a subroutine only returns once. Coroutines are similar except they can also exit by yielding to, or calling, other coroutines, which allows them to be re-entered at that point again; from the coroutine's point of view, it is not exiting at all but simply calling another coroutine.

Any subroutine can be implemented as a coroutine by simply not using the yielding feature of coroutines.

To implement a programming language with subroutines, only a single stack that can be preallocated at the beginning of program execution is needed. In contrast, coroutines, able to call on other coroutines as peers, are best implemented using continuations.Continuations may require allocation of additional stacks and therefore are more commonly implemented in garbage-collected high-level languages. Coroutine creation can be done cheaply by preallocating stacks or caching previously allocated stacks.

Here is a simple example of how coroutines can be useful. Suppose you have a consumer-producer relationship where one routine creates items and adds them to a queue and another removes items from the queue and uses them. For reasons of efficiency, you want to add and remove several items at once. The code might look like this:

var q := new queue

coroutine produce
while q is not full
create some new items
add the items to q
yield to consume

coroutine consume
while q is not empty
remove some items from q
use the items
yield to produce

The queue is then completely filled or emptied before yielding control to the other coroutine using the yield command. The further coroutines calls are starting right after the yield, in the outer coroutine loop.
Although this example is often used to introduce multithreading, it is not necessary to have two threads to achieve this: the yield statement can be implemented by a branch directly from one routine into the other.