Tumble upon johl

About

Serendipity wandering online. Wherein snippets from the World Wide Web are thrown in that deal with memetic nonsense, programmatic geekery, political and philosophical thoughts, all of the above, and beyond.

Newer posts are loading.

You are at the newest post.
Click here to check if anything new just came in.

January 15 2010

19:01

stream – Lazily-evaluated, parallelizable pipeline

Streams are iterables with a pipelining mechanism to enable data-flow programming and easy parallelization. The idea is to take the output of a function that turns an iterable into another iterable and plug that as the input of another such function. While you can already do this using function composition, this package provides an elegant notation for it by overloading the >> operator. This approach focuses the programming on processing streams of data, step by step. A pipeline usually starts with a producer, then passes through a number of processors. Multiple streams can be branched and combined. Finally, the output is fed to an accumulator, which can be any function of one iterable argument.