Split wrappers:
single input, multiple outputs
See IoTPy/IoTPy/agent_types/merge.py
split encapsulators read a single input stream and produce multiple output streams. The split functions are the inverse of corresponding merge functions; for example unzip is the inverse of zip, and unmix is the inverse of mix. (Note: unmix is also called separate because unmix is not in the dictionary.) When a split encapsulator encapsulates a function then typically, the encapsulated function reads a single element of the single input stream and returns a list where the j-th element of the list is appended to the j-th output stream.
UNZIP
unzip(in_stream, out_streams)
where in_stream is a single stream and out_streams is a list of streams. Each element of in_stream must be a list (or tuple) of length equal to the number of streams in out_streams. Each element of in_stream is a list, and the j-th element of this list is appended to the j-th out_stream. So:
out_streams[j][n] = in_stream[n][j] for j = 0, 1, ...where out_streams[j][n] is the n-th element of stream out_streams[j] and where in_stream[n] is the n-th element of in_stream, and in_stream[n][j] is the j-th member of in_stream[n].
unzip is the inverse of zip_stream:
zip_stream([x,y], z)
unzip(z, [u, v])makes u, v the same as x, y.
Example
unzip(z,[x,y])If the elements of z are [0, "A"], [10, "B"], [20, "C"], [30, "D"], .... then the elements of x are 0, 10, 20, 30, ... and those of y are "A", "B", "C", "D", ...
Example illustrating padding of the input stream
unzip(s,[u, v, w])If the elements of s are [0], [1, 10, 100, 1000], [2, 20, 200], [3, 30]... then the elements of u are 0, 1, 2, 3, ... the elements of v are None, 10, 20, 30,... and those of w are None, 100, 200, None, ...
TIMED_UNZIP
timed_unzip(in_stream, out_streams)timed_unzip is the inverse of timed_zip and is similar to unzip. For example, with streams x and y, timed_unzip(timed_zip([x, y]), [u,v]) makes stream u equal to x and makes stream v to equal y.
Each element of in_stream should be a tuple (t, v) where is v a list or tuple of length num_out_streams (the number of output streams), and t is a timestamp. The j-th element of v is appended to the j-th out_stream provided that this element is not None.
With z as defined in the example for timed_zip:
[(1, ["A", None]), (5, ["B", "a"]), (7, [None, "b"]), (9, ["C", "c"]), (10, [None, "d"]), ..
and with timed_unzip_f(z, [x, y]), we get:
x is [(1, "A"), (5, "B"), (9, "C"), (12, "D"), ....], and
y is [(5, "a"), (7, "b"), (9, "c"), (10, "d"), ....]
SEPARATE OR UNMIX
separate(in_stream, out_streams)separate is the inverse of mix:
mix([x,y], z)
separate(x, [u, v])makes u, v equal to x, y, respectively.
The elements of in_stream must be pairs of the form (j, v) where j is an integer. When an element (j, v) appears in in_stream the value v is appended to out_streams[j] provided 0 <= j < len(out_streams). If j is outside this range then the element is not placed on any output stream.
Example
separate_f(z,[x,y])If the elements of z are (0, "A"), (0, "B"), (1, 10), (0, "C"), (1, 20), (-1, 1000), (0,"D"), (1, 30), ... , then the elements of x are "A", "B", "C", "D",... and those of y are 10, 20, 30, ...
SPLIT_ELEMENT
split_element(func, in_stream, out_streams, state=None,
**kwargs)
The function, with the specified state and keyword arguments, kwargs, is applied to the elements of the input stream, just as in map_stream. The return value of the function must be a list of length equal to the length of out_streams. The i-th element of the returned list is added to the i-th output stream.
EXAMPLE
def f(v): return [v+10, v*2]
split_element(f,z,[x,y])If the input stream z is [0, 1, 2, 3, ...] then x is [10, 11, 12, 13, ...] and y is [0, 2, 4, 6, ..]
EXAMPLE WITH ADDITIONAL keyword ARGUMENTS
def f(v, addend, multiplier):
return [v+addend, v*multiplier]
split_element(function=f, in_stream=z, [x,y]
addend=10, multiplier=4)
If z is [0, 1, 2, 3, ...] then x is [10, 11, 12, 13, ...] and y is [0, 4, 8, 12, ....].
SPLIT_LIST
split_list(func, in_stream, out_streams, state=None,
name='split_list', **kwargs)split_list is the same as split_element except that func in split_list operates on a list and returns a list of lists whereas func in split_element operates on a single element and returns a single list.
EXAMPLE
def f(lst):
return [v*2 for v in lst], [v*10 for v in lst]
split_list(f, x, [y, z])If the content of stream x is [0, 1, 2, 3, ...] then the contents of y and and z are [0, 2, 4, 6,...] and [0, 10, 20, 30, ...] respectively.
SPLIT_WINDOW
split_window(
func, in_stream, out_streams, window_size, step_size,
state=None, name='split_window', **kwargs)func operates on a window (i.e. a list) of length window_size and returns a list or tuple with one element for each output stream. The window is then moved forward by step_size.
EXAMPLE
def f(window): return max(window), min(window)
split_window(f, x, [y, z], window_size=3, step_size=3)If the content of stream x is [0, 1, 2, 3, 4, 5, 6, 7, 8, ...] then the content of y is [2, 5, 8, ....] and the content of z is [0, 3, 6, ...]. This is because the function f operates on a sequence of windows which are [0, 1, 2], [3, 4, 5], [6, 7, 8], …. with max values [2, 5, 8, ….] and min values [0, 3, 6, …]
CODE REPOSITORY
agent_types/split.py. Tests are in
tests/split_test.py