Tutorial

The lazyarray module contains a single class, larray.

>>> from lazyarray import larray

Creating a lazy array

Lazy arrays may be created from single numbers, from sequences (lists, NumPy arrays), from iterators, from generators, or from a certain class of functions. Here are some examples:

>>> from_number = larray(20.0)
>>> from_list = larray([0, 1, 1, 2, 3, 5, 8])
>>> import numpy as np
>>> from_array = larray(np.arange(6).reshape((2, 3)))
>>> from_iter = larray(iter(range(8)))
>>> from_gen = larray((x**2 + 2*x + 3 for x in range(5)))

To create a lazy array from a function or other callable, the function must accept one or more integers as arguments (depending on the dimensionality of the array) and return a single number.

>>> def f(i, j):
...     return i*np.sin(np.pi*j/100)
>>> from_func = larray(f)

Specifying array shape

Where the larray is created from something that does not already have a known shape (i.e. from something that is not a list or array), it is possible to specify the shape of the array at the time of construction:

>>> from_func2 = larray(lambda i: 2*i, shape=(6,))
>>> print(from_func2.shape)
(6,)

For sequences, the shape is introspected:

>>> from_list.shape
(7,)
>>> from_array.shape
(2, 3)

Otherwise, the shape attribute is set to None, and must be set later before the array can be evaluated.

>>> print(from_number.shape)
None
>>> print(from_iter.shape)
None
>>> print(from_gen.shape)
None
>>> print(from_func.shape)
None

Evaluating a lazy array

The simplest way to evaluate a lazy array is with the evaluate() method, which returns a NumPy array:

>>> from_list.evaluate()
array([0, 1, 1, 2, 3, 5, 8])
>>> from_array.evaluate()
array([[0, 1, 2],
       [3, 4, 5]])
>>> from_number.evaluate()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/andrew/dev/lazyarray/lazyarray.py", line 35, in wrapped_meth
    raise ValueError("Shape of larray not specified")
ValueError: Shape of larray not specified
>>> from_number.shape = (2, 2)
>>> from_number.evaluate()
array([[ 20.,  20.],
       [ 20.,  20.]])

Note that an larray can only be evaluated once its shape has been defined. Note also that a lazy array created from a single number evaluates to a homogeneous array containing that number. To obtain just the value, use the simplify argument:

>>> from_number.evaluate(simplify=True)
20.0

Evaluating a lazy array created from an iterator or generator fills the array in row-first order. The number of values generated by the iterator must fit within the array shape:

>>> from_iter.shape = (2, 4)
>>> from_iter.evaluate()
array([[ 0.,  1.,  2.,  3.],
       [ 4.,  5.,  6.,  7.]])
>>> from_gen.shape = (5,)
>>> from_gen.evaluate()
array([  3.,   6.,  11.,  18.,  27.])

If it doesn’t, an Exception is raised:

>>> from_iter.shape = (7,)
>>> from_iter.evaluate()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
    from_iter.evaluate()
  File "/Users/andrew/dev/lazyarray/lazyarray.py", line 36, in wrapped_meth
    return meth(self, *args, **kwargs)
  File "/Users/andrew/dev/lazyarray/lazyarray.py", line 235, in evaluate
    x = x.reshape(self.shape)
ValueError: total size of new array must be unchanged

When evaluating a lazy array created from a callable, the function is called with the indices of each element of the array:

>>> from_func.shape = (3, 4)
>>> from_func.evaluate()
array([[ 0.        ,  0.        ,  0.        ,  0.        ],
       [ 0.        ,  0.03141076,  0.06279052,  0.09410831],
       [ 0.        ,  0.06282152,  0.12558104,  0.18821663]])

It is also possible to evaluate only parts of an array. This is explained below.

Performing operations on a lazy array

Just as with a normal NumPy array, it is possible to perform elementwise arithmetic operations:

>>> a = from_list + 2
>>> b = 2*a
>>> print(type(b))
<class 'lazyarray.larray'>

However, these operations are not carried out immediately, rather they are queued up to be carried out later, which can lead to large time and memory savings if the evaluation step turns out later not to be needed, or if only part of the array needs to be evaluated.

>>> b.evaluate()
array([ 4,  6,  6,  8, 10, 14, 20])

Some more examples:

>>> a = 1.0/(from_list + 1)
>>> a.evaluate()
array([ 1.        ,  0.5       ,  0.5       ,  0.33333333,  0.25      ,
        0.16666667,  0.11111111])
>>> (from_list < 2).evaluate()
array([ True,  True,  True, False, False, False, False], dtype=bool)
>>> (from_list**2).evaluate()
array([ 0,  1,  1,  4,  9, 25, 64])
>>> x = from_list
>>> (x**2 - 2*x + 5).evaluate()
array([ 5,  4,  4,  5,  8, 20, 53])

Numpy ufuncs cannot be used directly with lazy arrays, as NumPy does not know what to do with larray objects. The lazyarray module therefore provides lazy array-compatible versions of a subset of the NumPy ufuncs, e.g.:

>>> from lazyarray import sqrt
>>> sqrt(from_list).evaluate()
array([ 0.        ,  1.        ,  1.        ,  1.41421356,  1.73205081,
        2.23606798,  2.82842712])

For any other function that operates on a NumPy array, it can be applied to a lazy array using the apply() method:

>>> def g(x):
...    return x**2 - 2*x + 5
>>> from_list.apply(g)
>>> from_list.evaluate()
array([ 5,  4,  4,  5,  8, 20, 53])

Partial evaluation

When accessing a single element of an array, only that element is evaluated, where possible, not the whole array:

>>> x = larray(lambda i,j: 2*i + 3*j, shape=(4, 5))
>>> x[3, 2]
12
>>> y = larray(lambda i: i*(2-i), shape=(6,))
>>> y[4]
-8

The same is true for accessing individual rows or columns:

>>> x[1]
array([ 2,  5,  8, 11, 14])
>>> x[:, 4]
array([12, 14, 16, 18])
>>> x[:, (0, 4)]
array([[ 0, 12],
       [ 2, 14],
       [ 4, 16],
       [ 6, 18]])

Table Of Contents

Previous topic

Installation

Next topic

Performance

This Page