Pyjo.Iterator - Lazy iterator¶
from Pyjo.Iterator import Iterator
# Elements of an iterable, one by one, through a chain
for word in Iterator(['just', 'works']).map(lambda e: e.upper()):
print(word)
# Or import the module, as in Google Python Style Guide
import Pyjo.Iterator
# Three requests at a time, and their titles in the order of the URLs
titles = Pyjo.Iterator.new(urls).map(ua.get_p, concurrency=3).map(lambda tx: tx.result.dom().at('title').text)
for title in titles:
print(title)
# The same in a coroutine
async for title in titles:
print(title)
# A collection, lazily
from Pyjo.Collection import c
c(1, 2, 3).iterate().map(lambda e: e * 2).join(', ').say()
Pyjo.Iterator is a lazy iterator of the elements of an iterable or of an
asynchronous iterable, with the methods of Pyjo.Collection. Each step
of a chain passes an element on to the next one as soon as it has it, so that
the first element is processed before the last one is taken from the source.
It is iterated with for and with async for alike, and each iteration
takes the elements of the source anew, which a source that is iterated once,
as a generator or a stream, gives once.
An iteration that ends before the end of the source, as of
first_p(), of head() or of an exception, closes the
asynchronous iterators of the chain with their aclose, as
contextlib.aclosing() does, down to the one of the source, as an async
generator or a stream, so that the garbage collector does not finalize them
later.
A callback may return a Pyjo.Promise or another awaitable, or be a
coroutine function, and the iterator passes on the value that it waits for.
async for awaits it, and for waits for it with
wait(), which runs the event loop, so that
for cannot wait in an event loop that runs already, as in an action of a
server, where it raises a RuntimeError, and where async for or a
method whose name ends with _p waits instead. for takes the elements of an
asynchronous iterable, as a stream, the same way.
# The messages of a stream as JSON, in a coroutine
async for request in messages.iterate().map(Pyjo.JSON.decode_json):
...
See also¶
Pyjoyment, https://mojolicious.org.
It has no counterpart in Mojolicious; it is inspired by the iterator helpers of
JavaScript, with the methods of Pyjo.Collection.
- class Pyjo.Iterator.Pyjo_Iterator(iterable: Iterable | AsyncIterable = ())[source]¶
Pyjo.Iteratorinherits all methods fromPyjo.Baseand implements the following new ones.- __aiter__() AsyncIterator[source]¶
async for e in iterator: ...
Take the elements in a coroutine, and await the values of the callbacks.
- __iter__() Iterator[source]¶
for e in iterator: ...
Take the elements, and wait for the values of the callbacks with the event loop.
- compact() Self[source]¶
new = iterator.compact()
Pass on all elements that are not
Noneand not an empty string.# "0, 1, 2, 3" Pyjo.Iterator.new([0, 1, None, 2, '', 3]).compact().join(', ')
- each(cb: Callable[[T, int], Any]) Self[source]¶
iterator = iterator.each(lambda e, num: ...)
Evaluate callback for each element, one after another, and wait for its value. The element will be the first argument passed to the callback, and its number the second.
# Make a numbered list iterator.each(lambda e, num: print(f'{num}: {e}'))
- each_p(cb: Callable[[T, int], Any]) Pyjo_Promise[source]¶
promise = iterator.each_p(lambda e, num: ...)
Same as
each(), but returns aPyjo.Promiseobject, which is fulfilled with the iterator once the callback has been evaluated for each element.# Process the messages one after another await messages.iterate().each_p(handle)
- first(cb: Callable[[T], Anything] | None = None, /) T | None[source]¶
- first(cb: Callable[[...], Any] | str | Pattern[str], /, *args: Any, **kwargs: Any) T | None
first = iterator.first() first = iterator.first(re.compile(r'foo')) first = iterator.first(lambda e: ...) first = iterator.first('some_method') first = iterator.first('some_method', *args)
Evaluate regular expression/callback for, or call method on, each element and return the first one that matched the regular expression, or for which the callback/method returned true, without taking the elements after it.
# Find first value that is greater than 5 greater = iterator.first(lambda e: e > 5)
- first_p(cb: Callable[[...], Any] | str | Pattern[str] | None = None, /, *args: Any, **kwargs: Any) Pyjo_Promise[source]¶
promise = iterator.first_p(lambda e: ...)
Same as
first(), but returns aPyjo.Promiseobject, which is fulfilled with the element, orNone.
- flatten() Pyjo_Iterator[Any][source]¶
new = iterator.flatten()
Flatten nested collections/lists/tuples recursively and pass on all elements.
# "1, 2, 3, 4, 5, 6, 7" Pyjo.Iterator.new([1, [2, [3, 4], 5, [6]], 7]).flatten().join(', ')
- grep(cb: Callable[[T], Anything], /) Self[source]¶
- grep(cb: Callable[[...], Any] | str | Pattern[str], /, *args: Any, **kwargs: Any) Self
new = iterator.grep(re.compile(r'foo')) new = iterator.grep(lambda e: ...) new = iterator.grep('some_method') new = iterator.grep('some_method', *args)
Evaluate regular expression/callback for, or call method on, each element and pass on all elements that matched the regular expression, or for which the callback/method returned true.
# Find all values that are greater than 5 greater = iterator.grep(lambda e: e > 5)
- head(size: int) Self[source]¶
new = iterator.head(4)
Pass on up to the specified number of elements from the beginning, and take no more of them.
# "A B C" Pyjo.Iterator.new(['A', 'B', 'C', 'D', 'E']).head(3).join(' ')
- join(string: str | None = None) Pyjo_String_Unicode[source]¶
string = iterator.join() string = iterator.join("\n")
Turn elements into
Pyjo.String.Unicode.# Join all values with commas iterator.join(', ').say()
- join_p(string: str | None = None) Pyjo_Promise[source]¶
promise = iterator.join_p(', ')
Same as
join(), but returns aPyjo.Promiseobject, which is fulfilled with thePyjo.String.Unicode.
- map(cb: Callable[[T], Awaitable[U]], /, *, concurrency: int = 1) Pyjo_Iterator[U][source]¶
- map(cb: Callable[[T], U], /, *, concurrency: int = 1) Pyjo_Iterator[U]
- map(cb: Callable[[...], Any] | str, /, *args: Any, concurrency: int = 1, **kwargs: Any) Pyjo_Iterator[Any]
new = iterator.map(lambda e: ...) new = iterator.map(lambda e: ..., concurrency=3) new = iterator.map('some_method') new = iterator.map('some_method', *args)
Evaluate callback for, or call method on, each element and pass on its value. The element will be the first argument passed to the callback.
# Append the word "pyjo" to all values pyjoified = iterator.map(lambda e: e + 'pyjo')
A callback that returns a
Pyjo.Promiseor another awaitable, or a coroutine function, is evaluated ahead for up toconcurrencyelements, so that as many promises are in progress at the same time, while the values are passed on in the order of the elements.# Perform 3 requests at a time concurrently iterator.map(ua.get_p, concurrency=3).each(lambda tx, num: print(tx.result.dom().at('title').text))
These options are currently available:
- concurrency
concurrency=3
The maximum number of elements whose callbacks are in progress at the same time,
1by default, as a source may have no end.
- classmethod new(iterable: Iterable | AsyncIterable = ()) Self[source]¶
iterator = Pyjo.Iterator.new([1, 2, 3]) iterator = Pyjo.Iterator.new(n for n in range(3)) iterator = Pyjo.Iterator.new(stream)
Construct a new
Pyjo.Iteratorobject of an iterable or of an asynchronous iterable.
- reduce(cb: Callable[[Any, T], Any], *initial: Any) Any[source]¶
result = iterator.reduce(lambda a, b: ...) result = iterator.reduce(lambda a, b: ..., initial)
Reduce elements with a callback and return its final result, passing
aandbeach time the callback is executed. The first timeawill be an optional initial value or the first element. And from then onawill be the value of the callback, whilebwill always be the next element.# Calculate the sum of all values total = iterator.reduce(lambda a, b: a + b)
- reduce_p(cb: Callable[[Any, T], Any], *initial: Any) Pyjo_Promise[source]¶
promise = iterator.reduce_p(lambda a, b: ...) promise = iterator.reduce_p(lambda a, b: ..., initial)
Same as
reduce(), but returns aPyjo.Promiseobject, which is fulfilled with the final result.
- to_list() Pyjo.Collection.object[T][source]¶
collection = iterator.to_list()
Take all elements and turn them into a
Pyjo.Collection, which is alist.
- to_list_p() Pyjo_Promise[source]¶
promise = iterator.to_list_p()
Same as
to_list(), but returns aPyjo.Promiseobject, which is fulfilled with thePyjo.Collection.# The titles of all pages, 3 at a time titles = await Pyjo.Iterator.new(urls).map(ua.get_p, concurrency=3).map(title).to_list_p()
- uniq(cb: Callable[[...], Any] | str | None = None, *args: Any, **kwargs: Any) Self[source]¶
new = iterator.uniq() new = iterator.uniq(lambda e: ...) new = iterator.uniq('some_method') new = iterator.uniq('some_method', *args)
Pass on the elements without duplicates, using the string representation of either the elements or the value of the callback/method to decide uniqueness. Note that
Noneand empty string are treated the same.# "foo bar baz" Pyjo.Iterator.new(['foo', 'bar', 'bar', 'baz']).uniq().join(' ')
- Pyjo.Iterator.new(iterable: Iterable | AsyncIterable = ()) Self[source]¶
iterator = Pyjo.Iterator.new([1, 2, 3]) iterator = Pyjo.Iterator.new(n for n in range(3)) iterator = Pyjo.Iterator.new(stream)
Construct a new
Pyjo.Iteratorobject of an iterable or of an asynchronous iterable.