Pyjo.Util - Portable utility functions¶
from Pyjo.Util import b64_encode, url_escape, url_unescape
string = 'test=23'
escaped = url_escape(string)
print(url_unescape(escaped))
print(b64_encode(escaped.encode(), ''))
Pyjo.Util provides portable utility functions for Pyjo.
Next to the functions of Mojolicious, it has functions of its own that turn
statements into expressions, so that they can be used in a lambda, a callback
or a chain: die(), getenv(), isalive(), notnone(),
setenv(), slurp(), spurt() and warn().
home = getenv('HOME') or die('HOME is not defined')
See also¶
Pyjoyment, https://mojolicious.org.
Ported from Mojo::Util and Mojo::BaseUtil of Mojolicious 9.52.
Functions¶
- Pyjo.Util.b64_decode(b64: str | bytes) bytes[source]¶
data = b64_decode(b64)
Base64 decode bytes, ignoring the characters that are not Base64 and the ones after the padding, as
MIME::Base64of Perl does.
- Pyjo.Util.b64_encode(data: bytes, eol: str = '\n') str[source]¶
b64 = b64_encode(data) b64 = b64_encode(data, "\n")
Base64 encode bytes, the line ending defaults to a newline.
- Pyjo.Util.b64url_decode(b64: str | bytes) bytes[source]¶
data = b64url_decode(b64)
Base64 decode the variant for URLs of RFC 4648, with
-and_in the place of+and/, with or without the padding, asb64_decode()does, ignoring the characters that are not Base64.# b"\xfb\xff\xfe" b64url_decode('-__-')
- Pyjo.Util.b64url_encode(data: bytes) str[source]¶
b64 = b64url_encode(data)
Base64 encode bytes in the variant for URLs of RFC 4648, with
-and_in the place of+and/, and without the padding and the line endings.# "-__-" b64url_encode(b'\xfb\xff\xfe')
- Pyjo.Util.camelize(string: str) str[source]¶
camelcase = camelize(snakecase)
Convert
snake_casestring toCamelCaseand replace-with::.# "FooBar" camelize('foo_bar') # "FooBar::Baz" camelize('foo_bar-baz') # "FooBar::Baz" camelize('FooBar::Baz')
- Pyjo.Util.class_to_file(class_: str) str[source]¶
file = class_to_file('Foo.Bar')
Convert a class name to a file.
# "foo_bar" class_to_file('Foo.Bar') # "foobar" class_to_file('FOO.Bar') # "foo_bar" class_to_file('FooBar') # "foobar" class_to_file('FOOBar')
- Pyjo.Util.class_to_path(class_: str) str[source]¶
path = class_to_path('Foo.Bar')
Convert class name to path, as a module is found in a directory of
sys.path.# "Foo/Bar.py" class_to_path('Foo.Bar') # "FooBar.py" class_to_path('FooBar')
- Pyjo.Util.decamelize(string: str) str[source]¶
snakecase = decamelize(camelcase)
Convert
CamelCasestring tosnake_caseand replace::with-.# "foo_bar" decamelize('FooBar') # "foo_bar-baz" decamelize('FooBar::Baz') # "foo_bar-baz" decamelize('foo_bar-baz')
- Pyjo.Util.decode(charset: str, data: str | bytes) str | None[source]¶
chars = decode('UTF-8', data)
Decode bytes to characters, or return
Noneif decoding failed. A string of the characters 0-255 is decoded as the bytes it holds, like a Perl byte string; one with other characters does not decode.
- Pyjo.Util.deprecated(*args: object) None[source]¶
deprecated('foo is DEPRECATED in favor of bar')
Warn about deprecated feature from perspective of caller, with a
DeprecationWarning. You can also set thePYJO_FATAL_DEPRECATIONSenvironment variable to make them raise it instead.
- Pyjo.Util.die(e: BaseException | str) NoReturn[source]¶
die(Exception('Something went wrong')) die('Exit immediately')
Raise an exception.
SystemExitexception is raised if parameter is not an exception already.os.environ.get('HOME') or die('HOME is not defined')
- Pyjo.Util.dumper(*values: object) str[source]¶
python = dumper({'some': 'data'})
Dump a Python data structure with
pprint.
- Pyjo.Util.encode(charset: str, chars: str) bytes[source]¶
data = encode('UTF-8', chars)
Encode characters to bytes. A character that the encoding cannot represent becomes a replacement character.
- Pyjo.Util.extract_usage(file: str | PathLike[str] | None = None, section: str | None = None) str[source]¶
usage = extract_usage() usage = extract_usage('/home/sri/foo.py') usage = extract_usage(section='myscript')
Extract usage message from the synopsis of the docstring of a module, its first literal block, defaults to using the file this function was called from.
# "Usage: APPLICATION test [OPTIONS]\n" extract_usage() """ Test command ============ :: Usage: APPLICATION test [OPTIONS] """
With a section, the first literal block after its heading, as the usage of a script that is a function of the module.
# "Usage: myscript [OPTIONS]\n" extract_usage(section='myscript') """ Test module =========== :: test = Test.new() myscript -------- :: Usage: myscript [OPTIONS] """
- Pyjo.Util.generate_secret() str[source]¶
secret = generate_secret()
Generate a random secret with a cryptographically secure random number generator.
- Pyjo.Util.getenv(name: str) str | None[source]¶
- Pyjo.Util.getenv(name: str, default: str) str
envvar = getenv(name) envvar = getenv(name, default)
Get the environment variable. Returns default value or
Noneif variable is not defined.
- Pyjo.Util.getopt(specs: dict[str, Pyjo.GetOpt.Target], /, *, pass_through: bool = False, require_order: bool = False, env_prefix: str | None = None) Pyjo.GetOpt.Pyjo_GetOpt_Result[source]¶
- Pyjo.Util.getopt(args: list[str], specs: dict[str, Pyjo.GetOpt.Target], /, *, pass_through: bool = False, require_order: bool = False, env_prefix: str | None = None) Pyjo.GetOpt.Pyjo_GetOpt_Result
opts = getopt({'H|headers=s': headers, 't|timeout=i': None, 'v|verbose': None}) opts = getopt(args, {'H|headers=s': headers, 't|timeout=i': None, 'v|verbose': None}) opts = getopt(args, {'H|headers=s': headers, 't|timeout=i': None}, pass_through=True) opts = getopt(args, {'v|verbose': None}, require_order=True) opts = getopt(args, {'t|timeout=i': None}, env_prefix='PYJO_')
Extract options from a list with
Pyjo.GetOpt, defaults to using the arguments ofsys.argv. The target of an option is a list for its values, a callback, which gets its name and its value, orNonefor its value in the result, which is false after an error. Withrequire_order, the options end at the first argument that is not one. Withenv_prefix, an option that is not given has the value of the environment variable of its name, asPYJO_TIMEOUTfor--timeout, seePyjo.GetOpt.# Extract "charset" option opts = getopt(['--charset', 'UTF-8'], {'charset=s': None}) print(opts.charset)
- Pyjo.Util.gunzip(compressed: bytes) bytes[source]¶
uncompressed = gunzip(compressed)
Uncompress bytes with
gzip.
- Pyjo.Util.gzip(uncompressed: bytes) bytes[source]¶
compressed = gzip(uncompressed)
Compress bytes with
gzip.
- Pyjo.Util.header_params(value: str) tuple[dict[str, str], str][source]¶
params, remainder = header_params('one=foo; two="bar", three=baz')
Extract HTTP header field parameters until the first comma according to RFC 5987.
- Pyjo.Util.hmac_sha1_bytes(data: bytes, secret: bytes) bytes[source]¶
checksum = hmac_sha1_bytes(data, b'passw0rd')
Generate binary HMAC-SHA1 checksum for bytes with
hmac.
- Pyjo.Util.hmac_sha1_sum(data: bytes, secret: bytes) str[source]¶
checksum = hmac_sha1_sum(data, b'passw0rd')
Generate HMAC-SHA1 checksum for bytes with
hmac.# "11cedfd5ec11adc0ec234466d8a0f2a83736aa68" hmac_sha1_sum(b'foo', b'passw0rd')
- Pyjo.Util.hmac_sha256_bytes(data: bytes, secret: bytes) bytes[source]¶
checksum = hmac_sha256_bytes(data, b'passw0rd')
Generate binary HMAC-SHA256 checksum for bytes with
hmac, as for a key derived in steps, each the key of the next one.# The signing key of Signature Version 4 of AWS key = b'AWS4' + secret for step in (b'20150830', b'us-east-1', b'iam', b'aws4_request'): key = hmac_sha256_bytes(step, key)
- Pyjo.Util.hmac_sha256_sum(data: bytes, secret: bytes) str[source]¶
checksum = hmac_sha256_sum(data, b'passw0rd')
Generate HMAC-SHA256 checksum for bytes with
hmac.# "e54ce0e8e81828088bc8e81b9f2d96816e4a56ffe691d40f1426365e83623c8c" hmac_sha256_sum(b'foo', b'passw0rd')
- Pyjo.Util.hmac_sha512_bytes(data: bytes, secret: bytes) bytes[source]¶
checksum = hmac_sha512_bytes(data, b'passw0rd')
Generate binary HMAC-SHA512 checksum for bytes with
hmac.
- Pyjo.Util.hmac_sha512_sum(data: bytes, secret: bytes) str[source]¶
checksum = hmac_sha512_sum(data, b'passw0rd')
Generate HMAC-SHA512 checksum for bytes with
hmac.# "cd43bd8e691c5aff08152da9918221d91cc1c0f4cb63a80b5e1160387b98791f..." hmac_sha512_sum(b'foo', b'passw0rd')
- Pyjo.Util.html_attr_unescape(string: str) str[source]¶
string = html_attr_unescape(escaped)
Same as
html_unescape(), but handles special rules from the HTML Living Standard for HTML attributes.# "foo=bar<est=baz" html_attr_unescape('foo=bar<est=baz') # "foo=bar<est=baz" html_attr_unescape('foo=bar<est=baz')
- Pyjo.Util.html_unescape(string: str) str[source]¶
string = html_unescape(escaped)
Unescape all HTML entities in string.
# "<div>" html_unescape('<div>')
- Pyjo.Util.humanize_bytes(size: int | float) str[source]¶
string = humanize_bytes(1234)
Turn number of bytes into a simplified human readable format.
# "1B" humanize_bytes(1) # "7.5GiB" humanize_bytes(8007188480) # "13GiB" humanize_bytes(13443399680) # "-685MiB" humanize_bytes(-717946880)
- Pyjo.Util.isalive(obj: object) bool[source]¶
boolean = isalive(weakref.proxy(obj))
Check if object behind weak reference proxy still exists.
- Pyjo.Util.md5_bytes(data: bytes) bytes[source]¶
checksum = md5_bytes(data)
Generate binary MD5 checksum for bytes with
hashlib.
- Pyjo.Util.md5_sum(data: bytes) str[source]¶
checksum = md5_sum(data)
Generate MD5 checksum for bytes with
hashlib.# "acbd18db4cc2f85cedef654fccc4a4d8" md5_sum(b'foo')
- Pyjo.Util.monkey_patch(target: type | ModuleType, **patch: Callable[[...], Any]) None[source]¶
monkey_patch(cls, foo=lambda: ...) monkey_patch(module, foo=lambda: ...) monkey_patch(cls, foo=lambda: ..., bar=lambda: ...)
Monkey patch functions into a class or a module, with names of their own.
monkey_patch(MyApp, one=lambda: print('One!'), two=lambda: print('Two!'), three=lambda: print('Three!'))
- Pyjo.Util.network_contains(cidr: str, addr: str) bool[source]¶
boolean = network_contains(network, address)
Check that a given address is contained within a network in CIDR form. If the network is a single address, the addresses must be equivalent.
# True network_contains('10.0.0.0/8', '10.10.10.10') network_contains('10.10.10.10', '10.10.10.10') network_contains('fc00::/7', 'fc00::c0:ff:ee') # False network_contains('10.0.0.0/29', '10.10.10.10') network_contains('10.10.10.12', '10.10.10.10') network_contains('fc00::/7', '::1')
- Pyjo.Util.notnone(*args: T | Callable[[], T] | None) T | None[source]¶
value = notnone(value, another_value, default_value) value = notnone(value, lambda: default_value)
Return the first argument that is not
None. An argument that is callable is called, and its value is returned (lazy evaluation).from Pyjo.Util import getenv def print_log(msg, **kwargs): level = notnone(kwargs.get('log_level'), lambda: int(getenv('LOG_LEVEL', '3'))) ...
- Pyjo.Util.punycode_decode(punycode: str) str[source]¶
string = punycode_decode(punycode)
Punycode decode string as described in RFC 3492.
# "bücher" punycode_decode('bcher-kva')
- Pyjo.Util.punycode_encode(string: str) str[source]¶
punycode = punycode_encode(string)
Punycode encode string as described in RFC 3492.
# "bcher-kva" punycode_encode('bücher')
- Pyjo.Util.quote(string: str) str[source]¶
- Pyjo.Util.quote(string: bytes) bytes
quoted = quote(string)
Quote string.
- Pyjo.Util.random_bytes(count: int) bytes[source]¶
random = random_bytes(count)
Generate
countbytes worth of high entropy randomness.
- Pyjo.Util.scope_guard(cb: Callable[[], object]) _Guard[source]¶
with scope_guard(lambda: ...): ...
Create anonymous scope guard object that will execute the passed callback when the
withblock ends.# Execute callback at end of block with scope_guard(lambda: print('Mojo!')): print('Hello')
- Pyjo.Util.secure_compare(one: str | bytes, two: str | bytes) bool[source]¶
boolean = secure_compare(string1, string2)
Constant time comparison algorithm to prevent timing attacks. The secret string should be the second argument, to avoid leaking information about the length of the string.
- Pyjo.Util.setenv(name: str, value: str | None) None[source]¶
setenv(name, value)
Set the environment variable or remove it if value is
None.
- Pyjo.Util.sha1_bytes(data: bytes) bytes[source]¶
checksum = sha1_bytes(data)
Generate binary SHA1 checksum for bytes with
hashlib.
- Pyjo.Util.sha1_sum(data: bytes) str[source]¶
checksum = sha1_sum(data)
Generate SHA1 checksum for bytes with
hashlib.# "0beec7b5ea3f0fdbc95d0dd47f3c5bc275da8a33" sha1_sum(b'foo')
- Pyjo.Util.sha256_bytes(data: bytes) bytes[source]¶
checksum = sha256_bytes(data)
Generate binary SHA256 checksum for bytes with
hashlib.
- Pyjo.Util.sha256_sum(data: bytes) str[source]¶
checksum = sha256_sum(data)
Generate SHA256 checksum for bytes with
hashlib.# "2c26b46b68ffc68ff99b453c1d30413413422d706483bfa0f98a5e886266e7ae" sha256_sum(b'foo')
- Pyjo.Util.sha512_bytes(data: bytes) bytes[source]¶
checksum = sha512_bytes(data)
Generate binary SHA512 checksum for bytes with
hashlib.
- Pyjo.Util.sha512_sum(data: bytes) str[source]¶
checksum = sha512_sum(data)
Generate SHA512 checksum for bytes with
hashlib.# "f7fbba6e0636f890e56fbbf3283e524c6fa3204ae298382d624741d0dc663832..." sha512_sum(b'foo')
- Pyjo.Util.slugify(value: str | int, allow_unicode: bool = False) str[source]¶
slug = slugify(string) slug = slugify(string, boolean)
Returns a URL slug generated from the input string. Non-word characters are removed, the string is trimmed and lowercased, and whitespace characters are replaced by a dash. By default, non-ASCII characters are normalized to ASCII word characters or removed, but if a true value is passed as the second parameter, all word characters will be allowed in the result according to unicode semantics.
# "joel-is-a-slug" slugify('Joel is a slug') # "this-is-my-resume" slugify('This is: my - résumé! ☃ ') # "this-is-my-résumé" slugify('This is: my - résumé! ☃ ', True)
- Pyjo.Util.slurp(path: str | PathLike[str], charset: str = 'utf-8') str[source]¶
string = slurp('/etc/passwd') string = slurp('/etc/passwd', 'utf-8')
Read all data at once from file as characters. A byte order mark of Unicode switches
utf-8, the default, to its encoding, and is not a part of the characters.
- Pyjo.Util.split_cookie_header(string: str) list[list[tuple[str, str | None]]][source]¶
tree = split_cookie_header('a=b; expires=Thu, 07 Aug 2008 07:07:59 GMT')
Same as
split_header(), but handlesexpiresvalues from RFC 6265.
- Pyjo.Util.split_header(string: str) list[list[tuple[str, str | None]]][source]¶
tree = split_header('foo="bar baz"; test=123, yada')
Split HTTP header value into key/value pairs, each comma separated part gets its own list of
(key, value)tuples, and keys without a value getNoneassigned.# 'one' split_header('one; two="three four", five=six')[0][0][0] # 'two' split_header('one; two="three four", five=six')[0][1][0] # 'three four' split_header('one; two="three four", five=six')[0][1][1] # 'five' split_header('one; two="three four", five=six')[1][0][0] # 'six' split_header('one; two="three four", five=six')[1][0][1]
- Pyjo.Util.spurt(content: str, path: str | PathLike[str], charset: str = 'utf-8') int[source]¶
written = spurt(string, '/etc/passwd') written = spurt(string, '/etc/passwd', 'utf-8')
Write all characters at once to file, and return the number of bytes written.
- Pyjo.Util.steady_time() float[source]¶
time = steady_time()
High resolution time elapsed from an arbitrary fixed point in the past, resilient to time jumps if a monotonic clock is available through
time.monotonic().
- Pyjo.Util.tablify(rows: Sequence[Sequence[object]]) str[source]¶
table = tablify([['foo', 'bar'], ['baz', 'yada']])
Row-oriented generator for text tables.
# "foo bar\nyada yada\nbaz yada\n" tablify([['foo', 'bar'], ['yada', 'yada'], ['baz', 'yada']])
- Pyjo.Util.term_escape(string: str) str[source]¶
escaped = term_escape(string)
Escape all POSIX control characters except for
\n.# "foo\\x09bar\\x0d\n" term_escape("foo\tbar\r\n")
- Pyjo.Util.trim(string: str) str[source]¶
trimmed = trim(string)
Trim whitespace characters from both ends of string.
# "foo bar" trim(' foo bar ')
- Pyjo.Util.unindent(string: str) str[source]¶
unindented = unindent(string)
Unindent multi-line string.
# "foo\nbar\nbaz\n" unindent(" foo\n bar\n baz\n")
- Pyjo.Util.unquote(string: str) str[source]¶
- Pyjo.Util.unquote(string: bytes) bytes
string = unquote(quoted)
Unquote string.
- Pyjo.Util.url_escape(string: str, pattern: str | None = None) str[source]¶
- Pyjo.Util.url_escape(string: bytes, pattern: bytes | None = None) bytes
escaped = url_escape(string) escaped = url_escape(string, r'^A-Za-z0-9\-._~')
Percent encode unsafe characters in string as described in RFC 3986, the pattern used defaults to
^A-Za-z0-9\-._~.# "foo%3Bbar" url_escape('foo;bar')
- Pyjo.Util.url_unescape(string: str) str[source]¶
- Pyjo.Util.url_unescape(string: bytes) bytes
string = url_unescape(escaped)
Decode percent encoded characters in string as described in RFC 3986. A percent encoded byte in a string becomes the character with the same code, as in Perl.
# "foo;bar" url_unescape('foo%3Bbar')
- Pyjo.Util.warn(*args: object) None[source]¶
warn(string) warn(string, value)
Output the arguments to
sys.stderr, joined as Perl’swarnjoins a list, with a newline at the end unless there is one already.
- Pyjo.Util.xml_escape(string: str | Markup | None) str[source]¶
escaped = xml_escape(string)
Escape unsafe characters
&,<,>,"and'in string, but do not escape markup: an object with the method__html__, asPyjo.String.HTML, which gives the markup.# "<div>" xml_escape('<div>') # "<div>" from Pyjo.String.HTML import html xml_escape(html('<div>'))