that I find useful and hope you will too.
Code is under `src/pyutils/`. Most code includes documentation and inline
-doctests.
+doctests. I've tried to organize it into logical packages based on the
+code's functionality.
Unit and integration tests are under `tests/*`. To run all tests::
./run_tests.py --all [--coverage]
See the README under `tests/` and the code of `run_tests.py` for more
-options / information.
+options / information about running tests.
This package generates Sphinx docs which are available at:
https://wannabe.guru.org/pydocs/pyutils/pyutils.html
Package code is checked into a local git server and available to clone
-from git at https://wannabe.guru.org/git/pyutils.git or from a web browser
-at:
+from git at https://wannabe.guru.org/git/pyutils.git or to view in a
+web browser at:
https://wannabe.guru.org/gitweb/?p=pyutils.git;a=summary
For a long time this was just a local library on my machine that my
-tools imported but I've now decided to release it on PyPi. Early
+tools imported but I've now decided to release it on PyPi. Earlier
development happened in a different git repo:
https://wannabe.guru.org/gitweb/?p=python_utils.git;a=summary
-I hope you find this useful. LICENSE and NOTICE describe reusing this
-code and where everything came from. Drop me a line if you are using
-this, find a bug, have a question, or have a suggestion:
+The LICENSE and NOTICE files at the root of the project describe
+reusing this code and where everything came from. Drop me a line
+if you are using this, find a bug, have a question, or have a
+suggestion:
.. toctree::
:maxdepth: 4
+ :name: mastertoc
pyutils.collectionz
pyutils.compress
-------------------
This file mainly contains code for changing the nature of text printed
-to the console via ANSI escape sequences. e.g. it can be used to emit
-text that is bolded, underlined, italicised, colorized, etc...
+to the console via ANSI escape sequences. For example, it can be used
+to emit text that is bolded, underlined, italicised, colorized, etc...
-It also contains a colorizing context that will apply color patterns
-based on regular expressions to any data emitted to stdout that may be
-useful in adding color to other programs' outputs, for instance.
+It does *not* contain ANSI escape sequences that do things like move
+the cursor or record/restore its position. It is focused on text style
+only.
-Here are the predefined color names it knows:
+The file includes a colorizing context that will apply color patterns
+based on regular expressions / callables to any data emitted to stdout
+that may be useful in adding color to other programs' outputs, for
+instance.
+
+In addition, it contains a mapping from color name to RGB value that it
+uses to enable friendlier color names in most of its functions. Here
+is the list of predefined color names it knows:
.. raw:: html
.. note::
You can also use raw RGB values with this module so you do not have to use
- the predefined color names.
+ these predefined color names unless you want to.
+
+---
.. automodule:: pyutils.ansi
:members:
pyutils.argparse\_utils module
------------------------------
-I use the Python internal :py:mod:`argparse` module for commandline argument
-parsing but found it lacking in some ways. This module contains code to
-fill those gaps. It include stuff like:
+I use and love the Python internal :py:mod:`argparse` module for
+commandline argument parsing but found it lacking in some ways. This
+module contains code to fill those gaps. See also :py:mod:`pyutils.config`.
- - An :class:`argparse.Action` to create pairs of flags such as
- `--feature` and `--no_feature`.
- - A helper to parse and validate bools, IP addresses, MAC
- addresses, filenames, percentages, dates, datetimes, and
- durations passed as flags.
+---
.. automodule:: pyutils.argparse_utils
:members:
pyutils.bootstrap module
------------------------
-Bootstrap module defines a decorator meant to wrap your main function.
-This decorator will do several things for you:
+The bootstrap module defines a decorator meant to wrap your main function.
+This is optional, of course: you can use this library without using the
+bootstrap decorator on your main. If you choose to use it, though, it will
+do some work for you automatically.
- - If your code uses the :file:`config.py` module (see below), it invokes
- :py:meth:`parse` automatically to initialize `config.config` from commandline
- flags, environment variables, or other sources.
- - It will optionally break into pdb in response to an unhandled
- Exception at the top level of your code.
- - It initializes logging for your program (see :file:`logging.py`).
- - It can optionally run a code and/or memory profiler on your code.
+---
.. automodule:: pyutils.bootstrap
:members:
pyutils.config module
---------------------
-This module reads the program's configuration parameters from: the
-commandline (using :py:mod:`argparse`), environment variables and/or a
-shared zookeeper-based configuration. It stores this configuration
-state in a dict-like structure that can be queried by your code at
-runtime.
-
-It handles creating a nice `--help` message for your code.
+The config module is an opinionated way to set up input parameters
+to your program. It is enabled by using the :py:mod:`pyutils.bootstrap`
+decorator or by simply calling :py:meth:`pyutils.config.parse` early in main
+(which is what :py:meth:`pyutils.bootstrap.initialize` does for you).
-It can optionally react to dynamic config changes and change state at
-runtime (if and only if you name your flag with the *dynamic_* prefix
-and are using zookeeper-based configs).
+If you use this module, input parameters to your program can come from
+the commandline (and are configured using Python's :py:mod:`argparse`).
+But they can also be be augmented or replaced using saved configuration
+files stored either on the local filesystem or on Apache Zookeeper.
+There is a provision for enabling dynamic arguments (i.e. that can change
+during runtime) via Zookeeper.
-It can optionally save and retrieve sets of arguments from files on
-the local disk or on Zookeeper.
-
-All of my examples use this as does the pyutils library itself.
+---
.. automodule:: pyutils.config
:members:
This is a grab bag of decorators.
+---
+
.. automodule:: pyutils.decorator_utils
:members:
:undoc-members:
A bunch of helpers for dealing with Python dicts.
+---
+
.. automodule:: pyutils.dict_utils
:members:
:undoc-members:
Helper code for dealing with subprocesses.
+---
+
.. automodule:: pyutils.exec_utils
:members:
:undoc-members:
Helper util for dealing with functions.
+---
+
.. automodule:: pyutils.function_utils
:members:
:undoc-members:
Generate unique identifiers.
+---
+
.. automodule:: pyutils.id_generator
:members:
:undoc-members:
Iterator utilities including a :py:class:`PeekingIterator`, :py:class:`PushbackIterator`,
and :py:class:`SamplingIterator`.
+---
+
.. automodule:: pyutils.iter_utils
:members:
:undoc-members:
Utilities for dealing with Python lists.
+---
+
.. automodule:: pyutils.list_utils
:members:
:undoc-members:
- Log probalistically,
- Clear rogue logging handlers added by other imports.
+---
+
.. automodule:: pyutils.logging_utils
:members:
:undoc-members:
computation, literate float truncation, percentage <-> multiplier, prime
number determination, etc...
+---
+
.. automodule:: pyutils.math_utils
:members:
:undoc-members:
Miscellaneous utilities: are we running as root, and is a debugger attached?
+---
+
.. automodule:: pyutils.misc_utils
:members:
:undoc-members:
from some external location and (optionally / conditionally) save their
state to an external location at shutdown.
+---
+
.. automodule:: pyutils.persistent
:members:
:undoc-members:
run pickled code on a remote machine. It is used by code marked with
`@parallelize(method=Method.REMOTE)` in the parallelize framework.
+---
+
.. automodule:: pyutils.remote_worker
:members:
:undoc-members:
(either automatically or when invoked to poll) and allow their callers
to wait on state changes.
+---
+
.. automodule:: pyutils.state_tracker
:members:
:undoc-members:
This is a stopwatch context that just times how long something took to
execute.
+---
+
.. automodule:: pyutils.stopwatch
:members:
:undoc-members:
starting library from Davide Zanotti, I've added a pile of other string
functions so hopefully it will handle all of your string-needs.
+---
+
.. automodule:: pyutils.string_utils
:members:
:undoc-members:
- create a header line,
- draw a box around some text.
+---
+
.. automodule:: pyutils.text_utils
:members:
:undoc-members:
Utilities to support smarter unit tests.
+---
+
.. automodule:: pyutils.unittest_utils
:members:
:undoc-members:
Unscramble scrambled English words quickly.
+---
+
.. automodule:: pyutils.unscrambler
:members:
:undoc-members:
A helper module for dealing with Zookeeper that adds some functionality.
+---
+
.. automodule:: pyutils.zookeeper
:members:
:undoc-members:
Module contents
---------------
+---
+
.. automodule:: pyutils
:members:
:undoc-members: