pyutils package
===============
+When I was writing little tools in Python and found myself implementing
+a generally useful pattern I stuffed it into a local library. That
+library grew into pyutils: a set of collections, helpers and utilities
+that I find useful and hope you will too.
+
+Code is under `src/pyutils/`. Most code includes documentation and inline
+doctests.
+
+Unit and integration tests are under `tests/*`. To run all tests::
+
+ cd tests/
+ ./run_tests.py --all [--coverage]
+
+See the README under `tests/` and the code of `run_tests.py` for more
+options / information.
+
+This package generates Sphinx docs which are available at:
+
+ https://wannabe.guru.org/pydocs/pyutils/pyutils.html
+
+Package code is checked into a local git server and available to clone
+from git at https://wannabe.guru.org/git/pyutils.git or from a web browser
+at:
+
+ https://wannabe.guru.org/gitweb/?p=pyutils.git;a=summary
+
+For a long time this was just a local library on my machine that my
+tools imported but I've now decided to release it on PyPi. Early
+development happened in a different git repo:
+
+ https://wannabe.guru.org/gitweb/?p=python_utils.git;a=summary
+
+I hope you find this useful. LICENSE and NOTICE describe reusing it
+and where everything came from. Drop me a line if you are using this,
+find a bug, or have a question:
+
+
+
Subpackages
-----------
pyutils.ansi module
-------------------
+This file mainly contains code for changing the nature of text printed
+to the console via ANSI escape sequences. e.g. it can be used to emit
+text that is bolded, underlined, italicised, colorized, etc...
+
+It also contains a colorizing context that will apply color patterns
+based on regular expressions to any data emitted to stdout that may be
+useful in adding color to other programs' outputs, for instance.
+
.. automodule:: pyutils.ansi
:members:
:undoc-members:
pyutils.argparse\_utils module
------------------------------
+I use the Python internal `argparse` module for commandline argument
+parsing but found it lacking in some ways. This module contains code to
+fill those gaps. It include stuff like:
+
+ - An `argparse.Action` to create pairs of flags such as
+ `--feature` and `--no_feature`.
+ - A helper to parse and validate bools, IP addresses, MAC
+ addresses, filenames, percentages, dates, datetimes, and
+ durations passed as flags.
+
.. automodule:: pyutils.argparse_utils
:members:
:undoc-members:
pyutils.bootstrap module
------------------------
+Bootstrap module defines a decorator meant to wrap your main function.
+This decorator will do several things for you:
+
+ - If your code uses the :file:`config.py` module (see below), it invokes
+ `parse` automatically to initialize `config.config` from commandline
+ flags, environment variables, or other sources.
+ - It will optionally break into pdb in response to an unhandled
+ Exception at the top level of your code.
+ - It initializes logging for your program (see :file:`logging.py`).
+ - It can optionally run a code and/or memory profiler on your code.
+
.. automodule:: pyutils.bootstrap
:members:
:undoc-members:
pyutils.config module
---------------------
+This module reads the program's configuration parameters from: the
+commandline (using argparse), environment variables and/or a shared
+zookeeper-based configuration. It stores this configuration state in
+a dict-like structure that can be queried by your code at runtime.
+
+It handles creating a nice `--help` message for your code.
+
+It can optionally react to dynamic config changes and change state at
+runtime (iff you name your flag with the *dynamic_* prefix and are
+using zookeeper-based configs).
+
+It can optionally save and retrieve sets of arguments from files on
+the local disk or on zookeeper.
+
+All of my examples use this as does the pyutils library itself.
+
.. automodule:: pyutils.config
:members:
:undoc-members:
pyutils.decorator\_utils module
-------------------------------
+This is a grab bag of decorators.
+
.. automodule:: pyutils.decorator_utils
:members:
:undoc-members:
pyutils.dict\_utils module
--------------------------
+A bunch of helpers for dealing with Python dicts.
+
.. automodule:: pyutils.dict_utils
:members:
:undoc-members:
pyutils.exec\_utils module
--------------------------
+Helper code for dealing with subprocesses.
+
.. automodule:: pyutils.exec_utils
:members:
:undoc-members:
pyutils.function\_utils module
------------------------------
+Helper util for dealing with functions.
+
.. automodule:: pyutils.function_utils
:members:
:undoc-members:
pyutils.id\_generator module
----------------------------
+Generate unique identifiers.
+
.. automodule:: pyutils.id_generator
:members:
:undoc-members:
pyutils.iter\_utils module
--------------------------
+Iterator utilities including a :class:PeekingIterator, :class:PushbackIterator,
+and :class:SamplingIterator.
+
.. automodule:: pyutils.iter_utils
:members:
:undoc-members:
pyutils.list\_utils module
--------------------------
+Utilities for dealing with Python lists.
+
.. automodule:: pyutils.list_utils
:members:
:undoc-members:
pyutils.logging\_utils module
-----------------------------
+This is a module that offers an opinionated take on how whole program logging
+should be initialized and controlled. It uses standard Python logging but gives
+you control, via commandline config, to:
+
+ - Set the logging level of the program including overriding the
+ logging level for individual modules,
+ - Define the logging message format including easily adding a
+ PID/TID marker on all messages to help with multithreaded debugging,
+ - Control the destination (file, sys.stderr, syslog) of messages,
+ - Control the facility and logging level used with syslog,
+ - Squelch repeated messages,
+ - Log probalistically,
+ - Clear rogue logging handlers added by other imports.
+
.. automodule:: pyutils.logging_utils
:members:
:undoc-members:
pyutils.math\_utils module
--------------------------
+Helper utilities that are "mathy" such as a :class:NumericPopulation that
+makes population summary statistics available to your code quickly, GCD
+computation, literate float truncation, percentage <-> multiplier, prime
+number determination, etc...
+
.. automodule:: pyutils.math_utils
:members:
:undoc-members:
pyutils.misc\_utils module
--------------------------
+Miscellaneous utilities: are we running as root, and is a debugger attached?
+
.. automodule:: pyutils.misc_utils
:members:
:undoc-members:
pyutils.persistent module
-------------------------
+Persistent defines a class hierarchy and decorator for creating
+singleton classes that (optionally/conditionally) load their state
+from some external location and (optionally/conditionally) stave their
+state to an external location at shutdown.
+
.. automodule:: pyutils.persistent
:members:
:undoc-members:
pyutils.remote\_worker module
-----------------------------
+This module defines a helper that is invoked by the remote executor to
+run pickled code on a remote machine. It is used by code marked with
+@parallelize(method=Method.REMOTE) in the parallelize framework.
+
.. automodule:: pyutils.remote_worker
:members:
:undoc-members:
pyutils.state\_tracker module
-----------------------------
+This module defines several classes (:class:StateTracker,
+:class:AutomaticStateTracker, and
+:class:WaitableAutomaticStateTracker) that can be used as base
+classes. These class patterns are meant to encapsulate and represent
+state that dynamically changes. These classes update their state
+(either automatically or when invoked to poll) and allow their callers
+to wait on state changes.
+
.. automodule:: pyutils.state_tracker
:members:
:undoc-members:
pyutils.stopwatch module
------------------------
+This is a stopwatch context that just times how long something took to
+execute.
+
.. automodule:: pyutils.stopwatch
:members:
:undoc-members:
pyutils.string\_utils module
----------------------------
+A bunch of utilities for dealing with strings. Based on a really great
+starting library from Davide Zanotti, I've added a pile of other string
+functions so hopefully it will handle all of your string-needs.
+
.. automodule:: pyutils.string_utils
:members:
:undoc-members:
pyutils.text\_utils module
--------------------------
+Utilities for dealing with and creating text chunks. For example:
+
+ - Make a bar graph,
+ - make a spark line,
+ - left, right, center, justify text,
+ - word wrap text,
+ - indent text,
+ - create a header line,
+ - draw a box around some text.
+
.. automodule:: pyutils.text_utils
:members:
:undoc-members:
pyutils.unittest\_utils module
------------------------------
+Utilities to support smarter unit tests.
+
.. automodule:: pyutils.unittest_utils
:members:
:undoc-members:
pyutils.unscrambler module
--------------------------
+Unscramble scrambled English words quickly.
+
.. automodule:: pyutils.unscrambler
:members:
:undoc-members:
pyutils.zookeeper module
------------------------
+A helper module for dealing with Zookeeper that adds some functionality.
+
.. automodule:: pyutils.zookeeper
:members:
:undoc-members:
projects / pyutils.git / blobdiff