X-Git-Url: https://wannabe.guru.org/gitweb/?a=blobdiff_plain;f=docs%2Fpyutils.rst;h=97a222129519c8104728f10c9200519bd565357e;hb=5bbcfd4dfccd411df0afd03f9790f67b4c852b9c;hp=f548756452a5cf18f4af50a4eb8464de92431556;hpb=91334e2f21a891c38c4c9432c96f52132bafa801;p=pyutils.git diff --git a/docs/pyutils.rst b/docs/pyutils.rst index f548756..97a2221 100644 --- a/docs/pyutils.rst +++ b/docs/pyutils.rst @@ -7,7 +7,8 @@ library grew into pyutils: a set of collections, helpers and utilities that I find useful and hope you will too. Code is under `src/pyutils/`. Most code includes documentation and inline -doctests. +doctests. I've tried to organize it into logical packages based on the +code's functionality. Unit and integration tests are under `tests/*`. To run all tests:: @@ -15,27 +16,28 @@ Unit and integration tests are under `tests/*`. To run all tests:: ./run_tests.py --all [--coverage] See the README under `tests/` and the code of `run_tests.py` for more -options / information. +options / information about running tests. This package generates Sphinx docs which are available at: https://wannabe.guru.org/pydocs/pyutils/pyutils.html Package code is checked into a local git server and available to clone -from git at https://wannabe.guru.org/git/pyutils.git or from a web browser -at: +from git at https://wannabe.guru.org/git/pyutils.git or to view in a +web browser at: https://wannabe.guru.org/gitweb/?p=pyutils.git;a=summary For a long time this was just a local library on my machine that my -tools imported but I've now decided to release it on PyPi. Early +tools imported but I've now decided to release it on PyPi. Earlier development happened in a different git repo: https://wannabe.guru.org/gitweb/?p=python_utils.git;a=summary -I hope you find this useful. LICENSE and NOTICE describe reusing this -code and where everything came from. Drop me a line if you are using -this, find a bug, have a question, or have a suggestion: +The LICENSE and NOTICE files at the root of the project describe +reusing this code and where everything came from. Drop me a line +if you are using this, find a bug, have a question, or have a +suggestion: --Scott Gasch (scott.gasch@gmail.com) @@ -45,6 +47,7 @@ Subpackages .. toctree:: :maxdepth: 4 + :name: mastertoc pyutils.collectionz pyutils.compress @@ -62,14 +65,21 @@ pyutils.ansi module ------------------- This file mainly contains code for changing the nature of text printed -to the console via ANSI escape sequences. e.g. it can be used to emit -text that is bolded, underlined, italicised, colorized, etc... +to the console via ANSI escape sequences. For example, it can be used +to emit text that is bolded, underlined, italicised, colorized, etc... -It also contains a colorizing context that will apply color patterns -based on regular expressions to any data emitted to stdout that may be -useful in adding color to other programs' outputs, for instance. +It does *not* contain ANSI escape sequences that do things like move +the cursor or record/restore its position. It is focused on text style +only. -Here are the predefined color names it knows: +The file includes a colorizing context that will apply color patterns +based on regular expressions / callables to any data emitted to stdout +that may be useful in adding color to other programs' outputs, for +instance. + +In addition, it contains a mapping from color name to RGB value that it +uses to enable friendlier color names in most of its functions. Here +is the list of predefined color names it knows: .. raw:: html @@ -2135,7 +2145,9 @@ Here are the predefined color names it knows: .. note:: You can also use raw RGB values with this module so you do not have to use - the predefined color names. + these predefined color names unless you want to. + +--- .. automodule:: pyutils.ansi :members: @@ -2145,15 +2157,11 @@ Here are the predefined color names it knows: pyutils.argparse\_utils module ------------------------------ -I use the Python internal :py:mod:`argparse` module for commandline argument -parsing but found it lacking in some ways. This module contains code to -fill those gaps. It include stuff like: +I use and love the Python internal :py:mod:`argparse` module for +commandline argument parsing but found it lacking in some ways. This +module contains code to fill those gaps. See also :py:mod:`pyutils.config`. - - An :class:`argparse.Action` to create pairs of flags such as - `--feature` and `--no_feature`. - - A helper to parse and validate bools, IP addresses, MAC - addresses, filenames, percentages, dates, datetimes, and - durations passed as flags. +--- .. automodule:: pyutils.argparse_utils :members: @@ -2163,16 +2171,12 @@ fill those gaps. It include stuff like: pyutils.bootstrap module ------------------------ -Bootstrap module defines a decorator meant to wrap your main function. -This decorator will do several things for you: +The bootstrap module defines a decorator meant to wrap your main function. +This is optional, of course: you can use this library without using the +bootstrap decorator on your main. If you choose to use it, though, it will +do some work for you automatically. - - If your code uses the :file:`config.py` module (see below), it invokes - :py:meth:`parse` automatically to initialize `config.config` from commandline - flags, environment variables, or other sources. - - It will optionally break into pdb in response to an unhandled - Exception at the top level of your code. - - It initializes logging for your program (see :file:`logging.py`). - - It can optionally run a code and/or memory profiler on your code. +--- .. automodule:: pyutils.bootstrap :members: @@ -2182,22 +2186,19 @@ This decorator will do several things for you: pyutils.config module --------------------- -This module reads the program's configuration parameters from: the -commandline (using :py:mod:`argparse`), environment variables and/or a -shared zookeeper-based configuration. It stores this configuration -state in a dict-like structure that can be queried by your code at -runtime. - -It handles creating a nice `--help` message for your code. +The config module is an opinionated way to set up input parameters +to your program. It is enabled by using the :py:mod:`pyutils.bootstrap` +decorator or by simply calling :py:meth:`pyutils.config.parse` early in main +(which is what :py:meth:`pyutils.bootstrap.initialize` does for you). -It can optionally react to dynamic config changes and change state at -runtime (if and only if you name your flag with the *dynamic_* prefix -and are using zookeeper-based configs). +If you use this module, input parameters to your program can come from +the commandline (and are configured using Python's :py:mod:`argparse`). +But they can also be be augmented or replaced using saved configuration +files stored either on the local filesystem or on Apache Zookeeper. +There is a provision for enabling dynamic arguments (i.e. that can change +during runtime) via Zookeeper. -It can optionally save and retrieve sets of arguments from files on -the local disk or on Zookeeper. - -All of my examples use this as does the pyutils library itself. +--- .. automodule:: pyutils.config :members: @@ -2209,6 +2210,8 @@ pyutils.decorator\_utils module This is a grab bag of decorators. +--- + .. automodule:: pyutils.decorator_utils :members: :undoc-members: @@ -2219,6 +2222,8 @@ pyutils.dict\_utils module A bunch of helpers for dealing with Python dicts. +--- + .. automodule:: pyutils.dict_utils :members: :undoc-members: @@ -2229,6 +2234,8 @@ pyutils.exec\_utils module Helper code for dealing with subprocesses. +--- + .. automodule:: pyutils.exec_utils :members: :undoc-members: @@ -2239,6 +2246,8 @@ pyutils.function\_utils module Helper util for dealing with functions. +--- + .. automodule:: pyutils.function_utils :members: :undoc-members: @@ -2249,6 +2258,8 @@ pyutils.id\_generator module Generate unique identifiers. +--- + .. automodule:: pyutils.id_generator :members: :undoc-members: @@ -2260,6 +2271,8 @@ pyutils.iter\_utils module Iterator utilities including a :py:class:`PeekingIterator`, :py:class:`PushbackIterator`, and :py:class:`SamplingIterator`. +--- + .. automodule:: pyutils.iter_utils :members: :undoc-members: @@ -2270,6 +2283,8 @@ pyutils.list\_utils module Utilities for dealing with Python lists. +--- + .. automodule:: pyutils.list_utils :members: :undoc-members: @@ -2292,6 +2307,8 @@ logging should be initialized and controlled. It uses standard Python - Log probalistically, - Clear rogue logging handlers added by other imports. +--- + .. automodule:: pyutils.logging_utils :members: :undoc-members: @@ -2305,6 +2322,8 @@ makes population summary statistics available to your code quickly, GCD computation, literate float truncation, percentage <-> multiplier, prime number determination, etc... +--- + .. automodule:: pyutils.math_utils :members: :undoc-members: @@ -2315,6 +2334,8 @@ pyutils.misc\_utils module Miscellaneous utilities: are we running as root, and is a debugger attached? +--- + .. automodule:: pyutils.misc_utils :members: :undoc-members: @@ -2328,6 +2349,8 @@ singleton classes that (optionally / conditionally) load their state from some external location and (optionally / conditionally) save their state to an external location at shutdown. +--- + .. automodule:: pyutils.persistent :members: :undoc-members: @@ -2340,6 +2363,8 @@ This module defines a helper that is invoked by the remote executor to run pickled code on a remote machine. It is used by code marked with `@parallelize(method=Method.REMOTE)` in the parallelize framework. +--- + .. automodule:: pyutils.remote_worker :members: :undoc-members: @@ -2356,6 +2381,8 @@ state that dynamically changes. These classes update their state (either automatically or when invoked to poll) and allow their callers to wait on state changes. +--- + .. automodule:: pyutils.state_tracker :members: :undoc-members: @@ -2367,6 +2394,8 @@ pyutils.stopwatch module This is a stopwatch context that just times how long something took to execute. +--- + .. automodule:: pyutils.stopwatch :members: :undoc-members: @@ -2379,6 +2408,8 @@ A bunch of utilities for dealing with strings. Based on a really great starting library from Davide Zanotti, I've added a pile of other string functions so hopefully it will handle all of your string-needs. +--- + .. automodule:: pyutils.string_utils :members: :undoc-members: @@ -2397,6 +2428,8 @@ Utilities for dealing with and creating text chunks. For example: - create a header line, - draw a box around some text. +--- + .. automodule:: pyutils.text_utils :members: :undoc-members: @@ -2407,6 +2440,8 @@ pyutils.unittest\_utils module Utilities to support smarter unit tests. +--- + .. automodule:: pyutils.unittest_utils :members: :undoc-members: @@ -2417,6 +2452,8 @@ pyutils.unscrambler module Unscramble scrambled English words quickly. +--- + .. automodule:: pyutils.unscrambler :members: :undoc-members: @@ -2427,6 +2464,8 @@ pyutils.zookeeper module A helper module for dealing with Zookeeper that adds some functionality. +--- + .. automodule:: pyutils.zookeeper :members: :undoc-members: @@ -2435,6 +2474,8 @@ A helper module for dealing with Zookeeper that adds some functionality. Module contents --------------- +--- + .. automodule:: pyutils :members: :undoc-members: