Welcome to cliapp’s documentation!

cliapp is a Python framework for Unix-like command line programs, which typically have the following characteristics:

  • non-interactive
  • the programs read input files named on the command line, or the standard input
  • each line of input is processed individually
  • output is to the standard output
  • there are various options to modify how the program works
  • certain options are common to all: --help, --version

Programs like the above are often used as filters in a pipeline. The scaffoling to set up a command line parser, open each input file, read each line of input, etc, is the same in each program. Only the logic of what to do with each line differs.

cliapp is not restricted to line-based filters, but is a more general framework. It provides ways for its users to override most behavior. For example:

  • you can treat command line arguments as URLs, or record identfiers in a database, or whatever you like
  • you can read input files in whatever chunks you like, or not at all, rather than forcing a line-based paradigm

Despite all the flexibility, writing simple line-based filters remains very straightforward. The point is to get the framework to do all the usual things, and avoid repeating code across users of the framework.

Example

class ExampleApp(cliapp.Application):

    def add_settings(self):
        self.settings.string_list(['pattern', 'e'],
                                  'search for regular expression PATTERN',
                                  metavar='REGEXP')

    # We override process_inputs to be able to do something after the last
    # input line.
    def process_inputs(self, args):
        self.matches = 0
        cliapp.Application.process_inputs(self, args)
        self.output.write('There were %s matches.\\n' % self.matches)

    def process_input_line(self, name, line):
        for pattern in self.settings['pattern']:
            if pattern in line:
                self.output.write('%s:%s: %s' % (name, self.lineno, line))
                self.matches += 1
                logging.debug('Match: %s line %d' % (name, self.lineno))


if __name__ == '__main__':
    ExampleApp().run()

Walkthrough

Every application should be a class that subclasses cliapp.Application. The subclass should provide specific methods. Read the documentation for the cliapp.Application class to see all methods, but a rough summary is here:

  • the settings attribute is the cliapp.Settings instance used by the application
  • override add_settings to add new settings for the application
  • override process_* methods to override various stages in how arguments and input files are processed
  • override process_args to decide how each argument is processed; by default, this called process_inputs or handles subcommands
  • process_inputs calls process_input (note singular) for each argument, or on - to process standard input if no files are named on the command line
  • process_input calls open_input to open each file, then calls process_input_line for each input line
  • process_input_line does nothing, by default

This cascade of overrideable methods is started by the run method, which also sets up logging, loads configuration files, parses the command line, and handles reporting of exceptions. It can also run the rest of the code under the Python profiler, if the appropriate environment variable is set.

Logging

Logging support: by default, no log file is written, it must be requested explicitly by the user. The default log level is info.

Subcommands

Sometimes a command line tool needs to support subcommands. For example, version control tools often do this: git commit, git clone, etc. To do this with cliapp, you need to add methods with names like cmd_commit and cmd_clone:

class VersionControlTool(cliapp.Application):

    def cmd_commit(self, args):
        pass
    def cmd_clone(self, args):
        pass

If any such methods exist, cliapp automatically supports subcommands. The name of the method, without the cmd_ prefix, forms the name of the subcommand. Any underscores in the method name get converted to dashes in the command line. Case is preserved.

Subcommands may also be added using the add_subcommand method.

All options are global, not specific to the subcommand. All non-option arguments are passed to the method in its only argument.

Subcommands are implemented by the process_args method. If you override that method, you need to support subcommands yourself (perhaps by calling the cliapp implementation).

Manual pages

cliapp provides a way to fill in a manual page template, in troff format, with information about all options. This allows you to write the rest of the manual page without having to remember to update all options. This is a compromise between ease-of-development and manual page quality.

A high quality manual page probably needs to be written from scratch. For example, the description of each option in a manual page should usually be longer than what is suitable for --help output. However, it is tedious to write option descriptions many times.

To use this, use the --generate-manpage=TEMPLATE option, where TEMPLATE is the name of the template file. See example.1 in the cliapp source tree for an example.

Profiling support

If sys.argv[0] is foo, and the environment variable FOO_PROFILE is set, then the execution of the application (the run method) is profiled, using cProfile, and the profile written to the file named in the environment variable.

Reference manual

exception cliapp.AppException

Base class for application specific exceptions.

Any exceptions that are subclasses of this one get printed as nice errors to the user. Any other exceptions cause a Python stack trace to be written to stderr.

class cliapp.Settings(progname, version, usage=None, description=None, epilog=None)

Settings for a cliapp application.

You probably don’t need to create a settings object yourself, since cliapp.Application does it for you.

Settings are read from configuration files, and parsed from the command line. Every setting has a type, name, and help text, and may have a default value as well.

For example:

settings.boolean(['verbose', 'v'], 'show what is going on')

This would create a new setting, verbose, with a shorter alias v. On the command line, the options --verbose and -v would work equally well. There can be any number of aliases.

The help text is shown if the user uses --help or --generate-manpage. You can use the metavar keyword argument to set the name shown in the generated option lists; the default name is whatever optparse decides (i.e., name of option).

Use load_configs to read configuration files, and parse_args to parse command line arguments.

The current value of a setting can be accessed by indexing the settings class:

settings['verbose']

The list of configuration files for the appliation is stored in config_files. Add or remove from the list if you wish. The files need to exist: those that don’t are silently ignored.

as_cp()

Return a ConfigParser instance with current values of settings.

boolean(names, help, default=False, **kwargs)

Add a setting with a boolean value.

build_parser(configs_only=False, arg_synopsis=None, cmd_synopsis=None)

Build OptionParser for parsing command line.

bytesize(names, help, default=0, **kwargs)

Add a setting with a size in bytes.

The user can use suffixes for kilo/mega/giga/tera/kibi/mibi/gibi/tibi.

choice(names, possibilities, help, **kwargs)

Add a setting which chooses from list of acceptable values.

An example would be an option to set debugging level to be one of a set of accepted names: debug, info, warning, etc.

The default value is the first possibility.

dump_config(output)
integer(names, help, default=0, **kwargs)

Add an integer setting.

load_configs(open=<built-in function open>)

Load all config files in self.config_files.

Silently ignore files that do not exist.

parse_args(args, parser=None, suppress_errors=False, configs_only=False, arg_synopsis=None, cmd_synopsis=None)

Parse the command line.

Return list of non-option arguments. args would usually be sys.argv[1:].

require(name)

Raise exception if setting has not been set.

Option must have a value, and a default value is OK.

set_from_raw_string(name, raw_string)

Set value of a setting from a raw, unparsed string value.

string(names, help, default='', **kwargs)

Add a setting with a string value.

string_list(names, help, default=None, **kwargs)

Add a setting which have multiple string values.

An example would be an option that can be given multiple times on the command line, e.g., “–exclude=foo –exclude=bar”.

class cliapp.Application(progname=None, version='0.0.0', description=None, epilog=None)

A framework for Unix-like command line programs.

The user should subclass this base class for each application. The subclass does not need code for the mundane, boilerplate parts that are the same in every utility, and can concentrate on the interesting part that is unique to it.

To start the application, call the run method.

The progname argument sets tne name of the program, which is used for various purposes, such as determining the name of the configuration file.

Similarly, version sets the version number of the program.

description and epilog are included in the output of --help. They are formatted to fit the screen. Unlike the default behavior of optparse, empty lines separate paragraphs.

add_settings()

Add application specific settings.

add_subcommand(name, func, arg_synopsis=None)

Add a subcommand.

Normally, subcommands are defined by add cmd_foo methods to the application class. However, sometimes it is more convenient to have them elsewhere (e.g., in plugins). This method allows doing that.

The callback function must accept a list of command line non-option arguments.

cleanup()

Clean up after process_args.

This method is called just after process_args. By default it does nothing, but subclasses may override it with a suitable implementation. This is easier than overriding process_args itself.

dump_memory_profile(msg)

Log memory profiling information.

Get the memory profiling method from the dump-memory-profile setting, and log the results at DEBUG level. msg is a message the caller provides to identify at what point the profiling happens.

open_input(name, mode='r')

Open an input file for reading.

The default behaviour is to open a file named on the local filesystem. A subclass might override this behavior for URLs, for example.

The optional mode argument speficies the mode in which the file gets opened. It should allow reading. Some files should perhaps be opened in binary mode (‘rb’) instead of the default text mode.

parse_args(args, configs_only=False)

Parse the command line.

Return list of non-option arguments.

process_args(args)

Process command line non-option arguments.

The default is to call process_inputs with the argument list, or to invoke the requested subcommand, if subcommands have been defined.

process_input(name, stdin=<open file '<stdin>', mode 'r' at 0x559cd020>)

Process a particular input file.

The stdin argument is meant for unit test only.

process_input_line(filename, line)

Process one line of the input file.

Applications that are line-oriented can redefine only this method in a subclass, and should not need to care about the other methods.

process_inputs(args)

Process all arguments as input filenames.

The default implementation calls process_input for each input filename. If no filenames were given, then process_input is called with - as the argument name. This implements the usual Unix command line practice of reading from stdin if no inputs are named.

The attributes fileno, global_lineno, and lineno are set, and count files and lines. The global line number is the line number as if all input files were one.

run(args=None, stderr=<open file '<stderr>', mode 'w' at 0x559cd0d0>, sysargv=['/usr/bin/sphinx-build', '-b', 'html', '-d', '_build/doctrees', '.', '_build/html'], log=<function critical at 0x876e144>)

Run the application.

runcmd(argv, *args, **kwargs)

Run external command or pipeline.

Example: ``runcmd([‘grep’, ‘foo’], [‘wc’, ‘-l’],
feed_stdin=’foo

bar ‘)``

Return the standard output of the command.

Raise cliapp.AppException if external command returns non-zero exit code. *args and **kwargs are passed onto subprocess.Popen.

runcmd_unchecked(argv, *argvs, **kwargs)

Run external command or pipeline.

Return the exit code, and contents of standard output and error of the command.

See also runcmd.

setup()

Prepare for process_args.

This method is called just before process_args. By default it does nothing, but subclasses may override it with a suitable implementation. This is easier than overriding process_args itself.

setup_logging()

Set up logging.

Indices and tables

Table Of Contents

This Page