jip.options¶

This module contains the essential parts to handle tool and pipeline options both within the python API as well as from the command line.

The module provides access to two classes:

jip.options.Option

wraps a single option and its value

jip.options.Options

represents a set of options and provides indexed access to the options instances

The options are typically used to represent tool and pipeline inputs, outputs, and options. Because the JIP system needs to be able to identify which files are consumed and which files are created by a given tool, this information is encoded in the Option instance in their option_type attribute. The following types are supported:

jip.options.TYPE_INPUT¶: Option type to identify input options

jip.options.TYPE_OUTPUT¶: Option type to identify output options

jip.options.TYPE_OPTION¶: Option type to identify general options

The Options instance can be created manually or auto-generated from either a string or an argparse.ArgumentParser instance. The string parsing is done using a slightly modified version of the docopt library in order to support input and output blocks. You can use the Options.from_docopt() function to parse a docopt string:

import sys
from jip.options import Options

opts = Options.from_docopt('''
My tool description

Usage:
    tool -i <input> -o <output> [-b]

Inputs:
    -i, --input <input>     The input
                            [default: stdin]

Outputs:
    -o, --output <output>  The output
                            [default: stdout]

Options:
    -b, --boolean           A boolean flag
''')

assert opts['input'].raw() == sys.stdin
assert opts['output'].raw() == sys.stdout

In this example, we created all three option blocks, Inputs:, Outputs:, and Options: explicitly. The parser also detects inputs and output by their option name and if their default value is set to sys.stdin or sys.stdout.

The same options can also be created using pythons argparse.ArgumentParser:

import argparse
parser = argparse.ArgumentParser("tool")
parser.add_argument('-i', '--input', default=sys.stdin)
parser.add_argument('-o', '--output', default=sys.stdout)
parser.add_argument('-b', '--boolean', action="store_true")

opts = Options.from_argparse(parser, inputs=['input'], outputs=['output'])
assert opts['input'].raw() == sys.stdin
assert opts['output'].raw() == sys.stdout

We again specify the inputs and outputs explicitly, but this time in the call to the from_argparse method. There is currently no way other than the option name and its default value to indicate inputs and output using argparse other than specifying them explicitly. If you are using the JIP library as an API, it likely that you define your tool functions or classes using the decorators. These all allow you to specify the input and output options explicitly.

class jip.options.Options(source=None)¶

Container instance for a set of options.

The options container offers a set of static method to create Options instances from docopt options or arparse instances. In addition to creation, the options instance allows for simplified access to options using the options name, for example:

input = options['input']

This will assign the jip.options.Option instance to input. The options instance is also iterable in order to quickly go through the option instances, i.e.:

for o in options:
    print o.name

In addition, get_default_input() and get_default_output() can be used to access the default options configured for input and output. If you are interested in a specify option type, you can use the get_by_type() function to get an iterator over the options of the specified type:

for inopt in options.get_by_type(TYPE_INPUT):
    print inopt.name

Option values can also be set directly using the dictionary notation. For example:

>>> opts = Options()
>>> opts.add_input('input')
<no-source>.input[]
>>> opts['input'] = "data.txt"

This assigned the value data.txt to the input option.

If a source is specified, this becomes the source instance for all options added.

__init__(source=None)¶

add(option)¶

Adds an options to the options set. The source is applied to an option added.

Parameters:	option (`Option`) – the option to add to the set

add_input(name, value=None, nargs=None, hidden=True, **kwargs)¶

Add additional, hidden, input option. The default value for this option is None, but you can pass a value here that will be set after the option is added.

If no value is specified, the option by default is a single value option. You can overwrite this by specifying the nargs argument.

By default, the new option is hidden and will not be listed in the default options printer. You can overwrite this with the hidden flag.

Parameters:	name (string) – the name of the new option value – optional value applied to the option nargs – multiplicity specifier. If this is not set explicitly but a value is provided, the value is inspected to guess a multiplicity. hidden – set this to False to create a visible option kwargs – all additional keyword arguments are passed to the new option as they are
Returns:	the added option
Return type:	`jip.options.Option`

add_option(name, value=None, nargs=None, hidden=True, type='option', **kwargs)¶

Add additional, hidden, option. The default value for this option is None, but you can pass a value here that will be set after the option is added.

If no value is specified, the option by default is a single value option. You can overwrite this by specifying the nargs argument.

By default, the new option is hidden and will not be listed in the default options printer. You can overwrite this with the hidden flag.

Parameters:	name (string) – the name of the new option value – optional value applied to the option nargs – multiplicity specifier. If this is not set explicitly but a value is provided, the value is inspected to guess a multiplicity. hidden – set this to False to create a visible option kwargs – all additional keyword arguments are passed to the new option as they are
Returns:	the added option
Return type:	`jip.options.Option`

add_output(name, value=None, nargs=None, hidden=True, **kwargs)¶

Add additional, hidden, output option. The default value for this option is None, but you can pass a value here that will be set after the option is added.

If no value is specified, the option by default is a single value option. You can overwrite this by specifying the nargs argument.

By default, the new option is hidden and will not be listed in the default options printer. You can overwrite this with the hidden flag.

Parameters:	name (string) – the name of the new option value – optional value applied to the option nargs – multiplicity specifier. If this is not set explicitly but a value is provided, the value is inspected to guess a multiplicity. hidden – set this to False to create a visible option kwargs – all additional keyword arguments are passed to the new option as they are
Returns:	the added option
Return type:	`jip.options.Option`

classmethod from_argparse(parser, inputs=None, outputs=None, source=None)¶

Create Options from a given argparse parser.

The inputs and outputs can be set to options names to set a specific type.

If no input or output options are specified explicitly, options named input are assigned as the default input and options named output are assigned as default output output.

In addition, the default types are checked and options where the default points to stdin or stdout are given the type TYPE_INPUT and TYPE_OUTPUT respectively.

Parameters:	parser – the argparse instance inputs – list of names of `TYPE_INPUT` options output – list of names of `TYPE_OUTPUT` options
Returns:	a new Options instance
Return type:	`Options`

classmethod from_docopt(doc, inputs=None, outputs=None, source=None)¶

Create Options from a help string using the docopt parser

The inputs and outputs can be set to options names to set a specific type.

If no input or output options are specified explicitly, options named input are assigned as the default input and options named output are assigned as default output output.

In addition, the default types are checked and options where the default points to stdin or stdout are given the type TYPE_INPUT and TYPE_OUTPUT respectively.

Here is an example of how an Options instance can be created from a doc string:

Options.from_docopt('''            usage:
    tool -i <input>
options:
    -i, --input <input>  the input option
''')

Parameters:	parser – the argparse instance inputs – list of names of `TYPE_INPUT` options output – list of names of `TYPE_OUTPUT` options
Returns:	a new Options instance
Return type:	`Options`

get_by_type(options_type)¶

Generator function that yields all options of the specified type. The type should be one of TYPE_OUTPUT, TYPE_INPUT or TYPE_OPTION.

Parameters:	options_type (string) – the type
Returns:	generator of all options of the specified type
Return type:	list of `Option`

get_default_input()¶

Returns the first input option that is found in the list of options that has a non-null value. If no input option is found, a LookupError is raised

Raises LookupError:
Returns:	the default input option
Return type:	`Option`
	if no default option was found

get_default_output()¶

Returns the first output option that is found in the list of options that has a non-null value. If no output option is found, a LookupError is raised

Raises LookupError:
Returns:	the default output option
Return type:	`Option`
	if no default option was found

glob_inputs()¶: Resolve file wildcards on input options.

help()¶: Returns the help message :returns: the help message :rtype: string

make_absolute(path)¶

Render input and output options absolute. Output options are made absolute relative the given path and input options are made absolute relative to the current working directory

Parameters:	path (string) – the parent path for output options

parse(args)¶

Parse the given arguments and full the options values. A ParserException is raised if help is requested (-h or –help) or if an option error occurs. The exceptions error message is set accordingly.

The given args list should contain all command line argument to parse without the program name.

Parameters:	args (list) – the arguments

to_cmd()¶

Render all non hidden options to a single command line representation. For example:

>>> from jip.options import Options
>>> opts = Options()
>>> opts.add_input("input", short="-i", value="data.in", hidden=False)
<no-source>.input['data.in']
>>> opts.add_output("output", short="-o", value="data.out", hidden=False)
<no-source>.output['data.out']
>>> assert opts.to_cmd() == '-i data.in -o data.out'

Returns:	command line representation of all non-hidden options
Return type:	string

to_dict(raw=False)¶

Convert the options to a read-only dictionary pointing to the raw values of the options.

Returns:	read-only dictionary of the options raw values

usage()¶

Returns the usage message

Returns:	usage message
Return type:	string

validate()¶: Validate all options

class jip.options.Option(name, short=None, long=None, type=None, nargs=None, default=None, value=None, required=False, streamable=None, hidden=False, join=' ', option_type='option', const=None, sticky=False)¶

This class manages a single option of a JIP Tool.

The option instance itself it usually wrapped and accessed through a Options instance, and provides the basic functionality to work with its value and its command line representation.

The most commonly used properties of an option are its name and its value. In addition the option instance carries various other information, for example,the option type, its value type, its multiplicity.

Internally, each instance can carry a set of values, independent of its multiplicity (nargs) setting. In fact, option value are always stored in a list. This is used in pipeline extensions and expansions and you should keep it in mind when accessing the option value.

Access can be done in three different ways. The most prominent one is the options get() method, which returns a string representation of the options value. Please read the methods description to understand the translation rules.

The Options constructor will automatically translate a value set to the string stdin, stdout or stderr to the corresponding system streams.

Parameters:

name – the options name
short – the short option name, i.e, -h
long – the long options name, i.e., --help
type – the values type, for example, int or string
nargs – number of supported arguments. Supports 0 for boolean options, 1 for options that take a single value, + for options that take at least on argument, and * for options that take none or more arguments.
default – the options default value
value – the options initial value
required – set to True to make the option non-mandatory
hidden – mark the option as hidden
join – specify a string that is used to join list of elements for string representations
streamable – enable streams for this option
option_type – the option type, one of TYPE_INPUT, TYPE_OUTPUT or TYPE_OPTION
sticky – mark the option as sticky. Sticky option values are ignored during a cleanup

append(value)¶

Append a value to the list of option values.

Parameters:	value – the value to append

check_file()¶

Validate this option and check that, if the options is not set through a dependency, all string values represent existing files.

Raises ValueError:
	if an expected file is not found

check_files()¶: Alias for check_file()

copy()¶

Create a clone of this option instance

Returns:	clone of this option
Return type:	`Option`

expand()¶

Returns the raw values of the list but expanded to contain options multiple times in case the _value contains options.

This can be used in fanout mode for pipeline nodes where you need to resolve the full list of values n times.

Returns:	list of expanded values

get(converter=<type 'str'>)¶

Get a representation for the current value, default to string representation

The get method translates the current value in the following way:

if the options nargs is set to 0 and the option represents a boolean flag, an empty string is returned

if the options nargs is set to 1 and the option should contains a single value but contains more than one value, a ValueError is raised.

if the options nargs allows for a list of values, each value is resolved independently and the list is joined using the options join string.

if the option is required and no value is set, an ParseException is raised.

Option values are resolved before returned and boolean values and file streams are resolved to an empty string. All other values are resolved to their string representations.

Parameters:	converter – the converter function, defaults to `str`
Returns:	string representation of the current option value
Return type:	string
Raises:	ValueError – if the option contains more elements that allowed ParseException – if the option is required but no value is set

get_opt()¶

Return the short or long representation of this option, starting with the short option. If that is not set, the options long name is returned.

Returns:	options short or long name
Return type:	string

glob()¶: Resolve wildcards used in this option. The results is sorted by name and applied as values to this option.

is_dependency()¶

Returns true if this options value is coming from another tool execution, hence this option depends on another option and the tool containing this option depends on another tool.

Returns:	True if this option has a dependency to another option from another tool
Return type:	boolean

is_list()¶

Return true if this option takes lists of values.

Returns:	True if this option accepts a list of values

is_stream()¶

Return true if the current value is a stream or a list of streams.

Returns:	True if current value is a stream

make_absolute(path)¶

Converts the option values to absolute paths relative to the given parent path.

Parameters:	path (string) – the parent path

raw()¶

Get raw value(s) wrapped by this options.

No require checks are performed.

If the options carries a single value and nargs == 1, this first value in the options list is returned. Otherwise the list of values is returned.

Returns:	the raw options value depending on `nargs` a single value or a list of values

set(new_value)¶

Set the options value

Parameters:	new_value – the new value

to_cmd()¶

Return the command line representation for this option. An exception is raised if the option setting is not valid. For example:

>>> o = Option("input", "-i", "--long", value="data.csv")
>>> assert o.to_cmd() == '-i data.csv'

Options with False or None value are represented as empty string.

Returns:	the cull command line representation of this option

validate()¶

Validate the option and raise a ValueError if the option is required but no value is set.

Raises ValueError:
	if the option is `required` but not set

value¶

The list of values wrapped by this option.

The list of values is rendered and resolved on access type. For this, the options render_context must be set. This context is then used to render any string value as a template.

Getter:	Returns the fully rendered and resolved list of current values
Setter:	Set the current option value
Type:	single object or list of objects

exception jip.options.ParserException(message, options=None, status=0)¶: Exception raised by the Options argument parser

jip.options¶

Table Of Contents

Links

This Page

Navigation

jip.options¶

Table Of Contents

Links

This Page

Quick search