Declarative Parser - showcase¶
Powerful like click, integrated like argparse, declarative as sqlalchemy. MIT licenced.
Features:¶
Installation and support¶
To install, use pip (which is installed by default with Python 3.6):
python3 -m pip install declarative_parser
In case of any problem, please open an issue on GitHub repository page. Please, feel free to star and contribute if you find this package interesting.
Built on top of argparse¶
The basic API of the DeclarativeParser is compatible with argparse, so you do not need to learn from start.
This is the arparse way:
import argparse
parser = argparse.ArgumentParser()
parser.add_argument("square", help="display a square of a given number")
args = parser.parse_args()
print(args.square**2)
This is the declarative way:
from declarative_parser import Parser, Argument
class MyParser(Parser):
square = Argument(help='display a square of a given number')
parser = MyParser()
args = parser.parse_args()
print(args.square**2)
Nested parsers¶
DeclarativeParser allows you to nest parsers one in another, just like ‘git commit’ or ‘git push’:
class TheParser(Parser):
help = 'This takes only one argument, but it is required'
arg = Argument(optional=False, help='This is required')
class MyParser(Parser):
description = 'This should be a longer text'
my_argument = Argument(type=int, help='some number')
my_sub_parser = TheParser()
epilog = 'You can create a footer with this'
To execute the parser use:
parser = MyParser()
# The commands will usually be `sys.argv[1:]`
commands = '--my_argument 4 my_sub_parser value'.split()
namespace = parser.parse_args(commands)
Resultant namespace is a normal argparse.Namespace
assert namespace.my_argument == 4
assert namespace.my_sub_parser.arg == 'value'
Parallel parsers¶
You can have multiple sub-parsers on the same level, like:
supported_formats = ['png', 'jpeg', 'gif']
class InputOptions(Parser):
path = Argument(type=argparse.FileType('rb'), optional=False)
format = Argument(default='png', choices=supported_formats)
class OutputOptions(Parser):
format = Argument(default='jpeg', choices=supported_formats)
scale = Argument(type=int, default=100, help='Rescale image to % of original size')
class ImageConverter(Parser):
description = 'This app converts images'
verbose = Argument(action='store_true')
input = InputOptions()
output = OutputOptions()
parser = ImageConverter()
commands = '--verbose input image.png output --format gif --scale 50'.split()
namespace = parser.parse_args(commands)
assert namespace.input.format == 'png'
assert namespace.output.format == 'gif'
As simple as it looks!
Production pattern¶
Do you want to introduce sophisticated behaviour to your parser, but keep the logic away from the core of your app? DeclarativeParser enables you to add “produce” method to each parser, which will transform the arguments namespace in a way you wish it to be done!
Have a look on this example of advanced file parsing:
from declarative_parser import Parser, Argument
from declarative_parser.types import Slice, Indices, Range, one_of
def slice_file(file, columns_selector=None, delimiter=None):
pass
class FileSubsetFactory(Parser):
"""Parse user options and load desired part of given file.
The files should come in Delimiter Separated Values format
(like .csv or .tsv). The default delimiter is a tab character.
To use only a subset of columns from given file,
specify column numbers with --columns.
"""
file = Argument(
type=argparse.FileType('r'),
optional=False
)
columns = Argument(
# we want to handle either ":4", "5:" or even "1,2,3"
type=one_of(Slice, Indices, Range),
# user may (but do not have to) specify columns
# to be extracted from given file(s).
help='Columns to be extracted from files: '
'either a comma delimited list of 0-based numbers (e.g. 0,2,3) '
'or a range defined using Python slice notation (e.g. 3:10). '
'Columns for each of files should be separated by space.'
)
delimiter = Argument(
default='\t',
help='Delimiter of the provided file(s). Default: tabulation mark.'
)
def produce(self, unknown_args=None):
opts = self.namespace
opts.file_subset = slice_file(
opts.file,
columns_selector=opts.columns.get_iterator if opts.columns else None,
delimiter=opts.delimiter,
)
return opts
After parsing file_subset will become a part of your resultant namespace.
Batteries included¶
Powerful validation, additional types and more.
Do you want to allow user to provide distinct options for each of provided files, but not to validate the number of arguments every single time? No problem, just use as_many_as=files.
class AdvancedFileFactory(Parser):
"""Parse user options and load given file(s).
To use only a subset of columns from files(s) specify column numbers
(--columns) or column names (--names) of desired columns.
"""
files = Argument(
type=argparse.FileType('r'),
# at least one file is always required
nargs='+',
optional=False
)
names = Argument(
type=dsv(str),
nargs='*',
as_many_as=files,
help='Names of columns to be extracted from the file. '
'Names are determined from the first non-empty row. '
'Use a comma to separate column names. '
'Column names for each of files should be separated by space.'
)
columns = Argument(
# we want to handle either ":4", "5:" or even "1,2,3"
type=one_of(Slice, Indices, Range),
# user may (but do not have to) specify columns
# to be extracted from given file(s).
nargs='*',
as_many_as=files,
help='Columns to be extracted from files: '
'either a comma delimited list of 0-based numbers (e.g. 0,2,3) '
'or a range defined using Python slice notation (e.g. 3:10). '
'Columns for each of files should be separated by space.'
)
def produce(self, unknown_args=None):
opts = self.namespace
file_chunks = []
for i, file_obj in enumerate(opts.files):
file_chunks.append(slice_file(
opts.file,
names=opts.names[i] if opts.names else None,
columns_selector=opts.columns[i].get_iterator if opts.columns else None,
))
opts.file_subset = merge_chunks(file_chunks)
return opts
To further explore additional types, see: Utility types.
Arguments deduction (typing, docstrings, kwargs)¶
What about automatic parser creation? You can use ClassParser of FunctionParser for that!
Just feed constructor_parser.ClassParser
with your main class and it will take care of it.
Arguments defined in your __init__ and in body of your class (i.e. class variables) will be used to create a parser;
Type annotations (as long as based on real types, not typing module) will be used to define types of your arguments;
Default: from keyword arguments. Positional arguments will be always required.
Docstring descriptions will be used to provide help for your arguments.
Following docstring formats are supported: Google, NumPy and reStructuredText, with the default being Google. To change the format, pass docstring_type=’numpy’ or docstring_type=’rst’ respectively.
When an argument is defined in both: __init__ and declarative_parser.Argument()
variable, the class variable overwrites the values from __init__ .
import argparse
from declarative_parser import Argument
from declarative_parser.constructor_parser import ClassParser
class MyProgram:
database = Argument(
type=argparse.FileType('r'),
help='Path to file with the database'
)
def __init__(self, text: str, threshold: float=0.05, database=None):
"""My program does XYZ.
Arguments:
threshold: a floating-point value defining threshold, default 0.05
database: file object to the database if any
"""
print(text, threshold, None)
parser = ClassParser(MyProgram)
options = parser.parse_args()
program = parser.constructor(**vars(options))
And it works quite intuitively:
$ ./my_program.py test --threshold 0.6
test 0.6 None
$ ./my_program.py test --threshold f
usage: my_program.py [-h] [--database DATABASE] [--threshold THRESHOLD] text {} ...
my_program.py: error: argument --threshold: invalid float value: 'f'
$ ./my_program.py --threshold 0.6
usage: my_program.py [-h] [--database DATABASE] [--threshold THRESHOLD] text {} ...
my_program.py: error: the following arguments are required: text
You could then implement run method and call program.run() to start you application.
Likewise, constructor_parser.FunctionParser
will create a parser using your function:
def calc_exponent(base: float, exponent: int=2):
return base ** exponent
parser = FunctionParser(calc_exponent)
commands = '2 --exponent 3'.split()
options = parser.parse_args(commands)
result = parser.constructor(**vars(options))
assert result == 2 * 2 * 2
Actions¶
What if you only want to show licence of your program? or version? It there a need to write a separate logic?
DeclarativeParser gives you utility decorator: @action which utilizes the power of argparse.Action
,
leaving behind the otherwise necessary boilerplate code.
__version__ = 2.0
import argparse
from declarative_parser import action
from declarative_parser.constructor_parser import ConstructorParser
class MyProgram:
def __init__(self, threshold: float=0.05):
"""My program does XYZ.
Arguments:
threshold: a floating-point value, default 0.05
"""
pass
@action
def version(options):
print(__version__)
parser = ConstructorParser(MyProgram)
options = parser.parse_args()
program = parser.constructor(**vars(options))
The execution of an action will (by default) cause the program to exit immediately when finished. See following run as example:
$ ./my_program.py --version
2.0
$
Acknowledgements¶
This module was originally developed for https://github.com/kn-bibs/pathways-analysis project. Big thanks go to @hansiu, @sienkie and @pjanek for early feedback, inspiration and some valuable insights.