[prev in list] [next in list] [prev in thread] [next in thread]
List: python-ideas
Subject: Re: [Python-ideas] More Metadata for Variable Annotations
From: Wes Turner <wes.turner () gmail ! com>
Date: 2017-08-18 21:23:50
Message-ID: CACfEFw95LGikUcJyWM569wOF-hadF18_jM5MXnjTB4Zw-M5-aA () mail ! gmail ! com
[Download RAW message or body]
[Attachment #2 (multipart/alternative)]
PyContracts supports things like numpy array constraints
https://andreacensi.github.io/contracts/reference.html#contracts-language-reference
> You can specify that the value must be a list, and specify optional
constraints for its length and for its elements.
...
You mentioned JSONschema. For RDF (e.g. JSONLD, #CSVW), there are a number
of relevant variable annotations that should be useful as schema:
> A data table with 7 metadata header rows (column label, property URI
path, DataType, unit, accuracy, precision, significant figures):
https://wrdrd.github.io/docs/consulting/linkedreproducibility#csv-csvw-and-metadata-rows
On Friday, August 18, 2017, Bagrat Aznauryan <bagrat@aznauryan.org> wrote:
> # Abstract
>
> Before the holly PEP-526 the only option for type hints were comments. And
> before PEP-484 the docstrings were the main place where variable metadata
> would go. That variable metadata would include:
>
> * the type
> * the human-readable description
> * some value constraints (e.g. a range for integer variable)
>
> PEP-526 introduced the awesome syntax sugar, which made the first part of
> the metadata - the type, easily introspectable during the runtime. However,
> if you still need to add the description and the value constraints to the
> variable metadata, you still need to fallback to the docstring option.
>
> The idea is to make it possible to be able to include all of the mentioned
> metadata in the variable annotations.
>
> # Rationale
>
> Having the type specified using the supported annotation syntax and the
> rest of the metadata in the docstrings, adds duplication and complexity for
> further maintenance. Moreover, if you need the docstring-contained metadata
> to be used in the runtime, you need to implement a parser or pick one from
> existing ones which adds another dependency to your application.
>
> The need for the rest of the metadata other than the type, might be proven
> to be common. A typical example is generating the JSON Schema for a class,
> e.g. to be used for OpenAPI definition of your API.
>
> # Possible Solutions
>
> ## A wrapper
>
> The proposal is to introduce a new wrapper (probably a function), that
> will accept the type as the first positional argument and additional
> keyword arguments for metadata. The wrapper will map the keyword arguments
> to the type object as attributes and return it. The code would look like
> this:
>
> ```
> foo: wrapper(
> int,
> description="bar",
> minimum=0,
> maximum=100
> )
> ```
>
> Later, the metadata can be accessed as the annotation attributes, like
> e.g.:
>
> ```
> __annotations__['foo'].description
> ```
>
> ## Annotation as a tuple
>
> This solution does not require any code change in Python, but will force
> other tools change the parsing (e.g. mypy). The proposal is that when the
> annotation is optionally a tuple instance, use the first element as the
> type of the variable, and ignore the rest or treat as additional metadata.
> This will make it possible to add the metadata into a separate dictionary
> as the second element of the annotation tuple. For example:
>
> ```
> foo: (
> int,
> {
> description="bar",
> minimum=0,
> maximum=100
> }
> )
> ```
>
> The annotation will be stored as is, so to access the metadata in the
> runtime, one would need to explicitly access the second item of the
> annotation tuple.
>
> # Summary
>
> This option would help to have a well annotated code which will be
> self-descriptive and provide abilities to generate schemas and other
> definitions (e.g. OpenAPI) automatically and without duplication.
>
> The proposed solutions are definitely not perfect and not the main point
> of this email. The target is to describe the idea and motivation and start
> a discussion.
>
[Attachment #5 (text/html)]
PyContracts supports things like numpy array constraints<div><a \
href="https://andreacensi.github.io/contracts/reference.html#contracts-language-refere \
nce">https://andreacensi.github.io/contracts/reference.html#contracts-language-reference</a></div><div><br></div><div>> \
You can specify that the value must be a list, and specify optional constraints for \
its length and for its \
elements.</div><div><br></div><div>...</div><div><br></div><div>You mentioned \
JSONschema. For RDF (e.g. JSONLD, #CSVW), there are a number of relevant variable \
annotations that should be useful as schema:</div><div><br></div><div>> A data \
table with 7 metadata header rows (column label, property URI path, DataType, unit, \
accuracy, precision, significant figures):<br></div><div><br></div><div><a \
href="https://wrdrd.github.io/docs/consulting/linkedreproducibility#csv-csvw-and-metad \
ata-rows">https://wrdrd.github.io/docs/consulting/linkedreproducibility#csv-csvw-and-metadata-rows</a><br></div><div><br>On \
Friday, August 18, 2017, Bagrat Aznauryan <<a \
href="mailto:bagrat@aznauryan.org">bagrat@aznauryan.org</a>> wrote:<br><blockquote \
class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc \
solid;padding-left:1ex"><div dir="ltr"><div># \
Abstract</div><div><br></div><div>Before the holly PEP-526 the only option for type \
hints were comments. And before PEP-484 the docstrings were the main place where \
variable metadata would go. That variable metadata would \
include:</div><div><br></div><div>* the type</div><div>* the human-readable \
description</div><div>* some value constraints (e.g. a range for integer \
variable)</div><div><br></div><div>PEP-526 introduced the awesome syntax sugar, which \
made the first part of the metadata - the type, easily introspectable during the \
runtime. However, if you still need to add the description and the value constraints \
to the variable metadata, you still need to fallback to the docstring \
option.<br></div><div><br></div><div>The idea is to make it possible to be able to \
include all of the mentioned metadata in the variable \
annotations.</div><div><br></div><div># Rationale</div><div><br></div><div>Having the \
type specified using the supported annotation syntax and the rest of the metadata in \
the docstrings, adds duplication and complexity for further maintenance. Moreover, if \
you need the docstring-contained metadata to be used in the runtime, you need to \
implement a parser or pick one from existing ones which adds another dependency to \
your application.</div><div><br></div><div>The need for the rest of the metadata \
other than the type, might be proven to be common. A typical example is generating \
the JSON Schema for a class, e.g. to be used for OpenAPI definition of your \
API.</div><div><br></div><div># Possible Solutions</div><div><br></div><div>## A \
wrapper</div><div><br></div><div>The proposal is to introduce a new wrapper (probably \
a function), that will accept the type as the first positional argument and \
additional keyword arguments for metadata. The wrapper will map the keyword arguments \
to the type object as attributes and return it. The code would look like \
this:</div><div><br></div><div>```</div><div>foo: wrapper(</div><div> \
int,</div><div> description="bar",</div><div> \
minimum=0,</div><div> \
maximum=100</div><div>)</div><div>```</div><div><br></div><div>Later, the metadata \
can be accessed as the annotation attributes, like \
e.g.:</div><div><br></div><div>```</div><div>__annotations__['foo'].<wbr>description</div><div>```</div><div><br></div><div>## \
Annotation as a tuple</div><div><br></div><div>This solution does not require any \
code change in Python, but will force other tools change the parsing (e.g. mypy). The \
proposal is that when the annotation is optionally a tuple instance, use the first \
element as the type of the variable, and ignore the rest or treat as additional \
metadata. This will make it possible to add the metadata into a separate dictionary \
as the second element of the annotation tuple. For \
example:</div><div><br></div><div><div>```</div><div>foo: (</div><div> \
int,</div><div> {</div><div> description="bar",</div><div> \
minimum=0,</div><div> maximum=100</div><div> \
}</div><div>)</div><div>```</div><br></div><div>The annotation will be stored as is, \
so to access the metadata in the runtime, one would need to explicitly access the \
second item of the annotation tuple.</div><div><br></div><div># \
Summary</div><div><br></div><div>This option would help to have a well annotated code \
which will be self-descriptive and provide abilities to generate schemas and other \
definitions (e.g. OpenAPI) automatically and without \
duplication.</div><div><br></div><div>The proposed solutions are definitely not \
perfect and not the main point of this email. The target is to describe the idea and \
motivation and start a discussion.</div></div> </blockquote></div>
_______________________________________________
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/
[prev in list] [next in list] [prev in thread] [next in thread]
Configure |
About |
News |
Add a list |
Sponsored by KoreLogic