Python package template project for kick-starting new Python projects.

AlexIoannides, updated 🕥 2022-12-08 01:18:01

Python Package Template Project

The py-template-project package allows users to download the contents of this GiHub repository, containing a skeleton Python package project to be used as a template for kick-starting development of any type of Package; destined for upload to PyPI, or just for local install using Pip. The downloaded package includes the following components to aid rapid development without having to spend time cloning existing set-ups from other projects:

a minimal setup.py file;
testing with PyTest;
documentation (HTML and PDF) generated using Sphinx with auto-documentation setup;
an entry-point that allows the package to execute functions directly from the command line - e.g. to start a server, interact with a user, download a GitHub repository, etc.; and,
automated testing and deployment using Travis CI.

A description of how to work with (and modify) each of these components, is provided in more detail in the sections that follow-on below, as well as in the documentation and within the example code bundled with the package.

This is obviously a opinionated view of how a Python package project ought to be structured, based largely on my own experiences and requirements. Where I have needed guidance on this subject, I have leant heavily on the advice given by the Python Packaging Authority (PyPA) and used the excellent Requests and Flask projects as references for 'best practices'.

Installing

Install and update using pip:

bash pip3 install py-template-project

Downloading a Python Package Template Project

To down load the latest version of the Python Package Template project located in this GiHub repository, execute the following command from the command line:

bash py-package-template install

This will be downloaded to the current directory and will contain the following directory structure:

The Python Package Template Project

We now describe the various components of the template project and the workflows associated with it. The template package project contains two modules to get things started:

curves.py
entry_points.py

The curves.py module contains sample code for modelling economic supply and demand curves and makes for a useful demonstration of how Python type annotation and interface definition via abstract base classes, can make code easier to read, document and reason about (I am a big fan). The test suite for this module is contained in the tests folder and demonstrates how to get up-and-running with PyTest.

The entry_points.py module is referenced in the setup.py file via the entry_points definitions:

python entry_points={ 'console_scripts': ['py-package-template=py_pkg.entry_points:main'], }

It enables the declared entry point - py_pkg.entry_points.main - to be invoked when py-package-template is called from the command line. This is what enables the template project to be downloaded programmatically (check the code for the full details). This could easily be extended to start a server (e.g. using Flask), or run any other type of script.

Project Dependencies

We use pipenv for managing project dependencies and Python environments (i.e. virtual environments). These dependencies are not to be confused with the package installation dependencies for the package under developement - i.e. those that need to be defined in the install_requires section of setup.py. All of the direct packages dependencies required to run the project's code (e.g. NumPy for tensors), as well as all the packages used during development (e.g. flake8 for code linting and IPython for interactive console sessions), are described in the Pipfile. Their precise downstream dependencies are crystallised in Pipfile.lock, which is used to guarentee repeatable (i.e. deterministic) builds.

Installing Pipenv

To get started with Pipenv, first of all download it - assuming that there is a 'global' version of Python available on your system and on the PATH, then this can be achieved by running the following command,

bash pip3 install pipenv

For more information, including advanced configuration options, see the official pipenv documentation.

Installing this Projects' Dependencies

Make sure that you're in the project's root directory (the same one in which Pipfile resides), and then run,

bash pipenv install --dev

This will install all of the direct project dependencies as well as the development dependencies (the latter a consequence of the --dev flag). To add and remove dependencies as required for your new project, use pipenv install and pipenv uninstall as required, using the --dev flag for development-only dependencies.

Running Python and IPython from the Project's Virtual Environment

In order to open a Python REPL using within an environment that precisely mimics the one the project is being developed with, use Pipenv from the command line as follows,

bash pipenv run python3

The python3 command could just as well be ipython3.

Automatic Loading of Environment Variables

Pipenv will automatically pick-up and load any environment variables declared in the .env file, located in the package's root directory. For example, adding,

bash SPARK_HOME=applications/spark-2.3.1/bin

Will enable access to this variable within any Python program, via a call to os.environ['SPARK_HOME']. Note, that if any security credentials are placed here, then this file must be removed from source control - i.e. add .env to the .gitignore file to prevent potential security risks.

Pipenv Shells

Prepending pipenv to every command you want to run within the context of your Pipenv-managed virtual environment, can get (very) tedious. This can be avoided by entering into a Pipenv-managed shell,

bash pipenv shell

Which is equivalent to 'activating' the virtual environment. Any command will now be executed within the virtual environment. Use exit to leave the shell session.

Running Unit Tests

All test have been written using the PyTest package. Tests are kept in the tests folder and can be run from the command line by - e.g. by invoking,

bash pipenv run pytest

The test suite is structured as an independent Python package as follows:

The conftest.py module is used by PyTest - in this particular instance for loading test data and building objects that will then be used by potentially many other tests. These are referred to as 'fixtures' in PyTest - more details can be found here.

Linting Code

I prefer to use flake8 for style guide enforcement. This can be invoked from the command line by running,

bash pipenv run flake8 py_pkg

Flake8 could easily be swapped-out for another tool by using Pipenv as described above.

Static Type Checking

We have used the Python type annotation framework, together with the MyPy package, to perform static type checks on the codebase. Analogous to any linter or unit testing framework, MyPy can be run from the command line as follows,

bash pipenv run python -m mypy py_pkg/*.py

MyPy options for this project can be defined in the mypy.ini file that MyPy will look for by default. For more information on the full set of options, see the mypy documentation.

Examples of type annotation and type checking for library development can be found in the py_pkg.curves.py module. This should also be cross-referenced with the improvement to readability (and usability) that this has on package documentation.

Documentation

The documentation in the docs folder has been built using Sphinx. We have used the default 'quickstart' automatic configuration, which was originally triggered by executing,

bash pipenv run sphinx-quickstart

The output is based primarily on the Docstrings in the source code, using the autodoc extension within Sphinx (specified during the 'quickstart'). The contents for the entry point into the docs (index.html), is defined in the index.rst file, which itself imports the modules.rst file that lists all of the modules to document. The documentation can be built by running the following command,

bash pipenv run sphinx-build -b html docs/source docs/build_html

The resulting HTML documentation can be accessed by opening docs/build_html/index.html in a web browser.

My preferred third party theme from Read the Docs has also been used, by installing the sphinx_rtd_theme as a development dependency and modifying docs/source/config.py as follows:

python import sphinx_rtd_theme html_theme = "sphinx_rtd_theme" html_theme_path = [sphinx_rtd_theme.get_html_theme_path()]

Creating a PDF Version Using LaTeX

So long as a LaTex distribution is present on your system (e.g. MikTeX for Mac OS X), then it is possible to create a PDF version of the documentation, as well. Start by building the prerequisite LaTex version from the ReStructured Text originals,

bash pipenv run sphinx-build -b latex docs/source docs/build_latex

Then, navigate to docs/build_latex and run,

bash make

Both LaTeX and PDF versions can then be found in docs/build_latex.

Building Deployable Distributions

The recommended (and most pragmatic) way of deploy this package is to build a Python wheel and to then to install it in a fresh virtual environment on the target system. The exact build configuration is determined by the parameters in setup.py. Note, that this requires that all package dependencies also be specified in the install_requires declaration in setup.py, regardless of their entry in Pipfile. For more information on Python packaging refer to the Python Packaging User Guide and the accompanying sample project. To create the Python wheel run,

bash pipenv run python setup.py bdist_wheel

This will create build, py_package_template.egg-info and dist directories - the wheel can be found in the latter. This needs to be copied to the target system (which we are assuming has Python and Pipenv available as a minimum), where it can be installed into a new virtual environment, together with all downstream dependencies, using,

bash pipenv install path/to/your-package.whl

Automated Testing and Deployment using Travis CI

We have chosen Travis for Continuous Integration (CI) as it integrates very easily with Python and GitHub (where I have granted it access to my public repositories). The configuration details are kept in the .travis.yaml file in the root directory:

```yaml ncsudo: required

language: python

python: - 3.7-dev

install: - pip install pipenv - pipenv install --dev

script: - pipenv run pytest

deploy: provider: pypi user: alexioannides password: secure: my-encrypted-pypi-password on: tags: true distributions: bdist_wheel ```

Briefly, this instructs the Travis build server to:

download, build and install Python 3.7;
install Pipenv
use Pipenv and Pipfile.lock to install all dependencies (dev dependencies are necessary for running PyTest);
run all unit tests using PyTest;
if the tests were run successfully and if we have pushed a new tag (i.e. a release) to the master branch then:
- build a Python wheel; and,
- push it to PyPI.org using my PyPI account credentials.

Note that we provide Travis with an encrypted password, that was made using the Travis command line tool (downloaded using HomeBrew on OS X). For more details on this and PyPI deployment more generally see the Travis CI documentation.

Issues

Bump certifi from 2018.10.15 to 2022.12.7

opened on 2022-12-08 01:18:00 by dependabot[bot]

Bumps certifi from 2018.10.15 to 2022.12.7.

Commits

9e9e840 2022.12.07
b81bdb2 2022.09.24
939a28f 2022.09.14
aca828a 2022.06.15.2
de0eae1 Only use importlib.resources's new files() / Traversable API on Python ≥3.11 ...
b8eb5e9 2022.06.15.1
47fb7ab Fix deprecation warning on Python 3.11 (#199)
b0b48e0 fixes #198 -- update link in license
9d514b4 2022.06.15
4151e88 Add py.typed to MANIFEST.in to package in sdist (#196)
Additional commits viewable in compare view

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) - `@dependabot use these labels` will set the current labels as the default for future PRs for this repo and language - `@dependabot use these reviewers` will set the current reviewers as the default for future PRs for this repo and language - `@dependabot use these assignees` will set the current assignees as the default for future PRs for this repo and language - `@dependabot use this milestone` will set the current milestone as the default for future PRs for this repo and language You can disable automated security fix PRs for this repo from the [Security Alerts page](https://github.com/AlexIoannides/py-package-template/network/alerts).

Fix typos

opened on 2022-05-30 11:19:35 by endolith None

curves is not visible from py_pkg, np is visible from curves

opened on 2022-05-30 11:16:57 by endolith

Is this the intended behavior? I'm trying to learn the "Right" way to cleanly structure packages and imports.

If I do:

``` In [1]: import py_pkg

In [2]: py_pkg. `` and then press Tab, there is no auto-completion forcurves` or anything else. I would expect to be able to see what is available.

On the other hand, if I do

``` In [3]: from py_pkg import curves

In [4]: curves. `` and press Tab, it shows irrelevant things likenp,abc,Dict,Tupleetc. that are imported intocurves.pybut are not part of thispy_package_templatepackage. Is that intentional? I would expect those sorts of imports to be hidden, and only things likeDemandMonotonicityErrorandPriceRanges` to show up.

Bump ipython from 7.1.1 to 7.16.3

opened on 2022-01-21 19:43:52 by dependabot[bot]

Bumps ipython from 7.1.1 to 7.16.3.

Commits

d43c7c7 release 7.16.3
5fa1e40 Merge pull request from GHSA-pq7m-3gw7-gq5x
8df8971 back to dev
9f477b7 release 7.16.2
138f266 bring back release helper from master branch
5aa3634 Merge pull request #13341 from meeseeksmachine/auto-backport-of-pr-13335-on-7...
bcae8e0 Backport PR #13335: What's new 7.16.2
8fcdcd3 Pin Jedi to <0.17.2.
2486838 release 7.16.1
20bdc6f fix conda build
Additional commits viewable in compare view

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Dependabot commands and options

Bump babel from 2.6.0 to 2.9.1

opened on 2021-10-21 18:20:21 by dependabot[bot]

Bumps babel from 2.6.0 to 2.9.1.

Release notes

Sourced from babel's releases.

Version 2.9.1

Bugfixes

The internal locale-data loading functions now validate the name of the locale file to be loaded and only allow files within Babel's data directory. Thank you to Chris Lyne of Tenable, Inc. for discovering the issue!

Version 2.9.0

Upcoming version support changes

This version, Babel 2.9, is the last version of Babel to support Python 2.7, Python 3.4, and Python 3.5.

Improvements

CLDR: Use CLDR 37 – Aarni Koskela (#734)

Dates: Handle ZoneInfo objects in get_timezone_location, get_timezone_name - Alessio Bogon (#741)

Numbers: Add group_separator feature in number formatting - Abdullah Javed Nesar (#726)

Bugfixes

Dates: Correct default Format().timedelta format to 'long' to mute deprecation warnings – Aarni Koskela

Import: Simplify iteration code in "import_cldr.py" – Felix Schwarz

Import: Stop using deprecated ElementTree methods "getchildren()" and "getiterator()" – Felix Schwarz

Messages: Fix unicode printing error on Python 2 without TTY. – Niklas Hambüchen

Messages: Introduce invariant that _invalid_pofile() takes unicode line. – Niklas Hambüchen

Tests: fix tests when using Python 3.9 – Felix Schwarz

Tests: Remove deprecated 'sudo: false' from Travis configuration – Jon Dufresne

Tests: Support Py.test 6.x – Aarni Koskela

Utilities: LazyProxy: Handle AttributeError in specified func – Nikiforov Konstantin (#724)

Utilities: Replace usage of parser.suite with ast.parse – Miro Hrončok

Documentation

Update parse_number comments – Brad Martin (#708)

Add iter to Catalog documentation – @CyanNani123

Version 2.8.1

This patch version only differs from 2.8.0 in that it backports in #752.

Version 2.8.0

Improvements

CLDR: Upgrade to CLDR 36.0 - Aarni Koskela (#679)

Messages: Don't even open files with the "ignore" extraction method - @sebleblanc (#678)

Bugfixes

Numbers: Fix formatting very small decimals when quantization is disabled - Lev Lybin, @miluChen (#662)

Messages: Attempt to sort all messages – Mario Frasca (#651, #606)

Docs

... (truncated)

Changelog

Sourced from babel's changelog.

Version 2.9.1

Bugfixes

* The internal locale-data loading functions now validate the name of the locale file to be loaded and only allow files within Babel's data directory. Thank you to Chris Lyne of Tenable, Inc. for discovering the issue! Version 2.9.0

Upcoming version support changes

This version, Babel 2.9, is the last version of Babel to support Python 2.7, Python 3.4, and Python 3.5.

Improvements


* CLDR: Use CLDR 37 – Aarni Koskela ([#734](https://github.com/python-babel/babel/issues/734))
* Dates: Handle ZoneInfo objects in get_timezone_location, get_timezone_name - Alessio Bogon ([#741](https://github.com/python-babel/babel/issues/741))
* Numbers: Add group_separator feature in number formatting - Abdullah Javed Nesar ([#726](https://github.com/python-babel/babel/issues/726))
Bugfixes

* Dates: Correct default Format().timedelta format to 'long' to mute deprecation warnings – Aarni Koskela
* Import: Simplify iteration code in &quot;import_cldr.py&quot; – Felix Schwarz
* Import: Stop using deprecated ElementTree methods &quot;getchildren()&quot; and &quot;getiterator()&quot; – Felix Schwarz
* Messages: Fix unicode printing error on Python 2 without TTY. – Niklas Hambüchen
* Messages: Introduce invariant that _invalid_pofile() takes unicode line. – Niklas Hambüchen
* Tests: fix tests when using Python 3.9 – Felix Schwarz
* Tests: Remove deprecated 'sudo: false' from Travis configuration – Jon Dufresne
* Tests: Support Py.test 6.x – Aarni Koskela
* Utilities: LazyProxy: Handle AttributeError in specified func – Nikiforov Konstantin ([#724](https://github.com/python-babel/babel/issues/724))
* Utilities: Replace usage of parser.suite with ast.parse – Miro Hrončok

Documentation
</code></pre>
<ul>
<li>Update parse_number comments – Brad Martin (<a href="https://github-redirect.dependabot.com/python-babel/babel/issues/708">#708</a>)</li>
<li>Add <strong>iter</strong> to Catalog documentation – <a href="https://github.com/CyanNani123"><code>@CyanNani123</code></a></li>
</ul>
<h2>Version 2.8.1</h2>
<p>This is solely a patch release to make running tests on Py.test 6+ possible.</p>
<p>Bugfixes</p>
<!-- raw HTML omitted -->
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Commits</summary>

<ul>
<li><a href="https://github.com/python-babel/babel/commit/a99fa2474c808b51ebdabea18db871e389751559"><code>a99fa24</code></a> Use 2.9.0's setup.py for 2.9.1</li>
<li><a href="https://github.com/python-babel/babel/commit/60b33e083801109277cb068105251e76d0b7c14e"><code>60b33e0</code></a> Become 2.9.1</li>
<li><a href="https://github.com/python-babel/babel/commit/412015ef642bfcc0d8ba8f4d05cdbb6aac98d9b3"><code>412015e</code></a> Merge pull request <a href="https://github-redirect.dependabot.com/python-babel/babel/issues/782">#782</a> from python-babel/locale-basename</li>
<li><a href="https://github.com/python-babel/babel/commit/5caf717ceca4bd235552362b4fbff88983c75d8c"><code>5caf717</code></a> Disallow special filenames on Windows</li>
<li><a href="https://github.com/python-babel/babel/commit/3a700b5b8b53606fd98ef8294a56f9510f7290f8"><code>3a700b5</code></a> Run locale identifiers through <code>os.path.basename()</code></li>
<li><a href="https://github.com/python-babel/babel/commit/5afe2b2f11dcdd6090c00231d342c2e9cd1bdaab"><code>5afe2b2</code></a> Merge pull request <a href="https://github-redirect.dependabot.com/python-babel/babel/issues/754">#754</a> from python-babel/github-ci</li>
<li><a href="https://github.com/python-babel/babel/commit/58de8342f865df88697a4a166191e880e3c84d82"><code>58de834</code></a> Replace Travis + Appveyor with GitHub Actions (WIP)</li>
<li><a href="https://github.com/python-babel/babel/commit/d1bbc08e845d03d8e1f0dfa0e04983d755f39cb5"><code>d1bbc08</code></a> import_cldr: use logging; add -q option</li>
<li><a href="https://github.com/python-babel/babel/commit/156b7fb9f377ccf58c71cf01dc69fb10c7b69314"><code>156b7fb</code></a> Quiesce CLDR download progress bar if requested (or not a TTY)</li>
<li><a href="https://github.com/python-babel/babel/commit/613dc1700f91c3d40b081948c0dd6023d8ece057"><code>613dc17</code></a> Make the import warnings about unsupported number systems less verbose</li>
<li>Additional commits viewable in <a href="https://github.com/python-babel/babel/compare/v2.6.0...v2.9.1">compare view</a></li>
</ul>
</details>

<br />



[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=babel&package-manager=pip&previous-version=2.6.0&new-version=2.9.1)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---


Dependabot commands and options



You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
- `@dependabot use these labels` will set the current labels as the default for future PRs for this repo and language
- `@dependabot use these reviewers` will set the current reviewers as the default for future PRs for this repo and language
- `@dependabot use these assignees` will set the current assignees as the default for future PRs for this repo and language
- `@dependabot use this milestone` will set the current milestone as the default for future PRs for this repo and language

You can disable automated security fix PRs for this repo from the [Security Alerts page](https://github.com/AlexIoannides/py-package-template/network/alerts).



Updated README to correct the command to pip install the package

opened on 2021-06-29 05:50:31 by jfthuong
None


    
        
            
                
                    
                    
                    Alex Ioannides
                    
                
                
                machine learning engineer - (data)scientist - reformed quant - habitual coder - PhD
                
                
                    GitHub Repository
                
                
            
        
        
        
            
                
                    
                    python
                    
                    package
                    
                    pip
                    
                    pypi
                    
                    template-project
                    
                    travis-ci
                    
                    automated-testing
                    
                    automated-deployment
                    
                    continuous-delivery
                    
                    pytest
                    
                    mypy
                    
                    sphinx
                    
                    flake8
                    
                    example-project