Optimal numerical differentiation of noisy time series data in python.

andgoldschmidt, updated 🕥 2022-12-08 12:57:59

|RTD| |PyPI| |Zenodo| |LIC|

Numerical differentiation of noisy time series data in python ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

derivative is a Python package for differentiating noisy data. The package showcases a variety of improvements that can be made over finite differences when data is not clean.

Want to see an example of how derivative can help? This package is part of PySINDy (github.com/dynamicslab/pysindy <https://github.com/dynamicslab/pysindy/>_), a sparse-regression framework for discovering nonlinear dynamical systems from data.

This package binds common differentiation methods to a single easily implemented differentiation interface to encourage user adaptation. Numerical differentiation methods for noisy time series data in python includes:

  1. Symmetric finite difference schemes using arbitrary window size.

  2. Savitzky-Galoy derivatives (aka polynomial-filtered derivatives) of any polynomial order with independent left and right window parameters.

  3. Spectral derivatives with optional filter.

  4. Spline derivatives of any order.

  5. Polynomial-trend-filtered derivatives generalizing methods like total variational derivatives.

  6. Kalman derivatives find the maximum likelihood estimator for a derivative described by a Brownian motion.

.. code-block:: python

from derivative import dxdt
import numpy as np

t = np.linspace(0,2*np.pi,50)
x = np.sin(x)

# 1. Finite differences with central differencing using 3 points.
result1 = dxdt(x, t, kind="finite_difference", k=1)

# 2. Savitzky-Golay using cubic polynomials to fit in a centered window of length 1
result2 = dxdt(x, t, kind="savitzky_golay", left=.5, right=.5, order=3)

# 3. Spectral derivative
result3 = dxdt(x, t, kind="spectral")

# 4. Spline derivative with smoothing set to 0.01
result4 = dxdt(x, t, kind="spline", s=1e-2)

# 5. Total variational derivative with regularization set to 0.01
result5 = dxdt(x, t, kind="trend_filtered", order=0, alpha=1e-2)

# 6. Kalman derivative with smoothing set to 1
result6 = dxdt(x, t, kind="kalman", alpha=1)

Contributors:

Thanks to the members of the community who have contributed!

+-----------------------------------------------------------------+----------------------------------------------------------------------------------+ | Jacob Stevens-Haas <https://github.com/Jacob-Stevens-Haas> | Kalman derivatives #12 <https://github.com/andgoldschmidt/derivative/pull/12> |
+-----------------------------------------------------------------+----------------------------------------------------------------------------------+

References:

[1] Numerical differentiation of experimental data: local versus global methods- K. Ahnert and M. Abel

[2] Numerical Differentiation of Noisy, Nonsmooth Data- Rick Chartrand

[3] The Solution Path of the Generalized LASSO- R.J. Tibshirani and J. Taylor

Citing derivative:

The derivative package is a contribution to PySINDy <https://github.com/dynamicslab/pysindy/>_; this work has been published in the Journal of Open Source Software (JOSS). If you use derivative in your work, please cite it using the following reference:

Kaptanoglu et al., (2022). PySINDy: A comprehensive Python package for robust sparse system identification. Journal of Open Source Software, 7(69), 3994, https://doi.org/10.21105/joss.03994

.. code-block:: text

  @article{kaptanoglu2022pysindy,
doi = {10.21105/joss.03994},
url = {https://doi.org/10.21105/joss.03994},
year = {2022},
publisher = {The Open Journal},
volume = {7},
number = {69},
pages = {3994},
author = {Alan A. Kaptanoglu and Brian M. de Silva and Urban Fasel and Kadierdan Kaheman and Andy J. Goldschmidt and Jared Callaham and Charles B. Delahunt and Zachary G. Nicolaou and Kathleen Champion and Jean-Christophe Loiseau and J. Nathan Kutz and Steven L. Brunton},
title = {PySINDy: A comprehensive Python package for robust sparse system identification},
journal = {Journal of Open Source Software}
}

.. |RTD| image:: https://readthedocs.org/projects/derivative/badge/?version=latest :target: https://derivative.readthedocs.io/en/latest/?badge=latest :alt: Documentation Status

.. |LIC| image:: https://img.shields.io/badge/License-MIT-blue.svg :target: https://derivative.readthedocs.io/en/latest/license.html :alt: MIT License

.. |PyPI| image:: https://badge.fury.io/py/derivative.svg :target: https://pypi.org/project/derivative/

.. |Zenodo| image:: https://zenodo.org/badge/DOI/10.5281/zenodo.6617446.svg :target: https://doi.org/10.5281/zenodo.6617446

Issues

Bump certifi from 2021.10.8 to 2022.12.7

opened on 2022-12-08 12:57:58 by dependabot[bot]

Bumps certifi from 2021.10.8 to 2022.12.7.

Commits


Dependabot compatibility score

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) - `@dependabot use these labels` will set the current labels as the default for future PRs for this repo and language - `@dependabot use these reviewers` will set the current reviewers as the default for future PRs for this repo and language - `@dependabot use these assignees` will set the current assignees as the default for future PRs for this repo and language - `@dependabot use this milestone` will set the current milestone as the default for future PRs for this repo and language You can disable automated security fix PRs for this repo from the [Security Alerts page](https://github.com/andgoldschmidt/derivative/network/alerts).

Bump pillow from 9.1.1 to 9.3.0

opened on 2022-11-22 11:10:40 by dependabot[bot]

Bumps pillow from 9.1.1 to 9.3.0.

Release notes

Sourced from pillow's releases.

9.3.0

https://pillow.readthedocs.io/en/stable/releasenotes/9.3.0.html

Changes

... (truncated)

Changelog

Sourced from pillow's changelog.

9.3.0 (2022-10-29)

  • Limit SAMPLESPERPIXEL to avoid runtime DOS #6700 [wiredfool]

  • Initialize libtiff buffer when saving #6699 [radarhere]

  • Inline fname2char to fix memory leak #6329 [nulano]

  • Fix memory leaks related to text features #6330 [nulano]

  • Use double quotes for version check on old CPython on Windows #6695 [hugovk]

  • Remove backup implementation of Round for Windows platforms #6693 [cgohlke]

  • Fixed set_variation_by_name offset #6445 [radarhere]

  • Fix malloc in _imagingft.c:font_setvaraxes #6690 [cgohlke]

  • Release Python GIL when converting images using matrix operations #6418 [hmaarrfk]

  • Added ExifTags enums #6630 [radarhere]

  • Do not modify previous frame when calculating delta in PNG #6683 [radarhere]

  • Added support for reading BMP images with RLE4 compression #6674 [npjg, radarhere]

  • Decode JPEG compressed BLP1 data in original mode #6678 [radarhere]

  • Added GPS TIFF tag info #6661 [radarhere]

  • Added conversion between RGB/RGBA/RGBX and LAB #6647 [radarhere]

  • Do not attempt normalization if mode is already normal #6644 [radarhere]

... (truncated)

Commits


Dependabot compatibility score

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) - `@dependabot use these labels` will set the current labels as the default for future PRs for this repo and language - `@dependabot use these reviewers` will set the current reviewers as the default for future PRs for this repo and language - `@dependabot use these assignees` will set the current assignees as the default for future PRs for this repo and language - `@dependabot use this milestone` will set the current milestone as the default for future PRs for this repo and language You can disable automated security fix PRs for this repo from the [Security Alerts page](https://github.com/andgoldschmidt/derivative/network/alerts).

Bump nbconvert from 6.5.0 to 6.5.1

opened on 2022-08-23 19:07:45 by dependabot[bot]

Bumps nbconvert from 6.5.0 to 6.5.1.

Commits


Dependabot compatibility score

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) - `@dependabot use these labels` will set the current labels as the default for future PRs for this repo and language - `@dependabot use these reviewers` will set the current reviewers as the default for future PRs for this repo and language - `@dependabot use these assignees` will set the current assignees as the default for future PRs for this repo and language - `@dependabot use this milestone` will set the current milestone as the default for future PRs for this repo and language You can disable automated security fix PRs for this repo from the [Security Alerts page](https://github.com/andgoldschmidt/derivative/network/alerts).

Releases

derivative 0.4.2 2022-05-18 03:20:05

derivative 0.3.1 2020-10-07 19:55:41

https://pypi.org/project/derivative/0.3.1/

Andy Goldschmidt

EPiQC Postdoc with UChicago CS

GitHub Repository Homepage

differentiation numerical-differentiation experimental-data