Release of the whole cell E. coli model.

CovertLab, updated 🕥 2023-02-16 03:35:05

Whole Cell Model - Escherichia coli

Notice: This repository contains previous release snapshots of the Covert Lab's Whole Cell Model for Escherichia coli. For the most recent versions of the E. coli whole-cell model that are undergoing active development, please visit the wcEcoli and the vivarium-ecoli repositories. This repository should only be used for the purpose of replicating model outputs generated for previous publications listed below. We do not plan to merge Pull Requests into this repository except documentation and installation fixes.

You can reach us at WholeCellTeam.

This repository contains code for the following publications: - Simultaneous cross-evaluation of heterogeneous E. coli datasets via mechanistic simulation published in Science, 24 July 2020. (Release) - The E. coli whole-cell modeling project published in EcoSal Plus, 9 July 2021. - An expanded whole-cell model of E. coli links cellular physiology with mechanisms of growth rate control published in npj Systems Biology and Applications, 19 August 2022. (Release)

Setup

See docs/README.md for docs on how to set up and run the model.

In short, there are two alternative ways to set up to run the model: in a Docker container or in a pyenv Python virtual environment. Docker containers are easier to build and isolated from your development computer, but they run slower. (PyCharm should support debugging into a Docker container but we haven't tested that.) pyenv virtual environments take more steps to build and depend on your computer's OS, but are lighter weight and easier for debugging. With Docker, you can start running a simulation with these steps: 1. Create a github personal access token with at least the read:packages permission selected. 1. Place the token in github_personal_access_token.txt. 1. Log in to docker.pkg.github.com: bash cat github_personal_access_token.txt | docker login https://docker.pkg.github.com -u USERNAME --password-stdin You should see an output message like Login Succeeded 1. Pull the Docker image: shell script docker pull docker.pkg.github.com/covertlab/wholecellecolirelease/wcm-full:latest 1. Run the Docker container: shell script docker run --name=wcm -it --rm docker.pkg.github.com/covertlab/wholecellecolirelease/wcm-full 1. Inside the container, run the model: shell script python runscripts/manual/runSim.py

Quick start

When running this code, prepare with these steps (the wcm-code Docker container already prepares this for you):

cd to the top level of your wcEcoli directory.
Set the $PYTHONPATH:

bash export PYTHONPATH="$PWD"

In the wcEcoli directory, compile the Cython code:

bash make clean compile

Ways to run the model:

Use the manual runscripts.

They run each step directly in-process, which is particularly handy to use with a debugger. But you're responsible for properly sequencing all the steps: parameter calculation, cell simulation generations, and analyses. The manual runscripts work with a Docker container and also with a pyenv virtual environment.
Queue up a Fireworks workflow, then run it.

You configure it for the desired variants, number of generations, and other options, then Fireworks will automatically run all the steps including parameter calculation, simulations, and all the analysis plots.

The workflow tasks can be distributed over multiple processes or even multiple computers, but they must all access a shared file system such as NFS and the (or copies of the) pyenv virtual environment. We have not tested Fireworks with Docker containers.
Run on the Google Cloud Platform using Docker containers and our custom workflow software.
Use the multi-scale agent-based framework.

This can run several cells interactively on a simulated microscope slide.

Using the manual runscripts

These scripts will:

run the parameter calculator (ParCa),
run cell simulations, and
run analysis plots

All these steps run directly, in-process, without any workflow software or MongoDB. This is handy for development, e.g. running under the PyCharm debugger. But you're responsible for running the scripts in order and for re-running the ParCa after relevant code changes.

You can run just the parts you want and rerun them as needed but the manual scripts don't automate dependency management. It's on you to rerun code if things change, runSim before analysis, or delete runSim output before running it again. (That last part should be improved! Also note that some analysis scripts get confused if the sim runs are more varied than expected. See Issue #199.)

These scripts have command line interfaces built on argparse, so you can use shorter option names as long as they're unambiguous, and also one-letter forms so you can use --cpus 8, or --cpu 8, or -c8.

NOTE: Use the -h or --help switch to get complete, up-to-date documentation on the command line options. Below are just some of the command line options.

To run the parameter calculator (ParCa), which is needed to prepare data for the simulation: bash python runscripts/manual/runParca.py [-h] [--cpus CPUS] [sim_outdir]

To simulate one or more cell generations with optional variants:

bash python runscripts/manual/runSim.py [-h] [--variant VARIANT_TYPE FIRST_INDEX LAST_INDEX] [--generations GENERATIONS] [--init-sims INIT_SIMS] [--seed SEED] [sim_dir]

To interactively select from the data that is saved during a simulation for visualization:

bash python runscripts/manual/analysis_interactive.py [-h] [sim_dir]

Running the command without any arguments will populate drop down menus for each set of simulations in out/ where you can select the desired variant/seed/generation/daughter and view the available values that are saved during simulations. Some simple data processing options are available. This interface mainly lets you select time traces or create scatter plots that can be used to compare different variants, generations, etc.

To run predefined analysis plots on the simulation output in a given sim_dir (use the -h parameter to get complete help on the command line options):

```bash python runscripts/manual/analysisParca.py [-h] [-p PLOT [PLOT ...]] [--cpus CPUS] [sim_dir]

python runscripts/manual/analysisVariant.py [-h] [--plot PLOT [PLOT ...]] [--cpus CPUS] [sim_dir]

python runscripts/manual/analysisCohort.py [-h] [--plot PLOT [PLOT ...]] [--cpus CPUS] [--variant-index VARIANT_INDEX] [--variant-range START_VARIANT END_VARIANT] [sim_dir]

python runscripts/manual/analysisMultigen.py [-h] [--plot PLOT [PLOT ...]] [--cpus CPUS] [--variant-index VARIANT_INDEX] [--seed SEED] [--variant-range START_VARIANT END_VARIANT] [--seed-range START_SEED END_SEED] [sim_dir]

python runscripts/manual/analysisSingle.py [-h] [--plot PLOT [PLOT ...]] [--cpus CPUS] [--variant-index VARIANT_INDEX] [--seed SEED] [--generation GENERATION] [--daughter DAUGHTER] [--variant-range START_VARIANT END_VARIANT] [--seed-range START_SEED END_SEED] [--generation-range START_GENERATION END_GENERATION] [sim_dir]

python runscripts/manual/analysis_interactive.py [-h] [sim_dir] ```

If you default the analysis parameters, these scripts will pick the latest simulation directory, the first variant, the first generation, and so on. To get full analyses across all variants, generations, etc., run:

analysisVariant.py
analysisCohort.py for each --variant_index you simulated
analysisMultigen.py for each combination of --variant_index and --seed you simulated
analysisSingle.py for each combination of --variant_index, --seed, and --generation you simulated

The --plot (or -p) optional parameter lets you pick one or more specific PLOTS to run. The list of PLOTs can include analysis class filenames like aaCounts (or aaCounts.py) and analysis group TAGS like CORE. See the __init__.py file in each analysis class directory for the available analysis classes and group TAGS. The default is to run the DEFAULT tag, which will run the CORE group of plots that are recommended for everyday development and any variant specific plots with the corresponding variant tag.

For example, to run two analysis plots on simulation variant #3 and put a filename prefix "v3_" on their output files (to distinguish them from other analysis runs):

bash python runscripts/manual/analysisCohort.py --plot compositionFitting.py figure2e.py --variant_index 3 --output_prefix v3_

Set the environment variable DEBUG_GC=1 if you want to check for Python memory leaks when running the analysis plots.

There's another way run an individual analysis plot:

bash python models/ecoli/analysis/cohort/transcriptFrequency.py [-h] [-o OUTPUT_PREFIX] [-v VARIANT_INDEX] [sim_dir]

Running a Fireworks workflow

See wholecell/fireworks/README.md for instructions to set up MongoDB as needed to run Fireworks.

The command line program fw_queue.py queues up a Fireworks workflow including parameter calculations, the simulation itself, and analysis plots.

The fw_queue.py source code begins with documentation on its many options. The options are set via environment variables. Below are a few usage examples.

But first, note that you can reset the Fireworks queue (if needed) via:

bash lpad reset

Single simulation

To queue up a single simulation in Fireworks, including parameter calculations and analysis plots:

bash DESC="Example run of a single simulation." python runscripts/fireworks/fw_queue.py

The DESC text should be more descriptive than this so you can readily distinguish your runs.

Multiple simulations

To queue multiple simulations, e.g. 4 simulations, each with a different initial seed:

bash DESC="Example run of multiple simulations." N_INIT_SIMS=4 python runscripts/fireworks/fw_queue.py

Multiple generations

To queue multiple generations, e.g. 4 generations from a single mother cell:

bash DESC="Example run of multiple generations." N_GENS=4 python runscripts/fireworks/fw_queue.py

To queue multiple generations (in this case 3 generations) from multiple mother cells (in this case 2 mother cells:

bash DESC="Example run of multiple generations from multiple mother cells." N_GENS=3 N_INIT_SIMS=2 python runscripts/fireworks/fw_queue.py

Shifting media conditions

To queue a simulation that switches between environments, use the "timeline" variant and give the range of indices (in this case from 1 to 1) specifying conditions defined in wcEcoli/environment/condition/timelines:

bash DESC="Example run of nutrient shifts." VARIANT="timeline" FIRST_VARIANT_INDEX=1 LAST_VARIANT_INDEX=1 python runscripts/fireworks/fw_queue.py

Using the cached sim data object

To use the cached sim data file, set the CACHED_SIM_DATA environment variable (TODO: Explain what creates a cached sim data file):

bash DESC="Example run with cached sim data." CACHED_SIM_DATA=1 python runscripts/fireworks/fw_queue.py

Using an interactive Sherlock node to run a Fireworks workflow

To run queued simulations on an interactive Sherlock node:

bash rlaunch rapidfire

You probably only want to do this if you're running or debugging a single simulation (one initial seed, generation, and variant).

Don't do this on a Sherlock login node.

Using the SLURM scheduler on Linux to run a Fireworks workflow

To run simulations on a Sherlock cluster (helpful when running more than one simulation):

bash qlaunch -r rapidfire --nlaunches infinite --sleep 5

The qlaunch command will run forever. Hit Ctrl-C to kill it once the console logs shows that all the simulation and analysis steps have finished.

qlaunch is relatively lightweight, so it might work on a Sherlock login node.

qlaunch will create block directories with stdout and stderr from each Firework. To troubleshoot any errors or just to see the output you would normally see from an interactive session, use the following commands to search the block directories for your desired fw_id: bash ./runscripts/fw_info.sh out 1 ./runscripts/fw_info.sh error 1 This will display the stdout and stderr from the execution of a firework with fw_id of 1.

Output

The output is stored as a time-stamped sub-directory of the out directory, for example out/20180703.215222.029168__multi-sim/, where DESC="multi-sim" was one of the arguments to fw_queue.py.

Within this directory, there is a metadata sub-directory which stores the git revision information as well as the description provided by the DESC variable, a kb sub-directory which stores kb objects (after the simulations and analysis are done the objects are compressed using bzip2), and sub-directories (maybe only a single sub-directory) containing different variants (e.g., gene knockouts or other perturbations).

Within variant sub-directories, there are N_INIT_SIMS (which defaults to 1) numbered sub-directories such as 000000 corresponding to "family trees".

Within each "family tree" sub-directory are generation sub-directories such as generation_000000.

Within each generation sub-directory are numbered individual simulation directories that contain simOut (for simulation data) and plotOut (for plots) sub-directories.

A family tree for 3 generations showing the relationship between numbered individual simulations is shown here:

gen 0: 0 gen 1: 0 1 gen 2: 0 1 2 3

Google Cloud Platform

You can run wcEcoli cell simulations on the Google Cloud Platform using Docker containers and our custom workflow software.

NOTE: So far the documentation assumes you're part of the Covert lab and able to access our Allen Discovery Center project on Google Cloud Platform.

See How to run the Whole Cell Model on the Google Cloud Platform.

Parameter search and optimization

Another runscript will allow you to search for parameters or optimize parameters for a desired objective, which can be any output from the simulation. This script will run the parca and simulations with varying sets of parameters and can update the parameters based on the results of each iteration. Define any optimization approach you would like to use in a file in wholecell/optimization/ and the method, which will define the set of parameters and objective you are interested in, in models/ecoli/sim/parameter_search/ and then run the script with your options (add -c for the number of CPUs to use in parallel for faster iteration): runscripts/manual/parameter_search.py output-dir --solver spsa --method quick_example -c 4

Issues

Bump werkzeug from 1.0.1 to 2.2.3

opened on 2023-02-16 03:35:04 by dependabot[bot]

Bumps werkzeug from 1.0.1 to 2.2.3.

Release notes

Sourced from werkzeug's releases.

2.2.3

This is a fix release for the 2.2.x release branch.

Changes: https://werkzeug.palletsprojects.com/en/2.2.x/changes/#version-2-2-3

Milestone: https://github.com/pallets/werkzeug/milestone/26?closed=1

This release contains security fixes for:

https://github.com/pallets/werkzeug/security/advisories/GHSA-xg9f-g7g7-2323

https://github.com/pallets/werkzeug/security/advisories/GHSA-px8h-6qxv-m22q

2.2.2

This is a fix release for the 2.2.0 feature release.

Changes: https://werkzeug.palletsprojects.com/en/2.2.x/changes/#version-2-2-2

Milestone: https://github.com/pallets/werkzeug/milestone/25?closed=1

2.2.1

This is a fix release for the 2.2.0 feature release.

Changes: https://werkzeug.palletsprojects.com/en/2.2.x/changes/#version-2-2-1

Milestone: https://github.com/pallets/werkzeug/milestone/24?closed=1

2.2.0

This is a feature release, which includes new features and removes previously deprecated features. The 2.2.x branch is now the supported bugfix branch, the 2.1.x branch will become a tag marking the end of support for that branch. We encourage everyone to upgrade, and to use a tool such as pip-tools to pin all dependencies and control upgrades.

Changes: https://werkzeug.palletsprojects.com/en/2.2.x/changes/#version-2-2-0

Milestone: https://github.com/pallets/werkzeug/milestone/20?closed=1

2.1.2

This is a fix release for the 2.1.0 feature release.

Changes: https://werkzeug.palletsprojects.com/en/2.1.x/changes/#version-2-1-2

Milestone: https://github.com/pallets/werkzeug/milestone/22?closed=1

2.1.1

This is a fix release for the 2.1.0 feature release.

Changes: https://werkzeug.palletsprojects.com/en/2.1.x/changes/#version-2-1-1

Milestone: https://github.com/pallets/werkzeug/milestone/19?closed=1

2.1.0

This is a feature release, which includes new features and removes previously deprecated features. The 2.1.x branch is now the supported bugfix branch, the 2.0.x branch will become a tag marking the end of support for that branch. We encourage everyone to upgrade, and to use a tool such as pip-tools to pin all dependencies and control upgrades.

Changes: https://werkzeug.palletsprojects.com/en/2.1.x/changes/#version-2-1-0

Milestone: https://github.com/pallets/werkzeug/milestone/16?closed=1

2.0.3

Changes: https://werkzeug.palletsprojects.com/en/2.0.x/changes/#version-2-0-3

Milestone: https://github.com/pallets/werkzeug/milestone/18?closed=1

... (truncated)

Changelog

Sourced from werkzeug's changelog.

Version 2.2.3

Released 2023-02-14

Ensure that URL rules using path converters will redirect with strict slashes when the trailing slash is missing. :issue:2533

Type signature for get_json specifies that return type is not optional when silent=False. :issue:2508

parse_content_range_header returns None for a value like bytes */-1 where the length is invalid, instead of raising an AssertionError. :issue:2531

Address remaining ResourceWarning related to the socket used by run_simple. Remove prepare_socket, which now happens when creating the server. :issue:2421

Update pre-existing headers for multipart/form-data requests with the test client. :issue:2549

Fix handling of header extended parameters such that they are no longer quoted. :issue:2529

LimitedStream.read works correctly when wrapping a stream that may not return the requested size in one read call. :issue:2558

A cookie header that starts with = is treated as an empty key and discarded, rather than stripping the leading ==.

Specify a maximum number of multipart parts, default 1000, after which a RequestEntityTooLarge exception is raised on parsing. This mitigates a DoS attack where a larger number of form/file parts would result in disproportionate resource use.

Version 2.2.2

Released 2022-08-08

Fix router to restore the 2.1 strict_slashes == False behaviour whereby leaf-requests match branch rules and vice versa. :pr:2489

Fix router to identify invalid rules rather than hang parsing them, and to correctly parse / within converter arguments. :pr:2489

Update subpackage imports in :mod:werkzeug.routing to use the import as syntax for explicitly re-exporting public attributes. :pr:2493

Parsing of some invalid header characters is more robust. :pr:2494

When starting the development server, a warning not to use it in a production deployment is always shown. :issue:2480

LocalProxy.__wrapped__ is always set to the wrapped object when the proxy is unbound, fixing an issue in doctest that would cause it to fail. :issue:2485

Address one ResourceWarning related to the socket used by run_simple. :issue:2421

... (truncated)

Commits

22a254f release version 2.2.3
517cac5 Merge pull request from GHSA-xg9f-g7g7-2323
babc8d9 rewrite docs about request data limits
09449ee clean up docs
fe899d0 limit the maximum number of multipart form parts
cf275f4 Merge pull request from GHSA-px8h-6qxv-m22q
8c2b4b8 don't strip leading = when parsing cookie
7c7ce5c [pre-commit.ci] pre-commit autoupdate (#2585)
19ae03e [pre-commit.ci] auto fixes from pre-commit.com hooks
a83d3b8 [pre-commit.ci] pre-commit autoupdate
Additional commits viewable in compare view

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) - `@dependabot use these labels` will set the current labels as the default for future PRs for this repo and language - `@dependabot use these reviewers` will set the current reviewers as the default for future PRs for this repo and language - `@dependabot use these assignees` will set the current assignees as the default for future PRs for this repo and language - `@dependabot use this milestone` will set the current milestone as the default for future PRs for this repo and language You can disable automated security fix PRs for this repo from the [Security Alerts page](https://github.com/CovertLab/WholeCellEcoliRelease/network/alerts).

Bump ipython from 7.16.1 to 8.10.0

opened on 2023-02-10 23:07:36 by dependabot[bot]

Bumps ipython from 7.16.1 to 8.10.0.

Release notes

Sourced from ipython's releases.

See https://pypi.org/project/ipython/

We do not use GitHub release anymore. Please see PyPI https://pypi.org/project/ipython/

Commits

15ea1ed release 8.10.0
560ad10 DOC: Update what's new for 8.10 (#13939)
7557ade DOC: Update what's new for 8.10
385d693 Merge pull request from GHSA-29gw-9793-fvw7
e548ee2 Swallow potential exceptions from showtraceback() (#13934)
0694b08 MAINT: mock slowest test. (#13885)
8655912 MAINT: mock slowest test.
a011765 Isolate the attack tests with setUp and tearDown methods
c7a9470 Add some regression tests for this change
fd34cf5 Swallow potential exceptions from showtraceback()
Additional commits viewable in compare view

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Dependabot commands and options

Bump certifi from 2020.6.20 to 2022.12.7

opened on 2022-12-08 12:29:06 by dependabot[bot]

Bumps certifi from 2020.6.20 to 2022.12.7.

Commits

9e9e840 2022.12.07
b81bdb2 2022.09.24
939a28f 2022.09.14
aca828a 2022.06.15.2
de0eae1 Only use importlib.resources's new files() / Traversable API on Python ≥3.11 ...
b8eb5e9 2022.06.15.1
47fb7ab Fix deprecation warning on Python 3.11 (#199)
b0b48e0 fixes #198 -- update link in license
9d514b4 2022.06.15
4151e88 Add py.typed to MANIFEST.in to package in sdist (#196)
Additional commits viewable in compare view

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Dependabot commands and options

Bump pillow from 7.2.0 to 9.3.0

opened on 2022-11-22 06:58:02 by dependabot[bot]

Bumps pillow from 7.2.0 to 9.3.0.

Release notes

Sourced from pillow's releases.

9.3.0

https://pillow.readthedocs.io/en/stable/releasenotes/9.3.0.html

Changes

Initialize libtiff buffer when saving #6699 [@radarhere]

Limit SAMPLESPERPIXEL to avoid runtime DOS #6700 [@wiredfool]

Inline fname2char to fix memory leak #6329 [@nulano]

Fix memory leaks related to text features #6330 [@nulano]

Use double quotes for version check on old CPython on Windows #6695 [@hugovk]

GHA: replace deprecated set-output command with GITHUB_OUTPUT file #6697 [@nulano]

Remove backup implementation of Round for Windows platforms #6693 [@cgohlke]

Upload fribidi.dll to GitHub Actions #6532 [@nulano]

Fixed set_variation_by_name offset #6445 [@radarhere]

Windows build improvements #6562 [@nulano]

Fix malloc in _imagingft.c:font_setvaraxes #6690 [@cgohlke]

Only use ASCII characters in C source file #6691 [@cgohlke]

Release Python GIL when converting images using matrix operations #6418 [@hmaarrfk]

Added ExifTags enums #6630 [@radarhere]

Do not modify previous frame when calculating delta in PNG #6683 [@radarhere]

Added support for reading BMP images with RLE4 compression #6674 [@npjg]

Decode JPEG compressed BLP1 data in original mode #6678 [@radarhere]

pylint warnings #6659 [@marksmayo]

Added GPS TIFF tag info #6661 [@radarhere]

Added conversion between RGB/RGBA/RGBX and LAB #6647 [@radarhere]

Do not attempt normalization if mode is already normal #6644 [@radarhere]

Fixed seeking to an L frame in a GIF #6576 [@radarhere]

Consider all frames when selecting mode for PNG save_all #6610 [@radarhere]

Don't reassign crc on ChunkStream close #6627 [@radarhere]

Raise a warning if NumPy failed to raise an error during conversion #6594 [@radarhere]

Only read a maximum of 100 bytes at a time in IMT header #6623 [@radarhere]

Show all frames in ImageShow #6611 [@radarhere]

Allow FLI palette chunk to not be first #6626 [@radarhere]

If first GIF frame has transparency for RGB_ALWAYS loading strategy, use RGBA mode #6592 [@radarhere]

Round box position to integer when pasting embedded color #6517 [@radarhere]

Removed EXIF prefix when saving WebP #6582 [@radarhere]

Pad IM palette to 768 bytes when saving #6579 [@radarhere]

Added DDS BC6H reading #6449 [@ShadelessFox]

Added support for opening WhiteIsZero 16-bit integer TIFF images #6642 [@JayWiz]

Raise an error when allocating translucent color to RGB palette #6654 [@jsbueno]

Moved mode check outside of loops #6650 [@radarhere]

Added reading of TIFF child images #6569 [@radarhere]

Improved ImageOps palette handling #6596 [@PososikTeam]

Defer parsing of palette into colors #6567 [@radarhere]

Apply transparency to P images in ImageTk.PhotoImage #6559 [@radarhere]

Use rounding in ImageOps contain() and pad() #6522 [@bibinhashley]

Fixed GIF remapping to palette with duplicate entries #6548 [@radarhere]

Allow remap_palette() to return an image with less than 256 palette entries #6543 [@radarhere]

Corrected BMP and TGA palette size when saving #6500 [@radarhere]

... (truncated)

Changelog

Sourced from pillow's changelog.

9.3.0 (2022-10-29)

Limit SAMPLESPERPIXEL to avoid runtime DOS #6700 [wiredfool]

Initialize libtiff buffer when saving #6699 [radarhere]

Inline fname2char to fix memory leak #6329 [nulano]

Fix memory leaks related to text features #6330 [nulano]

Use double quotes for version check on old CPython on Windows #6695 [hugovk]

Remove backup implementation of Round for Windows platforms #6693 [cgohlke]

Fixed set_variation_by_name offset #6445 [radarhere]

Fix malloc in _imagingft.c:font_setvaraxes #6690 [cgohlke]

Release Python GIL when converting images using matrix operations #6418 [hmaarrfk]

Added ExifTags enums #6630 [radarhere]

Do not modify previous frame when calculating delta in PNG #6683 [radarhere]

Added support for reading BMP images with RLE4 compression #6674 [npjg, radarhere]

Decode JPEG compressed BLP1 data in original mode #6678 [radarhere]

Added GPS TIFF tag info #6661 [radarhere]

Added conversion between RGB/RGBA/RGBX and LAB #6647 [radarhere]

Do not attempt normalization if mode is already normal #6644 [radarhere]

... (truncated)

Commits

d594f4c Update CHANGES.rst [ci skip]
909dc64 9.3.0 version bump
1a51ce7 Merge pull request #6699 from hugovk/security-libtiff_buffer
2444cdd Merge pull request #6700 from hugovk/security-samples_per_pixel-sec
744f455 Added release notes
0846bfa Add to release notes
799a6a0 Fix linting
00b25fd Hide UserWarning in logs
05b175e Tighter test case
13f2c5a Prevent DOS with large SAMPLESPERPIXEL in Tiff IFD
Additional commits viewable in compare view

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Dependabot commands and options

Bump protobuf from 3.15.8 to 3.18.3

opened on 2022-09-23 23:21:05 by dependabot[bot]

Bumps protobuf from 3.15.8 to 3.18.3.

Release notes

Sourced from protobuf's releases.

Protocol Buffers v3.18.3

C++

Reduce memory consumption of MessageSet parsing

This release addresses a Security Advisory for C++ and Python users

Protocol Buffers v3.16.1

Java

Improve performance characteristics of UnknownFieldSet parsing (#9371)

Protocol Buffers v3.18.2

Java

Improve performance characteristics of UnknownFieldSet parsing (#9371)

Protocol Buffers v3.18.1

Python

Update setup.py to reflect that we now require at least Python 3.5 (#8989)

Performance fix for DynamicMessage: force GetRaw() to be inlined (#9023)

Ruby

Update ruby_generator.cc to allow proto2 imports in proto3 (#9003)

Protocol Buffers v3.18.0

C++

Fix warnings raised by clang 11 (#8664)

Make StringPiece constructible from std::string_view (#8707)

Add missing capability attributes for LLVM 12 (#8714)

Stop using std::iterator (deprecated in C++17). (#8741)

Move field_access_listener from libprotobuf-lite to libprotobuf (#8775)

Fix #7047 Safely handle setlocale (#8735)

Remove deprecated version of SetTotalBytesLimit() (#8794)

Support arena allocation of google::protobuf::AnyMetadata (#8758)

Fix undefined symbol error around SharedCtor() (#8827)

Fix default value of enum(int) in json_util with proto2 (#8835)

Better Smaller ByteSizeLong

Introduce event filters for inject_field_listener_events

Reduce memory usage of DescriptorPool

For lazy fields copy serialized form when allowed.

Re-introduce the InlinedStringField class

v2 access listener

Reduce padding in the proto's ExtensionRegistry map.

GetExtension performance optimizations

Make tracker a static variable rather than call static functions

Support extensions in field access listener

Annotate MergeFrom for field access listener

Fix incomplete types for field access listener

Add map_entry/new_map_entry to SpecificField in MessageDifferencer. They record the map items which are different in MessageDifferencer's reporter.

Reduce binary size due to fieldless proto messages

TextFormat: ParseInfoTree supports getting field end location in addition to start.

... (truncated)

Commits

a902b39 No-op whitespace change
ae62acd Updating version.json and repo version numbers to: 18.3
f43ac49 Merge pull request #10542 from deannagarcia/3.18.x
9efdf55 Add missing includes
d1635e1 Apply patch
5b37c91 Update version.json with "lts": true (#10534)
c39d622 Merge pull request #10529 from protocolbuffers/deannagarcia-patch-5
f77d3b6 Update version.json
8178b06 Merge pull request #10503 from deannagarcia/3.18.x
24ca839 Add version file
Additional commits viewable in compare view

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Dependabot commands and options

Bump numpy from 1.14.6 to 1.22.0

opened on 2022-06-21 21:49:37 by dependabot[bot]

Bumps numpy from 1.14.6 to 1.22.0.

Release notes

Sourced from numpy's releases.

v1.22.0

NumPy 1.22.0 Release Notes

NumPy 1.22.0 is a big release featuring the work of 153 contributors spread over 609 pull requests. There have been many improvements, highlights are:

Annotations of the main namespace are essentially complete. Upstream is a moving target, so there will likely be further improvements, but the major work is done. This is probably the most user visible enhancement in this release.

A preliminary version of the proposed Array-API is provided. This is a step in creating a standard collection of functions that can be used across application such as CuPy and JAX.

NumPy now has a DLPack backend. DLPack provides a common interchange format for array (tensor) data.

New methods for quantile, percentile, and related functions. The new methods provide a complete set of the methods commonly found in the literature.

A new configurable allocator for use by downstream projects.

These are in addition to the ongoing work to provide SIMD support for commonly used functions, improvements to F2PY, and better documentation.

The Python versions supported in this release are 3.8-3.10, Python 3.7 has been dropped. Note that 32 bit wheels are only provided for Python 3.8 and 3.9 on Windows, all other wheels are 64 bits on account of Ubuntu, Fedora, and other Linux distributions dropping 32 bit support. All 64 bit wheels are also linked with 64 bit integer OpenBLAS, which should fix the occasional problems encountered by folks using truly huge arrays.

Expired deprecations

Deprecated numeric style dtype strings have been removed

Using the strings "Bytes0", "Datetime64", "Str0", "Uint32", and "Uint64" as a dtype will now raise a TypeError.

(gh-19539)

Expired deprecations for loads, ndfromtxt, and mafromtxt in npyio

numpy.loads was deprecated in v1.15, with the recommendation that users use pickle.loads instead. ndfromtxt and mafromtxt were both deprecated in v1.17 - users should use numpy.genfromtxt instead with the appropriate value for the usemask parameter.

(gh-19615)

... (truncated)

Commits

4adc87d Merge pull request #20685 from charris/prepare-for-1.22.0-release
fd66547 REL: Prepare for the NumPy 1.22.0 release.
125304b wip
c283859 Merge pull request #20682 from charris/backport-20416
5399c03 Merge pull request #20681 from charris/backport-20954
f9c45f8 Merge pull request #20680 from charris/backport-20663
794b36f Update armccompiler.py
d93b14e Update test_public_api.py
7662c07 Update init.py
311ab52 Update armccompiler.py
Additional commits viewable in compare view

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Dependabot commands and options

Releases

npj-sba-2022-08-19 2022-08-23 14:47:00

This is the newest snapshot release to support the manuscript titled An expanded whole-cell model of E. coli links cellular physiology with mechanisms of growth rate control published in npj Systems Biology and Applications, 19 August 2022.

One of the most significant changes is an upgrade from Python 2.7 to Python 3.8.

Biologically, the new version supports several new features by selecting simulation options in runSim.py with the noted flags or in fw_queue.py with the noted environment variables. Additional details are in the Methods section of the paper: - ppGpp dynamics and regulation of gene expression, RNAP binding, and translation (--ppgpp-regulation or PPGPP_REGULATION=1) - Dynamics of amino acid biosynthesis (--mechanistic-translation-supply or MECHANISTIC_TRANSLATION_SUPPLY=1) - Dynamics of amino acid transport (--mechanistic-aa-transport or MECHANISTIC_AA_TRANSPORT=1) - Dynamics of tRNA charging and ribosome elongation (--trna-charging or TRNA_CHARGING=1) - tRNA attenuation (--trna-attenuation or TRNA_ATTENUATION=1) - New environmental conditions (add or remove single amino acids, amino acid combinations, acetate and succinate) with new variants (--variant or VARIANT)

To run simulations as described in the paper with the entire set of new growth rate control options enabled, the following set of parameters should be used with fw_queue.py (along with any other desired options): TRNA_ATTENUATION=1 PPGPP_REGULATION=1 MECHANISTIC_TRANSLATION_SUPPLY=1 MECHANISTIC_AA_TRANSPORT=1 \ AA_SUPPLY_IN_CHARGING=1 D_PERIOD_DIVISION=1 MECHANISTIC_REPLISOME=0 TIMESTEP_MAX=1 \ python runscripts/fireworks/fw_queue.py

Other updates from the previously released version: - Framework to run simulations with Google Cloud Platform - Flat files describing genes and reactions have been updated to reflect newer versions on EcoCyc and additional data sources for other parameters have been added - Many analysis plots have been added or updated - Improvements to the modeling framework and helper classes and functions - Bug fixes and performance improvements

Science-2020-07-24 2020-11-18 22:07:39

This release snapshot Science-2020-07-24 goes with the paper Simultaneous cross-evaluation of heterogeneous E. coli datasets via mechanistic simulation published in Science, 24 July 2020.

See docs/README.md for info on setting up the Python 2.7 runtime environment to run this release.

(The next release will contain lots of work done since this snapshot forked off, and it runs on Python 3.8.)

Covert Lab

GitHub Repository

Release of the whole cell E. coli model.

Whole Cell Model - Escherichia coli

Setup

Quick start

Using the manual runscripts

Running a Fireworks workflow

Single simulation

Multiple simulations

Multiple generations

Shifting media conditions

Using the cached sim data object

Using an interactive Sherlock node to run a Fireworks workflow

Using the SLURM scheduler on Linux to run a Fireworks workflow

Output

Google Cloud Platform

Parameter search and optimization

Issues

Bump werkzeug from 1.0.1 to 2.2.3

2.2.3

2.2.2

2.2.1

2.2.0

2.1.2

2.1.1

2.1.0

2.0.3

Version 2.2.3

Version 2.2.2

Bump ipython from 7.16.1 to 8.10.0

See https://pypi.org/project/ipython/

Bump certifi from 2020.6.20 to 2022.12.7

Bump pillow from 7.2.0 to 9.3.0

9.3.0

Changes

9.3.0 (2022-10-29)

Bump protobuf from 3.15.8 to 3.18.3

Protocol Buffers v3.18.3

C++

Protocol Buffers v3.16.1

Java

Protocol Buffers v3.18.2

Java

Protocol Buffers v3.18.1

Python

Ruby

Protocol Buffers v3.18.0

C++

Bump numpy from 1.14.6 to 1.22.0

v1.22.0

NumPy 1.22.0 Release Notes

Expired deprecations

Deprecated numeric style dtype strings have been removed

Expired deprecations for loads, ndfromtxt, and mafromtxt in npyio

Releases

npj-sba-2022-08-19 2022-08-23 14:47:00

Science-2020-07-24 2020-11-18 22:07:39

Covert Lab

Expired deprecations for `loads`, `ndfromtxt`, and `mafromtxt` in npyio