US Bill Summarization Corpus

FiscalNote, updated 🕥 2023-02-11 01:13:13

BillSum

Code for the paper: BillSum: A Corpus for Automatic Summarization of US Legislation (Kornilova and Eidelman, 2019)

This paper was be presented at EMNLP 2019 Workshop on New Frontiers in Summarization. Link to slides from workshop

Accessing the Dataset: This dataset was updated on 12/3/2019, if you accessed the dataset prior to this date, please redownload it.

Link to Google Drive

TensorFlow Datasets - does not contain the "clean" versions of the texts

If you do something cool with the data, share on our Kaggle Page!

Information on how the dataset was collected is under BillSum_Data_Documentation.md

Data Structure The data is stored in a jsonlines format, with one bill per line.

text: bill text
clean_text: a preprocessed version of the text that was used to train the models in the paper
summary: (human-written) bill summary
title: bill title (can be used for generating a summary)
bill_id: An identified for the bill - in US data it is SESSION_BILL-ID, for CA BILL-ID

Set-up

Install python dependencies (If using conda, use env.lst. If using pip, use requirements.txt)
Set the env BILLSUM_PREFIX to the base directory for all the data. (Download from link above)
Set PYTHONPATH=. to run code from this directory.
Install packages from environment.lst (we used conda, but you should be able to use pip

Experiments

The results for the intermediate steps (explained below) can be found here

For all the experiments described in the paper, the texts were first cleaned using the script billsum/data_prep/clean_text.py. Results will be saved into the BILLSUM_PREFIX/clean_final directory.

Sumy baselines

Clone sumy and checkout the branch ak_fork (This is a minor modification on the original sumy library that allows it to work with my sentence selection logic).
In that directory run pip install -e .
From this directory, run bill_sum/sumy_baselines.py

Supervised Experiments

Preparing the data

Run billsum/data_prep/clean_text.py to clean up the whitespace formatting in the dataset. Outputs new jsonlines files with 'clean_text' field + original fields to BILLSUM_PREFIX/clean_data
Run billsum/data_prep/label_sentences.py to create labeled dataset.

This script takes each document, splits it into sentences, processes them with Spacy to get useful syntactic features and calculates the Rouge Score relative to the summary.

Outputs for each dataset part will be a pickle file with a dict of (bill_id, sentence data) pairs. (Stored under PREFIX/sent_data/) directory

Bill_id --> [ ('The monthly limitation for each coverage month during the taxable year is an amount equal to the lesser of 50 percent of the amount paid for qualified health insurance for such month, or an amount equal to 112 of in the case of self-only coverage, $1,320, and in the case of family coverage, $3,480. ', [('The ', 186, 'the', '', 'O', 'DET', 'det', 188), ('monthly ', 187, 'monthly', 'DATE', 'B', 'ADJ', 'amod', 188), ('limitation ', 188, 'limitation', '', 'O', 'NOUN', 'nsubj', 197), ...] {'rouge-1': {'f': 0.2545454500809918, 'p': 0.3783783783783784, 'r': 0.1917808219178082}, 'rouge-2': {'f': 0.09459459021183367, 'p': 0.14583333333333334, 'r': 0.07}, 'rouge-l': {'f': 0.16757568176139123, 'p': 0.2972972972972973, 'r': 0.1506849315068493}}), ...]

Running Bert Models

Clone https://github.com/google-research/bert. Replace the run_classifier.py file with billsum/bert_helpers/run_classifier.py (adds custom code to read data in and out of files). Install dependencies as described in this repo.
Create train.tsv / test.tsv files with billsum/bert_helpers/prep_bert.py. These will be stored under PREFIX/bert_data (set $BERT_DATA_DIR to point here)
Download the Bert-Large, Uncased model.
Set $BERT_BASE_DIR environment variable to point to directory where you downloaded the model
Pretrain the Bert Model (run from the cloned bert repo)

python create_pretraining_data.py \ --input_file=$BERT_DATA_DIR/all_texts_us_train.txt \ --output_file=$BERT_DATA_DIR/all_texts_us_train.tfrecord \ --vocab_file=$BERT_BASE_DIR/vocab.txt \ --do_lower_case=True \ --max_seq_length=128 \ --max_predictions_per_seq=20 \ --masked_lm_prob=0.15 \ --random_seed=12345 \ --dupe_factor=5

Set $BERT_MODEL_DIR to the directory where you want to store your pretrained model.

python run_pretraining.py \ --input_file=$BERT_DATA_DIR/all_texts_us_train.tfrecord\ --output_dir=$BERT_MODEL_DIR \ --do_train=True \ --do_eval=True \ --bert_config_file=$BERT_BASE_DIR/bert_config.json \ --init_checkpoint=$BERT_BASE_DIR/bert_model.ckpt \ --train_batch_size=32 \ --max_seq_length=128 \ --max_predictions_per_seq=20 \ --num_train_steps=20000 \ --num_warmup_steps=10 \ --learning_rate=2e-5

This will take a while to run.

To train the classifier model run (from bert repo):

python run_classifier.py --task_name=simple --do_train=true --do_predict=true --do_predict_ca=true --data_dir=$BERT_DATA_DIR --vocab_file=$BERT_BASE_DIR/vocab.txt --bert_config_file=$BERT_BASE_DIR/bert_config.json --init_checkpoint=$BERT_MODEL_DIR/model.ckpt-40000 --max_seq_length=128 --train_batch_size=32 --num_train_epochs=3.0 --output_dir=$BERT_CLASSIFIER_DIR

Change BERT_CLASSIFIER_DIR to the directory where you want to store the classifier - should be different from pretraining directory. This script will create a model in the BERT_CLASSIFIER_DIR and store the sentence predictions in BERT_CLASSIFIER_DIR/ dir.

For clarity: - BERT_BASE_DIR: directory of the original downloaded model (same as for step 3) - BERT_MODEL_DIR: directory where the output of the pretraining was stored - BERT_DATA_DIR: directory with all train/test examples - BERT_CLASSIFIER_DIR: directory where new model should

After this procedure is run, two files will be generated in the BERT_CLASSIFIER_DIR: test_results.tsv / ca_test_results.tsv -- this contain sentence level predictions for each test sentence. Rename the test_results.tsv file to us_test_results.tsv. Then copy both of them over to the bert_data folder.

Evaluate results using bill_sum/bert_helpers/evaluate_bert.py. Change the prefix variable to point to BERT_CLASSIFIER_DIR from above.

Results will be stored under BILLSUM_PREFIX/score_data/

Running feature classifier + ensemble

Run bill_sum/train_wrapper.py. Results will be stored under BILLSUM_PREFIX/score_data/

To get computations for the ensemble method run billsum/evaluate_ensemble.py

Final Result aggregation

The PrintFinalScores.ipynb will compute the summary statistics for each method + generate the Oracle scores.

Issues

Bump ipython from 7.8.0 to 8.10.0

opened on 2023-02-11 01:13:09 by dependabot[bot]

Bumps ipython from 7.8.0 to 8.10.0.

Release notes

Sourced from ipython's releases.

See https://pypi.org/project/ipython/

We do not use GitHub release anymore. Please see PyPI https://pypi.org/project/ipython/

7.9.0

No release notes provided.

Commits

15ea1ed release 8.10.0
560ad10 DOC: Update what's new for 8.10 (#13939)
7557ade DOC: Update what's new for 8.10
385d693 Merge pull request from GHSA-29gw-9793-fvw7
e548ee2 Swallow potential exceptions from showtraceback() (#13934)
0694b08 MAINT: mock slowest test. (#13885)
8655912 MAINT: mock slowest test.
a011765 Isolate the attack tests with setUp and tearDown methods
c7a9470 Add some regression tests for this change
fd34cf5 Swallow potential exceptions from showtraceback()
Additional commits viewable in compare view

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) - `@dependabot use these labels` will set the current labels as the default for future PRs for this repo and language - `@dependabot use these reviewers` will set the current reviewers as the default for future PRs for this repo and language - `@dependabot use these assignees` will set the current assignees as the default for future PRs for this repo and language - `@dependabot use this milestone` will set the current milestone as the default for future PRs for this repo and language You can disable automated security fix PRs for this repo from the [Security Alerts page](https://github.com/FiscalNote/BillSum/network/alerts).

Bump cryptography from 2.7 to 39.0.1

opened on 2023-02-07 22:40:15 by dependabot[bot]

Bumps cryptography from 2.7 to 39.0.1.

Changelog

Sourced from cryptography's changelog.

39.0.1 - 2023-02-07
* **SECURITY ISSUE** - Fixed a bug where ``Cipher.update_into`` accepted Python
  buffer protocol objects, but allowed immutable buffers. **CVE-2023-23931**
* Updated Windows, macOS, and Linux wheels to be compiled with OpenSSL 3.0.8.
.. _v39-0-0:
39.0.0 - 2023-01-01
BACKWARDS INCOMPATIBLE: Support for OpenSSL 1.1.0 has been removed. Users on older version of OpenSSL will need to upgrade.

BACKWARDS INCOMPATIBLE: Dropped support for LibreSSL < 3.5. The new minimum LibreSSL version is 3.5.0. Going forward our policy is to support versions of LibreSSL that are available in versions of OpenBSD that are still receiving security support.

BACKWARDS INCOMPATIBLE: Removed the encode_point and from_encoded_point methods on :class:~cryptography.hazmat.primitives.asymmetric.ec.EllipticCurvePublicNumbers, which had been deprecated for several years. :meth:~cryptography.hazmat.primitives.asymmetric.ec.EllipticCurvePublicKey.public_bytes and :meth:~cryptography.hazmat.primitives.asymmetric.ec.EllipticCurvePublicKey.from_encoded_point should be used instead.

BACKWARDS INCOMPATIBLE: Support for using MD5 or SHA1 in :class:~cryptography.x509.CertificateBuilder, other X.509 builders, and PKCS7 has been removed.

BACKWARDS INCOMPATIBLE: Dropped support for macOS 10.10 and 10.11, macOS users must upgrade to 10.12 or newer.

ANNOUNCEMENT: The next version of cryptography (40.0) will change the way we link OpenSSL. This will only impact users who build cryptography from source (i.e., not from a wheel), and specify their own version of OpenSSL. For those users, the CFLAGS, LDFLAGS, INCLUDE, LIB, and CRYPTOGRAPHY_SUPPRESS_LINK_FLAGS environment variables will no longer be respected. Instead, users will need to configure their builds as documented here_.

Added support for :ref:disabling the legacy provider in OpenSSL 3.0.x<legacy-provider>.

Added support for disabling RSA key validation checks when loading RSA keys via :func:~cryptography.hazmat.primitives.serialization.load_pem_private_key, :func:~cryptography.hazmat.primitives.serialization.load_der_private_key, and :meth:~cryptography.hazmat.primitives.asymmetric.rsa.RSAPrivateNumbers.private_key. This speeds up key loading but is :term:unsafe if you are loading potentially attacker supplied keys.

Significantly improved performance for :class:~cryptography.hazmat.primitives.ciphers.aead.ChaCha20Poly1305

... (truncated)

Commits

d6951dc changelog + security fix backport (#8231)
138da90 workaround scapy bug in downstream tests (#8218) (#8228)
69527bc bookworm is py311 now (#8200)
111deef backport main branch CI to 39.0.x (#8153)
338a65a 39.0.0 version bump (#7954)
84a3cd7 automatically download and upload circleci wheels (#7949)
525c0b3 Type annotate release.py (#7951)
46d2a94 Use the latest 3.10 release when wheel building (#7953)
f150dc1 fix CI to work with ubuntu 22.04 (#7950)
8867724 fix README for python3 (#7947)
Additional commits viewable in compare view

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Dependabot commands and options

Bump certifi from 2019.9.11 to 2022.12.7

opened on 2022-12-08 06:39:17 by dependabot[bot]

Bumps certifi from 2019.9.11 to 2022.12.7.

Commits

9e9e840 2022.12.07
b81bdb2 2022.09.24
939a28f 2022.09.14
aca828a 2022.06.15.2
de0eae1 Only use importlib.resources's new files() / Traversable API on Python ≥3.11 ...
b8eb5e9 2022.06.15.1
47fb7ab Fix deprecation warning on Python 3.11 (#199)
b0b48e0 fixes #198 -- update link in license
9d514b4 2022.06.15
4151e88 Add py.typed to MANIFEST.in to package in sdist (#196)
Additional commits viewable in compare view

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Dependabot commands and options

Bump joblib from 0.13.2 to 1.2.0

opened on 2022-09-30 19:45:38 by dependabot[bot]

Bumps joblib from 0.13.2 to 1.2.0.

Changelog

Sourced from joblib's changelog.

Release 1.2.0

Fix a security issue where eval(pre_dispatch) could potentially run arbitrary code. Now only basic numerics are supported. joblib/joblib#1327

Make sure that joblib works even when multiprocessing is not available, for instance with Pyodide joblib/joblib#1256

Avoid unnecessary warnings when workers and main process delete the temporary memmap folder contents concurrently. joblib/joblib#1263

Fix memory alignment bug for pickles containing numpy arrays. This is especially important when loading the pickle with mmap_mode != None as the resulting numpy.memmap object would not be able to correct the misalignment without performing a memory copy. This bug would cause invalid computation and segmentation faults with native code that would directly access the underlying data buffer of a numpy array, for instance C/C++/Cython code compiled with older GCC versions or some old OpenBLAS written in platform specific assembly. joblib/joblib#1254

Vendor cloudpickle 2.2.0 which adds support for PyPy 3.8+.

Vendor loky 3.3.0 which fixes several bugs including:

robustly forcibly terminating worker processes in case of a crash (joblib/joblib#1269);

avoiding leaking worker processes in case of nested loky parallel calls;

reliability spawn the correct number of reusable workers.

Release 1.1.0

Fix byte order inconsistency issue during deserialization using joblib.load in cross-endian environment: the numpy arrays are now always loaded to use the system byte order, independently of the byte order of the system that serialized the pickle. joblib/joblib#1181

Fix joblib.Memory bug with the ignore parameter when the cached function is a decorated function.

... (truncated)

Commits

5991350 Release 1.2.0
3fa2188 MAINT cleanup numpy warnings related to np.matrix in tests (#1340)
cea26ff CI test the future loky-3.3.0 branch (#1338)
8aca6f4 MAINT: remove pytest.warns(None) warnings in pytest 7 (#1264)
067ed4f XFAIL test_child_raises_parent_exits_cleanly with multiprocessing (#1339)
ac4ebd5 MAINT add back pytest warnings plugin (#1337)
a23427d Test child raises parent exits cleanly more reliable on macos (#1335)
ac09691 [MAINT] various test updates (#1334)
4a314b1 Vendor loky 3.2.0 (#1333)
bdf47e9 Make test_parallel_with_interactively_defined_functions_default_backend timeo...
Additional commits viewable in compare view

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Dependabot commands and options

Bump nbconvert from 5.6.0 to 6.5.1

opened on 2022-08-23 17:53:14 by dependabot[bot]

Bumps nbconvert from 5.6.0 to 6.5.1.

Release notes

Sourced from nbconvert's releases.

Release 6.5.1

No release notes provided.

6.5.0

What's Changed

Drop dependency on testpath. by @anntzer in jupyter/nbconvert#1723

Adopt pre-commit by @blink1073 in jupyter/nbconvert#1744

Add pytest settings and handle warnings by @blink1073 in jupyter/nbconvert#1745

Apply Autoformatters by @blink1073 in jupyter/nbconvert#1746

Add git-blame-ignore-revs by @blink1073 in jupyter/nbconvert#1748

Update flake8 config by @blink1073 in jupyter/nbconvert#1749

support bleach 5, add packaging and tinycss2 dependencies by @bollwyvl in jupyter/nbconvert#1755

[pre-commit.ci] pre-commit autoupdate by @pre-commit-ci in jupyter/nbconvert#1752

update cli example by @leahecole in jupyter/nbconvert#1753

Clean up pre-commit by @blink1073 in jupyter/nbconvert#1757

Clean up workflows by @blink1073 in jupyter/nbconvert#1750

New Contributors

@pre-commit-ci made their first contribution in jupyter/nbconvert#1752

Full Changelog: https://github.com/jupyter/nbconvert/compare/6.4.5...6.5

6.4.3

What's Changed

Add section to customizing showing how to use template inheritance by @stefanv in jupyter/nbconvert#1719

Remove ipython genutils by @rgs258 in jupyter/nbconvert#1727

Update changelog for 6.4.3 by @blink1073 in jupyter/nbconvert#1728

New Contributors

@stefanv made their first contribution in jupyter/nbconvert#1719

@rgs258 made their first contribution in jupyter/nbconvert#1727

Full Changelog: https://github.com/jupyter/nbconvert/compare/6.4.2...6.4.3

6.4.0

What's Changed

Optionally speed up validation by @gwincr11 in jupyter/nbconvert#1672

Adding missing div compared to JupyterLab DOM structure by @SylvainCorlay in jupyter/nbconvert#1678

Allow passing extra args to code highlighter by @yuvipanda in jupyter/nbconvert#1683

Prevent page breaks in outputs when printing by @SylvainCorlay in jupyter/nbconvert#1679

Add collapsers to template by @SylvainCorlay in jupyter/nbconvert#1689

Fix recent pandoc latex tables by adding calc and array (#1536, #1566) by @cgevans in jupyter/nbconvert#1686

Add an invalid notebook error by @gwincr11 in jupyter/nbconvert#1675

Fix typos in execute.py by @TylerAnderson22 in jupyter/nbconvert#1692

Modernize latex greek math handling (partially fixes #1673) by @cgevans in jupyter/nbconvert#1687

Fix use of deprecated API and update test matrix by @blink1073 in jupyter/nbconvert#1696

Update nbconvert_library.ipynb by @letterphile in jupyter/nbconvert#1695

Changelog for 6.4 by @blink1073 in jupyter/nbconvert#1697

New Contributors

... (truncated)

Commits

7471b75 Release 6.5.1
c1943e0 Fix pre-commit
8685e93 Fix tests
0abf290 Run black and prettier
418d545 Run test on 6.x branch
bef65d7 Convert input to string prior to escape HTML
0818628 Check input type before escaping
b206470 GHSL-2021-1017, GHSL-2021-1020, GHSL-2021-1021
a03cbb8 GHSL-2021-1026, GHSL-2021-1025
48fe71e GHSL-2021-1024
Additional commits viewable in compare view

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Dependabot commands and options

Bump mistune from 0.8.4 to 2.0.3

opened on 2022-07-29 22:56:39 by dependabot[bot]

Bumps mistune from 0.8.4 to 2.0.3.

Release notes

Sourced from mistune's releases.

Version 2.0.2

Fix escape_url via lepture/mistune#295

Version 2.0.1

Fix XSS for image link syntax.

Version 2.0.0

First release of Mistune v2.

Version 2.0.0 RC1

In this release, we have a Security Fix for harmful links.

Version 2.0.0 Alpha 1

This is the first release of v2. An alpha version for users to have a preview of the new mistune.

Changelog

Sourced from mistune's changelog.

Changelog

Here is the full history of mistune v2.

Version 2.0.4
Released on Jul 15, 2022

Fix url plugin in &lt;a&gt; tag
Fix * formatting

Version 2.0.3
Released on Jun 27, 2022

Fix table plugin

Security fix for CVE-2022-34749

Version 2.0.2
Released on Jan 14, 2022
Fix escape_url
Version 2.0.1
Released on Dec 30, 2021

XSS fix for image link syntax.

Version 2.0.0
Released on Dec 5, 2021
This is the first non-alpha release of mistune v2.
Version 2.0.0rc1
Released on Feb 16, 2021

Version 2.0.0a6
</tr></table> 

... (truncated)

Commits

3f422f1 Version bump 2.0.3
a6d4321 Fix asteris emphasis regex CVE-2022-34749
5638e46 Merge pull request #307 from jieter/patch-1
0eba471 Fix typo in guide.rst
61e9337 Fix table plugin
76dec68 Add documentation for renderer heading when TOC enabled
799cd11 Version bump 2.0.2
babb0cf Merge pull request #295 from dairiki/bug.escape_url
fc2cd53 Make mistune.util.escape_url less aggressive
3e8d352 Version bump 2.0.1
Additional commits viewable in compare view

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Dependabot commands and options

FiscalNote, Inc.

GitHub Repository

nlp law summarization dataset natural-language-processing