Concreteness
victorssilva, updated
🕥
2022-11-22 03:49:24
Concreteness
An implementation of Quantifying the Visual Concreteness of Words and Topics in Multimodal Datasets with PyTorch.
It uses a ResNet50 along with Spotify's Annoy library to compute the visual concreteness scores of words from MIRFLICKR.
Requirements
To install the basic requirements, run this:
pip install -r requirements.txt
If you'd like use a Jupyter Notebook for interacting with the concreteness scores after computing them, you'll also need:
pip install -r requirements-notebook.txt
As of now, the existing code has only been tested with Python3.6 and Python 3.7.
For the MSCOCO dataset, you'd also have to download the English model for SpaCy by
python -m spacy download en
Usage
Downloading the dataset
Before running, you'll need to download the MIRFLICKR dataset. You can do that with:
cd data
./get_mirflickr.sh
It's 120GB, so it may take a while.
Similarly, you can get the MSCOCO dataset with:
cd data
./get_mscoco.sh
Shell usage
Once your download is finished, you can compute the concreteness scores with:
python main.py -d <mirflickr_directory> -c <cache_directory> -v
Swap in the path to where the mirflickr dataset was downloaded to and a directory of your choice to use for caching.
For the MSCOCO dataset, run with
python main.py -d <mscoco_directory> -c <cache_directory> -v -t mscoco
Jupyter Notebook
If you prefer, you can also run the provided Jupyter Notebook:
jupyter notebook concreteness.ipynb
TODO
- Improve Jupyter Notebook formatting
Thanks to
@jmhessel for helpful pointers and a great paper.
Citation:
@inproceedings{hessel2018concreteness,
title={Quantifying the visual concreteness of words and topics in multimodal datasets},
author={Hessel, Jack and Mimno, David and Lee, Lillian},
booktitle={NAACL},
year={2018}
}
Issues
Bump pillow from 5.3.0 to 9.3.0
opened on 2022-11-22 03:49:20 by dependabot[bot]Bumps pillow from 5.3.0 to 9.3.0.
Release notes
Sourced from pillow's releases.
9.3.0
https://pillow.readthedocs.io/en/stable/releasenotes/9.3.0.html
Changes
- Initialize libtiff buffer when saving #6699 [
@radarhere]- Limit SAMPLESPERPIXEL to avoid runtime DOS #6700 [
@wiredfool]- Inline fname2char to fix memory leak #6329 [
@nulano]- Fix memory leaks related to text features #6330 [
@nulano]- Use double quotes for version check on old CPython on Windows #6695 [
@hugovk]- GHA: replace deprecated set-output command with GITHUB_OUTPUT file #6697 [
@nulano]- Remove backup implementation of Round for Windows platforms #6693 [
@cgohlke]- Upload fribidi.dll to GitHub Actions #6532 [
@nulano]- Fixed set_variation_by_name offset #6445 [
@radarhere]- Windows build improvements #6562 [
@nulano]- Fix malloc in _imagingft.c:font_setvaraxes #6690 [
@cgohlke]- Only use ASCII characters in C source file #6691 [
@cgohlke]- Release Python GIL when converting images using matrix operations #6418 [
@hmaarrfk]- Added ExifTags enums #6630 [
@radarhere]- Do not modify previous frame when calculating delta in PNG #6683 [
@radarhere]- Added support for reading BMP images with RLE4 compression #6674 [
@npjg]- Decode JPEG compressed BLP1 data in original mode #6678 [
@radarhere]- pylint warnings #6659 [
@marksmayo]- Added GPS TIFF tag info #6661 [
@radarhere]- Added conversion between RGB/RGBA/RGBX and LAB #6647 [
@radarhere]- Do not attempt normalization if mode is already normal #6644 [
@radarhere]- Fixed seeking to an L frame in a GIF #6576 [
@radarhere]- Consider all frames when selecting mode for PNG save_all #6610 [
@radarhere]- Don't reassign crc on ChunkStream close #6627 [
@radarhere]- Raise a warning if NumPy failed to raise an error during conversion #6594 [
@radarhere]- Only read a maximum of 100 bytes at a time in IMT header #6623 [
@radarhere]- Show all frames in ImageShow #6611 [
@radarhere]- Allow FLI palette chunk to not be first #6626 [
@radarhere]- If first GIF frame has transparency for RGB_ALWAYS loading strategy, use RGBA mode #6592 [
@radarhere]- Round box position to integer when pasting embedded color #6517 [
@radarhere]- Removed EXIF prefix when saving WebP #6582 [
@radarhere]- Pad IM palette to 768 bytes when saving #6579 [
@radarhere]- Added DDS BC6H reading #6449 [
@ShadelessFox]- Added support for opening WhiteIsZero 16-bit integer TIFF images #6642 [
@JayWiz]- Raise an error when allocating translucent color to RGB palette #6654 [
@jsbueno]- Moved mode check outside of loops #6650 [
@radarhere]- Added reading of TIFF child images #6569 [
@radarhere]- Improved ImageOps palette handling #6596 [
@PososikTeam]- Defer parsing of palette into colors #6567 [
@radarhere]- Apply transparency to P images in ImageTk.PhotoImage #6559 [
@radarhere]- Use rounding in ImageOps contain() and pad() #6522 [
@bibinhashley]- Fixed GIF remapping to palette with duplicate entries #6548 [
@radarhere]- Allow remap_palette() to return an image with less than 256 palette entries #6543 [
@radarhere]- Corrected BMP and TGA palette size when saving #6500 [
@radarhere]
... (truncated)
Changelog
Sourced from pillow's changelog.
9.3.0 (2022-10-29)
Limit SAMPLESPERPIXEL to avoid runtime DOS #6700 [wiredfool]
Initialize libtiff buffer when saving #6699 [radarhere]
Inline fname2char to fix memory leak #6329 [nulano]
Fix memory leaks related to text features #6330 [nulano]
Use double quotes for version check on old CPython on Windows #6695 [hugovk]
Remove backup implementation of Round for Windows platforms #6693 [cgohlke]
Fixed set_variation_by_name offset #6445 [radarhere]
Fix malloc in _imagingft.c:font_setvaraxes #6690 [cgohlke]
Release Python GIL when converting images using matrix operations #6418 [hmaarrfk]
Added ExifTags enums #6630 [radarhere]
Do not modify previous frame when calculating delta in PNG #6683 [radarhere]
Added support for reading BMP images with RLE4 compression #6674 [npjg, radarhere]
Decode JPEG compressed BLP1 data in original mode #6678 [radarhere]
Added GPS TIFF tag info #6661 [radarhere]
Added conversion between RGB/RGBA/RGBX and LAB #6647 [radarhere]
Do not attempt normalization if mode is already normal #6644 [radarhere]
... (truncated)
Commits
d594f4cUpdate CHANGES.rst [ci skip]909dc649.3.0 version bump1a51ce7Merge pull request #6699 from hugovk/security-libtiff_buffer2444cddMerge pull request #6700 from hugovk/security-samples_per_pixel-sec744f455Added release notes0846bfaAdd to release notes799a6a0Fix linting00b25fdHide UserWarning in logs05b175eTighter test case13f2c5aPrevent DOS with large SAMPLESPERPIXEL in Tiff IFD- Additional commits viewable in compare view
Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) - `@dependabot use these labels` will set the current labels as the default for future PRs for this repo and language - `@dependabot use these reviewers` will set the current reviewers as the default for future PRs for this repo and language - `@dependabot use these assignees` will set the current assignees as the default for future PRs for this repo and language - `@dependabot use this milestone` will set the current milestone as the default for future PRs for this repo and language You can disable automated security fix PRs for this repo from the [Security Alerts page](https://github.com/victorssilva/concreteness/network/alerts).
Bump numpy from 1.15.4 to 1.22.0
opened on 2022-06-21 21:32:16 by dependabot[bot]Bumps numpy from 1.15.4 to 1.22.0.
Release notes
Sourced from numpy's releases.
v1.22.0
NumPy 1.22.0 Release Notes
NumPy 1.22.0 is a big release featuring the work of 153 contributors spread over 609 pull requests. There have been many improvements, highlights are:
- Annotations of the main namespace are essentially complete. Upstream is a moving target, so there will likely be further improvements, but the major work is done. This is probably the most user visible enhancement in this release.
- A preliminary version of the proposed Array-API is provided. This is a step in creating a standard collection of functions that can be used across application such as CuPy and JAX.
- NumPy now has a DLPack backend. DLPack provides a common interchange format for array (tensor) data.
- New methods for
quantile,percentile, and related functions. The new methods provide a complete set of the methods commonly found in the literature.- A new configurable allocator for use by downstream projects.
These are in addition to the ongoing work to provide SIMD support for commonly used functions, improvements to F2PY, and better documentation.
The Python versions supported in this release are 3.8-3.10, Python 3.7 has been dropped. Note that 32 bit wheels are only provided for Python 3.8 and 3.9 on Windows, all other wheels are 64 bits on account of Ubuntu, Fedora, and other Linux distributions dropping 32 bit support. All 64 bit wheels are also linked with 64 bit integer OpenBLAS, which should fix the occasional problems encountered by folks using truly huge arrays.
Expired deprecations
Deprecated numeric style dtype strings have been removed
Using the strings
"Bytes0","Datetime64","Str0","Uint32", and"Uint64"as a dtype will now raise aTypeError.(gh-19539)
Expired deprecations for
loads,ndfromtxt, andmafromtxtin npyio
numpy.loadswas deprecated in v1.15, with the recommendation that users usepickle.loadsinstead.ndfromtxtandmafromtxtwere both deprecated in v1.17 - users should usenumpy.genfromtxtinstead with the appropriate value for theusemaskparameter.(gh-19615)
... (truncated)
Commits
4adc87dMerge pull request #20685 from charris/prepare-for-1.22.0-releasefd66547REL: Prepare for the NumPy 1.22.0 release.125304bwipc283859Merge pull request #20682 from charris/backport-204165399c03Merge pull request #20681 from charris/backport-20954f9c45f8Merge pull request #20680 from charris/backport-20663794b36fUpdate armccompiler.pyd93b14eUpdate test_public_api.py7662c07Update init.py311ab52Update armccompiler.py- Additional commits viewable in compare view
Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) - `@dependabot use these labels` will set the current labels as the default for future PRs for this repo and language - `@dependabot use these reviewers` will set the current reviewers as the default for future PRs for this repo and language - `@dependabot use these assignees` will set the current assignees as the default for future PRs for this repo and language - `@dependabot use this milestone` will set the current milestone as the default for future PRs for this repo and language You can disable automated security fix PRs for this repo from the [Security Alerts page](https://github.com/victorssilva/concreteness/network/alerts).
Bump ipython from 7.1.1 to 7.16.3
opened on 2022-01-21 19:36:53 by dependabot[bot]Bumps ipython from 7.1.1 to 7.16.3.
Commits
d43c7c7release 7.16.35fa1e40Merge pull request from GHSA-pq7m-3gw7-gq5x8df8971back to dev9f477b7release 7.16.2138f266bring back release helper from master branch5aa3634Merge pull request #13341 from meeseeksmachine/auto-backport-of-pr-13335-on-7...bcae8e0Backport PR #13335: What's new 7.16.28fcdcd3Pin Jedi to <0.17.2.2486838release 7.16.120bdc6ffix conda build- Additional commits viewable in compare view
Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) - `@dependabot use these labels` will set the current labels as the default for future PRs for this repo and language - `@dependabot use these reviewers` will set the current reviewers as the default for future PRs for this repo and language - `@dependabot use these assignees` will set the current assignees as the default for future PRs for this repo and language - `@dependabot use this milestone` will set the current milestone as the default for future PRs for this repo and language You can disable automated security fix PRs for this repo from the [Security Alerts page](https://github.com/victorssilva/concreteness/network/alerts).
Dumped tags.json keys are changed from int to str
opened on 2020-06-19 03:14:19 by VimosHi, the dumped tags.json cannot be used correctly when loaded back. This is because the key data type from int to str. We may patch the code like the following:
diff --git a/mscoco.py b/mscoco.py
index 2f0dc9d..9e422cb 100644
--- a/mscoco.py
+++ b/mscoco.py
@@ -43,6 +43,7 @@ def build_tags_json(annotation_path, tags_json_filename):
def load_tags_json(tags_json_filename):
with open(tags_json_filename, 'r') as tags_json_file:
image_paths, image_tags = json.load(tags_json_file)
+ image_tags = {int(k): v for k, v in image_tags.items()}
return image_paths, image_tags