Translation layer between the GDC Data Dictionary and psqlgraph

NCI-GDC, updated 🕥 2023-03-20 18:57:05

GDC Data Model

Repo to keep information about the GDC data model design.

GDC Data Model
Installation
Jupyter + Graphviz
Documentation
- Visual representation
Dependencies
- Project Dependencies
Example validation usage
Tests
Setup pre-commit hook to check for secrets
Contributing

Installation

To install the gdcdatamodel library run the setup script: ❯ python setup.py install

Jupyter + Graphviz

It's helpful to examine the relationships between nodes visually. One way to do this is to run a Jupyter notebook with a Python2 kernel. When used with Graphviz's SVG support, you can view a graphical representation of a subgraph directly in a REPL. To do so, install the dev-requirements.txt dependencies. There is an example Jupyter notebook at examples/jupyter_example.ipynb (replicated in examples/jupyter_example.py for clarity)

pip install -r dev-requirements PG_USER=* PG_HOST=* PG_DATABASE=* PG_PASSWORD=* jupyter notebook examples/jupyter_example.ipynb

Documentation

Visual representation

For instructions on how to build the Graphviz representation of the datamodel, see the docs readme.

Dependencies

Before continuing you must have the following programs installed:

Python 2.7+

The gdcdatamodel library requires the following pip dependencies

Project Dependencies

Project dependencies are managed using PIP

Example validation usage

``` from gdcdatamodel import node_avsc_object from gdcdatamodel.mappings import get_participant_es_mapping, get_file_es_mapping from avro.io import validate import json

with open('examples/nodes/aliquot_valid.json', 'r') as f: node = json.loads(f.read()) print validate(node_avsc_object, node) # if valid, prints True

print(get_participant_es_mapping()) # Prints participant elasticsearch mapping print(get_file_es_mapping()) # Prints file elasticsearch mapping ```

Tests

``` ❯ nosetests -v test_invalid_aliquot_node (test_avro_schemas.TestAvroSchemaValidation) ... ok test_valid_aliquot_node (test_avro_schemas.TestAvroSchemaValidation) ... ok

Ran 2 tests in 0.033s

OK ```

Setup pre-commit hook to check for secrets

We use pre-commit to setup pre-commit hooks for this repo. We use detect-secrets to search for secrets being committed into the repo.

To install the pre-commit hook, run pre-commit install

To update the .secrets.baseline file run detect-secrets scan --update .secrets.baseline

.secrets.baseline contains all the string that were caught by detect-secrets but are not stored in plain text. Audit the baseline to view the secrets .

detect-secrets audit .secrets.baseline

Contributing

Read how to contribute here

Issues

replace https with ssh for github links in deps

opened on 2023-03-20 18:54:37 by qiaouchicago None

Bump ipython from 7.9.0 to 8.10.0

opened on 2023-02-11 01:27:34 by dependabot[bot]

Bumps ipython from 7.9.0 to 8.10.0.

Release notes

Sourced from ipython's releases.

See https://pypi.org/project/ipython/

We do not use GitHub release anymore. Please see PyPI https://pypi.org/project/ipython/

Commits

15ea1ed release 8.10.0
560ad10 DOC: Update what's new for 8.10 (#13939)
7557ade DOC: Update what's new for 8.10
385d693 Merge pull request from GHSA-29gw-9793-fvw7
e548ee2 Swallow potential exceptions from showtraceback() (#13934)
0694b08 MAINT: mock slowest test. (#13885)
8655912 MAINT: mock slowest test.
a011765 Isolate the attack tests with setUp and tearDown methods
c7a9470 Add some regression tests for this change
fd34cf5 Swallow potential exceptions from showtraceback()
Additional commits viewable in compare view

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) - `@dependabot use these labels` will set the current labels as the default for future PRs for this repo and language - `@dependabot use these reviewers` will set the current reviewers as the default for future PRs for this repo and language - `@dependabot use these assignees` will set the current assignees as the default for future PRs for this repo and language - `@dependabot use this milestone` will set the current milestone as the default for future PRs for this repo and language You can disable automated security fix PRs for this repo from the [Security Alerts page](https://github.com/NCI-GDC/gdcdatamodel/network/alerts).

Update README.md

opened on 2023-01-24 19:32:06 by stilesj-uchicago None

chore(deps-dev): bump jupyter-core from 4.6.3 to 4.11.2

opened on 2023-01-04 15:10:21 by dependabot[bot]

Bumps jupyter-core from 4.6.3 to 4.11.2.

Release notes

Sourced from jupyter-core's releases.

4.11.1

What's Changed

Fix inclusion of jupyter file and check in CI by @blink1073 in jupyter/jupyter_core#276

Full Changelog: https://github.com/jupyter/jupyter_core/compare/4.11.0...4.11.1

4.11.0

What's Changed

Use hatch backend by @blink1073 in jupyter/jupyter_core#265

is_hidden: Use normalized paths by @martinRenou in jupyter/jupyter_core#271

New Contributors

@martinRenou made their first contribution in jupyter/jupyter_core#271

Full Changelog: https://github.com/jupyter/jupyter_core/compare/4.10.0...4.11.0

4.10.0

What's Changed

Update changelog for 4.9.2 by @blink1073 in jupyter/jupyter_core#252

Include all files from jupyter_core by @jonringer in jupyter/jupyter_core#253

Add project URLs to setup.cfg by @tlinhart in jupyter/jupyter_core#254

Set up pre-commit by @blink1073 in jupyter/jupyter_core#255

[pre-commit.ci] pre-commit autoupdate by @pre-commit-ci in jupyter/jupyter_core#257

Add flake8 and mypy settings by @blink1073 in jupyter/jupyter_core#256

Clean up CI by @blink1073 in jupyter/jupyter_core#258

Update changelog for 4.10 Release by @blink1073 in jupyter/jupyter_core#259

New Contributors

@jonringer made their first contribution in jupyter/jupyter_core#253

@tlinhart made their first contribution in jupyter/jupyter_core#254

@pre-commit-ci made their first contribution in jupyter/jupyter_core#257

Full Changelog: https://github.com/jupyter/jupyter_core/compare/4.9.2...4.10.0

4.9.2

What's Changed

set proper sys.argv[0] for subcommand by @bnavigator in jupyter/jupyter_core#248

Add explicit encoding in open calls by @dlukes in jupyter/jupyter_core#249

jupyter_config_dir - reorder home_dir initialization by @dharmaquark in jupyter/jupyter_core#251

New Contributors

@bnavigator made their first contribution in jupyter/jupyter_core#248

@dlukes made their first contribution in jupyter/jupyter_core#249

@dharmaquark made their first contribution in jupyter/jupyter_core#251

Full Changelog: https://github.com/jupyter/jupyter_core/compare/4.9.1...4.9.2

Changelog

Sourced from jupyter-core's changelog.

Changes in jupyter-core

5.1.2

(Full Changelog)

Maintenance and upkeep improvements

Add spell check and ensure docstrings #327 (@blink1073)

Maint: Some more precise typing. #326 (@Carreau)

Documentation improvements

Add api docs #328 (@blink1073)

Contributors to this release

(GitHub contributors page for this release)

@blink1073 | @Carreau

5.1.1

(Full Changelog)

Enhancements made

Only prefer envs owned by the current user #323 (@minrk)

Bugs fixed

Don't treat the conda root env as an env #324 (@minrk)

Maintenance and upkeep improvements

Fix lint #325 (@blink1073)

Adopt ruff and address lint #321 (@blink1073)

Contributors to this release

(GitHub contributors page for this release)

@blink1073 | @jasongrout | @minrk | @pre-commit-ci

5.1.0

... (truncated)

Commits

a8eac8c Release 4.11.2
1118c8c Merge pull request from GHSA-m678-f26j-3hrp
d3f61f3 Release 4.11.1
e7eeb9e Fix inclusion of jupyter file and check in CI (#276)
035bf11 Release 4.11.0
45aa28b [pre-commit.ci] pre-commit autoupdate (#273)
73401cc [pre-commit.ci] pre-commit autoupdate (#272)
bc6b771 is_hidden: Use normalized paths (#271)
be38e52 [pre-commit.ci] pre-commit autoupdate (#270)
7b790d8 [pre-commit.ci] pre-commit autoupdate (#269)
Additional commits viewable in compare view

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Dependabot commands and options

Bump pyyaml from 5.3.1 to 5.4

opened on 2021-03-26 00:00:34 by dependabot[bot]

Bumps pyyaml from 5.3.1 to 5.4.

Changelog

Sourced from pyyaml's changelog.

5.4 (2021-01-19)

yaml/pyyaml#407 -- Build modernization, remove distutils, fix metadata, build wheels, CI to GHA

yaml/pyyaml#472 -- Fix for CVE-2020-14343, moves arbitrary python tags to UnsafeLoader

yaml/pyyaml#441 -- Fix memory leak in implicit resolver setup

yaml/pyyaml#392 -- Fix py2 copy support for timezone objects

yaml/pyyaml#378 -- Fix compatibility with Jython

Commits

58d0cb7 5.4 release
a60f7a1 Fix compatibility with Jython
ee98abd Run CI on PR base branch changes
ddf2033 constructor.timezone: _copy & deepcopy
fc914d5 Avoid repeatedly appending to yaml_implicit_resolvers
a001f27 Fix for CVE-2020-14343
fe15062 Add 3.9 to appveyor file for completeness sake
1e1c7fb Add a newline character to end of pyproject.toml
0b6b7d6 Start sentences and phrases for capital letters
c976915 Shell code improvements
Additional commits viewable in compare view

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Dependabot commands and options

[Snyk] Security upgrade py from 1.8.1 to 1.10.0

opened on 2020-12-15 04:31:46 by snyk-bot

Snyk has created this PR to fix one or more vulnerable packages in the `pip` dependencies of this project.

Changes included in this PR

Changes to the following files to upgrade the vulnerable dependencies to a fixed version:
- dev-requirements.txt

Vulnerabilities that will be fixed

By pinning:

Severity | Priority Score () | Issue | Upgrade | Breaking Change | Exploit Maturity :-------------------------:|-------------------------|:-------------------------|:-------------------------|:-------------------------|:------------------------- | 608/1000
Why?* Proof of Concept exploit, Recently disclosed, Has a fix available, CVSS 4.3 | Regular Expression Denial of Service (ReDoS)
SNYK-PYTHON-PY-1049546 | py:
1.8.1 -> 1.10.0
| No | Proof of Concept

(*) Note that the real score may have changed since the PR was raised.

Some vulnerabilities couldn't be fully fixed and so Snyk will still find them when the project is tested again. This may be because the vulnerability existed within more than one direct dependency, but not all of the effected dependencies could be upgraded.

Check the changes in this PR to ensure they won't cause issues with your project.

Note: You are seeing this because you or someone else with access to this repository has authorized Snyk to open fix PRs.

For more information: 🧐 View latest project report

🛠 Adjust project settings

Releases

Elion 2023-01-26 22:15:13

What's Changed

Release/curie by @ProfOak in https://github.com/NCI-GDC/gdcdatamodel/pull/371
Release/dr32 by @ProfOak in https://github.com/NCI-GDC/gdcdatamodel/pull/379
Chore: Fix build errors in release/darwin by @mpsolano in https://github.com/NCI-GDC/gdcdatamodel/pull/384
Release/darwin by @mpsolano in https://github.com/NCI-GDC/gdcdatamodel/pull/383
Merge master to develop after darwin release by @mpsolano in https://github.com/NCI-GDC/gdcdatamodel/pull/385
DEV 1403 fix bug for backrefs by @qiaouchicago in https://github.com/NCI-GDC/gdcdatamodel/pull/387
Feat/dev 1406 upgrade pre commit hooks for gdcdatamodel and reformat by @qiaouchicago in https://github.com/NCI-GDC/gdcdatamodel/pull/388
Feat/dev 1484 add isort ymlfmt toml sort and reformat by @qiaouchicago in https://github.com/NCI-GDC/gdcdatamodel/pull/390
dev 1483 add black and reformat by @qiaouchicago in https://github.com/NCI-GDC/gdcdatamodel/pull/389
Feat/dev 1627 drop py2 by @qiaouchicago in https://github.com/NCI-GDC/gdcdatamodel/pull/391
Chore: Update tags for Elion by @mpsolano in https://github.com/NCI-GDC/gdcdatamodel/pull/393
Update gdcdictionary to fix min max issue. by @stilesj-uchicago in https://github.com/NCI-GDC/gdcdatamodel/pull/395
Update gdcdictionary version. by @stilesj-uchicago in https://github.com/NCI-GDC/gdcdatamodel/pull/396

New Contributors

@stilesj-uchicago made their first contribution in https://github.com/NCI-GDC/gdcdatamodel/pull/395

Full Changelog: https://github.com/NCI-GDC/gdcdatamodel/compare/3.5.0-rc.0...3.6.5

Darwin 2022-06-21 18:22:07

What's Changed

Darwin release tag update by @mpsolano in https://github.com/NCI-GDC/gdcdatamodel/pull/382

New Contributors

@mpsolano made their first contribution in https://github.com/NCI-GDC/gdcdatamodel/pull/382

Full Changelog: https://github.com/NCI-GDC/gdcdatamodel/compare/3.4.3-rc.2...3.5.0-rc.0

3.4.3-rc.2 2022-05-31 21:22:18

What's Changed

remove parent tag caching by @kulgan in https://github.com/NCI-GDC/gdcdatamodel/pull/375
Release/dr32 to develop by @ProfOak in https://github.com/NCI-GDC/gdcdatamodel/pull/378
DEV-1035: Add support for skipping nodes during tagging/versioning by @kulgan in https://github.com/NCI-GDC/gdcdatamodel/pull/380

Full Changelog: https://github.com/NCI-GDC/gdcdatamodel/compare/3.4.2...3.4.3-rc.2

3.4.2 2022-03-08 19:09:44

3.4.2-rc.2 2021-11-19 19:16:38

What's Changed

Release/curie - merge to develop by @ProfOak in https://github.com/NCI-GDC/gdcdatamodel/pull/372
DEV-941: use six.ensure_str to fix tag mismatch issue by @kulgan in https://github.com/NCI-GDC/gdcdatamodel/pull/373
DICT-171: bump gdcdictionary by @kulgan in https://github.com/NCI-GDC/gdcdatamodel/pull/374

Full Changelog: https://github.com/NCI-GDC/gdcdatamodel/compare/3.4.1...3.4.2-rc.2

3.4.2-rc.1 2021-11-05 16:07:01

Genomic Data Commons

GitHub Repository Homepage

core library