This repo is a vulnerability database and package search for sources such as Aqua Security vuln-list, OSV, NVD, GitHub, and NPM. Vulnerability data are downloaded from the sources and stored in a custom file based storage with indexes to allow offline access and quick searches.
pip install appthreat-vulnerability-db
This package is ideal as a library for managing vulnerabilities. This is used by dep-scan, a free open-source dependency audit tool. However, there is a limited cli capability available with few features to test this tool directly.
Use the ORAS cli to download a pre-built database containing all application and OS vulnerabilities.
oras pull ghcr.io/appthreat/vdb:v5 -o $VDB_HOME
Cache application vulnerabilities
Typical size of this database is over 1.1 GB.
Cache application and OS vulnerabilities
Note the size of the database with OS vulnerabilities is over 3.1 GB.
Cache from just OSV
vdb --cache --only-osv
It is possible to customise the cache behaviour by increasing the historic data period to cache by setting the following environment variables.
To periodically sync the latest vulnerabilities and update the database cache.
It is possible to perform simple search using the cli.
```bash vdb --search android:8.0
vdb --search google:android:8.0
vdb --search android:8.0,simplesamlphp:1.14.11 ```
Syntax is package:version,package:version or vendor : package : version (Without space)
v5 gave the needed performance boost for searches. However, one of the things I am not happy with is the hardcoded batch size while storing a group of records.
I picked this number out of thin air. Bad hotel internet means I could never experiment with various batch sizes to measure the impact on storage vs. search performance.
After adding support for OS vulnerabilities, it is clear that this project's simple file-based search and indexing approach is no longer enough to support large containers, OS and cloud scans. Instead of going the traditional RDBMS route, I am working on a replacement project to use graph db.
Will update this thread with progress.
While working on a new risk scoring feature for dep-scan, I realized the need for a database for package metadata to prevent querying npm and pypi datasources for each invocation.
This perhaps will be a separate file with its own index to prevent the vulnerability database from becoming large. Separate flag will be added to perform package metadata fetching.
vulnerability-detection cve vulnerability-database nvd cli sca