QAConv

Interpretable and Generalizable Person Re-Identification with Query-Adaptive Convolution and Temporal Lifting

This is the official PyTorch code for the QAConv method proposed in our paper [1] and the QAConv-GS with Graph Sampling proposed in our paper [2]. A Chinese blog is available in 再见，迁移学习？可解释和泛化的行人再辨识.

Updates

• 1/3/2023: Improved testing efficiency by using half precision, and optimized memory usage in testing. Included ClonedPerson, and removed DukeMTMC-reID.
• 3/3/2022: The Graph Sampling work (QAConv-GS/QAConv 2.1) has been accepted by CVPR 2022.
• 9/29/2022: TransMatcher has been accepted by NeurIPS 2021.
• 9/19/2021: Include TransMatcher, a transformer based deep image matching method based on QAConv 2.0.
• 9/16/2021: QAConv 2.1: simplify graph sampling, implement the Einstein summation for QAConv, use the batch hard triplet loss, design an adaptive epoch and learning rate scheduling method, and apply the automatic mixed precision training.
• 4/1/2021: QAConv 2.0 [2]: include a new sampler called Graph Sampler (GS), and remove the class memory. This version is much more efficient in learning. See the updated results.
• 3/31/2021: QAConv 1.2: include some popular data augmentation methods, and change the ranking.py implementation to the original open-reid version, so that it is more consistent to most other implementations (e.g. open-reid, torch-reid, fast-reid).
• 2/7/2021: QAConv 1.1: an important update, which includes a pre-training function for a better initialization, so that the results are now more stable.
• 11/26/2020: Include the IBN-Net as backbone, and the RandPerson dataset.

Illustrations

Fig. 1. Illustration of the proposed query-adaptive convolution (QAConv).

Fig. 2. Examples of local correspondences obtained by QAConv.

Fig. 3. QAConv network architecture in training.

Fig. 4. Illustration of the proposed temporal lifting (TLift).

Requirements

• Pytorch (>1.0)
• sklearn
• scipy

Usage

Download some public datasets (e.g. Market-1501, CUHK03-NP, MSMT, RandPerson, ClonedPerson) on your own, extract them in some folder, and then run the followings.

Training and test

python main.py --dataset market --testset cuhk03_np_detected[,msmt] [--data-dir ./data] [--exp-dir ./Exp]

For more options, run "python main.py --help". For example, if you want to use the ResNet-152 as backbone, specify "-a resnet152". If you want to train on the whole dataset (as done in our paper for the MSMT17), specify "--combine_all".

The main file is updated with the QAConv 2.1 version, that is the CVPR 2022 version with the Graph Sampler and sole triplet loss. For other earlier versions, please check Releases.

Test only

python main.py --dataset market --testset cuhk03_np_detected[,market,msmt] [--data-dir ./data] [--exp-dir ./Exp] --evaluate

Performance

Performance (%) of QAConv (QAConv 1.0) and QAConv-GS (QAConv 2.1) under direct cross-dataset evaluation without transfer learning or domain adaptation:

Training Data	Version	Training Hours	CUHK03-NP		Market-1501		MSMT17
			Rank-1	mAP	Rank-1	mAP	Rank-1	mAP
Market	QAConv 1.0	1.33	9.9	8.6	-	-	22.6	7.0
	QAConv 2.1	0.25	19.1	18.1	-	-	45.9	17.2
MSMT	QAConv 2.1	0.73	20.9	20.6	79.1	49.5	-	-
MSMT (all)	QAConv 1.0	26.90	25.3	22.6	72.6	43.1	-	-
	QAConv 2.1	3.42	27.6	28.0	82.4	56.9	-	-
RandPerson	QAConv 2.1	2.0	18.4	16.1	76.7	46.7	45.1	15.5

Contacts

Shengcai Liao
Inception Institute of Artificial Intelligence (IIAI)
[email protected]

Citation

[1] Shengcai Liao and Ling Shao, "Interpretable and Generalizable Person Re-Identification with Query-Adaptive Convolution and Temporal Lifting." In the 16th European Conference on Computer Vision (ECCV), 23-28 August, 2020.

[2] Shengcai Liao and Ling Shao, "Graph Sampling Based Deep Metric Learning for Generalizable Person Re-Identification." In CVF/IEEE Conference on Computer Vision and Pattern Recognition, 2022.

``` @inproceedings{Liao-ECCV2020-QAConv,
title={{Interpretable and Generalizable Person Re-Identification with Query-Adaptive Convolution and Temporal Lifting}},
author={Shengcai Liao and Ling Shao},
booktitle={European Conference on Computer Vision (ECCV)},
year={2020}
}

@article{Liao-CVPR2022-GraphSampling, author = {Shengcai Liao and Ling Shao}, title = {{Graph Sampling Based Deep Metric Learning for Generalizable Person Re-Identification}}, booktitle = {CVF/IEEE Conference on Computer Vision and Pattern Recognition (CVPR)}, month = {June}, year = {2022} } ```

Releases

QAConv 2.1 2021-09-16 16:31:05

• Simplified graph sampling
• Einstein summation for QAConv
• Hard triplet loss
• Adaptive epoch and learning rate scheduling
• Automatic mixed precision training

QAConv 2.0 2021-04-01 13:19:24

• Include a new sampler called Graph Sampler (GS).
• Remove the class memory based loss. Instead, a pairwise matching loss is implemented.
• This version is much more efficient in learning.

QAConv 1.2 2021-03-31 16:43:07

Include some popular data augmentation methods, and change the ranking.py implementation to the original open-reid version, so that it is more consistent to most other implementations (e.g. open-reid, torch-reid, fast-reid).

QAConv 1.1 2021-02-08 08:13:07

• Include the IBN-Net as backbone, and the RandPerson dataset.
• Include a pre-training function for a better initialization, so that the results are now more stable.

ECCV 2020 Version 2020-08-06 06:23:02

This is the ECCV 2020 version.