Emergence of exploratory look-around behaviors through active observation completion

A journal version of this work in conjunction with our prior work on Learning to Look Around: Intelligently Exploring Unseen Environments for Unknown Tasks has been published in Science Robotics 2019.

Emergence of exploratory look-around behaviors through active observation completion
Santhosh K. Ramakrishnan, Dinesh Jayaraman, Kristen Grauman
Science Robotics 2019

A cleaned version of this codebase along with new transfer tasks are available at https://github.com/srama2512/visual-exploration.

Sidekick Policy Learning

This repository contains code and data for the paper

Sidekick Policy Learning for Active Visual Exploration
Santhosh K. Ramakrishnan, Kristen Grauman
ECCV 2018

Setup

• First install anaconda and setup a new environment. Install anaconda from: https://www.anaconda.com/download/

conda create -n spl python=2.7 source activate spl - Clone this project repository and setup requirements using pip.

git clone https://github.com/srama2512/sidekicks.git cd sidekicks pip install -r requirements.txt

• Download preprocessed SUN360 and ModelNet data.

wget http://vision.cs.utexas.edu/projects/sidekicks/data.zip unzip data.zip

• Sidekick scores for ours-rew, ours-demo, rnd-rewards on both datasets have been provided here. The one-view model used to generate them have also been provided.

Evaluating pre-trained models

All the pre-trained models have been provided here. To evaluate them, download them to the models directory. To reproduce results from the paper:

wget http://vision.cs.utexas.edu/projects/sidekicks/models.zip unzip models.zip sh evaluation_script_final.sh

Evaluation examples

• Evaluating SUN360 one-view baseline on the test data with avg metric:

python eval.py --h5_path data/sun360/sun360_processed.h5 --dataset 0 \ --model_path models/sun360/one-view.net --T 1 --M 8 --N 4 \ --start_view 2 --save_path dummy/

• Evaluating SUN360 ltla baseline on the test data with avg metric:

python eval.py --h5_path data/sun360/sun360_processed.h5 --dataset 0 \ --model_path models/sun360/ltla.net --T 4 --M 8 --N 4 \ --start_view 2 --save_path dummy/ - Evaluating SUN360 ltla baseline on the test data with adv metric:

python eval.py --h5_path data/sun360/sun360_processed.h5 --dataset 0 \ --model_path models/sun360/ltla.net --T 4 --M 8 --N 4 \ --start_view 2 --save_path dummy/ - Evaluating SUN360 rnd-actions baseline on test data with avg metric:

python eval.py --h5_path data/sun360/sun360_processed.h5 --dataset 0 \ --model_path models/sun360/rnd-actions.net --T 4 --M 8 --N 4 \ --start_view 2 --actorType random --save_path dummy/ - Evaluating ModelNet Hard one-view baseline on test (seen and unseen) data with avg metric:

python eval.py --h5_path modelnet30_processed.h5 \ --h5_path_unseen modelnet10_processed.h5 --dataset 1 \ --model_path models/modelnet_hard/one-view.net --T 1 --M 9 --N 5 \ --start_view 2 --save_path dummy/

Training models

Ensure that the pre-trained models and pre-computed scores are downloaded and extracted.

• Training one-view model on SUN360 with default settings:

python main.py --T 1 --training_setting 0 --epochs 100 \ --save_path saved_models/sun360/one-view - Training ltla baseline on SUN360 with default settings (starting from pre-trained one-view model):

python main.py --T 4 --training_setting 1 --epochs 1000 \ --save_path saved_models/sun360/ltla/ \ --load_model models/sun360/one-view.net - Training ours-rew on SUN360 with default settings (with pre-computed score):

python main.py --T 4 --training_setting 1 --epochs 1000 \ --save_path saved_models/sun360/ours-rew/ \ --load_model models/sun360/one-view.net --expert_rewards True \ --rewards_h5_path scores/sun360/ours-rew-scores.h5 - Training ours-demo on SUN360 with default settings (with pre-computed score):

python main.py --T 4 --training_setting 1 --epochs 1000 \ --save_path saved_models/sun360/ours-demo/ \ --load_model models/sun360/one-view.net --expert_trajectories True \ --utility_h5_path scores/sun360/ours-demo-scores.h5 - Training ltla baseline on ModelNet Hard with default settings (starting from pre-trained one-view model):

python main.py --h5_path data/modelnet_hard/modelnet30_processed.h5 \ --training_setting 1 --dataset 1 --T 4 --M 9 --N 5 \ --load_model models/modelnet_hard/one-view.net \ --save_path saved_models/modelnet_hard/ltla/

The other ModelNet Hard models can be trained similar to SUN360 models. To train actor critic models, set --baselineType critic. To add full observability to the critic (for asymm-ac), set --critic_full_obs True.

Visualization

From the repository directory, start jupyter notebook and open visualize_policy_paper.ipynb. Perform the TODOs mentioned in the comments (setting the correct paths) and run the entire script. It will generate tensorboard files contained visualized heatmaps on several examples.

Issues

Bump ipython from 5.5.0 to 7.16.3

opened on 2022-01-21 19:42:37 by dependabot[bot]

Bumps ipython from 5.5.0 to 7.16.3.

Commits

• d43c7c7 release 7.16.3
• 5fa1e40 Merge pull request from GHSA-pq7m-3gw7-gq5x
• 8df8971 back to dev
• 9f477b7 release 7.16.2
• 138f266 bring back release helper from master branch
• 5aa3634 Merge pull request #13341 from meeseeksmachine/auto-backport-of-pr-13335-on-7...
• bcae8e0 Backport PR #13335: What's new 7.16.2
• 8fcdcd3 Pin Jedi to <0.17.2.
• 2486838 release 7.16.1
• 20bdc6f fix conda build
• Additional commits viewable in compare view

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

---

Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) - `@dependabot use these labels` will set the current labels as the default for future PRs for this repo and language - `@dependabot use these reviewers` will set the current reviewers as the default for future PRs for this repo and language - `@dependabot use these assignees` will set the current assignees as the default for future PRs for this repo and language - `@dependabot use this milestone` will set the current milestone as the default for future PRs for this repo and language You can disable automated security fix PRs for this repo from the [Security Alerts page](https://github.com/srama2512/sidekicks/network/alerts).