BMW-Anonymization-Api

Data privacy and individuals’ anonymity are and always have been a major concern for data-driven companies.

Therefore, we designed and implemented an anonymization API that localizes and obfuscates (i.e. hides) sensitive information in images/videos in order to preserve the individuals' anonymity. The main features of our anonymization tool are the following: * Agnostic in terms of localization techniques: our API currently supports Semantic segmentation or Object Detection. * Modular in terms of sensitive information: the user can train a Deep Learning (DL) model for object detection and semantic segmentation to localize the sensitive information she/he wishes to protect, e.g., individual's face or body, personal belongings, vehicles... * Scalable in terms of anonymization techniques: our API currently supports pixelating, blurring, blackening (masking). Also, additinal anonymization techniques can be configured as stated below. For the highest level of privacy, we recommend using the blackening technique with degree 1. * Supports DL-based models optimized via the Intel® OpenVINO™ toolkit v2021.1 for CPU usage: DL-based models optimized and deployed via the Openvino Segmentation Inference API and the Openvino Detection Inference API can also be used. * Compatible with the BMW Deep Learning tools: DL models trained via our training and deployed via our inference APIs are compatible with this anonymization API.

General Architecture & Deployment Mode:

Our anonymization API receives an image along with a JSON object through which the user specifies mainly: * The sensitive information she/he wishes to obfuscate. * The anonymization technique. * The anonymization degree. * The localization technique.

You can deploy the anonymization API either: * As a standalone docker container which can be connected to other inference APIs (object detection or semantic segmentation) deployed within a standalone docker container as well. * As a network of docker containers along with other inference APIs running on the same machine via docker-compose. (please check the following link for the docker-compose deployment).

Prerequisites:

• docker
• docker-compose

Check for prerequisites

To check if docker-ce is installed:

sh docker --version

To check if docker-compose is installed:

sh docker-compose --version

Install prerequisites

Ubuntu

To install Docker and Docker Compose on Ubuntu, please follow the link.

Windows 10

To install Docker on Windows, please follow the link.

P.S: For Windows users, open the Docker Desktop menu by clicking the Docker Icon in the Notifications area. Select Settings, and then Advanced tab to adjust the resources available to Docker Engine.

Build The Docker Image

As mentioned before, this container can be deployed using either docker or docker-compose.

• If you wish to deploy this API using docker-compose, please refer to following link. After deploying the API with docker compose, please consider returning to this documentation for further information about the API Endpoints and use configuration file sample sections.

• If you wish to deploy this API using docker, please continue with the following docker build and run commands.

In order to build the project run the following command from the project's root directory:

sh docker build -t anonymization_api -f docker/dockerfile .

Build behind a proxy

In order to build the image behind a proxy use the following command in the project's root directory: sh docker build --build-arg http_proxy='your_proxy' --build-arg https_proxy='your_proxy' -t anonymization_api -f ./docker/dockerfile .

Run the docker container

To run the API, go to the API's directory and run the following:

Using Linux based docker:

sh sudo docker run -itv $(pwd)/src/main:/main -v $(pwd)/jsonFiles:/jsonFiles -p <port_of_your_choice>:4343 anonymization_api

Behind a proxy:

sh sudo docker run -itv $(pwd)/src/main:/main -v $(pwd)/jsonFiles:/jsonFiles --env HTTP_PROXY="" --env HTTPS_PROXY="" --env http_proxy="" --env https_proxy="" -p 5555:4343 anonymization_api

Using Windows based docker:

sh docker run -itv ${PWD}/src/main:/main -v ${PWD}/jsonFiles:/jsonFiles -p <port_of_your_choice>:4343 anonymization_api

The API file will be run automatically, and the service will listen to http requests on the chosen port.

API Endpoints

To see all available endpoints, open your favorite browser and navigate to:

http://<machine_IP>:<docker_host_port>/docs

Endpoints summary

Configuration

/set_url (POST)

Set the URL of the inference API that you wish to connect to the Anonymization API. If the specified URL is unreachable due to connection problems, it will not be added to the JSON url_configuration file. The URL should be specified in the following format "http://ip:port/".

/list_urls (GET)

Returns the URLs of the inference APIs that were already configured via the /set_url POST request.

/remove_url (POST)

Removes the specified URL from the JSON url_configuration file

/remove_all_urls (POST)

Removes all available urls from the JSON url_configuration file

/available_methods/ (GET)

After setting the inference URLs via the /set_url request, the user can view the Anonymization API's configuration by issuing the /available_methods request. Mainly the user can view (i) the supported sensitive information (label_names) , (ii) the supported localization techniques, (iii) the inference URLs and (iv) the DL model name that are configured in the deployed anonymization API as seen below.

Anonymization

/anonymize/ (POST)

Anonymizes the input image based on the user's JSON configuration file

/anonymize_video/ (POST)

Anonymizes a video based on the user's sensitive info and save the anonymized video in src/main/anonymized_videos under _TIMESTAMP.mp4

Video Anonymization Time

The video might take a while, actually you can estimate the time that it may take by using the following formula: Video_Anonymization_Time = Video_Length x Number_Of_Frames_Per_Second x Anonymization_Time_Of_Each_Frame

User configuration file sample

In order to anonymize an image, the user should specify the different details in the user's JSON configuration file

Please check a sample in the below image:

Note that the URL field is an optional field that you can add in case you wanted to use a specific URL of a running API. You can just add the URL as an optional field in this file as shown in the first sensitive info. In case this field is not specified, the URL defined in the url_configuration.json file will be used by default if it matches all the requirements.

To add a new technique to the API:

Please refer to the following link add new technique documentation for more information on how to add a new anonymization technique to the APIs with common and custom labels.

Benchmark

Object Detection

|GPU|Network |Width |Height |Inference Time (s) |Anonymization Time (s) |Total Time (s) | |:-:|:-:|:-:|:-:|:-:|:-:|:-:| |Titan RTX | yolov4 | 640 | 768| 0.2 |0.07 |0.27 | |Titan RTX | yolov4 | 1024 | 768| 0.4 |0.14 |0.54 | |Titan RTX | yolov4 | 2048 | 1024| 1.2 |0.6 |1.8 | |Titan RTX | yolov4 | 3840 | 2160| 4.8 |0.6 |5.4 |

Object Detection with OpenVINO model and Intel Core i7-1185G7

The model was trained with the TensorFlow Object Detection API (TF version 1.14) and then converted to OpenVINO IR using Intel® OpenVINO™ toolkit v2021.4
Results may vary. For workloads and configurations visit: www.intel.com/PerformanceIndex and Legal Information.

|CPU|Network |Precision |Width |Height |Inference Time (s) |Anonymization Time (s) |Total Time (s)
for Avg, Max, Min| |:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:| |Intel Core
i7-1185G7 | Faster R-CNN
Input Shape: [3,600,600] | FP32 | 1024| 768 |0.51 |0.09 |0.60, 0.67, 0.54 | |Intel Core
i7-1185G7 | Faster R-CNN
Input Shape: [3,600,600] | FP32 | 2048 | 1536 |0.56 |0.24 |0.80, 0.97, 0.70 | |Intel Core
i7-1185G7 | Faster R-CNN
Input Shape: [3,600,600] | INT8 | 1024| 768 |0.16 |0.09 |0.25, 0.27, 0.22 | |Intel Core
i7-1185G7 | Faster R-CNN
Input Shape: [3,600,600] | INT8| 2048 | 1536 |0.19 |0.24 |0.43, 0.56, 0.36 |

Semantic Segmentation

|GPU|Network |Width |Height |Inference Time (s) |Anonymization Time (s) |Total Time (s) | |:-:|:-:|:-:|:-:|:-:|:-:|:-:| |Titan RTX | psp resnet 101 | 640 | 768| 0.2 |0.8 |1.1 | |Titan RTX | psp resnet 101 | 1024 | 768| 0.3 |0.8 |1.1 | |Titan RTX | psp resnet 101 | 2048 | 1024| 0.9 |1.0 |1.9 | |Titan RTX | psp resnet 101 | 3840 | 2160| 2.0 |3.0 |5.0 |

Possible Error

• You may encounter the below error when running the docker container at startup in standalone version or docker-compose version

• In case you do, please make sure that the URL of the inference APIs listed in the jsonFiles/url_configuration.json are still recheable. A possible solution would be to empty jsonFiles/url_configuration.json as seen below before starting the container:

  { "urls": [ ] }

Acknowledgments

Ghenwa Aoun

Antoine Charbel, inmind.ai, Beirut, Lebanon

Roy Anwar

Fady Dib

Jimmy Tekli, BMW Innovation Lab, Munich, Germany

OpenVINO Toolkit

intel.com

robotron.de

Issues

bmw-annomization-api does not start

opened on 2022-09-20 16:43:35 by marmile

Hi,

I failed to get the docker-compose path to work. Links to underling services in docker-compose path does not point to repos anymore. I used the links from the main page https://github.com/BMW-InnovationLab/BMW-IntelOpenVINO-Detection-Inference-API, https://github.com/BMW-InnovationLab/BMW-IntelOpenVINO-Detection-Inference-API.

I see following errors during docker-compose up: bmw-anonymization-api-openvino_detection_api-1 | {"loglevel": "debug", "workers": 1, "bind": "0.0.0.0:80", "timeout": 3600, "workers_per_core": 1.0, "host": "0.0.0.0", "port": "80"} bmw-anonymization-api-openvino_detection_api-1 | 2022-09-20 16:16:27,530 [gunicorn.error] INFO Starting gunicorn 20.0.4 bmw-anonymization-api-openvino_detection_api-1 | 2022-09-20 16:16:27,530 [gunicorn.error] INFO Listening at: http://0.0.0.0:80 (1) bmw-anonymization-api-openvino_detection_api-1 | 2022-09-20 16:16:27,530 [gunicorn.error] INFO Using worker: uvicorn.workers.UvicornWorker bmw-anonymization-api-openvino_detection_api-1 | 2022-09-20 16:16:27,532 [gunicorn.error] INFO Booting worker with pid: 25 bmw-anonymization-api-openvino_segmentation_api-1 | {"loglevel": "debug", "workers": 1, "bind": "0.0.0.0:80", "timeout": 3600, "workers_per_core": 1.0, "host": "0.0.0.0", "port": "80"} bmw-anonymization-api-openvino_segmentation_api-1 | 2022-09-20 16:16:27,598 [gunicorn.error] INFO Starting gunicorn 20.0.4 bmw-anonymization-api-openvino_segmentation_api-1 | 2022-09-20 16:16:27,599 [gunicorn.error] INFO Listening at: http://0.0.0.0:80 (1) bmw-anonymization-api-openvino_segmentation_api-1 | 2022-09-20 16:16:27,599 [gunicorn.error] INFO Using worker: uvicorn.workers.UvicornWorker bmw-anonymization-api-openvino_segmentation_api-1 | 2022-09-20 16:16:27,600 [gunicorn.error] INFO Booting worker with pid: 25 bmw-anonymization-api-openvino_detection_api-1 | 2022-09-20 16:16:33,145 [uvicorn.access] INFO 172.18.0.4:43614 - "GET /models HTTP/1.1" 200 bmw-anonymization-api-openvino_detection_api-1 | Incorrect weights in bin file! bmw-anonymization-api-openvino_detection_api-1 | Error loading model bmw-anonymization-api-openvino_detection_api-1 | 2022-09-20 16:16:33,325 [uvicorn.access] INFO 172.18.0.4:43616 - "GET /models/detect_cups/labels HTTP/1.1" 500 bmw-anonymization-api-openvino_detection_api-1 | Incorrect weights in bin file! bmw-anonymization-api-openvino_detection_api-1 | Error loading model bmw-anonymization-api-openvino_detection_api-1 | 2022-09-20 16:16:33,332 [uvicorn.access] INFO 172.18.0.4:43618 - "GET /models/detect_cups/config HTTP/1.1" 500 bmw-anonymization-api-anonymization_api-1 | Traceback (most recent call last): bmw-anonymization-api-anonymization_api-1 | File "/usr/local/bin/uvicorn", line 8, in <module> bmw-anonymization-api-anonymization_api-1 | sys.exit(main()) bmw-anonymization-api-anonymization_api-1 | File "/usr/local/lib/python3.7/site-packages/click/core.py", line 1130, in __call__ bmw-anonymization-api-anonymization_api-1 | return self.main(*args, **kwargs) bmw-anonymization-api-anonymization_api-1 | File "/usr/local/lib/python3.7/site-packages/click/core.py", line 1055, in main bmw-anonymization-api-anonymization_api-1 | rv = self.invoke(ctx) bmw-anonymization-api-anonymization_api-1 | File "/usr/local/lib/python3.7/site-packages/click/core.py", line 1404, in invoke bmw-anonymization-api-anonymization_api-1 | return ctx.invoke(self.callback, **ctx.params) bmw-anonymization-api-anonymization_api-1 | File "/usr/local/lib/python3.7/site-packages/click/core.py", line 760, in invoke bmw-anonymization-api-anonymization_api-1 | return __callback(*args, **kwargs) bmw-anonymization-api-anonymization_api-1 | File "/usr/local/lib/python3.7/site-packages/uvicorn/main.py", line 426, in main bmw-anonymization-api-anonymization_api-1 | run(app, **kwargs) bmw-anonymization-api-anonymization_api-1 | File "/usr/local/lib/python3.7/site-packages/uvicorn/main.py", line 452, in run bmw-anonymization-api-anonymization_api-1 | server.run() bmw-anonymization-api-anonymization_api-1 | File "/usr/local/lib/python3.7/site-packages/uvicorn/server.py", line 68, in run bmw-anonymization-api-anonymization_api-1 | return asyncio.run(self.serve(sockets=sockets)) bmw-anonymization-api-anonymization_api-1 | File "/usr/local/lib/python3.7/asyncio/runners.py", line 43, in run bmw-anonymization-api-anonymization_api-1 | return loop.run_until_complete(main) bmw-anonymization-api-anonymization_api-1 | File "/usr/local/lib/python3.7/asyncio/base_events.py", line 587, in run_until_complete bmw-anonymization-api-anonymization_api-1 | return future.result() bmw-anonymization-api-anonymization_api-1 | File "/usr/local/lib/python3.7/site-packages/uvicorn/server.py", line 76, in serve bmw-anonymization-api-anonymization_api-1 | config.load() bmw-anonymization-api-anonymization_api-1 | File "/usr/local/lib/python3.7/site-packages/uvicorn/config.py", line 456, in load bmw-anonymization-api-anonymization_api-1 | self.loaded_app = import_from_string(self.app) bmw-anonymization-api-anonymization_api-1 | File "/usr/local/lib/python3.7/site-packages/uvicorn/importer.py", line 21, in import_from_string bmw-anonymization-api-anonymization_api-1 | module = importlib.import_module(module_str) bmw-anonymization-api-anonymization_api-1 | File "/usr/local/lib/python3.7/importlib/__init__.py", line 127, in import_module bmw-anonymization-api-anonymization_api-1 | return _bootstrap._gcd_import(name[level:], package, level) bmw-anonymization-api-anonymization_api-1 | File "<frozen importlib._bootstrap>", line 1006, in _gcd_import bmw-anonymization-api-anonymization_api-1 | File "<frozen importlib._bootstrap>", line 983, in _find_and_load bmw-anonymization-api-anonymization_api-1 | File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked bmw-anonymization-api-anonymization_api-1 | File "<frozen importlib._bootstrap>", line 677, in _load_unlocked bmw-anonymization-api-anonymization_api-1 | File "<frozen importlib._bootstrap_external>", line 728, in exec_module bmw-anonymization-api-anonymization_api-1 | File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed bmw-anonymization-api-anonymization_api-1 | File "./start.py", line 23, in <module> bmw-anonymization-api-anonymization_api-1 | config.master_dict = labels_methods() bmw-anonymization-api-anonymization_api-1 | File "./labels.py", line 15, in labels_methods bmw-anonymization-api-anonymization_api-1 | api_client = ApiClient() bmw-anonymization-api-anonymization_api-1 | File "./APIClient.py", line 15, in __init__ bmw-anonymization-api-anonymization_api-1 | self.get_api_configuration() bmw-anonymization-api-anonymization_api-1 | File "./APIClient.py", line 39, in get_api_configuration bmw-anonymization-api-anonymization_api-1 | self.get_models(url) bmw-anonymization-api-anonymization_api-1 | File "./APIClient.py", line 59, in get_models bmw-anonymization-api-anonymization_api-1 | model_type = self.get_model_configuration(url, model_name) bmw-anonymization-api-anonymization_api-1 | File "./APIClient.py", line 90, in get_model_configuration bmw-anonymization-api-anonymization_api-1 | return response.json()["data"]["type"] bmw-anonymization-api-anonymization_api-1 | TypeError: 'NoneType' object is not subscriptable bmw-anonymization-api-anonymization_api-1 exited with code 1

I'm not able to resolve labels in listed model: mw-anonymization-api-openvino_detection_api-1 | 2022-09-20 16:27:15,424 [uvicorn.access] INFO 172.18.0.1:51654 - "GET /models/detect_cups/labels HTTP/1.1" 500

HW reference:

01: None 00.0: 10103 CPU [Created at cpu.465] Unique ID: rdCR.j8NaKXDZtZ6 Hardware Class: cpu Arch: X86-64 Vendor: "GenuineIntel" Model: 6.158.10 "Intel(R) Core(TM) i7-8700K CPU @ 3.70GHz" Features: fpu,vme,de,pse,tsc,msr,pae,mce,cx8,apic,sep,mtrr,pge,mca,cmov,pat,pse36,clflush,dts,acpi,mmx,fxsr,sse,sse2,ss,ht,tm,pbe,syscall,nx,pdpe1gb,rdtscp,lm,constant_tsc,art,arch_perfmon,pebs,bts,rep_good,nopl,xtopology,nonstop_tsc,cpuid,aperfmperf,pni,pclmulqdq,dtes64,monitor,ds_cpl,vmx,smx,est,tm2,ssse3,sdbg,fma,cx16,xtpr,pdcm,pcid,sse4_1,sse4_2,x2apic,movbe,popcnt,tsc_deadline_timer,aes,xsave,avx,f16c,rdrand,lahf_lm,abm,3dnowprefetch,cpuid_fault,invpcid_single,pti,ssbd,ibrs,ibpb,stibp,tpr_shadow,vnmi,flexpriority,ept,vpid,ept_ad,fsgsbase,tsc_adjust,bmi1,avx2,smep,bmi2,erms,invpcid,mpx,rdseed,adx,smap,clflushopt,intel_pt,xsaveopt,xsavec,xgetbv1,xsaves,dtherm,ida,arat,pln,pts,hwp,hwp_notify,hwp_act_window,hwp_epp,md_clear,flush_l1d,arch_capabilities Clock: 3700 MHz BogoMips: 7399.70 Cache: 12288 kb Units/Processor: 16 Config Status: cfg=new, avail=yes, need=no, active=unknown