Categorize tools (#4501)

* Categorize tools

* Update docs

* Rename tools

* Tool organization

configs/cityscapes/README.md

@@ -17,7 +17,7 @@
 - All models were trained on `cityscapes_train`, and tested on `cityscapes_val`.
 - 1x training schedule indicates 64 epochs which corresponds to slightly less than the 24k iterations reported in the original schedule from the [Mask R-CNN paper](https://arxiv.org/abs/1703.06870)
 - COCO pre-trained weights are used to initialize.
-- A conversion [script](../../tools/convert_datasets/cityscapes.py) is provided to convert Cityscapes into COCO format. Please refer to [install.md](../../docs/1_exist_data_model.md#prepare-datasets) for details.
+- A conversion [script](../../tools/dataset_converters/cityscapes.py) is provided to convert Cityscapes into COCO format. Please refer to [install.md](../../docs/1_exist_data_model.md#prepare-datasets) for details.
 - `CityscapesDataset` implemented three evaluation methods. `bbox` and `segm` are standard COCO bbox/mask AP. `cityscapes` is the cityscapes dataset official evaluation, which may be slightly higher than COCO.
 
 ### Faster R-CNN

configs/legacy_1.x/README.md

@@ -20,10 +20,10 @@ There are three main difference in the model weights between V1.x and V2.0 codeb
 3. For two-stage detectors, their wegihts need to be upgraded since MMDetection V2.0 refactors all the two-stage detectors with `RoIHead`.
 
 The users can do the same modification as mentioned above for the self-implemented
-detectors. We provide a scripts `tools/upgrade_model_version.py` to convert the model weights in the V1.x model zoo.
+detectors. We provide a scripts `tools/model_converters/upgrade_model_version.py` to convert the model weights in the V1.x model zoo.
 
 ```bash
-python tools/upgrade_model_version.py ${OLD_MODEL_PATH} ${NEW_MODEL_PATH} --num-classes ${NUM_CLASSES}
+python tools/model_converters/upgrade_model_version.py ${OLD_MODEL_PATH} ${NEW_MODEL_PATH} --num-classes ${NUM_CLASSES}
 
 ```
 

configs/regnet/README.md

@@ -34,7 +34,7 @@ For more general usage, we also provide script `regnet2mmdet.py` in the tools di
 ResNet-style checkpoints used in MMDetection.
 
 ```bash
-python -u tools/regnet2mmdet.py ${PRETRAIN_PATH} ${STORE_PATH}
+python -u tools/model_converters/regnet2mmdet.py ${PRETRAIN_PATH} ${STORE_PATH}
 ```
 
 This script convert model from `PRETRAIN_PATH` and store the converted model in `STORE_PATH`.

docs/1_exist_data_model.md

@@ -174,12 +174,12 @@ mmdetection
 
 ```
 
-The [cityscapes](https://www.cityscapes-dataset.com/) annotations need to be converted into the coco format using `tools/convert_datasets/cityscapes.py`:
+The [cityscapes](https://www.cityscapes-dataset.com/) annotations need to be converted into the coco format using `tools/dataset_converters/cityscapes.py`:
 
 ```shell
 pip install cityscapesscripts
 
-python tools/convert_datasets/cityscapes.py \
+python tools/dataset_converters/cityscapes.py \
     ./data/cityscapes \
     --nproc 8 \
     --out-dir ./data/cityscapes/annotations

docs/3_exist_data_new_model.md

@@ -41,11 +41,11 @@ mmdetection
 
 ```
 
-The cityscapes annotations have to be converted into the coco format using `tools/convert_datasets/cityscapes.py`:
+The cityscapes annotations have to be converted into the coco format using `tools/dataset_converters/cityscapes.py`:
 
 ```shell
 pip install cityscapesscripts
-python tools/convert_datasets/cityscapes.py ./data/cityscapes --nproc 8 --out-dir ./data/cityscapes/annotations
+python tools/dataset_converters/cityscapes.py ./data/cityscapes --nproc 8 --out-dir ./data/cityscapes/annotations
 ```
 
 Currently the config files in `cityscapes` use COCO pre-trained weights to initialize.

docs/compatibility.md

@@ -77,6 +77,6 @@ model-level compatibility but slightly improves the performance. The major ones
 
 ## Upgrade Models from 1.x to 2.0
 
-To convert the models trained by MMDetection V1.x to MMDetection V2.0, the users can use the script `tools/upgrade_model_version.py` to convert
+To convert the models trained by MMDetection V1.x to MMDetection V2.0, the users can use the script `tools/model_converters/upgrade_model_version.py` to convert
 their models. The converted models can be run in MMDetection V2.0 with slightly dropped performance (less than 1% AP absolute).
 Details can be found in `configs/legacy`.

docs/model_zoo.md

@@ -11,7 +11,7 @@ You can replace `https://s3.ap-northeast-2.amazonaws.com/open-mmlab` with `https
 - We use distributed training.
 - All pytorch-style pretrained backbones on ImageNet are from PyTorch model zoo, caffe-style pretrained backbones are converted from the newly released model from detectron2.
 - For fair comparison with other codebases, we report the GPU memory as the maximum value of `torch.cuda.max_memory_allocated()` for all 8 GPUs. Note that this value is usually less than what `nvidia-smi` shows.
-- We report the inference time as the total time of network forwarding and post-processing, excluding the data loading time. Results are obtained with the script [benchmark.py](https://github.com/open-mmlab/mmdetection/blob/master/tools/benchmark.py) which computes the average time on 2000 images.
+- We report the inference time as the total time of network forwarding and post-processing, excluding the data loading time. Results are obtained with the script [benchmark.py](https://github.com/open-mmlab/mmdetection/blob/master/tools/analysis_tools/benchmark.py) which computes the average time on 2000 images.
 
 ## Baselines
 

docs/tutorials/config.md

 # Tutorial 1: Learn about Configs
 
 We incorporate modular and inheritance design into our config system, which is convenient to conduct various experiments.
-If you wish to inspect the config file, you may run `python tools/print_config.py /PATH/TO/CONFIG` to see the complete config.
+If you wish to inspect the config file, you may run `python tools/misc/print_config.py /PATH/TO/CONFIG` to see the complete config.
 
 ## Modify config through script arguments
 

docs/tutorials/customize_dataset.md

@@ -152,7 +152,7 @@ Here is a valid example of annotations:
  ]
 ```
 
-We use this way to support CityScapes dataset. The script is in [cityscapes.py](https://github.com/open-mmlab/mmdetection/blob/master/tools/convert_datasets/cityscapes.py) and we also provide the finetuning [configs](https://github.com/open-mmlab/mmdetection/blob/master/configs/cityscapes).
+We use this way to support CityScapes dataset. The script is in [cityscapes.py](https://github.com/open-mmlab/mmdetection/blob/master/tools/dataset_converters/cityscapes.py) and we also provide the finetuning [configs](https://github.com/open-mmlab/mmdetection/blob/master/configs/cityscapes).
 
 **Note**
 
@@ -203,7 +203,7 @@ There are two ways to work with custom datasets.
 - offline conversion
 
   You can convert the annotation format to the expected format above and save it to
-  a pickle or json file, like [pascal_voc.py](https://github.com/open-mmlab/mmdetection/blob/master/tools/convert_datasets/pascal_voc.py).
+  a pickle or json file, like [pascal_voc.py](https://github.com/open-mmlab/mmdetection/blob/master/tools/dataset_converters/pascal_voc.py).
   Then you can simply use `CustomDataset`.
 
 ### An example of customized dataset

docs/tutorials/pytorch2onnx.md

@@ -26,7 +26,7 @@
 ### Usage
 
 ```bash
-python tools/pytorch2onnx.py \
+python tools/deployment/pytorch2onnx.py \
     ${CONFIG_FILE} \
     ${CHECKPOINT_FILE} \
     --output-file ${OUTPUT_FILE} \
@@ -60,7 +60,7 @@ Description of all arguments:
 Example:
 
 ```bash
-python tools/pytorch2onnx.py \
+python tools/deployment/pytorch2onnx.py \
     configs/yolo/yolov3_d53_mstrain-608_273e_coco.py \
     checkpoints/yolo/yolov3_d53_mstrain-608_273e_coco.pth \
     --output-file checkpoints/yolo/yolov3_d53_mstrain-608_273e_coco.onnx \

docs/useful_tools.md

@@ -3,11 +3,11 @@ Apart from training/testing scripts, We provide lots of useful tools under the
 
 ## Log Analysis
 
-`tools/analyze_logs.py` plots loss/mAP curves given a training
+`tools/analysis_tools/analyze_logs.py` plots loss/mAP curves given a training
  log file. Run `pip install seaborn` first to install the dependency.
 
  ```shell
-python tools/analyze_logs.py plot_curve [--keys ${KEYS}] [--title ${TITLE}] [--legend ${LEGEND}] [--backend ${BACKEND}] [--style ${STYLE}] [--out ${OUT_FILE}]
+python tools/analysis_tools/analyze_logs.py plot_curve [--keys ${KEYS}] [--title ${TITLE}] [--legend ${LEGEND}] [--backend ${BACKEND}] [--style ${STYLE}] [--out ${OUT_FILE}]
 ```
 
 ![loss curve image](../resources/loss_curve.png)
@@ -17,25 +17,25 @@ Examples:
 - Plot the classification loss of some run.
 
     ```shell
-    python tools/analyze_logs.py plot_curve log.json --keys loss_cls --legend loss_cls
+    python tools/analysis_tools/analyze_logs.py plot_curve log.json --keys loss_cls --legend loss_cls
     ```
 
 - Plot the classification and regression loss of some run, and save the figure to a pdf.
 
     ```shell
-    python tools/analyze_logs.py plot_curve log.json --keys loss_cls loss_bbox --out losses.pdf
+    python tools/analysis_tools/analyze_logs.py plot_curve log.json --keys loss_cls loss_bbox --out losses.pdf
     ```
 
 - Compare the bbox mAP of two runs in the same figure.
 
     ```shell
-    python tools/analyze_logs.py plot_curve log1.json log2.json --keys bbox_mAP --legend run1 run2
+    python tools/analysis_tools/analyze_logs.py plot_curve log1.json log2.json --keys bbox_mAP --legend run1 run2
     ```
 
 - Compute the average training speed.
 
     ```shell
-    python tools/analyze_logs.py cal_train_time log.json [--include-outliers]
+    python tools/analysis_tools/analyze_logs.py cal_train_time log.json [--include-outliers]
     ```
 
     The output is expected to be like the following.
@@ -52,12 +52,12 @@ Examples:
 
 ### Visualize Datasets
 
-`tools/browse_dataset.py` helps the user to browse a detection dataset (both
+`tools/misc/browse_dataset.py` helps the user to browse a detection dataset (both
  images and bounding box annotations) visually, or save the image to a
   designated directory.
 
 ```shell
-python tools/browse_dataset.py ${CONFIG} [-h] [--skip-type ${SKIP_TYPE[SKIP_TYPE...]}] [--output-dir ${OUTPUT_DIR}] [--not-show] [--show-interval ${SHOW_INTERVAL}]
+python tools/misc/browse_dataset.py ${CONFIG} [-h] [--skip-type ${SKIP_TYPE[SKIP_TYPE...]}] [--output-dir ${OUTPUT_DIR}] [--not-show] [--show-interval ${SHOW_INTERVAL}]
 ```
 
 ### Visualize Models
@@ -74,20 +74,20 @@ If you need a lightweight GUI for visualizing the detection results, you can ref
 
 ## Error Analysis
 
-`tools/coco_error_analysis.py` analyzes COCO results per category and by
+`tools/analysis_tools/coco_error_analysis.py` analyzes COCO results per category and by
  different criterion. It can also make a plot to provide useful
   information.
 
 ```shell
-python tools/coco_error_analysis.py ${RESULT} ${OUT_DIR} [-h] [--ann ${ANN}] [--types ${TYPES[TYPES...]}]
+python tools/analysis_tools/coco_error_analysis.py ${RESULT} ${OUT_DIR} [-h] [--ann ${ANN}] [--types ${TYPES[TYPES...]}]
 ```
 
 ## Model Complexity
 
-`tools/get_flops.py` is a script adapted from [flops-counter.pytorch](https://github.com/sovrasov/flops-counter.pytorch) to compute the FLOPs and params of a given model.
+`tools/analysis_tools/get_flops.py` is a script adapted from [flops-counter.pytorch](https://github.com/sovrasov/flops-counter.pytorch) to compute the FLOPs and params of a given model.
 
 ```shell
-python tools/get_flops.py ${CONFIG_FILE} [--shape ${INPUT_SHAPE}]
+python tools/analysis_tools/get_flops.py ${CONFIG_FILE} [--shape ${INPUT_SHAPE}]
 ```
 
 You will get the results like this.
@@ -116,43 +116,43 @@ Params: 37.74 M
 We provide a script to convert model to [ONNX](https://github.com/onnx/onnx) format. We also support comparing the output results between Pytorch and ONNX model for verification.
 
 ```shell
-python tools/pytorch2onnx.py ${CONFIG_FILE} ${CHECKPOINT_FILE} --output_file ${ONNX_FILE} [--shape ${INPUT_SHAPE} --verify]
+python tools/deployment/pytorch2onnx.py ${CONFIG_FILE} ${CHECKPOINT_FILE} --output_file ${ONNX_FILE} [--shape ${INPUT_SHAPE} --verify]
 ```
 
 **Note**: This tool is still experimental. Some customized operators are not supported for now. For a detailed description of the usage and the list of supported models, please refer to [pytorch2onnx](tutorials/pytorch2onnx.md).
 
 ### MMDetection 1.x model to MMDetection 2.x
 
-`tools/upgrade_model_version.py` upgrades a previous MMDetection checkpoint
+`tools/model_converters/upgrade_model_version.py` upgrades a previous MMDetection checkpoint
  to the new version. Note that this script is not guaranteed to work as some
   breaking changes are introduced in the new version. It is recommended to
    directly use the new checkpoints.
 
 ```shell
-python tools/upgrade_model_version.py ${IN_FILE} ${OUT_FILE} [-h] [--num-classes NUM_CLASSES]
+python tools/model_converters/upgrade_model_version.py ${IN_FILE} ${OUT_FILE} [-h] [--num-classes NUM_CLASSES]
 ```
 
 ### RegNet model to MMDetection
 
-`tools/regnet2mmdet.py` convert keys in pycls pretrained RegNet models to
+`tools/model_converters/regnet2mmdet.py` convert keys in pycls pretrained RegNet models to
  MMDetection style.
 
 ```shell
-python tools/regnet2mmdet.py ${SRC} ${DST} [-h]
+python tools/model_converters/regnet2mmdet.py ${SRC} ${DST} [-h]
 ```
 
 ### Detectron ResNet to Pytorch
 
-`tools/detectron2pytorch.py` converts keys in the original detectron pretrained
+`tools/model_converters/detectron2pytorch.py` converts keys in the original detectron pretrained
  ResNet models to PyTorch style.
 
 ```shell
-python tools/detectron2pytorch.py ${SRC} ${DST} ${DEPTH} [-h]
+python tools/model_converters/detectron2pytorch.py ${SRC} ${DST} ${DEPTH} [-h]
 ```
 
 ### Prepare a model for publishing
 
-`tools/publish_model.py` helps users to prepare their model for publishing.
+`tools/model_converters/publish_model.py` helps users to prepare their model for publishing.
 
 Before you upload a model to AWS, you may want to
 
@@ -162,47 +162,47 @@ Before you upload a model to AWS, you may want to
  filename.
 
 ```shell
-python tools/publish_model.py ${INPUT_FILENAME} ${OUTPUT_FILENAME}
+python tools/model_converters/publish_model.py ${INPUT_FILENAME} ${OUTPUT_FILENAME}
 ```
 
 E.g.,
 
 ```shell
-python tools/publish_model.py work_dirs/faster_rcnn/latest.pth faster_rcnn_r50_fpn_1x_20190801.pth
+python tools/model_converters/publish_model.py work_dirs/faster_rcnn/latest.pth faster_rcnn_r50_fpn_1x_20190801.pth
 ```
 
 The final output filename will be `faster_rcnn_r50_fpn_1x_20190801-{hash id}.pth`.
 
 ## Dataset Conversion
 
-`tools/convert_datasets/` contains tools to convert the Cityscapes dataset
+`tools/data_converters/` contains tools to convert the Cityscapes dataset
  and Pascal VOC dataset to the COCO format.
 
 ```shell
-python tools/convert_datasets/cityscapes.py ${CITYSCAPES_PATH} [-h] [--img-dir ${IMG_DIR}] [--gt-dir ${GT_DIR}] [-o ${OUT_DIR}] [--nproc ${NPROC}]
-python tools/convert_datasets/pascal_voc.py ${DEVKIT_PATH} [-h] [-o ${OUT_DIR}]
+python tools/dataset_converters/cityscapes.py ${CITYSCAPES_PATH} [-h] [--img-dir ${IMG_DIR}] [--gt-dir ${GT_DIR}] [-o ${OUT_DIR}] [--nproc ${NPROC}]
+python tools/dataset_converters/pascal_voc.py ${DEVKIT_PATH} [-h] [-o ${OUT_DIR}]
 ```
 
 ## Miscellaneous
 
 ### Evaluating a metric
 
-`tools/eval_metric.py` evaluates certain metrics of a pkl result file
+`tools/analysis_tools/eval_metric.py` evaluates certain metrics of a pkl result file
  according to a config file.
 
 ```shell
-python tools/eval_metric.py ${CONFIG} ${PKL_RESULTS} [-h] [--format-only] [--eval ${EVAL[EVAL ...]}]
+python tools/analysis_tools/eval_metric.py ${CONFIG} ${PKL_RESULTS} [-h] [--format-only] [--eval ${EVAL[EVAL ...]}]
                       [--cfg-options ${CFG_OPTIONS [CFG_OPTIONS ...]}]
                       [--eval-options ${EVAL_OPTIONS [EVAL_OPTIONS ...]}]
 ```
 
 ### Print the entire config
 
-`tools/print_config.py` prints the whole config verbatim, expanding all its
+`tools/misc/print_config.py` prints the whole config verbatim, expanding all its
  imports.
 
 ```shell
-python tools/print_config.py ${CONFIG} [-h] [--options ${OPTIONS [OPTIONS...]}]
+python tools/misc/print_config.py ${CONFIG} [-h] [--options ${OPTIONS [OPTIONS...]}]
 ```
 
 ### Test the robustness of detectors

mmdet/core/evaluation/eval_hooks.py

@@ -14,7 +14,7 @@ class EvalHook(Hook):
 
     Notes:
         If new arguments are added for EvalHook, tools/test.py,
-        tools/eval_metric.py may be effected.
+        tools/analysis_tools/eval_metric.py may be effected.
 
     Attributes:
         dataloader (DataLoader): A PyTorch dataloader.

mmdet/models/detectors/single_stage.py

@@ -59,7 +59,7 @@ class SingleStageDetector(BaseDetector):
     def forward_dummy(self, img):
         """Used for computing network flops.
 
-        See `mmdetection/tools/get_flops.py`
+        See `mmdetection/tools/analysis_tools/get_flops.py`
         """
         x = self.extract_feat(img)
         outs = self.bbox_head(x)

mmdet/models/detectors/sparse_rcnn.py

@@ -92,7 +92,7 @@ class SparseRCNN(TwoStageDetector):
     def forward_dummy(self, img):
         """Used for computing network flops.
 
-        See `mmdetection/tools/get_flops.py`
+        See `mmdetection/tools/analysis_tools/get_flops.py`
         """
         # backbone
         x = self.extract_feat(img)

mmdet/models/detectors/two_stage.py

@@ -87,7 +87,7 @@ class TwoStageDetector(BaseDetector):
     def forward_dummy(self, img):
         """Used for computing network flops.
 
-        See `mmdetection/tools/get_flops.py`
+        See `mmdetection/tools/analysis_tools/get_flops.py`
         """
         outs = ()
         # backbone

mmdet/models/detectors/yolact.py

@@ -32,7 +32,7 @@ class YOLACT(SingleStageDetector):
     def forward_dummy(self, img):
         """Used for computing network flops.
 
-        See `mmdetection/tools/get_flops.py`
+        See `mmdetection/tools/analysis_tools/get_flops.py`
         """
         raise NotImplementedError
 

setup.cfg

@@ -3,7 +3,7 @@ line_length = 79
 multi_line_output = 0
 known_standard_library = setuptools
 known_first_party = mmdet
-known_third_party = PIL,asynctest,cityscapesscripts,cv2,matplotlib,mmcv,numpy,onnx,onnxruntime,pycocotools,pytest,robustness_eval,seaborn,six,terminaltables,torch
+known_third_party = PIL,asynctest,cityscapesscripts,cv2,matplotlib,mmcv,numpy,onnx,onnxruntime,pycocotools,pytest,seaborn,six,terminaltables,torch
 no_lines_before = STDLIB,LOCALFOLDER
 default_section = THIRDPARTY