re-organized the doc

docs/tutorials/1_exist_data_model.md → docs/1_exist_data_model.md

-# Tutorial 1: Inference, testing, and training with predefined models and standard datasets
+# Case 1: Inference, testing, and training with predefined models and standard datasets
 
-Welcome to MMDetection's tutorial.
-
-MMDetection provides hundreds of predefined and pretrained detection models in [Model Zoo](https://mmdetection.readthedocs.io/en/latest/model_zoo.html)), and supports multiple standard datasets, including Pascal VOC, COCO, CityScapes, LVIS, etc. This tutorial will show how to perform common tasks on these pretrained models and standard datasets, including:
+MMDetection provides hundreds of predefined and pretrained detection models in [Model Zoo](https://mmdetection.readthedocs.io/en/latest/model_zoo.html)), and supports multiple standard datasets, including Pascal VOC, COCO, CityScapes, LVIS, etc. This note will show how to perform common tasks on these pretrained models and standard datasets, including:
 
 - Use existing models to inference on given images.
 - Test pretrained models on standard datasets.
@@ -186,7 +184,6 @@ python tools/convert_datasets/cityscapes.py \
 ```
 
 TODO: CHANGE TO THE NEW PATH
-For using custom datasets, please refer to [Tutorials 2: Adding New Dataset](tutorials/new_dataset.md).
 
 ### Test pretrained models
 
@@ -318,7 +315,6 @@ Assume that you have already downloaded the checkpoints to the directory `checkp
 
 MMDetection also provides out-of-the-box tools for training detection models.
 This section will show how to train _predefined_ models (under [configs](https://github.com/open-mmlab/mmdetection/tree/master/configs)) on standard datasets i.e. COCO.
-For training self-defined models, or training with custom datasets. See [Tutorial 2]() and [Tutorial 3]() for details.
 
 **Important**: The default learning rate in config files is for 8 GPUs and 2 img/gpu (batch size = 8\*2 = 16).
 According to the [linear scaling rule](https://arxiv.org/abs/1706.02677), you need to set the learning rate proportional to the batch size if you use different GPUs or images per GPU, e.g., `lr=0.01` for 4 GPUs \* 2 imgs/gpu and `lr=0.08` for 16 GPUs \* 4 imgs/gpu.

docs/tutorials/2_new_data_model.md → docs/2_new_data_model.md

-# Tutorial 2: Inference, testing, and training with predefined models and customized datasets
+# Case 2: Inference, testing, and training with predefined models and customized datasets
 
-In this tutorial, you will know how to inference, test, and train predefined models with customized datasets. We use the [ballon dataset](https://github.com/matterport/Mask_RCNN/tree/master/samples/balloon) as an example to describe the whole process.
+In this case, you will know how to inference, test, and train predefined models with customized datasets. We use the [ballon dataset](https://github.com/matterport/Mask_RCNN/tree/master/samples/balloon) as an example to describe the whole process.
 
 The basic steps are as below:
 
@@ -18,7 +18,7 @@ There are three ways to support a new dataset in MMDetection:
 
 Usually we recommend to use the first two methods which are usually easier than the third.
 
-In this tutorial, we give an example for converting the data into COCO format.
+In this note, we give an example for converting the data into COCO format.
 
 **Note**: MMDetection only supports evaluating mask AP of dataset in COCO format for now.
 So for instance segmentation task users should convert the data into coco format.
@@ -207,7 +207,7 @@ def convert_balloon_to_coco(ann_file, out_file, image_prefix):
 Using the function above, users can successfully convert the annotation file into json format, then we can use `CocoDataset` to train and evaluate the model.
 
 
-### Prepare a config
+## Prepare a config
 
 The second step is to prepare a config thus the dataset could be successfully loaded. Assume that we want to use Mask R-CNN with FPN, the config to train the detector on ballon dataset is as below. Assume the config is under directory `configs/ballon/` and named as `mask_rcnn_r50_caffe_fpn_mstrain-poly_1x_balloon.py`, the config is as below.
 
@@ -242,7 +242,7 @@ data = dict(
 load_from = 'checkpoints/mask_rcnn_r50_caffe_fpn_mstrain-poly_3x_coco_bbox_mAP-0.408__segm_mAP-0.37_20200504_163245-42aa3d00.pth'
 ```
 
-### Train a new model
+## Train a new model
 
 To train a model with the new config, you can simply run
 
@@ -250,9 +250,9 @@ To train a model with the new config, you can simply run
 python tools/train.py configs/ballon/mask_rcnn_r50_caffe_fpn_mstrain-poly_1x_balloon.py
 ```
 
-For more detailed usages, please refer to the tutorial 1.
+For more detailed usages, please refer to the [Case 1](1_exist_data_model.md).
 
-### Test and inference
+## Test and inference
 
 To test the trained model, you can simply run
 
@@ -260,4 +260,4 @@ To test the trained model, you can simply run
 python tools/test.py configs/ballon/mask_rcnn_r50_caffe_fpn_mstrain-poly_1x_balloon.py work_dirs/mask_rcnn_r50_caffe_fpn_mstrain-poly_1x_balloon.py/latest.pth --eval bbox segm
 ```
 
-For more detailed usages, please refer to the tutorial 1.
+For more detailed usages, please refer to the [Case 1](1_exist_data_model.md).

docs/tutorials/3_exist_data_new_model.md → docs/3_exist_data_new_model.md

-# Tutorial 3: Inference, testing, and training with predefined models and standard datasets
+# Case 3: Inference, testing, and training with predefined models and standard datasets
 
-In this tutorial, you will know how to inference, test, and train predefined models with your own settings for standard datasets. We use the cityscapes dataset to train a COCO pretrained Cascade Mask R-CNN model as an example to describe the whole process.
+In this note, you will know how to inference, test, and train predefined models with your own settings for standard datasets. We use the cityscapes dataset to train a COCO pretrained Cascade Mask R-CNN model as an example to describe the whole process.
 
 The basic steps are as below:
 
@@ -10,9 +10,45 @@ The basic steps are as below:
 
 ### Prepare the standard dataset
 
-In this tutorial, as we use the standard cityscapes dataset as an example, to prepare the dataset please follow the tutorial [getting_started](https://github.com/open-mmlab/mmdetection/blob/master/docs/getting_started.md).
+In this note, as we use the standard cityscapes dataset as an example.
 
+It is recommended to symlink the dataset root to `$MMDETECTION/data`.
+If your folder structure is different, you may need to change the corresponding paths in config files.
 
+```
+mmdetection
+├── mmdet
+├── tools
+├── configs
+├── data
+│   ├── coco
+│   │   ├── annotations
+│   │   ├── train2017
+│   │   ├── val2017
+│   │   ├── test2017
+│   ├── cityscapes
+│   │   ├── annotations
+│   │   ├── leftImg8bit
+│   │   │   ├── train
+│   │   │   ├── val
+│   │   ├── gtFine
+│   │   │   ├── train
+│   │   │   ├── val
+│   ├── VOCdevkit
+│   │   ├── VOC2007
+│   │   ├── VOC2012
+
+```
+
+The cityscapes annotations have to be converted into the coco format using `tools/convert_datasets/cityscapes.py`:
+
+```shell
+pip install cityscapesscripts
+python tools/convert_datasets/cityscapes.py ./data/cityscapes --nproc 8 --out-dir ./data/cityscapes/annotations
+```
+
+Currently the config files in `cityscapes` use COCO pre-trained weights to initialize.
+You could download the pre-trained models in advance if network is unavailable or slow, otherwise it would cause errors at the beginning of training.
 
 ### Prepare a config
 
@@ -115,7 +151,7 @@ To train a model with the new config, you can simply run
 python tools/train.py configs/cityscapes/cascade_mask_rcnn_r50_fpn_1x_cityscapes.py
 ```
 
-For more detailed usages, please refer to the tutorial 1.
+For more detailed usages, please refer to the [Case 1](1_exist_data_model.md).
 
 ### Test and inference
 
@@ -125,4 +161,4 @@ To test the trained model, you can simply run
 python tools/test.py configs/cityscapes/cascade_mask_rcnn_r50_fpn_1x_cityscapes.py work_dirs/cascade_mask_rcnn_r50_fpn_1x_cityscapes/latest.pth --eval bbox segm
 ```
 
-For more detailed usages, please refer to the tutorial 1.
+For more detailed usages, please refer to the [Case 1](1_exist_data_model.md).

docs/index.rst

@@ -4,9 +4,21 @@ Welcome to MMDetection's documentation!
 .. toctree::
    :maxdepth: 2
 
-   install.md
    getting_started.md
-   model_zoo.md
+
+.. toctree::
+   :maxdepth: 2
+   :caption: Quick Run
+
+   1_exist_data_model.md
+   2_new_data_model.md
+   3_exist_data_new_model.md
+
+.. toctree::
+   :maxdepth: 2
+   :caption: Useful Tools
+
+   usefultools.md
 
 .. toctree::
    :maxdepth: 2

docs/usefultools.md

+Apart from training/testing scripts, We provide lots of useful tools under the
+ `tools/` directory.
+
+## Visualizations
+
+### Visualizing and analyzing logs
+
+`tools/analyze_logs.py` plots loss/mAP curves given a training
+ log file. Run `pip install seaborn` first to install the dependency.
+
+ ```shell
+python tools/analyze_logs.py plot_curve [--keys ${KEYS}] [--title ${TITLE}] [--legend ${LEGEND}] [--backend ${BACKEND}] [--style ${STYLE}] [--out ${OUT_FILE}]
+```
+
+![loss curve image](../resources/loss_curve.png)
+
+Examples:
+
+- Plot the classification loss of some run.
+
+    ```shell
+    python tools/analyze_logs.py plot_curve log.json --keys loss_cls --legend loss_cls
+    ```
+
+- Plot the classification and regression loss of some run, and save the figure to a pdf.
+
+    ```shell
+    python tools/analyze_logs.py plot_curve log.json --keys loss_cls loss_bbox --out losses.pdf
+    ```
+
+- Compare the bbox mAP of two runs in the same figure.
+
+    ```shell
+    python tools/analyze_logs.py plot_curve log1.json log2.json --keys bbox_mAP --legend run1 run2
+    ```
+
+- Compute the average training speed.
+
+    ```shell
+    python tools/analyze_logs.py cal_train_time log.json [--include-outliers]
+    ```
+
+    The output is expected to be like the following.
+
+    ```
+    -----Analyze train time of work_dirs/some_exp/20190611_192040.log.json-----
+    slowest epoch 11, average time is 1.2024
+    fastest epoch 1, average time is 1.1909
+    time std over epochs is 0.0028
+    average iter time: 1.1959 s/iter
+    ```
+
+### Visualizing datasets
+
+`tools/browse_dataset.py` helps the user to browse a detection dataset (both
+ images and bounding box annotations) visually, or save the image to a
+  designated directory.
+
+```shell
+python tools/browse_dataset.py ${CONFIG} [-h] [--skip-type ${SKIP_TYPE[SKIP_TYPE...]}] [--output-dir ${OUTPUT_DIR}] [--not-show] [--show-interval ${SHOW_INTERVAL}]
+```
+
+### Visualizing models
+
+First, convert the model to ONNX as described
+[here](#convert-mmdetection-model-to-onnx-experimental).
+Note that currently only RetinaNet is supported, support for other models
+ will be coming in later versions.
+The converted model could be visualized by tools like [Netron](https://github.com/lutzroeder/netron).
+
+### Visualizing the output results
+
+If you need a lightweight GUI for visualizing the detection results, you can refer [DetVisGUI project](https://github.com/Chien-Hung/DetVisGUI/tree/mmdetection).
+
+## Analysis
+
+### Analyzing COCO errors
+
+`tools/coco_error_analysis.py` analyzes COCO results per category and by
+ different criterion. It can also make a plot to provide useful
+  information.
+
+```shell
+python tools/coco_error_analysis.py ${RESULT} ${OUT_DIR} [-h] [--ann ${ANN}] [--types ${TYPES[TYPES...]}]
+```
+
+### Get the FLOPs and number of params (experimental)
+
+`tools/get_flops.py` is a script adapted from [flops-counter.pytorch](https://github.com/sovrasov/flops-counter.pytorch) to compute the FLOPs and params of a given model.
+
+```shell
+python tools/get_flops.py ${CONFIG_FILE} [--shape ${INPUT_SHAPE}]
+```
+
+You will get the results like this.
+
+```
+==============================
+Input shape: (3, 1280, 800)
+Flops: 239.32 GFLOPs
+Params: 37.74 M
+==============================
+```
+
+**Note**: This tool is still experimental and we do not guarantee that the
+ number is absolutely correct. You may well use the result for simple
+  comparisons, but double check it before you adopt it in technical reports or papers.
+
+1. FLOPs are related to the input shape while parameters are not. The default
+ input shape is (1, 3, 1280, 800).
+2. Some operators are not counted into FLOPs like GN and custom operators
+. Refer to [`mmcv.cnn.get_model_complexity_info()`](https://github.com/open-mmlab/mmcv/blob/master/mmcv/cnn/utils/flops_counter.py) for details.
+3. The FLOPs of two-stage detectors is dependent on the number of proposals.
+
+### Print the entire config
+
+`tools/print_config.py` prints the whole config verbatim, expanding all its
+ imports.
+
+```shell
+python tools/print_config.py ${CONFIG} [-h] [--options ${OPTIONS [OPTIONS...]}]
+```
+
+### Evaluating a metric
+
+`tools/eval_metric.py` evaluates certain metrics of a pkl result file
+ according to a config file.
+
+```shell
+python tools/eval_metric.py ${CONFIG} ${PKL_RESULTS} [-h] [--format-only] [--eval ${EVAL[EVAL ...]}]
+                      [--cfg-options ${CFG_OPTIONS [CFG_OPTIONS ...]}]
+                      [--eval-options ${EVAL_OPTIONS [EVAL_OPTIONS ...]}]
+```
+
+## Model conversion
+
+### Prepare a model for publishing
+
+`tools/publish_model.py` helps users to prepare their model for publishing.
+
+Before you upload a model to AWS, you may want to
+1. convert model weights to CPU tensors
+2. delete the optimizer states and
+3. compute the hash of the checkpoint file and append the hash id to the
+ filename.
+
+```shell
+python tools/publish_model.py ${INPUT_FILENAME} ${OUTPUT_FILENAME}
+```
+
+E.g.,
+
+```shell
+python tools/publish_model.py work_dirs/faster_rcnn/latest.pth faster_rcnn_r50_fpn_1x_20190801.pth
+```
+
+The final output filename will be `faster_rcnn_r50_fpn_1x_20190801-{hash id}.pth`.
+
+
+### MMDetection model to ONNX (experimental)
+
+We provide a script to convert model to [ONNX](https://github.com/onnx/onnx) format. We also support comparing the output results between Pytorch and
+ ONNX model for verification.
+
+```shell
+python tools/pytorch2onnx.py ${CONFIG_FILE} ${CHECKPOINT_FILE} --output_file ${ONNX_FILE} [--shape ${INPUT_SHAPE} --verify]
+```
+
+**Note**: This tool is still experimental. Some customized operators are not supported for now. We only support exporting RetinaNet model at this moment.
+
+### MMDetection 1.x model to MMDetection 2.x
+
+`tools/upgrade_model_version.py` upgrades a previous MMDetection checkpoint
+ to the new version. Note that this script is not guaranteed to work as some
+  breaking changes are introduced in the new version. It is recommended to
+   directly use the new checkpoints.
+
+```shell
+python tools/upgrade_model_version.py ${IN_FILE} ${OUT_FILE} [-h] [--num-classes NUM_CLASSES]
+```
+
+### Regnet model to MMDetection
+
+`tools/regnet2mmdet.py` convert keys in pycls pretrained RegNet models to
+ MMDetection style.
+
+```shell
+python tools/regnet2mmdet.py ${SRC} ${DST} [-h]
+```
+
+### Detectron ResNet to Pytorch
+
+`tools/detectron2pytorch.py` converts keys in the original detectron pretrained
+ ResNet models to PyTorch style.
+
+```shell
+python tools/detectron2pytorch.py ${SRC} ${DST} ${DEPTH} [-h]
+```
+
+## Dataset Conversion
+
+`tools/convert_datasets/` contains tools to convert the Cityscapes dataset
+ and Pascal VOC dataset to the COCO format.
+
+```shell
+python tools/convert_datasets/cityscapes.py ${CITYSCAPES_PATH} [-h] [--img-dir ${IMG_DIR}] [--gt-dir ${GT_DIR}] [-o ${OUT_DIR}] [--nproc ${NPROC}]
+python tools/convert_datasets/pascal_voc.py ${DEVKIT_PATH} [-h] [-o ${OUT_DIR}]
+```
+
+## Miscellaneous
+
+
+### Test the robustness of detectors
+
+Please refer to [robustness_benchmarking.md](robustness_benchmarking.md).