diff --git a/docs/tutorials/1_exist_data_model.md b/docs/1_exist_data_model.md similarity index 97% rename from docs/tutorials/1_exist_data_model.md rename to docs/1_exist_data_model.md index 24e12537a45acc1a2e582c1948e9e9acfa5d7e75..e1f5b87b6db398bdbc76c1cf1446f27408d11050 100644 --- a/docs/tutorials/1_exist_data_model.md +++ b/docs/1_exist_data_model.md @@ -1,8 +1,6 @@ -# Tutorial 1: Inference, testing, and training with predefined models and standard datasets +# Case 1: Inference, testing, and training with predefined models and standard datasets -Welcome to MMDetection's tutorial. - -MMDetection provides hundreds of predefined and pretrained detection models in [Model Zoo](https://mmdetection.readthedocs.io/en/latest/model_zoo.html)), and supports multiple standard datasets, including Pascal VOC, COCO, CityScapes, LVIS, etc. This tutorial will show how to perform common tasks on these pretrained models and standard datasets, including: +MMDetection provides hundreds of predefined and pretrained detection models in [Model Zoo](https://mmdetection.readthedocs.io/en/latest/model_zoo.html)), and supports multiple standard datasets, including Pascal VOC, COCO, CityScapes, LVIS, etc. This note will show how to perform common tasks on these pretrained models and standard datasets, including: - Use existing models to inference on given images. - Test pretrained models on standard datasets. @@ -186,7 +184,6 @@ python tools/convert_datasets/cityscapes.py \ ``` TODO: CHANGE TO THE NEW PATH -For using custom datasets, please refer to [Tutorials 2: Adding New Dataset](tutorials/new_dataset.md). ### Test pretrained models @@ -318,7 +315,6 @@ Assume that you have already downloaded the checkpoints to the directory `checkp MMDetection also provides out-of-the-box tools for training detection models. This section will show how to train _predefined_ models (under [configs](https://github.com/open-mmlab/mmdetection/tree/master/configs)) on standard datasets i.e. COCO. -For training self-defined models, or training with custom datasets. See [Tutorial 2]() and [Tutorial 3]() for details. **Important**: The default learning rate in config files is for 8 GPUs and 2 img/gpu (batch size = 8\*2 = 16). According to the [linear scaling rule](https://arxiv.org/abs/1706.02677), you need to set the learning rate proportional to the batch size if you use different GPUs or images per GPU, e.g., `lr=0.01` for 4 GPUs \* 2 imgs/gpu and `lr=0.08` for 16 GPUs \* 4 imgs/gpu. diff --git a/docs/tutorials/2_new_data_model.md b/docs/2_new_data_model.md similarity index 90% rename from docs/tutorials/2_new_data_model.md rename to docs/2_new_data_model.md index a5e713e7760c9bd4b0266d7c64b3443f0f413f3a..2ca2a0cc1ea3fea4752007142a3dce570380d30b 100644 --- a/docs/tutorials/2_new_data_model.md +++ b/docs/2_new_data_model.md @@ -1,6 +1,6 @@ -# Tutorial 2: Inference, testing, and training with predefined models and customized datasets +# Case 2: Inference, testing, and training with predefined models and customized datasets -In this tutorial, you will know how to inference, test, and train predefined models with customized datasets. We use the [ballon dataset](https://github.com/matterport/Mask_RCNN/tree/master/samples/balloon) as an example to describe the whole process. +In this case, you will know how to inference, test, and train predefined models with customized datasets. We use the [ballon dataset](https://github.com/matterport/Mask_RCNN/tree/master/samples/balloon) as an example to describe the whole process. The basic steps are as below: @@ -18,7 +18,7 @@ There are three ways to support a new dataset in MMDetection: Usually we recommend to use the first two methods which are usually easier than the third. -In this tutorial, we give an example for converting the data into COCO format. +In this note, we give an example for converting the data into COCO format. **Note**: MMDetection only supports evaluating mask AP of dataset in COCO format for now. So for instance segmentation task users should convert the data into coco format. @@ -207,7 +207,7 @@ def convert_balloon_to_coco(ann_file, out_file, image_prefix): Using the function above, users can successfully convert the annotation file into json format, then we can use `CocoDataset` to train and evaluate the model. -### Prepare a config +## Prepare a config The second step is to prepare a config thus the dataset could be successfully loaded. Assume that we want to use Mask R-CNN with FPN, the config to train the detector on ballon dataset is as below. Assume the config is under directory `configs/ballon/` and named as `mask_rcnn_r50_caffe_fpn_mstrain-poly_1x_balloon.py`, the config is as below. @@ -242,7 +242,7 @@ data = dict( load_from = 'checkpoints/mask_rcnn_r50_caffe_fpn_mstrain-poly_3x_coco_bbox_mAP-0.408__segm_mAP-0.37_20200504_163245-42aa3d00.pth' ``` -### Train a new model +## Train a new model To train a model with the new config, you can simply run @@ -250,9 +250,9 @@ To train a model with the new config, you can simply run python tools/train.py configs/ballon/mask_rcnn_r50_caffe_fpn_mstrain-poly_1x_balloon.py ``` -For more detailed usages, please refer to the tutorial 1. +For more detailed usages, please refer to the [Case 1](1_exist_data_model.md). -### Test and inference +## Test and inference To test the trained model, you can simply run @@ -260,4 +260,4 @@ To test the trained model, you can simply run python tools/test.py configs/ballon/mask_rcnn_r50_caffe_fpn_mstrain-poly_1x_balloon.py work_dirs/mask_rcnn_r50_caffe_fpn_mstrain-poly_1x_balloon.py/latest.pth --eval bbox segm ``` -For more detailed usages, please refer to the tutorial 1. +For more detailed usages, please refer to the [Case 1](1_exist_data_model.md). diff --git a/docs/tutorials/3_exist_data_new_model.md b/docs/3_exist_data_new_model.md similarity index 70% rename from docs/tutorials/3_exist_data_new_model.md rename to docs/3_exist_data_new_model.md index 04c2015cfeb5755a00882acdcb04d7784a761b16..c69424a14f2c4c61084815a360dd7c46716376af 100644 --- a/docs/tutorials/3_exist_data_new_model.md +++ b/docs/3_exist_data_new_model.md @@ -1,6 +1,6 @@ -# Tutorial 3: Inference, testing, and training with predefined models and standard datasets +# Case 3: Inference, testing, and training with predefined models and standard datasets -In this tutorial, you will know how to inference, test, and train predefined models with your own settings for standard datasets. We use the cityscapes dataset to train a COCO pretrained Cascade Mask R-CNN model as an example to describe the whole process. +In this note, you will know how to inference, test, and train predefined models with your own settings for standard datasets. We use the cityscapes dataset to train a COCO pretrained Cascade Mask R-CNN model as an example to describe the whole process. The basic steps are as below: @@ -10,9 +10,45 @@ The basic steps are as below: ### Prepare the standard dataset -In this tutorial, as we use the standard cityscapes dataset as an example, to prepare the dataset please follow the tutorial [getting_started](https://github.com/open-mmlab/mmdetection/blob/master/docs/getting_started.md). +In this note, as we use the standard cityscapes dataset as an example. +It is recommended to symlink the dataset root to `$MMDETECTION/data`. +If your folder structure is different, you may need to change the corresponding paths in config files. +``` +mmdetection +鈹溾攢鈹€ mmdet +鈹溾攢鈹€ tools +鈹溾攢鈹€ configs +鈹溾攢鈹€ data +鈹� 鈹溾攢鈹€ coco +鈹� 鈹� 鈹溾攢鈹€ annotations +鈹� 鈹� 鈹溾攢鈹€ train2017 +鈹� 鈹� 鈹溾攢鈹€ val2017 +鈹� 鈹� 鈹溾攢鈹€ test2017 +鈹� 鈹溾攢鈹€ cityscapes +鈹� 鈹� 鈹溾攢鈹€ annotations +鈹� 鈹� 鈹溾攢鈹€ leftImg8bit +鈹� 鈹� 鈹� 鈹溾攢鈹€ train +鈹� 鈹� 鈹� 鈹溾攢鈹€ val +鈹� 鈹� 鈹溾攢鈹€ gtFine +鈹� 鈹� 鈹� 鈹溾攢鈹€ train +鈹� 鈹� 鈹� 鈹溾攢鈹€ val +鈹� 鈹溾攢鈹€ VOCdevkit +鈹� 鈹� 鈹溾攢鈹€ VOC2007 +鈹� 鈹� 鈹溾攢鈹€ VOC2012 + +``` + +The cityscapes annotations have to be converted into the coco format using `tools/convert_datasets/cityscapes.py`: + +```shell +pip install cityscapesscripts +python tools/convert_datasets/cityscapes.py ./data/cityscapes --nproc 8 --out-dir ./data/cityscapes/annotations +``` + +Currently the config files in `cityscapes` use COCO pre-trained weights to initialize. +You could download the pre-trained models in advance if network is unavailable or slow, otherwise it would cause errors at the beginning of training. ### Prepare a config @@ -115,7 +151,7 @@ To train a model with the new config, you can simply run python tools/train.py configs/cityscapes/cascade_mask_rcnn_r50_fpn_1x_cityscapes.py ``` -For more detailed usages, please refer to the tutorial 1. +For more detailed usages, please refer to the [Case 1](1_exist_data_model.md). ### Test and inference @@ -125,4 +161,4 @@ To test the trained model, you can simply run python tools/test.py configs/cityscapes/cascade_mask_rcnn_r50_fpn_1x_cityscapes.py work_dirs/cascade_mask_rcnn_r50_fpn_1x_cityscapes/latest.pth --eval bbox segm ``` -For more detailed usages, please refer to the tutorial 1. +For more detailed usages, please refer to the [Case 1](1_exist_data_model.md). diff --git a/docs/getting_started.md b/docs/getting_started.md index e86ba3ecbaecce1ba3ced018768b8511ce2022d3..380f3c35a5f5d2a78fc3d708f80c2f711c830ee9 100644 --- a/docs/getting_started.md +++ b/docs/getting_started.md @@ -2,9 +2,54 @@ This page provides basic tutorials about the usage of MMDetection. -## Installation +## Prerequisites + +- Linux or macOS (Windows is not currently officially supported) +- Python 3.6+ +- PyTorch 1.3+ +- CUDA 9.2+ (If you build PyTorch from source, CUDA 9.0 is also compatible) +- GCC 5+ +- [mmcv](https://mmcv.readthedocs.io/en/latest/#installation) + + The required versions of MMCV for different versions of MMDetection since V2.0 are as below. Please install the correct version of MMCV to avoid installation issues. + + | MMDetection version | MMCV version | + |:-------------------:|:-------------------:| + | master | mmcv-full>=1.1.1, <=1.2| + | 2.4.0 | mmcv-full>=1.1.1, <=1.2| + | 2.3.0 | mmcv-full==1.0.5| + | 2.3.0rc0 | mmcv-full>=1.0.2 | + | 2.2.1 | mmcv==0.6.2 | + | 2.2.0 | mmcv==0.6.2 | + | 2.1.0 | mmcv>=0.5.9, <=0.6.1| + | 2.0.0 | mmcv>=0.5.1, <=0.5.8| -### Install MMDetection + Note: You need to run `pip uninstall mmcv` first if you have mmcv installed. + If mmcv and mmcv-full are both installed, there will be `ModuleNotFoundError`. + + We **highly recommend** you to install the pre-build mmcv as below. + + ```shell + pip install mmcv-full==latest+torch1.6.0+cu101 -f https://download.openmmlab.com/mmcv/dist/index.html + ``` + + See [here](https://github.com/open-mmlab/mmcv#install-with-pip) for different versions of MMCV compatible to different PyTorch and CUDA versions. + + Optionally you can choose to compile mmcv from source by the following command + + ```shell + git clone https://github.com/open-mmlab/mmcv.git + cd mmcv + MMCV_WITH_OPS=1 pip install -e . # package mmcv-full will be installed after this step + cd .. + ``` + Or directly run + + ```shell + pip install mmcv-full + ``` + +## Installation 1. Create a conda virtual environment and activate it. @@ -169,621 +214,27 @@ To use the default MMDetection installed in the environment rather than that you PYTHONPATH="$(dirname $0)/..":$PYTHONPATH ``` +## Verification -## Prerequisites - -- Linux or macOS (Windows is not currently officially supported) -- Python 3.6+ -- PyTorch 1.3+ -- CUDA 9.2+ (If you build PyTorch from source, CUDA 9.0 is also compatible) -- GCC 5+ -- [mmcv](https://mmcv.readthedocs.io/en/latest/#installation) - - The required versions of MMCV for different versions of MMDetection since V2.0 are as below. Please install the correct version of MMCV to avoid installation issues. - - | MMDetection version | MMCV version | - |:-------------------:|:-------------------:| - | master | mmcv-full>=1.1.1, <=1.2| - | 2.4.0 | mmcv-full>=1.1.1, <=1.2| - | 2.3.0 | mmcv-full==1.0.5| - | 2.3.0rc0 | mmcv-full>=1.0.2 | - | 2.2.1 | mmcv==0.6.2 | - | 2.2.0 | mmcv==0.6.2 | - | 2.1.0 | mmcv>=0.5.9, <=0.6.1| - | 2.0.0 | mmcv>=0.5.1, <=0.5.8| - - Note: You need to run `pip uninstall mmcv` first if you have mmcv installed. - If mmcv and mmcv-full are both installed, there will be `ModuleNotFoundError`. - - We **highly recommend** you to install the pre-build mmcv as below. - - ```shell - pip install mmcv-full==latest+torch1.6.0+cu101 -f https://download.openmmlab.com/mmcv/dist/index.html - ``` - - See [here](https://github.com/open-mmlab/mmcv#install-with-pip) for different versions of MMCV compatible to different PyTorch and CUDA versions. - - Optionally you can choose to compile mmcv from source by the following command - - ```shell - git clone https://github.com/open-mmlab/mmcv.git - cd mmcv - MMCV_WITH_OPS=1 pip install -e . # package mmcv-full will be installed after this step - cd .. - ``` - Or directly run - - ```shell - pip install mmcv-full - ``` - - -## Prepare datasets - -It is recommended to symlink the dataset root to `$MMDETECTION/data`. -If your folder structure is different, you may need to change the corresponding paths in config files. - -``` -mmdetection -鈹溾攢鈹€ mmdet -鈹溾攢鈹€ tools -鈹溾攢鈹€ configs -鈹溾攢鈹€ data -鈹� 鈹溾攢鈹€ coco -鈹� 鈹� 鈹溾攢鈹€ annotations -鈹� 鈹� 鈹溾攢鈹€ train2017 -鈹� 鈹� 鈹溾攢鈹€ val2017 -鈹� 鈹� 鈹溾攢鈹€ test2017 -鈹� 鈹溾攢鈹€ cityscapes -鈹� 鈹� 鈹溾攢鈹€ annotations -鈹� 鈹� 鈹溾攢鈹€ leftImg8bit -鈹� 鈹� 鈹� 鈹溾攢鈹€ train -鈹� 鈹� 鈹� 鈹溾攢鈹€ val -鈹� 鈹� 鈹溾攢鈹€ gtFine -鈹� 鈹� 鈹� 鈹溾攢鈹€ train -鈹� 鈹� 鈹� 鈹溾攢鈹€ val -鈹� 鈹溾攢鈹€ VOCdevkit -鈹� 鈹� 鈹溾攢鈹€ VOC2007 -鈹� 鈹� 鈹溾攢鈹€ VOC2012 - -``` - -The cityscapes annotations have to be converted into the coco format using `tools/convert_datasets/cityscapes.py`: - -```shell -pip install cityscapesscripts -python tools/convert_datasets/cityscapes.py ./data/cityscapes --nproc 8 --out-dir ./data/cityscapes/annotations -``` - -Currently the config files in `cityscapes` use COCO pre-trained weights to initialize. -You could download the pre-trained models in advance if network is unavailable or slow, otherwise it would cause errors at the beginning of training. - -For using custom datasets, please refer to [Tutorials 2: Customize Datasets](tutorials/customize_dataset.md). - -## Inference with pretrained models - -We provide testing scripts to evaluate a whole dataset (COCO, PASCAL VOC, Cityscapes, etc.), -and also some high-level apis for easier integration to other projects. +To verify whether MMDetection and the required environment are installed correctly, we can run sample python codes to initialize a detector and inference a demo image: -### Test a dataset +From the command line, type: -- single GPU -- single node multiple GPU -- multiple node - -You can use the following commands to test a dataset. - -```shell -# single-gpu testing -python tools/test.py ${CONFIG_FILE} ${CHECKPOINT_FILE} [--out ${RESULT_FILE}] [--eval ${EVAL_METRICS}] [--show] [--cfg-options] - -# multi-gpu testing -./tools/dist_test.sh ${CONFIG_FILE} ${CHECKPOINT_FILE} ${GPU_NUM} [--out ${RESULT_FILE}] [--eval ${EVAL_METRICS}] [--cfg-options] +```bash +python ``` -Optional arguments: -- `RESULT_FILE`: Filename of the output results in pickle format. If not specified, the results will not be saved to a file. -- `EVAL_METRICS`: Items to be evaluated on the results. Allowed values depend on the dataset, e.g., `proposal_fast`, `proposal`, `bbox`, `segm` are available for COCO, `mAP`, `recall` for PASCAL VOC. Cityscapes could be evaluated by `cityscapes` as well as all COCO metrics. -- `--show`: If specified, detection results will be plotted on the images and shown in a new window. It is only applicable to single GPU testing and used for debugging and visualization. Please make sure that GUI is available in your environment, otherwise you may encounter the error like `cannot connect to X server`. -- `--show-dir`: If specified, detection results will be plotted on the images and saved to the specified directory. It is only applicable to single GPU testing and used for debugging and visualization. You do NOT need a GUI available in your environment for using this option. -- `--show-score-thr`: If specified, detections with score below this threshold will be removed. -- `--cfg-options`: If specified, some setting in the used config will be overridden. - -Examples: - -Assume that you have already downloaded the checkpoints to the directory `checkpoints/`. - -1. Test Faster R-CNN and visualize the results. Press any key for the next image. - - ```shell - python tools/test.py configs/faster_rcnn/faster_rcnn_r50_fpn_1x_coco.py \ - checkpoints/faster_rcnn_r50_fpn_1x_coco_20200130-047c8118.pth \ - --show - ``` - -2. Test Faster R-CNN and save the painted images for latter visualization. - - ```shell - python tools/test.py configs/faster_rcnn/faster_rcnn_r50_fpn_1x.py \ - checkpoints/faster_rcnn_r50_fpn_1x_coco_20200130-047c8118.pth \ - --show-dir faster_rcnn_r50_fpn_1x_results - ``` - -3. Test Faster R-CNN on PASCAL VOC (without saving the test results) and evaluate the mAP. - - ```shell - python tools/test.py configs/pascal_voc/faster_rcnn_r50_fpn_1x_voc.py \ - checkpoints/SOME_CHECKPOINT.pth \ - --eval mAP - ``` - -4. Test Mask R-CNN with 8 GPUs, and evaluate the bbox and mask AP. - - ```shell - ./tools/dist_test.sh configs/mask_rcnn/mask_rcnn_r50_fpn_1x_coco.py \ - checkpoints/mask_rcnn_r50_fpn_1x_coco_20200205-d4b0c5d6.pth \ - 8 --out results.pkl --eval bbox segm - ``` - -5. Test Mask R-CNN with 8 GPUs, and evaluate the **classwise** bbox and mask AP. - - ```shell - ./tools/dist_test.sh configs/mask_rcnn/mask_rcnn_r50_fpn_1x_coco.py \ - checkpoints/mask_rcnn_r50_fpn_1x_coco_20200205-d4b0c5d6.pth \ - 8 --out results.pkl --eval bbox segm --eval-options "classwise=True" - ``` - -6. Test Mask R-CNN on COCO test-dev with 8 GPUs, and generate the json file to be submit to the official evaluation server. - - ```shell - ./tools/dist_test.sh configs/mask_rcnn/mask_rcnn_r50_fpn_1x_coco.py \ - checkpoints/mask_rcnn_r50_fpn_1x_coco_20200205-d4b0c5d6.pth \ - 8 --format-only --eval-options "jsonfile_prefix=./mask_rcnn_test-dev_results" - ``` - - You will get two json files `mask_rcnn_test-dev_results.bbox.json` and `mask_rcnn_test-dev_results.segm.json`. - -7. Test Mask R-CNN on Cityscapes test with 8 GPUs, and generate the txt and png files to be submit to the official evaluation server. - - ```shell - ./tools/dist_test.sh configs/cityscapes/mask_rcnn_r50_fpn_1x_cityscapes.py \ - checkpoints/mask_rcnn_r50_fpn_1x_cityscapes_20200227-afe51d5a.pth \ - 8 --format-only --eval-options "txtfile_prefix=./mask_rcnn_cityscapes_test_results" - ``` - - The generated png and txt would be under `./mask_rcnn_cityscapes_test_results` directory. - -### Image demo - -We provide a demo script to test a single image. - -```shell -python demo/image_demo.py ${IMAGE_FILE} ${CONFIG_FILE} ${CHECKPOINT_FILE} [--device ${GPU_ID}] [--score-thr ${SCORE_THR}] -``` - -Examples: - -```shell -python demo/image_demo.py demo/demo.jpg configs/faster_rcnn/faster_rcnn_r50_fpn_1x_coco.py \ - checkpoints/faster_rcnn_r50_fpn_1x_coco_20200130-047c8118.pth --device cpu -``` - -### Webcam demo - -We provide a webcam demo to illustrate the results. - -```shell -python demo/webcam_demo.py ${CONFIG_FILE} ${CHECKPOINT_FILE} [--device ${GPU_ID}] [--camera-id ${CAMERA-ID}] [--score-thr ${SCORE_THR}] -``` - -Examples: - -```shell -python demo/webcam_demo.py configs/faster_rcnn/faster_rcnn_r50_fpn_1x_coco.py \ - checkpoints/faster_rcnn_r50_fpn_1x_coco_20200130-047c8118.pth -``` - - -### High-level APIs for testing images - -#### Synchronous interface -Here is an example of building the model and test given images. +then enter the following code: ```python from mmdet.apis import init_detector, inference_detector -import mmcv config_file = 'configs/faster_rcnn/faster_rcnn_r50_fpn_1x_coco.py' -checkpoint_file = 'checkpoints/faster_rcnn_r50_fpn_1x_coco_20200130-047c8118.pth' - -# build the model from a config file and a checkpoint file -model = init_detector(config_file, checkpoint_file, device='cuda:0') - -# test a single image and show the results -img = 'test.jpg' # or img = mmcv.imread(img), which will only load it once -result = inference_detector(model, img) -# visualize the results in a new window -model.show_result(img, result) -# or save the visualization results to image files -model.show_result(img, result, out_file='result.jpg') - -# test a video and show the results -video = mmcv.VideoReader('video.mp4') -for frame in video: - result = inference_detector(model, frame) - model.show_result(frame, result, wait_time=1) +device = 'cuda:0' +# init a detector +model = init_detector(config_file, device=device) +# inference the demo image +inference_detector(model, 'demo/demo.jpg') ``` -A notebook demo can be found in [demo/inference_demo.ipynb](https://github.com/open-mmlab/mmdetection/blob/master/demo/inference_demo.ipynb). - -#### Asynchronous interface - supported for Python 3.7+ - -Async interface allows not to block CPU on GPU bound inference code and enables better CPU/GPU utilization for single threaded application. Inference can be done concurrently either between different input data samples or between different models of some inference pipeline. - -See `tests/async_benchmark.py` to compare the speed of synchronous and asynchronous interfaces. - -```python -import asyncio -import torch -from mmdet.apis import init_detector, async_inference_detector -from mmdet.utils.contextmanagers import concurrent - -async def main(): - config_file = 'configs/faster_rcnn/faster_rcnn_r50_fpn_1x_coco.py' - checkpoint_file = 'checkpoints/faster_rcnn_r50_fpn_1x_coco_20200130-047c8118.pth' - device = 'cuda:0' - model = init_detector(config_file, checkpoint=checkpoint_file, device=device) - - # queue is used for concurrent inference of multiple images - streamqueue = asyncio.Queue() - # queue size defines concurrency level - streamqueue_size = 3 - - for _ in range(streamqueue_size): - streamqueue.put_nowait(torch.cuda.Stream(device=device)) - - # test a single image and show the results - img = 'test.jpg' # or img = mmcv.imread(img), which will only load it once - - async with concurrent(streamqueue): - result = await async_inference_detector(model, img) - - # visualize the results in a new window - model.show_result(img, result) - # or save the visualization results to image files - model.show_result(img, result, out_file='result.jpg') - - -asyncio.run(main()) - -``` - - -## Train a model - -MMDetection implements distributed training and non-distributed training, -which uses `MMDistributedDataParallel` and `MMDataParallel` respectively. - -All outputs (log files and checkpoints) will be saved to the working directory, -which is specified by `work_dir` in the config file. - -By default we evaluate the model on the validation set after each epoch, you can change the evaluation interval by adding the interval argument in the training config. -```python -evaluation = dict(interval=12) # This evaluate the model per 12 epoch. -``` - -**\*Important\***: The default learning rate in config files is for 8 GPUs and 2 img/gpu (batch size = 8*2 = 16). -According to the [Linear Scaling Rule](https://arxiv.org/abs/1706.02677), you need to set the learning rate proportional to the batch size if you use different GPUs or images per GPU, e.g., lr=0.01 for 4 GPUs * 2 img/gpu and lr=0.08 for 16 GPUs * 4 img/gpu. - -### Train with a single GPU - -```shell -python tools/train.py ${CONFIG_FILE} [optional arguments] -``` - -If you want to specify the working directory in the command, you can add an argument `--work-dir ${YOUR_WORK_DIR}`. - -### Train with multiple GPUs - -```shell -./tools/dist_train.sh ${CONFIG_FILE} ${GPU_NUM} [optional arguments] -``` - -Optional arguments are: - -- `--no-validate` (**not suggested**): By default, the codebase will perform evaluation at every k (default value is 1, which can be modified like [this](https://github.com/open-mmlab/mmdetection/blob/master/configs/mask_rcnn/mask_rcnn_r50_fpn_1x_coco.py#L174)) epochs during the training. To disable this behavior, use `--no-validate`. -- `--work-dir ${WORK_DIR}`: Override the working directory specified in the config file. -- `--resume-from ${CHECKPOINT_FILE}`: Resume from a previous checkpoint file. -- `--cfg-options 'Key=value'`: Overide some settings in the used config. - -**Note**: - -- `resume-from` loads both the model weights and optimizer status, and the epoch is also inherited from the specified checkpoint. It is usually used for resuming the training process that is interrupted accidentally. -- For more clear usage, the original `load-from` is deprecated and you can use `--cfg-options 'load_from="path/to/you/model"'` instead. It only loads the model weights and the training epoch starts from 0 which is usually used for finetuning. - -### Train with multiple machines - -If you run MMDetection on a cluster managed with [slurm](https://slurm.schedmd.com/), you can use the script `slurm_train.sh`. (This script also supports single machine training.) - -```shell -[GPUS=${GPUS}] ./tools/slurm_train.sh ${PARTITION} ${JOB_NAME} ${CONFIG_FILE} ${WORK_DIR} -``` - -Here is an example of using 16 GPUs to train Mask R-CNN on the dev partition. - -```shell -GPUS=16 ./tools/slurm_train.sh dev mask_r50_1x configs/mask_rcnn/mask_rcnn_r50_fpn_1x_coco.py /nfs/xxxx/mask_rcnn_r50_fpn_1x -``` - -You can check [slurm_train.sh](https://github.com/open-mmlab/mmdetection/blob/master/tools/slurm_train.sh) for full arguments and environment variables. - -If you have just multiple machines connected with ethernet, you can refer to -PyTorch [launch utility](https://pytorch.org/docs/stable/distributed.html#launch-utility). -Usually it is slow if you do not have high speed networking like InfiniBand. - -### Launch multiple jobs on a single machine - -If you launch multiple jobs on a single machine, e.g., 2 jobs of 4-GPU training on a machine with 8 GPUs, -you need to specify different ports (29500 by default) for each job to avoid communication conflict. - -If you use `dist_train.sh` to launch training jobs, you can set the port in commands. - -```shell -CUDA_VISIBLE_DEVICES=0,1,2,3 PORT=29500 ./tools/dist_train.sh ${CONFIG_FILE} 4 -CUDA_VISIBLE_DEVICES=4,5,6,7 PORT=29501 ./tools/dist_train.sh ${CONFIG_FILE} 4 -``` - -If you use launch training jobs with Slurm, there are two ways to specify the ports. - -1. Set the port through `--cfg-options`. This is more recommended since it does not change the original configs. - - ```shell - CUDA_VISIBLE_DEVICES=0,1,2,3 GPUS=4 ./tools/slurm_train.sh ${PARTITION} ${JOB_NAME} config1.py ${WORK_DIR} --options 'dist_params.port=29500' - CUDA_VISIBLE_DEVICES=4,5,6,7 GPUS=4 ./tools/slurm_train.sh ${PARTITION} ${JOB_NAME} config2.py ${WORK_DIR} --options 'dist_params.port=29501' - ``` - -2. Modify the config files (usually the 6th line from the bottom in config files) to set different communication ports. - - In `config1.py`, - - ```python - dist_params = dict(backend='nccl', port=29500) - ``` - - In `config2.py`, - - ```python - dist_params = dict(backend='nccl', port=29501) - ``` - - Then you can launch two jobs with `config1.py` ang `config2.py`. - - ```shell - CUDA_VISIBLE_DEVICES=0,1,2,3 GPUS=4 ./tools/slurm_train.sh ${PARTITION} ${JOB_NAME} config1.py ${WORK_DIR} - CUDA_VISIBLE_DEVICES=4,5,6,7 GPUS=4 ./tools/slurm_train.sh ${PARTITION} ${JOB_NAME} config2.py ${WORK_DIR} - ``` - -## Useful tools - -Apart from training/testing scripts, We provide lots of useful tools under the - `tools/` directory. - -### Visualizations - -#### Visualizing and analyzing logs - -`tools/analyze_logs.py` plots loss/mAP curves given a training - log file. Run `pip install seaborn` first to install the dependency. - - ```shell -python tools/analyze_logs.py plot_curve [--keys ${KEYS}] [--title ${TITLE}] [--legend ${LEGEND}] [--backend ${BACKEND}] [--style ${STYLE}] [--out ${OUT_FILE}] -``` - - - -Examples: - -- Plot the classification loss of some run. - - ```shell - python tools/analyze_logs.py plot_curve log.json --keys loss_cls --legend loss_cls - ``` - -- Plot the classification and regression loss of some run, and save the figure to a pdf. - - ```shell - python tools/analyze_logs.py plot_curve log.json --keys loss_cls loss_bbox --out losses.pdf - ``` - -- Compare the bbox mAP of two runs in the same figure. - - ```shell - python tools/analyze_logs.py plot_curve log1.json log2.json --keys bbox_mAP --legend run1 run2 - ``` - -- Compute the average training speed. - - ```shell - python tools/analyze_logs.py cal_train_time log.json [--include-outliers] - ``` - - The output is expected to be like the following. - - ``` - -----Analyze train time of work_dirs/some_exp/20190611_192040.log.json----- - slowest epoch 11, average time is 1.2024 - fastest epoch 1, average time is 1.1909 - time std over epochs is 0.0028 - average iter time: 1.1959 s/iter - ``` - -#### Visualizing datasets - -`tools/browse_dataset.py` helps the user to browse a detection dataset (both - images and bounding box annotations) visually, or save the image to a - designated directory. - -```shell -python tools/browse_dataset.py ${CONFIG} [-h] [--skip-type ${SKIP_TYPE[SKIP_TYPE...]}] [--output-dir ${OUTPUT_DIR}] [--not-show] [--show-interval ${SHOW_INTERVAL}] -``` - -#### Visualizing models - -First, convert the model to ONNX as described -[here](#convert-mmdetection-model-to-onnx-experimental). -Note that currently only RetinaNet is supported, support for other models - will be coming in later versions. -The converted model could be visualized by tools like [Netron](https://github.com/lutzroeder/netron). - -#### Visualizing the output results - -If you need a lightweight GUI for visualizing the detection results, you can refer [DetVisGUI project](https://github.com/Chien-Hung/DetVisGUI/tree/mmdetection). - -### Analysis - -#### Analyzing COCO errors - -`tools/coco_error_analysis.py` analyzes COCO results per category and by - different criterion. It can also make a plot to provide useful - information. - -```shell -python tools/coco_error_analysis.py ${RESULT} ${OUT_DIR} [-h] [--ann ${ANN}] [--types ${TYPES[TYPES...]}] -``` - -#### Get the FLOPs and number of params (experimental) - -`tools/get_flops.py` is a script adapted from [flops-counter.pytorch](https://github.com/sovrasov/flops-counter.pytorch) to compute the FLOPs and params of a given model. - -```shell -python tools/get_flops.py ${CONFIG_FILE} [--shape ${INPUT_SHAPE}] -``` - -You will get the results like this. - -``` -============================== -Input shape: (3, 1280, 800) -Flops: 239.32 GFLOPs -Params: 37.74 M -============================== -``` - -**Note**: This tool is still experimental and we do not guarantee that the - number is absolutely correct. You may well use the result for simple - comparisons, but double check it before you adopt it in technical reports or papers. - -1. FLOPs are related to the input shape while parameters are not. The default - input shape is (1, 3, 1280, 800). -2. Some operators are not counted into FLOPs like GN and custom operators -. Refer to [`mmcv.cnn.get_model_complexity_info()`](https://github.com/open-mmlab/mmcv/blob/master/mmcv/cnn/utils/flops_counter.py) for details. -3. The FLOPs of two-stage detectors is dependent on the number of proposals. - -#### Print the entire config - -`tools/print_config.py` prints the whole config verbatim, expanding all its - imports. - -```shell -python tools/print_config.py ${CONFIG} [-h] [--options ${OPTIONS [OPTIONS...]}] -``` - -#### Evaluating a metric - -`tools/eval_metric.py` evaluates certain metrics of a pkl result file - according to a config file. - -```shell -python tools/eval_metric.py ${CONFIG} ${PKL_RESULTS} [-h] [--format-only] [--eval ${EVAL[EVAL ...]}] - [--cfg-options ${CFG_OPTIONS [CFG_OPTIONS ...]}] - [--eval-options ${EVAL_OPTIONS [EVAL_OPTIONS ...]}] -``` - -### Model conversion - -#### Prepare a model for publishing - -`tools/publish_model.py` helps users to prepare their model for publishing. - -Before you upload a model to AWS, you may want to -1. convert model weights to CPU tensors -2. delete the optimizer states and -3. compute the hash of the checkpoint file and append the hash id to the - filename. - -```shell -python tools/publish_model.py ${INPUT_FILENAME} ${OUTPUT_FILENAME} -``` - -E.g., - -```shell -python tools/publish_model.py work_dirs/faster_rcnn/latest.pth faster_rcnn_r50_fpn_1x_20190801.pth -``` - -The final output filename will be `faster_rcnn_r50_fpn_1x_20190801-{hash id}.pth`. - - -#### MMDetection model to ONNX (experimental) - -We provide a script to convert model to [ONNX](https://github.com/onnx/onnx) format. We also support comparing the output results between Pytorch and - ONNX model for verification. - -```shell -python tools/pytorch2onnx.py ${CONFIG_FILE} ${CHECKPOINT_FILE} --output_file ${ONNX_FILE} [--shape ${INPUT_SHAPE} --verify] -``` - -**Note**: This tool is still experimental. Some customized operators are not supported for now. We only support exporting RetinaNet model at this moment. - -#### MMDetection 1.x model to MMDetection 2.x - -`tools/upgrade_model_version.py` upgrades a previous MMDetection checkpoint - to the new version. Note that this script is not guaranteed to work as some - breaking changes are introduced in the new version. It is recommended to - directly use the new checkpoints. - -```shell -python tools/upgrade_model_version.py ${IN_FILE} ${OUT_FILE} [-h] [--num-classes NUM_CLASSES] -``` - -#### Regnet model to MMDetection - -`tools/regnet2mmdet.py` convert keys in pycls pretrained RegNet models to - MMDetection style. - -```shell -python tools/regnet2mmdet.py ${SRC} ${DST} [-h] -``` - -#### Detectron ResNet to Pytorch - -`tools/detectron2pytorch.py` converts keys in the original detectron pretrained - ResNet models to PyTorch style. - -```shell -python tools/detectron2pytorch.py ${SRC} ${DST} ${DEPTH} [-h] -``` - -### Dataset Conversion - -`tools/convert_datasets/` contains tools to convert the Cityscapes dataset - and Pascal VOC dataset to the COCO format. - -```shell -python tools/convert_datasets/cityscapes.py ${CITYSCAPES_PATH} [-h] [--img-dir ${IMG_DIR}] [--gt-dir ${GT_DIR}] [-o ${OUT_DIR}] [--nproc ${NPROC}] -python tools/convert_datasets/pascal_voc.py ${DEVKIT_PATH} [-h] [-o ${OUT_DIR}] -``` - -### Miscellaneous - - -#### Test the robustness of detectors - -Please refer to [robustness_benchmarking.md](robustness_benchmarking.md). - - - -## Tutorials - -Currently, we provide four tutorials for users to [finetune models](tutorials/finetune.md), [customize datasets](tutorials/customize_dataset.md), [design data pipeline](tutorials/data_pipeline.md), [customize modules](tutorials/customize_models.md), [customize runtime settings](tutorials/customize_runtime.md), and [customize losses](tutorials/customize_losses.md). -We also provide a full description about the [config system](config.md). +The above code is supposed to run successfully upon you finish the installation. diff --git a/docs/index.rst b/docs/index.rst index 8d34206f7d432250401ef09263ff475b05f7be56..6403219deb73f6d3c8feeaadcbca2f094ded663d 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -4,9 +4,21 @@ Welcome to MMDetection's documentation! .. toctree:: :maxdepth: 2 - install.md getting_started.md - model_zoo.md + +.. toctree:: + :maxdepth: 2 + :caption: Quick Run + + 1_exist_data_model.md + 2_new_data_model.md + 3_exist_data_new_model.md + +.. toctree:: + :maxdepth: 2 + :caption: Useful Tools + + usefultools.md .. toctree:: :maxdepth: 2 diff --git a/docs/usefultools.md b/docs/usefultools.md new file mode 100644 index 0000000000000000000000000000000000000000..53e17ece86bd1cba8e654fe4abdc0549600fa850 --- /dev/null +++ b/docs/usefultools.md @@ -0,0 +1,215 @@ +Apart from training/testing scripts, We provide lots of useful tools under the + `tools/` directory. + +## Visualizations + +### Visualizing and analyzing logs + +`tools/analyze_logs.py` plots loss/mAP curves given a training + log file. Run `pip install seaborn` first to install the dependency. + + ```shell +python tools/analyze_logs.py plot_curve [--keys ${KEYS}] [--title ${TITLE}] [--legend ${LEGEND}] [--backend ${BACKEND}] [--style ${STYLE}] [--out ${OUT_FILE}] +``` + + + +Examples: + +- Plot the classification loss of some run. + + ```shell + python tools/analyze_logs.py plot_curve log.json --keys loss_cls --legend loss_cls + ``` + +- Plot the classification and regression loss of some run, and save the figure to a pdf. + + ```shell + python tools/analyze_logs.py plot_curve log.json --keys loss_cls loss_bbox --out losses.pdf + ``` + +- Compare the bbox mAP of two runs in the same figure. + + ```shell + python tools/analyze_logs.py plot_curve log1.json log2.json --keys bbox_mAP --legend run1 run2 + ``` + +- Compute the average training speed. + + ```shell + python tools/analyze_logs.py cal_train_time log.json [--include-outliers] + ``` + + The output is expected to be like the following. + + ``` + -----Analyze train time of work_dirs/some_exp/20190611_192040.log.json----- + slowest epoch 11, average time is 1.2024 + fastest epoch 1, average time is 1.1909 + time std over epochs is 0.0028 + average iter time: 1.1959 s/iter + ``` + +### Visualizing datasets + +`tools/browse_dataset.py` helps the user to browse a detection dataset (both + images and bounding box annotations) visually, or save the image to a + designated directory. + +```shell +python tools/browse_dataset.py ${CONFIG} [-h] [--skip-type ${SKIP_TYPE[SKIP_TYPE...]}] [--output-dir ${OUTPUT_DIR}] [--not-show] [--show-interval ${SHOW_INTERVAL}] +``` + +### Visualizing models + +First, convert the model to ONNX as described +[here](#convert-mmdetection-model-to-onnx-experimental). +Note that currently only RetinaNet is supported, support for other models + will be coming in later versions. +The converted model could be visualized by tools like [Netron](https://github.com/lutzroeder/netron). + +### Visualizing the output results + +If you need a lightweight GUI for visualizing the detection results, you can refer [DetVisGUI project](https://github.com/Chien-Hung/DetVisGUI/tree/mmdetection). + +## Analysis + +### Analyzing COCO errors + +`tools/coco_error_analysis.py` analyzes COCO results per category and by + different criterion. It can also make a plot to provide useful + information. + +```shell +python tools/coco_error_analysis.py ${RESULT} ${OUT_DIR} [-h] [--ann ${ANN}] [--types ${TYPES[TYPES...]}] +``` + +### Get the FLOPs and number of params (experimental) + +`tools/get_flops.py` is a script adapted from [flops-counter.pytorch](https://github.com/sovrasov/flops-counter.pytorch) to compute the FLOPs and params of a given model. + +```shell +python tools/get_flops.py ${CONFIG_FILE} [--shape ${INPUT_SHAPE}] +``` + +You will get the results like this. + +``` +============================== +Input shape: (3, 1280, 800) +Flops: 239.32 GFLOPs +Params: 37.74 M +============================== +``` + +**Note**: This tool is still experimental and we do not guarantee that the + number is absolutely correct. You may well use the result for simple + comparisons, but double check it before you adopt it in technical reports or papers. + +1. FLOPs are related to the input shape while parameters are not. The default + input shape is (1, 3, 1280, 800). +2. Some operators are not counted into FLOPs like GN and custom operators +. Refer to [`mmcv.cnn.get_model_complexity_info()`](https://github.com/open-mmlab/mmcv/blob/master/mmcv/cnn/utils/flops_counter.py) for details. +3. The FLOPs of two-stage detectors is dependent on the number of proposals. + +### Print the entire config + +`tools/print_config.py` prints the whole config verbatim, expanding all its + imports. + +```shell +python tools/print_config.py ${CONFIG} [-h] [--options ${OPTIONS [OPTIONS...]}] +``` + +### Evaluating a metric + +`tools/eval_metric.py` evaluates certain metrics of a pkl result file + according to a config file. + +```shell +python tools/eval_metric.py ${CONFIG} ${PKL_RESULTS} [-h] [--format-only] [--eval ${EVAL[EVAL ...]}] + [--cfg-options ${CFG_OPTIONS [CFG_OPTIONS ...]}] + [--eval-options ${EVAL_OPTIONS [EVAL_OPTIONS ...]}] +``` + +## Model conversion + +### Prepare a model for publishing + +`tools/publish_model.py` helps users to prepare their model for publishing. + +Before you upload a model to AWS, you may want to +1. convert model weights to CPU tensors +2. delete the optimizer states and +3. compute the hash of the checkpoint file and append the hash id to the + filename. + +```shell +python tools/publish_model.py ${INPUT_FILENAME} ${OUTPUT_FILENAME} +``` + +E.g., + +```shell +python tools/publish_model.py work_dirs/faster_rcnn/latest.pth faster_rcnn_r50_fpn_1x_20190801.pth +``` + +The final output filename will be `faster_rcnn_r50_fpn_1x_20190801-{hash id}.pth`. + + +### MMDetection model to ONNX (experimental) + +We provide a script to convert model to [ONNX](https://github.com/onnx/onnx) format. We also support comparing the output results between Pytorch and + ONNX model for verification. + +```shell +python tools/pytorch2onnx.py ${CONFIG_FILE} ${CHECKPOINT_FILE} --output_file ${ONNX_FILE} [--shape ${INPUT_SHAPE} --verify] +``` + +**Note**: This tool is still experimental. Some customized operators are not supported for now. We only support exporting RetinaNet model at this moment. + +### MMDetection 1.x model to MMDetection 2.x + +`tools/upgrade_model_version.py` upgrades a previous MMDetection checkpoint + to the new version. Note that this script is not guaranteed to work as some + breaking changes are introduced in the new version. It is recommended to + directly use the new checkpoints. + +```shell +python tools/upgrade_model_version.py ${IN_FILE} ${OUT_FILE} [-h] [--num-classes NUM_CLASSES] +``` + +### Regnet model to MMDetection + +`tools/regnet2mmdet.py` convert keys in pycls pretrained RegNet models to + MMDetection style. + +```shell +python tools/regnet2mmdet.py ${SRC} ${DST} [-h] +``` + +### Detectron ResNet to Pytorch + +`tools/detectron2pytorch.py` converts keys in the original detectron pretrained + ResNet models to PyTorch style. + +```shell +python tools/detectron2pytorch.py ${SRC} ${DST} ${DEPTH} [-h] +``` + +## Dataset Conversion + +`tools/convert_datasets/` contains tools to convert the Cityscapes dataset + and Pascal VOC dataset to the COCO format. + +```shell +python tools/convert_datasets/cityscapes.py ${CITYSCAPES_PATH} [-h] [--img-dir ${IMG_DIR}] [--gt-dir ${GT_DIR}] [-o ${OUT_DIR}] [--nproc ${NPROC}] +python tools/convert_datasets/pascal_voc.py ${DEVKIT_PATH} [-h] [-o ${OUT_DIR}] +``` + +## Miscellaneous + + +### Test the robustness of detectors + +Please refer to [robustness_benchmarking.md](robustness_benchmarking.md).