Further changes to the TCVC implementation for merging

video-colorization/tcvc/README.md

@@ -1,11 +1,12 @@
 # TCVC Example by NNabla
-This is a NNabla implementation of the "Automatic Temporally Coherent Video Colorization" (TCVC).
+This is a NNabla implementation of the "[Automatic Temporally Coherent Video Colorization (TCVC)](https://arxiv.org/abs/1904.09527)".
 
 This work is done by Mingcheng Yuan during his internship at Sony R&D in 2021
 
-## Requirements
+<br/>
 
-### python environments
+## Requirements
+### Python Environments
 You can install all dependencies from `requirements.txt`.
 ```bash
 pip install -r ./requirements.txt
@@ -15,45 +16,53 @@ Note that `nnabla-ext-cuda110`(requiring CUDA 11.0.3 and CUDNN 8.0.5) will be in
 If you've installed other version of CUDA, please modify `requirements.txt` to meet your environment.  
 For more information, see [NNabla CUDA extension package installation guild](https://nnabla.readthedocs.io/en/latest/python/pip_installation_cuda.html).
 
+<br/>
+
 ## Dataset
-Please prepare the dataset by extracting frames (require `ffmpeg`, currently won't work when installed with pip but works with conda) from anime episodes with 
+Please prepare the dataset by extracting frames from anime episodes (users should prepare their own arbitrary anime clips)  
+Extract with the following command (requires `ffmpeg`, currently won't work when installed with pip but works with conda)
 ```
 python -m external.extract_video_frames --input-path path-to-video 
 ```
-and extract the line-art with "[sketchKeras](https://github.com/lllyasviel/sketchKeras)"
+After extracting the frame, extract the line-art with "[sketchKeras](https://github.com/lllyasviel/sketchKeras)"
 
+<br/>
 
 ## Running Code
-### Train with the original tcvc (or reference-based version)
+### Train with the original tcvc ( _feat for reference-based version)
 ```
 # by a single GPU
-./scripts/generate_tcvc.sh
+./scripts/train_tcvc.sh
 or
-./scripts/generate_tcvc_feat.sh
+./scripts/train_tcvc_ref.sh
 
 # by multi GPUs
-./scripts/generate_tcvc_mgpu.sh
+./scripts/train_tcvc_mgpu.sh
 or
-./scripts/generate_tcvc_feat_mgpu.sh
+./scripts/train_tcvc_ref_mgpu.sh
 ```
 Trained model will be saved under the log directory (default is `./result/tcvc_01/`).
 Tensorboard logs will be saved under `./result/tcvc_01/tb_logs`.
 
-### Generate image 
+### Generate Image 
 ```
-# by a single GPU
+# original tcvc 
 ./scripts/generate_tcvc.sh
 
-# by multi GPUs
-./scripts/generate_tcvc_feat.sh
+# reference-based version
+./scripts/generate_tcvc_ref.sh
 ```
+
 In the default setting, model parameter is loaded from `./results/tcvc_01/015_param.h5` which is created at the end of 16th epoch during training script.  
 Generated images will be saved under the log directory (default is `./results/tcvc_01/gen_results`).
 
+<br/>
 
 ## Note 
 * All configurations are listed in config.yaml. You can change model settings by editing this.
 
+<br/>
+
 ## Reference
 - Thasarathan, Harrish, Kamyar Nazeri, and Mehran Ebrahimi,  "[Automatic Temporally Coherent Video Colorization](https://github.com/iver56/automatic-video-colorization)"
 

video-colorization/tcvc/args.py

@@ -1,3 +1,17 @@
+# Copyright 2021 Sony Group Corporation
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
 import os
 import argparse
 

video-colorization/tcvc/data/__init__.py

@@ -1,4 +1,4 @@
-# Copyright (c) 2017 Sony Corporation. All Rights Reserved.
+# Copyright 2021 Sony Group Corporation
 #
 # Licensed under the Apache License, Version 2.0 (the "License");
 # you may not use this file except in compliance with the License.

video-colorization/tcvc/data/tcvc_dataset.py

@@ -1,3 +1,17 @@
+# Copyright 2021 Sony Group Corporation
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
 import re
 from pathlib import Path
 import glob

video-colorization/tcvc/generate.py

@@ -1,3 +1,17 @@
+# Copyright 2021 Sony Group Corporation
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
 import os
 import argparse
 import numpy as np

video-colorization/tcvc/models.py

@@ -1,3 +1,17 @@
+# Copyright 2021 Sony Group Corporation
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
 import nnabla as nn
 import nnabla.functions as F
 import nnabla.parametric_functions as PF

video-colorization/tcvc/neu_utils/__init__.py

@@ -1,4 +1,4 @@
-# Copyright (c) 2019 Sony Corporation. All Rights Reserved.
+# Copyright 2021 Sony Group Corporation
 #
 # Licensed under the Apache License, Version 2.0 (the "License");
 # you may not use this file except in compliance with the License.