Detection Model TextNet (TF+PT) #1292

nikokks · 2023-08-29T22:37:00Z

make test-torch
USE_TORCH='1' coverage run -m pytest tests/pytorch/
=========================================================================================== test session starts ===========================================================================================
platform linux -- Python 3.8.10, pytest-5.4.1, py-1.11.0, pluggy-0.13.1
rootdir: /home/nikkokks/Desktop/github/doctr_FAST
plugins: hydra-core-1.2.0, typeguard-2.13.3
collected 216 items                                                                                                                                                                                       

tests/pytorch/test_datasets_pt.py ...................................................                                                                                                               [ 23%]
tests/pytorch/test_file_utils_pt.py .                                                                                                                                                               [ 24%]
tests/pytorch/test_io_image_pt.py ...                                                                                                                                                               [ 25%]
tests/pytorch/test_models_classification_pt.py ................................                                                                                                                     [ 40%]
tests/pytorch/test_models_detection_pt.py ........................                                                                                                                                  [ 51%]
tests/pytorch/test_models_factory.py ..........................                                                                                                                                     [ 63%]
tests/pytorch/test_models_obj_detection_pt.py ..                                                                                                                                                    [ 64%]
tests/pytorch/test_models_preprocessor_pt.py ..........                                                                                                                                             [ 68%]
tests/pytorch/test_models_recognition_pt.py ................................ss..                                                                                                                    [ 85%]
tests/pytorch/test_models_utils_pt.py ....                                                                                                                                                          [ 87%]
tests/pytorch/test_models_zoo_pt.py .........                                                                                                                                                       [ 91%]
tests/pytorch/test_transforms_pt.py ..................                                                                                                                                              [100%]

============================================================================================ warnings summary =============================================================================================
tests/pytorch/test_models_classification_pt.py::test_models_onnx_export[vit_b-input_shape10-output_size10]
tests/pytorch/test_models_recognition_pt.py::test_models_onnx_export[vitstr_small-input_shape5]
tests/pytorch/test_models_recognition_pt.py::test_models_onnx_export[parseq-input_shape6]
  /home/nikkokks/Desktop/github/doctr_FAST/doctr/models/modules/vision_transformer/pytorch.py:71: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
    assert H % self.patch_size[0] == 0, "Image height must be divisible by patch height"

tests/pytorch/test_models_classification_pt.py::test_models_onnx_export[vit_b-input_shape10-output_size10]
tests/pytorch/test_models_recognition_pt.py::test_models_onnx_export[vitstr_small-input_shape5]
tests/pytorch/test_models_recognition_pt.py::test_models_onnx_export[parseq-input_shape6]
  /home/nikkokks/Desktop/github/doctr_FAST/doctr/models/modules/vision_transformer/pytorch.py:72: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
    assert W % self.patch_size[1] == 0, "Image width must be divisible by patch width"

tests/pytorch/test_models_classification_pt.py::test_models_onnx_export[vit_b-input_shape10-output_size10]
  /home/nikkokks/Desktop/github/doctr_FAST/doctr/models/modules/vision_transformer/pytorch.py:44: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
    if num_patches == num_positions and height == width:

tests/pytorch/test_models_classification_pt.py::test_models_onnx_export[vit_b-input_shape10-output_size10]
tests/pytorch/test_models_recognition_pt.py::test_models_onnx_export[vitstr_small-input_shape5]
tests/pytorch/test_models_recognition_pt.py::test_models_onnx_export[parseq-input_shape6]
  /home/nikkokks/Desktop/github/doctr_FAST/doctr/models/modules/transformer/pytorch.py:49: TracerWarning: Converting a tensor to a Python float might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
    scores = torch.matmul(query, key.transpose(-2, -1)) / math.sqrt(query.size(-1))

tests/pytorch/test_models_detection_pt.py: 12 tests with warnings
  /home/nikkokks/Desktop/github/doctr_FAST/doctr/models/detection/differentiable_binarization/base.py:197: RuntimeWarning: invalid value encountered in sqrt
    result = np.sqrt(square_dist_1 * square_dist_2 * square_sin / square_dist)

tests/pytorch/test_models_preprocessor_pt.py: 5 tests with warnings
tests/pytorch/test_models_recognition_pt.py: 8 tests with warnings
  /home/nikkokks/.local/lib/python3.8/site-packages/torchvision/transforms/functional.py:1603: UserWarning: The default value of the antialias parameter of all the resizing transforms (Resize(), RandomResizedCrop(), etc.) will change from None to True in v0.17, in order to be consistent across the PIL and Tensor backends. To suppress this warning, directly pass antialias=True (recommended, future default), antialias=None (current default, which means False for Tensors and True for PIL), or antialias=False (only works on Tensors - PIL will still use antialiasing). This also applies if you are using the inference transforms from the models weights: update the call to weights.transforms(antialias=True).
    warnings.warn(

tests/pytorch/test_models_recognition_pt.py::test_models_onnx_export[crnn_vgg16_bn-input_shape0]
tests/pytorch/test_models_recognition_pt.py::test_models_onnx_export[crnn_mobilenet_v3_small-input_shape1]
tests/pytorch/test_models_recognition_pt.py::test_models_onnx_export[crnn_mobilenet_v3_large-input_shape2]
  /home/nikkokks/.local/lib/python3.8/site-packages/torch/onnx/symbolic_opset9.py:4476: UserWarning: Exporting a model to ONNX with a batch_size other than 1, with a variable length with LSTM can cause an error when running the ONNX model with a different batch size. Make sure to save the model with a batch size of 1, or define the initial states (h0/c0) as inputs of the model. 
    warnings.warn(

tests/pytorch/test_models_recognition_pt.py::test_models_onnx_export[parseq-input_shape6]
  /home/nikkokks/Desktop/github/doctr_FAST/doctr/models/recognition/parseq/pytorch.py:283: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
    if max_len is None and (ys == self.vocab_size).any(dim=-1).all():

tests/pytorch/test_models_recognition_pt.py::test_models_onnx_export[parseq-input_shape6]
  /home/nikkokks/.local/lib/python3.8/site-packages/torch/onnx/symbolic_opset9.py:3825: DeprecationWarning: an integer is required (got type float).  Implicit conversion to integers using __int__ is deprecated, and may be removed in a future version of Python.
    value_t=torch.tensor([fill_value], dtype=scalar_type.dtype()),

-- Docs: https://docs.pytest.org/en/latest/warnings.html
========================================================================= 214 passed, 2 skipped, 40 warnings in 533.02s (0:08:53) =========================================================================

 make test-tf
USE_TF='1' coverage run -m pytest tests/tensorflow/
=========================================================================================== test session starts ===========================================================================================
platform linux -- Python 3.8.10, pytest-5.4.1, py-1.11.0, pluggy-0.13.1
rootdir: /home/nikkokks/Desktop/github/doctr_FAST
plugins: hydra-core-1.2.0, typeguard-2.13.3
collected 222 items                                                                                                                                                                                       

tests/tensorflow/test_datasets_loader_tf.py .                                                                                                                                                       [  0%]
tests/tensorflow/test_datasets_tf.py ...................................................                                                                                                            [ 23%]
tests/tensorflow/test_file_utils_tf.py .                                                                                                                                                            [ 23%]
tests/tensorflow/test_io_image_tf.py ...                                                                                                                                                            [ 25%]
tests/tensorflow/test_models_classification_tf.py ......................ssssss...                                                                                                                   [ 39%]
tests/tensorflow/test_models_detection_tf.py ..................sss                                                                                                                                  [ 48%]
tests/tensorflow/test_models_factory.py ......................                                                                                                                                      [ 58%]
tests/tensorflow/test_models_preprocessor_tf.py ........                                                                                                                                            [ 62%]
tests/tensorflow/test_models_recognition_tf.py .....................................sss                                                                                                             [ 80%]
tests/tensorflow/test_models_utils_tf.py ....                                                                                                                                                       [ 81%]
tests/tensorflow/test_models_zoo_tf.py .........                                                                                                                                                    [ 86%]
tests/tensorflow/test_transforms_tf.py ...............................                                                                                                                              [100%]

============================================================================================ warnings summary =============================================================================================
tests/tensorflow/test_models_zoo_tf.py::test_ocrpredictor[False-False]
tests/tensorflow/test_models_zoo_tf.py::test_kiepredictor[False-False]
  /home/nikkokks/Desktop/github/doctr_FAST/doctr/utils/geometry.py:286: RuntimeWarning: divide by zero encountered in true_divide
    return float(np.median(np.arctan((yleft - yright) / (xright - xleft))) * 180 / np.pi)  # Y axis from top to bottom!

-- Docs: https://docs.pytest.org/en/latest/warnings.html
======================================================================== 210 passed, 12 skipped, 2 warnings in 1302.87s (0:21:42) =========================================================================

… test-tf

felixdittrich92

Hi @nikokks 👋,

Thanks a lot for starting with this looks pretty good for a first draft 👍

I left some comments :)

BTW: To fix the style issues you can run mmake style and to check the mypy issues make quality to fix them afterwards manually

Documentation: you can add the models in:
https://github.com/mindee/doctr/blob/main/docs/source/modules/models.rst

doctr/models/classification/__init__.py

doctr/models/classification/textnetFast/pytorch.py

doctr/models/classification/textnetFast/tensorflow.py

doctr/models/modules/layers/pytorch.py

doctr/models/modules/layers/tensorflow.py

doctr/models/utils/tensorflow.py

tests/tensorflow/test_models_classification_tf.py

nikokks · 2023-08-30T11:09:31Z

make style
isort .
Skipped 44 files
black .
reformatted /home/nikkokks/Desktop/github/doctr_FAST/doctr/models/classification/textnet_fast/tensorflow.py

All done! ✨ 🍰 ✨
1 file reformatted, 251 files left unchanged.
ruff --fix .

make quality
isort . -c
Skipped 44 files
ruff check .
black --check .
All done! ✨ 🍰 ✨
252 files would be left unchanged.
mypy doctr/
doctr/models/artefacts/face.py:32: error: Unused "type: ignore" comment
                cv2.data.haarcascades + "haarcascade_frontalface_default.xml"  # type: ignore[attr-defined]
                ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
doctr/models/modules/layers/pytorch.py:39: error: Incompatible types in assignment (expression has type "None", variable has type "Conv2d")  [assignment]
                self.ver_conv, self.ver_bn = None, None
                                             ^~~~
doctr/models/modules/layers/pytorch.py:39: error: Incompatible types in assignment (expression has type "None", variable has type "BatchNorm2d")  [assignment]
                self.ver_conv, self.ver_bn = None, None
                                                   ^~~~
doctr/models/modules/layers/pytorch.py:54: error: Incompatible types in assignment (expression has type "None", variable has type "Conv2d")  [assignment]
                self.hor_conv, self.hor_bn = None, None
                                             ^~~~
doctr/models/modules/layers/pytorch.py:54: error: Incompatible types in assignment (expression has type "None", variable has type "BatchNorm2d")  [assignment]
                self.hor_conv, self.hor_bn = None, None
                                                   ^~~~
Found 5 errors in 2 files (checked 161 source files)
make: *** [Makefile:7: quality] Error 1

doctr/models/classification/textnet_fast/pytorch.py

…ng test method to deploy model

felixT2K · 2023-09-05T12:05:55Z

@nikokks Could you please remove all the TF parts and let the CI run onces ? :)

There are some changes required we need to do.
Unfortunately i don't have the time to review it careful this week.
So i would suggest you continue with the neck + head and post processing.

nikokks · 2023-09-08T16:18:46Z

make style
isort .
Skipped 45 files
black .
Skipping .ipynb files as Jupyter dependencies are not installed.
You can fix this by running ``pip install "black[jupyter]"``
All done! ✨ 🍰 ✨
253 files left unchanged.
ruff --fix .

make quality
isort . -c
Skipped 45 files
ruff check .
black --check .
Skipping .ipynb files as Jupyter dependencies are not installed.
You can fix this by running ``pip install "black[jupyter]"``
All done! ✨ 🍰 ✨
253 files would be left unchanged.
mypy doctr/
Success: no issues found in 161 source files
pydocstyle doctr/

make test-tf
USE_TF='1' coverage run -m pytest tests/tensorflow/
=========================================================================================== test session starts ===========================================================================================
platform linux -- Python 3.8.10, pytest-5.4.1, py-1.11.0, pluggy-0.13.1
rootdir: /home/nikkokks/Desktop/github/doctr_FAST
plugins: hydra-core-1.2.0, typeguard-2.13.3
collected 222 items                                                                                                                                                                                       

tests/tensorflow/test_datasets_loader_tf.py .                                                                                                      [  0%]
tests/tensorflow/test_datasets_tf.py ...................................................                                                           [ 23%]
tests/tensorflow/test_file_utils_tf.py .                                                                                                           [ 23%]
tests/tensorflow/test_io_image_tf.py ...                                                                                                           [ 25%]
tests/tensorflow/test_models_classification_tf.py ......................ssssss...                                                                  [ 39%]
tests/tensorflow/test_models_detection_tf.py ..................sss                                                                                 [ 48%]
tests/tensorflow/test_models_factory.py ......................                                                                                     [ 58%]
tests/tensorflow/test_models_preprocessor_tf.py ........                                                                                           [ 62%]
tests/tensorflow/test_models_recognition_tf.py .....................................sss                                                            [ 80%]
tests/tensorflow/test_transforms_tf.py ...............................                                                                             [100%]

============================================================================================ warnings summary =============================================================================================
tests/tensorflow/test_models_zoo_tf.py::test_ocrpredictor[False-False]
tests/tensorflow/test_models_zoo_tf.py::test_kiepredictor[False-False]
  /home/nikkokks/Desktop/github/doctr_FAST/doctr/utils/geometry.py:286: RuntimeWarning: divide by zero encountered in true_divide
    return float(np.median(np.arctan((yleft - yright) / (xright - xleft))) * 180 / np.pi)  # Y axis from top to bottom!

-- Docs: https://docs.pytest.org/en/latest/warnings.html
========================================================================= 210 passed, 12 skipped, 2 warnings in 907.17s (0:15:07)
==========================================================================

make test-torch
USE_TORCH='1' coverage run -m pytest tests/pytorch/
=========================================================================================== test session starts ===========================================================================================
platform linux -- Python 3.8.10, pytest-5.4.1, py-1.11.0, pluggy-0.13.1
rootdir: /home/nikkokks/Desktop/github/doctr_FAST
plugins: hydra-core-1.2.0, typeguard-2.13.3
collected 216 items                                                                                                                                                                                       

tests/pytorch/test_datasets_pt.py ...................................................                                                                   [ 23%]
tests/pytorch/test_file_utils_pt.py .                                                                                                                   [ 24%]
tests/pytorch/test_io_image_pt.py ...                                                                                                                   [ 25%]
tests/pytorch/test_models_classification_pt.py ................................                                                                         [ 40%]
tests/pytorch/test_models_detection_pt.py ........................                                                                                      [ 51%]
tests/pytorch/test_models_factory.py ..........................                                                                                         [ 63%]
tests/pytorch/test_models_obj_detection_pt.py ..                                                                                                        [ 64%]
tests/pytorch/test_models_preprocessor_pt.py ..........                                                                                                 [ 68%]
tests/pytorch/test_models_recognition_pt.py ................................ss..                                                                        [ 85%]
tests/pytorch/test_models_utils_pt.py ....                                                                                                              [ 87%]
tests/pytorch/test_models_zoo_pt.py .........                                                                                                           [ 91%]
tests/pytorch/test_transforms_pt.py ..................                                                                                                  [100%]

======================================================================== 214 passed, 2 skipped, 268 warnings in 234.90s (0:03:54) =========================================================================

nikokks · 2023-09-08T16:20:30Z

I still have a problem to switch textNetFast tensorflow model to eval, I think the problem is due to a problem of init weights.
Can someone have a look at it plz ?

nikokks · 2023-09-08T16:21:37Z

I think maybe there is a second problem initializing block of layers in TF.

…fast model (not working for the moment)

…lity tests

…ot working (forward,compute_loss,postprocessor))

… implementation of fast torche model

felixdittrich92 · 2023-09-15T05:19:11Z

As discussed closing this up to december :)

nikokks added 10 commits August 29, 2023 10:16

adding TextNet backbone : first try

03c5834

second try: uploading TextNet backbone

ceeca80

correcting some syntax for textnet backbone

151e65f

changing import layer for textnet

57d33f8

renaming textnet to textnetfast

dae10b2

adding textnetfast test and ran it good

2465696

correcting some stuff to run TextNetFast : not working for the moment

ab5b5b3

run test_classification_pytorch OK

564f9eb

first commit of textnetfast model in tensorflow

7f48b06

ending implementing tensorflow textnet classification model => go for…

346bcd4

… test-tf

felixdittrich92 self-assigned this Aug 30, 2023

felixdittrich92 added module: models Related to doctr.models framework: pytorch Related to PyTorch backend framework: tensorflow Related to TensorFlow backend topic: character classification Related to the task of character classification type: new feature New feature labels Aug 30, 2023

felixdittrich92 added this to the 0.7.0 milestone Aug 30, 2023

felixdittrich92 requested changes Aug 30, 2023

View reviewed changes

some changes + make style + make quality

eb9299e

nikokks added 6 commits August 30, 2023 13:12

some changes

fd3d85b

some changes

10e7e05

some changes3

d49c3f2

[skip ci] some changes 4

6599951

[skip ci] some changes 5

491ddd5

[skip ci] some changes 5

2f2e769

felixdittrich92 marked this pull request as draft August 30, 2023 16:25

felixT2K reviewed Aug 31, 2023

View reviewed changes

doctr/models/classification/textnet_fast/pytorch.py Show resolved Hide resolved

nikokks added 2 commits August 31, 2023 10:16

[skip ci] removing ingore_keys in tf textnet model

f315122

[skip ci] removing ingore_keys in tf textnet model

ad1bc73

nikokks added 8 commits September 1, 2023 20:24

[skip ci] adding fast inference fo textnet model

064ca6b

[skip ci] some changes 6

53c144b

[skip ci] override eval method for speed up eval mode of TextNetFast

774138d

[skip ci] override eval and train mode for textNetFast done

a7e8d73

[skip ci] make style and make quality Done

ae5f7c4

[skip ci] changing eval and train method to switch repconvlayer

7c0bba3

[skip ci] changing eval and train method to switch repconvlayer setti…

6814718

…ng test method to deploy model

[skip ci] TextNetFast pytorch model done

eac952e

TextNetFast model implemented in torch

0ef122f

felixdittrich92 modified the milestones: 0.7.0, 0.7.1 Sep 8, 2023

adding textNetFast tensorflow implementation

dcd2ece

nikokks added 12 commits September 9, 2023 14:51

starting to solving eval mode of textnetFadt model

a8ac914

[skip ci] Last modification for switch train to eval mode for textnet…

48fb8e5

…fast model (not working for the moment)

[skip ci] dleting tensorflow textnetfast code for futher integreation

77d78b6

[skip ci] first commit on adding neck,head and fast model in torch

d7b9ca2

[skip ci] backbone+neck+head of FAst torch ready, remains fast class

2e1ea2f

[skip ci] correcting some stuff for Fast Torch Model on style and qua…

e83b2ac

…lity tests

[skip ci] correcting some stuff for convlayer

e3aa43f

[skip ci] correcting some stuff for Fast Torch Model on style and qua…

cd35227

…lity tests

[skip ci] implementation of Fast torch model update (working(init), n…

6a5b5a6

…ot working (forward,compute_loss,postprocessor))

[skip ci] update losses for Fast Torch model

1bc5ba4

[skip ci] forward and compute_loss seems to be ok, need postprocessor…

018a3c9

… implementation of fast torche model

[skip ci] advancements in Fast torch model forward method

5de4fff

felixdittrich92 closed this Sep 15, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Detection Model TextNet (TF+PT) #1292

Detection Model TextNet (TF+PT) #1292

nikokks commented Aug 29, 2023 •

edited

Loading

felixdittrich92 left a comment

nikokks commented Aug 30, 2023

felixT2K commented Sep 5, 2023

nikokks commented Sep 8, 2023

nikokks commented Sep 8, 2023 •

edited

Loading

nikokks commented Sep 8, 2023

felixdittrich92 commented Sep 15, 2023

Detection Model TextNet (TF+PT) #1292

Detection Model TextNet (TF+PT) #1292

Conversation

nikokks commented Aug 29, 2023 • edited Loading

felixdittrich92 left a comment

Choose a reason for hiding this comment

nikokks commented Aug 30, 2023

felixT2K commented Sep 5, 2023

nikokks commented Sep 8, 2023

nikokks commented Sep 8, 2023 • edited Loading

nikokks commented Sep 8, 2023

felixdittrich92 commented Sep 15, 2023

nikokks commented Aug 29, 2023 •

edited

Loading

nikokks commented Sep 8, 2023 •

edited

Loading