Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Integration inference #39

Open
wants to merge 229 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
229 commits
Select commit Hold shift + click to select a range
ba9076f
updated readme
Vishal-S-P Apr 6, 2023
bca22af
updated readme
Vishal-S-P Apr 6, 2023
3b47bb2
The dataloader output None
Vishal-S-P Apr 7, 2023
d30d4c4
testing modeling folder
Vishal-S-P Apr 8, 2023
5062ec8
maskformer arch test successful.
Vishal-S-P Apr 8, 2023
c3aba68
started debugging loss function
Vishal-S-P Apr 10, 2023
3f6e030
linear sum assignment does not work for rectangular cost matrix
Vishal-S-P Apr 10, 2023
79cf603
fixing cross entropy loss calcualtion
Vishal-S-P Apr 15, 2023
2915e03
Made cost matrix square and compatible with matcher
Vishal-S-P Apr 15, 2023
266592f
verified the scipy and tf version of matcher gives same indices as ou…
Vishal-S-P Apr 16, 2023
77f7d62
updated NestedTensor
Vishal-S-P Apr 16, 2023
8dbc212
latest
Vishal-S-P Apr 16, 2023
59b066a
fixed large CE loss issue dice and focal loss do not match with Pytor…
Vishal-S-P Apr 18, 2023
2ea7cda
fixed th e issue with the assignments between pytorch and tensorflow …
Vishal-S-P Apr 19, 2023
f8ce6c6
focal loss matches with the pytorch code
Vishal-S-P Apr 19, 2023
8e5d782
fixed all losses working on aux losses
Vishal-S-P Apr 21, 2023
cc7bcde
dataloader not working as expected
Vishal-S-P Apr 25, 2023
bfefafe
working on dataloader
Vishal-S-P Apr 29, 2023
30cd1a4
dataloader working issue
Vishal-S-P May 4, 2023
42e34d6
binary masks done
Vishal-S-P May 5, 2023
365b400
almost close to training
Vishal-S-P May 6, 2023
f67ee4f
first version of the working training code
Vishal-S-P May 7, 2023
7774725
new branch
Vishal-S-P May 17, 2023
52e675b
update
Vishal-S-P May 17, 2023
88f0fd2
update for TPU
Vishal-S-P May 19, 2023
180b0fd
updated path
Vishal-S-P May 19, 2023
94097ec
updated shapes
Vishal-S-P May 19, 2023
563ca07
upated loss loop
Vishal-S-P May 19, 2023
2594f76
updated code
Vishal-S-P May 19, 2023
540a6fe
tpu update
Vishal-S-P May 19, 2023
21e92d3
updated for tpu
Vishal-S-P May 19, 2023
c3bd482
updated for tpu
Vishal-S-P May 19, 2023
d2e9a56
no loss loop
Vishal-S-P May 19, 2023
5d6495d
update
Vishal-S-P May 19, 2023
82c72d9
update
Vishal-S-P May 19, 2023
f7d398a
update
Vishal-S-P May 19, 2023
5070106
padded
Vishal-S-P May 19, 2023
2862fc1
update
Vishal-S-P May 19, 2023
7f693e3
updated with padding
Vishal-S-P May 19, 2023
0baa601
tpu update
Vishal-S-P May 19, 2023
63cf9a8
added zone
Vishal-S-P May 19, 2023
734b9a2
update
Vishal-S-P May 19, 2023
a8c8a04
update
Vishal-S-P May 19, 2023
3bc4696
added tpu test file
Vishal-S-P May 19, 2023
c24a3c4
tpu location
Vishal-S-P May 19, 2023
0499089
updated
Vishal-S-P May 19, 2023
5de4f9f
update :
Vishal-S-P May 19, 2023
88116ab
working tpu test
Vishal-S-P May 19, 2023
e17026e
updated zone and project id for TPU
Vishal-S-P May 19, 2023
8589783
added bash file for training
Vishal-S-P May 19, 2023
59b27db
updated bash
Vishal-S-P May 19, 2023
0e863ad
updated bash
Vishal-S-P May 19, 2023
31a6720
pdtaed bash
Vishal-S-P May 19, 2023
ce072d2
update
Vishal-S-P May 19, 2023
92febc0
update
Vishal-S-P May 19, 2023
4df5686
TPU fix
Vishal-S-P May 20, 2023
52a9f2a
updated
Vishal-S-P May 20, 2023
ae9ffc1
update
Vishal-S-P May 20, 2023
be6465a
update
Vishal-S-P May 20, 2023
e0e5a9b
dataset update
Vishal-S-P May 20, 2023
87cd6f0
dataset update
Vishal-S-P May 20, 2023
08ace54
dataset update
Vishal-S-P May 20, 2023
e2c08dc
update
Vishal-S-P May 20, 2023
3bfc639
update
Vishal-S-P May 20, 2023
4c44097
update
Vishal-S-P May 20, 2023
ad05cd6
update
Vishal-S-P May 20, 2023
105fdea
update
Vishal-S-P May 20, 2023
2ef491a
update
Vishal-S-P May 20, 2023
c81f81c
update
Vishal-S-P May 20, 2023
9a6c25c
update
Vishal-S-P May 22, 2023
39eb6ac
update
Vishal-S-P May 22, 2023
d8dba05
update
Vishal-S-P May 22, 2023
63c3564
updated code
Vishal-S-P May 22, 2023
73b647d
update
Vishal-S-P May 22, 2023
7f919f3
update
Vishal-S-P May 22, 2023
aa779a6
update
Vishal-S-P May 27, 2023
40930a9
update
Vishal-S-P May 27, 2023
ceb89cf
update
Vishal-S-P May 27, 2023
346f2b3
Set tensorarray dynamic to False
Wenxin-Jiang May 27, 2023
4b0b092
Change the tensorarray size
Wenxin-Jiang May 27, 2023
121db6b
Update panoptic_input.py
Wenxin-Jiang May 27, 2023
39b169b
Remove padding in loss
Wenxin-Jiang May 27, 2023
f6f424a
Change 101 to 100
Wenxin-Jiang May 27, 2023
3cfe8ea
Update panoptic_input.py
Wenxin-Jiang May 27, 2023
5c28744
Update panoptic_input.py
Wenxin-Jiang May 27, 2023
6b9686f
Remove the optimizer config for now
Wenxin-Jiang May 27, 2023
773bdca
Use yolo optimizer config
Wenxin-Jiang May 27, 2023
44821c1
Fix error
Wenxin-Jiang May 27, 2023
7002031
Use yolo optimization
Wenxin-Jiang May 27, 2023
151d337
updated tensorarray
Vishal-S-P May 28, 2023
b9cf71f
updated tensorarray in losses
Vishal-S-P May 28, 2023
f15aaec
added -1 to the loop
Vishal-S-P May 28, 2023
1ea1a27
updated ignore label
Vishal-S-P May 28, 2023
092694c
cheking the input shapes
Vishal-S-P May 28, 2023
751b7c6
cheking the input shapes
Vishal-S-P May 28, 2023
76afe1f
updated the tensorarray loop
Vishal-S-P May 28, 2023
0500905
removed dynamic size from all tensor arrays
Vishal-S-P May 28, 2023
7526ae0
tf.range --> range
Vishal-S-P May 28, 2023
5bd6c9f
updated
Vishal-S-P May 28, 2023
ccd10da
updated the for loops
Vishal-S-P May 30, 2023
69f5d37
fixed bug with for loop
Vishal-S-P May 30, 2023
472935d
testing
Vishal-S-P May 31, 2023
f0909ba
update
Vishal-S-P May 31, 2023
388b0cb
debug
Vishal-S-P May 31, 2023
d0565bd
update
Vishal-S-P May 31, 2023
04d423a
debug
Vishal-S-P May 31, 2023
916baf7
update
Vishal-S-P May 31, 2023
9c72f39
update
Vishal-S-P May 31, 2023
cd9e6d4
update
Vishal-S-P May 31, 2023
61b2de5
update
Vishal-S-P May 31, 2023
0280b06
update
Vishal-S-P May 31, 2023
c4b3cf7
int64 not supported on tpu for labels
Vishal-S-P May 31, 2023
45b5f18
bfloat16 to float32
Vishal-S-P May 31, 2023
e41f19b
update
Vishal-S-P May 31, 2023
52bb2b7
updated
Vishal-S-P May 31, 2023
aeb6930
updated tpu
Vishal-S-P May 31, 2023
8530bee
updated tpu location
Vishal-S-P May 31, 2023
d1a274e
update
Vishal-S-P May 31, 2023
c40bf45
update
Vishal-S-P May 31, 2023
26047ae
update
Vishal-S-P May 31, 2023
aff1523
debugging for larger batch size
Vishal-S-P May 31, 2023
1fd42c2
updated
Vishal-S-P May 31, 2023
455da2d
updated proba massk
Vishal-S-P May 31, 2023
faaf9ae
vectorized for loop
Vishal-S-P Jun 1, 2023
0d9dea8
update
Vishal-S-P Jun 1, 2023
bd3ce56
update
Vishal-S-P Jun 1, 2023
468c374
update
Vishal-S-P Jun 1, 2023
10eeb82
update
Vishal-S-P Jun 1, 2023
efe4fee
update
Vishal-S-P Jun 1, 2023
cd134e5
soft device placement
Vishal-S-P Jun 1, 2023
5564cd6
updated resize
Vishal-S-P Jun 1, 2023
9a04abd
update
Vishal-S-P Jun 1, 2023
a3fe2c2
updtae
Vishal-S-P Jun 1, 2023
d297ebe
update
Vishal-S-P Jun 1, 2023
3e0cbd6
update
Vishal-S-P Jun 1, 2023
dd51bee
squeezed dimension
Vishal-S-P Jun 1, 2023
8db790e
update
Vishal-S-P Jun 1, 2023
284a6c6
updated matcher loop
Vishal-S-P Jun 1, 2023
8d72d9d
update
Vishal-S-P Jun 1, 2023
80cd394
update
Vishal-S-P Jun 1, 2023
9d07b95
update
Vishal-S-P Jun 1, 2023
d6f52c5
update
Vishal-S-P Jun 1, 2023
8345301
update
Vishal-S-P Jun 1, 2023
1812d8e
update
Vishal-S-P Jun 1, 2023
1f5d766
update
Vishal-S-P Jun 1, 2023
1972272
update
Vishal-S-P Jun 1, 2023
57f2f8f
update
Vishal-S-P Jun 1, 2023
4f8b9f0
update
Vishal-S-P Jun 1, 2023
ad607aa
updtae
Vishal-S-P Jun 1, 2023
9c1cbec
updtae
Vishal-S-P Jun 1, 2023
3e4bc4a
updtae
Vishal-S-P Jun 1, 2023
cb3b96e
updtae
Vishal-S-P Jun 1, 2023
5a7f19c
update
Vishal-S-P Jun 1, 2023
889ab4a
update
Vishal-S-P Jun 1, 2023
8631fbd
update
Vishal-S-P Jun 1, 2023
de54053
update
Vishal-S-P Jun 1, 2023
aa893c7
update
Vishal-S-P Jun 1, 2023
42ce9fb
update
Vishal-S-P Jun 1, 2023
695f87c
update
Vishal-S-P Jun 1, 2023
fd4a9bd
update
Vishal-S-P Jun 1, 2023
e6147ff
update
Vishal-S-P Jun 1, 2023
1383e99
update
Vishal-S-P Jun 1, 2023
9fb0ca8
update
Vishal-S-P Jun 1, 2023
225988f
update
Vishal-S-P Jun 1, 2023
cb06086
update
Vishal-S-P Jun 1, 2023
64c83bd
update
Vishal-S-P Jun 1, 2023
cbcf0aa
update
Vishal-S-P Jun 1, 2023
a61d2c3
update
Vishal-S-P Jun 1, 2023
a478e53
update
Vishal-S-P Jun 1, 2023
3b3f82a
removed data cache
Vishal-S-P Jun 1, 2023
3ea047c
updated tpu zone
Vishal-S-P Jun 1, 2023
92c32d6
updated tpu zone for tf-train-256
Vishal-S-P Jun 1, 2023
528f87c
updated batch size
Vishal-S-P Jun 1, 2023
27d585d
updated batch size
Vishal-S-P Jun 1, 2023
32950c2
update
Vishal-S-P Jun 1, 2023
87019b6
updated the individual mask loop and removed hashtable which was not …
Vishal-S-P Jun 6, 2023
a24d2b5
added code profiling
Vishal-S-P Jun 6, 2023
9b3eb0f
removed tf.unique
Vishal-S-P Jun 6, 2023
55b90e8
updated
Vishal-S-P Jun 17, 2023
5fa8610
updated
Vishal-S-P Jun 17, 2023
8c3ec79
updated
Vishal-S-P Jun 17, 2023
b7697d0
updated
Vishal-S-P Jun 17, 2023
e653403
updated
Vishal-S-P Jun 17, 2023
8be91de
updated
Vishal-S-P Jun 17, 2023
8ef0e84
updated
Vishal-S-P Jun 17, 2023
f247c9c
updated
Vishal-S-P Jun 17, 2023
37c33bb
updated
Vishal-S-P Jun 17, 2023
398ed76
updated
Vishal-S-P Jun 17, 2023
2c2acb5
updated
Vishal-S-P Jun 17, 2023
df61eb3
updated
Vishal-S-P Jun 17, 2023
334baf3
updated
Vishal-S-P Jun 17, 2023
b15b454
updated
Vishal-S-P Jun 17, 2023
2af9550
update
Vishal-S-P Jun 17, 2023
22c35a1
update
Vishal-S-P Jun 17, 2023
56f5aa6
update
Vishal-S-P Jun 17, 2023
63de744
updated
Vishal-S-P Jun 18, 2023
f94ecf8
updated
Vishal-S-P Jun 18, 2023
fc389e1
updated
Vishal-S-P Jun 18, 2023
dfc1f99
working tpu v2-256 europe code
Vishal-S-P Jun 18, 2023
b6ac86b
working tpu v2-256 europe code
Vishal-S-P Jun 18, 2023
12430d2
working tpu v2-256 europe code
Vishal-S-P Jun 18, 2023
6ce00d6
working tpu v2-256 US code
Vishal-S-P Jun 18, 2023
cc4e496
working tpu v2-256 US code
Vishal-S-P Jun 18, 2023
f182bfa
working tpu v2-256 US code
Vishal-S-P Jun 18, 2023
cf0a8ff
working tpu v2-256 US code
Vishal-S-P Jun 18, 2023
03c9272
working tpu v2-256 US code
Vishal-S-P Jun 18, 2023
3352782
added clip or pad to fixed size
Vishal-S-P Jun 18, 2023
2c4448a
updated batch size to 64, number of steps per epoch to 100
Vishal-S-P Jun 18, 2023
0bd746c
update
Vishal-S-P Jun 18, 2023
a30d2c2
undo the batch size to 8
Vishal-S-P Jun 18, 2023
4ef5286
added a line for prifiler
Vishal-S-P Jun 19, 2023
fa7fded
for tf-debug-1
Vishal-S-P Jun 19, 2023
38d3293
for tf-debug-1
Vishal-S-P Jun 19, 2023
2d1e4c8
for tf-debug-1
Vishal-S-P Jun 19, 2023
c3bb923
code failed to run on tf-train-1
Vishal-S-P Jun 19, 2023
1554325
eu tpu v3-8
Vishal-S-P Jun 19, 2023
ab1a956
added data cache and added profiler code
Vishal-S-P Jun 19, 2023
c7462d6
removed profile server and added profile capture programatically
Vishal-S-P Jun 19, 2023
ddf957a
updated code to run on faster VM
Vishal-S-P Jun 19, 2023
a801143
updated code to run on faster VM
Vishal-S-P Jun 19, 2023
e7a8c5b
updated code to run on faster VM
Vishal-S-P Jun 19, 2023
be65563
updated code to run on faster VM
Vishal-S-P Jun 19, 2023
3a8bd0c
added bfloat16 to the parser which used float32 before
Vishal-S-P Jun 20, 2023
fed7003
updated batch size to 64 and made dtype bfloat16
Vishal-S-P Jun 20, 2023
1d6865a
updated batch size to 8 and made dtype bfloat16
Vishal-S-P Jun 20, 2023
1e1d8e7
added mixed precision training bfloat16
Vishal-S-P Jun 20, 2023
75def0e
first changes
InAIBot Jun 27, 2023
3aae0a0
fixing inference and maskformer parameters
InAIBot Jun 27, 2023
d82ff0e
Update maskformer_test.py
InAIBot Jun 27, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
Binary file added category_mask.npy
Binary file not shown.
Binary file added img.npy
Binary file not shown.
Binary file added individual_masks.npy
Binary file not shown.
Binary file added instance_mask.npy
Binary file not shown.
Binary file added models/official/__pycache__/__init__.cpython-38.pyc
Binary file not shown.
Binary file modified models/official/__pycache__/__init__.cpython-39.pyc
Binary file not shown.
Binary file not shown.
Binary file modified models/official/common/__pycache__/__init__.cpython-39.pyc
Binary file not shown.
Binary file modified models/official/common/__pycache__/dataset_fn.cpython-39.pyc
Binary file not shown.
Binary file not shown.
Binary file modified models/official/common/__pycache__/distribute_utils.cpython-39.pyc
Binary file not shown.
3 changes: 2 additions & 1 deletion models/official/common/distribute_utils.py
Original file line number Diff line number Diff line change
Expand Up @@ -82,8 +82,9 @@ def tpu_initialize(tpu_address):
Returns:
A TPUClusterResolver.
"""

cluster_resolver = tf.distribute.cluster_resolver.TPUClusterResolver(
tpu=tpu_address)
tpu=tpu_address, project=os.environ["TPU_PROJECT"], zone=os.environ["TPU_ZONE"])
if tpu_address not in ("", "local"):
tf.config.experimental_connect_to_cluster(cluster_resolver)
tf.tpu.experimental.initialize_tpu_system(cluster_resolver)
Expand Down
Binary file not shown.
Binary file modified models/official/core/__pycache__/__init__.cpython-39.pyc
Binary file not shown.
Binary file not shown.
Binary file modified models/official/core/__pycache__/actions.cpython-39.pyc
Binary file not shown.
Binary file not shown.
Binary file modified models/official/core/__pycache__/base_task.cpython-39.pyc
Binary file not shown.
Binary file not shown.
Binary file modified models/official/core/__pycache__/base_trainer.cpython-39.pyc
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file modified models/official/core/__pycache__/exp_factory.cpython-39.pyc
Binary file not shown.
Binary file modified models/official/core/__pycache__/export_base.cpython-39.pyc
Binary file not shown.
Binary file modified models/official/core/__pycache__/file_writers.cpython-39.pyc
Binary file not shown.
Binary file modified models/official/core/__pycache__/input_reader.cpython-39.pyc
Binary file not shown.
Binary file modified models/official/core/__pycache__/registry.cpython-39.pyc
Binary file not shown.
Binary file not shown.
Binary file modified models/official/core/__pycache__/task_factory.cpython-39.pyc
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file modified models/official/core/__pycache__/train_lib.cpython-39.pyc
Binary file not shown.
Binary file modified models/official/core/__pycache__/train_utils.cpython-39.pyc
Binary file not shown.
22 changes: 11 additions & 11 deletions models/official/core/actions.py
Original file line number Diff line number Diff line change
Expand Up @@ -222,15 +222,15 @@ def get_train_actions(
)
train_actions.append(recover_action)

if (
params.trainer.preemption_on_demand_checkpoint
and trainer.strategy.cluster_resolver
):
on_demand_checkpoint_action = orbit.actions.SaveCheckpointIfPreempted(
trainer.strategy.cluster_resolver,
checkpoint_manager,
trainer.global_step,
keep_running_after_save=True,
)
train_actions.append(on_demand_checkpoint_action)
# if (
# params.trainer.preemption_on_demand_checkpoint
# and trainer.strategy.cluster_resolver
# ):
# on_demand_checkpoint_action = orbit.actions.SaveCheckpointIfPreempted(
# trainer.strategy.cluster_resolver,
# checkpoint_manager,
# trainer.global_step,
# keep_running_after_save=True,
# )
# train_actions.append(on_demand_checkpoint_action)
return train_actions
1 change: 1 addition & 0 deletions models/official/core/base_trainer.py
Original file line number Diff line number Diff line change
Expand Up @@ -335,6 +335,7 @@ def train_loop_end(self):
# Maybe a self-implemented optimizer does not have `optimizer.iterations`.
# So just to be safe here.
if hasattr(self.optimizer, "iterations"):

logs["learning_rate"] = self.optimizer.learning_rate(
self.optimizer.iterations)
else:
Expand Down
Binary file not shown.
Binary file modified models/official/modeling/__pycache__/__init__.cpython-39.pyc
Binary file not shown.
Binary file modified models/official/modeling/__pycache__/performance.cpython-39.pyc
Binary file not shown.
Binary file modified models/official/modeling/__pycache__/tf_utils.cpython-39.pyc
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file modified models/official/modeling/privacy/__pycache__/ops.cpython-39.pyc
Binary file not shown.
Binary file modified models/official/nlp/__pycache__/__init__.cpython-39.pyc
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file modified models/official/projects/__pycache__/__init__.cpython-39.pyc
Binary file not shown.
Binary file not shown.
Binary file not shown.
2 changes: 1 addition & 1 deletion models/official/projects/detr/dataloaders/coco.py
Original file line number Diff line number Diff line change
Expand Up @@ -116,7 +116,7 @@ def preprocess(self, inputs):
classes, self._params.max_num_boxes),
'boxes':
preprocess_ops.clip_or_pad_to_fixed_size(
boxes, self._params.max_num_boxes)
boxes, self._params.max_num_boxes) # [4, 100]
}
if not self._params.is_training:
labels.update({
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,5 +2,6 @@
python3 official/projects/detr/train.py \
--experiment=detr_coco \
--mode=train_and_eval \
--model_dir=/tmp/logging_dir/ \
--model_dir="./" \
--params_override=task.init_checkpoint='gs://tf_model_garden/vision/resnet50_imagenet/ckpt-62400',trainer.train_steps=554400,trainer.optimizer_config.learning_rate.stepwise.boundaries="[369600]"

Binary file not shown.
Binary file not shown.
Binary file not shown.
3 changes: 2 additions & 1 deletion models/official/projects/detr/ops/matchers.py
Original file line number Diff line number Diff line change
Expand Up @@ -27,9 +27,9 @@
Based on the original implementation by Jiquan Ngiam <[email protected]>.
"""
import tensorflow as tf
# from official.modeling import tf_utils
from official.modeling import tf_utils


def _prepare(weights):
"""Prepare the cost matrix.

Expand Down Expand Up @@ -73,6 +73,7 @@ def _greedy_assignment(adj_matrix):
Each row and column can have at most one true element. Some of the rows
and columns may not be matched.
"""

_, num_elems, _ = tf_utils.get_shape_list(adj_matrix, expected_rank=3)
adj_matrix = tf.transpose(adj_matrix, [1, 0, 2])

Expand Down
5 changes: 2 additions & 3 deletions models/official/projects/detr/ops/matchers_test.py
Original file line number Diff line number Diff line change
Expand Up @@ -18,9 +18,8 @@
from scipy import optimize
import tensorflow as tf

from official.projects.detr.ops import matchers


# from official.projects.detr.ops import matchers
import matchers
class MatchersOpsTest(tf.test.TestCase):

def testLinearSumAssignment(self):
Expand Down
Binary file not shown.
4 changes: 2 additions & 2 deletions models/official/projects/detr/tasks/detection.py
Original file line number Diff line number Diff line change
Expand Up @@ -152,7 +152,7 @@ def _compute_cost(self, cls_outputs, box_outputs, cls_targets, box_targets):
tf.cast(tf.not_equal(cls_targets, 0), dtype=total_cost.dtype), axis=1)
total_cost = (1 - valid) * max_cost + valid * total_cost

# Set inf of nan to large constant
# Set inf or nan to large constant
total_cost = tf.where(
tf.logical_or(tf.math.is_nan(total_cost), tf.math.is_inf(total_cost)),
max_cost * tf.ones_like(total_cost, dtype=total_cost.dtype),
Expand All @@ -176,7 +176,7 @@ def build_losses(self, outputs, labels, aux_losses=None):
target_index = tf.math.argmax(indices, axis=1)
cls_assigned = tf.gather(cls_outputs, target_index, batch_dims=1, axis=1)
box_assigned = tf.gather(box_outputs, target_index, batch_dims=1, axis=1)

background = tf.equal(cls_targets, 0)
num_boxes = tf.reduce_sum(
tf.cast(tf.logical_not(background), tf.float32), axis=-1)
Expand Down
4 changes: 4 additions & 0 deletions models/official/projects/maskformer/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
.gitignore
myreadme.md
ckpts/
.npy
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
{
"cells": [],
"metadata": {},
"nbformat": 4,
"nbformat_minor": 4
}
Binary file not shown.
Binary file not shown.
Binary file not shown.
Original file line number Diff line number Diff line change
@@ -0,0 +1,212 @@
# Copyright 2022 The TensorFlow Authors. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

"""MaskFormer configurations."""

import dataclasses
import os
from typing import List, Optional, Union

from official.core import config_definitions as cfg
from official.core import exp_factory
from official.modeling import hyperparams
from official.vision.configs import backbones
from official.vision.configs import common
from official.projects.maskformer import optimization


@dataclasses.dataclass
class Parser(hyperparams.Config):
"""Config definitions for parser"""
output_size: List[int] = None
min_scale: float = 0.3
aspect_ratio_range: List[float] = (0.5, 2.0)
min_overlap_params: List[float] = (0.0, 1.4, 0.2, 0.1)
max_retry: int = 50
pad_output: bool = False
resize_eval_groundtruth: bool = True
groundtruth_padded_size: Optional[List[int]] = None
ignore_label: int = 0
aug_rand_hflip: bool = True
aug_scale_min: float = 1.0
aug_scale_max: float = 1.0
color_aug_ssd: bool = False
brightness: float = 0.2
saturation: float = 0.3
contrast: float = 0.5
aug_type: Optional[common.Augmentation] = None
sigma: float = 8.0
small_instance_area_threshold: int = 4096
small_instance_weight: float = 3.0
dtype: str = 'float32'
seed: int = None

@dataclasses.dataclass
class DataConfig(cfg.DataConfig):
"""Input config for training."""
input_path: str = ''
tfds_name: str = ''
tfds_split: str = 'train'
global_batch_size: int = 0
is_training: bool = False
regenerate_source_id: bool = False
# TODO : Change the dtype to bloat16 for TPU training
dtype: str = 'bfloat16'
decoder: common.DataDecoder = common.DataDecoder()
shuffle_buffer_size: int = 10000
file_type: str = 'tfrecord'
drop_remainder: bool = True
parser: Parser = Parser()


@dataclasses.dataclass
class Losses(hyperparams.Config):
# TODO update these for maskformer
class_offset: int = 0
lambda_cls: float = 1.0
lambda_box: float = 5.0
lambda_giou: float = 2.0
background_cls_weight: float = 0.1
l2_weight_decay: float = 1e-4


@dataclasses.dataclass
class MaskFormer(hyperparams.Config):
# TODO update these for maskformer
"""MaskFormer model definations."""
num_queries: int = 100
hidden_size: int = 256
# TODO: Actually there are 133 classes for panoptic segmentation
num_classes: int = 133 # 0: background
num_encoder_layers: int = 6
num_decoder_layers: int = 6
input_size: List[int] = dataclasses.field(default_factory=list)
backbone: backbones.Backbone = backbones.Backbone(
type='resnet', resnet=backbones.ResNet(model_id=50, bn_trainable=False))
norm_activation: common.NormActivation = common.NormActivation()
backbone_endpoint_name: str = '5'


@dataclasses.dataclass
class MaskFormerTask(cfg.TaskConfig):
model: MaskFormer = MaskFormer()
train_data: cfg.DataConfig = cfg.DataConfig()
validation_data: cfg.DataConfig = cfg.DataConfig()
losses: Losses = Losses()
init_checkpoint: Optional[str] = None
init_checkpoint_modules: Union[str, List[str]] = 'all' # all, backbone
annotation_file: Optional[str] = None
per_category_metrics: bool = False

# TODO : we should pass this via cmd
# COCO_INPUT_PATH_BASE = '/depot/davisjam/data/vishal/datasets/coco/'
COCO_INPUT_PATH_BASE = 'gs://cam2-datasets/coco_panoptic/'
COCO_TRAIN_EXAMPLES = 118287
COCO_VAL_EXAMPLES = 5000


@exp_factory.register_config_factory('maskformer_coco_panoptic')
def maskformer_coco_panoptic() -> cfg.ExperimentConfig:
"""Config to get results that matches the paper."""
train_batch_size = 8
eval_batch_size = 8
steps_per_epoch = 100
# steps_per_epoch = COCO_TRAIN_EXAMPLES // train_batch_size
train_steps = 300 * steps_per_epoch # 300 epochs
decay_at = train_steps - 100 * steps_per_epoch # 200 epochs
config = cfg.ExperimentConfig(
task=MaskFormerTask(
init_checkpoint='',
init_checkpoint_modules='backbone',
annotation_file=os.path.join(COCO_INPUT_PATH_BASE,'annotations'
'instances_train2017.json'),
model=MaskFormer(
input_size=[640, 640, 3],
norm_activation=common.NormActivation()),
losses=Losses(),
train_data=DataConfig(
input_path=os.path.join(COCO_INPUT_PATH_BASE, 'tfrecords/train*'),
is_training=True,
global_batch_size=train_batch_size,
shuffle_buffer_size=1000,
parser = Parser(
output_size = [640,640],
min_scale = 0.3,
aspect_ratio_range = (0.5, 2.0),
min_overlap_params = (0.0, 1.4, 0.2, 0.1),
max_retry = 50,
pad_output = False,
resize_eval_groundtruth = True,
groundtruth_padded_size = None,
ignore_label = 0,
aug_rand_hflip = True,
aug_scale_min = 1.0,
aug_scale_max = 1.0,
color_aug_ssd = False,
brightness = 0.2,
saturation = 0.3,
contrast = 0.5,
# TODO choose appropriate augmentation
aug_type = None,
sigma = 8.0,
small_instance_area_threshold = 4096,
small_instance_weight = 3.0,
dtype = 'bfloat16',
seed = 2045,
)
),
validation_data=DataConfig(
input_path=os.path.join(COCO_INPUT_PATH_BASE, 'tfrecords/val*'),
is_training=False,
global_batch_size=eval_batch_size,
drop_remainder=False,
parser = Parser(
output_size = [640,640],
pad_output = True,
seed = 4096,
)

)),
trainer=cfg.TrainerConfig(
train_steps=train_steps,
validation_steps=COCO_VAL_EXAMPLES // eval_batch_size,
steps_per_loop=steps_per_epoch,
summary_interval=steps_per_epoch,
checkpoint_interval=steps_per_epoch,
validation_interval= 5 * steps_per_epoch,
max_to_keep=1,
best_checkpoint_export_subdir='best_ckpt',
# TODO: Not defined the metric
optimizer_config=optimization.OptimizationConfig({
'optimizer': {
'type': 'detr_adamw',
'detr_adamw': {
'weight_decay_rate': 1e-4,
'global_clipnorm': 0.1,
# Avoid AdamW legacy behavior.
'gradient_clip_norm': 0.0
}
},
'learning_rate': {
'type': 'stepwise',
'stepwise': {
'boundaries': [decay_at],
'values': [0.0001, 1.0e-05]
}
},
})),
restrictions=[
'task.train_data.is_training != None',
])
return config
Binary file not shown.
Loading