Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Reproducing results from the paper #16

Open
lvkd84 opened this issue Feb 19, 2021 · 4 comments
Open

Reproducing results from the paper #16

lvkd84 opened this issue Feb 19, 2021 · 4 comments

Comments

@lvkd84
Copy link

lvkd84 commented Feb 19, 2021

Hi,
I'm trying to reproduce the results from the paper. I downloaded the repo as is and upload it to Google Colab Pro. Then, I used the following commands to execute the codes:

%tensorflow_version 1.12
!pip install scipy==1.5.1
!python build_graph.py R8 3
!python train.py --dataset R8 --early_stopping 10

The results that I got for R8 dataset were quite low compared to that reported in the paper. Only around or below 0.97. Here is the log from the most recent run of the last command.

The python version is 3.6.9.

/content/utils.py:82: VisibleDeprecationWarning: Creating an ndarray from ragged nested sequences (which is a list-or-tuple of lists-or-tuples-or ndarrays with different lengths or shapes) is deprecated. If you meant to do this, you must specify 'dtype=object' when creating the ndarray
train_adj = np.array(train_adj)
/content/utils.py:83: VisibleDeprecationWarning: Creating an ndarray from ragged nested sequences (which is a list-or-tuple of lists-or-tuples-or ndarrays with different lengths or shapes) is deprecated. If you meant to do this, you must specify 'dtype=object' when creating the ndarray
val_adj = np.array(val_adj)
/content/utils.py:84: VisibleDeprecationWarning: Creating an ndarray from ragged nested sequences (which is a list-or-tuple of lists-or-tuples-or ndarrays with different lengths or shapes) is deprecated. If you meant to do this, you must specify 'dtype=object' when creating the ndarray
test_adj = np.array(test_adj)
/content/utils.py:85: VisibleDeprecationWarning: Creating an ndarray from ragged nested sequences (which is a list-or-tuple of lists-or-tuples-or ndarrays with different lengths or shapes) is deprecated. If you meant to do this, you must specify 'dtype=object' when creating the ndarray
train_embed = np.array(train_embed)
/content/utils.py:86: VisibleDeprecationWarning: Creating an ndarray from ragged nested sequences (which is a list-or-tuple of lists-or-tuples-or ndarrays with different lengths or shapes) is deprecated. If you meant to do this, you must specify 'dtype=object' when creating the ndarray
val_embed = np.array(val_embed)
/content/utils.py:87: VisibleDeprecationWarning: Creating an ndarray from ragged nested sequences (which is a list-or-tuple of lists-or-tuples-or ndarrays with different lengths or shapes) is deprecated. If you meant to do this, you must specify 'dtype=object' when creating the ndarray
test_embed = np.array(test_embed)
loading training set
100% 4937/4937 [00:04<00:00, 1153.54it/s]
tcmalloc: large alloc 3344564224 bytes == 0x10ed0c000 @ 0x7f1e78dc91e7 0x7f1e7626041e 0x7f1e762b0c2b 0x7f1e762b3e33 0x7f1e762b402b 0x7f1e76355701 0x50a4a5 0x50beb4 0x5095c8 0x50a2fd 0x50beb4 0x507be4 0x50ad03 0x634e72 0x634f27 0x6386df 0x639281 0x4b0dc0 0x7f1e789c6bf7 0x5b259a
100% 4937/4937 [00:01<00:00, 3453.25it/s]
tcmalloc: large alloc 3448004608 bytes == 0x1d6b98000 @ 0x7f1e78dc91e7 0x7f1e7626041e 0x7f1e762b0c2b 0x7f1e762b3e33 0x7f1e762b402b 0x7f1e76355701 0x50a4a5 0x50beb4 0x5095c8 0x50a2fd 0x50beb4 0x507be4 0x50ad03 0x634e72 0x634f27 0x6386df 0x639281 0x4b0dc0 0x7f1e789c6bf7 0x5b259a
loading validation set
100% 548/548 [00:00<00:00, 1785.99it/s]
100% 548/548 [00:00<00:00, 3361.42it/s]
loading test set
100% 2189/2189 [00:01<00:00, 1843.75it/s]
tcmalloc: large alloc 1363156992 bytes == 0x9f68e000 @ 0x7f1e78dc91e7 0x7f1e7626041e 0x7f1e762b0c2b 0x7f1e762b3e33 0x7f1e762b402b 0x7f1e76355701 0x50a4a5 0x50beb4 0x5095c8 0x50a2fd 0x50beb4 0x507be4 0x50ad03 0x634e72 0x634f27 0x6386df 0x639281 0x4b0dc0 0x7f1e789c6bf7 0x5b259a
100% 2189/2189 [00:00<00:00, 3550.98it/s]
WARNING:tensorflow:From train.py:67: The name tf.placeholder is deprecated. Please use tf.compat.v1.placeholder instead.

WARNING:tensorflow:From train.py:71: The name tf.placeholder_with_default is deprecated. Please use tf.compat.v1.placeholder_with_default instead.

WARNING:tensorflow:From /content/models.py:145: The name tf.train.AdamOptimizer is deprecated. Please use tf.compat.v1.train.AdamOptimizer instead.

build...
WARNING:tensorflow:From /content/models.py:41: The name tf.variable_scope is deprecated. Please use tf.compat.v1.variable_scope instead.

WARNING:tensorflow:From /content/inits.py:14: The name tf.random_uniform is deprecated. Please use tf.random.uniform instead.

WARNING:tensorflow:From /content/layers.py:120: The name tf.summary.histogram is deprecated. Please use tf.compat.v1.summary.histogram instead.

WARNING:tensorflow:From /content/layers.py:222: calling dropout (from tensorflow.python.ops.nn_ops) with keep_prob is deprecated and will be removed in a future version.
Instructions for updating:
Please use rate instead of keep_prob. Rate should be set to rate = 1 - keep_prob.
WARNING:tensorflow:From /content/models.py:53: The name tf.get_collection is deprecated. Please use tf.compat.v1.get_collection instead.

WARNING:tensorflow:From /content/models.py:53: The name tf.GraphKeys is deprecated. Please use tf.compat.v1.GraphKeys instead.

WARNING:tensorflow:From /content/models.py:155: The name tf.trainable_variables is deprecated. Please use tf.compat.v1.trainable_variables instead.

WARNING:tensorflow:From /content/metrics.py:6: softmax_cross_entropy_with_logits (from tensorflow.python.ops.nn_ops) is deprecated and will be removed in a future version.
Instructions for updating:

Future major versions of TensorFlow will allow gradients to flow
into the labels input on backprop by default.

See tf.nn.softmax_cross_entropy_with_logits_v2.

WARNING:tensorflow:From train.py:86: The name tf.Session is deprecated. Please use tf.compat.v1.Session instead.

2021-02-19 22:59:40.288206: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1
2021-02-19 22:59:40.313953: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-02-19 22:59:40.314912: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1639] Found device 0 with properties:
name: Tesla P100-PCIE-16GB major: 6 minor: 0 memoryClockRate(GHz): 1.3285
pciBusID: 0000:00:04.0
2021-02-19 22:59:40.315288: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1
2021-02-19 22:59:40.317432: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10
2021-02-19 22:59:40.319180: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10
2021-02-19 22:59:40.319574: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10
2021-02-19 22:59:40.321476: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10
2021-02-19 22:59:40.322424: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10
2021-02-19 22:59:40.326427: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7
2021-02-19 22:59:40.326587: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-02-19 22:59:40.327541: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-02-19 22:59:40.328407: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1767] Adding visible gpu devices: 0
2021-02-19 22:59:40.333859: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 2299995000 Hz
2021-02-19 22:59:40.334256: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7e4dd40 initialized for platform Host (this does not guarantee that XLA will be used). Devices:
2021-02-19 22:59:40.334312: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version
2021-02-19 22:59:40.416292: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-02-19 22:59:40.417420: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7b49d40 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices:
2021-02-19 22:59:40.417493: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Tesla P100-PCIE-16GB, Compute Capability 6.0
2021-02-19 22:59:40.417668: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-02-19 22:59:40.418522: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1639] Found device 0 with properties:
name: Tesla P100-PCIE-16GB major: 6 minor: 0 memoryClockRate(GHz): 1.3285
pciBusID: 0000:00:04.0
2021-02-19 22:59:40.418619: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1
2021-02-19 22:59:40.418649: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10
2021-02-19 22:59:40.418705: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10
2021-02-19 22:59:40.418732: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10
2021-02-19 22:59:40.418771: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10
2021-02-19 22:59:40.418830: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10
2021-02-19 22:59:40.418876: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7
2021-02-19 22:59:40.418987: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-02-19 22:59:40.420152: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-02-19 22:59:40.420862: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1767] Adding visible gpu devices: 0
2021-02-19 22:59:40.420919: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1
2021-02-19 22:59:40.422479: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1180] Device interconnect StreamExecutor with strength 1 edge matrix:
2021-02-19 22:59:40.422508: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1186] 0
2021-02-19 22:59:40.422519: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1199] 0: N
2021-02-19 22:59:40.422666: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-02-19 22:59:40.423590: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-02-19 22:59:40.424456: W tensorflow/core/common_runtime/gpu/gpu_bfc_allocator.cc:39] Overriding allow_growth setting because the TF_FORCE_GPU_ALLOW_GROWTH environment variable is set. Original config value was 0.
2021-02-19 22:59:40.424508: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1325] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 15224 MB memory) -> physical GPU (device: 0, name: Tesla P100-PCIE-16GB, pci bus id: 0000:00:04.0, compute capability: 6.0)
WARNING:tensorflow:From train.py:100: The name tf.global_variables_initializer is deprecated. Please use tf.compat.v1.global_variables_initializer instead.

train start...
tcmalloc: large alloc 2860646400 bytes == 0x7f1cccfda000 @ 0x7f1e78dc91e7 0x7f1e7626041e 0x7f1e762b0c2b 0x7f1e762b0cc8 0x7f1e76343349 0x7f1e76345a55 0x50c19e 0x507be4 0x50ad03 0x634e72 0x634f27 0x6386df 0x639281 0x4b0dc0 0x7f1e789c6bf7 0x5b259a
tcmalloc: large alloc 2774827008 bytes == 0x7f1c27192000 @ 0x7f1e78dc91e7 0x7f1e7626041e 0x7f1e762b0c2b 0x7f1e762b0cc8 0x7f1e76343349 0x7f1e76345a55 0x50c19e 0x507be4 0x50ad03 0x634e72 0x634f27 0x6386df 0x639281 0x4b0dc0 0x7f1e789c6bf7 0x5b259a
2021-02-19 22:59:44.780344: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10
Epoch: 0001 train_loss= 2.19713 train_acc= 0.12619 val_loss= 1.18276 val_acc= 0.54015 test_acc= 0.49749 time= 7.16459
tcmalloc: large alloc 2860646400 bytes == 0x7f1cccfda000 @ 0x7f1e78dc91e7 0x7f1e7626041e 0x7f1e762b0c2b 0x7f1e762b0cc8 0x7f1e76343349 0x7f1e76345a55 0x50c19e 0x507be4 0x50ad03 0x634e72 0x634f27 0x6386df 0x639281 0x4b0dc0 0x7f1e789c6bf7 0x5b259a
tcmalloc: large alloc 2774827008 bytes == 0x7f1ba11b6000 @ 0x7f1e78dc91e7 0x7f1e7626041e 0x7f1e762b0c2b 0x7f1e762b0cc8 0x7f1e76343349 0x7f1e76345a55 0x50c19e 0x507be4 0x50ad03 0x634e72 0x634f27 0x6386df 0x639281 0x4b0dc0 0x7f1e789c6bf7 0x5b259a
Epoch: 0002 train_loss= 1.27117 train_acc= 0.55358 val_loss= 0.98709 val_acc= 0.75000 test_acc= 0.78483 time= 5.48131
tcmalloc: large alloc 2860646400 bytes == 0x7f1cccfda000 @ 0x7f1e78dc91e7 0x7f1e7626041e 0x7f1e762b0c2b 0x7f1e762b0cc8 0x7f1e76343349 0x7f1e76345a55 0x50c19e 0x507be4 0x50ad03 0x634e72 0x634f27 0x6386df 0x639281 0x4b0dc0 0x7f1e789c6bf7 0x5b259a
tcmalloc: large alloc 2774827008 bytes == 0x7f1ba11b6000 @ 0x7f1e78dc91e7 0x7f1e7626041e 0x7f1e762b0c2b 0x7f1e762b0cc8 0x7f1e76343349 0x7f1e76345a55 0x50c19e 0x507be4 0x50ad03 0x634e72 0x634f27 0x6386df 0x639281 0x4b0dc0 0x7f1e789c6bf7 0x5b259a
Epoch: 0003 train_loss= 1.03188 train_acc= 0.69799 val_loss= 0.79159 val_acc= 0.75912 test_acc= 0.78026 time= 5.26398
Epoch: 0004 train_loss= 0.87602 train_acc= 0.74134 val_loss= 0.65070 val_acc= 0.77190 test_acc= 0.78118 time= 5.21352
Epoch: 0005 train_loss= 0.74427 train_acc= 0.75329 val_loss= 0.61101 val_acc= 0.78832 test_acc= 0.79397 time= 5.24498
Epoch: 0006 train_loss= 0.69071 train_acc= 0.76484 val_loss= 0.51856 val_acc= 0.83577 test_acc= 0.84011 time= 5.27161
Epoch: 0007 train_loss= 0.61153 train_acc= 0.79644 val_loss= 0.47803 val_acc= 0.84307 test_acc= 0.85975 time= 5.27141
Epoch: 0008 train_loss= 0.57619 train_acc= 0.80089 val_loss= 0.44037 val_acc= 0.84672 test_acc= 0.85290 time= 5.26265
Epoch: 0009 train_loss= 0.51257 train_acc= 0.81527 val_loss= 0.41687 val_acc= 0.85219 test_acc= 0.86067 time= 5.24975
Epoch: 0010 train_loss= 0.49260 train_acc= 0.81973 val_loss= 0.38203 val_acc= 0.86314 test_acc= 0.88305 time= 5.23994
Epoch: 0011 train_loss= 0.46670 train_acc= 0.83654 val_loss= 0.36208 val_acc= 0.86496 test_acc= 0.88488 time= 5.24392
Epoch: 0012 train_loss= 0.44745 train_acc= 0.84363 val_loss= 0.32356 val_acc= 0.89599 test_acc= 0.90726 time= 5.28619
Epoch: 0013 train_loss= 0.41033 train_acc= 0.85740 val_loss= 0.29557 val_acc= 0.91058 test_acc= 0.91366 time= 5.25913
Epoch: 0014 train_loss= 0.36834 train_acc= 0.87361 val_loss= 0.27699 val_acc= 0.90511 test_acc= 0.90544 time= 5.24832
Epoch: 0015 train_loss= 0.33900 train_acc= 0.87786 val_loss= 0.24892 val_acc= 0.91606 test_acc= 0.92188 time= 5.40275
Epoch: 0016 train_loss= 0.30960 train_acc= 0.89224 val_loss= 0.23543 val_acc= 0.91971 test_acc= 0.92828 time= 5.43948
Epoch: 0017 train_loss= 0.29585 train_acc= 0.89528 val_loss= 0.23820 val_acc= 0.91058 test_acc= 0.93193 time= 5.22677
Epoch: 0018 train_loss= 0.28628 train_acc= 0.89427 val_loss= 0.21523 val_acc= 0.92883 test_acc= 0.94198 time= 5.26715
Epoch: 0019 train_loss= 0.25932 train_acc= 0.90480 val_loss= 0.20096 val_acc= 0.93796 test_acc= 0.94107 time= 5.23199
Epoch: 0020 train_loss= 0.25889 train_acc= 0.91209 val_loss= 0.18539 val_acc= 0.93978 test_acc= 0.94518 time= 5.24492
Epoch: 0021 train_loss= 0.23199 train_acc= 0.91979 val_loss= 0.17907 val_acc= 0.94343 test_acc= 0.95021 time= 5.24143
Epoch: 0022 train_loss= 0.22114 train_acc= 0.92789 val_loss= 0.17115 val_acc= 0.94708 test_acc= 0.95066 time= 5.27204
Epoch: 0023 train_loss= 0.20715 train_acc= 0.92728 val_loss= 0.15791 val_acc= 0.95073 test_acc= 0.95158 time= 5.21739
Epoch: 0024 train_loss= 0.21088 train_acc= 0.92728 val_loss= 0.14660 val_acc= 0.95620 test_acc= 0.95934 time= 5.23590
Epoch: 0025 train_loss= 0.19642 train_acc= 0.93802 val_loss= 0.14318 val_acc= 0.95255 test_acc= 0.95889 time= 5.26726
Epoch: 0026 train_loss= 0.18673 train_acc= 0.93539 val_loss= 0.14005 val_acc= 0.95985 test_acc= 0.96117 time= 5.24012
Epoch: 0027 train_loss= 0.17085 train_acc= 0.94673 val_loss= 0.13166 val_acc= 0.95803 test_acc= 0.95751 time= 5.24573
Epoch: 0028 train_loss= 0.16628 train_acc= 0.94592 val_loss= 0.12531 val_acc= 0.96715 test_acc= 0.96071 time= 5.22776
Epoch: 0029 train_loss= 0.16193 train_acc= 0.94592 val_loss= 0.12951 val_acc= 0.96533 test_acc= 0.96482 time= 5.21652
Epoch: 0030 train_loss= 0.15526 train_acc= 0.95118 val_loss= 0.13832 val_acc= 0.95620 test_acc= 0.96528 time= 5.25098
Epoch: 0031 train_loss= 0.15074 train_acc= 0.95098 val_loss= 0.12373 val_acc= 0.96350 test_acc= 0.96437 time= 5.24724
Epoch: 0032 train_loss= 0.15112 train_acc= 0.94875 val_loss= 0.11347 val_acc= 0.96533 test_acc= 0.96163 time= 5.22761
Epoch: 0033 train_loss= 0.14135 train_acc= 0.95301 val_loss= 0.11771 val_acc= 0.96350 test_acc= 0.96574 time= 5.20961
Epoch: 0034 train_loss= 0.14083 train_acc= 0.95443 val_loss= 0.12411 val_acc= 0.95985 test_acc= 0.96437 time= 5.23837
Epoch: 0035 train_loss= 0.13634 train_acc= 0.95422 val_loss= 0.10965 val_acc= 0.96715 test_acc= 0.96071 time= 5.23632
Epoch: 0036 train_loss= 0.13367 train_acc= 0.95584 val_loss= 0.10600 val_acc= 0.97445 test_acc= 0.96574 time= 5.25844
Epoch: 0037 train_loss= 0.12564 train_acc= 0.95746 val_loss= 0.11893 val_acc= 0.97080 test_acc= 0.96437 time= 5.23042
Epoch: 0038 train_loss= 0.12295 train_acc= 0.95888 val_loss= 0.11020 val_acc= 0.97263 test_acc= 0.96482 time= 5.23870
Epoch: 0039 train_loss= 0.12087 train_acc= 0.96010 val_loss= 0.10628 val_acc= 0.97080 test_acc= 0.96802 time= 5.23002
Epoch: 0040 train_loss= 0.11964 train_acc= 0.96152 val_loss= 0.10099 val_acc= 0.97263 test_acc= 0.96300 time= 5.21768
Epoch: 0041 train_loss= 0.11953 train_acc= 0.96293 val_loss= 0.10075 val_acc= 0.97445 test_acc= 0.96482 time= 5.22469
Epoch: 0042 train_loss= 0.11595 train_acc= 0.95969 val_loss= 0.10223 val_acc= 0.96898 test_acc= 0.96711 time= 5.22958
Epoch: 0043 train_loss= 0.11599 train_acc= 0.96212 val_loss= 0.09982 val_acc= 0.97080 test_acc= 0.96437 time= 5.25619
Epoch: 0044 train_loss= 0.11026 train_acc= 0.96415 val_loss= 0.09549 val_acc= 0.97263 test_acc= 0.96894 time= 5.24579
Epoch: 0045 train_loss= 0.10430 train_acc= 0.96739 val_loss= 0.09941 val_acc= 0.97445 test_acc= 0.96894 time= 5.26075
Epoch: 0046 train_loss= 0.10363 train_acc= 0.96435 val_loss= 0.10249 val_acc= 0.96898 test_acc= 0.96437 time= 5.25947
Epoch: 0047 train_loss= 0.10309 train_acc= 0.96678 val_loss= 0.10327 val_acc= 0.97080 test_acc= 0.97305 time= 5.22501
Epoch: 0048 train_loss= 0.10143 train_acc= 0.96638 val_loss= 0.10224 val_acc= 0.97263 test_acc= 0.96802 time= 5.21243
Early stopping...
Optimization Finished!
Best epoch: 44
Test set results: cost= 0.08943 accuracy= 0.96894
Test Precision, Recall and F1-Score...
precision recall f1-score support

       0     0.8947    0.8395    0.8662        81
       1     0.9600    0.6667    0.7869        36
       2     0.9070    0.9669    0.9360       121
       3     0.9868    0.9684    0.9775       696
       4     0.9481    0.9733    0.9605        75
       5     0.9809    0.9954    0.9881      1083
       6     1.0000    0.8000    0.8889        10
       7     0.8587    0.9080    0.8827        87

accuracy                         0.9689      2189

macro avg 0.9420 0.8898 0.9109 2189
weighted avg 0.9693 0.9689 0.9684 2189

Macro average Test Precision, Recall and F1-Score...
(0.9420219683340437, 0.8897835372547763, 0.9108535401669325, None)
Micro average Test Precision, Recall and F1-Score...
(0.9689355870260393, 0.9689355870260393, 0.9689355870260393, None)
@lvkd84
Copy link
Author

lvkd84 commented Feb 20, 2021 •
edited
Loading

By changing to Numpy 1.18.5 I removed the warning about ragged arrays but the results are still the same.

@Magicat128
Copy link
Collaborator

Hi @lvkd84

Please try running full 200 epochs without early stopping.

@lvkd84
Copy link
Author

lvkd84 commented Feb 22, 2021

Thank you for replying. I did what you recommended using the following command:

!python train.py --dataset R8

The result I got from the last execution (it did improve):

Optimization Finished!
Best epoch: 141
Test set results: cost= 0.08810 accuracy= 0.97487
Test Precision, Recall and F1-Score...
precision recall f1-score support

       0     0.8022    0.9012    0.8488        81
       1     0.9881    0.9935    0.9908      1083
       2     0.9748    0.9587    0.9667       121
       3     0.9474    1.0000    0.9730        36
       4     0.8846    0.7931    0.8364        87
       5     1.0000    0.9000    0.9474        10
       6     0.9733    0.9733    0.9733        75
       7     0.9884    0.9799    0.9841       696

accuracy                         0.9749      2189

macro avg 0.9448 0.9375 0.9401 2189
weighted avg 0.9753 0.9749 0.9749 2189

Macro average Test Precision, Recall and F1-Score...
(0.9448466371093606, 0.9374713207116196, 0.9400576400859745, None)
Micro average Test Precision, Recall and F1-Score...
(0.9748743718592965, 0.9748743718592965, 0.9748743718592965, None)

I run the command a few times and these are the test accuracies I got so far:
[0.97305, 0.97487, 0.97533, 0.97487]
I also inspected the epochs and I could not find one with test accuracy 0.98. I wonder which combination of parameters should I use to get the results reported?

@susanhhhhhh
Copy link

Hi, I'm trying to reproduce the results from the paper. I downloaded the repo as is and upload it to Google Colab Pro. Then, I used the following commands to execute the codes:

%tensorflow_version 1.12 !pip install scipy==1.5.1 !python build_graph.py R8 3 !python train.py --dataset R8 --early_stopping 10

The results that I got for R8 dataset were quite low compared to that reported in the paper. Only around or below 0.97. Here is the log from the most recent run of the last command.

The python version is 3.6.9.

/content/utils.py:82: VisibleDeprecationWarning: Creating an ndarray from ragged nested sequences (which is a list-or-tuple of lists-or-tuples-or ndarrays with different lengths or shapes) is deprecated. If you meant to do this, you must specify 'dtype=object' when creating the ndarray train_adj = np.array(train_adj) /content/utils.py:83: VisibleDeprecationWarning: Creating an ndarray from ragged nested sequences (which is a list-or-tuple of lists-or-tuples-or ndarrays with different lengths or shapes) is deprecated. If you meant to do this, you must specify 'dtype=object' when creating the ndarray val_adj = np.array(val_adj) /content/utils.py:84: VisibleDeprecationWarning: Creating an ndarray from ragged nested sequences (which is a list-or-tuple of lists-or-tuples-or ndarrays with different lengths or shapes) is deprecated. If you meant to do this, you must specify 'dtype=object' when creating the ndarray test_adj = np.array(test_adj) /content/utils.py:85: VisibleDeprecationWarning: Creating an ndarray from ragged nested sequences (which is a list-or-tuple of lists-or-tuples-or ndarrays with different lengths or shapes) is deprecated. If you meant to do this, you must specify 'dtype=object' when creating the ndarray train_embed = np.array(train_embed) /content/utils.py:86: VisibleDeprecationWarning: Creating an ndarray from ragged nested sequences (which is a list-or-tuple of lists-or-tuples-or ndarrays with different lengths or shapes) is deprecated. If you meant to do this, you must specify 'dtype=object' when creating the ndarray val_embed = np.array(val_embed) /content/utils.py:87: VisibleDeprecationWarning: Creating an ndarray from ragged nested sequences (which is a list-or-tuple of lists-or-tuples-or ndarrays with different lengths or shapes) is deprecated. If you meant to do this, you must specify 'dtype=object' when creating the ndarray test_embed = np.array(test_embed) loading training set 100% 4937/4937 [00:04<00:00, 1153.54it/s] tcmalloc: large alloc 3344564224 bytes == 0x10ed0c000 @ 0x7f1e78dc91e7 0x7f1e7626041e 0x7f1e762b0c2b 0x7f1e762b3e33 0x7f1e762b402b 0x7f1e76355701 0x50a4a5 0x50beb4 0x5095c8 0x50a2fd 0x50beb4 0x507be4 0x50ad03 0x634e72 0x634f27 0x6386df 0x639281 0x4b0dc0 0x7f1e789c6bf7 0x5b259a 100% 4937/4937 [00:01<00:00, 3453.25it/s] tcmalloc: large alloc 3448004608 bytes == 0x1d6b98000 @ 0x7f1e78dc91e7 0x7f1e7626041e 0x7f1e762b0c2b 0x7f1e762b3e33 0x7f1e762b402b 0x7f1e76355701 0x50a4a5 0x50beb4 0x5095c8 0x50a2fd 0x50beb4 0x507be4 0x50ad03 0x634e72 0x634f27 0x6386df 0x639281 0x4b0dc0 0x7f1e789c6bf7 0x5b259a loading validation set 100% 548/548 [00:00<00:00, 1785.99it/s] 100% 548/548 [00:00<00:00, 3361.42it/s] loading test set 100% 2189/2189 [00:01<00:00, 1843.75it/s] tcmalloc: large alloc 1363156992 bytes == 0x9f68e000 @ 0x7f1e78dc91e7 0x7f1e7626041e 0x7f1e762b0c2b 0x7f1e762b3e33 0x7f1e762b402b 0x7f1e76355701 0x50a4a5 0x50beb4 0x5095c8 0x50a2fd 0x50beb4 0x507be4 0x50ad03 0x634e72 0x634f27 0x6386df 0x639281 0x4b0dc0 0x7f1e789c6bf7 0x5b259a 100% 2189/2189 [00:00<00:00, 3550.98it/s] WARNING:tensorflow:From train.py:67: The name tf.placeholder is deprecated. Please use tf.compat.v1.placeholder instead.

WARNING:tensorflow:From train.py:71: The name tf.placeholder_with_default is deprecated. Please use tf.compat.v1.placeholder_with_default instead.

WARNING:tensorflow:From /content/models.py:145: The name tf.train.AdamOptimizer is deprecated. Please use tf.compat.v1.train.AdamOptimizer instead.

build... WARNING:tensorflow:From /content/models.py:41: The name tf.variable_scope is deprecated. Please use tf.compat.v1.variable_scope instead.

WARNING:tensorflow:From /content/inits.py:14: The name tf.random_uniform is deprecated. Please use tf.random.uniform instead.

WARNING:tensorflow:From /content/layers.py:120: The name tf.summary.histogram is deprecated. Please use tf.compat.v1.summary.histogram instead.

WARNING:tensorflow:From /content/layers.py:222: calling dropout (from tensorflow.python.ops.nn_ops) with keep_prob is deprecated and will be removed in a future version. Instructions for updating: Please use rate instead of keep_prob. Rate should be set to rate = 1 - keep_prob. WARNING:tensorflow:From /content/models.py:53: The name tf.get_collection is deprecated. Please use tf.compat.v1.get_collection instead.

WARNING:tensorflow:From /content/models.py:53: The name tf.GraphKeys is deprecated. Please use tf.compat.v1.GraphKeys instead.

WARNING:tensorflow:From /content/models.py:155: The name tf.trainable_variables is deprecated. Please use tf.compat.v1.trainable_variables instead.

WARNING:tensorflow:From /content/metrics.py:6: softmax_cross_entropy_with_logits (from tensorflow.python.ops.nn_ops) is deprecated and will be removed in a future version. Instructions for updating:

Future major versions of TensorFlow will allow gradients to flow into the labels input on backprop by default.

See tf.nn.softmax_cross_entropy_with_logits_v2.

WARNING:tensorflow:From train.py:86: The name tf.Session is deprecated. Please use tf.compat.v1.Session instead.

2021-02-19 22:59:40.288206: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1 2021-02-19 22:59:40.313953: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero 2021-02-19 22:59:40.314912: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1639] Found device 0 with properties: name: Tesla P100-PCIE-16GB major: 6 minor: 0 memoryClockRate(GHz): 1.3285 pciBusID: 0000:00:04.0 2021-02-19 22:59:40.315288: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2021-02-19 22:59:40.317432: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2021-02-19 22:59:40.319180: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2021-02-19 22:59:40.319574: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2021-02-19 22:59:40.321476: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2021-02-19 22:59:40.322424: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2021-02-19 22:59:40.326427: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2021-02-19 22:59:40.326587: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero 2021-02-19 22:59:40.327541: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero 2021-02-19 22:59:40.328407: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1767] Adding visible gpu devices: 0 2021-02-19 22:59:40.333859: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 2299995000 Hz 2021-02-19 22:59:40.334256: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7e4dd40 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2021-02-19 22:59:40.334312: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version 2021-02-19 22:59:40.416292: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero 2021-02-19 22:59:40.417420: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7b49d40 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices: 2021-02-19 22:59:40.417493: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Tesla P100-PCIE-16GB, Compute Capability 6.0 2021-02-19 22:59:40.417668: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero 2021-02-19 22:59:40.418522: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1639] Found device 0 with properties: name: Tesla P100-PCIE-16GB major: 6 minor: 0 memoryClockRate(GHz): 1.3285 pciBusID: 0000:00:04.0 2021-02-19 22:59:40.418619: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2021-02-19 22:59:40.418649: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2021-02-19 22:59:40.418705: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2021-02-19 22:59:40.418732: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2021-02-19 22:59:40.418771: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2021-02-19 22:59:40.418830: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2021-02-19 22:59:40.418876: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2021-02-19 22:59:40.418987: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero 2021-02-19 22:59:40.420152: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero 2021-02-19 22:59:40.420862: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1767] Adding visible gpu devices: 0 2021-02-19 22:59:40.420919: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2021-02-19 22:59:40.422479: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1180] Device interconnect StreamExecutor with strength 1 edge matrix: 2021-02-19 22:59:40.422508: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1186] 0 2021-02-19 22:59:40.422519: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1199] 0: N 2021-02-19 22:59:40.422666: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero 2021-02-19 22:59:40.423590: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero 2021-02-19 22:59:40.424456: W tensorflow/core/common_runtime/gpu/gpu_bfc_allocator.cc:39] Overriding allow_growth setting because the TF_FORCE_GPU_ALLOW_GROWTH environment variable is set. Original config value was 0. 2021-02-19 22:59:40.424508: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1325] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 15224 MB memory) -> physical GPU (device: 0, name: Tesla P100-PCIE-16GB, pci bus id: 0000:00:04.0, compute capability: 6.0) WARNING:tensorflow:From train.py💯 The name tf.global_variables_initializer is deprecated. Please use tf.compat.v1.global_variables_initializer instead.

train start... tcmalloc: large alloc 2860646400 bytes == 0x7f1cccfda000 @ 0x7f1e78dc91e7 0x7f1e7626041e 0x7f1e762b0c2b 0x7f1e762b0cc8 0x7f1e76343349 0x7f1e76345a55 0x50c19e 0x507be4 0x50ad03 0x634e72 0x634f27 0x6386df 0x639281 0x4b0dc0 0x7f1e789c6bf7 0x5b259a tcmalloc: large alloc 2774827008 bytes == 0x7f1c27192000 @ 0x7f1e78dc91e7 0x7f1e7626041e 0x7f1e762b0c2b 0x7f1e762b0cc8 0x7f1e76343349 0x7f1e76345a55 0x50c19e 0x507be4 0x50ad03 0x634e72 0x634f27 0x6386df 0x639281 0x4b0dc0 0x7f1e789c6bf7 0x5b259a 2021-02-19 22:59:44.780344: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 Epoch: 0001 train_loss= 2.19713 train_acc= 0.12619 val_loss= 1.18276 val_acc= 0.54015 test_acc= 0.49749 time= 7.16459 tcmalloc: large alloc 2860646400 bytes == 0x7f1cccfda000 @ 0x7f1e78dc91e7 0x7f1e7626041e 0x7f1e762b0c2b 0x7f1e762b0cc8 0x7f1e76343349 0x7f1e76345a55 0x50c19e 0x507be4 0x50ad03 0x634e72 0x634f27 0x6386df 0x639281 0x4b0dc0 0x7f1e789c6bf7 0x5b259a tcmalloc: large alloc 2774827008 bytes == 0x7f1ba11b6000 @ 0x7f1e78dc91e7 0x7f1e7626041e 0x7f1e762b0c2b 0x7f1e762b0cc8 0x7f1e76343349 0x7f1e76345a55 0x50c19e 0x507be4 0x50ad03 0x634e72 0x634f27 0x6386df 0x639281 0x4b0dc0 0x7f1e789c6bf7 0x5b259a Epoch: 0002 train_loss= 1.27117 train_acc= 0.55358 val_loss= 0.98709 val_acc= 0.75000 test_acc= 0.78483 time= 5.48131 tcmalloc: large alloc 2860646400 bytes == 0x7f1cccfda000 @ 0x7f1e78dc91e7 0x7f1e7626041e 0x7f1e762b0c2b 0x7f1e762b0cc8 0x7f1e76343349 0x7f1e76345a55 0x50c19e 0x507be4 0x50ad03 0x634e72 0x634f27 0x6386df 0x639281 0x4b0dc0 0x7f1e789c6bf7 0x5b259a tcmalloc: large alloc 2774827008 bytes == 0x7f1ba11b6000 @ 0x7f1e78dc91e7 0x7f1e7626041e 0x7f1e762b0c2b 0x7f1e762b0cc8 0x7f1e76343349 0x7f1e76345a55 0x50c19e 0x507be4 0x50ad03 0x634e72 0x634f27 0x6386df 0x639281 0x4b0dc0 0x7f1e789c6bf7 0x5b259a Epoch: 0003 train_loss= 1.03188 train_acc= 0.69799 val_loss= 0.79159 val_acc= 0.75912 test_acc= 0.78026 time= 5.26398 Epoch: 0004 train_loss= 0.87602 train_acc= 0.74134 val_loss= 0.65070 val_acc= 0.77190 test_acc= 0.78118 time= 5.21352 Epoch: 0005 train_loss= 0.74427 train_acc= 0.75329 val_loss= 0.61101 val_acc= 0.78832 test_acc= 0.79397 time= 5.24498 Epoch: 0006 train_loss= 0.69071 train_acc= 0.76484 val_loss= 0.51856 val_acc= 0.83577 test_acc= 0.84011 time= 5.27161 Epoch: 0007 train_loss= 0.61153 train_acc= 0.79644 val_loss= 0.47803 val_acc= 0.84307 test_acc= 0.85975 time= 5.27141 Epoch: 0008 train_loss= 0.57619 train_acc= 0.80089 val_loss= 0.44037 val_acc= 0.84672 test_acc= 0.85290 time= 5.26265 Epoch: 0009 train_loss= 0.51257 train_acc= 0.81527 val_loss= 0.41687 val_acc= 0.85219 test_acc= 0.86067 time= 5.24975 Epoch: 0010 train_loss= 0.49260 train_acc= 0.81973 val_loss= 0.38203 val_acc= 0.86314 test_acc= 0.88305 time= 5.23994 Epoch: 0011 train_loss= 0.46670 train_acc= 0.83654 val_loss= 0.36208 val_acc= 0.86496 test_acc= 0.88488 time= 5.24392 Epoch: 0012 train_loss= 0.44745 train_acc= 0.84363 val_loss= 0.32356 val_acc= 0.89599 test_acc= 0.90726 time= 5.28619 Epoch: 0013 train_loss= 0.41033 train_acc= 0.85740 val_loss= 0.29557 val_acc= 0.91058 test_acc= 0.91366 time= 5.25913 Epoch: 0014 train_loss= 0.36834 train_acc= 0.87361 val_loss= 0.27699 val_acc= 0.90511 test_acc= 0.90544 time= 5.24832 Epoch: 0015 train_loss= 0.33900 train_acc= 0.87786 val_loss= 0.24892 val_acc= 0.91606 test_acc= 0.92188 time= 5.40275 Epoch: 0016 train_loss= 0.30960 train_acc= 0.89224 val_loss= 0.23543 val_acc= 0.91971 test_acc= 0.92828 time= 5.43948 Epoch: 0017 train_loss= 0.29585 train_acc= 0.89528 val_loss= 0.23820 val_acc= 0.91058 test_acc= 0.93193 time= 5.22677 Epoch: 0018 train_loss= 0.28628 train_acc= 0.89427 val_loss= 0.21523 val_acc= 0.92883 test_acc= 0.94198 time= 5.26715 Epoch: 0019 train_loss= 0.25932 train_acc= 0.90480 val_loss= 0.20096 val_acc= 0.93796 test_acc= 0.94107 time= 5.23199 Epoch: 0020 train_loss= 0.25889 train_acc= 0.91209 val_loss= 0.18539 val_acc= 0.93978 test_acc= 0.94518 time= 5.24492 Epoch: 0021 train_loss= 0.23199 train_acc= 0.91979 val_loss= 0.17907 val_acc= 0.94343 test_acc= 0.95021 time= 5.24143 Epoch: 0022 train_loss= 0.22114 train_acc= 0.92789 val_loss= 0.17115 val_acc= 0.94708 test_acc= 0.95066 time= 5.27204 Epoch: 0023 train_loss= 0.20715 train_acc= 0.92728 val_loss= 0.15791 val_acc= 0.95073 test_acc= 0.95158 time= 5.21739 Epoch: 0024 train_loss= 0.21088 train_acc= 0.92728 val_loss= 0.14660 val_acc= 0.95620 test_acc= 0.95934 time= 5.23590 Epoch: 0025 train_loss= 0.19642 train_acc= 0.93802 val_loss= 0.14318 val_acc= 0.95255 test_acc= 0.95889 time= 5.26726 Epoch: 0026 train_loss= 0.18673 train_acc= 0.93539 val_loss= 0.14005 val_acc= 0.95985 test_acc= 0.96117 time= 5.24012 Epoch: 0027 train_loss= 0.17085 train_acc= 0.94673 val_loss= 0.13166 val_acc= 0.95803 test_acc= 0.95751 time= 5.24573 Epoch: 0028 train_loss= 0.16628 train_acc= 0.94592 val_loss= 0.12531 val_acc= 0.96715 test_acc= 0.96071 time= 5.22776 Epoch: 0029 train_loss= 0.16193 train_acc= 0.94592 val_loss= 0.12951 val_acc= 0.96533 test_acc= 0.96482 time= 5.21652 Epoch: 0030 train_loss= 0.15526 train_acc= 0.95118 val_loss= 0.13832 val_acc= 0.95620 test_acc= 0.96528 time= 5.25098 Epoch: 0031 train_loss= 0.15074 train_acc= 0.95098 val_loss= 0.12373 val_acc= 0.96350 test_acc= 0.96437 time= 5.24724 Epoch: 0032 train_loss= 0.15112 train_acc= 0.94875 val_loss= 0.11347 val_acc= 0.96533 test_acc= 0.96163 time= 5.22761 Epoch: 0033 train_loss= 0.14135 train_acc= 0.95301 val_loss= 0.11771 val_acc= 0.96350 test_acc= 0.96574 time= 5.20961 Epoch: 0034 train_loss= 0.14083 train_acc= 0.95443 val_loss= 0.12411 val_acc= 0.95985 test_acc= 0.96437 time= 5.23837 Epoch: 0035 train_loss= 0.13634 train_acc= 0.95422 val_loss= 0.10965 val_acc= 0.96715 test_acc= 0.96071 time= 5.23632 Epoch: 0036 train_loss= 0.13367 train_acc= 0.95584 val_loss= 0.10600 val_acc= 0.97445 test_acc= 0.96574 time= 5.25844 Epoch: 0037 train_loss= 0.12564 train_acc= 0.95746 val_loss= 0.11893 val_acc= 0.97080 test_acc= 0.96437 time= 5.23042 Epoch: 0038 train_loss= 0.12295 train_acc= 0.95888 val_loss= 0.11020 val_acc= 0.97263 test_acc= 0.96482 time= 5.23870 Epoch: 0039 train_loss= 0.12087 train_acc= 0.96010 val_loss= 0.10628 val_acc= 0.97080 test_acc= 0.96802 time= 5.23002 Epoch: 0040 train_loss= 0.11964 train_acc= 0.96152 val_loss= 0.10099 val_acc= 0.97263 test_acc= 0.96300 time= 5.21768 Epoch: 0041 train_loss= 0.11953 train_acc= 0.96293 val_loss= 0.10075 val_acc= 0.97445 test_acc= 0.96482 time= 5.22469 Epoch: 0042 train_loss= 0.11595 train_acc= 0.95969 val_loss= 0.10223 val_acc= 0.96898 test_acc= 0.96711 time= 5.22958 Epoch: 0043 train_loss= 0.11599 train_acc= 0.96212 val_loss= 0.09982 val_acc= 0.97080 test_acc= 0.96437 time= 5.25619 Epoch: 0044 train_loss= 0.11026 train_acc= 0.96415 val_loss= 0.09549 val_acc= 0.97263 test_acc= 0.96894 time= 5.24579 Epoch: 0045 train_loss= 0.10430 train_acc= 0.96739 val_loss= 0.09941 val_acc= 0.97445 test_acc= 0.96894 time= 5.26075 Epoch: 0046 train_loss= 0.10363 train_acc= 0.96435 val_loss= 0.10249 val_acc= 0.96898 test_acc= 0.96437 time= 5.25947 Epoch: 0047 train_loss= 0.10309 train_acc= 0.96678 val_loss= 0.10327 val_acc= 0.97080 test_acc= 0.97305 time= 5.22501 Epoch: 0048 train_loss= 0.10143 train_acc= 0.96638 val_loss= 0.10224 val_acc= 0.97263 test_acc= 0.96802 time= 5.21243 Early stopping... Optimization Finished! Best epoch: 44 Test set results: cost= 0.08943 accuracy= 0.96894 Test Precision, Recall and F1-Score... precision recall f1-score support

       0     0.8947    0.8395    0.8662        81
       1     0.9600    0.6667    0.7869        36
       2     0.9070    0.9669    0.9360       121
       3     0.9868    0.9684    0.9775       696
       4     0.9481    0.9733    0.9605        75
       5     0.9809    0.9954    0.9881      1083
       6     1.0000    0.8000    0.8889        10
       7     0.8587    0.9080    0.8827        87

accuracy                         0.9689      2189

macro avg 0.9420 0.8898 0.9109 2189 weighted avg 0.9693 0.9689 0.9684 2189

Macro average Test Precision, Recall and F1-Score... (0.9420219683340437, 0.8897835372547763, 0.9108535401669325, None) Micro average Test Precision, Recall and F1-Score... (0.9689355870260393, 0.9689355870260393, 0.9689355870260393, None)

hello, can you share your parameter settings?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@lvkd84 @Magicat128 @susanhhhhhh