Continuing Training from a Previous Checkpoint

[SebastianPaucar]

Hi,

I would like to continue the training in a new run (not as the second stage of a first training). I tried changing the agent_file in my config.toml to the .chkpt generated in a previous run, but it doesn't work. What can I do?

Thanks in advance

[halx]

Hi,

many thanks for your interest in REINVENT and welcome to the community!

You need to provide me with details, otherwise I won't be able to diagnose the issue. How have you concluded that it doesnät work? Was there an error message? What were the inputs, etc. ?

Many thanks,
Hannes.

[SebastianPaucar]

Thank you for your response @halx. I believe this could improve the robustness of REINVENT's operation. The issue I encountered when continuing the training is as follows: I tried to compare three scenarios:

1. Running REINVENT using a config.toml with two [[stage]] sections (and the same scoring functions) to perform a two-stage training (4 epochs for the first stage and 4 epochs for the second stage). This generated two .csv files, staged_learning_1.csv and staged_learning_2.csv. I will refer to these as files A and B, respectively.

2. Running REINVENT using a config.toml with a single [[stage]] section, to perform a single-stage training with 8 epochs. This generated a single .csv file, staged_learning_1.csv. I will call this file C.

3. Running REINVENT using the previous config.toml but with 4 epochs, generating a .csv file I will refer to as file D. From the checkpoint obtained in the fourth epoch, I ran REINVENT again starting from this new agent (by placing this checkpoint in the agent_file field of the config.toml) for another 4 epochs. This also generated a single .csv file, staged_learning_1.csv, which I will call file E.

In all cases, I used the same scoring functions and termination conditions in the config.toml files.

Results:

• File C (resulting from a single stage with 8 epochs) shows that it generated the same SMILES and scores as files A + B or B followed by A (resulting from two stages of 4 epochs each).

• File D (resulting from a single stage with 4 epochs) reflects only what was generated by file A and also by the first 4 epochs of file C.

The problem lies with file E, which does not respect the continuation of the training. File E should correctly start from the checkpoint and continue generating the same SMILES as if the training from file D had been continued. What I am looking for is for file E to contain the same as file B, which is the same as the last 4 epochs of file C.

More specifically:

• Training history (Scenario 1, files A and B):
  Two stages, 4 epochs per stage.
  File A (staged_learning_1.csv):

Agent,Prior,Target,Score,SMILES,SMILES_state,Molecular weight,Molecular weight (raw),Unwanted SMARTS,Unwanted SMARTS (raw),step
28.8788,28.8788,16.1095,0.3514709,COc1cccc(CN2CCC(N=c3[nH]c(N4CCN(C)CC4)nc4cc(OC)c(OC)cc34)CC2)c1,1,0.3514709,506.6510,1.0000000,1.0000,1
22.5991,22.5991,105.3932,0.9999398,CC(=O)c1cccc(NC(=O)CSCc2nc(-c3ccc(C)cc3)oc2C)c1,1,0.9999398,394.4960,1.0000000,1.0000,1
43.3108,43.3108,-43.3108,0.0000000,COc1ccc2[nH]c3c(c(=O)c2c1)C(CC(=O)OC1c2cc4c(cc2C(c2cc(OC)c(OC)c(OC)c2)C2C(=O)OCC12)OCO4)c1ccccc1-3,1,0.0000000,0.0000,0.0000000,0.0000,1
25.8675,25.8675,101.4203,0.9944361,Fc1cccc(C=Cc2nc3cccnc3s2)c1,1,0.9944361,256.3050,1.0000000,1.0000,1
34.2160,34.2840,-34.2840,0.0000000,COC(=O)CNC(=O)C1CC1C(=O)NC(C#N)Cc1ccc(C#Cc2ccccc2)cc1,1,0.0000000,0.0000,0.0000000,0.0000,2
21.1360,21.1656,106.8342,0.9999985,Cc1cc(C)cc(-n2ncc3c2CCCC3NC(=O)c2ccccc2)c1,1,0.9999985,345.4460,1.0000000,1.0000,2
18.7809,18.7439,109.2448,0.9999117,COc1cc(C(=O)NCc2ccccc2)cc(OC)c1OC,1,0.9999117,301.3420,1.0000000,1.0000,2
25.7743,25.8302,101.6824,0.9961920,O=C1CCc2ccc(C(=O)N(c3ccccc3)C3CCN(Cc4ccccc4)CC3)cc2N1,1,0.9961920,439.5590,1.0000000,1.0000,2
31.5609,31.4528,70.1004,0.7933842,O=C(N=c1nn[nH][nH]1)c1cc(N2CCN(c3ccc(C(F)(F)F)cc3)CC2)cc(C(F)(F)F)c1,1,0.7933842,485.3920,1.0000000,1.0000,3
25.5686,25.5522,36.4973,0.4847616,CCC(CC)C(O)CC1CCNCC1,1,0.4847616,199.3380,1.0000000,1.0000,3
30.5101,30.4866,97.5116,0.9999861,Cc1ncc(C(=O)N2CCC3(CC2)NC(=O)C32CCOC2)s1,1,0.9999861,321.4020,1.0000000,1.0000,3
26.3633,26.3665,101.6328,0.9999946,Brc1ccccc1Oc1cccc(CNCc2ccccc2)c1,1,0.9999946,368.2740,1.0000000,1.0000,3
24.2177,24.0738,-24.0738,0.0000000,CC1CCCCN1CCNC(=O)c1cnn(-c2ccccc2)c1C1CCN(C(=O)OC(C)(C)C)CC1,1,0.0000000,0.0000,0.0000000,0.0000,4
31.8345,31.7957,96.1958,0.9999331,COc1ccc2c(c1)c1c(=O)n(N)c(=S)[nH]c1n2C(C)C,1,0.9999331,304.3750,1.0000000,1.0000,4
22.4919,22.4614,105.5383,0.9999971,CCCCOc1ccc2c(c1)C(c1ccc(F)cc1)C(C#N)C(=N)O2,1,0.9999971,338.3820,1.0000000,1.0000,4
40.3546,40.3791,-40.2383,0.0011002,O=C(CC1CC2CCCC1C2)NCCCn1nc(-c2ccc(Cl)cc2Cl)cc1C(=O)Nc1ccc(Cl)cc1,1,0.0011002,573.9520,1.0000000,1.0000,4

File B (staged_learning_2.csv):

Agent,Prior,Target,Score,SMILES,SMILES_state,Molecular weight,Molecular weight (raw),Unwanted SMARTS,Unwanted SMARTS (raw),step
42.5189,42.5900,-42.5900,0.0000000,O=C(O)CS(=O)(=O)c1nc(C(=O)N2CCOCC2)nc2sc(CN3CCOCC3)cc12,1,0.0000000,0.0000,0.0000000,0.0000,1
23.3246,23.4071,-23.4071,0.0000000,CC(=O)OCC1CN(C(=O)c2ccccc2)c2ccccc2C(c2ccccc2)=N1,1,0.0000000,0.0000,0.0000000,0.0000,1
27.4569,27.6354,96.4237,0.9692122,Cc1ccc(CN2C(=O)c3ccccc3Oc3ccc(C(=O)NCCc4ccccc4)cc32)cc1,1,0.9692122,462.5490,1.0000000,1.0000,1
19.9347,19.7887,8.2013,0.2186720,N#Cc1ccc(-c2n[nH]c(=N)o2)cc1,1,0.2186720,186.1740,1.0000000,1.0000,1
36.5449,36.1825,91.8165,0.9999923,CC(=Nc1ccc(Cl)cc1[N+](=O)[O-])N1CCSC(C)(C)C1,1,0.9999923,327.8370,1.0000000,1.0000,2
25.3778,25.4466,-25.4466,0.0000000,COC(=O)c1cc(OCc2ccc3ccccc3n2)cc(C(=O)OC)n1,1,0.0000000,0.0000,0.0000000,0.0000,2
38.3508,38.5302,89.4682,0.9999874,CCOc1cc(N2CCC(C)(O)CC2)cnc1OCC(C)(C)C,1,0.9999874,322.4490,1.0000000,1.0000,2
19.0588,19.7478,108.2114,0.9996811,c1ccc(CCNCc2ccc(-c3ccccc3)cc2)cc1,1,0.9996811,287.4060,1.0000000,1.0000,2
21.5394,21.5115,-21.5115,0.0000000,Cc1c(C(=O)O)oc2ccc(Br)cc12,1,0.0000000,0.0000,0.0000000,0.0000,3
34.7336,34.8306,-34.8306,0.0000000,CN1CCc2cccc3c4c([nH]c23)C(=O)c2cc3c(cc2C41)[nH]c1ccccc13,1,0.0000000,0.0000,0.0000000,0.0000,3
29.1114,28.8993,99.0844,0.9998727,CCC(=O)NS(=O)(=O)c1ccc(S(C)(=O)=O)s1,1,0.9998727,297.3790,1.0000000,1.0000,3
21.9754,22.0098,97.9921,0.9375147,CC(=O)Nc1ccc(S(=O)(=O)N2CCC(C(=O)N3CCN(c4ccccc4)CC3)CC2)cc1,1,0.9375147,470.5950,1.0000000,1.0000,3
24.0361,23.8490,103.7794,0.9970970,CC1CCCC(NC(=O)C2CCN(S(=O)(=O)c3cccc4nsnc34)CC2)C1C,1,0.9970970,436.6030,1.0000000,1.0000,4
23.2951,23.5942,104.1223,0.9977854,CCc1ccc(C=CC(=O)NCc2ccccn2)cc1,1,0.9977854,266.3440,1.0000000,1.0000,4
22.5281,22.5278,87.3851,0.8586947,O=C(Nc1ccc(OC(F)(F)F)cc1)N1CCc2ccccc2C1c1ccc(C(F)(F)F)cc1,1,0.8586947,480.4080,1.0000000,1.0000,4
27.1197,27.2749,100.6536,0.9994416,O=C(N=c1cc[nH]n1CCn1ccnc1)c1ccccc1,1,0.9994416,281.3190,1.0000000,1.0000,4

• Training history (Scenario 2, file C):
  A single stage, 8 epochs.
  File C:

Agent,Prior,Target,Score,SMILES,SMILES_state,Molecular weight,Molecular weight (raw),Unwanted SMARTS,Unwanted SMARTS (raw),step
28.8788,28.8788,16.1095,0.3514709,COc1cccc(CN2CCC(N=c3[nH]c(N4CCN(C)CC4)nc4cc(OC)c(OC)cc34)CC2)c1,1,0.3514709,506.6510,1.0000000,1.0000,1
22.5991,22.5991,105.3932,0.9999398,CC(=O)c1cccc(NC(=O)CSCc2nc(-c3ccc(C)cc3)oc2C)c1,1,0.9999398,394.4960,1.0000000,1.0000,1
43.3108,43.3108,-43.3108,0.0000000,COc1ccc2[nH]c3c(c(=O)c2c1)C(CC(=O)OC1c2cc4c(cc2C(c2cc(OC)c(OC)c(OC)c2)C2C(=O)OCC12)OCO4)c1ccccc1-3,1,0.0000000,0.0000,0.0000000,0.0000,1
25.8675,25.8675,101.4203,0.9944361,Fc1cccc(C=Cc2nc3cccnc3s2)c1,1,0.9944361,256.3050,1.0000000,1.0000,1
34.2160,34.2840,-34.2840,0.0000000,COC(=O)CNC(=O)C1CC1C(=O)NC(C#N)Cc1ccc(C#Cc2ccccc2)cc1,1,0.0000000,0.0000,0.0000000,0.0000,2
21.1360,21.1656,106.8342,0.9999985,Cc1cc(C)cc(-n2ncc3c2CCCC3NC(=O)c2ccccc2)c1,1,0.9999985,345.4460,1.0000000,1.0000,2
18.7809,18.7439,109.2448,0.9999117,COc1cc(C(=O)NCc2ccccc2)cc(OC)c1OC,1,0.9999117,301.3420,1.0000000,1.0000,2
25.7743,25.8302,101.6824,0.9961920,O=C1CCc2ccc(C(=O)N(c3ccccc3)C3CCN(Cc4ccccc4)CC3)cc2N1,1,0.9961920,439.5590,1.0000000,1.0000,2
31.5609,31.4528,70.1004,0.7933842,O=C(N=c1nn[nH][nH]1)c1cc(N2CCN(c3ccc(C(F)(F)F)cc3)CC2)cc(C(F)(F)F)c1,1,0.7933842,485.3920,1.0000000,1.0000,3
25.5686,25.5522,36.4973,0.4847616,CCC(CC)C(O)CC1CCNCC1,1,0.4847616,199.3380,1.0000000,1.0000,3
30.5101,30.4866,97.5116,0.9999861,Cc1ncc(C(=O)N2CCC3(CC2)NC(=O)C32CCOC2)s1,1,0.9999861,321.4020,1.0000000,1.0000,3
26.3633,26.3665,101.6328,0.9999946,Brc1ccccc1Oc1cccc(CNCc2ccccc2)c1,1,0.9999946,368.2740,1.0000000,1.0000,3
24.2177,24.0738,-24.0738,0.0000000,CC1CCCCN1CCNC(=O)c1cnn(-c2ccccc2)c1C1CCN(C(=O)OC(C)(C)C)CC1,1,0.0000000,0.0000,0.0000000,0.0000,4
31.8345,31.7957,96.1958,0.9999331,COc1ccc2c(c1)c1c(=O)n(N)c(=S)[nH]c1n2C(C)C,1,0.9999331,304.3750,1.0000000,1.0000,4
22.4919,22.4614,105.5383,0.9999971,CCCCOc1ccc2c(c1)C(c1ccc(F)cc1)C(C#N)C(=N)O2,1,0.9999971,338.3820,1.0000000,1.0000,4
40.3546,40.3791,-40.2383,0.0011002,O=C(CC1CC2CCCC1C2)NCCCn1nc(-c2ccc(Cl)cc2Cl)cc1C(=O)Nc1ccc(Cl)cc1,1,0.0011002,573.9520,1.0000000,1.0000,4
42.5189,42.5900,-42.5900,0.0000000,O=C(O)CS(=O)(=O)c1nc(C(=O)N2CCOCC2)nc2sc(CN3CCOCC3)cc12,1,0.0000000,0.0000,0.0000000,0.0000,5
23.3246,23.4071,-23.4071,0.0000000,CC(=O)OCC1CN(C(=O)c2ccccc2)c2ccccc2C(c2ccccc2)=N1,1,0.0000000,0.0000,0.0000000,0.0000,5
27.4569,27.6354,96.4237,0.9692122,Cc1ccc(CN2C(=O)c3ccccc3Oc3ccc(C(=O)NCCc4ccccc4)cc32)cc1,1,0.9692122,462.5490,1.0000000,1.0000,5
19.9347,19.7887,8.2013,0.2186720,N#Cc1ccc(-c2n[nH]c(=N)o2)cc1,1,0.2186720,186.1740,1.0000000,1.0000,5
36.5449,36.1825,91.8165,0.9999923,CC(=Nc1ccc(Cl)cc1[N+](=O)[O-])N1CCSC(C)(C)C1,1,0.9999923,327.8370,1.0000000,1.0000,6
25.3778,25.4466,-25.4466,0.0000000,COC(=O)c1cc(OCc2ccc3ccccc3n2)cc(C(=O)OC)n1,1,0.0000000,0.0000,0.0000000,0.0000,6
38.3508,38.5302,89.4682,0.9999874,CCOc1cc(N2CCC(C)(O)CC2)cnc1OCC(C)(C)C,1,0.9999874,322.4490,1.0000000,1.0000,6
19.0588,19.7478,108.2114,0.9996811,c1ccc(CCNCc2ccc(-c3ccccc3)cc2)cc1,1,0.9996811,287.4060,1.0000000,1.0000,6
21.5394,21.5115,-21.5115,0.0000000,Cc1c(C(=O)O)oc2ccc(Br)cc12,1,0.0000000,0.0000,0.0000000,0.0000,7
34.7336,34.8306,-34.8306,0.0000000,CN1CCc2cccc3c4c([nH]c23)C(=O)c2cc3c(cc2C41)[nH]c1ccccc13,1,0.0000000,0.0000,0.0000000,0.0000,7
29.1114,28.8993,99.0844,0.9998727,CCC(=O)NS(=O)(=O)c1ccc(S(C)(=O)=O)s1,1,0.9998727,297.3790,1.0000000,1.0000,7
21.9754,22.0098,97.9921,0.9375147,CC(=O)Nc1ccc(S(=O)(=O)N2CCC(C(=O)N3CCN(c4ccccc4)CC3)CC2)cc1,1,0.9375147,470.5950,1.0000000,1.0000,7
24.0361,23.8490,103.7794,0.9970970,CC1CCCC(NC(=O)C2CCN(S(=O)(=O)c3cccc4nsnc34)CC2)C1C,1,0.9970970,436.6030,1.0000000,1.0000,8
23.2951,23.5942,104.1223,0.9977854,CCc1ccc(C=CC(=O)NCc2ccccn2)cc1,1,0.9977854,266.3440,1.0000000,1.0000,8
22.5281,22.5278,87.3851,0.8586947,O=C(Nc1ccc(OC(F)(F)F)cc1)N1CCc2ccccc2C1c1ccc(C(F)(F)F)cc1,1,0.8586947,480.4080,1.0000000,1.0000,8
27.1197,27.2749,100.6536,0.9994416,O=C(N=c1cc[nH]n1CCn1ccnc1)c1ccccc1,1,0.9994416,281.3190,1.0000000,1.0000,8

• Training history (Scenario 3):
  This consists of two sub-scenarios:
  • Training the first 4 epochs. This generates a .chkpt file. Result: File D.

Agent,Prior,Target,Score,SMILES,SMILES_state,Molecular weight,Molecular weight (raw),Unwanted SMARTS,Unwanted SMARTS (raw),step
28.8788,28.8788,16.1095,0.3514709,COc1cccc(CN2CCC(N=c3[nH]c(N4CCN(C)CC4)nc4cc(OC)c(OC)cc34)CC2)c1,1,0.3514709,506.6510,1.0000000,1.0000,1
22.5991,22.5991,105.3932,0.9999398,CC(=O)c1cccc(NC(=O)CSCc2nc(-c3ccc(C)cc3)oc2C)c1,1,0.9999398,394.4960,1.0000000,1.0000,1
43.3108,43.3108,-43.3108,0.0000000,COc1ccc2[nH]c3c(c(=O)c2c1)C(CC(=O)OC1c2cc4c(cc2C(c2cc(OC)c(OC)c(OC)c2)C2C(=O)OCC12)OCO4)c1ccccc1-3,1,0.0000000,0.0000,0.0000000,0.0000,1
25.8675,25.8675,101.4203,0.9944361,Fc1cccc(C=Cc2nc3cccnc3s2)c1,1,0.9944361,256.3050,1.0000000,1.0000,1
34.2160,34.2840,-34.2840,0.0000000,COC(=O)CNC(=O)C1CC1C(=O)NC(C#N)Cc1ccc(C#Cc2ccccc2)cc1,1,0.0000000,0.0000,0.0000000,0.0000,2
21.1360,21.1656,106.8342,0.9999985,Cc1cc(C)cc(-n2ncc3c2CCCC3NC(=O)c2ccccc2)c1,1,0.9999985,345.4460,1.0000000,1.0000,2
18.7809,18.7439,109.2448,0.9999117,COc1cc(C(=O)NCc2ccccc2)cc(OC)c1OC,1,0.9999117,301.3420,1.0000000,1.0000,2
25.7743,25.8302,101.6824,0.9961920,O=C1CCc2ccc(C(=O)N(c3ccccc3)C3CCN(Cc4ccccc4)CC3)cc2N1,1,0.9961920,439.5590,1.0000000,1.0000,2
31.5609,31.4528,70.1004,0.7933842,O=C(N=c1nn[nH][nH]1)c1cc(N2CCN(c3ccc(C(F)(F)F)cc3)CC2)cc(C(F)(F)F)c1,1,0.7933842,485.3920,1.0000000,1.0000,3
25.5686,25.5522,36.4973,0.4847616,CCC(CC)C(O)CC1CCNCC1,1,0.4847616,199.3380,1.0000000,1.0000,3
30.5101,30.4866,97.5116,0.9999861,Cc1ncc(C(=O)N2CCC3(CC2)NC(=O)C32CCOC2)s1,1,0.9999861,321.4020,1.0000000,1.0000,3
26.3633,26.3665,101.6328,0.9999946,Brc1ccccc1Oc1cccc(CNCc2ccccc2)c1,1,0.9999946,368.2740,1.0000000,1.0000,3
24.2177,24.0738,-24.0738,0.0000000,CC1CCCCN1CCNC(=O)c1cnn(-c2ccccc2)c1C1CCN(C(=O)OC(C)(C)C)CC1,1,0.0000000,0.0000,0.0000000,0.0000,4
31.8345,31.7957,96.1958,0.9999331,COc1ccc2c(c1)c1c(=O)n(N)c(=S)[nH]c1n2C(C)C,1,0.9999331,304.3750,1.0000000,1.0000,4
22.4919,22.4614,105.5383,0.9999971,CCCCOc1ccc2c(c1)C(c1ccc(F)cc1)C(C#N)C(=N)O2,1,0.9999971,338.3820,1.0000000,1.0000,4
40.3546,40.3791,-40.2383,0.0011002,O=C(CC1CC2CCCC1C2)NCCCn1nc(-c2ccc(Cl)cc2Cl)cc1C(=O)Nc1ccc(Cl)cc1,1,0.0011002,573.9520,1.0000000,1.0000,4

• Training starting from the checkpoint resulting from the first 4 epochs. Result: File E.

Agent,Prior,Target,Score,SMILES,SMILES_state,Molecular weight,Molecular weight (raw),Unwanted SMARTS,Unwanted SMARTS (raw),step
23.0097,28.8788,16.1095,0.3514709,COc1cccc(CN2CCC(N=c3[nH]c(N4CCN(C)CC4)nc4cc(OC)c(OC)cc34)CC2)c1,1,0.3514709,506.6510,1.0000000,1.0000,1
18.5941,22.9442,105.0465,0.9999275,COc1ccc(C)cc1NC(=O)CSCc1nc(-c2ccc(C)cc2)oc1C,1,0.9999275,396.5120,1.0000000,1.0000,1
43.3685,43.3108,-43.3108,0.0000000,COc1ccc2[nH]c3c(c(=O)c2c1)C(CC(=O)OC1c2cc4c(cc2C(c2cc(OC)c(OC)c(OC)c2)C2C(=O)OCC12)OCO4)c1ccccc1-3,1,0.0000000,0.0000,0.0000000,0.0000,1
21.8165,25.8675,101.4203,0.9944361,Fc1cccc(C=Cc2nc3cccnc3s2)c1,1,0.9944361,256.3050,1.0000000,1.0000,1
31.8270,31.8698,-31.8698,0.0000000,N#CC(Cc1ccc(C#Cc2ccccc2)cc1)NC(=O)C1CC1C(=O)NCC(F)(F)F,1,0.0000000,0.0000,0.0000000,0.0000,2
18.0534,21.1656,106.8342,0.9999985,Cc1cc(C)cc(-n2ncc3c2CCCC3NC(=O)c2ccccc2)c1,1,0.9999985,345.4460,1.0000000,1.0000,2
17.7096,18.7439,109.2448,0.9999117,COc1cc(C(=O)NCc2ccccc2)cc(OC)c1OC,1,0.9999117,301.3420,1.0000000,1.0000,2
21.4345,25.8302,101.6824,0.9961920,O=C1CCc2ccc(C(=O)N(c3ccccc3)C3CCN(Cc4ccccc4)CC3)cc2N1,1,0.9961920,439.5590,1.0000000,1.0000,2
33.0768,33.3720,94.6275,0.9999965,Cc1ccccc1CN1CCCC(N(C)C(=O)c2cccnc2C2CC2)C1,1,0.9999965,363.5050,1.0000000,1.0000,3
37.5463,37.5956,90.4016,0.9999781,Cc1cc(NC(=O)NC(=O)Cc2ccccc2)ccc1SCCN1CCC1,1,0.9999781,383.5170,1.0000000,1.0000,3
36.9132,37.0145,58.3782,0.7452553,O=c1[nH]cnc(O)c1C(c1ccc(Br)cc1)c1cn(Cc2cccc3ccccc23)nn1,1,0.7452553,488.3450,1.0000000,1.0000,3
37.6607,37.5283,90.4695,0.9999827,Cc1ccc(C(C)(C)n2cc(CCN3CCCC3)c3ccc(Cl)cc32)cc1,1,0.9999827,380.9630,1.0000000,1.0000,3
21.1501,21.1560,14.0820,0.2752963,O=C(C=Cc1ccc(Cl)c(Cl)c1)NC1CCC(CN2CCC(c3c[nH]c4ccccc34)CC2)CC1,1,0.2752963,510.5090,1.0000000,1.0000,4
33.7879,33.6017,94.3930,0.9999584,CC(C)(C)c1ccc(-c2cn(C3CN(CCO)CC(O)(CO)C3O)nn2)cc1,1,0.9999584,390.4840,1.0000000,1.0000,4
36.7092,36.6592,81.0195,0.9193643,Cn1cncc1C(O)(c1ccccc1)c1ccc2ccc(OCCc3c[nH]c4ccccc34)cc2c1,1,0.9193643,473.5760,1.0000000,1.0000,4
27.0402,26.9309,99.5875,0.9884255,Nc1ccc2c(C(=O)NCCCCO)[nH]nc2c1,1,0.9884255,248.2860,1.0000000,1.0000,4

I have used reinvent.prior as the prior, and therefore it was also used as the agent, except for the case (file E) where I used the checkpoint from the first 4 epochs as the agent. As can be seen, file E does not continue the training correctly. Just by looking at the first generated SMILES, which is COc1cccc(CN2CCC(N=c3[nH]c(N4CCN(C)CC4)nc4cc(OC)c(OC)cc34)CC2)c1 instead of O=C(O)CS(=O)(=O)c1nc(C(=O)N2CCOCC2)nc2sc(CN3CCOCC3)cc12. How could I successfully continue the learning process? Is Reinvent adapted for this?

Thank you very much in advance.

[halx]

REINVENT uses probabilistic models and as such is non-deterministic. Other than with setting the seed (to some extent) and using beam search with the transformers you cannot expect repeat runs to produce the same output. And that is perfectly fine. In fact, decorrelation will happen fairly quickly and what you should really do in practice, but probably few people do, is to combine results from multiple independent runs.

Set up a realistic example with realistic settings and check that for yourself.