question with Running libinvent in REINVENT 4

[jackhu5]

hello!

I have been working with REINVENT 4 and tried constructing a parameter file for reinforcement learning (RL) using the provided notebook and template configuration file for RL. The parameter file for stage_1 was successfully created and executed. However, I encountered some issues while reviewing the results.

I noticed that some molecules with good original docking scores and QED values, and without any structural alerts, were given a score of 0. Upon comparing the ligand-pocket complexes, I believe these are actually good molecules. This leads me to think that my parameter file might be incorrectly set up and that libinvent was not successfully deployed.

Could you please provide me with some guidance on this matter? Thank you for your assistance!

Best regards,
Jack Hu

here is the toml:

device = "cuda:0"
tb_logdir = "tb_stage1"
json_out_config = "_stage1.json"

[parameters]

prior_file = "/root/reinvent4/REINVENT4-4.3.5/priors/libinvent.prior"
agent_file = "/root/reinvent4/REINVENT4-4.3.5/priors/libinvent.prior"
smiles_file = "/root/reinvent4/abc1/Rgroup/54.smi"
summary_csv_prefix = "stage_learning"

batch_size = 100
use_checkpoint = false
purge_memories = false

[learning_strategy]

type = "dap"
sigma = 128
rate = 0.0001

[diversity_filter]  

type = "IdenticalMurckoScaffold" 
bucket_size = 25                
minscore = 0.4                   
minsimilarity = 0.4              
penalty_multiplier = 0.5         

[[stage]]

max_score = 1.0
max_steps = 50

chkpt_file = "stage1.chkpt"

[stage.scoring]
type = "geometric_mean"
scoring_function.type = "custom_product"

[[stage.scoring.component]]
[stage.scoring.component.custom_alerts]

[[stage.scoring.component.custom_alerts.endpoint]]
name = "Alerts"

params.smarts = [
    "[*;r8]",
    "[*;r9]",
    "[*;r10]",
    "[*;r11]",
    "[*;r12]",
    "[*;r13]",
    "[*;r14]",
    "[*;r15]",
    "[*;r16]",
    "[*;r17]",
    "[#8][#8]",
    "[#6;+]",
    "[#16][#16]",
    "[#7;!n][S;!$(S(=O)=O)]",
    "[#7;!n][#7;!n]",
    "C#C",
    "C(=[O,S])[O,S]",
    "[#7;!n][C;!$(C(=[O,N])[N,O])][#16;!s]",
    "[#7;!n][C;!$(C(=[O,N])[N,O])][#7;!n]",
    "[#7;!n][C;!$(C(=[O,N])[N,O])][#8;!o]",
    "[#8;!o][C;!$(C(=[O,N])[N,O])][#16;!s]",
    "[#8;!o][C;!$(C(=[O,N])[N,O])][#8;!o]",
    "[#16;!s][C;!$(C(=[O,N])[N,O])][#16;!s]"
]

[[stage.scoring.component]]
[stage.scoring.component.QED]
[[stage.scoring.component.QED.endpoint]]
name = "QED"
weight = 1


[[stage.scoring.component]]
[[stage.scoring.component.DockStream.endpoint]]
name = "Docking into 5D7E grid"
weight = 1
params.configuration_path = "/root/reinvent4/myt1/Rgroup/dockstream_rl_direct_uncs.json"
params.docker_script_path = "/root/dockstream1/DockStream-1.0.0/docker.py"
params.docker_python_path = "/root/miniconda3/envs/DockStream/bin/python"
transform.type = "reverse_sigmoid"
transform.high = -7
transform.low = -13.5
transform.k = 0.2

this is the confusing result:

this is the smi:
CNC1=NC=CC(C2=C([*:1])C(C#N)=C(C)N2)=N1

[jackhu5]

bty, I observed that in the demo notebook, the target is described as the negative log likelihood of augmentation, implying that the target value should be greater than -σ. However, in the output CSV file, the target values tend to increase with each iteration, suggesting that the augmented log likelihood is being used instead.

[halx]

Your are using a scaffold diversity filter (DF) which is conflated with duplicate counting. Duplicates are scores with zero. With Libinvent you need to be careful with the scaffold DF because you pre-define scaffolds which leaves just so many ways to escape those scaffolds (ring addition/deletion, atom swaps, ring size changes).

"Target" is synymous with augmented likelihood. NLLs, contrary to their names, are positive. See also eq 5 int the paper and the following discussion.