expose intermediate outputs in map calculation

[johnarevalo]

By disaggregating the run_pipeline function in a modular way

[shntnu]

@johnarevalo Does this issue also include the following feature?

I want to be able to know the number of positive and negative examples per row (which in turn is used to calculate the null)

[johnarevalo]

Yes. I think the values you are looking for are in null_confs variable inside run_pipeline:

copairs/src/copairs/map.py

Line 133 in 491ebaf

ap_scores, null_confs = compute.compute_ap_contiguos(rel_k_list, counts)

As quick workaround, you can copy the run_pipeline code in a separate script to inspect all the intermediate outputs.

[shntnu]

As quick workaround, you can copy the run_pipeline code in a separate script to inspect all the intermediate outputs.

Thanks!

For our notes, here's what I did: https://github.com/broadinstitute/neuronal-cell-painting/blob/81bef05320e93c33f18f5587863845be63593697/ncp/src/my_run_pipeline.py

[shntnu]

Update: I figured there's no downside to doing this:

https://github.com/broadinstitute/neuronal-cell-painting/blob/2486dfa6402154192d4dd127bb000bc1f9755f7a/ncp/src/my_run_pipeline.py#L65-L68

[johnarevalo]

@shntnu , I forgot to mention null_confs is a list of [num_positive, total], not [num_positive, num_negative].

PD: I just merged a typo Alan fixed. now it is compute_ap_contiguous.

[johnarevalo]

n_pos_pairs and n_total_pairs are now available in v0.3.1.

copairs/src/copairs/map.py

Lines 159 to 160 in 11570eb

	meta["n_pos_pairs"] = null_confs[:, 0]
	meta["n_total_pairs"] = null_confs[:, 1]