Test Spherize more extensively

[shntnu]

Spherize only tests for the default epsilon

pycytominer/pycytominer/tests/test_operations/test_transform.py

Lines 21 to 40 in dfd355d

	def test_spherize():
	spherize_methods = ["PCA", "ZCA", "PCA-cor", "ZCA-cor"]
	for method in spherize_methods:
	for center in [True, False]:
	scaler = Spherize(method=method, center=center)
	scaler = scaler.fit(data_df)
	transform_df = scaler.transform(data_df)
	# The transfomed data is expected to have uncorrelated samples
	result = (
	pd.DataFrame(np.cov(np.transpose(transform_df)))
	.abs()
	.round()
	.sum()
	.clip(1) # necessary for when center == False (numerically unstable)
	.sum()
	)
	expected_result = data_df.shape[1]
	assert int(result) == expected_result

[shntnu]

Hm – actually I don't think the test is correct
https://colab.research.google.com/drive/1KitZF-CgV_xgZpd1n0BieQpko-uD4ZQA?usp=sharing

[gwaybio]

i don't have any idea what that collab notebook means!

@shntnu - this is quickly becoming a pretty critical piece in the lincs cell painting dataset. I'd like to solve it once and for all. see broadinstitute/lincs-cell-painting#60.

Does the notebook mean the test needs to be fixed? Or that the test is showing that the spherize function is incorrect in some way? Does the notebook propose a new test?

[gwaybio]

In #132, I added a spherize_epsilon parameter. When we close this issue, we can address this variable here as well.

[shntnu]

Sorry for leaving this stray comment without an explanation :D

The notebook shows that this (covariance) matrix will pass the test even though it is not spherized.

But as we discussed, you needn't do more here. The code looks right!

[shntnu]

Oh I just noticed this #127 (comment)

nvm – feel free to reopen or use a new issue, as you see fit @gwaygenomics