log scale 2D data with zeros and negative values not plotted correctly (Trac #1220)

[pkienzle]

Sasview 4.2

Zero and negative values not plotted appropriately in log scale for 2D data.

Plotter should add invalid values to the mask.

Could instead use a transform something like the following:

data = data.copy()
zmin = min(data[data>0])/2 if (data>0).any() else 1
data[data<=zmin] = zmin
return log10(data)

If you know it is counts data, then log10(data+1) is good enough for visualization.

You may still have a problem with spurious cancellation, for example when looking at background subtracted data, where you might end up with values like 1e-100 while the majority of the values are much higher. Setting the cutoff for masking at 1/2 the 5th percentile, np.quantile(data[data>0], 0.05)/2, instead of min(data[data>0])/2 would be better.

Migrated from http://trac.sasview.org/ticket/1220

{
    "status": "closed",
    "changetime": "2019-03-05T17:33:51",
    "_ts": "2019-03-05 17:33:51.016917+00:00",
    "description": "Sasview 4.2\n\nZero and negative values not plotted appropriately in log scale for 2D data.\n\nPlotter should add invalid values to the mask.\n\nCould instead use a transform something like the following:\n{{{\ndata = data.copy()\nzmin = min(data[data>0])/2 if (data>0).any() else 1\ndata[data<=zmin] = zmin\nreturn log10(data)\n}}}\n\nIf you know it is counts data, then `log10(data+1)` is good enough for visualization.\n\nYou may still have a problem with spurious cancellation, for example when looking at background subtracted data, where you might end up with values like 1e-100 while the majority of the values are much higher.  Setting the cutoff for masking at 1/2 the 5th percentile, `np.quantile(data[data>0], 0.05)/2`, instead of min(data[data>0])/2 would be better.\n",
    "reporter": "pkienzle",
    "cc": "",
    "resolution": "fixed",
    "workpackage": "McSAS Integration Project",
    "time": "2018-12-11T18:18:07",
    "component": "SasView",
    "summary": "log scale 2D data with zeros and negative values not plotted correctly",
    "priority": "major",
    "keywords": "",
    "milestone": "SasView 4.3.0",
    "owner": "GitHub <[email protected]>",
    "type": "defect"
}

[sasview-bot]

Trac update at 2018/12/11 21:14:30: krzywon commented:

In sasgui/plottools/PlotPanel.py, lines 1460 and 1469 that handle negative values for log plotting are commented out. Removing the comments makes the plots look much better.

The percentile suggestion will take some extra work to handle values between 0 and 1 (log10(1<n<0) is a negative number).

[pkienzle]

Trac update at 2018/12/12 19:12:14: pkienzle commented:

See sasview [[https://github.com//pull/202 | PR #350]].

This uses the min(data>0) as the cutoff rather than the 5th percentile, but it includes a note on how to change from one to the other.

[sasview-bot]

Trac update at 2019/03/05 17:33:51:

• GitHub [email protected] commented:

In changeset 0a924c6:

#!CommitTicketReference repository="sasview" revision="0a924c6e6f8f8cc1aa158646b7939c1045aba95d"
Merge pull request #350 from SasView/ticket-1220

fix 2D log-scale plotting range when data contains zeros and negative values

as agreed at today's meeting
fixes #1249

• GitHub [email protected] changed owner from "" to "GitHub [email protected]"
• GitHub [email protected] changed resolution from "" to "fixed"
• GitHub [email protected] changed status from "new" to "closed"