Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WIP Breaking Change for DataValidation #4240

Draft
wants to merge 4 commits into
base: master
Choose a base branch
from

Conversation

oleibman
Copy link
Collaborator

Fix #797 (marked stale in 2018, but now reopened). Fix #4091. DataValidation is specified in the Xml at the sheet level rather than the cell level. PhpSpreadsheet, however, currently requires a cell and a data validation object for each cell in a range. As a consequence, it may exhaust memory allocating objects that are never really needed. If it reads a spreadsheet which applies DataValidation to an entire column, the reader creates a million cells and a million DataValidation objects. An additional problem is that, when it saves the spreadsheet, it creates a row entry for each of the million rows even if they contain no data.

The code is changed to locate DataValidation for a cell by running the DataValidation list to see if the Sqref for any entry matches the cell in question. This eliminates the need to require a DataValidation entry for each cell; indeed, it eliminates the need to create a cell just because it is subject to DataValidation. Initial tests are very encouraging. The memory-exhausting spreadsheet from 797 required over 1GB of memory. With this change in place, that is reduced to 6MB, and DataValidation works as expected for all cells in the range.

This change is a work in progress. It needs formal tests. I need to see how it works with inserting or removing rows and columns. I need to see how Excel handles conflicting entries. My early tests indicate that it at least doesn't do anything irrational (like claiming corruption) when dealing with conflicts. It appears that, if I have a DataValidation applied to column A, and another applied to cell A5, whichever appears first in the Xml is the ultimate arbiter of what rule applies to A5.

If this change is merged, it will be a breaking range. It will require a new major release (probably PhpSpreadsheet 4.0), and will not happen for at least a couple of months, and probably later.

This is:

  • a bugfix
  • a new feature
  • refactoring
  • additional unit tests

Checklist:

  • Changes are covered by unit tests
    • Changes are covered by existing unit tests
    • New unit tests have been added
  • Code style is respected
  • Commit message explains why the change is made (see https://github.com/erlang/otp/wiki/Writing-good-commit-messages)
  • CHANGELOG.md contains a short summary of the change and a link to the pull request if applicable
  • Documentation is updated as necessary

Why this change is needed?

Provide an explanation of why this change is needed, with links to any Issues (if appropriate).
If this is a bugfix or a new feature, and there are no existing Issues, then please also create an issue that will make it easier to track progress with this PR.

Fix PHPOffice#797 (marked stale in 2018, but now reopened). Fix PHPOffice#4091. DataValidation is specified in the Xml at the sheet level rather than the cell level. PhpSpreadsheet, however, currently requires a cell and a data validation object for each cell in a range. As a consequence, it may exhaust memory allocating objects that are never really needed. If it reads a spreadsheet which applies DataValidation to an entire column, the reader creates a million cells and a million DataValidation objects. An additional problem is that, when it saves the spreadsheet, it creates a row entry for each of the million rows even if they contain no data.

The code is changed to locate DataValidation for a cell by running the DataValidation list to see if the Sqref for any entry matches the cell in question. This eliminates the need to require a DataValidation entry for each cell; indeed, it eliminates the need to create a cell just because it is subject to DataValidation. Initial tests are very encouraging. The memory-exhausting spreadsheet from 797 required over 1GB of memory. With this change in place, that is reduced to 6MB, and DataValidation works as expected for all cells in the range.

This change is a work in progress. It needs formal tests. I need to see how it works with inserting or removing rows and columns. I need to see how Excel handles conflicting entries. My early tests indicate that it at least doesn't do anything irrational (like claiming corruption) when dealing with conflicts. It appears that, if I have a DataValidation applied to column A, and another applied to cell A5, whichever appears first in the Xml is the ultimate arbiter of what rule applies to A5.

If this change is merged, it will be a breaking range. It will require a new major release (probably PhpSpreadsheet 4.0), and will not happen for at least a couple of months, and probably later.
@oleibman
Copy link
Collaborator Author

Existing code for ReferenceHelper adjusts sqref for DataValidation when rows/columns inserted/deleted, but does not adjust formula (where list may be expressed as a range). That is a bug, and I think it needs to be fixed as part of this PR.

Also add first tests that DV formulas are updated correctly after row/column insert/delete. Many more tests are still needed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
1 participant