How to replace all visible and virtual items following a given one (was: White space at the bottom)

[jyothi530]

Hi @dhilt

I wanted to use ngx-ui-scroll. My requirement is having items with variable height. Your library is a good fit for my requirement but have some issues that I wanted to solve and need your suggestions.

1. uiScroll is leaving some white space at the bottom or top when viewport is not the immediate parent of element hosting uiScroll.
   On the settings object I see the viewPortElement property exposed which can take either HTMLElement or Function. Is it suggested to use that approach. Using that I am able to get rid of whitespace in the bottom and also no double scroll bar
2. I have to add/remove items dynamically to the datasource buffer. I am using adapter API's remove() and insert(). This works fine but I observed after using remove() and insert() in the DOM the items are not virtualized. I have to call clip() but that is causing inconsistent behavior.
   (Eg: You can relate the requirement to a scenario where there will be a certain set of questions. Answering a question can show/hide other questions)

Can you please suggest me the best solution.

Thanks for the wonderful library.

[dhilt]

@jyothi530 1. Sure, use viewportElement setting. It is marked as experimental due to no restrictions on the lib's side, while essentially there are some: viewport element must be a parent of a scroller element. But it is being used and will be fully backward-compatible at least until ngx-ui-scroll v2, and most likely after v2 (by the way, no plans for v2 for now).

2. Would be nice to have a demo of the issue (like this stackblitz), but the first thing that comes to my mind is that you need to check if you are protecting the Adapter instructions flow. You may read about it in the Adapter return value demo and in the Adapter relax method demo, but in the code it should look like the following:

async doInsert (params) {
  const { adapter } = this.myDatasource;
  await adapter.relax();
  await adapter.insert(params);
  await adapter.clip();
}

I'm glad to hear the lib is useful and happy to assist!

[jyothi530]

@dhilt Thanks for the quick reply. I am using exactly like the above code snippet you mentioned. Will revert with a stackblitz sample.

[jyothi530]

Hi @dhilt, Here is the stackblitz url (https://stackblitz.com/edit/angular-ivy-qm8h6g). I created an array with 100 elements. I am inserting a set of items after index 6. To perform this operation I am removing all the items after index 6. Adding the new items along with items that are removed after index 6 with insert() operation.
I know I don't have to perform remove() operation but in the sample I am trying to simulate my requirement where I have to remove/insert elements at different index positions after a particular start index (eg: one item at index 7, two items at index 10, remove one from index 11, insert 3 at index 15, etc). Removing all elements after the start index and reinserting the new array with newly added/removed is easy approach so planning to go that route.
Coming to the issues:

1. In the stackblitz sample what I observed is without clip() operation when I perform remove() and insert(). I can see the new items added after index 6 but also observed scrolling the entire list at the bottom I see elements again repeated for index 6 to 20. Also the list in the DOM is not virtualised. (The elements 6 to 20 are seen at the bottom if I haven't scrolled the entire list. When I scroll to the end of list and clicking the button again will not generate items 6 to 20)
2. I read using clip() will virtualise. After using clip() as expected the elements in the DOM are virtualised but scrolling to the bottom of list and scrolling up I observed the newly added elements are not there.
3. The other issue I wanted to point out which I observed in my application but not presented in stackblitz sample is when I used clip(), Let's say I have 9 elements initially. I want to add 1 element after index 2 (I performed remove all after index 2 and then insert new element along with removed after index 2 because of my requirement explained above). Then I used clip(). My list should have 10 elements but what I observed is my list have only 9 elements with newly added element but the element at index 3 is removed. Similarly if I perform the same operation repeatedly I always see only 9 elements with the new items seen but not the old ones.
   Step0: 1 2 3 4 5 6 7 8 9 10
   Insert new '1X' element at index 3
   Step 1: Expected: 1 2 1X 3 4 5 6 7 8 9 10 , Observed: 1 2 1X 4 5 6 7 8 9 10
   Insert new '2X' element at index 4
   Step 1: Expected: 1 2 1X 2X 3 4 5 6 7 8 9 10 , Observed: 1 2 1X 2X 5 6 7 8 9 10

Psuedo code for above operation

await adapter.relax();
await adapter.remove(predicate);
await adapter.relax();
await adapter.insert({items, after:predicate});
await adapter.relax();
await adapter.clip();

Without clip() I can see expected behavior but the items are not virtualised

[dhilt]

@jyothi530 Great! The demo is very helpful. I see two major problems with the approach you implemented there.

First, is on the App side. All changes made over the Scroller must be synced with the Datasource. We have 2 data levels: Scroller's Buffer (including virtual data) and App's Datasource (which provides data to the Scroller on demand, per scroll events). If you want to make any change in the Scroller's data, you need also to update the App Datasource. That's why you see duplications. This problem needs to be addressed anyway, because this is the main point of consistency.

Second, is on the Scroller's end. The remove process (triggering by Adapter.remove) performs a full cycle of the UIScroll Workflow: cut the required items off and ask for new items to fill the viewport. These new items are being taken via Datasource and are being rendered before you call Adapter.insert. I think, this problem can be solved, but we need to discuss it.

For example, you may try to change Datasource in the same manner as you make your updates via Adapter: cut off DS items, call Adapter remove, insert DS items, call DS insert. I'm not sure, but this might require also managing min/max indexes (via Adapter.fix) because the internal request to the data after Adapter.remove will update them in accordance with DS update made right before Adapter.remove.

Another option is to deal with visible items only. You know the pack you want to update, so you may split it. Say, there was 1...100 items initially, 1..6 visible items should remain pristine, 6..20 visible items needs to be changed, and there will be 20..90 virtual items because 10 virtual items should gone. First, you need update the Datasource, for DS.get should return index-count portion properly after the update; it should manage [1..90] items. Then you make 2 steps: update 6..20 items via Adapter (since ngx-ui-scroll v1.11.0-rc.1 you may use Adapter.replace for this purpose, it can replace items in the Scroller's Buffer in a single run) and shift maxIndex via Adapter.fix({ maxIndex: 90 }). This shifting is necessary only if the Scroller knows the right border of the data at the moment of these changes. It means, if the Scroller knew it can get 1..100 items via DS, and then you updated the DS to have only [1..90] items, then you need to provide this information to the Scroller by reducing maxIndex, otherwise he will show you scrollbar for 1..100. This is very rough thinking, but I hope it will be enough to give it some try.

Post another version of demo if any questions or concerns, I will take a look.

[dhilt]

@jyothi530 This depends on the App requirements. There are some examples, like this one, but the main idea is that the Datasource.get should serve the real data you want the App to deal with.

[dhilt]

@jyothi530 Well, I spent some time to accomplish virtual replacement demo based on your case:

https://stackblitz.com/edit/ngx-ui-scroll-virual-replacement

It implements the last option from my previous comment with no extra render. Conditions: the dataset boundaries are known, indexes start from 0. Current min/max Buffer indexes are calculated via Adapter.fix-updater API, which does not seem intuitive, so I created separate issue #236 for this. I hope the concept does make sense, and you can apply it in your environment. Indexes consistency is the first point you need to care of, and you have to maintain it across the Datasource-Buffer updates of any complexity.

Virtual replacement is very interesting use case. Currently the ngx-ui-scroll has limited API which can't fulfil this task in short and strict manner. But it is possible, and my demo shows it.

[jyothi530]

@dhilt Thank you for the demo app. I actually applied the first option in your previous comment in my application. It worked the way I expected. There was one scenario when if I perform the operations (remove then insert then clip) repeatedly say more than 100 times I got "Maximum call stack size reached". I tried to replicate the issue in the demo app to show you but was unsuccessful (Might be since demo app is lightweight and my actual application will be having other events that will be firing).
I thought of trying the second option as well over weekend or Monday but little bit concerned since the replace() API is not part of stable candidate and in rc to use in my application. I will definitely try the second option since it seems lot simpler than first and let you know the results.
I really appreciate your quick responses with demo app when needed. Thank you so much for your support

[jyothi530]

I think the title of the issue is not appropriate with the discussion we are having in this thread. I came up with that title when I use the viewport element that is not immediate parent of uiScroll. But figured out in settings of Datasource I can pass a function that determines the viewPortEle.
Please suggest/update the title of the issue

[dhilt]

@jyothi530 👍 By the way ngx-ui-scroll v1.11.0 is almost done, it will be released during the next week. The rc versions appeared due to the request for one-to-one replacement feature, someone couldn't wait for many-to-many is done.