[help] used in multiple tabs and support for mark imags by programmatically | 能否在多个选项卡里同时使用，以及标记功能的支持

[WSH032]

Used in multiple tabs

@picobyte and I are discussing how to embed iib in our own gradio extension tab

My current understanding is that in A1111 WebUI, iib interacts with the backend through JavaScript, which requires a fixed elem_id of gradio component

I am trying to create the same gradio.HTML component in another extension, which can indeed embed iib in another extension

but it has resulted in the original iib being unable to be used.

Is there any way to achieve this?

---

Mark imags

Our extension can provide images tagger, such as duplicate images or restricted level images.

Can iib identify these markings for us? We can provide a txt file with the same name of images, a JSON file, or a Python interface

Especially for duplicate image, we are not satisfied with the gradio gallery components

[zanllp]

The following are my ideas, and I believe they are all achievable, at least that's the case for me.

Start the backend server and configure it to use "foo" as the base.

AppUtils(base = "foo").wrap_app(app)

Add a container for IIB

+ gr.HTML("error", elem_id="bar_iib_container")
- gr.HTML("error", elem_id="infinite_image_browsing_container_wrapper")

Load index.js on the browser side.

const jscodeResp = await fetch("/file?path=/path/to/your/submodue-iib/index.js") // fake api
const jscode = await jscodeResp.text()
eval(jscode.replace("__root_conatiner__", "#bar_iib_container").replace("__base__", "foo"))

If you want to mark an image, you can achieve it by using postmessage, and the same goes for other operations.

const iib = gradioApp().querySelector('#bar_iib_container iframe').contentWindow

iib.postMessage({
  type: 'mark-images',
  data: [  { path: 'path1', tags: ['tag1', 'tag2'] } ,  { path: 'path2', tags: ['tag3', 'tag4'] }  ]
})

[picobyte]

Thank you for your answer, I will try it out. I noticed that Sean Wang implemented something similar, maybe something for css is required, but I will find out.

[zanllp]

By the way, it should be noted that these are just some feasible solutions, but I haven't implemented them on IIB yet. It may take a few days as I am currently quite busy.

[picobyte]

I made a fork of your repository, maybe I can figure it out.

[zanllp]

Okay, if you complete it successfully, you can consider submitting a PR to merge it into my repository.

[WSH032]

My question is, if the user has already installed the iib extension on A1111-WebUI, do we still need to enable the submodule-iib. I mean, enable the submodule-iib as standalone mode

script_callbacks.on_app_started(
    AppUtils().wrap_app
)

If need, will there be no conflicts ?
If not, can we reuse the iib A1111 extension that has already been installed

[zanllp]

Both methods are feasible. I don't think there will be conflicts even if it is installed as a submodule. The advantage of using iib as a submodule is that the version can be fixed. In addition, with the variables "base" and "root_container" isolation can be achieved to a large extent.

[WSH032]

If we want to reuse the installed iib extension, can we do something like these:

I noticed that iib implemented the jump to specified folder function in the update on 2023-6-24.

My extension performs clustering and duplication detection by folder (my extension does not support recursive processing of subfolders), and every time the clustering or duplication detection results come out, I send a jump request to iib, which will jump to the folder being processed if the current iib version supports.

At the same time, I send the clustering or duplication detection labels of each image in the folder to iib, and iib displays these labels temporarily, and once exited, these labels will be cleared.

如果要复用已经安装的iib扩展，是否能这样：
我注意到iib在2023-6-24的更新中，实现了跳转至指定文件夹功能。我的扩展在进行聚类和查重时是以文件夹为单位的（我的扩展不支持递归处理子文件夹），每当聚类或者查重结果出来后，我向iib发送一次跳转请求，如果当前iib版本支持的话，将会跳转至正在处理的文件夹。同时我向iib发送该文件夹内每张图片的聚类或查重标签，iib以临时的方式显示这些标签，一旦退出后，这些标签将被清除。

---

Or can we have a temporary workspace for users to process, like the update on 2023-7-22, where I will send the file paths and their labels of the images being processed to iib’s temporary workspace, just like sending them to gradio.Gallery.

或者我们是否能像2023-7-22的更新那样，有一个临时工作区提供给用户进行处理，我会向iib的临时工作区发送所处理的文件路径以及他们的标签，就像发送给gr.Gallery那样

---

This is the gallery that I have implemented in my extension.

[zanllp]

@WSH032

2023-7-22的更新 是指的那个批量下载的功能的功能把

不然直接把那些函数暴露给好了

AppUtils(base = "foo", export_frontend = True ).wrap_app(app)

const iib = gradioApp().querySelector('#bar_iib_container iframe').contentWindow;

iib.insertTabPane({
    tabIdx: 0,
    paneIdx: 0,
    pane: {
        type: 'gridView',
        files: [{
            size: '',
            type: 'file',
            created_time: '',
            name: 'xxxxx.png',
            date: '',
            bytes: 0,
            fullpath: '/path/to/xxx.png',
            tags: [{text: 'foo'}, {text: 'bar'}]
        }]
    }
});




iib.insertTabPane({
  tabIdx: 0,
  paneIdx: 0,
  pane: {
    type: 'local',
    path: '/path/to/your/target/directory'
  }
});

[WSH032]

@zanllp
抱歉回复晚了

---

2023-7-22的更新 是指的那个批量下载的功能的功能把

Yes, the archive tab. I think it’s also a good idea to use it as a temporary workspace, so that I don’t have to worry about cleaning up the temporary tags. User can just clear all the images after finishing. But when displaying the clustering results, I might use multiple archive tabs.

对的，就是那个归档，我觉得把它当成临时工作区也不错，这样就不用考虑临时标签需要清除的问题了，处理完直接把图片全清空就行。
但显示聚类结果时候，我可能会用到多个归档选项卡。

---

不然直接把那些函数暴露给好了

So, you recommend us to use the js method to implement adding tabs and passing image paths with tags, right?
This is my current implementation
I noticed that the insertTabPane method is not implemented yet, right? But at least iib.alert() works.

就是推荐我们使用 js 方法来完成 添加选项卡 和 传递带有标签的图片路径 是吗？
这是我目前的实现
我注意到 insertTabPane 方法还没实现对吗？ 但至少 iib.alert() 是能工作的

---

AppUtils(base = "foo", export_frontend = True ).wrap_app(app)

My extension may not have an iib submodule, I’m considering reusing the running iib extension
picobyte may use the standalone mode
Are both feasible?

我的扩展可能不会内置一个子模块的iib，我考虑复用正在运行的iib
picobyte可能会使用standalone模式
两者都是可行的吗

[zanllp]

So, you recommend us to use the js method to implement adding tabs and passing image paths with tags, right?
This is my current implementation
I noticed that the insertTabPane method is not implemented yet, right? But at least iib.alert() works.

Yes, we should agree on some APIs, and then I don't care about the implementation on your side, similar to the bridge between native and web. If you need any API, you can ask me to add it or you can submit a pull request yourself.

I haven't implemented the specific APIs yet, so it's more like an RFC. As @picobyte mentioned earlier, he wanted to try modifying it first, but I'm not sure about his progress.

However, the frontend of IIB is quite complex, and I feel that modifying it won't be an easy task if you haven't specialized in frontend development. If you encounter any difficulties, you can always hand over this work to me.

My extension may not have an iib submodule, I’m considering reusing the running iib extension
picobyte may use the standalone mode
Are both feasible?

I think this is feasible

[WSH032]

Yes, we should agree on some APIs

Yes, I totally agree with that. Don’t worry about the implementation on my side. Whether it’s python, js, http request or any other api, I will make the necessary adjustments.

是的，我十分同意这一点。不用在乎我这边的实现，无论是python, js, http request 或者别的api，我都会去做相应的调整的。

---

However, the frontend of IIB is quite complex, and I feel that modifying it won't be an easy task if you haven't specialized in frontend development. If you encounter any difficulties, you can always hand over this work to me.

Thank you very much. I know nothing about frontend development, I will wait patiently. I will do some other work first, don’t mind me.

十分感谢你。我对前端开发一无所知，我会耐心等待的。我会先做一些其他的工作，不用在乎我这边。

[picobyte]

Thank you for your reply. Because of my work I can mostly spend time in the weekends. A rudimentary version was working, mostly thanks to Sean Wang. working though disabled if the iib extension is in Automatic1111. I've actually mostly focused on other issues, sorry, I may have misunderstood.

[zanllp]

These features have been roughly implemented and are available in the feature/add-more-isolation-mechanisms-and-export-functions branch. There have been significant changes in this branch, and I may not merge it into the main branch for the next few weeks. However, even in the main branch, maintenance is expected to be the primary focus recently, and there won't be many new features added.

AppUtils(base = "/foo", export_fe_fn=True).wrap_app(app)

Add a container for IIB

+ gr.HTML("error", elem_id="bar_iib_container")
- gr.HTML("error", elem_id="infinite_image_browsing_container_wrapper")

Load index.js on the browser side.

// Make a fetch request to a fake API endpoint to retrieve the JavaScript code
const jscodeResp = await fetch("/file?path=/path/to/your/submodue-iib/index.js") // Replace this with the actual API endpoint that serves the IIB code

// Convert the response into a text string
const jsText = await jscodeResp.text()

// Replace certain strings in the JavaScript code with new values
const js = jsText
  .replace("__iib_root_container__", "'#bar_iib_container'") // Set the root container for the IIB
  .replace("__iib_should_maximize__", "false") // Set the initial maximized state for the IIB
  // Replace all occurrences of "/infinite_image_browsing" with "/foo"
  // Change the base URL used by the IIB to load its resources
  .replace(/\/infinite_image_browsing/g, "/foo") 

// Evaluate the modified JavaScript code
eval(js) // Initialize the IIB with the modified configuration

const iib = gradioApp().querySelector('#bar_iib_container iframe').contentWindow

const { insertTabPane, getTabList, getPageRef, createGridViewFile: f } = iib
// The createGridViewFile function is a helper function that simplifies the creation of a FileNodeInfo object.
const files = [
  // Create an array of files with their corresponding tags.
  f('/path/to/img/1', ['tag1', 'tag2']),
  f('/path/to/img/2', ['tag3', 'tag4', 'tag6']),
  f('/path/to/img/3', ['tag2', 'tag5']),
  f('/path/to/img/4', ['tag1', 'tag2'])
]

// Insert a new tab pane of grid view type and assign it to the gridView variable.
const gridView = insertTabPane({
  // Optional parameters for tab index and pane index.
  tabIdx: 0,
  paneIdx: 0,
  pane: {
    type: 'grid-view', // Other types are also available, see https://github.com/zanllp/sd-webui-infinite-image-browsing/tree/main/vue/src/store/useGlobalStore.ts#L15
    name: 'Grid View 1',
    removable: true, // Optional parameter to allow the files to be removed, default is false.
    allowDragAndDrop: true, // Optional parameter to allow drag and drop, default is false.
    files // Use the files array created earlier for this pane.
  }
})

// Retrieve the files from the gridView pane and set them back to the same pane.
const files = gridView.ref.getFiles()
gridView.ref.setFiles(files)

// Get the tab list
const tabList = getTabList()
tabList[0].panes.key

// Get the file list from the first pane of the first tab.
const pane = tabList[0].panes[0]
getPageRef(pane.key).getFiles()

// Insert a new tab pane of local type with the specified directory path.
const localDirPane = insertTabPane({
  pane: {
    type: 'local',
    path: 'E:/_归档/green'
  }
})
localDirPane.ref.close() // Closes the newly created tab pane

To learn more information, you can refer to the type definition in the following file:

sd-webui-infinite-image-browsing/vue/src/store/useGlobalStore.ts

Lines 16 to 98 in 80dc801

	interface TabPaneBase {
	name: string | VNode
	nameFallbackStr?: string
	readonly key: string
	}
	interface OtherTabPane extends TabPaneBase {
	type: 'empty' | 'global-setting' | 'tag-search' | 'fuzzy-search' | 'batch-download'
	}
	export type GridViewFileTag = WithRequired<Partial<Tag>, 'name'>;
	export interface GridViewFile extends FileNodeInfo {
	/**
	* Tags for displaying the file. The 'name' property is required,
	* while the other properties are optional.
	*/
	tags?: GridViewFileTag[];
	}
	/**
	* A tab pane that displays files in a grid view.
	*/
	interface GridViewTabPane extends TabPaneBase {
	type: 'grid-view'
	/**
	* Indicates whether the files in the grid view can be deleted.
	*/
	removable?: boolean
	/**
	* Indicates whether files can be dragged and dropped from other pages into the grid view.
	*/
	allowDragAndDrop?: boolean,
	files: GridViewFile[]
	}
	interface TagSearchMatchedImageGridTabPane extends TabPaneBase {
	type: 'tag-search-matched-image-grid'
	selectedTagIds: MatchImageByTagsReq
	id: string
	}
	export interface ImgSliTabPane extends TabPaneBase {
	type: 'img-sli'
	left: FileNodeInfo
	right: FileNodeInfo
	}
	export interface FileTransferTabPane extends TabPaneBase {
	type: 'local'
	path?: string
	walkModePath?: string
	stackKey?: string
	}
	export type TabPane =
	| FileTransferTabPane
	| OtherTabPane
	| TagSearchMatchedImageGridTabPane
	| ImgSliTabPane
	| GridViewTabPane
	/**
	* This interface represents a tab, which contains an array of panes, an ID, and a key
	*/
	export interface Tab {
	/**
	* An array of panes that belong to this tab
	*/
	panes: TabPane[]
	/**
	* A unique identifier for this tab
	*/
	id: string
	/**
	* A value indicating which pane is currently selected within the tab
	*/
	key: string
	}
	export interface Shortcut extends Record<`toggle_tag_${string}`, string | undefined> {
	delete: string
	}

and this file:https://github.com/zanllp/sd-webui-infinite-image-browsing/blob/80dc8017bfe1fdec73ba746dd58a468549a11dd3/vue/src/defineExportFunc.ts

[WSH032]

感谢！
不过有几个疑问
Thanks! But I got some problems.

grid-view是拿来做临时工作区的吗？还是单纯展示图片？

我注意到它不像local那样有右键选项，和更多选项等等

而且无法用ctrl批量选择图片；那个拖拽好像也有问题，我把图片从grid-view拖到别的地方，或者别的地方拖过来都会报错

是它就是被设计成这样的吗？我觉得只要他能像local那样，拿来做临时工作区还是挺不错的（接口挺方便的）。

Is grid-view for temporary workspaces? Or just for displaying images?
I noticed that it doesn’t have right-click options, and more options, etc. like local does.
And I can’t use ctrl to select multiple images; drag and drop seems to have a problem too. I get an error when I drag images from grid-view to somewhere else, or from somewhere else to grid-view.
Is it designed to be like this? If it can work like local, I think it's a good idea to use it as temporary workspaces

---

如果grid-view并不是做临时工作区的话，那就是用local来直接操作咯？
但现有接口好像不能修改标签？

If grid-view is not for temporary workspaces, then do I use local to operate directly?
But the current interface seems to not allow modifying tags?

---

如果要用batch-download做临时工作区的话，现有接口好像不接受('/path/to/img/1', ['tag1', 'tag2']) ？好像只能拖拽来添加图片？

If use batch-download as a temporary workspace, the current interface seems to not accept (‘/path/to/img/1’, [‘tag1’, ‘tag2’])?
It seems that I can only add images by drag and drop ?

---

iib是否已经实现过一个跳转功能呢？就是把当前界面切到iib去（在新网页打开也行）。

如果还没有的话，只要在A1111-WebUI里注册的Gradio.Tab的elem_id是固定的，我这边可以自己写一个js来点击那个Gradio.Tab选项卡完成切换。

有的话我就不造轮子了。

Has iib already implemented a jump function? I mean, jump to the page displayed by iib (it’s okay to open it in a new web page).
If not, as long as the elem_id of the Gradio.Tab registered in A1111-WebUI is fixed, I can write a js myself to click that Gradio.Tab tab to complete the jumping.

[zanllp]

First of all, I do not recommend usingbatch-download. It is a function responsible for batch downloading and archiving, and should not be used for other purposes.

grid-view is not only used for viewing images, and although it is currently in an early stage, it will definitely become more powerful in the future. I have also fixed the problem of incorrect dragging and dropping. As for why there is no right-click menu and multiple selection, I feel that they may not be necessary, but we can consider adding them later if needed.

The biggest difference between grid-view and local is the usage scenario. grid-view is a collection of file references, while local is a folder. When you drag some files from other pages to grid-view, or remove a file from grid-view, the actual files will not change. However, local is different. If you delete a file, it will be deleted.

In addition, I have added several useful functions.

const { setTagColor, setTags, getTags } = iib

setTagColor("tag1", "#1890ff")
setTagColor("tag2", "green")


const localDirPane = insertTabPane({
  pane: {
    type: 'local',
    path: 'E:/_归档/green'
  }
})
const path = E:\\_归档\\green\\1.png'
setTags(path, ['tag1', 'tag2', 'tag3'])
const tags = getTags(path)

You can use switch2IIB and openIIBInNewTab to implement navigation.

[WSH032]

Thank for your quick response.

---

First of all, I do not recommend using batch-download. It is a function responsible for batch downloading and archiving, and should not be used for other purposes.

那么，先让用户在grid-view选择完想要的图片后，再把他们拖拽到batch-download来实现确认结果，怎么样？

但更新后可以从别的地方拖拽图片到grid-view，却仍不能从grid-view向外拖拽。

如果采用这个方案的话，用户确实能通过removable: ture来排除不要的图片，但加上multiple selection会更利于选择。

right-click也可以让用户在选择的时候查看prompt等信息。

So, how about letting the user select the desired images in grid-view first, and then drag them to batch-download to achieve confirm?
But after the update, I can drag images from other places to grid-view, but still can’t drag them out of grid-view.
If adopt this scheme, users can exclude unwanted images by removable: ture, But multiple selection also helps with selection.
And, right-click can also let users view prompt and other information when selecting.

---

As for why there is no right-click menu and multiple selection, I feel that they may not be necessary, but we can consider adding them later if needed.

grid-view和先前版本一样，图片缩略图可能太大了，且无法调整，所以可能需要像local那样的更多设置。

此外，grid-view是否能像local那样支持图片排序呢？对于查重这一场景，需要把同一重复类显示在一起。虽然我能在调用inserTabPane之前先对files进行排序。

还有image_sli对比功能，我相信这个功能对于用户选择查重结果也是挺有用的。

The image thumbnails of grid-view may be too large and cannot be adjusted. So may need more settings like local.
In addition, can grid-view support image sorting like local? For the scenario of duplicate checking, need to display the same duplicate class together. Although I can sort the files before calling inserTabPane.
And, about image_sli comparison function, I believe this function is also useful for helping users to select desired images.

---

grid-view is not only used for viewing images, and although it is currently in an early stage, it will definitely become more powerful in the future.

我觉得，如果grid-view实现local目前的功能就挺强大的了。

I think, if grid-view implements the current functions like local, it would be quite powerful.

---

setTags好像不起作用？

然后我注意到了type: temp，这是否意味着这些是临时标签，我就不用手动清除他们了？

setTags seems not work？
Then I noticed type: temp, does it mean these tags are temporary tags, and I don’t have to manually clear them?

[zanllp]

Maybe you need to pay attention to the escaping of paths and the use of separators. By the way, setTag only works when no· tags is passed in the pane.files parameter. Also, these temporary tags are only saved in memory and will be cleared after refreshing, so you don't need to clear them manually

If it's not a critical feature, you can wait until I have some free time to see how to implement these features. I need to find a new job recently.
You can also try to manually modify it yourself. I have already implemented the most complex part of the framework, so you can follow the existing code to add it. And if you need, I can also give you some permissions on the repo so that you can push directly to my branch.

[WSH032]

Maybe you need to pay attention to the escaping of paths and the use of separators.

You are right. Now it works normally. Thanks.

---

If it's not a critical feature, you can wait until I have some free time to see how to implement these features. I need to find a new job recently.

It's ok, I can wait for it. Wishing you find a satisfying job.

---

You can also try to manually modify it yourself. I have already implemented the most complex part of the framework, so you can follow the existing code to add it. And if you need, I can also give you some permissions on the repo so that you can push directly to my branch.

Maybe I can understand some of the code, but writing a complete js code is very difficult for me right now, I can only do most of the work in python. But, thanks anyway.