feat(webapp): virtual background

[huanghuang358]

add a new feature accroding to #9 .

[a-wing]

@huanghuang358 @Rocket1184 Review

[rocka]

I've tried the virtual background stream locally and it works like a charm. Nice work! I'm here to make some suggestions to further imporove the maintainability of the codebase:

[rocka]

Maybe we should download the model and put it under /static instead of using google's URL

[rocka]

It's definitely a bad idea to use jsdelivr URL in production. Vite has the ability to import file as URL: https://vite.dev/guide/assets#explicit-url-imports

So we may just import those files from node_modules

[rocka]

async function can also return Promises, so no need to store streamForVirtualBackground globally and check it in an interval afterwards.

Suggested change

	async function enableSegmentation(deviceId: string) {
	try {
	if (!imageSegmenter) {
	await createImageSegmenter()
	}
	// 开始图像分割
	const stream = await navigator.mediaDevices.getUserMedia({ audio: false, video: { width: 480, height: 360, deviceId: deviceId } })
	video.srcObject = stream
	video.onloadeddata = async () => {
	video.play()
	webcamRunning = true
	await predictWebcam()
	streamForVirtualBackground = canvas.captureStream()
	}
	} catch (error) {
	console.error("启动摄像头失败:", error);
	}
	}
	async function enableSegmentation(deviceId: string): Promise<MediaStream | undefined> {
	try {
	if (!imageSegmenter) {
	await createImageSegmenter()
	}
	// 开始图像分割
	const stream = await navigator.mediaDevices.getUserMedia({ audio: false, video: { width: 480, height: 360, deviceId: deviceId } })
	return new Promise(resolve => {
	video.srcObject = stream
	video.onloadeddata = async () => {
	video.play()
	webcamRunning = true
	await predictWebcam()
	resolve(canvas.captureStream())
	}
	})
	} catch (error) {
	console.error("启动摄像头失败:", error);
	}
	}

[rocka]

Suggested change

	async function asyncGetStreamForVirtualBackground(deviceId: string): Promise<MediaStream> {
	await enableSegmentation(deviceId)
	while (streamForVirtualBackground === null) {
	await new Promise(resolve => setTimeout(resolve, 100)) // 每100ms检查一次
	}
	return streamForVirtualBackground
	}
	async function asyncGetStreamForVirtualBackground(deviceId: string): Promise<MediaStream> {
	return enableSegmentation(deviceId)
	}

[rocka]

There are too many global variables. Please consider wrapping all those state and actions into a class, maybe something like:

export class VirtualBackgroundStream {
  private deviceId: string
  private webcamRunning: boolean
  private imageSegmenter: ImageSegmenter
  // other more memebers ...

  constructor(deviceId: string) {
    this.deviceId = deviceId
  }

  public startStream(): Promise<MediaStream> {
    // enableSegmentation ...
  }

  public destroyStream:(): void {
    // disableSegmentation ...
  }
}

to make the lifecycle management more clear, and easier for future extensions