Windows multi-touch support

[Marocco2]

Is your feature request related to a problem? Please describe.

Current implementation on moonlight android involve specific hardcoded touch/gesture inputs for touchscreens (3 fingers for popping up Android keyboard eg.) but it will overwrite native Windows capabilities for touchscreens (scrolling, long press, pinch to zoom etc.)

Describe the solution you'd like

Add an option for preferring native multi-touch for supposed hosts (Windows)

Describe alternatives you've considered

Moonlight-qt is a perfect example for multi-touch support and I'd like to have the same implementation here on moonlight android

Screenshots

No response

[cgutman]

Yep, this is actually already implemented but disabled. The problem preventing us from shipping it on Android is that full native multi-touch interferes with the ability to launch the virtual keyboard using the 3-finger tap combo. We need to implement some kind of overlay or something to allow activation of the keyboard before we can turn it on. We should also probably add some type of zoom feature since touch targets can be very tiny on a phone/tablet screen.

This wasn't an issue with the PC client because it never had virtual keyboard support.

[Marocco2]

This wasn't an issue with the PC client because it never had virtual keyboard support.

I'm curious: are there cases where android virtual keyboard got the upper hand over Windows one? On my use case, Windows touch keyboard would pop up even on UAC prompts (on moonlight-qt) and it has keybindings with Win/Ctrl/Shift. So, perhaps, it's for Linux? I really don't know

[tianpengwan]

This paragraph was translated. I think it is necessary to support multiple touch screens for operation on mobile phones. On the contrary, there are more keyboard calls, and I haven't found this feature after playing for half a month. If this affects the function, it would be more than worth the loss. My suggestion is to change the number of fingers exhaled, such as 5 fingers exhaling without affecting its function, or give users a chance to choose
这段话使用了翻译。我认为手机上操作支持多个触屏是有必要的，反而呼出键盘有些多于，玩了半个月没发现有这功能如果说这影响了功能，这未免得不偿失。我的建议是更改呼出的手指数量，比如5指呼出不会影响其功能，或者给用户一个选择的机会

[tianpengwan]

太难受了哇哥
【PC版原神串流触屏-哔哩哔哩】 https://b23.tv/gBTZ9wz

[tianpengwan]

Yep, this is actually already implemented but disabled. The problem preventing us from shipping it on Android is that full native multi-touch interferes with the ability to launch the virtual keyboard using the 3-finger tap combo. We need to implement some kind of overlay or something to allow activation of the keyboard before we can turn it on. We should also probably add some type of zoom feature since touch targets can be very tiny on a phone/tablet screen.

This wasn't an issue with the PC client because it never had virtual keyboard support.

hihihi

[cxw620]

Yep, this is actually already implemented but disabled. The problem preventing us from shipping it on Android is that full native multi-touch interferes with the ability to launch the virtual keyboard using the 3-finger tap combo. We need to implement some kind of overlay or something to allow activation of the keyboard before we can turn it on. We should also probably add some type of zoom feature since touch targets can be very tiny on a phone/tablet screen.

This wasn't an issue with the PC client because it never had virtual keyboard support.

Does it mean that multi-touch support has been already implemented, but has not been released yet?

[Marocco2]

Yep, it’s in the code. You can compile it yourself and enable it

[ZirrorDmage]

Now I understand why we use long press to achieve right click instead of double finger click，maybe we can use another why to adapt these needs, such as floating pop-up window. Altough multi-touch game is not common on PC, but I think cut off this function is a loss. RD client is a good teacher for us to learn from, two input mode can fit different demands.

[ocelot233]

Yep, this is actually already implemented but disabled. The problem preventing us from shipping it on Android is that full native multi-touch interferes with the ability to launch the virtual keyboard using the 3-finger tap combo. We need to implement some kind of overlay or something to allow activation of the keyboard before we can turn it on. We should also probably add some type of zoom feature since touch targets can be very tiny on a phone/tablet screen.

This wasn't an issue with the PC client because it never had virtual keyboard support.

@cgutman I think that the three-finger gesture for bringing up the keyboard is not a good design choice, as seen in my XiaoMi Pad 5 Pro. It takes over all three-finger gestures: a long press with three takes a region screenshot, swiping down with three takes a screenshot, and swiping up with three opens the task manager. On its system, the keyboard invocation in your software becomes ineffective. Instead, sacrificing touch functionality to achieve this feature is unwise. Some games, such as Genshin Impact, require multi-touch gestures to play properly when touch UI is enabled through launch parameters.

I suggest taking inspiration from the design of Microsoft RDP. Consider adding a toggle button and/or a separate button for bringing up the keyboard, and/or include some settings that allow users to choose from various gesture-based operations.

I used machine translation. Below is the original text.
我认为三指呼出键盘不是一个好的设计，就像我的XiaoMi pad 5 pro，它占用了所有的三指操作逻辑：三指长按是区域截屏，三指向下划动是截屏，三指向上划动是任务管理，您软件中的呼出键盘在它的系统上就会失去作用。相反，为了实现这个功能牺牲触控的功能是不明智的。有些游戏，如genshin impact，可以通过启动参数启用触摸ui，需要多指触控才能正常游玩。我建议您参考Microsoft RDP的设计，添加一个切换按钮和/或独立的呼出键盘按钮，和/或添加一些设置项使用户可以自己选择几种操作逻辑中的一种。

[TrueZhuangJia]

want this feature on iOS moonlight pls

[Moeweb647252]

Yep, it’s in the code. You can compile it yourself and enable it

I think adding a configure entry of touch screen in settings is feasible

[nhac-lly]

Why not add this as option? I mean I never used 3 fingers gesture to use Android keyboard but built in Windows keyboard instead.

[samkitty]

We already have the option to hold start button for mouse mode, why not hold select button for keyboard? Do people use Moonlight without a controller that often? Wouldn't they also have the controller overlay (and thus a select button on screen which they can tap and hold)? I feel not having native touch support is one of Moonlight's biggest downsides and is the only reason I still use SuperDisplay. The fact this feature is so within reach but not enabled is quite frustrating. It can surely be added as an option in settings?

[Marocco2]

We already have the option to hold start button for mouse mode, why not hold select button for keyboard? Do people use Moonlight without a controller that often? Wouldn't they also have the controller overlay (and thus a select button on screen which they can tap and hold)? I feel not having native touch support is one of Moonlight's biggest downsides and is the only reason I still use SuperDisplay. The fact this feature is so within reach but not enabled is quite frustrating. It can surely be added as an option in settings?

Yes, it can surely be an option and it would be quite easy to make a PR.
I've made my own build for it as I don't use Android keyboard at all, so I don't need an option for it

[Marocco2]

You only need to remove /* and */ before compiling your own Moonlight build

moonlight-android/app/src/main/java/com/limelight/Game.java

Lines 2050 to 2057 in 6551076

	// TODO: Re-enable native touch when have a better solution for handling
	// cancelled touches from Android gestures and 3 finger taps to activate
	// the software keyboard.
	/*if (!prefConfig.touchscreenTrackpad && trySendTouchEvent(view, event)) {
	// If this host supports touch events and absolute touch is enabled,
	// send it directly as a touch event.
	return true;
	}*/

[samkitty]

You only need to remove /* and */ before compiling your own Moonlight build

Thanks for the info! I have since moved to a Chinese fork of the Moonlight client which includes native touch support as well as a handful of other features such as a full windows keyboard overlay and the ability to switch between touch/trackpad mode on the fly by using the back gesture on your phone to access a hidden menu.

Search for user 阿西西的日常 if interested.

[tianpengwan]

谢谢，我找到了~o(〃'▽'〃)o

…

---Original--- From: ***@***.***> Date: Tue, Mar 19, 2024 08:26 AM To: ***@***.***>; Cc: ***@***.******@***.***>; Subject: Re: [moonlight-stream/moonlight-android] Windows multi-touch support (Issue #1271) You only need to remove /* and */ before compiling your own Moonlight build Thanks for the info! I have since moved to a Chinese fork of the Moonlight client which includes native touch support as well as a handful of other features such as a full windows keyboard overlay and the ability to switch between touch/trackpad mode on the fly by using the back gesture on your phone to access a hidden menu. Search for user 阿西西的日常 if interested. IMG-20240317-WA0003.jpg (view on web) — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you commented.Message ID: ***@***.***>

[TrueZhuangJia]

want this feature on iOS moonlight pls

Just created a fork myself to implement multi-touch on iOS.
https://github.com/TrueZhuangJia/moonlight-ios-NativeMultiTouchPassthrough/tree/Moonlight-iOS-ZWM

[MuhamadRifkii]

I made a fork of moonlight for android with a few added feature like
gameback menu #1171
Perfomance overlay toggler #1219
And custom resolution #1349

I did not write the original code, just applying it for personal use.

https://github.com/MuhamadRifkii/moonlight-android

[protodevnan0]

I made a fork of moonlight for android with a few added feature like gameback menu #1171 Perfomance overlay toggler #1219 And custom resolution #1349

I did not write the original code, just applying it for personal use.

https://github.com/MuhamadRifkii/moonlight-android

No words can describe how much I love You for this.

[moi952]

I made a fork of moonlight for android with a few added feature like

gameback menu #1171

Perfomance overlay toggler #1219

And custom resolution #1349

I did not write the original code, just applying it for personal use.

https://github.com/MuhamadRifkii/moonlight-android

Thanks 🙏🏼

@cgutman maybe we can have this on an official release ?

And the last really great thing would be to have different setting profiles, and to assign these profiles to games.

Example:

Greedy game like Avatar in 2k60.

Lighter light game like God of war 4k120

And if we don't assign a profile to a game, it launches the stream with the profile

[SixelAlexiS90]

I made a fork of moonlight for android with a few added feature like gameback menu #1171 Perfomance overlay toggler #1219 And custom resolution #1349

I did not write the original code, just applying it for personal use.

https://github.com/MuhamadRifkii/moonlight-android

Hi! Does this fork get rid of the "hold to right click" as well?
In touch mode I want to keep holding left click instead. Thanks 😊

[MuhamadRifkii]

Hi! Does this fork get rid of the "hold to right click" as well? In touch mode I want to keep holding left click instead. Thanks 😊

Unfortunately no, it interfere with the windows native multitouch support, afaik this app only send touch event into the windows and the touch event is handled by windows.

But, there is another way to disable "hold to right click" by using the regedit.

Navigate to this entry,
HKEY_CURRENT_USER\Software\Microsoft\Wisp\Touch
And changing the value of "TouchMode_hold" from 1 to 0, and reboot.

Please mind that this will disable the right click function of the touchscreen entirely (you need to re enable that registry again to enable the right click function)

[tripleS-Dev]

I made a fork of moonlight for android with a few added feature like gameback menu #1171 Perfomance overlay toggler #1219 And custom resolution #1349

I did not write the original code, just applying it for personal use.

https://github.com/MuhamadRifkii/moonlight-android

This is very good!
By any chance, is it possible to send to touch even when moonlight's input device is a trackpad (mouse)?

[MuhamadRifkii]

This is very good! By any chance, is it possible to send to touch even when moonlight's input device is a trackpad (mouse)?

It's possible, but not practical. direct mouse control almost the same as the windows native touch anyway. Unless there is a quick way to change touch control mode (like sidebar when back button pressed)

[tripleS-Dev]

This is very good! By any chance, is it possible to send to touch even when moonlight's input device is a trackpad (mouse)?

It's possible, but not practical. direct mouse control almost the same as the windows native touch anyway. Unless there is a quick way to change touch control mode (like sidebar when back button pressed)

Could you make a fork that forwards touch input instead of mouse input on mouse, trackpad, and touchpad devices? My skills aren't enough to figure it out.

I'm experiencing this issue too, and I believe this might solve it.

https://ideas.moonlight-stream.org/posts/41/support-samsung-s7-keyboard-cover-touchpad
https://www.reddit.com/r/GalaxyTab/comments/mcdgxq/moonlight_with_keyboard_cover_on_s7/
https://www.reddit.com/r/GalaxyTab/comments/16idupx/tab_s9_book_cover_keyboard_trackpad_issues/

[MuhamadRifkii]

Could you make a fork that forwards touch input instead of mouse input on mouse, trackpad, and touchpad devices? My skills aren't enough to figure it out.

Well looks like i was wrong, there is no way to implement it if samsung is using they own API to implement touch event on the device.
Based on what i read there, looks like this is poor touchpad implementation by samsung. since its not using USB API, so it's not detected as Mouse input device.

[tripleS-Dev]

Well looks like i was wrong, there is no way to implement it if samsung is using they own API to implement touch event on the device. Based on what i read there, looks like this is poor touchpad implementation by samsung. since its not using USB API, so it's not detected as Mouse input device.

I’m using the latest Samsung Book Cover that was released about three months ago. Maybe the firmware has been updated? In most cases, including left-click and right-click, it behaves just like a mouse.

However, when using two-finger scrolling, the keyboard uses drag instead of mouse wheel scrolling on the screen. So, even if I don’t know exactly how it’s processed, it seems that sending all inputs to Windows in touch mode would make it function correctly in all cases, including scrolling.

If you watch the video I just posted (https://youtu.be/pMdNq2PGF28), you’ll understand. Click-and-drag actions require touch mode to work as scrolling.