Add small screen mode behaviour to Palettes - feature pitch

[blurymind]

The suggestion goes like this:

When chickenpaint starts, it detects if its running in portrait mode (screen width/heiight detects that).

If that is the case, then do the following:

• the palette close button disappears, as it is too easy to tap by mistake and bringing a palette back from the menu is a pain in the neck on a smartphone
• a new behaviour is added to the palette headers. Tapping on it makes it toggle the visibility of its body on/off. This allows us to easily and quickly toggle it when we need it
• On some controls - such as for example selecting a tool - the palette is smarth enough to automatically toggles itself off as it correctly assumes that the user has performed the actin (similar to how artrage and other mobile painting apps work). We can do the same for when the user selects another layer, swatch or brush mode.
• You can still move palettes around by dragging them by the header as before.

If not, keep it working exactly as is atm.

I have some code on a local branch that does this already, although it needs a bit of polish. I am putting this up here to see if its something that would be a welcome pr and if the points sound good. Here is a wip gif

as I say, the ultimate goal is to make this thing usable on a smartphone.Right now it is very hard to use or broken even as palettes are covering the screen and out of screan headers make them impossible to move around or even dismiss

for future PR

• the palette header can sometimes display an icon/thumbnail of the selected thing while its collapsed (case for active tool, swatch, stroke mode,layer name, etc)

[thenickdude]

That looks good! The collapsible palettes make that usable on the small display.

I think this makes just as much sense on a landscape orientation screen too right? It really needs to be applied to all small screens.

[blurymind]

it can be applied to landscape too - but in that case I need to change how it decides to run in this mode.
The goal of it being a mode is to keep the old behavior.

I can detect if its running on a mobile operating system instead?
https://stackoverflow.com/questions/11381673/detecting-a-mobile-browser

additionally I can even add a toggle in the menu somewhere to disable this mode, but it is key to me for chickenpaint to run in the right way from the start without needing to go to menus

[thenickdude]

Yes, because nothing stops a desktop browser window from being portrait shaped, but it won't be useful there.

[blurymind]

it would be nice if chickenpaint could also store the gui state to localStorage. That way when someone uses it - it remembers the position and collapse state of palettes from previous session - but maybe for a future pr. I want to do this in steps and see how it goes :)

[thenickdude]

Also tablets run mobile OSes but it won't be helpful there either (maybe removing the close buttons will), so it can really just be gated on reported screen size (in pixels) to target phones.

[blurymind]

maybe a screen width threshold could trigger this? How do we tell chickenpaint if it should start in this mode?

[thenickdude]

it would be nice if chickenpaint could also store the gui state to localStorage.

Right now it doesn't do that because palettes are automatically arranged to fit the window size. If the positions were saved then they might be crammed over on one side if your next session is on a bigger browser window.

On a mobile device with a fixed size screen saving the position would make perfect sense though.

[thenickdude]

maybe a screen width threshold could trigger this? How do we tell chickenpaint if it should start in this mode?

It might as well detect it during construction I think. I can't think of a good reason to make this behaviour website-controllable in the constructor options.

[blurymind]

maybe a screen width threshold could trigger this? How do we tell chickenpaint if it should start in this mode?

It might as well detect it during construction I think. I can't think of a good reason to make this behaviour website-controllable in the constructor options.

I think that it would be nice to be able to override it, but also for it to automatically detect the mode as well.

The tricky part for me is in letting it run on a smartphone in both portrait and landscape mode - for both cases the screen is tiny and I would like to be able to have it run in the proposed mode.

So I cant have a portrait detection as you say. I know how to detect if the os is mobile, but then that creates the problem with having it run that way even on big screen tablets.

So perhaps I can check width (with some threshold) and height (with threshold) on construct to automatically detect if it should run this mode. If width is less than x or height is less than y?

That is sort of what I am thinking, because palettes have a certain size and if that size takes a huge portion or even exceeds the screen size, we should be able to collapse them and do something about it

[blurymind]

I also added palette toggle buttons that will show up in place of that giant Chickenpaint text when on small screens:

they do the same thing the palettes menu in the dropdown does but in a single click instead of three clicks. It would be nice if they get icons of course, for now they are using the mnemonic single letter.

So we can both make use of the collapse and the visibility toggle on mobile in a convenient way. Using it for a while I found it much more comfortable on my galaxy note

My current way of detecting we are running on a tiny screen in js
const windowIsPortraitMode = window.innerHeight < 820 || window.innerWidth < 400;

and on css I updated the query to also look at the width of the screen. The height alone was failing to detect most newer phones

/* More compact styles for small screens */
@media (max-height: 768px), (max-width: 400px)  {

I wonder if it should also check if on mobile - or detecting the screen is small is enough? Newer phones have higher and higher resolutions with denser and denser pixels.

did a few minor tweaks to reclaim some vertical space too - the header was wasting alot of it even on tiny screens imo

One huge problem still is the lack of multi-touch panning/zooming. When more than one finger is used- it shouldnt read input as brush

[blurymind]

you can try this on the branch here
https://github.com/blurymind/chickenpaint/tree/mobile-layout-buttons

It only gets enabled on small screen, so either try on smartphone or set the debugger to phone resolutions

[thenickdude]

Looks good!

I wonder if it should also check if on mobile - or detecting the screen is small is enough? Newer phones have higher and higher resolutions with denser and denser pixels.

This isn't an issue because ultradense screens still report a low resolution in "CSS pixels", i.e. phones will report being 480 pixels high even though they're really 2000 pixels high. (This is to prevent webpages from looking super tiny)

One huge problem still is the lack of multi-touch panning/zooming. When more than one finger is used- it shouldnt read input as brush

Yes, but solving this turns out to be very difficult, because those gestures will have to be recognised from the Pointer Event stream and emulated in code, because it doesn't look like you can get the browser to handle just panning events itself, and pass single-pointer events through for drawing (the CSS "touch-action" property has very spotty support in browsers). I couldn't find any good libraries for recognising these gestures, this is why I didn't bother focusing on mobile just yet.

This can be improved a lot by adding a "grabbing hand" tool on mobile just for panning the canvas.

[blurymind]

Thanks :) I was hoping that would be the case.
I made a PR here
#34

I was careful to use the existing code as much as possible and not create any risky changes. Hope its ok . Let me know if this can be done in a better way

[blurymind]

Yes, but solving this turns out to be very difficult, because those gestures will have to be recognised from the Pointer Event stream and emulated in code, because it doesn't look like you can get the browser to handle just panning events itself, and pass single-pointer events through for drawing (the CSS "touch-action" property has very spotty support in browsers). I couldn't find any good libraries for recognising these gestures, this is why I didn't bother focusing on mobile just yet.

I could give it a try as another PR to get it to pan when more than one finger is used. Maybe there is a simple way to bind that to the already implemented panning with middle click drag.
Would you be interested in a solution like that?

Obviously zooming/rotating would be way trickier to do. Not sure I am up for that yet :D Having these would be a huge leap in usability though - even just the panning alone

[thenickdude]

I think that would be really difficult because if your fingers don't touch the screen simultaneously, the first finger will begin painting and then the second finger would begin a pan/zoom gesture (it'd have to be able to undo the painting when it was realised the second finger joined the party).

[blurymind]

we could use the pointerevent to detect when a finger is being used or a stylus (optionally?)

I wonder how other good mobile painting apps do it. Worth looking into artrage and some other ones - although they arent open source :(