Problem with non latin characters in username (Trac #1248)

[butlerpd]

This problem was first reported in ticket #773 in August 2016 and closed as fixed in 5.0.

However very recently a new report came from a student at the Moscow State University that she had purchased a new computer and while !SasView seemed to install with no problem it would not start. A look at the logfile showed exactly the same 3 line output described in the #773 ticket before ending without the stopping message. I asked her to try 5.0beta2 which still would not start after installation. Changing her username to an "English name" fixed the problem.

Clearly this is not fixed and will be very annoying to anybody using !SasView on computers with non latin characters.

Piotr believes this may be a build system problem so tagging as support infrastructure workpackage for now.

Migrated from http://trac.sasview.org/ticket/1248

{
    "status": "assigned",
    "changetime": "2019-03-27T08:32:32",
    "_ts": "2019-03-27 08:32:32.539366+00:00",
    "description": "This problem was first reported in ticket #641 in August 2016 and closed as fixed in 5.0.\n\nHowever very recently a new report came from a student at the Moscow State University that she had purchased a new computer and while !SasView seemed to install with no problem it would not start. A look at the logfile showed exactly the same 3 line output described in the #641 ticket before ending without the stopping message.  I asked her to try 5.0beta2 which still would not start after installation.  Changing her username to an \"English name\" fixed the problem.\n\nClearly this is not fixed and will be very annoying to anybody using !SasView on computers with non latin characters.\n\nPiotr believes this may be a build system problem so tagging as support infrastructure workpackage for now.",
    "reporter": "butler",
    "cc": "",
    "resolution": "",
    "workpackage": "Support Infrastructure",
    "time": "2019-03-19T12:41:27",
    "component": "SasView",
    "summary": "Problem with non latin characters in username",
    "priority": "blocker",
    "keywords": "",
    "milestone": "SasView 5.0.0",
    "owner": "piotr",
    "type": "defect"
}

[butlerpd]

Trac update at 2019/03/23 14:13:04:

• butler commented:

Raising for code camp (to prioritize most useful tasks) and only because it was supposedly fixed.

• butler changed priority from "major" to "blocker"

[rozyczko]

Trac update at 2019/03/27 08:32:32:

• piotr changed owner from "" to "piotr"
• piotr changed status from "new" to "assigned"

[rozyczko]

Original testing was done on non-UTF8 account names by installing sasview in UTF8 directories.
This works.
HOWEVER, as reported this issue is about the user home directory containing non-ASCII characters.
This doesn't work.
I made a windows-specific hacky solution so our chinese guests at the Code Camp could work with SasView but the proper solution reaches deep down to SasModels and tinycc. Tinycc creates a temporary file during compilation in the user temp directory, which may contain UTF8 characters.
This change requires concerted effort of several repositories merged at the same time, I'm afraid.

Probably not doable for 5.0, unless we want to ship the hacky, windows-only solution.

[pkienzle]

What changes were needed? Do you have them in a branch? Can you attach a patch?

[rozyczko]

This hack relies on the backward compatibility of all Windows versions to support the old 8.3 DOS style names.
(E.g. "C:\Program Files" -> "C:\Progra~1")
Even the UTF-8 strings get cast into the old syntax so I'm just relying on this system conversion.
Seems to work just fine but again, it's Windows only.
I'll make a patch and attach here.

[rozyczko]

UTF8_patch.zip

Patch for both sasmodels and sasview.
This uses ctypes.windll.kernel32.GetShortPathNameW

[lucas-wilkins]

@rozyczko Can we close this?

[butlerpd]

I'm not sure if this is the same issue but we received a new help request from Japan:

5.0.5 had a problem on the Windows 11.
Windows11 Pro 64bit (OS build 22621.1555)、Dell XPS15 7590、SasView-5.0.5

Attached error occurs after several runs the software and will disappear after reinstallation of the software, but it occurs again after several runs. It does not happen in the version 4.

To verify that it was not a problem with a corrupted custom_config file I asked for a copy of that which seems fine to me and is copied here as well (github does not allow attaching .py files apparently)

"""
Application appearance custom configuration
"""
DATAPANEL_WIDTH = -1
CLEANUP_PLOT = False
FIXED_PANEL = True
PLOPANEL_WIDTH = -1
DATALOADER_SHOW = True
GUIFRAME_HEIGHT = -1
GUIFRAME_WIDTH = -1
CONTROL_WIDTH = -1
CONTROL_HEIGHT = -1
DEFAULT_OPEN_FOLDER = None
WELCOME_PANEL_SHOW = False
TOOLBAR_SHOW = True
DEFAULT_PERSPECTIVE = "Fitting"
SAS_OPENCL = "None"
MARKETPLACE_URL = "http://marketplace.sasview.org/"

# Logging options
FILTER_DEBUG_LOGS = True

Looking at the error it seems that a Yen symbol is being substituted for a path separator (/)? Am I missing something? Maybe @krzywon, @rozyczko, or @smk78 can take a look to see if there are other clues?

If this is really a language issue --- Is the (heavy lift) solution mentioned by @rozyczko still valid?

[lucas-wilkins]

I think the yen symbol is a related but not identical issue. Is that the `custom_config.py` from the right directory? That file, which appears to be the default `custom_config.py`, has no bytes with a value of 0x93. It would have been good to have seen the end of the traceback rather than the start. It would have given more relevant information.