access.smb Cannot Connect to Azure Files SMB Shares

[GuyPaddock]

I'm trying out Pydio Community Edition 8.2.2 (2018-10-10) on a VM in Azure, with SMB shares provided by Azure Files.

I can add an SMB-based workspace with the Azure Files share, but browsing to the share yields the following error:

There was a problem trying to open folder pydio://xxx@yyy, please check your Administrator.

On Azure, I've tried toggling the "secure transfer required" setting on and off but that did not affect the ability to connect. That option is described as follows:

The secure transfer option enhances the security of your storage account by only allowing requests to the storage account by secure connection. For example, when calling REST APIs to access your storage accounts, you must connect using HTTPs. Any requests using HTTP will be rejected when 'secure transfer required' is enabled. When you are using the Azure files service, connection without encryption will fail, including scenarios using SMB 2.1, SMB 3.0 without encryption, and some flavors of the Linux SMB client. Because Azure storage doesn’t support HTTPs for custom domain names, this option is not applied when using a custom domain name.

I enabled server debug mode (AJXP_SERVER_DEBUG set to true in /etc/pydio/bootstrap_context.php, for anyone else who similarly is trying to figure out how to do that), and was able to get the arguments that were being passed down to the underlying command:

smbclient -N -O 'TCP_NODELAY IPTOS_LOWDELAY SO_KEEPALIVE SO_RCVBUF=8192 SO_SNDBUF=8192' -O 'TCP_NODELAY IPTOS_LOWDELAY SO_KEEPALIVE SO_RCVBUF=8192 SO_SNDBUF=8192' -d 0 '//yyy/zzz' -c 'dir "/"' 2>/dev/null [auth data]

After some digging, I found a few posts out there that indicate the -N option in the command is to blame. Here's an example of such a post: https://ubuntuforums.org/showthread.php?t=1501277

Sure enough, after monkey-patching the offending line in smb.php to remove the -N option, I got a different error in the logs that indicated that / cannot be found. I tried running smbclient directly with the maximum level of debugging output, both with and without the -N option, and found that when the option is set, smbclient does not set the NTLMSSP_NEGOTIATE_EXTENDED_SESSIONSECURITY option on the connection. When -N is absent, the NTLMSSP_NEGOTIATE_EXTENDED_SESSIONSECURITY option does get set. Presumably this is required in order to negotiate the connection with Azure Files.

I was able to fix that issue by commenting out the line that defaults an empty path to "/". Is there any reason this line is needed?

Lastly, I noticed that the options to smbclient are getting passed twice, so I removed the duplicate reference to $options.

Now, I can PR these changes into the project, but I don't fully understand what effect that's going to have on other SMB implementations. Is -N really necessary for other implementations to function? Do other implementations require dir to receive / to get an index of the share?

Separately from the SMB access Issue: I did notice that I can't delete multiple files in Pydio over SMB. I was getting this error message when I select multiple items and try to delete them:

Launching task Delete in background
User 'admin' - Cannot access to workspace with id/alias 6f383eb200749787cb45d2e27b699959

Deleting items one-by-one seemed to work fine.

I made no changes to permissions or roles, but today that workspace has now completely disappeared from my list of workspaces. It doesn't appear in the list of "My Workspaces" and when I click on it in the "Recent History", I now get this error:

User 'admin' - Cannot access to workspace with id/alias 6f383eb200749787cb45d2e27b699959

I was able to re-gain access to the workspace by finding my account under Settings and granting read + write access to that workspace, but that was the first time I visited that page. Any ideas as to why permissions on a new workspace would get lost after 12-24 hours? Do either of these issues seem to have anything to do with the changes I made to the SMB client, or is this a different functional area? Is this just misconfiguration on my end?

I'm new to Pydio, so my apologies for dropping so much into my first issue. I would just like to get some feedback from the team before I PR any changes.

[cdujeu]

Hi Guy
Thanks for this detailed report. Give us some time to review this and give you a feedback, our bandwith is pretty busy :-)
@c12simple can you have look?

[GuyPaddock]

Thanks, @cdujeu. As an update, I switched over to using meta.mount with CIFS and udevil, and was able to get a much higher performance setup. However, I still saw the following behavior:

• Trying to delete multiple files yields the workspace access error.
• After trying and failing to delete several files in a row, the whole page refreshed and the workspace I had just created that I was working with disappeared from my list of workspaces.
• I could only regain access to the workspace by editing my user account and granting myself read and write access to it again.
• Even after granting myself access, I still got the background worker error about not having workspace access.

Is it possible that when you create a workspace, you get temporary access to it in the session, but if the session gets reloaded, you lose that temporary access?

In any even, I can confirm that the workspace access errors at least seem unrelated to the changes I made to access.smb since this is a completely different driver.

As an aside: I have some changes I'd like to PR into meta.mount to try to make it a little easier to diagnose issues. I found that if your credentials aren't exactly right, you lose access to Pydio completely because trying to load any page yields a stack trace. What's worse is that the stack trace does not contain the error message from the underlying udevil / mount command. I found a way to include this output, but the code in this area is generally a mess so I'd like to clean that up for you guys.

[c12simple]

Hello @GuyPaddock
Thanks for your work. I start with access.smb

• Is "-N" options is necessary: Yes. Without option -N, the password should be add in command line of smbclient then the password can be seen in the log or ps -ef . Solution is using a file descriptor to specify a stdin for writhing password in each time Pydio launch smbclient process. Then, the -N forces smbclient to request for password.
• Is there any reason this line is needed? Yes, it's added to fix problem of listing in webdav.

Concerning problem of deletion multiple files, could you please try with disabled CLI in Pydio? (Pydio >> Settings >> Main option >> CLI)

[GuyPaddock]

@c12simple I think that you are incorrect about the -N option. Pydio appears to work better without the option being added. The password gets passed the same way and does not appear in the logs.

I will try with CLI disabled.

[GuyPaddock]

With CLI disabled, the workspace access errors go away on bulk actions. Is there a separate issue I can link to related to CLI not working properly on Ubuntu?

I am still having a problem where I lose workspace access after a few hours. I have to keep going back and adding myself in.

[GuyPaddock]

@c12simple According to the man page for smbclient:

       -N|--no-pass
           If specified, this parameter suppresses the normal password prompt from the client to the user. This is useful when accessing a service that does not
           require a password.

           Unless a password is specified on the command line or this parameter is specified, the client will request a password.

           If a password is specified on the command line and this option is also defined the password on the command line will be silently ingnored and no
           password will be used.

This means that -N is for accessing services that don't require a password. It also indicates that the password passed is silently ignored. Removing the option seems like the proper way to go here.

[GuyPaddock]

For reference to anyone else who runs into this issue, I've stashed my changes here:
https://github.com/GuyPaddock/pydio-core/commits/feature/fix-smb-azure