ssh protocol for LFS

[masics]

I'm trying to migrate to LFS with ssh protocol but instead I get an error:
ssh protocol is not supported
How to make it working? Thank you.

[bozaro]

Can you show running command and error message?

[masics]

Sure:
$ java -jar git-lfs-migrate/git-lfs-migrate.jar -s orig_bare -d work3 -l ssh://artifactory.domain.com:1339/artifactory/test "*.dll" Feb 19, 2017 1:43:10 AM org.apache.http.impl.execchain.RetryExec execute INFO: I/O exception (org.apache.http.conn.UnsupportedSchemeException) caught when processing request to {}->ssh://artifactory.domain.com:1339: ssh protocol is not supported Feb 19, 2017 1:43:10 AM org.apache.http.impl.execchain.RetryExec execute INFO: Retrying request to {}->ssh://artifactory.domain.com:1339 Feb 19, 2017 1:43:10 AM org.apache.http.impl.execchain.RetryExec execute INFO: I/O exception (org.apache.http.conn.UnsupportedSchemeException) caught when processing request to {}->ssh://artifactory.domain.com:1339: ssh protocol is not supported Feb 19, 2017 1:43:10 AM org.apache.http.impl.execchain.RetryExec execute INFO: Retrying request to {}->ssh://artifactory.domain.com:1339 Feb 19, 2017 1:43:10 AM org.apache.http.impl.execchain.RetryExec execute INFO: I/O exception (org.apache.http.conn.UnsupportedSchemeException) caught when processing request to {}->ssh://artifactory.domain.com:1339: ssh protocol is not supported Feb 19, 2017 1:43:10 AM org.apache.http.impl.execchain.RetryExec execute INFO: Retrying request to {}->ssh://artifactory.domain.com:1339 [main] INFO git.lfs.migrate.Main - LFS server: Batch API request exception org.apache.http.conn.UnsupportedSchemeException: ssh protocol is not supported at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:108) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:353) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:380) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:184) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:88) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:184) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55) at ru.bozaro.gitlfs.client.internal.HttpClientExecutor.executeMethod(HttpClientExecutor.java:26) at ru.bozaro.gitlfs.client.Client.doRequest(Client.java:275) at ru.bozaro.gitlfs.client.Client.lambda$postBatch$7(Client.java:118) at ru.bozaro.gitlfs.client.Client.doWork(Client.java:252) at ru.bozaro.gitlfs.client.Client.postBatch(Client.java:118) at git.lfs.migrate.Main.checkLfsAuthenticate(Main.java:113) at git.lfs.migrate.Main.main(Main.java:66) Feb 19, 2017 1:43:10 AM org.apache.http.impl.execchain.RetryExec execute INFO: I/O exception (org.apache.http.conn.UnsupportedSchemeException) caught when processing request to {}->ssh://artifactory.domain.com:1339: ssh protocol is not supported Feb 19, 2017 1:43:10 AM org.apache.http.impl.execchain.RetryExec execute INFO: Retrying request to {}->ssh://artifactory.domain.com:1339 Feb 19, 2017 1:43:10 AM org.apache.http.impl.execchain.RetryExec execute INFO: I/O exception (org.apache.http.conn.UnsupportedSchemeException) caught when processing request to {}->ssh://artifactory.domain.com:1339: ssh protocol is not supported Feb 19, 2017 1:43:10 AM org.apache.http.impl.execchain.RetryExec execute INFO: Retrying request to {}->ssh://artifactory.domain.com:1339 Feb 19, 2017 1:43:10 AM org.apache.http.impl.execchain.RetryExec execute INFO: I/O exception (org.apache.http.conn.UnsupportedSchemeException) caught when processing request to {}->ssh://artifactory.domain.com:1339: ssh protocol is not supported Feb 19, 2017 1:43:10 AM org.apache.http.impl.execchain.RetryExec execute INFO: Retrying request to {}->ssh://artifactory.domain.com:1339 [main] ERROR git.lfs.migrate.Main - LFS server: Invalid base URL

[bozaro]

As I known, Git LFS use only HTTP for transport.

As I known, artifactory uses URL like:

     -l http://<user>:<pwd>@<server>/artifactory/api/lfs/<lfs-repo>

I make some Git LFS investigations in Dec 2016 with trial Artifactory: git-as-svn/git-lfs-java#4

[masics]

Artifactory can use ssh for authentication (as any other LFS service). Then, all transactions are done via HTTP(s).
https://github.com/git-lfs/git-lfs/blob/master/docs/api/authentication.md

[masics]

Another question - why do you care about it? Can you just put this info into .lfsconfig file and let me push it with original LFS?

[bozaro]

You can specify Git repository URL with -g option:

-g ssh://artifactory.domain.com:1339/artifactory/test

But I'm not sure about port parameter...

[masics]

Git repository and LFS are separate services. Git is located at Bitbucket while LFS is at Artifactory.

[bozaro]

You can specify:

• With -l parameter: direct Git LFS server (HTTP only);
• With -g parameter: Git URL. This URL can as HTTP server (in this case LFS URL calculates by simple URL Path modification) and SSH URL (in this case LFS URL gets by invoking git-lfs-authenticate command).

[bozaro]

Another question - why do you care about it? Can you just put this info into .lfsconfig file and let me push it with original LFS?

I did not understand your question. You can even omit the -g and -l options, and then upload LFS files manually (it would located at objects/lfs directory).

[masics]

I did not understand your question. You can even omit the -g and -l options, and then upload LFS files manually (it would located at objects/lfs directory).
I can. But without proper .lfsconfig it will be hard for users to clone/checkout.

[bozaro]

But without proper .lfsconfig it will be hard for users to clone/checkout.

It's unrelated problems. This file is outside working copy and all users should create it manually.

[masics]

It's unrelated problems. This file is outside working copy and all users should create it manually.

Why users should create it manually??? It should be part of the repo. All LFS files are located at predefined server at predefined folder. It's a constant.

[bozaro]

Manual git-lfs service configutaions usually used for testing and working with unconfigured git-lfs vs git intergration.
In common case user simply make git clone. Git LFS client detect LFS service location by https://github.com/git-lfs/git-lfs/blob/master/docs/api/server-discovery.md logic without any additional configurations files in client environment or working copy.

[masics]

In common case user simply make clone. Git LFS client detect LFS service location by https://github.com/git-lfs/git-lfs/blob/master/docs/api/server-discovery.md logic without any additional configurations files in client environment or workingh copy.

This is working if Git server serves as LFS storage. If they are different you cannot "guess" configuration from git url. And here comes the .lfsconfig with LFS url.

[bozaro]

This is working if Git server serves as LFS storage.

No. Git server should provide only integration point:

• For SSH users: git-lfs-authenticate command.
• For HTTP users: https://<git repo>.git/info/lfs/batch proxy to LFS server.

If you want to use Git server and LFS storage without discovery integration, you will feel pain.

[masics]

If you want to use Git server and LFS storage without discovery integration, you will feel pain.

How exactly you can configure Git server to provide LFS url on external server? What Git server you are using that provides this functionality?

[bozaro]

What Git server you are using that provides this functionality?

In my case I use GitLab as Git server and Git as Subversion as LFS external storage.
But I do not think that this solution is suitable for everyone.

[masics]

In my case it is Bitbucket and Artifactory. And I cannot get Bitbucket to make authentication for Artifactory. I need .lfsconfig file to do that.

[bozaro]

Both of them (Bitbucket and Artifactory) is proprietary and I can't install them to help you.

I would also highly recommend you assess the feasibility of using Artifactory as a repository Git LFS: as at December 2016 the implementation of server strongly did not meet specifications (git-as-svn/git-lfs-java#4). I was under the impression that Git LFS storage feature is implemented in Artifactory only for marketing.

[masics]

They must have fixed that in version 5.0.1. Everything is working if I supply .lfsconfig file.
Unfortunately this configuration is what is forced on me and I have to live with that.

[gronka]

We have the same use case. We are using various git repositories (bitbucket, gerritt, etc) paired with Artifactory. I agree with @masics that splitting git from lfs would be nice. It's unfortunate that there is not a way for you to develop/test against it. I would work on a PR, but I think for us, simply modifying the repo after conversion will be good enough. We have many repos, but we also have many engineering teams.