WIndows to Windows OpenSSH: Shell opener command was unsuccessful: client_loop: send disconnect: Connection reset

[neatchee]

DESCRIPTION

When trying to connect from XPipe on Windows 11 to another Windows 11 device running OpenSSH (with pwsh 7 as the default shell on the remote machine), the connection fails due to a connection reset. Issuing the command defined in "Insights" from a local pwsh shell manually succeeds normally.

STEPS TO REPRODUCE

1. Set up OpenSSH Server on a Windows 11 machine using Windows' "Extra Features"
2. Configure the remote machine to use pwsh7 as the default shell for OpenSSH
3. Create a neew XPipe SSH connection on the local machine using an identity file for authentication
4. Attempt to connect from XPipe

RESULT

io.xpipe.core.process.ProcessOutputException: Shell opener command was unsuccessful:
client_loop: send disconnect: Connection reset
at io.xpipe.core.process.ProcessOutputException.withPrefix(ProcessOutputException.java:29)
at io.xpipe.ext.proc.util.f.a(SourceFile:317)
at io.xpipe.ext.proc.l.m(SourceFile:2554)
at io.xpipe.ext.proc.ssh.r.m(SourceFile:53)
at io.xpipe.ext.proc.l.D(SourceFile:471)
at io.xpipe.ext.proc.l.start(SourceFile:223)
at io.xpipe.ext.proc.l.a(SourceFile:328)
at io.xpipe.ext.proc.l.prepareIntermediateTerminalOpen(SourceFile:239)
at io.xpipe.ext.proc.h.prepareTerminalOpen(SourceFile:54)
at io.xpipe.app.util.TerminalLauncherManager.prepare(TerminalLauncherManager.java:50)
at io.xpipe.app.util.TerminalLauncherManager.lambda$submitAsync$0(TerminalLauncherManager.java:70)
at io.xpipe.app.util.ThreadHelper.lambda$wrap$0(ThreadHelper.java:19)
at java.lang.VirtualThread.run(VirtualThread.java:309)

NOTES

Tested using both Windows' default OpenSSH and XPipe-bundled OpenSSH binaries
The following command from 'Insights' succeeds from a local pwsh7 shell:
ssh [email protected] -oStrictHostKeyChecking=accept-new -oRemoteCommand=none -oNoHostAuthenticationForLocalhost=yes -t

[crschnick]

This is probably related to #235

In short, Windows OpenSSH has always caused some problems. What you can try is running the command in the terminal with -T instead of -t to see whether it still works.

Most users that had to deal with Windows SSH problems used WSL as a gateway in xpipe when creating SSH connections, that works fine. So it's definitely isolated to Windows.

[crschnick]

But since this is getting quite frequent now, do you have anything enabled like special firewall settings? The problem with the Windows SSH issues were that I was never able to reproduce them.

[neatchee]

Yes, I run MalwareBytes' "Windows Firewall Control" for strict inbound/outbound blocking with prompting for unrecognized connection attempts. However, I do not observe any meaningful/relevant firewall log entries at the time of the failure on the host or client machines.

I have a background in software QA and would love to assist you in debugging this issue :)
Please feel free to make any requests from me that would help and I'll gladly do what I can. Love your software, use it daily, and would very much like to do what I can to contribute.

[crschnick]

Would it be possible to just temporarily disable some/these rules to check whether they are interfering with the connection?

[neatchee]

Completely disabled the firewall on both machines, no change :(

[crschnick]

Do you have WSL set up?

[neatchee]

Yup. I can work around it with WSL if necessary, but was hoping to get a direct connection to Windows pwsh working as I do run a fair number of powershell scripts.

fwiw I can confirm that:

1. -T option reproduces the issue without xpipe, so this is probably an upstream bug that I can pursue
2. The default shell isn't the issue; setting the default shell in windows to CMD produces the same error

[crschnick]

The thing is that the -T is required for XPipe to interact with the system. You can enable the disabled system interaction option for a SSH connection in XPipe, but you won't have access to any advanced features other than launching the connection that way.

You could install pwsh in your WSL, that is also supported there.

You could install msys2/mingw/git for windows, that one has a straight port of the ssh executable as well that you can add to the PATH.

[neatchee]

Ohoho, possibly relevant: microsoft/vscode-remote-release#6521
Looks like this is similar to an issue that VS Code has, and is identified as a known issue with certain versions of OpenSSH server on Windows

Since you aren't able to reproduce the issue, can you tell me what version of OpenSSH you have installed on your Windows device (if you have one, of course)?

It looks like the version installed via Windows' "features" is 8.6. I'm going to try installing a more recent version via winget and see if that resolves the issue....

[crschnick]

I tested with multiple versions, they all work for me on multiple devices. It seems to be a more system specific issue.

[crschnick]

If you really want to spend time with this, the only thing that xpipe can do is pass some specific SSH options with -oKey=Value. If you find an option that fixes this, then it can be implemented.

[neatchee]

Upgrading to the latest version (9.5 beta) of OpenSSH (plus the obligatory system reboot) works.

Guidance for future users: On the host machine, run the following from a Powershell prompt...
winget install Microsoft.OpenSSH.Beta --source winget
...then reboot your host machine. Note that if you have existing inbound firewall rules, you will need to create new ones as the path for this package will change from C:\Windows\System32\OpenSSH\sshd.exe to C:\Program Files\OpenSSH\sshd.exe

[crschnick]

That is good to know. I tested with 8.1p1 and 9.5 back then and they both worked

[crschnick]

So interestingly I'm now facing the same issue as I'm working on a feature that starts up a custom sshd.

For my testing, if I run sshd manually in the command line it will fail like this but if I started as a service it will suddenly work as expected. It's not a firewall issue or anything

[neatchee]

I wonder if the Service registration and the Path env var are pointing at different executables in your setup?

[crschnick]

I tried it a little bit more and can confirm that it is caused by the service. I only run the system default sshd, no custom installation because that is what I have to work with in xpipe.

If I run sshd manually it will fail always, regardless of which user runs it. If I create a service for it and start this service, it works.

[crschnick]

I managed to run the default Windows sshd on a user level without service with this configuration

PidFile "C:\Users\Christopher Schnick\.xpipe\ssh_bridge\sshd.pid"
StrictModes no
SyslogFacility USER
LogLevel Debug3
Port 21722
PasswordAuthentication no
HostKey "C:\Users\Christopher Schnick\.xpipe\ssh_bridge\host_key"
PubkeyAuthentication yes
AuthorizedKeysFile "C:\Users\Christopher Schnick\.xpipe\ssh_bridge\identity.pub"

[crschnick]

This is now closed in favour of #340 which is a pinned issue and offers a solution without having to read the full thread