paramiko should be default client for Windows

That makes sense to me. It should be an easy change.

It would be nice to separately file a bug about paramiko using the uifactory to confirm the key.

I think there may be a bug for the plink behaviour.

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Martin Pool wrote:
> That makes sense to me. It should be an easy change.
>
> It would be nice to separately file a bug about paramiko using the
> uifactory to confirm the key.
>
> I think there may be a bug for the plink behaviour.
>
> ** Changed in: bzr
> Status: New => Confirmed
>
> ** Changed in: bzr
> Importance: Undecided => High
>

I'm sure that paramiko does use UI factory to get the key.

It uses Auth.get_password() which checks the netrc etc files, and then does:

if password is None:
    if prompt is None:
        # Create a default prompt suitable for most cases
        prompt = '%s' % scheme.upper() + ' %(user)s@%(host)s password'
    # Special handling for optional fields in the prompt
    if port is not None:
        prompt_host = '%s:%d' % (host, port)
    else:
        prompt_host = host
    password = ui.ui_factory.get_password(prompt,
                                          host=prompt_host, user=user)

plink and openssh obviously don't.

John
=:->
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Cygwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAkqNYEEACgkQJdeBCYSNAANGWACeLYo6AExzyPomJG85382O0XJa
a+QAoI4q5GD7+zn3vXUCGfP5HgRrWriS
=cIJa
-----END PGP SIGNATURE-----

John, I think you misunderstood. I'm talking not about user SSH key but about confirming fingerprint of SSH server on first run. This is the root of problem with plink.

When plink first time connecting to SSH server it don't "know" about yet it asks user about confirmation (using stdout/stdin):

C:\Utils\PuTTY>plink.exe bazaar.launchpad.net
The server's host key is not cached in the registry. You
have no guarantee that the server is the computer you
think it is.
The server's rsa2 key fingerprint is:
ssh-rsa 1024 9d:38:3a:63:b1:d5:6f:c4:44:67:53:49:2e:ee:fc:89
If you trust this host, enter "y" to add the key to
PuTTY's cache and carry on connecting.
If you want to carry on connecting just once, without
adding the key to the cache, enter "n".
If you do not trust this host, press Return to abandon the
connection.
Store key in cache? (y/n)

In the same situation paramiko just prints note about adding the server key to its own registry and move on.

I suspect some people may think such paramiko behavior maybe security hole.

Here is example of what paramiko does for unknown server:

C:\work\Bazaar\plugins\qbzr>bzr pull lp:qbzr
Connected (version 2.0, client Twisted)
Adding ssh-rsa host key for bazaar.launchpad.net: 9D383A63B1D56FC4446753492EEEFC89
Authentication (publickey) successful!
Secsh channel 1 opened.
No revisions to pull.

This line: "Adding ssh-rsa host key for bazaar.launchpad.net: 9D383A63B1D56FC4446753492EEEFC89"

Martin, bug about wrong plink behavior was raised long time ago. It was half-fixed then by Dma Vasiliev by introducing usage of -batch command-line option that prevents plink to asking user about caching key in registry. Unfortunately this hides problem with first connect much deeper.

Documentation on plink suggest connecting first time manually using Putty or plink, see http://the.earth.li/~sgtatham/putty/0.58/htmldoc/Chapter7.html, section 7.2.2:

To avoid being prompted for the server host key when using Plink for an automated connection, you should first make a manual connection (using either of PuTTY or Plink) to the same server, verify the host key (see section 2.2 for more information), and select Yes to add the host key to the Registry. After that, Plink commands connecting to that server should not give a host key prompt unless the host key changes.

About SSH host keys and plink.

The page http://www.straightrunning.com/XmingNotes/portableputty.php said PuTTY stores SSH server keys in the windows registry as [HKCU\Software\SimonTatham\PuTTY] and they are indeed there (on my machine).

So... in theory adding keys there automatically is possible OR at least bzrlib can check that registry *before* establishing connection using plink (bzrlib has separate class to wrap plink as SSH vendor IIRC) and explicitly use plink in the *asking* mode and allow user to say Yes. This might be tricky to implement (asking user is, but checking registry is really simple) but it will solve this nasty problem.

About SSH keys and paramiko.

Paramiko or IIUC bzrlib stores all collected SSH server keys in the %APPDATA%\bazaar\2.0\ssh_host_keys file and it has following format/content:

# SSH host keys collected by bzr
bazaar.launchpad.net ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAIEApuXd4MHTfr1qLXWeClxTTQYZQblCA+nHvbjAjowkEd2Y4kpvntJOVewoSwa22zTbiYSmmssCuCkFHwcpnZBZN5qMWewjizav30WfeyLR5Kng5qucxmFAEkNJjCJiu194wRNKu0cD99Uk/6X/AfsWGLgmL5pa5UFk62aW+iZLUQ8=

So, actually bzrlib knows when there is new server connected first time and therefore it can ask user for confirmation.

More about SSH keys and paramiko. There is code in bzrlib/transport/ssh.py

        server_key = t.get_remote_server_key()
        server_key_hex = paramiko.util.hexify(server_key.get_fingerprint())
        keytype = server_key.get_name()
        if host in SYSTEM_HOSTKEYS and keytype in SYSTEM_HOSTKEYS[host]:
            our_server_key = SYSTEM_HOSTKEYS[host][keytype]
            our_server_key_hex = paramiko.util.hexify(
                our_server_key.get_fingerprint())
        elif host in BZR_HOSTKEYS and keytype in BZR_HOSTKEYS[host]:
            our_server_key = BZR_HOSTKEYS[host][keytype]
            our_server_key_hex = paramiko.util.hexify(
                our_server_key.get_fingerprint())
        else:
            trace.warning('Adding %s host key for %s: %s'
                          % (keytype, host, server_key_hex))
            add = getattr(BZR_HOSTKEYS, 'add', None)
            if add is not None: # paramiko >= 1.X.X
                BZR_HOSTKEYS.add(host, keytype, server_key)
            else:
                BZR_HOSTKEYS.setdefault(host, {})[keytype] = server_key
            our_server_key = server_key
            our_server_key_hex = paramiko.util.hexify(
                our_server_key.get_fingerprint())
            save_host_keys()

So actually bzrlib checks host keys and therefore can ask user for confirmation.

It was easy change. Corresponding merge proposal: https://code.launchpad.net/~bialix/bzr/win32-paramiko-default/+merge/10563

2009/8/22 Alexander Belchenko <email address hidden>:
> More about SSH keys and paramiko. There is code in
> bzrlib/transport/ssh.py
>
>        server_key = t.get_remote_server_key()
>        server_key_hex = paramiko.util.hexify(server_key.get_fingerprint())
>        keytype = server_key.get_name()
>        if host in SYSTEM_HOSTKEYS and keytype in SYSTEM_HOSTKEYS[host]:
>            our_server_key = SYSTEM_HOSTKEYS[host][keytype]
>            our_server_key_hex = paramiko.util.hexify(
>                our_server_key.get_fingerprint())
>        elif host in BZR_HOSTKEYS and keytype in BZR_HOSTKEYS[host]:
>            our_server_key = BZR_HOSTKEYS[host][keytype]
>            our_server_key_hex = paramiko.util.hexify(
>                our_server_key.get_fingerprint())
>        else:
>            trace.warning('Adding %s host key for %s: %s'
>                          % (keytype, host, server_key_hex))
>            add = getattr(BZR_HOSTKEYS, 'add', None)
>            if add is not None: # paramiko >= 1.X.X
>                BZR_HOSTKEYS.add(host, keytype, server_key)
>            else:
>                BZR_HOSTKEYS.setdefault(host, {})[keytype] = server_key
>            our_server_key = server_key
>            our_server_key_hex = paramiko.util.hexify(
>                our_server_key.get_fingerprint())
>            save_host_keys()
>
> So actually bzrlib checks host keys and therefore can ask user for
> confirmation.

Yes, I thought so - just do a ui_factory.get_boolean or similar. We
may need to copy gracefully for noninteractive uis and it may be good
to add a more specific method so that eg qbzr can do something
specific.

--
Martin <http://launchpad.net/~mbp/>

I think general this would be reasonable to put in and OK for 2.0. Despite 2.0 being branched we shouldn't be _too_ paranoid about putting changes in.

The worst that can happen here is that people who do have putty working and have putty-specific configuration, it may break for them. It's pretty common that people on unix have things in ~/.ssh/config that would cause trouble if they fell back to paramiko, but maybe this is less common on Windows, and anyhow they can use the variable.

Martin Pool пишет:
> I think general this would be reasonable to put in and OK for 2.0.
> Despite 2.0 being branched we shouldn't be _too_ paranoid about putting
> changes in.
>
> The worst that can happen here is that people who do have putty working
> and have putty-specific configuration, it may break for them. It's
> pretty common that people on unix have things in ~/.ssh/config that
> would cause trouble if they fell back to paramiko, but maybe this is
> less common on Windows, and anyhow they can use the variable.

Well, I'm doubt plink working on Linux.

This is in the 2.0 branch and will be in 2.0b1

Note that Alexander's change doesn't actually make Paramiko the default, for people like me who have cygwin installed and thus have a valid 'ssh' implementation on the path. For this, you have to still specify "BZR_SSH=paramiko" as an environment variable.

So it would probably be good to take this one step farther and actually make 'paramiko' the default client if nothing else is specified. Something basically involving changing the order of lookup on Windows to look for 'paramiko' before it looks for 'ssh'.

I was somewhat against this originally when Mark Hammond proposed it. I've since been using paramiko for a while, and find that it will most likely give the best value for users. I miss being able to configure short names for hosts in .ssh/config, but I *really* prefer having Pageant handling my logins.

Not to mention that if we switch more to GUI level apps, then we can have QBzr give a password prompt, rather than having it show up on a terminal.

John A Meinel пишет:
> Note that Alexander's change doesn't actually make Paramiko the default,
> for people like me who have cygwin installed and thus have a valid 'ssh'
> implementation on the path. For this, you have to still specify
> "BZR_SSH=paramiko" as an environment variable.

I want only exclude plink from the game. I have no real experience (neither good or bad) with
cygwin+ssh.

> Not to mention that if we switch more to GUI level apps, then we can
> have QBzr give a password prompt, rather than having it show up on a
> terminal.

Yes, we have similar confusion even from Linux people, see bug https://bugs.launchpad.net/bugs/422656

Actually I'd like to go even further and suggest using paramiko as default client even on Linux, at
least with GUI tools.

2009/9/3 Alexander Belchenko <email address hidden>:
> John A Meinel пишет:
>> Note that Alexander's change doesn't actually make Paramiko the default,
>> for people like me who have cygwin installed and thus have a valid 'ssh'
>> implementation on the path. For this, you have to still specify
>> "BZR_SSH=paramiko" as an environment variable.
>
> I want only exclude plink from the game. I have no real experience (neither good or bad) with
> cygwin+ssh.
>
>> Not to mention that if we switch more to GUI level apps, then we can
>> have QBzr give a password prompt, rather than having it show up on a
>> terminal.
>
> Yes, we have similar confusion even from Linux people, see bug
> https://bugs.launchpad.net/bugs/422656
>
> Actually I'd like to go even further and suggest using paramiko as default client even on Linux, at
> least with GUI tools.

I think that gets more into the case that I was trying to explain
above: people on unix have openssh-specific configuration. For
instance I have an 89-line long .ssh/config, and without the proxy
configuration and other stuff in it I can't connect to several
machines. The impact of switching to paramiko would then be: are the
features I need still available there, and if not, is it reasonably
discoverable that I should switch back to external ssh. But I think
that's potentially resolvable.

I will also mention that there is a way to get ssh to use an external
program to ask for passwords or passphrases, and we could try to hook
into that. Whether it will give us such nice integration, and whether
it's worth writing that code for both paramiko and openssh is
questionable.

--
Martin <http://launchpad.net/~mbp/>

Martin Pool пишет:
> 2009/9/3 Alexander Belchenko <email address hidden>:
>> John A Meinel пишет:
>>> Note that Alexander's change doesn't actually make Paramiko the default,
>>> for people like me who have cygwin installed and thus have a valid 'ssh'
>>> implementation on the path. For this, you have to still specify
>>> "BZR_SSH=paramiko" as an environment variable.
>> I want only exclude plink from the game. I have no real experience (neither good or bad) with
>> cygwin+ssh.
>>
>>> Not to mention that if we switch more to GUI level apps, then we can
>>> have QBzr give a password prompt, rather than having it show up on a
>>> terminal.
>> Yes, we have similar confusion even from Linux people, see bug
>> https://bugs.launchpad.net/bugs/422656
>>
>> Actually I'd like to go even further and suggest using paramiko as default client even on Linux, at
>> least with GUI tools.
>
> I think that gets more into the case that I was trying to explain
> above: people on unix have openssh-specific configuration. For
> instance I have an 89-line long .ssh/config, and without the proxy
> configuration and other stuff in it I can't connect to several
> machines. The impact of switching to paramiko would then be: are the
> features I need still available there, and if not, is it reasonably
> discoverable that I should switch back to external ssh. But I think
> that's potentially resolvable.

OK, this is very valuable example. As I said before I have no experience with OpenSSH nor with
complex ssh configs. Of course your case change the status quo. So I no more blindly believe that
paramiko is the best.

> I will also mention that there is a way to get ssh to use an external
> program to ask for passwords or passphrases, and we could try to hook
> into that. Whether it will give us such nice integration, and whether
> it's worth writing that code for both paramiko and openssh is
> questionable.

So we have stalemate situation. And this is frustrating. :-/

2009/9/3 Alexander Belchenko <email address hidden>:
>> I will also mention that there is a way to get ssh to use an external
>> program to ask for passwords or passphrases, and we could try to hook
>> into that.  Whether it will give us such nice integration, and whether
>> it's worth writing that code for both paramiko and openssh is
>> questionable.
>
> So we have stalemate situation. And this is frustrating. :-/

No, I don't think it's a stalemate at all, it's just not quite trivial:

We can pursue either or both of two courses:

1- Encouraging use of paramiko on unix; a good first step would be to
ask unix-based devs to try it. This may be easier if there was a
config option as well as a variable. We can just ask on the list. In
fact I'll just try it now!

2- Hooking into openssh so it calls back to our uifactory to get the
password and other things.

--
Martin <http://launchpad.net/~mbp/>

>>>>> "martin" == Martin Pool writes:

    martin> 2009/9/3 Alexander Belchenko <email address hidden>:
    >>> I will also mention that there is a way to get ssh to use an external
    >>> program to ask for passwords or passphrases, and we could try to hook
    >>> into that.  Whether it will give us such nice integration, and whether
    >>> it's worth writing that code for both paramiko and openssh is
    >>> questionable.
    >>
    >> So we have stalemate situation. And this is frustrating. :-/

    martin> No, I don't think it's a stalemate at all, it's just not quite trivial:

    martin> We can pursue either or both of two courses:

    martin> 1- Encouraging use of paramiko on unix; a good first step would be to
    martin> ask unix-based devs to try it. This may be easier if there was a
    martin> config option as well as a variable. We can just ask on the list. In
    martin> fact I'll just try it now!

    martin> 2- Hooking into openssh so it calls back to our uifactory to get the
    martin> password and other things.

3 - Have platform/distribution specific policies. We should try to
    *avoid* resorting to such extremes (because of the associated
    costs in test, code and documentation), but we shouldn't rule
    that out either.

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Martin Pool wrote:
> 1- Encouraging use of paramiko on unix; a good first step would be to
> ask unix-based devs to try it. This may be easier if there was a
> config option as well as a variable. We can just ask on the list. In
> fact I'll just try it now!

mmm, if paramiko spoke to ssh-agent, I'd be a lot happier. Does it?

> 2- Hooking into openssh so it calls back to our uifactory to get the
> password and other things.

AFAIK openssh only allows you to do that if you are an agent: and this
is something I whole heartedly approve of.

On Modern linux machines, you will get a gui prompt for ssh connections
already; we don't need to do anything special to make this work.

- -Rob

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAkqfb8MACgkQ42zgmrPGrq7sYACg0VZoshdKEK7WPfl599rmcIUC
ewQAn0b5vI6LafHO3BuwXWfQQdypPn0e
=B9vv
-----END PGP SIGNATURE-----

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Martin Pool wrote:
> 2009/9/3 Alexander Belchenko <email address hidden>:
>>> I will also mention that there is a way to get ssh to use an external
>>> program to ask for passwords or passphrases, and we could try to hook
>>> into that. Whether it will give us such nice integration, and whether
>>> it's worth writing that code for both paramiko and openssh is
>>> questionable.
>> So we have stalemate situation. And this is frustrating. :-/
>
> No, I don't think it's a stalemate at all, it's just not quite trivial:
>
> We can pursue either or both of two courses:
>
> 1- Encouraging use of paramiko on unix; a good first step would be to
> ask unix-based devs to try it. This may be easier if there was a
> config option as well as a variable. We can just ask on the list. In
> fact I'll just try it now!
>

That is one thing I forgot. I haven't figured out any way to get
paramiko to be configured to do the crazy port forwarding we need to
connect to private Canonical servers. So when I need to connect there I
have to do:
BZR_SSH=ssh bzr foo bzr+ssh://rookery...

etc.

Nonetheless, we could still favor openssh over paramiko on linux and
favor paramiko over openssh on Windows in the medium term.

> 2- Hooking into openssh so it calls back to our uifactory to get the
> password and other things.
>

And I'd like to get that working. I don't really know what it entails.
Also, given that many users will have an ssh-agent already set up, which
also already knows how to pop up a GUI box to enter their password for
their ssh key, I don't know that we want to do all that work on our own.

Something to consider, but I feel like unix people already have a decent
tool chain when using openssh. And the main issue is that we don't have
that working on Windows, so we can do the work via paramiko there.

John
=:->
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Cygwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAkqf5BEACgkQJdeBCYSNAAMMLQCgqAZhL09A31It/WL4gWCdlsJa
NvIAoMsXPBZk0R28gwLCqmwT2AtbQBA7
=+Q15
-----END PGP SIGNATURE-----

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Robert Collins wrote:
> Martin Pool wrote:
>> 1- Encouraging use of paramiko on unix; a good first step would be to
>> ask unix-based devs to try it. This may be easier if there was a
>> config option as well as a variable. We can just ask on the list. In
>> fact I'll just try it now!
>
> mmm, if paramiko spoke to ssh-agent, I'd be a lot happier. Does it?

Last I checked, it does. It certainly knows how to talk to Pageant,
because I was the one that wrote that support. But I based it on the
existing ssh-agent support.

John
=:->

>
>> 2- Hooking into openssh so it calls back to our uifactory to get the
>> password and other things.
>
> AFAIK openssh only allows you to do that if you are an agent: and this
> is something I whole heartedly approve of.
>
> On Modern linux machines, you will get a gui prompt for ssh connections
> already; we don't need to do anything special to make this work.
>
> -Rob
>
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Cygwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAkqf5I0ACgkQJdeBCYSNAAMsogCfblqsNJC7XKX8p9ptS+l5yP5J
xcwAoLsSFe8uBW34oztW97Gf6ojMHtnC
=GgCQ
-----END PGP SIGNATURE-----

2009/9/3 Robert Collins <email address hidden>:
>> 2- Hooking into openssh so it calls back to our uifactory to get the
>> password and other things.
>
> AFAIK openssh only allows you to do that if you are an agent: and this
> is something I whole heartedly approve of.

Right, we'd perhaps need to be a passthrough proxy for the real agent.
 This may be more trouble than it's worth. gpg documents a protocol
whereby you can provide quite a powerful gui but openssh less so, at
least in the man pages.

> On Modern linux machines, you will get a gui prompt for ssh connections
> already; we don't need to do anything special to make this work.

However, you don't get a gui message afaik for eg
 * failed to connect
 * unknown host key
 * wrong host key

We could handle some of these a bit better by parsing the output of ssh.

Anyhow, I've split this to bug 424069 for the unix issue, as it's
mostly separate.

--
Martin <http://launchpad.net/~mbp/>