bzr can't find .ssh/known_hosts

Asked by jim bridges

I am coming from a Perforce background and trying to just get started. I created a ubuntu desktop, installed the bzr server there and the openssh client and libraries.

On the windows client, operating from the command line, I seem to consistently run up again authentication failures related to ssh. Here is my trace. Do I need to install some openssh package on my Windows XP system? Or is there some configuration option that I need to set? (Note: I editted file so that real user does not show - I am paranoid :)

The trace was created after I gave this command:
C:\Temp\BzrTest>bzr up sftp://mildtgekco/Xgig
Connected (version 2.0, client OpenSSH_5.3p1)
Adding ssh-rsa host key for mildtgekco: 744A56EBA84D83BE97478DF774344AEE
SSH <user>@mildtgekco password:
Authentication (password) failed.
bzr: ERROR: Connection error: Unable to authenticate to SSH host as
  <user>@mildtgekco
 Authentication failed.

Tue 2010-10-12 17:39:45 -0700
0.093 bazaar version: 2.2.1
0.093 bzr arguments: [u'up', u'sftp://mildtgekco/Xgig']
0.093 looking for plugins in C:/Documents and Settings/<user>/Application Data/bazaar/2.0/plugins
0.093 looking for plugins in C:/Program Files/Bazaar/plugins
0.218 encoding stdout as sys.stdout encoding 'cp437'
1.844 bzr-svn: using Subversion 1.6.6 ()
1.953 falling back to default implementation
1.953 Unable to look up default port for ssh
1.953 failed to load system host keys: [Errno 2] No such file or directory: 'C:\\Documents and Settings\\<user>/.ssh/known_hosts'
[ 5164] 2010-10-12 17:39:47.071 INFO: Connected (version 2.0, client OpenSSH_5.3p1)
[ 5164] 2010-10-12 17:39:47.180 WARNING: Adding ssh-rsa host key for mildtgekco: 744A56EBA84D83BE97478DF774344AEE
2.078 Unable to look up default port for ssh
2.156 Unable to look up default port for ssh
[ 5164] 2010-10-12 17:39:56.196 INFO: Authentication (password) failed.
11.109 Traceback (most recent call last):
  File "bzrlib\commands.pyo", line 912, in exception_to_return_code
  File "bzrlib\commands.pyo", line 1112, in run_bzr
  File "bzrlib\commands.pyo", line 690, in run_argv_aliases
  File "bzrlib\commands.pyo", line 705, in run
  File "bzrlib\cleanup.pyo", line 135, in run_simple
  File "bzrlib\cleanup.pyo", line 165, in _do_with_cleanups
  File "bzrlib\builtins.pyo", line 1398, in run
  File "bzrlib\workingtree.pyo", line 351, in open_containing
  File "bzrlib\bzrdir.pyo", line 967, in open_containing
  File "bzrlib\bzrdir.pyo", line 988, in open_containing_from_transport
  File "bzrlib\bzrdir.pyo", line 941, in open_from_transport
  File "bzrlib\lazy_import.pyo", line 125, in __call__
  File "bzrlib\transport\__init__.pyo", line 1657, in do_catching_redirections
  File "bzrlib\bzrdir.pyo", line 928, in find_format
  File "bzrlib\bzrdir.pyo", line 1949, in find_format
  File "bzrlib\bzrdir.pyo", line 1959, in probe_transport
  File "bzrlib\transport\sftp.pyo", line 435, in get_bytes
  File "bzrlib\transport\sftp.pyo", line 425, in get
  File "bzrlib\transport\sftp.pyo", line 397, in _get_sftp
  File "bzrlib\transport\sftp.pyo", line 389, in _create_connection
  File "bzrlib\transport\ssh.pyo", line 321, in connect_sftp
  File "bzrlib\transport\ssh.pyo", line 317, in _connect
  File "bzrlib\transport\ssh.pyo", line 551, in _paramiko_auth
ConnectionError: Connection error: Unable to authenticate to SSH host as
  <user>@mildtgekco
 Authentication failed.

11.109 Transferred: 0kB (0.0kB/s r:0kB w:0kB)
11.109 return code 3

Question information

Language:
English Edit question
Status:
Solved
For:
Bazaar Edit question
Assignee:
No assignee Edit question
Solved by:
jim bridges
Solved:
Last query:
Last reply:
Revision history for this message
Martin Pool (mbp) said :
#1

Hi,

This means your ssh server is not accepting your username/password/key. Check auth.log on the server.

Revision history for this message
J Bridges (jbridges-yahoo) said :
#2

Please explain why the client is looking for .ssh/known_hosts. And I don't find the error on the server.

It also says Adding ssh-rsa host key for mildtgekco: 744A56EBA84D83BE97478DF774344AEE

So my guess is that it got a message from the server with the above key and it is trying to put it in that file? Does this make sense? Please explain the details of what is going on.

Revision history for this message
Martin Pool (mbp) said :
#3

On 14 October 2010 03:13, J Bridges
<email address hidden> wrote:
> Question #129151 on Bazaar changed:
> https://answers.launchpad.net/bzr/+question/129151
>
> J Bridges requested for more information:
> Please explain why the client is looking for .ssh/known_hosts.   And I
> don't find the error on the server.
>
> It also says Adding ssh-rsa host key for mildtgekco:
> 744A56EBA84D83BE97478DF774344AEE
>
> So my guess is that it got a message from the server with the above key
> and it is trying to put it in that file?  Does this make sense?  Please
> explain the details of what is going on.

The SSH client remembers the public key of the server, and stores it
in .ssh, to make sure that it's always the same machine you're
connecting to. See eg
<http://www.symantec.com/connect/articles/ssh-host-key-protection>.
That's almost certainly not why your connection is being refused.

Can you try running

  ssh -v username@miltdgekco

and see what that says?

--
Martin

Revision history for this message
jim bridges (jim-bridges) said :
#4

Thanks for the helpful website info. However that article says: "When you said 'yes', above, your SSH client saved the server's host key locally in the file $HOME/.ssh/known_hosts. " But the client says it can't do that - in the trace. So something clearly is wrong at that point.

Also, my client is Windows XP (or Windows 7, tried with both) and I don't have a ssh client so I can't give the command

ssh -v username@miltdgekco

We don't use Linux in our lab, but I installed a Ubuntu desktop machine to serve as a central bzr repository because it seems that this is more common and I felt there could be fewer problems. But now, if I am to do the kind of debugging you suggest, it looks like I need to install ssh on my Windows client.

Since the client can't find .ssh/known_hosts, would you recommend creating it and putting in the key the server send (from above, this is 744A56EBA84D83BE97478DF774344AEE ).? I don't know the format of that file, though so I would need to understand that first.

Revision history for this message
jim bridges (jim-bridges) said :
#5

Adding more information. I tried using putty.exe, which has lots of options but I took all the defaults and it worked to allow me to login to mildtgekco from my Windows 7 client. I don't know if this proves anything. It does not give me any more help on why I can't connect with the bzr client.

Revision history for this message
Vincent Ladeuil (vila) said :
#6

I'm not a windows user myself but I think you're on right track. Did you read https://help.launchpad.net/YourAccount/CreatingAnSSHKeyPair#Windows%20%28PuTTY%29 ?

Revision history for this message
jim bridges (jim-bridges) said :
#7

I am not a launchpad user so this is not really relevant.

It would be a lot easier to track down the problem if I understood why any process (client or server) is trying to write or access /.bzr - obviously a bad thing to do. It is a bad thing to write a file in the root on a client also - whether a windows or linux client.

If it does NOT mean the root folder, then it must mean the home folder of some user. But it is NOT the user (in my case) who is running the bzr serve command. The log file for that command shows nothing except the startup parameters for the bzr serve.

Maybe I am on a wild goose chase here. But when I google for the full text of the permission denied command in my log, I find that several people have experienced it - mostly with launchpad. But in the cases reported, the path is never at the root of the drive. So why is this happening to me? Does it think my home folder is the root?? Why and what can I do to change that?

From my experience so far, I would not want to unleash Bazaar on my user community. The currently use Perforce and even a novice is up and running in 10 minutes: adding files, checking out files, submitting files and so forth. Many users are not engineers and use our existing system maybe once per month. They can't afford a long learning curve, which (apparently) bzr requires.

I would give up on Bazaar except for urging from my manager to push ahead.

Revision history for this message
Martin Pool (mbp) said :
#8

Hi Jim,

You didn't mention anything trying to access /.bzr before.

You could try creating the .ssh directory; then the file will be automatically created.

But I really think the known-hosts thing is a distraction. The actual problem here is that the server is not accepting your credentials. The log files on the server should have at least a record of your attempted connection; please try to find that in /var/log/auth.log

Revision history for this message
jim bridges (jim-bridges) said :
#9

When you mentioned auth log before, I had no idea where it was. Using the full path, I found this:

Oct 14 07:28:53 MilDTGekco sshd[9751]: reverse mapping checking getaddrinfo for mapletree.ds.jdsu.net [10.75.131.141] failed - POSSIBLE BREAK-IN ATTEMPT!
Oct 14 07:28:59 MilDTGekco sshd[9751]: Accepted password for jbridges from 10.75.131.141 port 57320 ssh2
Oct 14 07:28:59 MilDTGekco sshd[9751]: pam_unix(sshd:session): session opened for user jbridges by (uid=0)
Oct 14 07:28:59 MilDTGekco sshd[9815]: subsystem request for sftp
Oct 14 07:28:59 MilDTGekco sshd[9751]: pam_unix(sshd:session): session closed for user jbridges

The reverse mapping is correct as far as I can see. That IP, 10.75.131.141 is the IP of the Windows 7 Client that I was using for this next command. So there is no spoofing of the IP involved.

10.75.131.172 is the IP of mildtgekco. I switched to using it instead of the name to see if it made any difference. The apparent difference is that it thinks I am a new user and so gave me a new key. Here is also where you see the "/.bzr" path I mentioned.

D:\BZR\trunk>bzr push --use-existing-dir sftp://10.75.131.172
Working tree "D:/BZR/trunk/" has uncommitted changes (See bzr status). Uncommitted changes will not be pushed.
Connected (version 2.0, client OpenSSH_5.3p1)
Adding ssh-rsa host key for 10.75.131.172: 744A56EBA84D83BE97478DF774344AEE
SSH jbridges@10.75.131.172 password:
Authentication (password) successful!
Secsh channel 1 opened.
[chan 1] Opened sftp connection (server version 3)
bzr: ERROR: Permission denied: "/.bzr": [Errno 13] Permission denied

I apologize for not giving this information before. Before this all started, I had no knowledge at all of ssh and now I know next to nothing but at least know where the log is.

If that auth log entry is the source of the problem, it is still strange in the way bzr reports it.

However, as I said, the reverse mapping is correct. I used nslookup to check on what machine it thought the IP address belonged to and it gave the right answer.

So it does not really solve the problem quite yet. But I feel a little more hopeful now :)

Revision history for this message
Martin Pool (mbp) said :
#10

If the SSH server doesn't let us in, then it just closes the connection, so there is not much that bzr can tell the user about the specific problem - but perhaps we can at least give people a clue where the problem is. (Bug 436745)

I don't know what changed, but the server is now accepting your connection, which is good.

The 'dns spoofing' message means that the DNS entry for .141 does not map to and from the hostname correctly. That's pretty common for internal IP networks. You can either fix it in DNS, or just ignore it, or set "UseDNS no" in the sshd configuration.

The 'permission denied' message is because "sftp://10.75.131.172" means "the root directory of that machine". You probably actually want to store it in a subdirectory you have write access too, like sftp://10..../~jbridges/myproject or sftp://10.../bzr/myproject. The latter will put it in /bzr/myproject, which might be appropriate for a team repository.

I would also suggest you look at using bzr+ssh rather than sftp, which will be a bit faster.

hope that helps

Revision history for this message
jim bridges (jim-bridges) said :
#11

after googling for ssh_config,

I added
UseDNS no
to my /etc/ssh/ssh_config file
(it was not there, even commented out.)

This caused another problem, apparently with PAM
"requirement user ingroup nopasswdlogin not met by user jbridges"

Since I could not connect, I quickly took the command out (deleted the line) and I still can't remotely login to mildtgekco anymore. So now I am hosed. It appears that the change I made is still in effect even though after I changed it back I rebooted (several times) and rechecked the content of ssh_config

Apparently it does not like me messing around with that file.

I googled some more and someone else gave the advice to modify sshd_config instead. Apparently, I was messing with the client config. But, that makes no sense, because, if I did not mess with the server config, why is it not letting me in?

I guess I need a lot of study on ssh to figure out what is going on.

Since the authentication is not actually controlled by Bazaar, I realize that this is not really your responsibility so I thank you for your efforts. I will mark this solved from the Bazaar viewpoint.

I may have to re-install my Ubutunu system to get back where I started.

Revision history for this message
Martin Pool (mbp) said :
#12

Hi,

Yes, sshd_config controls the server and ssh_config is the client.

You shouldn't need to reinstall to fix this. Before you try anything radical:

1- have a look in /var/log/auth.log and /var/log/daemon.log for any
relevant error messages
2- on the Ubuntu machine, type 'sudo service ssh restart' and see if
that gives you an error.

--
Martin