Bazaar

How do I get smart server to use active directory

Asked by Jemsquash on 2010-06-02

I am trying to get active directory to integrate with smart server. From the documentation it seems like I can run the smart server behind an apache instance. I have also loaded the Active Directory integration and this seems to be working. I'm an apache novice so I'm going on trial and error.

It seems like something is going wrong either during the rewrite rule or after it. I can see in the access logs that it is accessing /smart. What I can't see is evidence of the rewrite happening but seeing as it is the rewrite making it go to .py I don't know how to check if it is working.

The problem I have is I always get a 401 error even though I've authenticated.

My apache config is as follows (with a few changes so as not to reveal too many details about my clients network).

<VirtualHost *:9090>
ServerAdmin <email address hidden>
DocumentRoot "c:/apache-httpd-2.2.9-dev/htdocs/bzr"

ServerName blahserver.blah.com
ServerAlias blahserver.blah.com blahserver

ErrorLog logs/blah-error.log
CustomLog logs/blah-access.log common

  <Proxy *>
    Order deny,allow
    Allow from all
    Satisfy all
  </Proxy>

  <IfModule dir_module>
    DirectoryIndex index.php index.html
  </IfModule>

  Alias /bzr C:/bzr-sandbox/sh-repo
  <Directory C:/bzr-sandbox/sh-repo>
      Options Indexes +FollowSymLinks
      RewriteEngine On
      RewriteBase /bzr
      #RewriteRule ^(.*/|)\.bzr/smart$ C:/bzr-sandbox/sh-repo/scripts/bzr-smart.fcgi
      RewriteRule ^(.*/|)\.bzr/smart$ C:/bzr-sandbox/sh-repo/scripts/bzr-smart.py

      AllowOverride All
      Options None
      Order allow,deny
      Allow from all
      # module specific
      AuthName "Bzr development"
      AuthType SSPI
      SSPIAuth On
      # module is the final authority
      SSPIAuthoritative On
      # only authenticate against DPI no others
      SSPIDomain DPI
      # keep domain name in userid string
      SSPIOmitDomain Off
      # let non-IE clients authenticate
      SSPIOfferBasic On
      # first we try NTLM authentication and then basic authentication
      SSPIBasicPreferred Off
      # need to be in dev Group to be able to access bzr
      Require group "OurDomain\Bazaar-grp"
  </Directory>

  Alias C:/bzr-sandbox/sh-repo/scripts/bzr-smart.py C:/bzr-sandbox/sh-repo/scripts/bzr-smart.py
  <Directory C:/bzr-sandbox/sh-repo/scripts>
      <Files bzr-smart.py>
          #PythonPath "sys.path+['C:/bzr-sandbox/sh-repo/scripts']"
          PythonPath "['c:/Program Files/Bazaar/lib']+sys.path+['C:/bzr-sandbox/sh-repo/scripts']"
          AddHandler python-program .py
          #AddHandler mod_python .py
          PythonHandler bzr-smart::handler
          PythonDebug On
      </Files>

</Directory>

  # Test active directory integration
  <Directory "c:/apache-httpd-2.2.9-dev/htdocs/bzr/adi">
      AllowOverride None
      Options None
      Order allow,deny
      Allow from all
      # module specific
      AuthName "Bzr development"
      AuthType SSPI
      SSPIAuth On
      # module is the final authority
      SSPIAuthoritative On
      # only authenticate against DPI no others
      SSPIDomain DPI
      # keep domain name in userid string
      SSPIOmitDomain Off
      # let non-IE clients authenticate
      SSPIOfferBasic On
      # first we try NTLM authentication and then basic authentication
      SSPIBasicPreferred Off
      # need to be in dev Group to be able to access Subversion
      Require group "OurDomain\Bazaar-grp"
  </Directory>

</VirtualHost>

Question information

Language:: English Edit question

Status:: Solved

For:: Bazaar Edit question

Assignee:: No assignee Edit question

Solved by:: Jemsquash

Solved:: 2010-06-10

Last query:: 2010-06-10

Last reply:: 2010-06-09

Link existing bug

Jemsquash (james-stewart-bigfoot) said on 2010-06-04:

I am using :
apache 2.2.9 on Windows XP professional
mod_python-3.3.1.win32-py2.3-Apache2.2.exe
mod_auth_sspi-1.0.4-2.2.2.zip
mod_proxy_html-3.0.1-w32.zip
python 2.3.5

Jemsquash (james-stewart-bigfoot) said on 2010-06-08:

I've gone back to basics and tried to get the smart server running without the active directory stuff.

I've got it to the point of importing the bzr-smart.py.

From the apache logs I get:

[Tue Jun 08 16:25:06 2010] [notice] mod_python (pid=8672, interpreter='blah.com'): Importing module 'C:\\bzr-sandbox\\sh-repo\\scripts\\bzr-smart.py'

I now get the following error:

URI: '/bzr/scripts/bzr-smart.py'
Location: None
Directory: 'C:/bzr-sandbox/sh-repo/scripts/'
Filename: 'C:/bzr-sandbox/sh-repo/scripts/bzr-smart.py'
PathInfo: ''

Phase: 'PythonHandler'
Handler: 'bzr-smart::handler'

Traceback (most recent call last):

File "C:\Python23\lib\site-packages\mod_python\importer.py", line 1537, in HandlerDispatch
default=default_handler, arg=req, silent=hlist.silent)

File "C:\Python23\lib\site-packages\mod_python\importer.py", line 1202, in _process_target
module = import_module(module_name, path=path)

File "C:\Python23\lib\site-packages\mod_python\importer.py", line 296, in import_module
log, import_path)

File "C:\Python23\lib\site-packages\mod_python\importer.py", line 680, in import_module
execfile(file, module.__dict__)

File "C:\bzr-sandbox\sh-repo\scripts\bzr-smart.py", line 2, in ?
from bzrlib.transport.http import wsgi

ImportError: No module named bzrlib.transport.http

Any ideas how I get this module and install it?

Parth Malwankar (parthm) said on 2010-06-08:

Hi Jemsquash,

bzr supports and is tested with Python 2.4 and above. Is it possible for you to try this with a later version of Python?

Jemsquash (james-stewart-bigfoot) said on 2010-06-09:

I've switched to using the wsgi architecture. It looks like it is using Python 2.6.5 now.

I'm still however stuck with it not finding the bzrlib.transport.http package.
From by bzr-error.log:

[Wed Jun 09 11:59:08 2010] [error] [client 10.37.1.54] mod_wsgi (pid=5892): Target WSGI script 'C:/bzr-sandbox/sh-repo/scripts/smartserver.wsgi' cannot be loaded as Python module.
[Wed Jun 09 11:59:08 2010] [error] [client 10.37.1.54] mod_wsgi (pid=5892): Exception occurred processing WSGI script 'C:/bzr-sandbox/sh-repo/scripts/smartserver.wsgi'.
[Wed Jun 09 11:59:08 2010] [error] [client 10.37.1.54] Traceback (most recent call last):
[Wed Jun 09 11:59:08 2010] [error] [client 10.37.1.54] File "C:/bzr-sandbox/sh-repo/scripts/smartserver.wsgi", line 1, in <module>
[Wed Jun 09 11:59:08 2010] [error] [client 10.37.1.54] from bzrlib.transport.http import wsgi
[Wed Jun 09 11:59:08 2010] [error] [client 10.37.1.54] ImportError: No module named bzrlib.transport.http
[Wed Jun 09 11:59:08 2010] [error] [client 10.37.1.54] File does not exist: C:/tools/apache/htdocs/favicon.ico
[Wed Jun 09 11:59:23 2010] [error] [client 10.37.1.54] File does not exist: C:/tools/apache/htdocs/favicon.ico
[Wed Jun 09 11:59:38 2010] [error] [client 10.37.1.54] File does not exist: C:/tools/apache/htdocs/favicon.ico
[Wed Jun 09 11:59:50 2010] [error] [client 10.37.1.54] mod_wsgi (pid=1884): Target WSGI script 'C:/bzr-sandbox/sh-repo/scripts/smartserver.wsgi' cannot be loaded as Python module.
[Wed Jun 09 11:59:50 2010] [error] [client 10.37.1.54] mod_wsgi (pid=1884): Exception occurred processing WSGI script 'C:/bzr-sandbox/sh-repo/scripts/smartserver.wsgi'.
[Wed Jun 09 11:59:50 2010] [error] [client 10.37.1.54] Traceback (most recent call last):
[Wed Jun 09 11:59:50 2010] [error] [client 10.37.1.54] File "C:/bzr-sandbox/sh-repo/scripts/smartserver.wsgi", line 1, in <module>
[Wed Jun 09 11:59:50 2010] [error] [client 10.37.1.54] from bzrlib.transport.http import wsgi
[Wed Jun 09 11:59:50 2010] [error] [client 10.37.1.54] ImportError: No module named bzrlib.transport.http
[Wed Jun 09 11:59:50 2010] [error] [client 10.37.1.54] File does not exist: C:/tools/apache/htdocs/favicon.ico

From my error.log which shows the module working with python 2.6.5:

[Wed Jun 09 13:42:33 2010] [warn] mod_wsgi: Compiled for Python/2.6.2.
[Wed Jun 09 13:42:33 2010] [warn] mod_wsgi: Runtime using Python/2.6.5.
[Wed Jun 09 13:42:33 2010] [notice] Apache/2.2.15 (Win32) mod_wsgi/3.0 Python/2.6.5 configured -- resuming normal operations
[Wed Jun 09 13:42:33 2010] [notice] Server built: Mar 4 2010 11:27:46
[Wed Jun 09 13:42:33 2010] [notice] Parent: Created child process 5884
[Wed Jun 09 13:42:33 2010] [warn] mod_wsgi: Compiled for Python/2.6.2.
[Wed Jun 09 13:42:33 2010] [warn] mod_wsgi: Runtime using Python/2.6.5.
[Wed Jun 09 13:42:33 2010] [notice] Child 5884: Child process is running
[Wed Jun 09 13:42:33 2010] [notice] Child 5884: Acquired the start mutex.
[Wed Jun 09 13:42:33 2010] [notice] Child 5884: Starting 64 worker threads.
[Wed Jun 09 13:42:33 2010] [notice] Child 5884: Starting thread to listen on port 7070.

My config now looks like this (the myapp page works correctly):

<VirtualHost mycomputer:7070>
ServerAdmin <email address hidden>
DocumentRoot "C:/tools/apache/htdocs"

ServerName mycomputer.blah.com
ServerAlias mycomputer.blah.com mycomputer

ErrorLog logs/bzr-error.log
CustomLog logs/bzr-access.log common

WSGIScriptAlias /myapp C:/bzr-sandbox/sh-repo/scripts/myapp.wsgi

  <Proxy *>
    Order allow,deny
    Allow from all
    Satisfy all
  </Proxy>

  <IfModule dir_module>
    DirectoryIndex index.php index.html
  </IfModule>

  WSGIScriptAliasMatch ^/bzr/.*/\.bzr/smart$ C:/bzr-sandbox/sh-repo/scripts/smartserver.wsgi
  #WSGIScriptAliasMatch ^/bzr/.*/\.bzr/smart$ C:/bzr-sandbox/sh-repo/scripts/myapp.wsgi
  #WSGIScriptAliasMatch ^/bzr/.*/\.bzr/smart$ C:/bzr-sandbox/sh-repo/scripts/bzr-smart.py
  RewriteEngine On
  RewriteCond %{REQUEST_URI} !^/bzr/.*/\.bzr/smart$
  RewriteRule ^/bzr/(.*/\.bzr/.*)$ C:/bzr-sandbox/sh-repo/$1

  <Directory C:/bzr-sandbox/sh-repo>
        Order allow,deny
        Allow from all
        WSGIApplicationGroup %{GLOBAL}
        SetEnv bzr_wsgi.root C:/bzr-sandbox/sh-repo
        SetEnv bzr_wsgi.prefix
  </Directory>

</VirtualHost>

My scripts/smartserver.wsgi looks like this:
from bzrlib.transport.http import wsgi

def application(environ, start_response):
    return wsgi.make_app(
        root=environ["bzr_wsgi.root"],
        prefix=environ["bzr_wsgi.prefix"],
        readonly=True,
        enable_logging=True
    )(environ, start_response)

Parth Malwankar (parthm) said on 2010-06-09:

Hi Jemsquash,

I don't know much about active directory on Windows but the following lines are curious:

> [Wed Jun 09 11:59:50 2010] [error] [client 10.37.1.54] from bzrlib.transport.http import wsgi
> [Wed Jun 09 11:59:50 2010] [error] [client 10.37.1.54] ImportError: No module named bzrlib.transport.http

Bazaar ships with bzrlib.transport.http.wsgi so that should work. E.g.

[bzrlib]% ls transport/http
ca_bundle.py __init__.py _pycurl.py response.py _urllib2_wrappers.py _urllib.py wsgi.py

^
Note the wsgi.py file above.

[bzrlib]% python
Python 2.6.5 (r265:79063, Apr 16 2010, 13:09:56)
[GCC 4.4.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from bzrlib.transport.http import wsgi
>>>

What version of bazaar are you using?
In case you are not using it already, on Windows you could try reinstalling[1]. The standalone installer comes with its own python, while the others use installed python. You should try the "Python 2.6 based" installer first. Following that you can try doing the import like I have done above to see if that works.

The following lines may not be anything to worry about but we can't be sure. If something has changed at the C binding level in Python it may be an issue. But the above definitely is a problem.

> [Wed Jun 09 13:42:33 2010] [warn] mod_wsgi: Compiled for Python/2.6.2.
> [Wed Jun 09 13:42:33 2010] [warn] mod_wsgi: Runtime using Python/2.6.5.

[1] http://wiki.bazaar.canonical.com/WindowsDownloads

Jemsquash (james-stewart-bigfoot) said on 2010-06-10:

You are right about the windows installer. I used the python installer and that issue has gone away.

I'm now at the stage where it seems like it should be working when I try and branch I get a 403 error:
C:\temp>bzr branch bzr+http://blah:7070/bzr/trunk trunk
bzr: ERROR: Generic bzr smart protocol error: Invalid http response for http://blah:7070/bzr/trunk/.bzr
/smart: Unknown response code 403

from my bzr.log:

Thu 2010-06-10 09:05:05 +0800
0.078 bazaar version: 2.1.1
0.078 bzr arguments: [u'branch', u'bzr+http://blah.dpi.wa.gov.au:7070/bzr/trunk', u'trunk']
0.078 looking for plugins in C:/Documents and Settings/jastewart/Application Data/bazaar/2.0/plugins
0.078 looking for plugins in C:\Python26\lib\site-packages\bzrlib\plugins
0.187 encoding stdout as sys.stdout encoding 'cp850'
0.265 Traceback (most recent call last):
  File "C:\Python26\lib\site-packages\bzrlib\commands.py", line 853, in exception_to_return_code
    return the_callable(*args, **kwargs)
  File "C:\Python26\lib\site-packages\bzrlib\commands.py", line 1055, in run_bzr
    ret = run(*run_argv)
  File "C:\Python26\lib\site-packages\bzrlib\commands.py", line 661, in run_argv_aliases
    return self.run_direct(**all_cmd_args)
  File "C:\Python26\lib\site-packages\bzrlib\commands.py", line 665, in run_direct
    return self._operation.run_simple(*args, **kwargs)
  File "C:\Python26\lib\site-packages\bzrlib\cleanup.py", line 122, in run_simple
    self.cleanups, self.func, *args, **kwargs)
  File "C:\Python26\lib\site-packages\bzrlib\cleanup.py", line 156, in _do_with_cleanups
    result = func(*args, **kwargs)
  File "C:\Python26\lib\site-packages\bzrlib\builtins.py", line 1194, in run
    from_location)
  File "C:\Python26\lib\site-packages\bzrlib\bzrdir.py", line 958, in open_tree_or_branch
    bzrdir = klass.open(location)
  File "C:\Python26\lib\site-packages\bzrlib\bzrdir.py", line 843, in open
    return BzrDir.open_from_transport(t, _unsupported=_unsupported)
  File "C:\Python26\lib\site-packages\bzrlib\bzrdir.py", line 873, in open_from_transport
    redirected)
  File "C:\Python26\lib\site-packages\bzrlib\lazy_import.py", line 125, in __call__
    return obj(*args, **kwargs)
  File "C:\Python26\lib\site-packages\bzrlib\transport\__init__.py", line 1644, in do_catching_redirections
    return action(transport)
  File "C:\Python26\lib\site-packages\bzrlib\bzrdir.py", line 860, in find_format
    transport, _server_formats=_server_formats)
  File "C:\Python26\lib\site-packages\bzrlib\bzrdir.py", line 1820, in find_format
    return format.probe_transport(transport)
  File "C:\Python26\lib\site-packages\bzrlib\bzrdir.py", line 1830, in probe_transport
    format_string = transport.get_bytes(".bzr/branch-format")
  File "C:\Python26\lib\site-packages\bzrlib\transport\remote.py", line 226, in get_bytes
    resp, response_handler = self._client.call_expecting_body('get', remote)
  File "C:\Python26\lib\site-packages\bzrlib\smart\client.py", line 145, in call_expecting_body
    method, args, expect_response_body=True)
  File "C:\Python26\lib\site-packages\bzrlib\smart\client.py", line 90, in _call_and_read_response
    readv_body=readv_body, body_stream=body_stream)
  File "C:\Python26\lib\site-packages\bzrlib\smart\client.py", line 63, in _send_request
    encoder.call(method, *args)
  File "C:\Python26\lib\site-packages\bzrlib\smart\protocol.py", line 1303, in call
    self._medium_request.finished_writing()
  File "C:\Python26\lib\site-packages\bzrlib\smart\medium.py", line 441, in finished_writing
    self._finished_writing()
  File "C:\Python26\lib\site-packages\bzrlib\transport\http\__init__.py", line 646, in _finished_writing
    data = self._medium.send_http_smart_request(self._buffer)
  File "C:\Python26\lib\site-packages\bzrlib\transport\http\__init__.py", line 621, in send_http_smart_request
    raise errors.SmartProtocolError(str(e))
SmartProtocolError: Generic bzr smart protocol error: Invalid http response for http://blah:7070/bzr/trunk/.bzr/smart: Unknown response code 403

0.265 Transferred: 1KiB (81.4K/s r:1K w:1K)
0.265 return code 3

There are no errors in any apache log.

I have changed the smartserver.wsgi to look as follows:

from bzrlib.transport.http import wsgi

def application(environ, start_response):
    return wsgi.make_app(
        root='C:/bzr-sandbox/sh-repo',
        prefix='/bzr',
        readonly=True,
        enable_logging=True
    )(environ, start_response)

My vhosts looks as follows:

<VirtualHost blah:7070>
  ServerAdmin <email address hidden>
  DocumentRoot "C:/vsfz/tools/apache/htdocs"
  #DocumentRoot "C:/bzr-sandbox/sh-repo"

ServerName blah.com
ServerAlias blah.com blah

ErrorLog logs/bzr-error.log
CustomLog logs/bzr-access.log common

WSGIScriptAlias /myapp C:/bzr-sandbox/sh-repo/scripts/myapp.wsgi

</VirtualHost>

Jemsquash (james-stewart-bigfoot) said on 2010-06-10:

I've analysed this a bit using wireshark and it seems to be sending my request to our proxy which is denying it. I don't think that it has anything to do with the apache config at the moment.

I have to now figure out why bzr is trying to send the request to the proxy and not my local machine.

Jemsquash (james-stewart-bigfoot) said on 2010-06-10:

OK - I've run: set no_proxy=blah
and then bzr branch bzr+http://blah:7070/bzr/trunk trunk
and voila:
Branched 2 revision(s).

Now to go back to the active directory stuff.

Jemsquash (james-stewart-bigfoot) said on 2010-06-10:

It seems to be working now with SSPI integration.

My directory part of the vhosts is as follows:

  <Directory C:/bzr-sandbox/sh-repo>
      AuthName "Bzr Development"
      AuthType SSPI
      SSPIAuth On
      # module is the final authority
      SSPIAuthoritative On
      # only authenticate against DEVDomain no others
      SSPIDomain DEVDOMAIN
      # keep domain name in userid string
      SSPIOmitDomain Off
      # let non-IE clients authenticate
      SSPIOfferBasic On
      # first we try NTLM authentication and then basic authentication
      SSPIBasicPreferred Off
      # need to be in dev Group to be able to access bzr
      Require group "DEVDOMAIN\DEV-Grp"

        Order allow,deny
        Allow from all
        WSGIApplicationGroup %{GLOBAL}
        SetEnv bzr_wsgi.root C:/bzr-sandbox/sh-repo
        SetEnv bzr_wsgi.prefix /bzr
      # module specific

</Directory>

Alexander Belchenko (bialix) said on 2010-06-10:

#10

Jemsquash пишет:
> Question #113185 on Bazaar changed:
> https://answers.launchpad.net/bzr/+question/113185
>
> Status: Open => Solved
>
> Jemsquash confirmed that the question is solved:
> It seems to be working now with SSPI integration.
>
> My directory part of the vhosts is as follows:
>
> <Directory C:/bzr-sandbox/sh-repo>
> AuthName "Bzr Development"
> AuthType SSPI
> SSPIAuth On
> # module is the final authority
> SSPIAuthoritative On
> # only authenticate against DEVDomain no others
> SSPIDomain DEVDOMAIN
> # keep domain name in userid string
> SSPIOmitDomain Off
> # let non-IE clients authenticate
> SSPIOfferBasic On
> # first we try NTLM authentication and then basic authentication
> SSPIBasicPreferred Off
> # need to be in dev Group to be able to access bzr
> Require group "DEVDOMAIN\DEV-Grp"
>
> Order allow,deny
> Allow from all
> WSGIApplicationGroup %{GLOBAL}
> SetEnv bzr_wsgi.root C:/bzr-sandbox/sh-repo
> SetEnv bzr_wsgi.prefix /bzr
> # module specific
>
> </Directory>
>

Can you share your experience or final solution in your blog or on bzr
wiki (http://wiki.bazaar.canonical.com) and post the link here so
other people can get the answer when they will need it? Thanks.

Jemsquash (james-stewart-bigfoot) said on 2010-06-11:

#11

I will try find a suitable spot to document what I've done. I'll also list the final versions of everything that I'm using here in case I don't ever get around to documenting it.

Windows XP professional service pack 2
apache 2.2.15
Python bazaar installer for bazaar 2.1.1
Python 2.6.5
mod_wsgi-win32-ap22py26-3.0.so (from http://code.google.com/p/modwsgi/downloads/list)
mod_rewrite - (I think it came with apache.)
mod_sspi - I think it is mod_auth_sspi-1.0.4-2.2.3-w32.zip (I have a couple of downloads of this but the size is the same).

Ignore references to apache 2.2.9 in my postings before (even in the final solution), I left them in there because I didn't want to copy the htdocs folders from 2.2.9 to the 2.2.15 folder (in case someone is worried about it). I seem to remember that the wsgi module did not work with 2.2.9 hence 2.2.15 although my memory is a bit vague about this.

I need to document this for our configuration managers so hopefully I can use the wiki for this purpose.

Jemsquash (james-stewart-bigfoot) said on 2010-06-16:

#12

I have tried to document this at the following location:

http://wiki.bazaar.canonical.com/BzrSmartWSGIApacheActiveDirectory

I have not put the downloads there but it might be useful. What do others think about attaching all dependencies to the page? I have put links there but they can get out of date pretty quickly. It can often be a real pain trying to find correct compiled versions of everything. Does the wiki have enough space to store an apache zip file plus all its dependencies?

Alexander Belchenko (bialix) said on 2010-06-16:

#13

Jemsquash пишет:
> Question #113185 on Bazaar changed:
> https://answers.launchpad.net/bzr/+question/113185
>
> Jemsquash posted a new comment:
> I have tried to document this at the following location:
>
> http://wiki.bazaar.canonical.com/BzrSmartWSGIApacheActiveDirectory
>
> I have not put the downloads there but it might be useful. What do
> others think about attaching all dependencies to the page? I have put
> links there but they can get out of date pretty quickly. It can often be
> a real pain trying to find correct compiled versions of everything. Does
> the wiki have enough space to store an apache zip file plus all its
> dependencies?

Thank you for your work! The document is very good.

I know that binaries tends to go out of date as time passed, but I think
the more important thing that you documented steps and required
dependencies. So it should be possible to use newer version of tools
when time passed.

Thank you again.

Andrew Bennetts (spiv) said on 2010-06-17:

#14

That looks great, thank you!

I suggest leaving just links, rather than attaching files. Links can go out of date, but if they do it's often for a good reason, e.g. a security update. It's a wiki, people can always fix the links when they break :)

To post a message you must log in.

Ask a question

Edit question

Bazaar

How do I get smart server to use active directory

Question information

Related bugs

Related FAQ:

Subscribers