OpenStack Compute (nova)

Merge lp:~vishvananda/nova/project-vpns into lp:~hudson-openstack/nova/trunk

  1. project-vpns
  2. Merge into trunk
Proposed by Vish Ishaya
Status: Merged
Approved by: Eric Day
Approved revision: 410
Merged at revision: 477
Proposed branch: lp:~vishvananda/nova/project-vpns
Merge into: lp:~hudson-openstack/nova/trunk
Diff against target: 1490 lines (+642/-310)
19 files modified
CA/geninter.sh (+17/-9)
CA/genrootca.sh (+1/-0)
CA/genvpn.sh (+36/-0)
CA/openssl.cnf.tmpl (+2/-1)
bin/nova-manage (+52/-32)
nova/api/__init__.py (+1/-3)
nova/api/cloudpipe/__init__.py (+0/-69)
nova/api/ec2/cloud.py (+10/-6)
nova/auth/manager.py (+43/-37)
nova/cloudpipe/bootscript.template (+16/-29)
nova/cloudpipe/pipelib.py (+85/-42)
nova/crypto.py (+150/-43)
nova/db/api.py (+48/-2)
nova/db/sqlalchemy/api.py (+103/-11)
nova/db/sqlalchemy/models.py (+11/-1)
nova/flags.py (+12/-6)
nova/network/linux_net.py (+2/-4)
nova/tests/auth_unittest.py (+7/-10)
nova/utils.py (+46/-5)
To merge this branch: bzr merge lp:~vishvananda/nova/project-vpns
Reviewer Review Type Date Requested Status
Eric Day (community) Approve
Soren Hansen (community) Approve
Todd Willey (community) Needs Fixing
Review via email: mp+40243@code.launchpad.net

Commit message

Fixes per-project vpns (cloudpipe) and adds manage commands and support for certificate revocation.

Description of the change

Project Specific VPNs!

Cloudpipe has been a part of Nova, but it is currently broken. This branch makes cloudpipe work again. It adds an option for per-project CAs, supports revocation of certs, and automatic vpn creation and port forwarding.

Description of cloudpipe can be found here:
https://code.arc.nasa.gov/nova/devguide/cloudpipe.html#cloudpipe
and an image showing how it works is here:
https://code.arc.nasa.gov/nova/_images/cloudpipe.png

To post a comment you must log in.
Revision history for this message
Youcef Laribi (youcef-laribi) wrote :

The link
https://code.arc.nasa.gov/nova/devguide/cloudpipe.html#cloudpipe

seems dead (returns a 404).

Revision history for this message
Vish Ishaya (vishvananda) wrote :

Sorry, moved to here:

https://code.arc.nasa.gov/nova/devref/cloudpipe.html#cloudpipe

On Nov 15, 2010, at 9:57 AM, Youcef Laribi wrote:

> The link
> https://code.arc.nasa.gov/nova/devguide/cloudpipe.html#cloudpipe
>
> seems dead (returns a 404).
> --
> https://code.launchpad.net/~vishvananda/nova/project-vpns/+merge/40243
> You are the owner of lp:~vishvananda/nova/project-vpns.

Revision history for this message
Todd Willey (xtoddx) wrote :

LOG variable doesn't exist in nova/auth/manager.

review: Needs Fixing
Revision history for this message
Vish Ishaya (vishvananda) wrote :

Todd: good catch. fixed.

Revision history for this message
Soren Hansen (soren) wrote :

Don't grep though /etc/passwd. Use getent instead.

I think the credential_rc_file flag change is confusing and error prone. The help text for it should at least mention that there has to be a %s in there somewhere.

Revision history for this message
Soren Hansen (soren) wrote :

Other than that, I think this is "good enough". If there are bugs here and there, they'll be easier to work out if it's in trunk so we can all help out ironing things out.

review: Needs Fixing
Revision history for this message
Vish Ishaya (vishvananda) wrote :

> Don't grep though /etc/passwd. Use getent instead.
>
> I think the credential_rc_file flag change is confusing and error prone. The
> help text for it should at least mention that there has to be a %s in there
> somewhere.

Fixed the note.

Switched to using getent

Revision history for this message
Soren Hansen (soren) wrote :

Rock!

review: Approve
Revision history for this message
Eric Day (eday) wrote :

So, it looks like the REST API is gone now? Otherwise lgtm.

review: Approve
Revision history for this message
OpenStack Infra (hudson-openstack) wrote :

Attempt to merge into lp:nova failed due to conflicts:

deleting parent in nova/api/cloudpipe
unversioned parent in nova/api/cloudpipe
contents conflict in nova/api/cloudpipe/__init__.py
text conflict in nova/auth/manager.py
text conflict in nova/cloudpipe/pipelib.py
text conflict in nova/crypto.py
text conflict in nova/db/sqlalchemy/models.py

lp:~vishvananda/nova/project-vpns updated
410. By Vish Ishaya

merged i8n and fixed conflicts

Revision history for this message
Vish Ishaya (vishvananda) wrote :

> So, it looks like the REST API is gone now? Otherwise lgtm.

Yes, allowing users to sign certs through the api seemed a little insecure, so now certs are signed in advance and provided to the cloudpipe instance through meta data

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== modified file 'CA/geninter.sh'
2--- CA/geninter.sh 2010-07-20 22:55:38 +0000
3+++ CA/geninter.sh 2010-12-22 17:33:04 +0000
4@@ -16,16 +16,24 @@
5 # License for the specific language governing permissions and limitations
6 # under the License.
7
8-# ARG is the id of the user
9-export SUBJ="/C=US/ST=California/L=MountainView/O=AnsoLabs/OU=NovaDev/CN=customer-intCA-$1"
10-mkdir INTER/$1
11-cd INTER/$1
12+# $1 is the id of the project and $2 is the subject of the cert
13+NAME=$1
14+SUBJ=$2
15+mkdir -p projects/$NAME
16+cd projects/$NAME
17 cp ../../openssl.cnf.tmpl openssl.cnf
18-sed -i -e s/%USERNAME%/$1/g openssl.cnf
19+sed -i -e s/%USERNAME%/$NAME/g openssl.cnf
20 mkdir certs crl newcerts private
21+openssl req -new -x509 -extensions v3_ca -keyout private/cakey.pem -out cacert.pem -days 365 -config ./openssl.cnf -batch -nodes
22 echo "10" > serial
23 touch index.txt
24-openssl genrsa -out private/cakey.pem 1024 -config ./openssl.cnf -batch -nodes
25-openssl req -new -sha2 -key private/cakey.pem -out ../../reqs/inter$1.csr -batch -subj "$SUBJ"
26-cd ../../
27-openssl ca -extensions v3_ca -days 365 -out INTER/$1/cacert.pem -in reqs/inter$1.csr -config openssl.cnf -batch
28+# NOTE(vish): Disabling intermediate ca's because we don't actually need them.
29+# It makes more sense to have each project have its own root ca.
30+# openssl genrsa -out private/cakey.pem 1024 -config ./openssl.cnf -batch -nodes
31+# openssl req -new -sha256 -key private/cakey.pem -out ../../reqs/inter$NAME.csr -batch -subj "$SUBJ"
32+openssl ca -gencrl -config ./openssl.cnf -out crl.pem
33+if [ "`id -u`" != "`grep nova /etc/passwd | cut -d':' -f3`" ]; then
34+ sudo chown -R nova:nogroup .
35+fi
36+# cd ../../
37+# openssl ca -extensions v3_ca -days 365 -out INTER/$NAME/cacert.pem -in reqs/inter$NAME.csr -config openssl.cnf -batch
38
39=== modified file 'CA/genrootca.sh'
40--- CA/genrootca.sh 2010-07-15 05:28:51 +0000
41+++ CA/genrootca.sh 2010-12-22 17:33:04 +0000
42@@ -25,4 +25,5 @@
43 openssl req -new -x509 -extensions v3_ca -keyout private/cakey.pem -out cacert.pem -days 365 -config ./openssl.cnf -batch -nodes
44 touch index.txt
45 echo "10" > serial
46+ openssl ca -gencrl -config ./openssl.cnf -out crl.pem
47 fi
48
49=== added file 'CA/genvpn.sh'
50--- CA/genvpn.sh 1970-01-01 00:00:00 +0000
51+++ CA/genvpn.sh 2010-12-22 17:33:04 +0000
52@@ -0,0 +1,36 @@
53+#!/bin/bash
54+# vim: tabstop=4 shiftwidth=4 softtabstop=4
55+
56+# Copyright 2010 United States Government as represented by the
57+# Administrator of the National Aeronautics and Space Administration.
58+# All Rights Reserved.
59+#
60+# Licensed under the Apache License, Version 2.0 (the "License"); you may
61+# not use this file except in compliance with the License. You may obtain
62+# a copy of the License at
63+#
64+# http://www.apache.org/licenses/LICENSE-2.0
65+#
66+# Unless required by applicable law or agreed to in writing, software
67+# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
68+# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
69+# License for the specific language governing permissions and limitations
70+# under the License.
71+
72+# This gets zipped and run on the cloudpipe-managed OpenVPN server
73+NAME=$1
74+SUBJ=$2
75+
76+mkdir -p projects/$NAME
77+cd projects/$NAME
78+
79+# generate a server priv key
80+openssl genrsa -out server.key 2048
81+
82+# generate a server CSR
83+openssl req -new -key server.key -out server.csr -batch -subj "$SUBJ"
84+
85+novauid=`getent passwd nova | awk -F: '{print $3}'`
86+if [ ! -z "${novauid}" ] && [ "`id -u`" != "${novauid}" ]; then
87+ sudo chown -R nova:nogroup .
88+fi
89
90=== modified file 'CA/openssl.cnf.tmpl'
91--- CA/openssl.cnf.tmpl 2010-07-15 05:28:51 +0000
92+++ CA/openssl.cnf.tmpl 2010-12-22 17:33:04 +0000
93@@ -24,7 +24,6 @@
94
95 [ ca ]
96 default_ca = CA_default
97-unique_subject = no
98
99 [ CA_default ]
100 serial = $dir/serial
101@@ -32,6 +31,8 @@
102 new_certs_dir = $dir/newcerts
103 certificate = $dir/cacert.pem
104 private_key = $dir/private/cakey.pem
105+unique_subject = no
106+default_crl_days = 365
107 default_days = 365
108 default_md = md5
109 preserve = no
110
111=== renamed directory 'CA/INTER' => 'CA/projects'
112=== modified file 'bin/nova-manage'
113--- bin/nova-manage 2010-12-14 23:22:03 +0000
114+++ bin/nova-manage 2010-12-22 17:33:04 +0000
115@@ -72,6 +72,7 @@
116 gettext.install('nova', unicode=1)
117
118 from nova import context
119+from nova import crypto
120 from nova import db
121 from nova import exception
122 from nova import flags
123@@ -96,47 +97,43 @@
124 self.manager = manager.AuthManager()
125 self.pipe = pipelib.CloudPipe()
126
127- def list(self):
128- """Print a listing of the VPNs for all projects."""
129+ def list(self, project=None):
130+ """Print a listing of the VPN data for one or all projects.
131+
132+ args: [project=all]"""
133 print "%-12s\t" % 'project',
134 print "%-20s\t" % 'ip:port',
135+ print "%-20s\t" % 'private_ip',
136 print "%s" % 'state'
137- for project in self.manager.get_projects():
138+ if project:
139+ projects = [self.manager.get_project(project)]
140+ else:
141+ projects = self.manager.get_projects()
142+ # NOTE(vish): This hits the database a lot. We could optimize
143+ # by getting all networks in one query and all vpns
144+ # in aother query, then doing lookups by project
145+ for project in projects:
146 print "%-12s\t" % project.name,
147-
148- try:
149- s = "%s:%s" % (project.vpn_ip, project.vpn_port)
150- except exception.NotFound:
151- s = "None"
152- print "%-20s\t" % s,
153-
154- vpn = self._vpn_for(project.id)
155+ ipport = "%s:%s" % (project.vpn_ip, project.vpn_port)
156+ print "%-20s\t" % ipport,
157+ ctxt = context.get_admin_context()
158+ vpn = db.instance_get_project_vpn(ctxt, project.id)
159 if vpn:
160- command = "ping -c1 -w1 %s > /dev/null; echo $?"
161- out, _err = utils.execute(command % vpn['private_dns_name'],
162- check_exit_code=False)
163- if out.strip() == '0':
164- net = 'up'
165- else:
166- net = 'down'
167- print vpn['private_dns_name'],
168- print vpn['node_name'],
169- print vpn['instance_id'],
170+ address = None
171+ state = 'down'
172+ if vpn.get('fixed_ip', None):
173+ address = vpn['fixed_ip']['address']
174+ if project.vpn_ip and utils.vpn_ping(project.vpn_ip,
175+ project.vpn_port):
176+ state = 'up'
177+ print address,
178+ print vpn['host'],
179+ print vpn['ec2_id'],
180 print vpn['state_description'],
181- print net
182-
183+ print state
184 else:
185 print None
186
187- def _vpn_for(self, project_id):
188- """Get the VPN instance for a project ID."""
189- for instance in db.instance_get_all(context.get_admin_context()):
190- if (instance['image_id'] == FLAGS.vpn_image_id
191- and not instance['state_description'] in
192- ['shutting_down', 'shutdown']
193- and instance['project_id'] == project_id):
194- return instance
195-
196 def spawn(self):
197 """Run all VPNs."""
198 for p in reversed(self.manager.get_projects()):
199@@ -149,6 +146,21 @@
200 """Start the VPN for a given project."""
201 self.pipe.launch_vpn_instance(project_id)
202
203+ def change(self, project_id, ip, port):
204+ """Change the ip and port for a vpn.
205+
206+ args: project, ip, port"""
207+ project = self.manager.get_project(project_id)
208+ if not project:
209+ print 'No project %s' % (project_id)
210+ return
211+ admin = context.get_admin_context()
212+ network_ref = db.project_get_network(admin, project_id)
213+ db.network_update(admin,
214+ network_ref['id'],
215+ {'vpn_public_address': ip,
216+ 'vpn_public_port': int(port)})
217+
218
219 class ShellCommands(object):
220 def bpython(self):
221@@ -295,6 +307,14 @@
222 is_admin = False
223 self.manager.modify_user(name, access_key, secret_key, is_admin)
224
225+ def revoke(self, user_id, project_id=None):
226+ """revoke certs for a user
227+ arguments: user_id [project_id]"""
228+ if project_id:
229+ crypto.revoke_certs_by_user_and_project(user_id, project_id)
230+ else:
231+ crypto.revoke_certs_by_user(user_id)
232+
233
234 class ProjectCommands(object):
235 """Class for managing projects."""
236
237=== modified file 'nova/api/__init__.py'
238--- nova/api/__init__.py 2010-11-23 18:46:07 +0000
239+++ nova/api/__init__.py 2010-12-22 17:33:04 +0000
240@@ -29,9 +29,7 @@
241 import webob.dec
242
243 from nova import flags
244-from nova import utils
245 from nova import wsgi
246-from nova.api import cloudpipe
247 from nova.api import ec2
248 from nova.api import openstack
249 from nova.api.ec2 import metadatarequesthandler
250@@ -41,6 +39,7 @@
251 'subdomain running the OpenStack API')
252 flags.DEFINE_string('ec2api_subdomain', 'ec2',
253 'subdomain running the EC2 API')
254+
255 FLAGS = flags.FLAGS
256
257
258@@ -80,7 +79,6 @@
259 mapper.connect('%s/{path_info:.*}' % s, controller=mrh,
260 conditions=ec2api_subdomain)
261
262- mapper.connect("/cloudpipe/{path_info:.*}", controller=cloudpipe.API())
263 super(API, self).__init__(mapper)
264
265 @webob.dec.wsgify
266
267=== removed directory 'nova/api/cloudpipe'
268=== removed file 'nova/api/cloudpipe/__init__.py'
269--- nova/api/cloudpipe/__init__.py 2010-12-11 20:23:40 +0000
270+++ nova/api/cloudpipe/__init__.py 1970-01-01 00:00:00 +0000
271@@ -1,69 +0,0 @@
272-# vim: tabstop=4 shiftwidth=4 softtabstop=4
273-
274-# Copyright 2010 United States Government as represented by the
275-# Administrator of the National Aeronautics and Space Administration.
276-# All Rights Reserved.
277-#
278-# Licensed under the Apache License, Version 2.0 (the "License"); you may
279-# not use this file except in compliance with the License. You may obtain
280-# a copy of the License at
281-#
282-# http://www.apache.org/licenses/LICENSE-2.0
283-#
284-# Unless required by applicable law or agreed to in writing, software
285-# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
286-# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
287-# License for the specific language governing permissions and limitations
288-# under the License.
289-
290-"""
291-REST API Request Handlers for CloudPipe
292-"""
293-
294-import logging
295-import urllib
296-import webob
297-import webob.dec
298-import webob.exc
299-
300-from nova import crypto
301-from nova import wsgi
302-from nova.auth import manager
303-from nova.api.ec2 import cloud
304-
305-
306-_log = logging.getLogger("api")
307-_log.setLevel(logging.DEBUG)
308-
309-
310-class API(wsgi.Application):
311-
312- def __init__(self):
313- self.controller = cloud.CloudController()
314-
315- @webob.dec.wsgify
316- def __call__(self, req):
317- if req.method == 'POST':
318- return self.sign_csr(req)
319- _log.debug(_("Cloudpipe path is %s") % req.path_info)
320- if req.path_info.endswith("/getca/"):
321- return self.send_root_ca(req)
322- return webob.exc.HTTPNotFound()
323-
324- def get_project_id_from_ip(self, ip):
325- # TODO(eday): This was removed with the ORM branch, fix!
326- instance = self.controller.get_instance_by_ip(ip)
327- return instance['project_id']
328-
329- def send_root_ca(self, req):
330- _log.debug(_("Getting root ca"))
331- project_id = self.get_project_id_from_ip(req.remote_addr)
332- res = webob.Response()
333- res.headers["Content-Type"] = "text/plain"
334- res.body = crypto.fetch_ca(project_id)
335- return res
336-
337- def sign_csr(self, req):
338- project_id = self.get_project_id_from_ip(req.remote_addr)
339- cert = self.str_params['cert']
340- return crypto.sign_csr(urllib.unquote(cert), project_id)
341
342=== modified file 'nova/api/ec2/cloud.py'
343--- nova/api/ec2/cloud.py 2010-12-22 15:40:24 +0000
344+++ nova/api/ec2/cloud.py 2010-12-22 17:33:04 +0000
345@@ -196,15 +196,19 @@
346 if FLAGS.region_list:
347 regions = []
348 for region in FLAGS.region_list:
349- name, _sep, url = region.partition('=')
350+ name, _sep, host = region.partition('=')
351+ endpoint = '%s://%s:%s%s' % (FLAGS.ec2_prefix,
352+ host,
353+ FLAGS.cc_port,
354+ FLAGS.ec2_suffix)
355 regions.append({'regionName': name,
356- 'regionEndpoint': url})
357+ 'regionEndpoint': endpoint})
358 else:
359 regions = [{'regionName': 'nova',
360- 'regionEndpoint': FLAGS.ec2_url}]
361- if region_name:
362- regions = [r for r in regions if r['regionName'] in region_name]
363- return {'regionInfo': regions}
364+ 'regionEndpoint': '%s://%s:%s%s' % (FLAGS.ec2_prefix,
365+ FLAGS.cc_host,
366+ FLAGS.cc_port,
367+ FLAGS.ec2_suffix)}]
368
369 def describe_snapshots(self,
370 context,
371
372=== modified file 'nova/auth/manager.py'
373--- nova/auth/manager.py 2010-12-20 15:15:07 +0000
374+++ nova/auth/manager.py 2010-12-22 17:33:04 +0000
375@@ -64,12 +64,9 @@
376 'Filename of private key in credentials zip')
377 flags.DEFINE_string('credential_cert_file', 'cert.pem',
378 'Filename of certificate in credentials zip')
379-flags.DEFINE_string('credential_rc_file', 'novarc',
380- 'Filename of rc in credentials zip')
381-flags.DEFINE_string('credential_cert_subject',
382- '/C=US/ST=California/L=MountainView/O=AnsoLabs/'
383- 'OU=NovaDev/CN=%s-%s',
384- 'Subject for certificate for users')
385+flags.DEFINE_string('credential_rc_file', '%src',
386+ 'Filename of rc in credentials zip, %s will be '
387+ 'replaced by name of the region (nova by default)')
388 flags.DEFINE_string('auth_driver', 'nova.auth.dbdriver.DbDriver',
389 'Driver that auth manager uses')
390
391@@ -543,11 +540,10 @@
392 """
393
394 network_ref = db.project_get_network(context.get_admin_context(),
395- Project.safe_id(project))
396+ Project.safe_id(project), False)
397
398- if not network_ref['vpn_public_port']:
399- raise exception.NotFound(_('project network data has not '
400- 'been set'))
401+ if not network_ref:
402+ return (None, None)
403 return (network_ref['vpn_public_address'],
404 network_ref['vpn_public_port'])
405
406@@ -629,27 +625,37 @@
407 def get_key_pairs(context):
408 return db.key_pair_get_all_by_user(context.elevated(), context.user_id)
409
410- def get_credentials(self, user, project=None):
411+ def get_credentials(self, user, project=None, use_dmz=True):
412 """Get credential zip for user in project"""
413 if not isinstance(user, User):
414 user = self.get_user(user)
415 if project is None:
416 project = user.id
417 pid = Project.safe_id(project)
418- rc = self.__generate_rc(user.access, user.secret, pid)
419- private_key, signed_cert = self._generate_x509_cert(user.id, pid)
420+ private_key, signed_cert = crypto.generate_x509_cert(user.id, pid)
421
422 tmpdir = tempfile.mkdtemp()
423 zf = os.path.join(tmpdir, "temp.zip")
424 zippy = zipfile.ZipFile(zf, 'w')
425- zippy.writestr(FLAGS.credential_rc_file, rc)
426+ if use_dmz and FLAGS.region_list:
427+ regions = {}
428+ for item in FLAGS.region_list:
429+ region, _sep, region_host = item.partition("=")
430+ regions[region] = region_host
431+ else:
432+ regions = {'nova': FLAGS.cc_host}
433+ for region, host in regions.iteritems():
434+ rc = self.__generate_rc(user.access,
435+ user.secret,
436+ pid,
437+ use_dmz,
438+ host)
439+ zippy.writestr(FLAGS.credential_rc_file % region, rc)
440+
441 zippy.writestr(FLAGS.credential_key_file, private_key)
442 zippy.writestr(FLAGS.credential_cert_file, signed_cert)
443
444- try:
445- (vpn_ip, vpn_port) = self.get_project_vpn_data(project)
446- except exception.NotFound:
447- vpn_ip = None
448+ (vpn_ip, vpn_port) = self.get_project_vpn_data(project)
449 if vpn_ip:
450 configfile = open(FLAGS.vpn_client_template, "r")
451 s = string.Template(configfile.read())
452@@ -662,7 +668,7 @@
453 else:
454 logging.warn(_("No vpn data for project %s"), pid)
455
456- zippy.writestr(FLAGS.ca_file, crypto.fetch_ca(user.id))
457+ zippy.writestr(FLAGS.ca_file, crypto.fetch_ca(pid))
458 zippy.close()
459 with open(zf, 'rb') as f:
460 read_buffer = f.read()
461@@ -670,38 +676,38 @@
462 shutil.rmtree(tmpdir)
463 return read_buffer
464
465- def get_environment_rc(self, user, project=None):
466+ def get_environment_rc(self, user, project=None, use_dmz=True):
467 """Get credential zip for user in project"""
468 if not isinstance(user, User):
469 user = self.get_user(user)
470 if project is None:
471 project = user.id
472 pid = Project.safe_id(project)
473- return self.__generate_rc(user.access, user.secret, pid)
474+ return self.__generate_rc(user.access, user.secret, pid, use_dmz)
475
476 @staticmethod
477- def __generate_rc(access, secret, pid):
478+ def __generate_rc(access, secret, pid, use_dmz=True, host=None):
479 """Generate rc file for user"""
480+ if use_dmz:
481+ cc_host = FLAGS.cc_dmz
482+ else:
483+ cc_host = FLAGS.cc_host
484+ # NOTE(vish): Always use the dmz since it is used from inside the
485+ # instance
486+ s3_host = FLAGS.s3_dmz
487+ if host:
488+ s3_host = host
489+ cc_host = host
490 rc = open(FLAGS.credentials_template).read()
491 rc = rc % {'access': access,
492 'project': pid,
493 'secret': secret,
494- 'ec2': FLAGS.ec2_url,
495- 's3': 'http://%s:%s' % (FLAGS.s3_host, FLAGS.s3_port),
496+ 'ec2': '%s://%s:%s%s' % (FLAGS.ec2_prefix,
497+ cc_host,
498+ FLAGS.cc_port,
499+ FLAGS.ec2_suffix),
500+ 's3': 'http://%s:%s' % (s3_host, FLAGS.s3_port),
501 'nova': FLAGS.ca_file,
502 'cert': FLAGS.credential_cert_file,
503 'key': FLAGS.credential_key_file}
504 return rc
505-
506- def _generate_x509_cert(self, uid, pid):
507- """Generate x509 cert for user"""
508- (private_key, csr) = crypto.generate_x509_cert(
509- self.__cert_subject(uid))
510- # TODO(joshua): This should be async call back to the cloud controller
511- signed_cert = crypto.sign_csr(csr, pid)
512- return (private_key, signed_cert)
513-
514- @staticmethod
515- def __cert_subject(uid):
516- """Helper to generate cert subject"""
517- return FLAGS.credential_cert_subject % (uid, utils.isotime())
518
519=== renamed file 'nova/cloudpipe/bootscript.sh' => 'nova/cloudpipe/bootscript.template'
520--- nova/cloudpipe/bootscript.sh 2010-07-29 21:48:10 +0000
521+++ nova/cloudpipe/bootscript.template 2010-12-22 17:33:04 +0000
522@@ -19,45 +19,32 @@
523
524 # This gets zipped and run on the cloudpipe-managed OpenVPN server
525
526-export SUPERVISOR="http://10.255.255.1:8773/cloudpipe"
527-export VPN_IP=`ifconfig | grep 'inet addr:'| grep -v '127.0.0.1' | cut -d: -f2 | awk '{print $1}'`
528-export BROADCAST=`ifconfig | grep 'inet addr:'| grep -v '127.0.0.1' | cut -d: -f3 | awk '{print $1}'`
529-export DHCP_MASK=`ifconfig | grep 'inet addr:'| grep -v '127.0.0.1' | cut -d: -f4 | awk '{print $1}'`
530+export VPN_IP=`ifconfig | grep 'inet addr:'| grep -v '127.0.0.1' | cut -d: -f2 | awk '{print $$1}'`
531+export BROADCAST=`ifconfig | grep 'inet addr:'| grep -v '127.0.0.1' | cut -d: -f3 | awk '{print $$1}'`
532+export DHCP_MASK=`ifconfig | grep 'inet addr:'| grep -v '127.0.0.1' | cut -d: -f4 | awk '{print $$1}'`
533 export GATEWAY=`netstat -r | grep default | cut -d' ' -f10`
534-export SUBJ="/C=US/ST=California/L=MountainView/O=AnsoLabs/OU=NovaDev/CN=customer-vpn-$VPN_IP"
535
536-DHCP_LOWER=`echo $BROADCAST | awk -F. '{print $1"."$2"."$3"." $4 - 10 }'`
537-DHCP_UPPER=`echo $BROADCAST | awk -F. '{print $1"."$2"."$3"." $4 - 1 }'`
538+DHCP_LOWER=`echo $$BROADCAST | awk -F. '{print $$1"."$$2"."$$3"." $$4 - ${num_vpn} }'`
539+DHCP_UPPER=`echo $$BROADCAST | awk -F. '{print $$1"."$$2"."$$3"." $$4 - 1 }'`
540
541 # generate a server DH
542 openssl dhparam -out /etc/openvpn/dh1024.pem 1024
543
544-# generate a server priv key
545-openssl genrsa -out /etc/openvpn/server.key 2048
546-
547-# generate a server CSR
548-openssl req -new -key /etc/openvpn/server.key -out /etc/openvpn/server.csr -batch -subj "$SUBJ"
549-
550-# URLEncode the CSR
551-CSRTEXT=`cat /etc/openvpn/server.csr`
552-CSRTEXT=$(python -c "import urllib; print urllib.quote('''$CSRTEXT''')")
553-
554-# SIGN the csr and save as server.crt
555-# CURL fetch to the supervisor, POSTing the CSR text, saving the result as the CRT file
556-curl --fail $SUPERVISOR -d "cert=$CSRTEXT" > /etc/openvpn/server.crt
557-curl --fail $SUPERVISOR/getca/ > /etc/openvpn/ca.crt
558-
559+cp crl.pem /etc/openvpn/
560+cp server.key /etc/openvpn/
561+cp ca.crt /etc/openvpn/
562+cp server.crt /etc/openvpn/
563 # Customize the server.conf.template
564 cd /etc/openvpn
565
566-sed -e s/VPN_IP/$VPN_IP/g server.conf.template > server.conf
567-sed -i -e s/DHCP_SUBNET/$DHCP_MASK/g server.conf
568-sed -i -e s/DHCP_LOWER/$DHCP_LOWER/g server.conf
569-sed -i -e s/DHCP_UPPER/$DHCP_UPPER/g server.conf
570+sed -e s/VPN_IP/$$VPN_IP/g server.conf.template > server.conf
571+sed -i -e s/DHCP_SUBNET/$$DHCP_MASK/g server.conf
572+sed -i -e s/DHCP_LOWER/$$DHCP_LOWER/g server.conf
573+sed -i -e s/DHCP_UPPER/$$DHCP_UPPER/g server.conf
574 sed -i -e s/max-clients\ 1/max-clients\ 10/g server.conf
575
576-echo "\npush \"route 10.255.255.1 255.255.255.255 $GATEWAY\"\n" >> server.conf
577-echo "\npush \"route 10.255.255.253 255.255.255.255 $GATEWAY\"\n" >> server.conf
578-echo "\nduplicate-cn\n" >> server.conf
579+echo "push \"route ${dmz_net} ${dmz_mask} $$GATEWAY\"" >> server.conf
580+echo "duplicate-cn" >> server.conf
581+echo "crl-verify /etc/openvpn/crl.pem" >> server.conf
582
583 /etc/init.d/openvpn start
584
585=== modified file 'nova/cloudpipe/pipelib.py'
586--- nova/cloudpipe/pipelib.py 2010-12-11 20:23:40 +0000
587+++ nova/cloudpipe/pipelib.py 2010-12-22 17:33:04 +0000
588@@ -22,13 +22,15 @@
589
590 """
591
592-import base64
593 import logging
594 import os
595+import string
596 import tempfile
597 import zipfile
598
599 from nova import context
600+from nova import crypto
601+from nova import db
602 from nova import exception
603 from nova import flags
604 from nova import utils
605@@ -39,8 +41,17 @@
606
607 FLAGS = flags.FLAGS
608 flags.DEFINE_string('boot_script_template',
609- utils.abspath('cloudpipe/bootscript.sh'),
610- 'Template for script to run on cloudpipe instance boot')
611+ utils.abspath('cloudpipe/bootscript.template'),
612+ _('Template for script to run on cloudpipe instance boot'))
613+flags.DEFINE_string('dmz_net',
614+ '10.0.0.0',
615+ _('Network to push into openvpn config'))
616+flags.DEFINE_string('dmz_mask',
617+ '255.255.255.0',
618+ _('Netmask to push into openvpn config'))
619+
620+
621+LOG = logging.getLogger('nova-cloudpipe')
622
623
624 class CloudPipe(object):
625@@ -48,64 +59,96 @@
626 self.controller = cloud.CloudController()
627 self.manager = manager.AuthManager()
628
629- def launch_vpn_instance(self, project_id):
630- logging.debug(_("Launching VPN for %s") % (project_id))
631- project = self.manager.get_project(project_id)
632+ def get_encoded_zip(self, project_id):
633 # Make a payload.zip
634 tmpfolder = tempfile.mkdtemp()
635 filename = "payload.zip"
636 zippath = os.path.join(tmpfolder, filename)
637 z = zipfile.ZipFile(zippath, "w", zipfile.ZIP_DEFLATED)
638-
639- z.write(FLAGS.boot_script_template, 'autorun.sh')
640+ shellfile = open(FLAGS.boot_script_template, "r")
641+ s = string.Template(shellfile.read())
642+ shellfile.close()
643+ boot_script = s.substitute(cc_dmz=FLAGS.cc_dmz,
644+ cc_port=FLAGS.cc_port,
645+ dmz_net=FLAGS.dmz_net,
646+ dmz_mask=FLAGS.dmz_mask,
647+ num_vpn=FLAGS.cnt_vpn_clients)
648+ # genvpn, sign csr
649+ crypto.generate_vpn_files(project_id)
650+ z.writestr('autorun.sh', boot_script)
651+ crl = os.path.join(crypto.ca_folder(project_id), 'crl.pem')
652+ z.write(crl, 'crl.pem')
653+ server_key = os.path.join(crypto.ca_folder(project_id), 'server.key')
654+ z.write(server_key, 'server.key')
655+ ca_crt = os.path.join(crypto.ca_path(project_id))
656+ z.write(ca_crt, 'ca.crt')
657+ server_crt = os.path.join(crypto.ca_folder(project_id), 'server.crt')
658+ z.write(server_crt, 'server.crt')
659 z.close()
660-
661- key_name = self.setup_key_pair(project.project_manager_id, project_id)
662 zippy = open(zippath, "r")
663- context = context.RequestContext(user=project.project_manager,
664- project=project)
665-
666- reservation = self.controller.run_instances(context,
667- # Run instances expects encoded userdata, it is decoded in the
668- # get_metadata_call. autorun.sh also decodes the zip file, hence
669- # the double encoding.
670- user_data=zippy.read().encode("base64").encode("base64"),
671+ # NOTE(vish): run instances expects encoded userdata, it is decoded
672+ # in the get_metadata_call. autorun.sh also decodes the zip file,
673+ # hence the double encoding.
674+ encoded = zippy.read().encode("base64").encode("base64")
675+ zippy.close()
676+ return encoded
677+
678+ def launch_vpn_instance(self, project_id):
679+ LOG.debug(_("Launching VPN for %s") % (project_id))
680+ project = self.manager.get_project(project_id)
681+ ctxt = context.RequestContext(user=project.project_manager,
682+ project=project)
683+ key_name = self.setup_key_pair(ctxt)
684+ group_name = self.setup_security_group(ctxt)
685+
686+ reservation = self.controller.run_instances(ctxt,
687+ user_data=self.get_encoded_zip(project_id),
688 max_count=1,
689 min_count=1,
690 instance_type='m1.tiny',
691 image_id=FLAGS.vpn_image_id,
692 key_name=key_name,
693- security_groups=["vpn-secgroup"])
694- zippy.close()
695-
696- def setup_key_pair(self, user_id, project_id):
697- key_name = '%s%s' % (project_id, FLAGS.vpn_key_suffix)
698+ security_group=[group_name])
699+
700+ def setup_security_group(self, context):
701+ group_name = '%s%s' % (context.project.id, FLAGS.vpn_key_suffix)
702+ if db.security_group_exists(context, context.project.id, group_name):
703+ return group_name
704+ group = {'user_id': context.user.id,
705+ 'project_id': context.project.id,
706+ 'name': group_name,
707+ 'description': 'Group for vpn'}
708+ group_ref = db.security_group_create(context, group)
709+ rule = {'parent_group_id': group_ref['id'],
710+ 'cidr': '0.0.0.0/0',
711+ 'protocol': 'udp',
712+ 'from_port': 1194,
713+ 'to_port': 1194}
714+ db.security_group_rule_create(context, rule)
715+ rule = {'parent_group_id': group_ref['id'],
716+ 'cidr': '0.0.0.0/0',
717+ 'protocol': 'icmp',
718+ 'from_port': -1,
719+ 'to_port': -1}
720+ db.security_group_rule_create(context, rule)
721+ # NOTE(vish): No need to trigger the group since the instance
722+ # has not been run yet.
723+ return group_name
724+
725+ def setup_key_pair(self, context):
726+ key_name = '%s%s' % (context.project.id, FLAGS.vpn_key_suffix)
727 try:
728- private_key, fingerprint = self.manager.generate_key_pair(user_id,
729- key_name)
730+ result = cloud._gen_key(context, context.user.id, key_name)
731+ private_key = result['private_key']
732 try:
733- key_dir = os.path.join(FLAGS.keys_path, user_id)
734+ key_dir = os.path.join(FLAGS.keys_path, context.user.id)
735 if not os.path.exists(key_dir):
736 os.makedirs(key_dir)
737- file_name = os.path.join(key_dir, '%s.pem' % key_name)
738- with open(file_name, 'w') as f:
739+ key_path = os.path.join(key_dir, '%s.pem' % key_name)
740+ with open(key_path, 'w') as f:
741 f.write(private_key)
742 except:
743 pass
744 except exception.Duplicate:
745 pass
746 return key_name
747-
748- # def setup_secgroups(self, username):
749- # conn = self.euca.connection_for(username)
750- # try:
751- # secgroup = conn.create_security_group("vpn-secgroup",
752- # "vpn-secgroup")
753- # secgroup.authorize(ip_protocol = "udp", from_port = "1194",
754- # to_port = "1194", cidr_ip = "0.0.0.0/0")
755- # secgroup.authorize(ip_protocol = "tcp", from_port = "80",
756- # to_port = "80", cidr_ip = "0.0.0.0/0")
757- # secgroup.authorize(ip_protocol = "tcp", from_port = "22",
758- # to_port = "22", cidr_ip = "0.0.0.0/0")
759- # except:
760- # pass
761
762=== modified file 'nova/crypto.py'
763--- nova/crypto.py 2010-12-11 20:23:40 +0000
764+++ nova/crypto.py 2010-12-22 17:33:04 +0000
765@@ -19,7 +19,6 @@
766 Wrappers around standard crypto data elements.
767
768 Includes root and intermediate CAs, SSH key_pairs and x509 certificates.
769-
770 """
771
772 import base64
773@@ -34,28 +33,57 @@
774
775 import M2Crypto
776
777-from nova import exception
778+from nova import context
779+from nova import db
780 from nova import flags
781
782
783 FLAGS = flags.FLAGS
784 flags.DEFINE_string('ca_file', 'cacert.pem', _('Filename of root CA'))
785+flags.DEFINE_string('key_file',
786+ os.path.join('private', 'cakey.pem'),
787+ _('Filename of private key'))
788+flags.DEFINE_string('crl_file', 'crl.pem',
789+ _('Filename of root Certificate Revokation List'))
790 flags.DEFINE_string('keys_path', '$state_path/keys',
791 _('Where we keep our keys'))
792 flags.DEFINE_string('ca_path', '$state_path/CA',
793 _('Where we keep our root CA'))
794-flags.DEFINE_boolean('use_intermediate_ca', False,
795- _('Should we use intermediate CAs for each project?'))
796-
797-
798-def ca_path(project_id):
799- if project_id:
800- return "%s/INTER/%s/cacert.pem" % (FLAGS.ca_path, project_id)
801- return "%s/cacert.pem" % (FLAGS.ca_path)
802+flags.DEFINE_boolean('use_project_ca', False,
803+ _('Should we use a CA for each project?'))
804+flags.DEFINE_string('user_cert_subject',
805+ '/C=US/ST=California/L=MountainView/O=AnsoLabs/'
806+ 'OU=NovaDev/CN=%s-%s-%s',
807+ _('Subject for certificate for users, '
808+ '%s for project, user, timestamp'))
809+flags.DEFINE_string('project_cert_subject',
810+ '/C=US/ST=California/L=MountainView/O=AnsoLabs/'
811+ 'OU=NovaDev/CN=project-ca-%s-%s',
812+ _('Subject for certificate for projects, '
813+ '%s for project, timestamp'))
814+flags.DEFINE_string('vpn_cert_subject',
815+ '/C=US/ST=California/L=MountainView/O=AnsoLabs/'
816+ 'OU=NovaDev/CN=project-vpn-%s-%s',
817+ _('Subject for certificate for vpns, '
818+ '%s for project, timestamp'))
819+
820+
821+def ca_folder(project_id=None):
822+ if FLAGS.use_project_ca and project_id:
823+ return os.path.join(FLAGS.ca_path, 'projects', project_id)
824+ return FLAGS.ca_path
825+
826+
827+def ca_path(project_id=None):
828+ return os.path.join(ca_folder(project_id), FLAGS.ca_file)
829+
830+
831+def key_path(project_id=None):
832+ return os.path.join(ca_folder(project_id), FLAGS.key_file)
833
834
835 def fetch_ca(project_id=None, chain=True):
836- if not FLAGS.use_intermediate_ca:
837+ if not FLAGS.use_project_ca:
838 project_id = None
839 buffer = ""
840 if project_id:
841@@ -92,8 +120,8 @@
842
843
844 def ssl_pub_to_ssh_pub(ssl_public_key, name='root', suffix='nova'):
845- pub_key_buffer = M2Crypto.BIO.MemoryBuffer(ssl_public_key)
846- rsa_key = M2Crypto.RSA.load_pub_key_bio(pub_key_buffer)
847+ buf = M2Crypto.BIO.MemoryBuffer(ssl_public_key)
848+ rsa_key = M2Crypto.RSA.load_pub_key_bio(buf)
849 e, n = rsa_key.pub()
850
851 key_type = 'ssh-rsa'
852@@ -106,53 +134,134 @@
853 return '%s %s %s@%s\n' % (key_type, b64_blob, name, suffix)
854
855
856-def generate_x509_cert(subject, bits=1024):
857+def revoke_cert(project_id, file_name):
858+ """Revoke a cert by file name"""
859+ start = os.getcwd()
860+ os.chdir(ca_folder(project_id))
861+ # NOTE(vish): potential race condition here
862+ utils.execute("openssl ca -config ./openssl.cnf -revoke '%s'" % file_name)
863+ utils.execute("openssl ca -gencrl -config ./openssl.cnf -out '%s'" %
864+ FLAGS.crl_file)
865+ os.chdir(start)
866+
867+
868+def revoke_certs_by_user(user_id):
869+ """Revoke all user certs"""
870+ admin = context.get_admin_context()
871+ for cert in db.certificate_get_all_by_user(admin, user_id):
872+ revoke_cert(cert['project_id'], cert['file_name'])
873+
874+
875+def revoke_certs_by_project(project_id):
876+ """Revoke all project certs"""
877+ # NOTE(vish): This is somewhat useless because we can just shut down
878+ # the vpn.
879+ admin = context.get_admin_context()
880+ for cert in db.certificate_get_all_by_project(admin, project_id):
881+ revoke_cert(cert['project_id'], cert['file_name'])
882+
883+
884+def revoke_certs_by_user_and_project(user_id, project_id):
885+ """Revoke certs for user in project"""
886+ admin = context.get_admin_context()
887+ for cert in db.certificate_get_all_by_user(admin, user_id, project_id):
888+ revoke_cert(cert['project_id'], cert['file_name'])
889+
890+
891+def _project_cert_subject(project_id):
892+ """Helper to generate user cert subject"""
893+ return FLAGS.project_cert_subject % (project_id, utils.isotime())
894+
895+
896+def _vpn_cert_subject(project_id):
897+ """Helper to generate user cert subject"""
898+ return FLAGS.vpn_cert_subject % (project_id, utils.isotime())
899+
900+
901+def _user_cert_subject(user_id, project_id):
902+ """Helper to generate user cert subject"""
903+ return FLAGS.user_cert_subject % (project_id, user_id, utils.isotime())
904+
905+
906+def generate_x509_cert(user_id, project_id, bits=1024):
907+ """Generate and sign a cert for user in project"""
908+ subject = _user_cert_subject(user_id, project_id)
909 tmpdir = tempfile.mkdtemp()
910 keyfile = os.path.abspath(os.path.join(tmpdir, 'temp.key'))
911 csrfile = os.path.join(tmpdir, 'temp.csr')
912- logging.debug("openssl genrsa -out %s %s" % (keyfile, bits))
913- utils.runthis(_("Generating private key: %s"),
914- "openssl genrsa -out %s %s" % (keyfile, bits))
915- utils.runthis(_("Generating CSR: %s"),
916- "openssl req -new -key %s -out %s -batch -subj %s" %
917+ utils.execute("openssl genrsa -out %s %s" % (keyfile, bits))
918+ utils.execute("openssl req -new -key %s -out %s -batch -subj %s" %
919 (keyfile, csrfile, subject))
920 private_key = open(keyfile).read()
921 csr = open(csrfile).read()
922 shutil.rmtree(tmpdir)
923- return (private_key, csr)
924-
925-
926-def sign_csr(csr_text, intermediate=None):
927- if not FLAGS.use_intermediate_ca:
928- intermediate = None
929- if not intermediate:
930- return _sign_csr(csr_text, FLAGS.ca_path)
931- user_ca = "%s/INTER/%s" % (FLAGS.ca_path, intermediate)
932- if not os.path.exists(user_ca):
933+ (serial, signed_csr) = sign_csr(csr, project_id)
934+ fname = os.path.join(ca_folder(project_id), "newcerts/%s.pem" % serial)
935+ cert = {'user_id': user_id,
936+ 'project_id': project_id,
937+ 'file_name': fname}
938+ db.certificate_create(context.get_admin_context(), cert)
939+ return (private_key, signed_csr)
940+
941+
942+def _ensure_project_folder(project_id):
943+ if not os.path.exists(ca_path(project_id)):
944 start = os.getcwd()
945- os.chdir(FLAGS.ca_path)
946- utils.runthis(_("Generating intermediate CA: %s"),
947- "sh geninter.sh %s" % (intermediate))
948+ os.chdir(ca_folder())
949+ utils.execute("sh geninter.sh %s %s" %
950+ (project_id, _project_cert_subject(project_id)))
951 os.chdir(start)
952- return _sign_csr(csr_text, user_ca)
953+
954+
955+def generate_vpn_files(project_id):
956+ project_folder = ca_folder(project_id)
957+ csr_fn = os.path.join(project_folder, "server.csr")
958+ crt_fn = os.path.join(project_folder, "server.crt")
959+
960+ if os.path.exists(crt_fn):
961+ return
962+ _ensure_project_folder(project_id)
963+ start = os.getcwd()
964+ os.chdir(ca_folder())
965+ # TODO(vish): the shell scripts could all be done in python
966+ utils.execute("sh genvpn.sh %s %s" %
967+ (project_id, _vpn_cert_subject(project_id)))
968+ with open(csr_fn, "r") as csrfile:
969+ csr_text = csrfile.read()
970+ (serial, signed_csr) = sign_csr(csr_text, project_id)
971+ with open(crt_fn, "w") as crtfile:
972+ crtfile.write(signed_csr)
973+ os.chdir(start)
974+
975+
976+def sign_csr(csr_text, project_id=None):
977+ if not FLAGS.use_project_ca:
978+ project_id = None
979+ if not project_id:
980+ return _sign_csr(csr_text, ca_folder())
981+ _ensure_project_folder(project_id)
982+ project_folder = ca_folder(project_id)
983+ return _sign_csr(csr_text, ca_folder(project_id))
984
985
986 def _sign_csr(csr_text, ca_folder):
987 tmpfolder = tempfile.mkdtemp()
988- csrfile = open("%s/inbound.csr" % (tmpfolder), "w")
989+ inbound = os.path.join(tmpfolder, "inbound.csr")
990+ outbound = os.path.join(tmpfolder, "outbound.csr")
991+ csrfile = open(inbound, "w")
992 csrfile.write(csr_text)
993 csrfile.close()
994 logging.debug(_("Flags path: %s") % ca_folder)
995 start = os.getcwd()
996 # Change working dir to CA
997 os.chdir(ca_folder)
998- utils.runthis(_("Signing cert: %s"),
999- "openssl ca -batch -out %s/outbound.crt "
1000- "-config ./openssl.cnf -infiles %s/inbound.csr" %
1001- (tmpfolder, tmpfolder))
1002+ utils.execute("openssl ca -batch -out %s -config "
1003+ "./openssl.cnf -infiles %s" % (outbound, inbound))
1004+ out, _err = utils.execute("openssl x509 -in %s -serial -noout" % outbound)
1005+ serial = out.rpartition("=")[2]
1006 os.chdir(start)
1007- with open("%s/outbound.crt" % (tmpfolder), "r") as crtfile:
1008- return crtfile.read()
1009+ with open(outbound, "r") as crtfile:
1010+ return (serial, crtfile.read())
1011
1012
1013 def mkreq(bits, subject="foo", ca=0):
1014@@ -160,8 +269,7 @@
1015 req = M2Crypto.X509.Request()
1016 rsa = M2Crypto.RSA.gen_key(bits, 65537, callback=lambda: None)
1017 pk.assign_rsa(rsa)
1018- # Should not be freed here
1019- rsa = None
1020+ rsa = None # should not be freed here
1021 req.set_pubkey(pk)
1022 req.set_subject(subject)
1023 req.sign(pk, 'sha512')
1024@@ -225,7 +333,6 @@
1025 # IN THE SOFTWARE.
1026 # http://code.google.com/p/boto
1027
1028-
1029 def compute_md5(fp):
1030 """
1031 :type fp: file
1032
1033=== modified file 'nova/db/api.py'
1034--- nova/db/api.py 2010-12-21 20:13:18 +0000
1035+++ nova/db/api.py 2010-12-22 17:33:04 +0000
1036@@ -130,6 +130,45 @@
1037 ###################
1038
1039
1040+def certificate_create(context, values):
1041+ """Create a certificate from the values dictionary."""
1042+ return IMPL.certificate_create(context, values)
1043+
1044+
1045+def certificate_destroy(context, certificate_id):
1046+ """Destroy the certificate or raise if it does not exist."""
1047+ return IMPL.certificate_destroy(context, certificate_id)
1048+
1049+
1050+def certificate_get_all_by_project(context, project_id):
1051+ """Get all certificates for a project."""
1052+ return IMPL.certificate_get_all_by_project(context, project_id)
1053+
1054+
1055+def certificate_get_all_by_user(context, user_id):
1056+ """Get all certificates for a user."""
1057+ return IMPL.certificate_get_all_by_user(context, user_id)
1058+
1059+
1060+def certificate_get_all_by_user_and_project(context, user_id, project_id):
1061+ """Get all certificates for a user and project."""
1062+ return IMPL.certificate_get_all_by_user_and_project(context,
1063+ user_id,
1064+ project_id)
1065+
1066+
1067+def certificate_update(context, certificate_id, values):
1068+ """Set the given properties on an certificate and update it.
1069+
1070+ Raises NotFound if service does not exist.
1071+
1072+ """
1073+ return IMPL.service_update(context, certificate_id, values)
1074+
1075+
1076+###################
1077+
1078+
1079 def floating_ip_allocate_address(context, host, project_id):
1080 """Allocate free floating ip and return the address.
1081
1082@@ -304,6 +343,11 @@
1083 return IMPL.instance_get_floating_address(context, instance_id)
1084
1085
1086+def instance_get_project_vpn(context, project_id):
1087+ """Get a vpn instance by project or return None."""
1088+ return IMPL.instance_get_project_vpn(context, project_id)
1089+
1090+
1091 def instance_get_by_internal_id(context, internal_id):
1092 """Get an instance by internal id."""
1093 return IMPL.instance_get_by_internal_id(context, internal_id)
1094@@ -473,12 +517,14 @@
1095 ###################
1096
1097
1098-def project_get_network(context, project_id):
1099+def project_get_network(context, project_id, associate=True):
1100 """Return the network associated with the project.
1101
1102- Raises NotFound if no such network can be found.
1103+ If associate is true, it will attempt to associate a new
1104+ network if one is not found, otherwise it returns None.
1105
1106 """
1107+
1108 return IMPL.project_get_network(context, project_id)
1109
1110
1111
1112=== modified file 'nova/db/sqlalchemy/api.py'
1113--- nova/db/sqlalchemy/api.py 2010-12-22 16:43:40 +0000
1114+++ nova/db/sqlalchemy/api.py 2010-12-22 17:33:04 +0000
1115@@ -252,6 +252,84 @@
1116 ###################
1117
1118
1119+@require_admin_context
1120+def certificate_get(context, certificate_id, session=None):
1121+ if not session:
1122+ session = get_session()
1123+
1124+ result = session.query(models.Certificate).\
1125+ filter_by(id=certificate_id).\
1126+ filter_by(deleted=can_read_deleted(context)).\
1127+ first()
1128+
1129+ if not result:
1130+ raise exception.NotFound('No certificate for id %s' % certificate_id)
1131+
1132+ return result
1133+
1134+
1135+@require_admin_context
1136+def certificate_create(context, values):
1137+ certificate_ref = models.Certificate()
1138+ for (key, value) in values.iteritems():
1139+ certificate_ref[key] = value
1140+ certificate_ref.save()
1141+ return certificate_ref
1142+
1143+
1144+@require_admin_context
1145+def certificate_destroy(context, certificate_id):
1146+ session = get_session()
1147+ with session.begin():
1148+ certificate_ref = certificate_get(context,
1149+ certificate_id,
1150+ session=session)
1151+ certificate_ref.delete(session=session)
1152+
1153+
1154+@require_admin_context
1155+def certificate_get_all_by_project(context, project_id):
1156+ session = get_session()
1157+ return session.query(models.Certificate).\
1158+ filter_by(project_id=project_id).\
1159+ filter_by(deleted=False).\
1160+ all()
1161+
1162+
1163+@require_admin_context
1164+def certificate_get_all_by_user(context, user_id):
1165+ session = get_session()
1166+ return session.query(models.Certificate).\
1167+ filter_by(user_id=user_id).\
1168+ filter_by(deleted=False).\
1169+ all()
1170+
1171+
1172+@require_admin_context
1173+def certificate_get_all_by_user_and_project(_context, user_id, project_id):
1174+ session = get_session()
1175+ return session.query(models.Certificate).\
1176+ filter_by(user_id=user_id).\
1177+ filter_by(project_id=project_id).\
1178+ filter_by(deleted=False).\
1179+ all()
1180+
1181+
1182+@require_admin_context
1183+def certificate_update(context, certificate_id, values):
1184+ session = get_session()
1185+ with session.begin():
1186+ certificate_ref = certificate_get(context,
1187+ certificate_id,
1188+ session=session)
1189+ for (key, value) in values.iteritems():
1190+ certificate_ref[key] = value
1191+ certificate_ref.save(session=session)
1192+
1193+
1194+###################
1195+
1196+
1197 @require_context
1198 def floating_ip_allocate_address(context, host, project_id):
1199 authorize_project_context(context, project_id)
1200@@ -653,6 +731,18 @@
1201 all()
1202
1203
1204+@require_admin_context
1205+def instance_get_project_vpn(context, project_id):
1206+ session = get_session()
1207+ return session.query(models.Instance).\
1208+ options(joinedload_all('fixed_ip.floating_ips')).\
1209+ options(joinedload('security_groups')).\
1210+ filter_by(project_id=project_id).\
1211+ filter_by(image_id=FLAGS.vpn_image_id).\
1212+ filter_by(deleted=can_read_deleted(context)).\
1213+ first()
1214+
1215+
1216 @require_context
1217 def instance_get_by_internal_id(context, internal_id):
1218 session = get_session()
1219@@ -1001,24 +1091,26 @@
1220
1221
1222 @require_context
1223-def project_get_network(context, project_id):
1224+def project_get_network(context, project_id, associate=True):
1225 session = get_session()
1226- rv = session.query(models.Network).\
1227- filter_by(project_id=project_id).\
1228- filter_by(deleted=False).\
1229- first()
1230- if not rv:
1231+ result = session.query(models.Network).\
1232+ filter_by(project_id=project_id).\
1233+ filter_by(deleted=False).\
1234+ first()
1235+ if not result:
1236+ if not associate:
1237+ return None
1238 try:
1239 return network_associate(context, project_id)
1240 except IntegrityError:
1241 # NOTE(vish): We hit this if there is a race and two
1242 # processes are attempting to allocate the
1243 # network at the same time
1244- rv = session.query(models.Network).\
1245- filter_by(project_id=project_id).\
1246- filter_by(deleted=False).\
1247- first()
1248- return rv
1249+ result = session.query(models.Network).\
1250+ filter_by(project_id=project_id).\
1251+ filter_by(deleted=False).\
1252+ first()
1253+ return result
1254
1255
1256 ###################
1257
1258=== modified file 'nova/db/sqlalchemy/models.py'
1259--- nova/db/sqlalchemy/models.py 2010-12-21 20:13:18 +0000
1260+++ nova/db/sqlalchemy/models.py 2010-12-22 17:33:04 +0000
1261@@ -151,6 +151,16 @@
1262 disabled = Column(Boolean, default=False)
1263
1264
1265+class Certificate(BASE, NovaBase):
1266+ """Represents a an x509 certificate"""
1267+ __tablename__ = 'certificates'
1268+ id = Column(Integer, primary_key=True)
1269+
1270+ user_id = Column(String(255))
1271+ project_id = Column(String(255))
1272+ file_name = Column(String(255))
1273+
1274+
1275 class Instance(BASE, NovaBase):
1276 """Represents a guest vm."""
1277 __tablename__ = 'instances'
1278@@ -555,7 +565,7 @@
1279 Volume, ExportDevice, IscsiTarget, FixedIp, FloatingIp,
1280 Network, SecurityGroup, SecurityGroupIngressRule,
1281 SecurityGroupInstanceAssociation, AuthToken, User,
1282- Project) # , Image, Host
1283+ Project, Certificate) # , Image, Host
1284 engine = create_engine(FLAGS.sql_connection, echo=False)
1285 for model in models:
1286 model.metadata.create_all(engine)
1287
1288=== modified file 'nova/flags.py'
1289--- nova/flags.py 2010-12-17 12:07:43 +0000
1290+++ nova/flags.py 2010-12-22 17:33:04 +0000
1291@@ -29,6 +29,8 @@
1292
1293 import gflags
1294
1295+from nova import utils
1296+
1297
1298 class FlagValues(gflags.FlagValues):
1299 """Extension of gflags.FlagValues that allows undefined and runtime flags.
1300@@ -211,7 +213,8 @@
1301 DEFINE_string('aws_access_key_id', 'admin', 'AWS Access ID')
1302 DEFINE_string('aws_secret_access_key', 'admin', 'AWS Access Key')
1303 DEFINE_integer('s3_port', 3333, 's3 port')
1304-DEFINE_string('s3_host', '127.0.0.1', 's3 host')
1305+DEFINE_string('s3_host', utils.get_my_ip(), 's3 host (for infrastructure)')
1306+DEFINE_string('s3_dmz', utils.get_my_ip(), 's3 dmz ip (for instances)')
1307 DEFINE_string('compute_topic', 'compute', 'the topic compute nodes listen on')
1308 DEFINE_string('scheduler_topic', 'scheduler',
1309 'the topic scheduler nodes listen on')
1310@@ -230,8 +233,11 @@
1311 DEFINE_integer('rabbit_retry_interval', 10, 'rabbit connection retry interval')
1312 DEFINE_integer('rabbit_max_retries', 12, 'rabbit connection attempts')
1313 DEFINE_string('control_exchange', 'nova', 'the main exchange to connect to')
1314-DEFINE_string('ec2_url', 'http://127.0.0.1:8773/services/Cloud',
1315- 'Url to ec2 api server')
1316+DEFINE_string('ec2_prefix', 'http', 'prefix for ec2')
1317+DEFINE_string('cc_host', utils.get_my_ip(), 'ip of api server')
1318+DEFINE_string('cc_dmz', utils.get_my_ip(), 'internal ip of api server')
1319+DEFINE_integer('cc_port', 8773, 'cloud controller port')
1320+DEFINE_string('ec2_suffix', '/services/Cloud', 'suffix for ec2')
1321
1322 DEFINE_string('default_image', 'ami-11111',
1323 'default image to use, testing only')
1324@@ -241,10 +247,10 @@
1325 'kernel image that indicates not to use a kernel,'
1326 ' but to use a raw disk image instead')
1327
1328-DEFINE_string('vpn_image_id', 'ami-CLOUDPIPE', 'AMI for cloudpipe vpn server')
1329+DEFINE_string('vpn_image_id', 'ami-cloudpipe', 'AMI for cloudpipe vpn server')
1330 DEFINE_string('vpn_key_suffix',
1331- '-key',
1332- 'Suffix to add to project name for vpn key')
1333+ '-vpn',
1334+ 'Suffix to add to project name for vpn key and secgroups')
1335
1336 DEFINE_integer('auth_token_ttl', 3600, 'Seconds for auth tokens to linger')
1337
1338
1339=== modified file 'nova/network/linux_net.py'
1340--- nova/network/linux_net.py 2010-12-11 20:23:40 +0000
1341+++ nova/network/linux_net.py 2010-12-22 17:33:04 +0000
1342@@ -46,9 +46,7 @@
1343 'network device for vlans')
1344 flags.DEFINE_string('dhcpbridge', _bin_file('nova-dhcpbridge'),
1345 'location of nova-dhcpbridge')
1346-flags.DEFINE_string('cc_host', utils.get_my_ip(), 'ip of api server')
1347-flags.DEFINE_integer('cc_port', 8773, 'cloud controller port')
1348-flags.DEFINE_string('routing_source_ip', '127.0.0.1',
1349+flags.DEFINE_string('routing_source_ip', utils.get_my_ip(),
1350 'Public IP of network host')
1351 flags.DEFINE_bool('use_nova_chains', False,
1352 'use the nova_ routing chains instead of default')
1353@@ -60,7 +58,7 @@
1354 """Create forwarding rule for metadata"""
1355 _confirm_rule("PREROUTING", "-t nat -s 0.0.0.0/0 "
1356 "-d 169.254.169.254/32 -p tcp -m tcp --dport 80 -j DNAT "
1357- "--to-destination %s:%s" % (FLAGS.cc_host, FLAGS.cc_port))
1358+ "--to-destination %s:%s" % (FLAGS.cc_dmz, FLAGS.cc_port))
1359
1360
1361 def init_host():
1362
1363=== modified file 'nova/tests/auth_unittest.py'
1364--- nova/tests/auth_unittest.py 2010-12-17 22:29:55 +0000
1365+++ nova/tests/auth_unittest.py 2010-12-22 17:33:04 +0000
1366@@ -208,17 +208,13 @@
1367 # so it probably belongs in crypto_unittest
1368 # but I'm leaving it where I found it.
1369 with user_and_project_generator(self.manager) as (user, project):
1370- # NOTE(todd): Should mention why we must setup controller first
1371- # (somebody please clue me in)
1372- cloud_controller = cloud.CloudController()
1373- cloud_controller.setup()
1374- _key, cert_str = self.manager._generate_x509_cert('test1',
1375- 'testproj')
1376+ # NOTE(vish): Setup runs genroot.sh if it hasn't been run
1377+ cloud.CloudController().setup()
1378+ _key, cert_str = crypto.generate_x509_cert(user.id, project.id)
1379 logging.debug(cert_str)
1380
1381- # Need to verify that it's signed by the right intermediate CA
1382- full_chain = crypto.fetch_ca(project_id='testproj', chain=True)
1383- int_cert = crypto.fetch_ca(project_id='testproj', chain=False)
1384+ full_chain = crypto.fetch_ca(project_id=project.id, chain=True)
1385+ int_cert = crypto.fetch_ca(project_id=project.id, chain=False)
1386 cloud_cert = crypto.fetch_ca()
1387 logging.debug("CA chain:\n\n =====\n%s\n\n=====" % full_chain)
1388 signed_cert = X509.load_cert_string(cert_str)
1389@@ -227,7 +223,8 @@
1390 cloud_cert = X509.load_cert_string(cloud_cert)
1391 self.assertTrue(signed_cert.verify(chain_cert.get_pubkey()))
1392 self.assertTrue(signed_cert.verify(int_cert.get_pubkey()))
1393- if not FLAGS.use_intermediate_ca:
1394+
1395+ if not FLAGS.use_project_ca:
1396 self.assertTrue(signed_cert.verify(cloud_cert.get_pubkey()))
1397 else:
1398 self.assertFalse(signed_cert.verify(cloud_cert.get_pubkey()))
1399
1400=== modified file 'nova/utils.py'
1401--- nova/utils.py 2010-12-17 16:10:11 +0000
1402+++ nova/utils.py 2010-12-22 17:33:04 +0000
1403@@ -21,13 +21,13 @@
1404 """
1405
1406 import datetime
1407-import functools
1408 import inspect
1409 import logging
1410 import os
1411 import random
1412 import subprocess
1413 import socket
1414+import struct
1415 import sys
1416 from xml.sax import saxutils
1417
1418@@ -35,11 +35,9 @@
1419 from eventlet import greenthread
1420
1421 from nova import exception
1422-from nova import flags
1423 from nova.exception import ProcessExecutionError
1424
1425
1426-FLAGS = flags.FLAGS
1427 TIME_FORMAT = "%Y-%m-%dT%H:%M:%SZ"
1428
1429
1430@@ -63,6 +61,51 @@
1431 return cls()
1432
1433
1434+def vpn_ping(address, port, timeout=0.05, session_id=None):
1435+ """Sends a vpn negotiation packet and returns the server session.
1436+
1437+ Returns False on a failure. Basic packet structure is below.
1438+
1439+ Client packet (14 bytes)::
1440+ 0 1 8 9 13
1441+ +-+--------+-----+
1442+ |x| cli_id |?????|
1443+ +-+--------+-----+
1444+ x = packet identifier 0x38
1445+ cli_id = 64 bit identifier
1446+ ? = unknown, probably flags/padding
1447+
1448+ Server packet (26 bytes)::
1449+ 0 1 8 9 13 14 21 2225
1450+ +-+--------+-----+--------+----+
1451+ |x| srv_id |?????| cli_id |????|
1452+ +-+--------+-----+--------+----+
1453+ x = packet identifier 0x40
1454+ cli_id = 64 bit identifier
1455+ ? = unknown, probably flags/padding
1456+ bit 9 was 1 and the rest were 0 in testing
1457+ """
1458+ if session_id is None:
1459+ session_id = random.randint(0, 0xffffffffffffffff)
1460+ sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
1461+ data = struct.pack("!BQxxxxxx", 0x38, session_id)
1462+ sock.sendto(data, (address, port))
1463+ sock.settimeout(timeout)
1464+ try:
1465+ received = sock.recv(2048)
1466+ except socket.timeout:
1467+ return False
1468+ finally:
1469+ sock.close()
1470+ fmt = "!BQxxxxxQxxxx"
1471+ if len(received) != struct.calcsize(fmt):
1472+ print struct.calcsize(fmt)
1473+ return False
1474+ (identifier, server_sess, client_sess) = struct.unpack(fmt, received)
1475+ if identifier == 0x40 and client_sess == session_id:
1476+ return server_sess
1477+
1478+
1479 def fetchfile(url, target):
1480 logging.debug(_("Fetching %s") % url)
1481 # c = pycurl.Curl()
1482@@ -151,8 +194,6 @@
1483
1484 def get_my_ip():
1485 """Returns the actual ip of the local machine."""
1486- if getattr(FLAGS, 'fake_tests', None):
1487- return '127.0.0.1'
1488 try:
1489 csock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
1490 csock.connect(('8.8.8.8', 80))