Unable to run compiz: Xlib: extension "GLX" missing on display ":0.0".

Ever since getting ubuntu I have been having problems with graphics drivers and my nvidia card.
The nouveau driver was slow, and then I tried installing the nvidia one from propietry drivers, but thta was very glitchy. Eventually, I uninstalled the nvidia driver from propietry drivers to try to go back to nouveau.

However when I rebooted I got no graphical output, I dropped to a the root command prompt in recovery mode, and installed the nvidia driver downloadable from nvidia's website, (I had downloaded it a few days ago, as for a bit I had thought about using it to see if there was any difference).
I rebooted, and lo and behold, I had graphical output again, and it was no longer glitchy.

However recently I installed a load of software updates, these included: the latest linux image (3.0.0-17-generic-pae) the latest nvidia driver (not sure if this was the one from nvidia's website or the propietry drivers one, I could have forgotten to remove a ppa, and that may explain the break that happened), and the latest version of X.

Then, when I restarted, I once again had no graphical output. I once again dropped to the root command line in recovery mode but this time I uninstalled the nvidia drivers off my system (but only the propietry drivers, not the ones from nvidia's website, for some reason when I deactivated them in propietry drivers they didn't uninstall, so I had both the drivers from nvidia's website and the propietry ones installed at the same time) (but my xorg.conf specified that I use the ones from nvidia's website).

Anway, after that I completely reinstalled X, using apt-get uninstall xserver-xorg (this removed a bunch of other packages that depend on it as well) and then apt-get install ubuntu-desktop (this should reinstall all those packages), I rebooted and everything worked fine apart from these things:

I could not run compiz (or any opengl applications) and /usr/lib/nux/unity_support_test -p gave this: Xlib: extension "GLX" missing on display ":0.0".

I still had the nvidia driver from nvidia's website installed, although X was not using it
I had no xorg.conf
I had no libglx.so in my /usr/lib/xorg/modules/extensions/
The X log in /var/log/Xorg.0.log looked like this (I've only pasted the bits that I think are important:
[ 16.168] (II) LoadModule: "glx"
[ 16.168] (WW) Warning, couldn't open module glx
[ 16.168] (II) UnloadModule: "glx"
[ 16.168] (II) Unloadingglx
[ 16.168] (EE) Failed to load module "glx" (module does not exist, 0)
..................................................................................................................................................................................
(there was some stuff between these two sections but this looked pretty normal and not relavant)
..................................................................................................................................................................................
[ 16.191] (==) Matched nvidia as autoconfigured driver 0
[ 16.191] (==) Matched nouveau as autoconfigured driver 1
[ 16.191] (==) Matched nv as autoconfigured driver 2
[ 16.191] (==) Matched vesa as autoconfigured driver 3
[ 16.191] (==) Matched fbdev as autoconfigured driver 4
[ 16.191] (==) Assigned the driver to the xf86ConfigLayout
[ 16.191] (II) LoadModule: "nvidia"
[ 16.191] (WW) Warning, couldn't open module nvidia
[ 16.191] (II) UnloadModule: "nvidia"
[ 16.191] (II) Unloading nvidia
[ 16.191] (EE) Failed to load module "nvidia" (module does not exist, 0)
[ 16.191] (II) LoadModule: "nouveau"
[ 16.191] (II) Loading /usr/lib/xorg/modules/drivers/nouveau_drv.so
[ 16.204] (II) Module nouveau: vendor="X.Org Foundation"
[ 16.204] compiled for 1.10.1, module version = 0.0.16
[ 16.204] Module class: X.Org Video Driver
[ 16.204] ABI class: X.Org Video Driver, version 10.0