Missing dependencies in Hardy 8.0.4.2

Hi -

I could use some help determining if this is really a bug. I'm pretty sure it is, but I don't want to submit an actual bug report without more confirmation than I was able to obtain via the forums, so ...

A moderately concise summary of well over 20 hours investigation:

Installing a supported, automatically-suggested update bundle to Hardy on my Gateway Core 2 Duo "M-series" laptop appears to have resulted in missing libraries/shared-objects (.so files) for the Mozilla XPCOM component framework that's included in Ubuntu. I base that statement on the results of using run-mozilla.sh to check dependencies: three .so files were reported as "not found" - i.e. not "registered" - despite their actual presence under the /usr/lib/xulrunner-1.9.0.7/ directory. ( Although probably present in the wrong subdir, I suspect. ) Further, I suspect this dependency issue may be why I can no longer watch the streaming video site www.fancast.com via Firefox, an activity I happened to be engaged in without any error or problem just a minute or two before I applied the updates. Extensive experimentation and investigation leads me to believe that this might represent an error (bug) in the recent officially-supplied update bundle for hardy, and perhaps in the xulrunner-1.9 update that was supplied as part of that, as I believe. As a non-programmer, I'd appreciate help confirming or disproving that, with a view toward initiation of a possible bug report in launchpad. ( I did post to the forums to get feedback from others about this; it also occurs in Intrepid and in Jaunty. See, e.g. http://ubuntuforums.org/showthread.php?t=1108123 .) I'd also very much appreciate any ideas as to what I might do as an interim workaround, if possible and not a burden, until the problem's status as either a package/build bug or a one-off configuration problem specific to my own pc can be conclusively determined. Finally, this appears (to me) to be somewhat related to https://bugs.launchpad.net/ubuntu/+source/firefox-3.0/+bug/217812 .

A *truly* verbose documentation of the same:

( This is so verbose because I was writing initially only for my own learning benefit, trying to understand something that's rather considerably above my level of formal training. Sorry to be so tediously long, but the following does include all the relevant info I could think of. )

( the following as per Rocket2DMn's "sticky" on how to post a bug report to launchpad ... not ready to do that, but I suppose this info will nevertheless be relevant: )

[code]
(1) output from: lsb_release -rd follows

Description: Ubuntu 8.04.2
Release: 8.04

(2) output from uname -a follows

Linux mypc 2.6.24-24-generic #1 SMP Thu Feb 19 08:00:07 UTC 2009 i686 GNU/Linux

(3) output from apt-cache policy xulrunner-1.9 follows

xulrunner-1.9:
  Installed: 1.9.0.7+nobinonly-0ubuntu0.8.04.1
  Candidate: 1.9.0.7+nobinonly-0ubuntu0.8.04.1
  Version table:
 *** 1.9.0.7+nobinonly-0ubuntu0.8.04.1 0
        500 http://mirrors.us.kernel.org hardy-updates/main Packages
        500 http://mirrors.us.kernel.org hardy-security/main Packages
        100 /var/lib/dpkg/status
     1.9~b5+nobinonly-0ubuntu3 0
        500 http://mirrors.us.kernel.org hardy/main Packages
[/code]

I found I couldn't play streaming videos at fancast.com after I acceded to the automatic download and installation of the latest Ubuntu-supplied updates on 20 March 2009, videos that I *had* been able to watch *immediately* before applying those updates. When I tried to watch streaming video at the site right after the update and reboot, the screen would never display any content, it just showed "loading... loading ...", ad nauseum, while the message at the bottom of the firefox window said "Done." This has been the case every time I've gone back to the fancast.com site.

The update bundle brought Firefox up to version 3.0.7, and - as I understand it - also may have updated or installed Mozilla XUL and XULRunner files that Firefox depends upon to function properly, since I hadn't updated previously since around early January, 2009, as I recall.

I have a Gateway Core 2 Duo, M-Series laptop, with 3 Gigs RAM, a 160G hard drive, and the only "non-standard" software that was present on my pc before the update bundle was downloaded and applied were Adblock Plus 0.7.5.4, Adobe Flash 9.x (don't recall exact version, but I know it was 9.something), the Ubuntu Firefox Modifications, Sun Java 6 and (?) related plugins, Firefox itself, and Wine 1.0. No virtualization software, and no Adobe Reader plugin (for .pdf files) installed.

I hadn't noticed any unusual system behavior or problems, at all, in the weeks and months prior to installing this most recent update bundle.

I suspected a Flash problem at first, but deleting and re-installing Flash (mutiple times, multiple ways) didn't help, nor did upgrading Flash from version 9-point-something to the newest 10.x production release.

( After-the-fact note: I have since carefully uninstalled both Firefox and Flash, and then reinstalled Firefox, with no add-ons and only the Default plugin, the Unix print example plugin, and Sun Java 6 installed. The results reported here don't change; the "dependencies" re the XPCOM component system and XULrunner, as reported by run-mozilla.sh , still occur without change or improvement. )

The Firefox error console ("Tools" ---> "Error Console" in Firefox) presented a great many lines concerning a "cascading style sheet" (.css file) from the fancast site that Firefox had evidently found hard to interpret and render correctly. Those lines were all classified as "warnings", and it's my understanding that such are never "show stoppers", that they're pretty generally inoccuous, and can be ignored or even turned off re user notification.

But the two lines that appeared at the very top in the Firefox error console were classified as "messages". That's a very rarely-used severity descriptor, according to the one web page I found that addressed the matter directly, although I couldn't find anything definitive about its rank order with respect to the other two descriptors, "Warning" and "Error" per se. But they did seem curious:

[code]
Failed to load XPCOM component: /usr/lib/xulrunner-1.9.0.7/components/pyabout.py
Failed to load XPCOM component: /usr/lib/xulrunner-1.9.0.7/components/libpyloader.so

Warning: Error in parsing value for property 'cursor'. Declaration dropped.
Source File: http://www.fancast.com/static-19052/css/base.css
Line: 28

Warning: Error in property 'resize'... (snip)
[/code]

So I decided to investigate, to learn what "Failed to load XPCOM component" re these two files

  /usr/lib/xulrunner-1.9.0.7/components/pyabout.py
  /usr/lib/xulrunner-1.9.0.7/components/libpyloader.so

might mean. I didn't know it, but that was pretty much a "down the rabbit hole" decision, like that scene in The Matrix when Neo takes the blue pill, or was it the red one? Anyway, the pill that sets him out on a whole new career, one that's difficult, lonely, overwhelming, one in which he fears the computers might win... That's a spot-on comparison, really. Wait, I wonder ... oh, whew!! It's alright! I *do* have a belly-button!

( For any non-ubergeeks - like myself - who may be trying to follow this thread, some definitions might be of use: Files with the ".so" extension are "shared objects", also called "shared libraries", and ".py" files are Python scripts. "XPCOM" means "Cross Platform Component Object Model", and refers to a framework for developing cross platform software that can be written in C, C++, or JavaScript with extensions for Perl and Python. Also, I learned from Wikipedia that XUL is an XML-based user-interface markup language developed by the Mozilla project, and which is used by cross-platform applications such as Firefox and Flock. I learned that XULRunner is a corresponding runtime environment, also developed by the Mozilla, to provide a common back-end for XUL applications. Finally, a file that's integral to the problem I'm documenting here is named "libpyxpcom.so", which I presume was named to reflect something like "a library Python uses to exchange info with (cross-platform) XPCOM components". As will be seen in the output from run-mozilla.sh, several lines below, that's one of three files/libraries/shared-objects that dont get registered properly, for some reason, and thus can't be found by libpyloader.so. Oh, this is helpful, also from Wikipedia: "PyXPCOM allows for communication between Python and XPCOM, such that a Python application can access XPCOM objects, and XPCOM can access any Python class that implements an XPCOM interface." I'm getting ahead of myself to say this just now, but I'd guess that if libpyxpcom.so is "missing in action", because it doesn't register properly, then all the Python code that needs to communicate with XPCOM components won't be able to do so. If all that is more or less correct, then it would seem to follow that the "Failed to load XPCOM component" messages I saw reported in the Firefox error console on my pc are actually rather important. )

The first significant progress I saw in this endeavor came from finding a Mozillazine forum post entitled "Failed to Load XPCOM Component [solved]", at

http://forums.mozillazine.org/viewtopic.php?f=19&t=787565

The post at the URL above links to another web page

https://developer.mozilla.org/en/Troubleshooting_XPCOM_components_registration

and THAT page (article), entitled "Troubleshooting XPCOM Components Registration", says one should respond to any XPCOM problems the Firefox error console reports by looking for missing dependent libraries via the "run-mozilla.sh" script. More fully, the article says:
[quote]
Linux-specific Hints

  Check if you have missing dependent libraries:

 From your Firefox (/XULRunner) install directory,
 run "./run-mozilla.sh `which ldd` -r path/to/your/component.so".

 There should be no "not found" entries and no undefined symbols.
 (The -r switch from GNU ldd lists function relocations; adjust
 as suitable for your version)[/quote]

Those "hints" also suggest using strace to see if Firefox is trying to load your components, and passing the the -Wl,-z,defs flag to gcc when linking your component to ensure that undefined symbols show an error at link time instead of failing at run time. Those seem like excellent suggestions, even if perhaps a bit beyond my abilities as basically an end-user. But in accordance with the first "hint" suggested, I opened a terminal window and executed:

[code]
cd /usr/lib/xulrunner-1.9.0.7/components , and then
/usr/lib/firefox-3.0.7/run-mozilla.sh `which ldd` -r ./libpyloader.so

( Or, identically: /usr/lib/firefox-3.0.7/run-mozilla.sh `which ldd` -r /usr/lib/xulrunner-1.9.0.7/components/libpyloader.so )
[/code]

I've reformatted the output results to make them a bit easier to read:

[code]
linux-gate.so.1 => (0xb7f46000)
libpthread.so.0 => /lib/tls/i686/cmov/libpthread.so.0 (0xb7f1d000)
libxpcom.so => not found <--------- libxpcom.so not found
libxul.so => not found <--------- libxul.so not found
libplds4.so.0d => /usr/lib/libplds4.so.0d (0xb7f19000)
libplc4.so.0d => /usr/lib/libplc4.so.0d (0xb7f15000)
libnspr4.so.0d => /usr/lib/libnspr4.so.0d (0xb7ee2000)
libdl.so.2 => /lib/tls/i686/cmov/libdl.so.2 (0xb7ede000)
libpython2.5.so.1.0 => /usr/lib/libpython2.5.so.1.0 (0xb7da8000)
libutil.so.1 => /lib/tls/i686/cmov/libutil.so.1 (0xb7da4000)
libpyxpcom.so => not found <--------- libpyxpcom.so not found
libstdc++.so.6 => /usr/lib/libstdc++.so.6 (0xb7cb1000)
libm.so.6 => /lib/tls/i686/cmov/libm.so.6 (0xb7c8c000)
libgcc_s.so.1 => /lib/libgcc_s.so.1 (0xb7c80000)
libc.so.6 => /lib/tls/i686/cmov/libc.so.6 (0xb7b31000)
/lib/ld-linux.so.2 (0xb7f47000)
[/code]

Note the three "not found" entries in the output above. Each of the three files

libxpcom.so,
libxul.so, and
libpyxpcom.so

seems not to have been where the "loader" library (libpyloader.so) expected to find them; thus - as I surmise - the symbols and XPCOM components that depend upon their correct registration will be unavailable. It's my guess that it's this which causes the "Failed to load XPCOM components" messages I saw in the Firefox error console.

They're absence is evidently more apparent than real, however. They're present on my pc as:

/usr/lib/xulrunner-1.9.0.7/libxpcom.so
/usr/lib/xulrunner-1.9.0.7/libxul.so
/usr/lib/xulrunner-1.9.0.7/libpyxpcom.so

So they're not missing, exactly. I suspect that what I'm assuming is the "loader" for them, libpyloader.so, might just be looking for them in the
wrong place. This suspicion is augmented by the somewhat unusual relative locations of the three "called" libraries with respect to the libpyloader.so library that I assume "calls" them. Somewhat strangely, the "loader" itself lives at:

/usr/lib/xulrunner-1.9.0.7/components/libpyloader.so

I would normally expect "called" libraries to occur "lower" in the directory tree than the "loader" that calls them but, here, the three "called" libraries occur *above* the one that calls them. ( Or so I speculate, anyway, re what calls what. )

I can't be sure that this is what's happening, of course, since I haven't been able to see the source code for libpyloader.so or the three libraries
that I'm presuming it calls. They're compiled binaries, of course, and although Ubuntu is supposed to be all open source, I don't know where to
start looking for the source code...

Oh, wait: I recall reading at http://ubuntuforums.org/showthread.php?t=661010 that the Debian package manager might be of help in such a case. I see that the command format I want is

dpkg -S /path/filename.so

Applying this command consecutively to each of

 /usr/lib/xulrunner-1.9.0.7/components/libpyloader.so ,
 /usr/lib/xulrunner-1.9.0.7/libxpcom.so ,
 /usr/lib/xulrunner-1.9.0.7/libxul.so, and
 /usr/lib/xulrunner-1.9.0.7/libpyxpcom.so

reveals that each is part of the "xulrunner-1.9" package, naturally enough.

Now that I think about it, that doesn't speak to which of the four files calls which other files, but I think it's reasonable to base a surmise
about that on the library (aka "shared object") names, as I did previously.

Presumably an app like the browser add-on "XPCOMViewer 0.9a" might be able to help me "enumerate all the components and interfaces availble to JavaScript and Firefox", which would go a long way toward seeing what calls what, I
should think, if I knew how to use it.

For now, however, I'm going with the idea that libpyloader.so does in fact "call" libxp.so , libxul.so , and libpyxpcom.so .

Since I've gone this far, I'll also just mention that one can apply the "strings" command to disclose any text strings tokens that may be embedded within the compiled .so file.

( Six undefined symbols were also reported, all associated with /usr/lib/xulrunner-1.9.0.7/components/libpyloader.so These were as follows: )

[code]
 _ZN14Py_nsISupports21PyObjectFromInterfaceEP11nsISupportsRK4nsIDii
 _Z16PyXPCOM_LogErrorPKcz
 _ZN14Py_nsISupports21InterfaceFromPyObjectEP7_objectRK4nsIDPP11nsISupportsii
 _Z24PyXPCOM_MakePendingCallsv
 _Z31PyXPCOM_EnsurePythonEnvironmentv
 _Z34PyXPCOM_SetCOMErrorFromPyExceptionv [/code]

I presume these six symbols must be defined in the three missing .so files, and would have been, had the libpyloader.so "loader"(?) file been able to find them where it expected to find them. But even though I think I more or less understand what's broken, I'm not at all sure what to do next. I'm not really a programmer or sysadmin at all, and have about reached the limit of my understanding here.

Is this a problem with the Ubuntu-supplied updates or with the Firefox 3.0.7 build, or is something else going on?

Should I try to upgrade to Firefox 3.1b, or just wait for a "fix for the fix" to be supplied from Ubuntu?

Could I just try copying the three missing .so files to new directories on my pc, to a new directory or directories where I *think* libpyloader.so might be "looking for them"?

Or should I try to install (say), the "XPCOMViewer 0.9a" add-on to my current Firefox installation, and try to figure out how to use that tool to "enumerate all the components and interfaces availble to JavaScript and Firefox"? ( That's language borrowed from the user comments for a different Firefox add-on that allows examination of XPCOM components. )

Still no videos, can't even re-install Flash, I find. Synaptic says it's installed, but it's nowhere to be seen when I look for in in Firefox ( Tools > Add-ons > Plugins ), nor do the directories that Firefox needs on disk get created. But that not a launchpad problem, I think, it's probably a user configuration one, in some way, after all the gyrations I've put my pc through to try to straighten out both that and what I conceive might be the underlying cause: the dependency problem I've described above.

Thanks very much for your time and effort in reviewing this, and for your work and great dedication more generally.

          ---------------------------------------------------------------------------------------

PS - (a note mostly to myself):

It appears from the text of the launchpad 217812 bug report that Alexander Sack may be the principal developer re Firefox, the one who integrates it into the Ubuntu package/update structure. Anyway, the final ( 21 Aug 2008 ) comment posted in https://bugs.launchpad.net/ubuntu/+source/firefox-3.0/+bug/217812 says:

"I've just installed the update for libnspr4-0d (4.7.1+1.9-ubuntu0.8.04.5) and everything is working fine now. It might be necessary to delete compreg.dat and xpti.dat from the used Fx profile to (re)register the component(s). I've just installed the update for libnspr4-0d (4.7.1+1.9-ubuntu0.8.04.5) and everything is working fine now."

Hmm ... He ( launchpad user 4711 ) refers to 8.04.5 here. If this is what broke my ability to watch streaming video in Hardy 8.04.2 then will I just have to upgrade to Intrepid or Jaunty to get that back? But then, I'm not *certain* that bug report even applies directly here.