Help with Building/Running on Mac

Asked by jeywalk on 2009-02-02

Hello,

I have installed the pre-requisites for Relex-0.98.2, and it seems to build okay (at least that's what it tells me).

However, when I try "ant run" to build and run the example, an error is given:

     [java] Info: Using file defined in wordnet.configfile:data/wordnet/file_properties.xml
     [java] Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/commons/logging/LogFactory
     [java] at net.didion.jwnl.util.MessageLog.<init>(MessageLog.java:11)
     [java] at net.didion.jwnl.dictionary.Dictionary.<clinit>(Dictionary.java:25)
     [java] at java.lang.Class.forName0(Native Method)
     [java] at java.lang.Class.forName(Class.java:164)
     [java] at net.didion.jwnl.util.factory.Element.install(Element.java:31)
     [java] at net.didion.jwnl.JWNL.initialize(JWNL.java:157)
     [java] at relex.morphy.MorphyFactory.initializeJWNL(MorphyFactory.java:68)
     [java] at relex.morphy.MorphyJWNL.initialize(MorphyJWNL.java:102)
     [java] at relex.morphy.MorphyFactory.getImplementation(MorphyFactory.java:39)
     [java] at relex.RelationExtractor.init(RelationExtractor.java:119)
     [java] at relex.RelationExtractor.<init>(RelationExtractor.java:113)
     [java] at relex.RelationExtractor.main(RelationExtractor.java:415)

I have tried to use different versions of JWNL (1.3.3 and 1.4rc1&2), and it doesnt make a difference. Same message.

I have provided in "file_properties.xml" the path for WordNet, which is "/usr/local/WordNet-3.0" on my computer. I have tried both including and omitting "/dict", and that doesn't change anything either.

To the best of my knowledge, I have properly installed link-grammar-4.4.2, GNU getopt for java (java-getopt-1.0.13.jar is the one i am using), and trove. I then provided the paths to these pre-requisites in build.xml and relation-extractor.sh files. I THINK that's okay.

Is there something that I am missing? If someone could provide some help, that would be greatly appreciated!

Question information

Language:
English Edit question
Status:
Solved
For:
RelEx Edit question
Assignee:
No assignee Edit question
Solved by:
linas
Solved:
2009-02-12
Last query:
2009-02-12
Last reply:
2009-02-08
linas (linasvepstas) said : #1

Unfortunately, Java stack traces can be painful to read.

The relevant problem is here:

  [java] Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/commons/logging/LogFactory

You need to install commons-logging.jar which, for Ubuntu, is supplied by the libcommons-logging-java package. Other systems may include "apache" in the name ...

jeywalk (cychang) said : #2

Thanks linas,

I have installed commons-logging.jar properly and that message went away.

However, now I am getting this message:

run:
     [java] Info: Using file defined in wordnet.configfile:data/wordnet/file_properties.xml
     [java] Feb 4, 2009 5:28:08 PM net.didion.jwnl.util.MessageLog doLog
     [java] INFO: Installing dictionary net.didion.jwnl.dictionary.FileBackedDictionary@7a1767
     [java] Exception in thread "main" java.lang.UnsatisfiedLinkError: no link-grammar-java in java.library.path
     [java] at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1682)
     [java] at java.lang.Runtime.loadLibrary0(Runtime.java:822)
     [java] at java.lang.System.loadLibrary(System.java:993)
     [java] at org.linkgrammar.LinkGrammar.<clinit>(LinkGrammar.java:37)
     [java] at relex.parser.LocalLGParser.getVersion(LocalLGParser.java:250)
     [java] at relex.RelationExtractor.getVersion(RelationExtractor.java:142)
     [java] at relex.RelationExtractor.main(RelationExtractor.java:419)

I have specified in relation-extractor.sh that Djava.library.path=/usr/lib:/usr/local/lib

all my liblink-grammar-java files are located in /usr/local/lib, so i think that is linked properly..

any idea what my problem is?

linas (linasvepstas) said : #3

Odd, I don't know what could be wrong.

Post the output of
ls -la /usr/local/lib/liblink-gr*
and also
ldd /usr/local/lib/liblink-grammar*so

A long shot would be to edit link-grammar/java/org/linkgrammar/LinkGrammar.java and right around line 37, load the other library as well.

What version of java are you using?
What version of linux/unix are you using?

linas (linasvepstas) said : #4

oh, you are using a Mac right.

It is possible that Mac does not have automated shared library loading. That means that the "long shot" might not be so long, and is a likely fix. i.e. just comment out the line that says "if windows.." and just always perform that step.

jeywalk (cychang) said : #5

Apparently I do not have any .so files in my lib. I also looked at the linkgrammar.java file and shared library loading was already being performed...

my linkgrammar folder is located at /Users/Jey/Documents/workspace/link-grammar-4.4.2/link-grammar

my Relex is located at /Users/Jey/Documents/workspace/relex-0.98.2

how does Relex know where my linkgrammar folder is? i don't remember specifying this anywhere

build.xml currently links to the linkgrammar-4.4.2.jar located in /usr/local/share/java

there is a another linkgrammar-4.4.2.jar located in /Users/Jey/Documents/workspace/link-grammar-4.4.2/

is there a difference between these two jar files? and which one should i link to in the build.xml?

the commands you provided resulted in the following output:

jerry-changs-computer:/usr/local/lib Jey$ ls -la /usr/local/lib/liblink-gr*
-rwxr-xr-x 1 root wheel 72744 Feb 1 01:16 /usr/local/lib/liblink-grammar-java.4.4.2.dylib
lrwxr-xr-x 1 root wheel 32 Feb 1 01:16 /usr/local/lib/liblink-grammar-java.4.dylib -> liblink-grammar-java.4.4.2.dylib
-rw-r--r-- 1 root wheel 62840 Feb 1 01:16 /usr/local/lib/liblink-grammar-java.a
lrwxr-xr-x 1 root wheel 32 Feb 1 01:16 /usr/local/lib/liblink-grammar-java.dylib -> liblink-grammar-java.4.4.2.dylib
-rwxr-xr-x 1 root wheel 952 Feb 1 01:16 /usr/local/lib/liblink-grammar-java.la
-rwxr-xr-x 1 root wheel 770760 Feb 1 01:16 /usr/local/lib/liblink-grammar.4.4.2.dylib
lrwxr-xr-x 1 root wheel 27 Feb 1 01:16 /usr/local/lib/liblink-grammar.4.dylib -> liblink-grammar.4.4.2.dylib
-rw-r--r-- 1 root wheel 848108 Feb 1 01:16 /usr/local/lib/liblink-grammar.a
lrwxr-xr-x 1 root wheel 27 Feb 1 01:16 /usr/local/lib/liblink-grammar.dylib -> liblink-grammar.4.4.2.dylib
-rwxr-xr-x 1 root wheel 883 Feb 1 01:16 /usr/local/lib/liblink-grammar.la

jerry-changs-computer:/usr/local/lib Jey$ otool -L /usr/local/lib/liblink-grammar*so
otool: can't open file: /usr/local/lib/liblink-grammar*so (No such file or directory)

linas (linasvepstas) said : #6

Clearly, MacOS called them "dylib" (dynamically loadable library) instead of "so" (shared object). I believe they are equivalent if not identical, except for the spurious name change (assuming the Apple engineers followed the processor ABI instead of tweaking it).

Relex does not need to know about either your link-grammar folder, nor about your relex folder. It only needs to find the linkgrammar-4.4.2.jar, which it has clearly found, and it needs to find and load the link-grammar shared library i.e. to find liblink-grammar-java.dylib Now, this is clearly visible in your listing, so the question is, "why does java not see it?"

Do you know java programming? The only way that I know of to debug this is to write a simple, three-line long program that attempts to load some dylib, any dylib, and try to get that to work. If that works, then attention needs to shift to why the link-grammar dylib failed to load.

I'm hoping you know at least some java programming, as RelEx is fairly useless all by itself -- it needs to be tailored for your specific needs ...

jeywalk (cychang) said : #7

I do know java...i wrote some simple codes to try and load the dylib.. and it seems to give me no errors:

public static void main(String[] args) {
 System.load("/usr/local/lib/liblink-grammar-java.dylib");
        System.out.println("Some text");

could it be that the .jar file is not properly calling the libs?

linas (linasvepstas) said : #8

> i wrote some simple codes to try and load the dylib..
> and it seems to give me no errors:
>
> public static void main(String[] args) {
> System.load("/usr/local/lib/liblink-grammar-java.dylib");
> System.out.println("Some text");

Yes, but you specified a full file-path name. This is not how
link-grammar is doing it -- it is doing the following:

  System.load("link-grammar-java");

Java is supposed to automatically deal with the directory
/usr/local/lib and the prefix lib and suffix .dylib. Are you
able to load the library with the "correct", short name?

linas (linasvepstas) said : #9

2009/2/5 jeywalk <email address hidden>:
> I do know java...i wrote some simple codes to try and load the dylib..
> and it seems to give me no errors:
>
> public static void main(String[] args) {
> System.load("/usr/local/lib/liblink-grammar-java.dylib");
> System.out.println("Some text");

Yes, but you specified a full file-path name. This is not how
link-grammar is doing it -- it is doing the following:

   System.load("link-grammar-java");

Java is supposed to automatically deal with the directory
/usr/local/lib and the prefix lib and suffix .dylib. Are you
able to load the library with the "correct", short name?

muriloq (muriloq) said : #10

You need to specify the directory using a system property:

java -Djava.library.path=/usr/local/lib

The JVM will choose the right extension (.so, .dll or .dylib), and if
necessary add the prefix lib at the beginning, automatically.

On Thu, Feb 5, 2009 at 8:41 PM, linas
<email address hidden>wrote:

> Question #59534 on RelEx changed:
> https://answers.launchpad.net/relex/+question/59534
>
> Status: Open => Needs information
>
> linas requested for more information:
> > i wrote some simple codes to try and load the dylib..
> > and it seems to give me no errors:
> >
> > public static void main(String[] args) {
> > System.load("/usr/local/lib/liblink-grammar-java.dylib");
> > System.out.println("Some text");
>
> Yes, but you specified a full file-path name. This is not how
> link-grammar is doing it -- it is doing the following:
>
> System.load("link-grammar-java");
>
> Java is supposed to automatically deal with the directory
> /usr/local/lib and the prefix lib and suffix .dylib. Are you
> able to load the library with the "correct", short name?
>
> You received this question notification because you are a member of
> RelEx Developers, which is an answer contact for RelEx.
>

--
Murilo Saraiva de Queiroz, MSc.
Senior Software Engineer
http://www.vettalabs.com
http://www.tecnologiainteligente.com.br

jeywalk (cychang) said : #11

Okay, I tried to load the library dynamically, using:

System.load("link-grammar-java")

you're right, it failed (i tried "link-grammar" as well). I then tried another dynamic library that I know is there, "sqlite3", and that failed too.

then i tried it with the -Djava.library.path=/usr/local/lib option, and I still could not load either dynamic library.

I also included /usr/local/lib:/usr/lib in the $DYLD_LIBRARY_PATH and $LD_LIBRARY_PATH environment variables. Again, that produced no difference.

Maybe I am giving it the wrong short-name, but linkgrammar-4.4.2.jar must be calling it with its correct short name. Could it be something wrong with how the JVM on my machine is handling dynamic libraries?

Thanks

jeywalk (cychang) said : #12

I tried using System.loadLibrary() as well, no difference

linas (linasvepstas) said : #13

2009/2/6 jeywalk <email address hidden>:

>Could it be something
> wrong with how the JVM on my machine is handling dynamic libraries?

That's what it would seem like, if you've really verified
that the library path and etc. are correct. But this seems
hard to beleive, as there would be other things that are
broken, too ...

What version of the Java JVM are you using?

Also: what does System.getProperty("os.name"); return?

I guess we could special-case handling for the Mac ...
assuming that other users might have this problem?!?

jeywalk (cychang) said : #14

I am running on Java(TM) 2 Runtime Environment, Standard Edition (build 1.5.0_16-b06-275)

System.getProperty("os.name") returns "Mac OS X"

Do you know of a dynamic library that Mac has for sure, and its correct short-name? I want to test it on another library to confirm the problem.

btw, when you asked me to edit LinkGrammar.java, i edited the one located in "/Users/Jey/Documents/workspace/link-grammar-4.4.2/link-grammar/java/org/linkgrammar/LinkGrammar.java". This is the only one on my computer. Out of interest, how does 'ant run' know to call this java file?

thanks

muriloq (muriloq) said : #15

Are you using Java 6 ? Maybe the problem is that the library is 32-bit and
the JVM 64-bit, according to this link:

http://forums.sun.com/thread.jspa?threadID=5361228&tstart=1

On Sat, Feb 7, 2009 at 3:52 AM, jeywalk <<email address hidden>
> wrote:

> Question #59534 on RelEx changed:
> https://answers.launchpad.net/relex/+question/59534
>
> Status: Answered => Open
>
> jeywalk is still having a problem:
> I am running on Java(TM) 2 Runtime Environment, Standard Edition (build
> 1.5.0_16-b06-275)
>
> System.getProperty("os.name") returns "Mac OS X"
>
> Do you know of a dynamic library that Mac has for sure, and its correct
> short-name? I want to test it on another library to confirm the problem.
>
> btw, when you asked me to edit LinkGrammar.java, i edited the one
> located in "/Users/Jey/Documents/workspace/link-grammar-4.4.2/link-
> grammar/java/org/linkgrammar/LinkGrammar.java". This is the only one on
> my computer. Out of interest, how does 'ant run' know to call this java
> file?
>
> thanks
>
> --
> You received this question notification because you are a member of
> RelEx Developers, which is an answer contact for RelEx.
>

--
Murilo Saraiva de Queiroz, MSc.
Senior Software Engineer
http://www.vettalabs.com
http://www.tecnologiainteligente.com.br

linas (linasvepstas) said : #16

2009/2/7 muriloq <email address hidden>:
> Are you using Java 6 ? Maybe the problem is that the library is 32-bit and
> the JVM 64-bit, according to this link:
>
> http://forums.sun.com/thread.jspa?threadID=5361228&tstart=1

Hmm. But if MacOS is 64-bit, wouldn't link-grammar
automatically build a 64-bit lib? Perhaps the default on Mac
is to build 32-bit systems (to minimize ram mem usage )
I don't know how to explicitly control this during
the build process. Maybe
./configure --build=(some 64-bit mac target ??)

linas (linasvepstas) said : #17

2009/2/6 jeywalk <email address hidden>:
> I am running on Java(TM) 2 Runtime Environment, Standard Edition (build
> 1.5.0_16-b06-275)

HMM OK, I think that's Java 5 not Java 6

> Do you know of a dynamic library that Mac has for sure, and its correct
> short-name?

No, but presumably hundreds .. almost everything in /lib and
/usr/lib will by a dynamically loadable library. (unless it ends with
the suffix .a, which is for development only,)

> btw, when you asked me to edit LinkGrammar.java, i edited the one
> located in "/Users/Jey/Documents/workspace/link-grammar-4.4.2/link-
> grammar/java/org/linkgrammar/LinkGrammar.java". This is the only one on
> my computer. Out of interest, how does 'ant run' know to call this java
> file?

???
If you edit that file, you will have to rebuild link-grammar (by
saying make) and then you''l have to install it ('make install')
RelEx only 'knows' about the jars specified in its class-path. For
your system, I'm guessing the link-grammar jar file ended up in
/usr/local/share/java and there is where RelEx is looking for it.

Instead of using 'ant run' with Relex, you will find it a zillion
times less awkward to edit the shell script "relation-extractor.sh" as
appropriate, and use that .

Best linas (linasvepstas) said : #18

2009/2/7 Linas Vepstas <email address hidden>:

>> btw, when you asked me to edit LinkGrammar.java, i edited the one
>> located in "/Users/Jey/Documents/workspace/link-grammar-4.4.2/link-
>> grammar/java/org/linkgrammar/LinkGrammar.java". This is the only one on
>> my computer. Out of interest, how does 'ant run' know to call this java
>> file?
>
> If you edit that file, you will have to rebuild link-grammar (by
> saying make) and then you''l have to install it ('make install')
> RelEx only 'knows' about the jars specified in its class-path. For
> your system, I'm guessing the link-grammar jar file ended up in
> /usr/local/share/java and there is where RelEx is looking for it.

Does it work, if you edit that file, and change the load
command to load "liblink-grammar-java.dylib", and recompile.
reinstall?

jeywalk (cychang) said : #19

I found the part of the problem, it's sort of a silly one.

When I edited LinkGrammar.java, and recompiled it, a new jar wasn't being created because the old one was still there. So I have to delete jar files both from the /usr/local/lib and /linkgrammar-4.4.2 folder before I recompile. Now I can make changes to LinkGrammar.java and have it reflected.

I still cannot dynamically load any libraries, instead I provided the full path. I loaded the two following dylibs:

        System.load("/usr/local/lib/liblink-grammar.dylib");
        System.load("/usr/local/lib/liblink-grammar-java.4.4.2.dylib");

After doing this, I seem to be able to at least run ./relation-extractor.sh with no problem.

I am not sure if I am missing any libraries by loading them using the full path, but when i tried to run ./batch-process.sh, i get the following error:

Exception in thread "main" java.lang.NoClassDefFoundError: org/linkgrammar/LGConfig
        at relex.parser.LGParser.<init>(LGParser.java:7)
        at relex.parser.LocalLGParser.<init>(LocalLGParser.java:15)
        at relex.RelationExtractor.init(RelationExtractor.java:117)
        at relex.RelationExtractor.<init>(RelationExtractor.java:109)
        at relex.WebFormat.<init>(WebFormat.java:50)
        at relex.WebFormat.main(WebFormat.java:118)

Clearly, LGConfig.java is there when i compiled it, so I am not sure why the above error is being produced.

What I am really trying to accomplish with RelEx is to load some text-reports (each 10-15 pages) and extract the tree and relations, so I can develop some patterns to get useful information from these reports.

I am guessing the best way to do this is learn the java codes for Relex, and adapt it for my needs?

thanks for the help

linas (linasvepstas) said : #20

2009/2/7 jeywalk <email address hidden>:
> After doing this, I seem to be able to at least run ./relation-
> extractor.sh with no problem.

Good.

> I am not sure if I am missing any libraries by loading them using the
> full path,

I think you're good.

> but when i tried to run ./batch-process.sh, i get the
> following error:

The classpath in that shell script is incorrect --
its looking for linkgrammar .jar in the wrong place.
If you fix it it will work.

> What I am really trying to accomplish with RelEx is to load some text-
> reports (each 10-15 pages) and extract the tree and relations, so I can
> develop some patterns to get useful information from these reports.
>
> I am guessing the best way to do this is learn the java codes for Relex,
> and adapt it for my needs?

Yes. re-read the section called "Using RelEx" in
the README file. Takes its words to heart.

--linas

jeywalk (cychang) said : #21

Thanks for the help. Things seem to have worked!

jeywalk (cychang) said : #22

Thanks linas, that solved my question.