Problem with importing "external" modules

Asked by CryoGenID

I have got (again *g*) a problem (funny thing is, that this once worked but won't work now anymore...):

I have a "main" sikuli.file (Runme.sikuli) which loads other "external" sikuli-files on demand and runs a pre-defined
routine "runModule()" inside of the loaded "external" modules. These are then doing their tests and the main sikuli
file loads the next module and so on...

All these files are opened via \\FILESERVER\Directory\Directory\Directory\Runme.sikuli (for the runme.sikuli) and
\\FILESERVER\Directory\Directory\Directory\Runme.sikuli\Tests\Test1.sikuli (for the first module).

The external modules "InitSystem" and "GlobalSettings" are opened and found correctly, they are saved here:
\\FILESERVER\Directory\Directory\Directory\Runme.sikuli\InitSystem.sikuli
\\FILESERVER\Directory\Directory\Directory\Runme.sikuli\GlobalSettings.sikuli

So the other external modules are one level more down (in the "Tests"-directory) than "InitSystem" and "GlobalSettings".

Here is the code (I have cut out all the other code which is not important here) of Runme.sikuli:
---
import InitSystem
import GlobalSettings

import os
dir = os.path.dirname(getBundlePath())
tests = os.path.join(dir, "Tests")

modulesToRun = ("Test1", "Test2", "Test_OCR")

print("Testing started: " + InitSystem.getCurrentTime())
popup("Abort test with ALT+SHIFT+C")

if (InitSystem.startUp(GlobalSettings.screenT, GlobalSettings.screenQ, GlobalSettings.screenP)==0): # Init the tests
    popup("ERROR during startup, cancelling all tests...")
    exit
else: # Startup was OK
    # Iterating through all Test-Modules
    popup("OK starting Tests in Modules")
    for currentModule in modulesToRun:
        popup("Working with module " + currentModule + " in directory " + tests)

        currDir = os.path.join(tests, currentModule+".sikuli")
        if not currDir in sys.path:
            sys.path.append(currDir)

        popup("Opening " + currentModule)
        # Position XXXX1
        exec("import " + currentModule)
        exec(currentModule + ".runModule()")

InitSystem.shutDown() # Shut down the test
print("Testing ended: " + getCurrentTime())
---

The contents of the variables are at Position XXXX1:
currentModule: Test1
currDir: \\FILESERVER\Directory\Directory\Runme.sikuli\Tests\Test1.sikuli

The Error I get is:
---
[error] Error message: Traceback (most recent call last):
 File "c:\temp\windows\sikuli-tmp2898954620263380028.py", line 27, in
 exec("import " + currentModule)
 File "", line 1, in
ImportError: No module named Test1
---

Can please anybody help me? :)

I hope that I could describe the setup here :-)

Thanks a lot and best regards,

Chris

Question information

Language:
English Edit question
Status:
Solved
For:
SikuliX Edit question
Assignee:
No assignee Edit question
Solved by:
CryoGenID
Solved:
Last query:
Last reply:
Revision history for this message
RaiMan (raimund-hocke) said :
#1

the directory, that has to be in sys.path, to <import xxx> stored at some-path/xxx.siklui has to be the path CONTAINING the .sikuli you want to import.

so it has to be:
        currDir = tests
        if not currDir in sys.path:
            sys.path.append(currDir)

and this can be outside the loop, since it is static for your loop.

Revision history for this message
CryoGenID (cryo-t) said :
#2

Hello RaiMan!

Thanks a lot again for your help :-)

Unfortunately it doesn't work :-(

Here is the "new" code, based on your comment above:
---
import InitSystem
import GlobalSettings

import os
dir = os.path.dirname(getBundlePath())
testsModulePath = os.path.join(dir, "Tests")

modulesToRun = ("Test1", "Test2", "Test_OCR")

print("Testing started: " + InitSystem.getCurrentTime())
popup("Abort test with ALT+SHIFT+C")

if (InitSystem.startUp(GlobalSettings.screenT, GlobalSettings.screenQ, GlobalSettings.screenP)==0): # Init the tests
    popup("ERROR during startup, cancelling all tests...")
    exit
else: # Startup was OK
    # Iterating through all Test-Modules
    popup("OK starting Tests in Modules")

    if not testsModulePath in sys.path:
        sys.path.append(testsModulePath)
    for currentModule in modulesToRun:
        popup("Working with module " + currentModule + " in directory " + testsModulePath)
        # --> Output: "Working with module Test1 in directory \\filecore1\xxx\xxx\xxx\Testing\Tests"
        popup("Opening " + currentModule)
       # --> Output: "Opening Test1"
        exec("import " + currentModule)
       # --> Output:
       # "File "c:\temp\windows\sikuli-tmp1197008925883503694.py", line 26, in
       # exec("import " + currentModule)
       # File "", line 1, in
       # ImportError: No module named Test1"
       # exec(currentModule + ".runModule()")

InitSystem.shutDown() # Shut down the test
print("Testing ended: " + getCurrentTime())
---

I have added the output of the MsgBoxes and the errormessage (after the "faulty" command) in the code
above as a comment.

Regarding the error:
I don't understand it, there is a directory in "Tests" called "Test1.sikuli", in there is "Test1.py", "Test1.html" and "Test1$py.class".

Please tell me that you have another idea ;-)

OS: Win7 x64

Thanks a lot and best regards,

Christian

Revision history for this message
RaiMan (raimund-hocke) said :
#3

Then the only possibility left:

exec ("import " + "Test1")

is not recognized by Sikuli's special handling of import statements for other Sikuli scripts - it is directly processed by the Jython interpreter, which looks for a Test1.py, which is not there.

So you have to switch your approach to Jython import combined with setBundlePath():

saved_sys_path = [for e in sys.path] # save sys.path content

for currentModule in modulesToRun:
    currentModuleSikuli = currentModule+".sikuli"
    sys.path.append(os.path.join(testsModulePath, currentModuleSikuli))
    setBundlePath(currentModuleSikuli)
    exec("import " + currentModule)
    exec(currentModule + ".runModule()")
    sys.path = [for e in saved_sys_path] # reset sys.path

BTW:
I do not really understand this complex approach.
Why do you not use Python's unit test (see faq 1804)

Revision history for this message
CryoGenID (cryo-t) said :
#4

Hello Raimund!

Thanks again for your help...
Wouldn't know how far we had come with the testings without you ;-)

But this topic here is really a problem...

If I use your code directly, I get:
---
error] Stopped
[error] An error occurs at line 29
[error] Error message: SyntaxError: ("no viable alternative at input 'for'", ('c:\\temp\\windows\\sikuli-tmp1337676288801423763.py', 29, 22, ' saved_sys_path = [for e in sys.path] # save sys.path content\n'))
---

If I simply leave the problematic statement out, I get the "old" error message again:
---
[error] Stopped
[error] An error occurs at line 35
[error] Error message: Traceback (most recent call last):
 File "c:\temp\windows\sikuli-tmp6340195059774416002.py", line 35, in
 exec("import " + currentModule)
 File "", line 1, in
ImportError: No module named Test1
---

I don't understand it... The whole thing already worked before and now THIS :-(

I will have a look into the Unit Test, but I think that my approach is not sooooo bad and complex, is it? ;-)

I have a "Main"-Module which makes the basic settings and then calls 1..n external Modules in which the tests
are conducted... Only that exactly that call is now making problems :-(

I really hope that we can solve this soon.... I really need it ;-)

Thanks and best regards from Munich!

Chris

Revision history for this message
RaiMan (raimund-hocke) said :
#5

Sorry for this simple syntax error :-(

must be:

saved_sys_path = [e for e in sys.path] # save sys.path content

and

    sys.path = [e for e in saved_sys_path] # reset sys.path

I am out now for some hours (on vacation ;-)

Revision history for this message
CryoGenID (cryo-t) said :
#6

Hello Raimund!

Oh vacation :-)
Bad for me, good for you ;-)

Unfortunately, still the same error:
---
[error] An error occurs at line 34
[error] Error message: Traceback (most recent call last):
 File "c:\temp\windows\sikuli-tmp7519093718142918498.py", line 34, in
 exec("import " + currentModule)
 File "", line 1, in
ImportError: No module named Test1
---

What can we do? :(

Thanks a lot and best regards,

Christian

Revision history for this message
RaiMan (raimund-hocke) said :
#7

I just made some tests on my Mac and the solution using the version with the Test1.sikuli worked for me.

I cannot test on Windows currently and the next days, so I can only ask you, to make some basic tests, wether your approach principally works.

# main.sikuli in some_directory
import os
dir = os.path.dirname(getBundlePath())
importDir = os.path.join(dir, "Tests")
if not importDir in sys.path:
    sys.path.append(importDir)
exec("import Test1")

# Test1.sikuli in some_directory\Tests\
print "I am imported now"

Revision history for this message
CryoGenID (cryo-t) said :
#8

Hello Raimund!

It is working now...
I could hit my head against the wall, such a dumb error ;-)

After I copied your demo-code and got the same error message again, I went on and printed out the content
of "sys.path" and there I found out that the directory of the "main"-sikuli script under which the "Tests" directory resides, was not part of that path.

So the simple solution was to include that:
---
testsModulePath = os.path.join(dir, "RUNME.sikuli\Tests")
---

And that's it ;-)

So if anybody is interested, here is the working code:
---
import InitSystem
import GlobalSettings
import os
dir = os.path.dirname(getBundlePath())
testsModulePath = os.path.join(dir, "RUNME.sikuli\Tests")

modulesToRun = ("Test1", "Test2", "Test_OCR")

print("Testing started: " + InitSystem.getCurrentTime())

if (InitSystem.startUp(GlobalSettings.screenT, GlobalSettings.screenQ, GlobalSettings.screenP)==0): # Init the tests
    popup("ERROR during startup, cancelling all tests...")
    exit
else: # Startup was OK
    # Iterating through all Test-Modules
    popup("OK starting Tests in Modules")

    if not testsModulePath in sys.path:
        sys.path.append(testsModulePath)
    for currentModule in modulesToRun:
       popup("Working with module " + currentModule + " in directory " + testsModulePath)
       popup("Opening " + currentModule)
       exec("import " + currentModule)
       exec(currentModule + ".runModule()")

InitSystem.shutDown() # Shut down the test
print("Testing ended: " + getCurrentTime())
---

Thanks a LOT Raimund and best regards,

Christian

Revision history for this message
RaiMan (raimund-hocke) said :
#9

Ok, but this only makes sense, if your main script IS IN FACT RUNME.sikuli (contains a RUNME.py).

From your initial question one could get the impression, that it CONTAINS a runme.sikuli:
... All these files are opened via \\FILESERVER\Directory\Directory\Directory\Runme.sikuli (for the runme.sikuli) ...

and using
dir = os.path.dirname(getBundlePath())

made this even more probable, since this gives the directory, that CONTAINS the some_script.sikuli, in your real case
\\FILESERVER\Directory\Directory\Directory\

so if your directory structure really is like this:

some-path\
  -- RUNME.sikuli
     -- RUNME.py
     -- InitSystem.sikuli
     -- GlobalSettings.sikuli
     -- Tests
         -- Test1.sikuli
         -- Test2.sikuli

then the real (general solution) is, to take the return of getBundlePath() as the base directory for the sys.path manipulation.

BTW: I guess in the last time it has to read
InitSystem.getCurrentTime()

# main script (content of RUNME.py)
import InitSystem
import GlobalSettings
import os
dir = getBundlePath() # changed ###
testsModulePath = os.path.join(dir, "Tests") # changed ###

modulesToRun = ("Test1", "Test2", "Test_OCR")

print("Testing started: " + InitSystem.getCurrentTime())

if (InitSystem.startUp(GlobalSettings.screenT, GlobalSettings.screenQ, GlobalSettings.screenP)==0): # Init the tests
    popup("ERROR during startup, cancelling all tests...")
    exit
else: # Startup was OK
    # Iterating through all Test-Modules
    popup("OK starting Tests in Modules")

    if not testsModulePath in sys.path:
        sys.path.append(testsModulePath)
    for currentModule in modulesToRun:
       popup("Working with module " + currentModule + " in directory " + testsModulePath)
       popup("Opening " + currentModule)
       exec("import " + currentModule)
       exec(currentModule + ".runModule()")

InitSystem.shutDown() # Shut down the test
print("Testing ended: " + InitSystem.getCurrentTime()) # changed ###

Revision history for this message
CryoGenID (cryo-t) said :
#10

Raimund,

again: THANKS :)

Now it works like a charm and is now independent of the name of the "main"-sikuli file :-)

Best regards,

Christian