Parallel Processing with Escript

I assume what you have in mind is to write a script which splits the available ranks into groups and run escript on each of the groups. This should no be a problem if you don't use the MPI version of escript (you still could use OpenMP parallelization in escript if your MPI allows you to do this).

If you still what to use the escript MPI version, this is possible but not out of the box as it needs some C++ programming. The problem is that the domain generators don't accept MPI communicators form python however the escript library supports them on the C++ level. the reason why we don't exposed this to the python level yet is that we didn't have the need and couldn't make up our mind which python interface for MPI to use.

There is also the possibility to execute escript as a separate process - via ssh if you need to perform escript remotely (with MPI and or OpenMP) and using sockets to transfer data (if there aren't many of them). There are other solution along these lines depending on your system,

I am sorry I am not familiar with your terminology when you are referring to ranks. I have the following situation: I have 10 different simulations using escript which I would like to run at the same time. I would like to run each on a separate processor to start, and was hoping to accomplish this using the Parallel Python package. However, when I do this I get an error, and thus I was wondering if escript was compatible with other packages besides MPI. I tried this without using the MPI version. I am currently operating on Windows 7 64 bit machine with 12 available threads. I wanted to start just by writing parallel code for my own machine, but am not ruling out moving to a larger network if necessary.

I am not familiar with "Parallel Python". From my understanding it is not using MPI but I think it will be difficult to use the escript MPI version in connection with Parallel python. However, I would expect that the non-MPI version is compatible with Parallel python. What error message do you get?

Ok this is what I thought. However, when I run the programs I get an error that says that one of modules does not support pickeling, or can't be pickled. This may not be a problem on your side, and I am in touch with the other developers to make sure my calls have the correct syntax and form to avoid any such errors. I will let you know when I get a resolution to this issue.

Unless it is buried somewhere in one of the libraries we use, we don't use pickling at all.
We don't implement any of the interfaces required for pickling so it might be that something in parallel python is trying to pickle a data object to send it to another rank.

This does appear to be an issue with PP. Using another technique I was able to parellelize this just fine with no hassles.