Save a dolfin log

On Fri, Jun 15, 2012 at 08:01:22AM -0000, Gabriel Balaban wrote:
> New question #200511 on DOLFIN:
> https://answers.launchpad.net/dolfin/+question/200511
>
> HI does anyone know how issue the saving of the dolfin log inside a python script?
>
> So far I have been using
>
> python myscript.py | tee mylog.log
>
> but as I am running myscript with about 48 different parameter sets, it is easy to mix up the logs. It would be very nice have the script name the log approriatly and save it.

I suggest reading/writing the parameters from XML files, and store
those along with the log in subdirectories. One subdirectory for each
case. The subdirectory should contain all parameters, the log file,
and all solution data. Name the directories to something sensible
(case_1, case_2 etc).

Look at the fsirun functionality in CBC.Solve. You can set up
something similar in your code by using the read/write to XML of
parameters. You can even have a parameter called "case" or similar and
have your script automatically create the directory, store data to
that directory and store the XML data to it.

Also note that DOLFIN has functionality for parsing of parameters from
the command-line so you could read those parameters and store to XML.

--
Anders

In C++ it is possible to pass a stream to replace std::out as output:

   dolfin::set_output_stream(std::ostream& out)

We have, however, not been able to port this to Python.

Johan

On 06/15/2012 12:06 PM, Anders Logg wrote:
> Question #200511 on DOLFIN changed:
> https://answers.launchpad.net/dolfin/+question/200511
>
> Status: Open => Answered
>
> Anders Logg proposed the following answer:
> On Fri, Jun 15, 2012 at 08:01:22AM -0000, Gabriel Balaban wrote:
>> New question #200511 on DOLFIN:
>> https://answers.launchpad.net/dolfin/+question/200511
>>
>> HI does anyone know how issue the saving of the dolfin log inside a python script?
>>
>> So far I have been using
>>
>> python myscript.py | tee mylog.log
>>
>> but as I am running myscript with about 48 different parameter sets, it is easy to mix up the logs. It would be very nice have the script name the log approriatly and save it.
>
> I suggest reading/writing the parameters from XML files, and store
> those along with the log in subdirectories. One subdirectory for each
> case. The subdirectory should contain all parameters, the log file,
> and all solution data. Name the directories to something sensible
> (case_1, case_2 etc).
>
> Look at the fsirun functionality in CBC.Solve. You can set up
> something similar in your code by using the read/write to XML of
> parameters. You can even have a parameter called "case" or similar and
> have your script automatically create the directory, store data to
> that directory and store the XML data to it.
>
> Also note that DOLFIN has functionality for parsing of parameters from
> the command-line so you could read those parameters and store to XML.
>
> --
> Anders
>

Thanks for the advice so far.
 @Anders. My scripts can automatically generate the folders, data and plots, it is just the log that is missing and must be done by hand.

On the cluster bigblue I have a script that can create 48 jobs and 48 logs and create the logs appropriately.

However on the laptop I cannot use the same script because the python for loop would be run in serial, that is one run would be done at a time, wheras I would like to run 8 tests at a time on the laptop, which I do at the moment manually. It is for this manual starting of jobs where I wish I could generate automaticall the logs as well.

On 06/15/2012 04:11 PM, Gabriel Balaban wrote:
> Question #200511 on DOLFIN changed:
> https://answers.launchpad.net/dolfin/+question/200511
>
> Status: Answered => Open
>
> Gabriel Balaban is still having a problem:
> Thanks for the advice so far.
> @Anders. My scripts can automatically generate the folders, data and plots, it is just the log that is missing and must be done by hand.
>
> On the cluster bigblue I have a script that can create 48 jobs and 48
> logs and create the logs appropriately.
>
> However on the laptop I cannot use the same script because the python
> for loop would be run in serial, that is one run would be done at a
> time, wheras I would like to run 8 tests at a time on the laptop, which
> I do at the moment manually. It is for this manual starting of jobs
> where I wish I could generate automaticall the logs as well.
>

If you are using dolfin_utils.pjobs.submit you might want to try

    submit(..., serial=True)

then the same jobs will be run in serial locally on what ever computer
you are using.

Johan

Thanks Chris! Your solution looks good, using this line
so = open('%s/logfile.log' % directoryname, 'w', 0)

you can create logs with variable names.

Thanks Christopher Laing, that solved my question.