Save and restore

Hi Jose,

I've asked a colleague to provide a technical answer to this question as I am not so familiar with the details of checkpoint-restart. To start, let me give you a bit of history though of checkpoint-restart in ESyS-Particle.

Some time ago the CheckPointer was added to ESyS with the intention being that it could be both used for simulation data output/visualisation and for restarting simulations. Sadly the restart facility using CheckPoint files was not very robust so now the CheckPointer is only useful for data output/visualisation. We plan to rename it to something like SnapShooter to better describe its current purpose.

All is not lost however! ESyS-Particle-2.0 has a new facility that does permit checkpoint-restart of simulations. It uses a similar algorithm for saving CheckPoint files but the file format is somewhat different (including more information specifically required to restore the model state).

To create checkpoint-restart files, you need to use LsmMpi.createRestartCheckPointer(..) and supply that subroutine with CheckPointPrms, in much the same way as createCheckPointer(..). The Restart files can be loaded in a simulation script to restore a model state using LsmMpi.loadCheckPoint(..). These files cannot however, be used to post-process/visualise simulation data using e.g. dump2vtk or grainextract. You will still need a CheckPointer for that.

Now a word of caution: This CheckPoint-Restart facility is fairly new and still a bit experimental. It does work for a variety of different simulation types but may require some work to write a restart script with everything in the correct order. I won't try to explain but rather defer to those who better understand the nuances of the current checkpoint-restart facility.

> can i change properties during the run (like density, gravity, etc)?

For the most part, properties cannot be changed during a simulation. This is not necessarily because it is a complex thing to do. Mostly it is because no-one has found an application yet requiring such a feature. Not long ago a colleague implemented a way to change the normal vector of simple walls during simulations. This small change allowed us to simulate a shear box in which side walls are vibrated backwards and forwards. A very nice little solution to a potentially complicated problem!

If you have a specific property in mind that you would like to change during a simulation, let me know and I'll look into how to implement that. A word of warning though: in numerical models using explicit finite-difference time-integration, you may run into stability problems if you suddenly change properties like density, elastic stiffness, particle radius etc.

Cheers,

Dion.

Thank you very much for the explanation Dion.
Indeed, i used the commnad you sugested:

sim.createRestartCheckPointer(
    CheckPointPrms(
   fileNamePrefix ="SAVE",
       beginTimeStep=10000,
       endTimeStep=100000,
       timeStepIncr=100000,
       )
)
again it creates two .txt files for each savepoint, like the ones i mentioned before, with the exception that the second one also contains the names, coordenates and normal vectors of all walls.
i try to restart the simulation in a different file where everything from the original is the same except that in the place where the balls and walls are created, i replace with:

sim.loadCheckPoint("SAVE_t=20000_1.txt")

however i get the following error:
RuntimeError: Unrecognised file type -1 expected LSMGeometry.

regarding my second question, i was able to overcome this by creating a pretimestepprunnable that starts an interaction only at the desired timestep. so for instance if i want to begin gravity of (0,-10,0) and then reduce it to (0, -1,0) at timestep 10.000, what i do is create an interaction group of (0,-10,0) and tell my prestimesteprunnable that i want another one of (0,9,0) starting at the timestep 10.000.

Hi Jose,

The error message is probably because you need to supply the name of the restart file ending with _0.txt i.e.
sim.loadCheckPoint("SAVE_t=20000_0.txt")

Just like the ordinary checkpoint files, the *_0.txt files are headers that aid the code to read the other files for that savepoint.

You may still experience a few unexpected problems even after eliminating your current error massage. Here are a couple of brief suggestions:

1) the restart script (containing the loadCheckPoint command) must explicitly set the simulation spatial domain using LsmMpi.setSpatialDomain(..). This should be done before calling loadCheckPoint(..). Ideally do this straight after calling initNeighbourSearch(..).

2) Some interaction groups need to be created before the loadCheckPoint command. Looking at my test script, these interaction groups define the bonded or unbonded interactions between particles. Particle-wall interactions and damping interactions must be created after the loadCheckPoint command.

3) Don't forget to change the total number of timesteps using setNumTimeSteps(..) to a value larger than the timestep number of the savepoint (i.e. > 20000 for your example) otherwise the simulation will simply terminate without doing any calculations.

I hope this helps and good luck!

Cheers,

Dion

Thank you for your efforts Dion, in fact i didn't mention the *_0.txt files becaus they also produce errors.
By following your advice i get the following message: RuntimeError: Expected 'Dimension', got 'SAVE_t=20000_1.txt'.

The header file SAVE_t=20000_0.txt has the following content:

V 2
1000000
0.001
20000
LSMGeometry 0
BoundingBox 0 0 0 0 0 0
PeriodicBoundaries 0 0 0
SAVE_t=20000_1.txt

So i guess it reads everything fine until the last line where it seems to be expecting something different. Do your header files have the same content?

Anyway, i will also paste my restart script up to the point where i try to load:

from esys.lsm import *
from esys.lsm.util import *
from esys.lsm.geometry import *
from WallLoader import *
from POVsnaps import POVsnaps
#instantiate a simulation object:
sim = LsmMpi (numWorkerProcesses = 1, mpiDimList = [1,1,1])
#initialise the neighbour search algorithm:
sim.initNeighbourSearch (
    particleType = "RotSphere",
    gridSpacing = 50.0,
    verletDist = 3.0
)
#set the number of timesteps and timestep increment:
sim.setNumTimeSteps (1000000)
sim.setTimeStepSize (1.0000e-03)

#specify the spatial domain for the simulation
domain = BoundingBox(Vec3(-400.0,-400.0,-400.0), Vec3(400.0,1200.0,400.0))
sim.setSpatialDomain (domain)

#initialise frictional interactions for unbonded particles:
sim.createInteractionGroup (
   RotFrictionPrms(
      name="friction",
      normalK=1E6,
      dynamicMu=0,
      staticMu=0.5,
      shearK=1E6,
      scaling = True
   )
)

sim.loadCheckPoint("SAVE_t=20000_0.txt")

sorry to resurrecti this topic but can anyone else help me?

Hi Jose,

I think i understand your problem. When you look at your header file:

V 2
1000000
0.001
20000
LSMGeometry 0
BoundingBox 0 0 0 0 0 0
PeriodicBoundaries 0 0 0
SAVE_t=20000_1.txt

Three lines are not set properly:
1. LSMGeometry 0 for a regular restart file will be LSMGeometry 1.2
2. Bounding Box will specify an actual box using 2 vectors for minimum and maximum. (This is your simulation domain)
3. You should also have another line "Dimension 3D" between PeriodicBoundaries and the file list.

These variables are actually internally set when using geometry files and the sim.readGeometry() function so that when you write out a checkpoint restart file they will also be written. A work around to this problem will be to try and change the header file manually with what i've given you above. If that fails i would suggest creating your geometry using LSMGenGeo and using the sim.readGeometry() function.

Let us know how you go...

Regards,
Will