batch-table

Asked by Lei Hang

Dear all,

When I practice using the batch-table in bouncing sphere script, the data file is empty. The modified script is as follows:
#
readParamsFromTable(damping=.2)
from yade.params import table

from yade import plot

O.bodies.append([
 # fixed: particle's position in space will not change (support)
 sphere(center=(0,0,0),radius=.5,fixed=True,color=(0,0,1)),
 # this particles is free, subject to dynamics
 sphere((0,0,2),.5,color=(0,0,1))
])

sphere1=O.bodies[-1]
z=sphere1.state.pos[2]-sphere1.state.refPos[2]

O.engines=[
 ForceResetter(),
 InsertionSortCollider([Bo1_Sphere_Aabb()]),
 InteractionLoop(
  [Ig2_Sphere_Sphere_ScGeom()], # collision geometry
  [Ip2_FrictMat_FrictMat_FrictPhys()], # collision "physics"
  [Law2_ScGeom_FrictPhys_CundallStrack()] # contact law -- apply forces
 ),
 # Apply gravity force to particles. damping: numerical dissipation of energy.
 NewtonIntegrator(gravity=(0,0,-9.81),damping=table.damping),
 PyRunner(command='checkUnbalanced()',realPeriod=2),
 PyRunner(command='addPlotData()',iterPeriod=1000)
]

O.dt=.5e-4*PWaveTimeStep()

O.trackEnergy=True

def checkUnbalanced():
 if unbalancedForce()<.05:
  O.pause()
  plot.saveDataTxt(O.tags['d.id']+'.data.bz2')

def addPlotData():
                plot.addData(i=O.iter,z=sphere1.state.pos[2]-sphere1.state.refPos[2],**O.energy)

plot.plots={'i':('z',None,O.energy.keys)}

plot.plot()

O.run()
waitIfBatch()

O.saveTmp()
#

After running the script, the error information is as follows,
#
Using python version: 3.8.2 (default, Apr 27 2020, 15:53:34)
[GCC 9.3.0]
Will run simulation(s) ['1batch.py'] using `/usr/bin/yade', nice value 10, using max 1 cores.
Will use table `params.txt', with available lines 2, 3, 4.
Will use lines 2 (damping=.6), 3 (damping=.4), 4 (damping=.2).
Master process pid 63773
Job summary:
   #0 (damping=.6): YADE_BATCH=params.txt:2 DISPLAY= /usr/bin/yade [threadspec] --nice=10 -x 1batch.py> 1batch.py.damping=.6.log 2>&1
   #1 (damping=.4): YADE_BATCH=params.txt:3 DISPLAY= /usr/bin/yade [threadspec] --nice=10 -x 1batch.py> 1batch.py.damping=.4.log 2>&1
   #2 (damping=.2): YADE_BATCH=params.txt:4 DISPLAY= /usr/bin/yade [threadspec] --nice=10 -x 1batch.py> 1batch.py.damping=.2.log 2>&1
http://localhost:9080 shows batch summary
#0 (damping=.6) started on Tue Jul 14 17:14:35 2020
#0 (damping=.6) FAILED (exit status 34304), duration 00:00:13, log 1batch.py.damping=.6.log
#1 (damping=.4) started on Tue Jul 14 17:14:48 2020
#1 (damping=.4) FAILED (exit status 34304), duration 00:00:15, log 1batch.py.damping=.4.log
#2 (damping=.2) started on Tue Jul 14 17:15:04 2020
#2 (damping=.2) FAILED (exit status 34304), duration 00:00:27, log 1batch.py.damping=.2.log
All jobs finished, total time 00:00:57
Log files: 1batch.py.damping=.6.log 1batch.py.damping=.4.log 1batch.py.damping=.2.log
Bye.
#

Could you tell me where is wrong in the script and how to deal with it?
Thanks a lot!

Question information

Language:
English Edit question
Status:
Solved
For:
Yade Edit question
Assignee:
No assignee Edit question
Solved by:
Jan Stránský
Solved:
Last query:
Last reply:
Revision history for this message
Jan Stránský (honzik) said :
#1

Hello,

please provide also the content of the table and which OS and Yade version you are using

> #0 (damping=.6) FAILED (exit status 34304), duration 00:00:13, log 1batch.py.damping=.6.log

FAILED means that there was some problem. Have a look at "1batch.py.damping=.6.log" file to see the problem (basically it is redirected output of the Yade run)

cheers
Jan

Revision history for this message
Lei Hang (h-stone) said :
#2

Hello,

Sorry, I don't know the meaning of 'OS'. I f it means operation system, the OS is Linux. The Yade version is 2020.01a. There is a "Fatal Python error" in the "1batch.py.damping=.6.log" file . But I don't understand this error and don't know how to deal with this error. The content of the table and the "1batch.py.damping=.6.log" file are as follows,

The content of the table:
#
damping
.6
.4
.2
#
The "1batch.py.damping=.6.log" file:
#
Welcome to Yade 2020.01a
Using python version: 3.8.2 (default, Apr 27 2020, 15:53:34)
[GCC 9.3.0]
Warning: no X rendering available (see https://bbs.archlinux.org/viewtopic.php?id=13189)
XMLRPC info provider on http://localhost:21000
TCP python prompt on localhost:9000, auth cookie `assduk'
Running script 1batch.py
/usr/lib/x86_64-linux-gnu/yade/py/yade/plot.py:444: MatplotlibDeprecationWarning:
The 'verts' kwarg was deprecated in Matplotlib 3.0 and will be removed in 3.2. Use 'marker' instead.
  scatter=pylab.scatter(scatterPt[0] if not math.isnan(scatterPt[0]) else 0,scatterPt[1] if not math.isnan(scatterPt[1]) else 0,s=scatterSize,color=line.get_color(),**scatterMarkerKw)
No handles with labels found to put in legend.
yade.plot: creating fake plot, since there are no y-data yet
Fatal Python error: This thread state must be current when releasing
Python runtime state: finalizing (tstate=0x15783b0)

Current thread 0x00007f83e1660700 (most recent call first):
  File "/usr/lib/python3.8/bz2.py", line 125 in close
  File "/usr/lib/x86_64-linux-gnu/yade/py/yade/plot.py", line 664 in saveDataTxt
  File "1batch.py", line 37 in checkUnbalanced
  File "<string>", line 1 in <module>

Thread 0x00007f83e43d4700 (most recent call first):
  File "/usr/lib/python3.8/selectors.py", line 415 in select
  File "/usr/lib/python3.8/socketserver.py", line 232 in serve_forever

Thread 0x00007f83e4bd5700 (most recent call first):
  File "/usr/lib/python3.8/selectors.py", line 415 in select
  File "/usr/lib/python3.8/socketserver.py", line 232 in serve_forever

Thread 0x00007f83f9e50740 (most recent call first):
  File "/usr/lib/python3.8/_weakrefset.py", line 38 in _remove
Aborted (core dumped)

=================== JOB SUMMARY ================
id : damping=.6
status : 34304 (FAILED)
duration: 00:00:13
command : YADE_BATCH=params.txt:2 DISPLAY= /usr/bin/yade --threads=1 --nice=10 -x 1batch.py> 1batch.py.damping=.6.log 2>&1
started : Wed Jul 15 09:02:36 2020
finished: Wed Jul 15 09:02:49 2020
#
Many thanks

Revision history for this message
Jan Stránský (honzik) said :
#3

yes, OS is operating system. Linux is too general, what is meant is distribution (Debian, Ubuntu) and version (e.g. Ubuntu 18.04)

> File "/usr/lib/x86_64-linux-gnu/yade/py/yade/plot.py", line 664 in saveDataTxt
> File "1batch.py", line 37 in checkUnbalanced

as the error says, the problem is on line 37, something about plot.saveDataTxt

> plot.saveDataTxt(O.tags['d.id']+'.data.bz2')

This worked for me:
###
...
def checkUnbalanced():
 if unbalancedForce()<.05:
  O.pause()
...
O.run()
waitIfBatch()
plot.saveDataTxt(O.tags['d.id']+'.data.bz2')
###

The problem is O.pause with O.wait (or waitIfBatch).
When O.pause() is called, the waiting is released. The only other (probably very quick) Python command is O.saveTmp().
This very quick command is executed and the script finishes, no matter that saving is in progress (it is called from C++ in another thread).

In the modified version, after waiting there are still Python commands (plot.saveDataTxt) to be processed and the script finishes after it is executed.

cheers
Jan

Revision history for this message
Lei Hang (h-stone) said :
#4

Thank you Jan,

The OS information is(Linux version 5.4.0-37-generic (buildd@lcy01-amd64-001) (gcc version 9.3.0 (Ubuntu 9.3.0-10ubuntu2)) #41-Ubuntu SMP Wed Jun 3 18:57:02 UTC 2020)

The method you provided solved my problem. But when I conduct the below script, why it can run normally? The position of "plot.saveDataTxt(O.tags['d.id']+'.data.bz2')" fllows the "O.pause()". It is not changed to behind the "waitIfBatch()".
The script:
#
readParamsFromTable(damping=.2)
from yade.params import table

from yade import pack, plot

O.bodies.append(geom.facetBox((.5,.5,.5),(.5,.5,.5),wallMask=31))

# create empty sphere packing
# sphere packing is not equivalent to particles in simulation, it contains only the pure geometry
sp=pack.SpherePack()
# generate randomly spheres with uniform radius distribution
sp.makeCloud((0,0,0),(1,1,1),rMean=.05,rRelFuzz=.5)
# add the sphere pack to the simulation

sp.toSimulation()

O.engines=[
 ForceResetter(),
 InsertionSortCollider([Bo1_Sphere_Aabb(),Bo1_Facet_Aabb()]),
 InteractionLoop(
  # handle sphere+sphere and facet+sphere collisions
  [Ig2_Sphere_Sphere_ScGeom(),Ig2_Facet_Sphere_ScGeom()],
  [Ip2_FrictMat_FrictMat_FrictPhys()],
  [Law2_ScGeom_FrictPhys_CundallStrack()]
 ),
 NewtonIntegrator(gravity=(0,0,-9.81),damping=table.damping),
 # call the checkUnbalanced function (defined below) every 2 seconds
 PyRunner(command='checkUnbalanced()',realPeriod=2),
 # call the addPlotData function every 200 steps
 PyRunner(command='addPlotData()',iterPeriod=100)
]
O.dt=.5*PWaveTimeStep()

# enable energy tracking; any simulation parts supporting it
# can create and update arbitrary energy types, which can be
# accessed as O.energy['energyName'] subsequently
O.trackEnergy=True

# if the unbalanced forces goes below .05, the packing
# is considered stabilized, therefore we stop collected
# data history and stop
def checkUnbalanced():
 if unbalancedForce()<.05:
  O.pause()
  plot.saveDataTxt(O.tags['d.id']+'.data.bz2')
  # plot.saveGnuplot('bbb') is also possible

# collect history of data which will be plotted
def addPlotData():
 # each item is given a names, by which it can be the unsed in plot.plots
 # the **O.energy converts dictionary-like O.energy to plot.addData arguments
 plot.addData(i=O.iter,unbalanced=unbalancedForce(),**O.energy)

# define how to plot data: 'i' (step number) on the x-axis, unbalanced force
# on the left y-axis, all energies on the right y-axis
# (O.energy.keys is function which will be called to get all defined energies)
# None separates left and right y-axis
plot.plots={'i':('unbalanced',None,O.energy.keys)}

# show the plot on the screen, and update while the simulation runs
plot.plot()

O.run() # run forever, until stopped by checkUnbalancedForce()
waitIfBatch()

O.saveTmp()
#

Revision history for this message
Best Jan Stránský (honzik) said :
#5

> But when I conduct the below script, why it can run normally?

I don't know. The evaluation is then "random", so sometimes it can be OK, sometimes not.

you can just try to put O.pause() at the end of checkUnbalanced, after plot.saveDataTxt.
Then it should work also in the batch mode.

cheers
Jan

Revision history for this message
Lei Hang (h-stone) said :
#6

Thank you!

Revision history for this message
Lei Hang (h-stone) said :
#7

Thanks Jan Stránský, that solved my question.