ESyS-Particle

What is the recommended method of adding a body force?

Asked by Feng Chen

Hi, All:

I am trying to add a customized body force over all particles, e.g.
add a buoyancy force to all particles Fbuoy=pi/6*d^3*rho*g
or a field magnetic force Fmag=beta*V_particle
basically need to loop over the particle list and then add the force for each time step
what is the recommended way of implementing this?

Thanks for any suggestions!

Question information

Language:
English Edit question
Status:
Solved
For:
ESyS-Particle Edit question
Assignee:
No assignee Edit question
Solved by:
Dion Weatherley
Solved:
Last query:
Last reply:
Revision history for this message
Best Dion Weatherley (d-weatherley) said : 		#1

Hi Feng,

To add a new body force, you will need to add an extra interaction group to the C++ source code. For the two interactions you mentioned, I'd suggest you use the existing Body Force Interaction Group implementation as a guide. There are a number of steps required to add a new body force interaction group:

1) implement a new subclass of BodyForceGroup and a new associated InteractionGroupParams (IGP) class

Start by looking at Model/BodyForceGroup.* to see how Gravity is implemented. The actual force calculation (for any given particle) is implemented in the calcForces() subroutine of your new class.

2) Update the LatticeMaster and SubLattice classes to add createMyBodyForceGroup(..) subroutines

Edits will be required in Parallel/LatticeMaster.* and Parallel/SubLattice.*. Search for "Gravity" and use the GravityIG implementation as a guide.

3) Add a Python wrapper interface for your new Interaction Group

You will need to add a wrapper to Python/esys/lsm/InteractionsPrmsPy.*. Again, search for "Gravity" as a guide for what to add.

After that you'll need to re-compile and install ESyS-Particle to start testing your implementation.

I realise this is a very brief set of instructions and it can be challenging to implement new interactions the first time. Perhaps Steffen Abe's tutorial on adding a new particle-pair interaction group might also be helpful:
https://twiki.esscc.uq.edu.au/pub/ESSCC/DocumentationAndPresentations/HowToAddInteraction.pdf
but be careful that this tutorial is designed for particle-pair interactions and not body forces.

Good luck and please ask more questions if you encounter difficulties.

Cheers,

Dion.

Revision history for this message
Feng Chen (fchen3-gmail) said : 		#2

Thanks a lot for such a detailed answer! Dr. Dion.

I think this is exactly the way I am looking for, currently I have only one question, in the second step:

"2) Update the LatticeMaster and SubLattice classes to add createMyBodyForceGroup(..) subroutines"

I understand ESys-particle is using the domain decomposition parallel algorithm, does this require handle the sub-domain boundary communication? Especially, when I need to find whether a particle falls within the magnetic field or not (Only the particles in the magnetic field need the F_mag). I searched previous posts, this could be similar to:

https://answers.launchpad.net/esys-particle/+question/88894

BTW, there might be an initiative for implementing the domain decomposition using GPU, I would also spend some time on this if this is really necessary, but this is just a thinking at this moment.

Thanks again for your instructions.

Revision history for this message
Feng Chen (fchen3-gmail) said : 		#3

Thanks Dion Weatherley, that solved my question.

Revision history for this message
Dion Weatherley (d-weatherley) said : 		#4

Hi Feng,

There is no need to worry about subdomain boundary communications when implementing new interaction groups. The additions to LatticeMaster and SubLattice are simply so the Master and Slave processes create the correct type of interaction group during initialisation of a simulation. Thereafter the subdomain boundary communications are handled via the parallel particle array and parallel interaction storage objects of each Slave process. This is done in a seamless way so the developer of a new interaction group needs only focus on implementing the details of how to calculate forces.

If your magnetic field interaction is dependent on spatial position, you will need to include a check in your calcForces() subroutine to ensure the current particle of interest is within the magnetic field's domain.

Regarding GPUs and ESyS-Particle, I recently attended a workshop on GPU programming and also had some lengthy discussions with various people more knowledgeable than I. It is my impression that almost all of ESyS-Particle's C++ engine would need to be re-written to gain any performance boost from GPU arrays. In ESyS-Particle, subdomain decomposition was written from scratch and doesn't use standard libraries or algorithms. Given the way it is currently implemented, and the memory footprint associated with that, I cannot see how it could be easily converted to execute on GPU arrays.

Although supercomputers including large GPU arrays may be prevalent in a few years, currently large clusters seem to be more common at present. ESyS-Particle works remarkably well as-is on cluster supercomputers. Consequently it is difficult to justify effort to port ESyS-Particle to GPU arrays, unless of course someone has the funding and the desire to do so...I would be happy to help by way of technical support in such a case.

Cheers,

Dion.

Revision history for this message
Feng Chen (fchen3-gmail) said : 		#5

Dr. Dion Weatherley,

Thanks a lot for sharing your experience about the parallel issue about ESys-particle, this gave me a big picture on how much effort would GPU parallelism require, at this moment I will first try to implement the custom body force engine and then if I really got a support for GPU in the future I will let you know.

My personal opinion(correct me if I am wrong), currently the limitation of DEM to the real engineering scale problem is still
(1) the number of particles due to the computational resource required for contact detection
(2) the number of time steps needed for explicit time integration, which is a function of dt=alpha*sqrt(m/k)
I think there seems to be several directions for future DEM:
1. increase the computational power, e.g. through super*super*... computers, however this imposes a limitation on hardware
2. use the scaling law, however whether the scaled model still conserve the same desired physical property will become a problem
3. use a implicit time integration scheme, like the DDA(Discontinuous Deformation Analysis) method,

I am also very curious about your opinion about these questions, which (or some way else) could be a better direction?

Best Regards,

Feng Chen