where are the shared vertices

Asked by Lizao (Larry) Li

I want to find out which CPU actually has the value at the shared vertex in parallel.
Example:
----------------------- test .py ----------------------------------------------
import numpy as np
from dolfin import *
mesh = UnitSquare(2, 2)
V = FunctionSpace(mesh, "CG", 1)
f = Expression("x[0]*x[0]")
u = interpolate(f, V)
print("#CPU #{}############\n{}\n{}\n{}\n{}".format(MPI.process_number(), str(u.vector().array()), str(mesh.parallel_data().global_entity_indices(0).array()), str(mesh.coordinates()), mesh.parallel_data().shared_vertices()))
-----------------------------------------------------------------------
In the above short code, "mesh.parallel_data().global_entity_indices(0).array()" tells you the map from global vertices to local vertices. But the output is always longer than len(f.vector().array()) on each CPU because it also tells you the map for shared vertices as well. "mesh.parallel_data().shared_vertices()" tells you which vertices are shared, but it does not tell you which CPU actually has the value of f at the shared vertices.

Output of "mpirun -np 3 python test.py":
----------------------- output ----------------------------------
#CPU #0############
[ 1. 0.25]
[0 1 2 4 5]
[[ 0. 0. ]
 [ 0.5 0. ]
 [ 1. 0. ]
 [ 0.5 0.5]
 [ 1. 0.5]]
{0: array([1], dtype=uint32), 4: array([1, 2], dtype=uint32), 5: array([2], dtype=uint32)}
#CPU #2############
[ 0. 1. 1. 0.25]
[7 8 3 4 5]
[[ 0.5 1. ]
 [ 1. 1. ]
 [ 0. 0.5]
 [ 0.5 0.5]
 [ 1. 0.5]]
{3: array([1], dtype=uint32), 4: array([0, 1], dtype=uint32), 5: array([0], dtype=uint32), 7: array([1], dtype=uint32)}
#CPU #1############
[ 0. 0.25 0. ]
[3 4 0 6 7]
[[ 0. 0.5]
 [ 0.5 0.5]
 [ 0. 0. ]
 [ 0. 1. ]
 [ 0.5 1. ]]
{0: array([0], dtype=uint32), 3: array([2], dtype=uint32), 4: array([0, 2], dtype=uint32), 7: array([2], dtype=uint32)}
--------------------------------------------------------------------------------------------------
I know it is possilbe to figure that out from the above output through quite some work. For example from the output above, CPU1 has global_to_local map for [0 1 2 4 5], and [0 4 5] are shared. f only has 2 values. So I know f.vector() must be the value of f at vertex 1 and 2. But for CPU2, this approach does not work any more and I have to look at the vertex coordinates and so on.

I am wondering if there is an easy to archive this.

Question information

Language:
English Edit question
Status:
Solved
For:
DOLFIN Edit question
Assignee:
No assignee Edit question
Solved by:
Garth Wells
Solved:
Last query:
Last reply:
Revision history for this message
Best Garth Wells (garth-wells) said :
#1

Your're mixing concepts.

For shared vertices, there is not owner. For vertices on a boundary, both processes have the vertex.

Dofs are owned by a process, and GenericDofMap has functions for working with this, e.g.

    GenericDofMap::off_process_owner()

returns a map (dofs on a process boundary that are not owned) -> (owner process index).

Revision history for this message
Lizao (Larry) Li (creatorlarryli) said :
#2

Thank you very much.

Revision history for this message
Lizao (Larry) Li (creatorlarryli) said :
#3

Thanks Garth Wells, that solved my question.