DOLFIN

MPI: seg fault 11 when using more than 1 node

Asked by Damiaan

Hello,

  I'm running some of the problems available in WIP (e.g. the channel problem).

I use the following submission script:
---
#!/bin/bash
# MOAB/Torque submission script for SciNet GPC
#
#PBS -l nodes=2:ppn=8,walltime=2:00:00

# DIRECTORY TO RUN - $PBS_O_WORKDIR is directory job was submitted from
cd $PBS_O_WORKDIR

# EXECUTION COMMAND; -np = nodes*ppn
mpirun -np 16 python ns channel ipcs_opt segregated=True &> $PBS_O_WORKDIR/log_`date +%y_%m_%d_%S`
----

So 8 cores per node; 2 nodes.

It then crashes with:

[gpc-f124n023:08782] *** Process received signal ***
[gpc-f124n023:08782] Signal: Segmentation fault (11)
[gpc-f124n023:08782] Signal code: Address not mapped (1)
[gpc-f124n023:08782] Failing at address: 0x31d8450

If I run it on 1 node, 2,4 or 8 cores, then it runs just fine.

Any suggestions?

Question information

Language:
English Edit question
Status:
Solved
For:
DOLFIN Edit question
Assignee:
No assignee Edit question
Solved by:
Damiaan
Solved:
Last query:
Last reply:
Revision history for this message
Kent-Andre Mardal (kent-and) said : 		#1

Could you provide us with some more info? E.g. the error log?

Kent

Revision history for this message
Damiaan (dhabets) said : 		#2

Kent, if this isn't what you were looking for, then let me know.

PATH variable is:
/home/s/steinman/dhabets/Root/FEniCS/bin:/scinet/gpc/mpi/openmpi/otpo/1.0.0//bin:/scinet/gpc/mpi/openmpi/1.4.4-intel-v12.1/bin:/home/s/steinman/dhabets/Root/FEniCS/bin:/home/s/steinman/dhabets/Root/FEniCS/bin:/scinet/gpc/tools/Python/Python272-shared/bin:/scinet/gpc/compilers/gcc-4.6.1/bin:/scinet/gpc/intel/ics/composer_xe_2011_sp1.9.293/bin/intel64:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/usr/lpp/mmfs/bin:/opt/torque/bin:/opt/torque/sbin:/usr/lpp/mmfs/bin:/opt/torque/bin:/opt/torque/sbin:/scinet/gpc/tools/editors/nano/nano-2.2.4/bin:/scinet/gpc/bin6:/home/s/steinman/dhabets/Root/bin:/home/s/steinman/dhabets/Root/bin:/usr/lpp/mmfs/bin:/opt/torque/bin:/opt/torque/sbin:/home/s/steinman/dhabets/Root/bin

Running main with args ['channel', 'ipcs_opt', 'segregated=True']
Making problem...
Process 0: Number of global vertices: 64
Process 0: Number of global cells: 98
Process 2: Compute dual graph.
Process 4: Compute dual graph.
Process 6: Compute dual graph.
Process 1: Compute dual graph.
Process 8: Compute dual graph.
Process 3: Compute dual graph.
Process 5: Compute dual graph.
Process 7: Compute dual graph.
Process 10: Compute dual graph.
Process 12: Compute dual graph.
Process 9: Compute dual graph.
Process 11: Compute dual graph.
Process 0: Compute dual graph.
Process 14: Compute dual graph.
Process 13: Compute dual graph.
Process 15: Compute dual graph.
Process 12: Compute local cell-cell connections
Process 12: Time to build vertex-cell connectivity map: 1.09673e-05
Process 12: Time to build local dual graph: : 3.8147e-06
Process 12: Finished computing local cell-cell connections
Process 12: Preparing data to to send off-process.
Process 2: Compute local cell-cell connections
Process 2: Time to build vertex-cell connectivity map: 8.10623e-06
Process 2: Time to build local dual graph: : 4.05312e-06
Process 2: Finished computing local cell-cell connections
Process 2: Preparing data to to send off-process.
Process 4: Compute local cell-cell connections
Process 4: Time to build vertex-cell connectivity map: 9.77516e-06
Process 4: Time to build local dual graph: : 5.00679e-06
Process 4: Finished computing local cell-cell connections
Process 4: Preparing data to to send off-process.
Process 8: Compute local cell-cell connections
Process 8: Time to build vertex-cell connectivity map: 1.00136e-05
Process 8: Time to build local dual graph: : 5.00679e-06
Process 8: Finished computing local cell-cell connections
Process 8: Preparing data to to send off-process.
Process 6: Compute local cell-cell connections
Process 6: Time to build vertex-cell connectivity map: 1.21593e-05
Process 6: Time to build local dual graph: : 5.00679e-06
Process 6: Finished computing local cell-cell connections
Process 6: Preparing data to to send off-process.
Process 10: Compute local cell-cell connections
Process 10: Time to build vertex-cell connectivity map: 1.00136e-05
Process 10: Time to build local dual graph: : 5.00679e-06
Process 10: Finished computing local cell-cell connections
Process 10: Preparing data to to send off-process.
Process 0: Compute local cell-cell connections
Process 0: Time to build vertex-cell connectivity map: 1.09673e-05
Process 0: Time to build local dual graph: : 5.00679e-06
Process 0: Finished computing local cell-cell connections
Process 0: Preparing data to to send off-process.
Process 14: Compute local cell-cell connections
Process 14: Time to build vertex-cell connectivity map: 9.05991e-06
Process 14: Time to build local dual graph: : 3.8147e-06
Process 14: Finished computing local cell-cell connections
Process 14: Preparing data to to send off-process.
Process 1: Compute local cell-cell connections
Process 1: Time to build vertex-cell connectivity map: 1.19209e-05
Process 1: Time to build local dual graph: : 4.05312e-06
Process 1: Finished computing local cell-cell connections
Process 1: Preparing data to to send off-process.
Process 3: Compute local cell-cell connections
Process 3: Time to build vertex-cell connectivity map: 9.05991e-06
Process 3: Time to build local dual graph: : 5.00679e-06
Process 3: Finished computing local cell-cell connections
Process 3: Preparing data to to send off-process.
Process 5: Compute local cell-cell connections
Process 5: Time to build vertex-cell connectivity map: 1.12057e-05
Process 5: Time to build local dual graph: : 5.00679e-06
Process 5: Finished computing local cell-cell connections
Process 5: Preparing data to to send off-process.
Process 7: Compute local cell-cell connections
Process 7: Time to build vertex-cell connectivity map: 8.82149e-06
Process 7: Time to build local dual graph: : 5.00679e-06
Process 7: Finished computing local cell-cell connections
Process 7: Preparing data to to send off-process.
Process 9: Compute local cell-cell connections
Process 9: Time to build vertex-cell connectivity map: 9.05991e-06
Process 9: Time to build local dual graph: : 4.05312e-06
Process 9: Finished computing local cell-cell connections
Process 9: Preparing data to to send off-process.
Process 11: Compute local cell-cell connections
Process 11: Time to build vertex-cell connectivity map: 1.00136e-05
Process 11: Time to build local dual graph: : 4.05312e-06
Process 11: Finished computing local cell-cell connections
Process 11: Preparing data to to send off-process.
Process 13: Compute local cell-cell connections
Process 13: Time to build vertex-cell connectivity map: 1.09673e-05
Process 13: Time to build local dual graph: : 3.8147e-06
Process 13: Finished computing local cell-cell connections
Process 13: Preparing data to to send off-process.
Process 15: Compute local cell-cell connections
Process 15: Time to build vertex-cell connectivity map: 1.19209e-05
Process 15: Time to build local dual graph: : 5.00679e-06
Process 15: Finished computing local cell-cell connections
Process 15: Preparing data to to send off-process.
Process 0: Compute graph ghost edges.
Process 0: Time to build local boundary vertex-cell connectivity map: 1.21593e-05
Process 0: Time to build ghost boundary vertex-cell connectivity map: 0
Process 0: Time to build ghost dual graph: : 3.09944e-06
Process 0: Time to build local boundary vertex-cell connectivity map: 5.96046e-06
Process 0: Time to build ghost boundary vertex-cell connectivity map: 5.96046e-06
Process 0: Time to build ghost dual graph: : 5.00679e-06
Process 0: Time to build local boundary vertex-cell connectivity map: 8.10623e-06
Process 0: Time to build ghost boundary vertex-cell connectivity map: 5.00679e-06
Process 0: Time to build ghost dual graph: : 5.00679e-06
Process 0: Time to build local boundary vertex-cell connectivity map: 5.96046e-06
Process 0: Time to build ghost boundary vertex-cell connectivity map: 5.00679e-06
Process 0: Time to build ghost dual graph: : 1.90735e-06
Process 0: Time to build local boundary vertex-cell connectivity map: 5.96046e-06
Process 0: Time to build ghost boundary vertex-cell connectivity map: 5.96046e-06
Process 0: Time to build ghost dual graph: : 9.53674e-07
Process 0: Time to build local boundary vertex-cell connectivity map: 6.19888e-06
Process 0: Time to build ghost boundary vertex-cell connectivity map: 5.96046e-06
Process 0: Time to build ghost dual graph: : 1.19209e-06
Process 0: Time to build local boundary vertex-cell connectivity map: 5.96046e-06
Process 0: Time to build ghost boundary vertex-cell connectivity map: 5.96046e-06
Process 0: Time to build ghost dual graph: : 9.53674e-07
Process 0: Time to build local boundary vertex-cell connectivity map: 5.96046e-06
Process 0: Time to build ghost boundary vertex-cell connectivity map: 5.96046e-06
Process 0: Time to build ghost dual graph: : 9.53674e-07
Process 0: Time to build local boundary vertex-cell connectivity map: 5.96046e-06
Process 0: Time to build ghost boundary vertex-cell connectivity map: 5.00679e-06
Process 0: Time to build ghost dual graph: : 9.53674e-07
Process 0: Time to build local boundary vertex-cell connectivity map: 5.96046e-06
Process 0: Time to build ghost boundary vertex-cell connectivity map: 5.00679e-06
Process 0: Time to build ghost dual graph: : 9.53674e-07
Process 0: Time to build local boundary vertex-cell connectivity map: 8.10623e-06
Process 0: Time to build ghost boundary vertex-cell connectivity map: 4.76837e-06
Process 0: Time to build ghost dual graph: : 9.53674e-07
Process 0: Time to build local boundary vertex-cell connectivity map: 7.15256e-06
Process 0: Time to build ghost boundary vertex-cell connectivity map: 6.19888e-06
Process 0: Time to build ghost dual graph: : 9.53674e-07
Process 0: Time to build local boundary vertex-cell connectivity map: 6.19888e-06
Process 0: Time to build ghost boundary vertex-cell connectivity map: 5.96046e-06
Process 0: Time to build ghost dual graph: : 9.53674e-07
Process 0: Time to build local boundary vertex-cell connectivity map: 6.91414e-06
Process 0: Time to build ghost boundary vertex-cell connectivity map: 5.96046e-06
Process 0: Time to build ghost dual graph: : 9.53674e-07
Process 0: Time to build local boundary vertex-cell connectivity map: 6.19888e-06
Process 0: Time to build ghost boundary vertex-cell connectivity map: 5.96046e-06
Process 0: Time to build ghost dual graph: : 9.53674e-07
Process 0: Time to build local boundary vertex-cell connectivity map: 7.15256e-06
Process 0: Time to build ghost boundary vertex-cell connectivity map: 5.96046e-06
Process 0: Time to build ghost dual graph: : 1.19209e-06
Process 0: Finish compute graph ghost edges.
Process 0: End compute dual graph.
Process 0: Start to compute partitions using SCOTCH
Process 4: Compute graph ghost edges.
Process 4: Time to build local boundary vertex-cell connectivity map: 1.50204e-05
Process 4: Time to build ghost boundary vertex-cell connectivity map: 9.05991e-06
Process 4: Time to build ghost dual graph: : 5.00679e-06
Process 4: Time to build local boundary vertex-cell connectivity map: 5.00679e-06
Process 4: Time to build ghost boundary vertex-cell connectivity map: 5.96046e-06
Process 4: Time to build ghost dual graph: : 4.05312e-06
Process 4: Time to build local boundary vertex-cell connectivity map: 5.00679e-06
Process 4: Time to build ghost boundary vertex-cell connectivity map: 5.00679e-06
Process 4: Time to build ghost dual graph: : 4.05312e-06
Process 4: Time to build local boundary vertex-cell connectivity map: 7.15256e-06
Process 4: Time to build ghost boundary vertex-cell connectivity map: 5.00679e-06
Process 4: Time to build ghost dual graph: : 1.90735e-06
Process 4: Time to build local boundary vertex-cell connectivity map: 5.00679e-06
Process 4: Time to build ghost boundary vertex-cell connectivity map: 0
Process 4: Time to build ghost dual graph: : 9.53674e-07
Process 4: Time to build local boundary vertex-cell connectivity map: 5.00679e-06
Process 4: Time to build ghost boundary vertex-cell connectivity map: 4.05312e-06
Process 4: Time to build ghost dual graph: : 1.90735e-06
Process 4: Time to build local boundary vertex-cell connectivity map: 5.96046e-06
Process 4: Time to build ghost boundary vertex-cell connectivity map: 5.00679e-06
Process 4: Time to build ghost dual graph: : 4.05312e-06
Process 4: Time to build local boundary vertex-cell connectivity map: 5.00679e-06
Process 4: Time to build ghost boundary vertex-cell connectivity map: 4.05312e-06
Process 4: Time to build ghost dual graph: : 2.86102e-06
Process 4: Time to build local boundary vertex-cell connectivity map: 5.00679e-06
Process 4: Time to build ghost boundary vertex-cell connectivity map: 5.96046e-06
Process 4: Time to build ghost dual graph: : 9.53674e-07
Process 4: Time to build local boundary vertex-cell connectivity map: 5.96046e-06
Process 4: Time to build ghost boundary vertex-cell connectivity map: 5.96046e-06
Process 4: Time to build ghost dual graph: : 9.53674e-07
Process 4: Time to build local boundary vertex-cell connectivity map: 5.00679e-06
Process 4: Time to build ghost boundary vertex-cell connectivity map: 5.96046e-06
Process 4: Time to build ghost dual graph: : 9.53674e-07
Process 4: Time to build local boundary vertex-cell connectivity map: 5.00679e-06
Process 4: Time to build ghost boundary vertex-cell connectivity map: 6.19888e-06
Process 4: Time to build ghost dual graph: : 1.19209e-06
Process 4: Time to build local boundary vertex-cell connectivity map: 5.00679e-06
Process 4: Time to build ghost boundary vertex-cell connectivity map: 5.00679e-06
Process 4: Time to build ghost dual graph: : 1.19209e-06
Process 4: Time to build local boundary vertex-cell connectivity map: 5.00679e-06
Process 4: Time to build ghost boundary vertex-cell connectivity map: 4.76837e-06
Process 4: Time to build ghost dual graph: : 1.19209e-06
Process 4: Time to build local boundary vertex-cell connectivity map: 5.96046e-06
Process 4: Time to build ghost boundary vertex-cell connectivity map: 5.96046e-06
Process 4: Time to build ghost dual graph: : 0
Process 4: Time to build local boundary vertex-cell connectivity map: 5.00679e-06
Process 4: Time to build ghost boundary vertex-cell connectivity map: 6.19888e-06
Process 4: Time to build ghost dual graph: : 1.19209e-06
Process 4: Finish compute graph ghost edges.
Process 4: End compute dual graph.
Process 4: Start to compute partitions using SCOTCH
Process 8: Compute graph ghost edges.
Process 8: Time to build local boundary vertex-cell connectivity map: 5.60284e-05
Process 8: Time to build ghost boundary vertex-cell connectivity map: 6.91414e-06
Process 8: Time to build ghost dual graph: : 1.21593e-05
Process 8: Time to build local boundary vertex-cell connectivity map: 4.05312e-06
Process 8: Time to build ghost boundary vertex-cell connectivity map: 5.00679e-06
Process 8: Time to build ghost dual graph: : 9.53674e-07
Process 8: Time to build local boundary vertex-cell connectivity map: 3.09944e-06
Process 8: Time to build ghost boundary vertex-cell connectivity map: 3.09944e-06
Process 8: Time to build ghost dual graph: : 9.53674e-07
Process 8: Time to build local boundary vertex-cell connectivity map: 3.09944e-06
Process 8: Time to build ghost boundary vertex-cell connectivity map: 3.09944e-06
Process 8: Time to build ghost dual graph: : 0
Process 8: Time to build local boundary vertex-cell connectivity map: 3.09944e-06
Process 8: Time to build ghost boundary vertex-cell connectivity map: 4.05312e-06
Process 8: Time to build ghost dual graph: : 9.53674e-07
Process 8: Time to build local boundary vertex-cell connectivity map: 3.09944e-06
Process 8: Time to build ghost boundary vertex-cell connectivity map: 2.86102e-06
Process 8: Time to build ghost dual graph: : 4.05312e-06
Process 8: Time to build local boundary vertex-cell connectivity map: 4.05312e-06
Process 8: Time to build ghost boundary vertex-cell connectivity map: 2.86102e-06
Process 8: Time to build ghost dual graph: : 3.09944e-06
Process 8: Time to build local boundary vertex-cell connectivity map: 3.09944e-06
Process 8: Time to build ghost boundary vertex-cell connectivity map: 2.86102e-06
Process 8: Time to build ghost dual graph: : 2.14577e-06
Process 8: Time to build local boundary vertex-cell connectivity map: 2.86102e-06
Process 8: Time to build ghost boundary vertex-cell connectivity map: 9.53674e-07
Process 8: Time to build ghost dual graph: : 9.53674e-07
Process 8: Time to build local boundary vertex-cell connectivity map: 3.09944e-06
Process 8: Time to build ghost boundary vertex-cell connectivity map: 4.05312e-06
Process 8: Time to build ghost dual graph: : 0
Process 8: Time to build local boundary vertex-cell connectivity map: 2.86102e-06
Process 8: Time to build ghost boundary vertex-cell connectivity map: 3.09944e-06
Process 8: Time to build ghost dual graph: : 3.09944e-06
Process 8: Time to build local boundary vertex-cell connectivity map: 2.86102e-06
Process 8: Time to build ghost boundary vertex-cell connectivity map: 3.09944e-06
Process 8: Time to build ghost dual graph: : 1.90735e-06
Process 8: Time to build local boundary vertex-cell connectivity map: 4.05312e-06
Process 8: Time to build ghost boundary vertex-cell connectivity map: 2.86102e-06
Process 8: Time to build ghost dual graph: : 0
Process 8: Time to build local boundary vertex-cell connectivity map: 2.86102e-06
Process 8: Time to build ghost boundary vertex-cell connectivity map: 4.05312e-06
Process 8: Time to build ghost dual graph: : 9.53674e-07
Process 8: Time to build local boundary vertex-cell connectivity map: 3.09944e-06
Process 8: Time to build ghost boundary vertex-cell connectivity map: 2.86102e-06
Process 8: Time to build ghost dual graph: : 1.19209e-06
Process 8: Time to build local boundary vertex-cell connectivity map: 3.8147e-06
Process 8: Time to build ghost boundary vertex-cell connectivity map: 3.09944e-06
Process 8: Time to build ghost dual graph: : 9.53674e-07
Process 8: Finish compute graph ghost edges.
Process 8: End compute dual graph.
Process 8: Start to compute partitions using SCOTCH
Process 6: Compute graph ghost edges.
Process 6: Time to build local boundary vertex-cell connectivity map: 1.00136e-05
Process 6: Time to build ghost boundary vertex-cell connectivity map: 6.19888e-06
Process 6: Time to build ghost dual graph: : 1.90735e-06
Process 6: Time to build local boundary vertex-cell connectivity map: 4.05312e-06
Process 6: Time to build ghost boundary vertex-cell connectivity map: 4.05312e-06
Process 6: Time to build ghost dual graph: : 0
Process 6: Time to build local boundary vertex-cell connectivity map: 4.05312e-06
Process 6: Time to build ghost boundary vertex-cell connectivity map: 3.09944e-06
Process 6: Time to build ghost dual graph: : 1.19209e-06
Process 6: Time to build local boundary vertex-cell connectivity map: 2.86102e-06
Process 6: Time to build ghost boundary vertex-cell connectivity map: 3.09944e-06
Process 6: Time to build ghost dual graph: : 4.05312e-06
Process 6: Time to build local boundary vertex-cell connectivity map: 2.86102e-06
Process 6: Time to build ghost boundary vertex-cell connectivity map: 3.09944e-06
Process 6: Time to build ghost dual graph: : 3.09944e-06
Process 6: Time to build local boundary vertex-cell connectivity map: 2.86102e-06
Process 6: Time to build ghost boundary vertex-cell connectivity map: 3.09944e-06
Process 6: Time to build ghost dual graph: : 1.90735e-06
Process 6: Time to build local boundary vertex-cell connectivity map: 4.05312e-06
Process 6: Time to build ghost boundary vertex-cell connectivity map: 0
Process 6: Time to build ghost dual graph: : 0
Process 6: Time to build local boundary vertex-cell connectivity map: 4.05312e-06
Process 6: Time to build ghost boundary vertex-cell connectivity map: 2.86102e-06
Process 6: Time to build ghost dual graph: : 9.53674e-07
Process 6: Time to build local boundary vertex-cell connectivity map: 3.8147e-06
Process 6: Time to build ghost boundary vertex-cell connectivity map: 2.14577e-06
Process 6: Time to build ghost dual graph: : 2.86102e-06
Process 6: Time to build local boundary vertex-cell connectivity map: 3.09944e-06
Process 6: Time to build ghost boundary vertex-cell connectivity map: 3.09944e-06
Process 6: Time to build ghost dual graph: : 1.90735e-06
Process 6: Time to build local boundary vertex-cell connectivity map: 3.09944e-06
Process 6: Time to build ghost boundary vertex-cell connectivity map: 2.86102e-06
Process 6: Time to build ghost dual graph: : 0
Process 6: Time to build local boundary vertex-cell connectivity map: 2.86102e-06
Process 6: Time to build ghost boundary vertex-cell connectivity map: 4.05312e-06
Process 6: Time to build ghost dual graph: : 0
Process 6: Time to build local boundary vertex-cell connectivity map: 3.09944e-06
Process 6: Time to build ghost boundary vertex-cell connectivity map: 3.8147e-06
Process 6: Time to build ghost dual graph: : 0
Process 6: Time to build local boundary vertex-cell connectivity map: 2.86102e-06
Process 6: Time to build ghost boundary vertex-cell connectivity map: 4.05312e-06
Process 6: Time to build ghost dual graph: : 9.53674e-07
Process 6: Time to build local boundary vertex-cell connectivity map: 3.09944e-06
Process 6: Time to build ghost boundary vertex-cell connectivity map: 4.05312e-06
Process 6: Time to build ghost dual graph: : 0
Process 6: Time to build local boundary vertex-cell connectivity map: 2.86102e-06
Process 6: Time to build ghost boundary vertex-cell connectivity map: 4.05312e-06
Process 6: Time to build ghost dual graph: : 9.53674e-07
Process 6: Finish compute graph ghost edges.
Process 6: End compute dual graph.
Process 6: Start to compute partitions using SCOTCH
Process 1: Compute graph ghost edges.
Process 1: Time to build local boundary vertex-cell connectivity map: 7.70092e-05
Process 1: Time to build ghost boundary vertex-cell connectivity map: 9.05991e-06
Process 1: Time to build ghost dual graph: : 9.05991e-06
Process 1: Time to build local boundary vertex-cell connectivity map: 7.86781e-06
Process 1: Time to build ghost boundary vertex-cell connectivity map: 0
Process 1: Time to build ghost dual graph: : 9.53674e-07
Process 1: Time to build local boundary vertex-cell connectivity map: 5.00679e-06
Process 1: Time to build ghost boundary vertex-cell connectivity map: 5.00679e-06
Process 1: Time to build ghost dual graph: : 9.53674e-07
Process 1: Time to build local boundary vertex-cell connectivity map: 5.96046e-06
Process 1: Time to build ghost boundary vertex-cell connectivity map: 5.00679e-06
Process 1: Time to build ghost dual graph: : 5.00679e-06
Process 1: Time to build local boundary vertex-cell connectivity map: 5.96046e-06
Process 1: Time to build ghost boundary vertex-cell connectivity map: 5.00679e-06
Process 1: Time to build ghost dual graph: : 3.09944e-06
Process 1: Time to build local boundary vertex-cell connectivity map: 5.96046e-06
Process 1: Time to build ghost boundary vertex-cell connectivity map: 4.76837e-06
Process 1: Time to build ghost dual graph: : 1.19209e-06
Process 1: Time to build local boundary vertex-cell connectivity map: 8.82149e-06
Process 1: Time to build ghost boundary vertex-cell connectivity map: 6.91414e-06
Process 1: Time to build ghost dual graph: : 9.53674e-07
Process 1: Time to build local boundary vertex-cell connectivity map: 5.96046e-06
Process 1: Time to build ghost boundary vertex-cell connectivity map: 8.10623e-06
Process 1: Time to build ghost dual graph: : 9.53674e-07
Process 1: Time to build local boundary vertex-cell connectivity map: 7.86781e-06
Process 1: Time to build ghost boundary vertex-cell connectivity map: 5.96046e-06
Process 1: Time to build ghost dual graph: : 9.53674e-07
Process 1: Time to build local boundary vertex-cell connectivity map: 5.96046e-06
Process 1: Time to build ghost boundary vertex-cell connectivity map: 6.91414e-06
Process 1: Time to build ghost dual graph: : 1.90735e-06
Process 1: Time to build local boundary vertex-cell connectivity map: 7.15256e-06
Process 1: Time to build ghost boundary vertex-cell connectivity map: 5.96046e-06
Process 1: Time to build ghost dual graph: : 9.53674e-07
Process 1: Time to build local boundary vertex-cell connectivity map: 5.96046e-06
Process 1: Time to build ghost boundary vertex-cell connectivity map: 5.96046e-06
Process 1: Time to build ghost dual graph: : 1.90735e-06
Process 1: Time to build local boundary vertex-cell connectivity map: 7.86781e-06
Process 1: Time to build ghost boundary vertex-cell connectivity map: 8.10623e-06
Process 1: Time to build ghost dual graph: : 9.53674e-07
Process 1: Time to build local boundary vertex-cell connectivity map: 6.19888e-06
Process 1: Time to build ghost boundary vertex-cell connectivity map: 6.91414e-06
Process 1: Time to build ghost dual graph: : 9.53674e-07
Process 1: Time to build local boundary vertex-cell connectivity map: 5.96046e-06
Process 1: Time to build ghost boundary vertex-cell connectivity map: 5.96046e-06
Process 1: Time to build ghost dual graph: : 9.53674e-07
Process 1: Time to build local boundary vertex-cell connectivity map: 5.96046e-06
Process 1: Time to build ghost boundary vertex-cell connectivity map: 5.96046e-06
Process 1: Time to build ghost dual graph: : 1.90735e-06
Process 1: Finish compute graph ghost edges.
Process 1: End compute dual graph.
Process 1: Start to compute partitions using SCOTCH
Process 11: Compute graph ghost edges.
Process 11: Time to build local boundary vertex-cell connectivity map: 1.19209e-05
Process 11: Time to build ghost boundary vertex-cell connectivity map: 8.10623e-06
Process 11: Time to build ghost dual graph: : 2.86102e-06
Process 11: Time to build local boundary vertex-cell connectivity map: 5.96046e-06
Process 11: Time to build ghost boundary vertex-cell connectivity map: 6.19888e-06
Process 11: Time to build ghost dual graph: : 9.53674e-07
Process 11: Time to build local boundary vertex-cell connectivity map: 5.00679e-06
Process 11: Time to build ghost boundary vertex-cell connectivity map: 4.76837e-06
Process 11: Time to build ghost dual graph: : 1.19209e-06
Process 11: Time to build local boundary vertex-cell connectivity map: 7.86781e-06
Process 11: Time to build ghost boundary vertex-cell connectivity map: 6.91414e-06
Process 11: Time to build ghost dual graph: : 9.53674e-07
Process 11: Time to build local boundary vertex-cell connectivity map: 5.96046e-06
Process 11: Time to build ghost boundary vertex-cell connectivity map: 7.15256e-06
Process 11: Time to build ghost dual graph: : 9.53674e-07
Process 11: Time to build local boundary vertex-cell connectivity map: 7.86781e-06
Process 11: Time to build ghost boundary vertex-cell connectivity map: 5.00679e-06
Process 11: Time to build ghost dual graph: : 9.53674e-07
Process 11: Time to build local boundary vertex-cell connectivity map: 5.00679e-06
Process 11: Time to build ghost boundary vertex-cell connectivity map: 6.91414e-06
Process 11: Time to build ghost dual graph: : 9.53674e-07
Process 11: Time to build local boundary vertex-cell connectivity map: 5.00679e-06
Process 11: Time to build ghost boundary vertex-cell connectivity map: 5.96046e-06
Process 11: Time to build ghost dual graph: : 9.53674e-07
Process 11: Time to build local boundary vertex-cell connectivity map: 5.00679e-06
Process 11: Time to build ghost boundary vertex-cell connectivity map: 5.00679e-06
Process 11: Time to build ghost dual graph: : 3.8147e-06
Process 11: Time to build local boundary vertex-cell connectivity map: 5.00679e-06
Process 11: Time to build ghost boundary vertex-cell connectivity map: 7.15256e-06
Process 11: Time to build ghost dual graph: : 4.05312e-06
Process 11: Time to build local boundary vertex-cell connectivity map: 5.96046e-06
Process 11: Time to build ghost boundary vertex-cell connectivity map: 5.96046e-06
Process 11: Time to build ghost dual graph: : 3.09944e-06
Process 11: Time to build local boundary vertex-cell connectivity map: 5.96046e-06
Process 11: Time to build ghost boundary vertex-cell connectivity map: 0
Process 11: Time to build ghost dual graph: : 1.19209e-06
Process 11: Time to build local boundary vertex-cell connectivity map: 7.15256e-06
Process 11: Time to build ghost boundary vertex-cell connectivity map: 5.00679e-06
Process 11: Time to build ghost dual graph: : 1.90735e-06
Process 11: Time to build local boundary vertex-cell connectivity map: 5.00679e-06
Process 11: Time to build ghost boundary vertex-cell connectivity map: 5.00679e-06
Process 11: Time to build ghost dual graph: : 3.8147e-06
Process 11: Time to build local boundary vertex-cell connectivity map: 5.96046e-06
Process 11: Time to build ghost boundary vertex-cell connectivity map: 5.00679e-06
Process 11: Time to build ghost dual graph: : 2.86102e-06
Process 11: Time to build local boundary vertex-cell connectivity map: 6.19888e-06
Process 11: Time to build ghost boundary vertex-cell connectivity map: 6.19888e-06
Process 11: Time to build ghost dual graph: : 9.53674e-07
Process 11: Finish compute graph ghost edges.
Process 11: End compute dual graph.
Process 11: Start to compute partitions using SCOTCH
Process 15: Compute graph ghost edges.
Process 15: Time to build local boundary vertex-cell connectivity map: 4.19617e-05
Process 15: Time to build ghost boundary vertex-cell connectivity map: 6.91414e-06
Process 15: Time to build ghost dual graph: : 1.28746e-05
Process 15: Time to build local boundary vertex-cell connectivity map: 5.00679e-06
Process 15: Time to build ghost boundary vertex-cell connectivity map: 3.8147e-06
Process 15: Time to build ghost dual graph: : 9.53674e-07
Process 15: Time to build local boundary vertex-cell connectivity map: 2.86102e-06
Process 15: Time to build ghost boundary vertex-cell connectivity map: 4.05312e-06
Process 15: Time to build ghost dual graph: : 9.53674e-07
Process 15: Time to build local boundary vertex-cell connectivity map: 4.05312e-06
Process 15: Time to build ghost boundary vertex-cell connectivity map: 2.86102e-06
Process 15: Time to build ghost dual graph: : 1.19209e-06
Process 15: Time to build local boundary vertex-cell connectivity map: 2.86102e-06
Process 15: Time to build ghost boundary vertex-cell connectivity map: 3.09944e-06
Process 15: Time to build ghost dual graph: : 0
Process 15: Time to build local boundary vertex-cell connectivity map: 3.09944e-06
Process 15: Time to build ghost boundary vertex-cell connectivity map: 3.09944e-06
Process 15: Time to build ghost dual graph: : 9.53674e-07
Process 15: Time to build local boundary vertex-cell connectivity map: 4.05312e-06
Process 15: Time to build ghost boundary vertex-cell connectivity map: 3.09944e-06
Process 15: Time to build ghost dual graph: : 0
Process 15: Time to build local boundary vertex-cell connectivity map: 3.09944e-06
Process 15: Time to build ghost boundary vertex-cell connectivity map: 3.09944e-06
Process 15: Time to build ghost dual graph: : 9.53674e-07
Process 15: Time to build local boundary vertex-cell connectivity map: 3.09944e-06
Process 15: Time to build ghost boundary vertex-cell connectivity map: 2.86102e-06
Process 15: Time to build ghost dual graph: : 9.53674e-07
Process 15: Time to build local boundary vertex-cell connectivity map: 3.09944e-06
Process 15: Time to build ghost boundary vertex-cell connectivity map: 4.05312e-06
Process 15: Time to build ghost dual graph: : 9.53674e-07
Process 15: Time to build local boundary vertex-cell connectivity map: 2.86102e-06
Process 15: Time to build ghost boundary vertex-cell connectivity map: 4.05312e-06
Process 15: Time to build ghost dual graph: : 9.53674e-07
Process 15: Time to build local boundary vertex-cell connectivity map: 3.09944e-06
Process 15: Time to build ghost boundary vertex-cell connectivity map: 4.05312e-06
Process 15: Time to build ghost dual graph: : 9.53674e-07
Process 15: Time to build local boundary vertex-cell connectivity map: 3.09944e-06
Process 15: Time to build ghost boundary vertex-cell connectivity map: 2.86102e-06
Process 15: Time to build ghost dual graph: : 4.05312e-06
Process 15: Time to build local boundary vertex-cell connectivity map: 3.09944e-06
Process 15: Time to build ghost boundary vertex-cell connectivity map: 2.86102e-06
Process 15: Time to build ghost dual graph: : 3.09944e-06
Process 15: Time to build local boundary vertex-cell connectivity map: 2.86102e-06
Process 15: Time to build ghost boundary vertex-cell connectivity map: 3.09944e-06
Process 15: Time to build ghost dual graph: : 1.90735e-06
Process 15: Time to build local boundary vertex-cell connectivity map: 2.86102e-06
Process 15: Time to build ghost boundary vertex-cell connectivity map: 0
Process 15: Time to build ghost dual graph: : 9.53674e-07
Process 15: Finish compute graph ghost edges.
Process 15: End compute dual graph.
Process 15: Start to compute partitions using SCOTCH
Process 2: Compute graph ghost edges.
Process 2: Time to build local boundary vertex-cell connectivity map: 1.00136e-05
Process 2: Time to build ghost boundary vertex-cell connectivity map: 5.00679e-06
Process 2: Time to build ghost dual graph: : 5.96046e-06
Process 2: Time to build local boundary vertex-cell connectivity map: 3.09944e-06
Process 2: Time to build ghost boundary vertex-cell connectivity map: 5.00679e-06
Process 2: Time to build ghost dual graph: : 9.53674e-07
Process 2: Time to build local boundary vertex-cell connectivity map: 3.09944e-06
Process 2: Time to build ghost boundary vertex-cell connectivity map: 0
Process 2: Time to build ghost dual graph: : 9.53674e-07
Process 2: Time to build local boundary vertex-cell connectivity map: 2.86102e-06
Process 2: Time to build ghost boundary vertex-cell connectivity map: 3.09944e-06
Process 2: Time to build ghost dual graph: : 9.53674e-07
Process 2: Time to build local boundary vertex-cell connectivity map: 3.09944e-06
Process 2: Time to build ghost boundary vertex-cell connectivity map: 2.86102e-06
Process 2: Time to build ghost dual graph: : 3.09944e-06
Process 2: Time to build local boundary vertex-cell connectivity map: 3.09944e-06
Process 2: Time to build ghost boundary vertex-cell connectivity map: 2.86102e-06
Process 2: Time to build ghost dual graph: : 2.14577e-06
Process 2: Time to build local boundary vertex-cell connectivity map: 2.86102e-06
Process 2: Time to build ghost boundary vertex-cell connectivity map: 4.05312e-06
Process 2: Time to build ghost dual graph: : 9.53674e-07
Process 2: Time to build local boundary vertex-cell connectivity map: 3.09944e-06
Process 2: Time to build ghost boundary vertex-cell connectivity map: 2.86102e-06
Process 2: Time to build ghost dual graph: : 1.19209e-06
Process 2: Time to build local boundary vertex-cell connectivity map: 4.05312e-06
Process 2: Time to build ghost boundary vertex-cell connectivity map: 3.09944e-06
Process 2: Time to build ghost dual graph: : 9.53674e-07
Process 2: Time to build local boundary vertex-cell connectivity map: 3.09944e-06
Process 2: Time to build ghost boundary vertex-cell connectivity map: 2.86102e-06
Process 2: Time to build ghost dual graph: : 9.53674e-07
Process 2: Time to build local boundary vertex-cell connectivity map: 4.05312e-06
Process 2: Time to build ghost boundary vertex-cell connectivity map: 3.09944e-06
Process 2: Time to build ghost dual graph: : 9.53674e-07
Process 2: Time to build local boundary vertex-cell connectivity map: 2.86102e-06
Process 2: Time to build ghost boundary vertex-cell connectivity map: 4.05312e-06
Process 2: Time to build ghost dual graph: : 9.53674e-07
Process 2: Time to build local boundary vertex-cell connectivity map: 3.09944e-06
Process 2: Time to build ghost boundary vertex-cell connectivity map: 4.05312e-06
Process 2: Time to build ghost dual graph: : 1.19209e-06
Process 2: Time to build local boundary vertex-cell connectivity map: 2.86102e-06
Process 2: Time to build ghost boundary vertex-cell connectivity map: 4.05312e-06
Process 2: Time to build ghost dual graph: : 9.53674e-07
Process 2: Time to build local boundary vertex-cell connectivity map: 3.09944e-06
Process 2: Time to build ghost boundary vertex-cell connectivity map: 2.86102e-06
Process 2: Time to build ghost dual graph: : 1.19209e-06
Process 2: Time to build local boundary vertex-cell connectivity map: 2.86102e-06
Process 2: Time to build ghost boundary vertex-cell connectivity map: 3.09944e-06
Process 2: Time to build ghost dual graph: : 9.53674e-07
Process 2: Finish compute graph ghost edges.
Process 2: End compute dual graph.
Process 2: Start to compute partitions using SCOTCH
Process 3: Compute graph ghost edges.
Process 3: Time to build local boundary vertex-cell connectivity map: 8.10623e-06
Process 3: Time to build ghost boundary vertex-cell connectivity map: 4.05312e-06
Process 3: Time to build ghost dual graph: : 3.09944e-06
Process 3: Time to build local boundary vertex-cell connectivity map: 3.09944e-06
Process 3: Time to build ghost boundary vertex-cell connectivity map: 3.09944e-06
Process 3: Time to build ghost dual graph: : 3.8147e-06
Process 3: Time to build local boundary vertex-cell connectivity map: 4.05312e-06
Process 3: Time to build ghost boundary vertex-cell connectivity map: 3.09944e-06
Process 3: Time to build ghost dual graph: : 1.90735e-06
Process 3: Time to build local boundary vertex-cell connectivity map: 4.05312e-06
Process 3: Time to build ghost boundary vertex-cell connectivity map: 0
Process 3: Time to build ghost dual graph: : 9.53674e-07
Process 3: Time to build local boundary vertex-cell connectivity map: 3.09944e-06
Process 3: Time to build ghost boundary vertex-cell connectivity map: 2.86102e-06
Process 3: Time to build ghost dual graph: : 1.19209e-06
Process 3: Time to build local boundary vertex-cell connectivity map: 2.86102e-06
Process 3: Time to build ghost boundary vertex-cell connectivity map: 3.09944e-06
Process 3: Time to build ghost dual graph: : 1.90735e-06
Process 3: Time to build local boundary vertex-cell connectivity map: 2.86102e-06
Process 3: Time to build ghost boundary vertex-cell connectivity map: 4.05312e-06
Process 3: Time to build ghost dual graph: : 1.90735e-06
Process 3: Time to build local boundary vertex-cell connectivity map: 4.05312e-06
Process 3: Time to build ghost boundary vertex-cell connectivity map: 3.09944e-06
Process 3: Time to build ghost dual graph: : 9.53674e-07
Process 3: Time to build local boundary vertex-cell connectivity map: 3.8147e-06
Process 3: Time to build ghost boundary vertex-cell connectivity map: 3.09944e-06
Process 3: Time to build ghost dual graph: : 0
Process 3: Time to build local boundary vertex-cell connectivity map: 3.09944e-06
Process 3: Time to build ghost boundary vertex-cell connectivity map: 3.8147e-06
Process 3: Time to build ghost dual graph: : 1.19209e-06
Process 3: Time to build local boundary vertex-cell connectivity map: 2.86102e-06
Process 3: Time to build ghost boundary vertex-cell connectivity map: 3.09944e-06
Process 3: Time to build ghost dual graph: : 0
Process 3: Time to build local boundary vertex-cell connectivity map: 3.09944e-06
Process 3: Time to build ghost boundary vertex-cell connectivity map: 3.8147e-06
Process 3: Time to build ghost dual graph: : 1.19209e-06
Process 3: Time to build local boundary vertex-cell connectivity map: 2.86102e-06
Process 3: Time to build ghost boundary vertex-cell connectivity map: 3.09944e-06
Process 3: Time to build ghost dual graph: : 0
Process 3: Time to build local boundary vertex-cell connectivity map: 3.09944e-06
Process 3: Time to build ghost boundary vertex-cell connectivity map: 4.05312e-06
Process 3: Time to build ghost dual graph: : 9.53674e-07
Process 3: Time to build local boundary vertex-cell connectivity map: 3.8147e-06
Process 3: Time to build ghost boundary vertex-cell connectivity map: 3.09944e-06
Process 3: Time to build ghost dual graph: : 9.53674e-07
Process 3: Time to build local boundary vertex-cell connectivity map: 3.09944e-06
Process 3: Time to build ghost boundary vertex-cell connectivity map: 2.86102e-06
Process 3: Time to build ghost dual graph: : 1.19209e-06
Process 3: Finish compute graph ghost edges.
Process 3: End compute dual graph.
Process 3: Start to compute partitions using SCOTCH
Process 5: Compute graph ghost edges.
Process 5: Time to build local boundary vertex-cell connectivity map: 9.05991e-06
Process 5: Time to build ghost boundary vertex-cell connectivity map: 6.19888e-06
Process 5: Time to build ghost dual graph: : 1.90735e-06
Process 5: Time to build local boundary vertex-cell connectivity map: 3.09944e-06
Process 5: Time to build ghost boundary vertex-cell connectivity map: 5.00679e-06
Process 5: Time to build ghost dual graph: : 9.53674e-07
Process 5: Time to build local boundary vertex-cell connectivity map: 3.09944e-06
Process 5: Time to build ghost boundary vertex-cell connectivity map: 2.86102e-06
Process 5: Time to build ghost dual graph: : 4.05312e-06
Process 5: Time to build local boundary vertex-cell connectivity map: 3.09944e-06
Process 5: Time to build ghost boundary vertex-cell connectivity map: 3.09944e-06
Process 5: Time to build ghost dual graph: : 2.86102e-06
Process 5: Time to build local boundary vertex-cell connectivity map: 3.09944e-06
Process 5: Time to build ghost boundary vertex-cell connectivity map: 2.86102e-06
Process 5: Time to build ghost dual graph: : 2.14577e-06
Process 5: Time to build local boundary vertex-cell connectivity map: 4.05312e-06
Process 5: Time to build ghost boundary vertex-cell connectivity map: 0
Process 5: Time to build ghost dual graph: : 0
Process 5: Time to build local boundary vertex-cell connectivity map: 4.05312e-06
Process 5: Time to build ghost boundary vertex-cell connectivity map: 3.09944e-06
Process 5: Time to build ghost dual graph: : 9.53674e-07
Process 5: Time to build local boundary vertex-cell connectivity map: 3.09944e-06
Process 5: Time to build ghost boundary vertex-cell connectivity map: 2.86102e-06
Process 5: Time to build ghost dual graph: : 3.09944e-06
Process 5: Time to build local boundary vertex-cell connectivity map: 3.09944e-06
Process 5: Time to build ghost boundary vertex-cell connectivity map: 2.86102e-06
Process 5: Time to build ghost dual graph: : 1.90735e-06
Process 5: Time to build local boundary vertex-cell connectivity map: 2.86102e-06
Process 5: Time to build ghost boundary vertex-cell connectivity map: 3.09944e-06
Process 5: Time to build ghost dual graph: : 9.53674e-07
Process 5: Time to build local boundary vertex-cell connectivity map: 3.09944e-06
Process 5: Time to build ghost boundary vertex-cell connectivity map: 2.86102e-06
Process 5: Time to build ghost dual graph: : 9.53674e-07
Process 5: Time to build local boundary vertex-cell connectivity map: 3.09944e-06
Process 5: Time to build ghost boundary vertex-cell connectivity map: 4.05312e-06
Process 5: Time to build ghost dual graph: : 9.53674e-07
Process 5: Time to build local boundary vertex-cell connectivity map: 3.09944e-06
Process 5: Time to build ghost boundary vertex-cell connectivity map: 2.86102e-06
Process 5: Time to build ghost dual graph: : 9.53674e-07
Process 5: Time to build local boundary vertex-cell connectivity map: 3.09944e-06
Process 5: Time to build ghost boundary vertex-cell connectivity map: 4.05312e-06
Process 5: Time to build ghost dual graph: : 0
Process 5: Time to build local boundary vertex-cell connectivity map: 4.05312e-06
Process 5: Time to build ghost boundary vertex-cell connectivity map: 2.86102e-06
Process 5: Time to build ghost dual graph: : 0
Process 5: Time to build local boundary vertex-cell connectivity map: 3.09944e-06
Process 5: Time to build ghost boundary vertex-cell connectivity map: 3.09944e-06
Process 5: Time to build ghost dual graph: : 9.53674e-07
Process 5: Finish compute graph ghost edges.
Process 5: End compute dual graph.
Process 5: Start to compute partitions using SCOTCH
Process 7: Compute graph ghost edges.
Process 7: Time to build local boundary vertex-cell connectivity map: 1.00136e-05
Process 7: Time to build ghost boundary vertex-cell connectivity map: 5.00679e-06
Process 7: Time to build ghost dual graph: : 1.90735e-06
Process 7: Time to build local boundary vertex-cell connectivity map: 4.05312e-06
Process 7: Time to build ghost boundary vertex-cell connectivity map: 3.8147e-06
Process 7: Time to build ghost dual graph: : 1.19209e-06
Process 7: Time to build local boundary vertex-cell connectivity map: 2.86102e-06
Process 7: Time to build ghost boundary vertex-cell connectivity map: 3.09944e-06
Process 7: Time to build ghost dual graph: : 9.53674e-07
Process 7: Time to build local boundary vertex-cell connectivity map: 3.09944e-06
Process 7: Time to build ghost boundary vertex-cell connectivity map: 2.86102e-06
Process 7: Time to build ghost dual graph: : 9.53674e-07
Process 7: Time to build local boundary vertex-cell connectivity map: 3.09944e-06
Process 7: Time to build ghost boundary vertex-cell connectivity map: 3.09944e-06
Process 7: Time to build ghost dual graph: : 3.8147e-06
Process 7: Time to build local boundary vertex-cell connectivity map: 3.09944e-06
Process 7: Time to build ghost boundary vertex-cell connectivity map: 3.09944e-06
Process 7: Time to build ghost dual graph: : 2.86102e-06
Process 7: Time to build local boundary vertex-cell connectivity map: 4.05312e-06
Process 7: Time to build ghost boundary vertex-cell connectivity map: 2.86102e-06
Process 7: Time to build ghost dual graph: : 2.14577e-06
Process 7: Time to build local boundary vertex-cell connectivity map: 4.05312e-06
Process 7: Time to build ghost boundary vertex-cell connectivity map: 0
Process 7: Time to build ghost dual graph: : 1.19209e-06
Process 7: Time to build local boundary vertex-cell connectivity map: 3.8147e-06
Process 7: Time to build ghost boundary vertex-cell connectivity map: 3.09944e-06
Process 7: Time to build ghost dual graph: : 9.53674e-07
Process 7: Time to build local boundary vertex-cell connectivity map: 3.09944e-06
Process 7: Time to build ghost boundary vertex-cell connectivity map: 2.86102e-06
Process 7: Time to build ghost dual graph: : 3.09944e-06
Process 7: Time to build local boundary vertex-cell connectivity map: 4.05312e-06
Process 7: Time to build ghost boundary vertex-cell connectivity map: 2.86102e-06
Process 7: Time to build ghost dual graph: : 3.09944e-06
Process 7: Time to build local boundary vertex-cell connectivity map: 4.05312e-06
Process 7: Time to build ghost boundary vertex-cell connectivity map: 4.05312e-06
Process 7: Time to build ghost dual graph: : 0
Process 7: Time to build local boundary vertex-cell connectivity map: 3.09944e-06
Process 7: Time to build ghost boundary vertex-cell connectivity map: 2.86102e-06
Process 7: Time to build ghost dual graph: : 9.53674e-07
Process 7: Time to build local boundary vertex-cell connectivity map: 2.86102e-06
Process 7: Time to build ghost boundary vertex-cell connectivity map: 4.05312e-06
Process 7: Time to build ghost dual graph: : 0
Process 7: Time to build local boundary vertex-cell connectivity map: 3.09944e-06
Process 7: Time to build ghost boundary vertex-cell connectivity map: 2.86102e-06
Process 7: Time to build ghost dual graph: : 9.53674e-07
Process 7: Time to build local boundary vertex-cell connectivity map: 2.86102e-06
Process 7: Time to build ghost boundary vertex-cell connectivity map: 3.09944e-06
Process 7: Time to build ghost dual graph: : 0
Process 7: Finish compute graph ghost edges.
Process 7: End compute dual graph.
Process 7: Start to compute partitions using SCOTCH
Process 9: Compute graph ghost edges.
Process 9: Time to build local boundary vertex-cell connectivity map: 1.09673e-05
Process 9: Time to build ghost boundary vertex-cell connectivity map: 5.96046e-06
Process 9: Time to build ghost dual graph: : 3.09944e-06
Process 9: Time to build local boundary vertex-cell connectivity map: 3.8147e-06
Process 9: Time to build ghost boundary vertex-cell connectivity map: 4.05312e-06
Process 9: Time to build ghost dual graph: : 1.19209e-06
Process 9: Time to build local boundary vertex-cell connectivity map: 2.86102e-06
Process 9: Time to build ghost boundary vertex-cell connectivity map: 3.09944e-06
Process 9: Time to build ghost dual graph: : 9.53674e-07
Process 9: Time to build local boundary vertex-cell connectivity map: 3.09944e-06
Process 9: Time to build ghost boundary vertex-cell connectivity map: 2.86102e-06
Process 9: Time to build ghost dual graph: : 9.53674e-07
Process 9: Time to build local boundary vertex-cell connectivity map: 2.86102e-06
Process 9: Time to build ghost boundary vertex-cell connectivity map: 4.05312e-06
Process 9: Time to build ghost dual graph: : 9.53674e-07
Process 9: Time to build local boundary vertex-cell connectivity map: 4.05312e-06
Process 9: Time to build ghost boundary vertex-cell connectivity map: 2.86102e-06
Process 9: Time to build ghost dual graph: : 1.19209e-06
Process 9: Time to build local boundary vertex-cell connectivity map: 2.86102e-06
Process 9: Time to build ghost boundary vertex-cell connectivity map: 3.09944e-06
Process 9: Time to build ghost dual graph: : 4.05312e-06
Process 9: Time to build local boundary vertex-cell connectivity map: 2.86102e-06
Process 9: Time to build ghost boundary vertex-cell connectivity map: 3.09944e-06
Process 9: Time to build ghost dual graph: : 3.09944e-06
Process 9: Time to build local boundary vertex-cell connectivity map: 2.86102e-06
Process 9: Time to build ghost boundary vertex-cell connectivity map: 3.09944e-06
Process 9: Time to build ghost dual graph: : 9.53674e-07
Process 9: Time to build local boundary vertex-cell connectivity map: 3.09944e-06
Process 9: Time to build ghost boundary vertex-cell connectivity map: 0
Process 9: Time to build ghost dual graph: : 9.53674e-07
Process 9: Time to build local boundary vertex-cell connectivity map: 2.86102e-06
Process 9: Time to build ghost boundary vertex-cell connectivity map: 3.09944e-06
Process 9: Time to build ghost dual graph: : 9.53674e-07
Process 9: Time to build local boundary vertex-cell connectivity map: 3.09944e-06
Process 9: Time to build ghost boundary vertex-cell connectivity map: 2.86102e-06
Process 9: Time to build ghost dual graph: : 1.90735e-06
Process 9: Time to build local boundary vertex-cell connectivity map: 4.05312e-06
Process 9: Time to build ghost boundary vertex-cell connectivity map: 3.09944e-06
Process 9: Time to build ghost dual graph: : 2.14577e-06
Process 9: Time to build local boundary vertex-cell connectivity map: 4.05312e-06
Process 9: Time to build ghost boundary vertex-cell connectivity map: 2.86102e-06
Process 9: Time to build ghost dual graph: : 0
Process 9: Time to build local boundary vertex-cell connectivity map: 2.86102e-06
Process 9: Time to build ghost boundary vertex-cell connectivity map: 4.05312e-06
Process 9: Time to build ghost dual graph: : 9.53674e-07
Process 9: Time to build local boundary vertex-cell connectivity map: 4.05312e-06
Process 9: Time to build ghost boundary vertex-cell connectivity map: 2.86102e-06
Process 9: Time to build ghost dual graph: : 1.19209e-06
Process 9: Finish compute graph ghost edges.
Process 9: End compute dual graph.
Process 9: Start to compute partitions using SCOTCH
Process 10: Compute graph ghost edges.
Process 10: Time to build local boundary vertex-cell connectivity map: 1.28746e-05
Process 10: Time to build ghost boundary vertex-cell connectivity map: 7.86781e-06
Process 10: Time to build ghost dual graph: : 1.90735e-06
Process 10: Time to build local boundary vertex-cell connectivity map: 5.00679e-06
Process 10: Time to build ghost boundary vertex-cell connectivity map: 7.86781e-06
Process 10: Time to build ghost dual graph: : 9.53674e-07
Process 10: Time to build local boundary vertex-cell connectivity map: 5.00679e-06
Process 10: Time to build ghost boundary vertex-cell connectivity map: 5.96046e-06
Process 10: Time to build ghost dual graph: : 1.19209e-06
Process 10: Time to build local boundary vertex-cell connectivity map: 5.96046e-06
Process 10: Time to build ghost boundary vertex-cell connectivity map: 5.00679e-06
Process 10: Time to build ghost dual graph: : 9.53674e-07
Process 10: Time to build local boundary vertex-cell connectivity map: 5.00679e-06
Process 10: Time to build ghost boundary vertex-cell connectivity map: 5.96046e-06
Process 10: Time to build ghost dual graph: : 9.53674e-07
Process 10: Time to build local boundary vertex-cell connectivity map: 5.96046e-06
Process 10: Time to build ghost boundary vertex-cell connectivity map: 5.00679e-06
Process 10: Time to build ghost dual graph: : 9.53674e-07
Process 10: Time to build local boundary vertex-cell connectivity map: 5.00679e-06
Process 10: Time to build ghost boundary vertex-cell connectivity map: 6.19888e-06
Process 10: Time to build ghost dual graph: : 9.53674e-07
Process 10: Time to build local boundary vertex-cell connectivity map: 5.00679e-06
Process 10: Time to build ghost boundary vertex-cell connectivity map: 5.00679e-06
Process 10: Time to build ghost dual graph: : 4.05312e-06
Process 10: Time to build local boundary vertex-cell connectivity map: 5.96046e-06
Process 10: Time to build ghost boundary vertex-cell connectivity map: 5.00679e-06
Process 10: Time to build ghost dual graph: : 3.09944e-06
Process 10: Time to build local boundary vertex-cell connectivity map: 6.19888e-06
Process 10: Time to build ghost boundary vertex-cell connectivity map: 4.05312e-06
Process 10: Time to build ghost dual graph: : 3.09944e-06
Process 10: Time to build local boundary vertex-cell connectivity map: 5.00679e-06
Process 10: Time to build ghost boundary vertex-cell connectivity map: 0
Process 10: Time to build ghost dual graph: : 9.53674e-07
Process 10: Time to build local boundary vertex-cell connectivity map: 5.96046e-06
Process 10: Time to build ghost boundary vertex-cell connectivity map: 6.19888e-06
Process 10: Time to build ghost dual graph: : 1.90735e-06
Process 13: Compute graph ghost edges.
Process 13: Time to build local boundary vertex-cell connectivity map: 1.09673e-05
Process 13: Time to build ghost boundary vertex-cell connectivity map: 5.96046e-06
Process 13: Time to build ghost dual graph: : 2.14577e-06
Process 13: Time to build local boundary vertex-cell connectivity map: 4.05312e-06
Process 13: Time to build ghost boundary vertex-cell connectivity map: 4.05312e-06
Process 13: Time to build ghost dual graph: : 0
Process 13: Time to build local boundary vertex-cell connectivity map: 3.09944e-06
Process 13: Time to build ghost boundary vertex-cell connectivity map: 4.05312e-06
Process 13: Time to build ghost dual graph: : 9.53674e-07
Process 13: Time to build local boundary vertex-cell connectivity map: 3.09944e-06
Process 13: Time to build ghost boundary vertex-cell connectivity map: 3.8147e-06
Process 13: Time to build ghost dual graph: : 9.53674e-07
Process 13: Time to build local boundary vertex-cell connectivity map: 2.86102e-06
Process 13: Time to build ghost boundary vertex-cell connectivity map: 4.05312e-06
Process 13: Time to build ghost dual graph: : 9.53674e-07
Process 13: Time to build local boundary vertex-cell connectivity map: 3.09944e-06
Process 13: Time to build ghost boundary vertex-cell connectivity map: 4.05312e-06
Process 13: Time to build ghost dual graph: : 9.53674e-07
Process 13: Time to build local boundary vertex-cell connectivity map: 3.09944e-06
Process 13: Time to build ghost boundary vertex-cell connectivity map: 3.8147e-06
Process 13: Time to build ghost dual graph: : 0
Process 13: Time to build local boundary vertex-cell connectivity map: 4.05312e-06
Process 13: Time to build ghost boundary vertex-cell connectivity map: 3.09944e-06
Process 13: Time to build ghost dual graph: : 9.53674e-07
Process 13: Time to build local boundary vertex-cell connectivity map: 3.09944e-06
Process 13: Time to build ghost boundary vertex-cell connectivity map: 3.8147e-06
Process 13: Time to build ghost dual graph: : 0
Process 13: Time to build local boundary vertex-cell connectivity map: 2.86102e-06
Process 13: Time to build ghost boundary vertex-cell connectivity map: 4.05312e-06
Process 13: Time to build ghost dual graph: : 9.53674e-07
Process 13: Time to build local boundary vertex-cell connectivity map: 4.05312e-06
Process 13: Time to build ghost boundary vertex-cell connectivity map: 2.86102e-06
Process 13: Time to build ghost dual graph: : 3.09944e-06
Process 13: Time to build local boundary vertex-cell connectivity map: 3.09944e-06
Process 13: Time to build ghost boundary vertex-cell connectivity map: 2.86102e-06
Process 13: Time to build ghost dual graph: : 3.09944e-06
Process 13: Time to build local boundary vertex-cell connectivity map: 3.09944e-06
Process 13: Time to build ghost boundary vertex-cell connectivity map: 3.09944e-06
Process 13: Time to build ghost dual graph: : 1.90735e-06
Process 13: Time to build local boundary vertex-cell connectivity map: 2.86102e-06
Process 13: Time to build ghost boundary vertex-cell connectivity map: 0
Process 13: Time to build ghost dual graph: : 9.53674e-07
Process 13: Time to build local boundary vertex-cell connectivity map: 3.09944e-06
Process 13: Time to build ghost boundary vertex-cell connectivity map: 2.86102e-06
Process 13: Time to build ghost dual graph: : 1.19209e-06
Process 13: Time to build local boundary vertex-cell connectivity map: 4.05312e-06
Process 13: Time to build ghost boundary vertex-cell connectivity map: 2.14577e-06
Process 13: Time to build ghost dual graph: : 2.86102e-06
Process 13: Finish compute graph ghost edges.
Process 13: End compute dual graph.
Process 13: Start to compute partitions using SCOTCH
Process 10: Time to build local boundary vertex-cell connectivity map: 5.00679e-06
Process 10: Time to build ghost boundary vertex-cell connectivity map: 5.00679e-06
Process 10: Time to build ghost dual graph: : 3.09944e-06
Process 10: Time to build local boundary vertex-cell connectivity map: 5.00679e-06
Process 10: Time to build ghost boundary vertex-cell connectivity map: 5.00679e-06
Process 10: Time to build ghost dual graph: : 2.86102e-06
Process 10: Time to build local boundary vertex-cell connectivity map: 5.96046e-06
Process 10: Time to build ghost boundary vertex-cell connectivity map: 5.00679e-06
Process 10: Time to build ghost dual graph: : 1.19209e-06
Process 10: Time to build local boundary vertex-cell connectivity map: 5.00679e-06
Process 10: Time to build ghost boundary vertex-cell connectivity map: 5.00679e-06
Process 10: Time to build ghost dual graph: : 9.53674e-07
Process 10: Finish compute graph ghost edges.
Process 10: End compute dual graph.
Process 10: Start to compute partitions using SCOTCH
Process 14: Compute graph ghost edges.
Process 14: Time to build local boundary vertex-cell connectivity map: 1.21593e-05
Process 14: Time to build ghost boundary vertex-cell connectivity map: 7.86781e-06
Process 14: Time to build ghost dual graph: : 3.09944e-06
Process 14: Time to build local boundary vertex-cell connectivity map: 5.96046e-06
Process 14: Time to build ghost boundary vertex-cell connectivity map: 7.15256e-06
Process 14: Time to build ghost dual graph: : 1.19209e-06
Process 14: Time to build local boundary vertex-cell connectivity map: 6.19888e-06
Process 14: Time to build ghost boundary vertex-cell connectivity map: 5.00679e-06
Process 14: Time to build ghost dual graph: : 1.90735e-06
Process 14: Time to build local boundary vertex-cell connectivity map: 5.00679e-06
Process 14: Time to build ghost boundary vertex-cell connectivity map: 5.00679e-06
Process 14: Time to build ghost dual graph: : 9.53674e-07
Process 14: Time to build local boundary vertex-cell connectivity map: 5.00679e-06
Process 14: Time to build ghost boundary vertex-cell connectivity map: 5.00679e-06
Process 14: Time to build ghost dual graph: : 0
Process 14: Time to build local boundary vertex-cell connectivity map: 5.00679e-06
Process 14: Time to build ghost boundary vertex-cell connectivity map: 5.00679e-06
Process 14: Time to build ghost dual graph: : 1.19209e-06
Process 14: Time to build local boundary vertex-cell connectivity map: 6.19888e-06
Process 14: Time to build ghost boundary vertex-cell connectivity map: 5.96046e-06
Process 14: Time to build ghost dual graph: : 9.53674e-07
Process 14: Time to build local boundary vertex-cell connectivity map: 5.00679e-06
Process 14: Time to build ghost boundary vertex-cell connectivity map: 5.00679e-06
Process 14: Time to build ghost dual graph: : 1.90735e-06
Process 14: Time to build local boundary vertex-cell connectivity map: 5.00679e-06
Process 14: Time to build ghost boundary vertex-cell connectivity map: 5.00679e-06
Process 14: Time to build ghost dual graph: : 9.53674e-07
Process 14: Time to build local boundary vertex-cell connectivity map: 5.00679e-06
Process 14: Time to build ghost boundary vertex-cell connectivity map: 5.00679e-06
Process 14: Time to build ghost dual graph: : 1.90735e-06
Process 14: Time to build local boundary vertex-cell connectivity map: 5.00679e-06
Process 14: Time to build ghost boundary vertex-cell connectivity map: 5.00679e-06
Process 14: Time to build ghost dual graph: : 9.53674e-07
Process 14: Time to build local boundary vertex-cell connectivity map: 5.96046e-06
Process 14: Time to build ghost boundary vertex-cell connectivity map: 5.00679e-06
Process 14: Time to build ghost dual graph: : 4.05312e-06
Process 14: Time to build local boundary vertex-cell connectivity map: 5.00679e-06
Process 14: Time to build ghost boundary vertex-cell connectivity map: 5.00679e-06
Process 14: Time to build ghost dual graph: : 3.09944e-06
Process 14: Time to build local boundary vertex-cell connectivity map: 6.19888e-06
Process 14: Time to build ghost boundary vertex-cell connectivity map: 5.00679e-06
Process 14: Time to build ghost dual graph: : 1.90735e-06
Process 14: Time to build local boundary vertex-cell connectivity map: 5.00679e-06
Process 14: Time to build ghost boundary vertex-cell connectivity map: 0
Process 14: Time to build ghost dual graph: : 1.19209e-06
Process 14: Time to build local boundary vertex-cell connectivity map: 5.00679e-06
Process 14: Time to build ghost boundary vertex-cell connectivity map: 5.00679e-06
Process 14: Time to build ghost dual graph: : 2.14577e-06
Process 14: Finish compute graph ghost edges.
Process 14: End compute dual graph.
Process 14: Start to compute partitions using SCOTCH
Process 12: Compute graph ghost edges.
Process 12: Time to build local boundary vertex-cell connectivity map: 1.21593e-05
Process 12: Time to build ghost boundary vertex-cell connectivity map: 7.86781e-06
Process 12: Time to build ghost dual graph: : 3.09944e-06
Process 12: Time to build local boundary vertex-cell connectivity map: 5.96046e-06
Process 12: Time to build ghost boundary vertex-cell connectivity map: 7.15256e-06
Process 12: Time to build ghost dual graph: : 9.53674e-07
Process 12: Time to build local boundary vertex-cell connectivity map: 5.00679e-06
Process 12: Time to build ghost boundary vertex-cell connectivity map: 5.00679e-06
Process 12: Time to build ghost dual graph: : 1.19209e-06
Process 12: Time to build local boundary vertex-cell connectivity map: 5.96046e-06
Process 12: Time to build ghost boundary vertex-cell connectivity map: 5.00679e-06
Process 12: Time to build ghost dual graph: : 9.53674e-07
Process 12: Time to build local boundary vertex-cell connectivity map: 5.00679e-06
Process 12: Time to build ghost boundary vertex-cell connectivity map: 5.96046e-06
Process 12: Time to build ghost dual graph: : 9.53674e-07
Process 12: Time to build local boundary vertex-cell connectivity map: 5.00679e-06
Process 12: Time to build ghost boundary vertex-cell connectivity map: 5.00679e-06
Process 12: Time to build ghost dual graph: : 9.53674e-07
Process 12: Time to build local boundary vertex-cell connectivity map: 5.00679e-06
Process 12: Time to build ghost boundary vertex-cell connectivity map: 6.19888e-06
Process 12: Time to build ghost dual graph: : 1.19209e-06
Process 12: Time to build local boundary vertex-cell connectivity map: 6.19888e-06
Process 12: Time to build ghost boundary vertex-cell connectivity map: 5.00679e-06
Process 12: Time to build ghost dual graph: : 9.53674e-07
Process 12: Time to build local boundary vertex-cell connectivity map: 5.00679e-06
Process 12: Time to build ghost boundary vertex-cell connectivity map: 5.96046e-06
Process 12: Time to build ghost dual graph: : 9.53674e-07
Process 12: Time to build local boundary vertex-cell connectivity map: 5.96046e-06
Process 12: Time to build ghost boundary vertex-cell connectivity map: 4.05312e-06
Process 12: Time to build ghost dual graph: : 3.8147e-06
Process 12: Time to build local boundary vertex-cell connectivity map: 5.00679e-06
Process 12: Time to build ghost boundary vertex-cell connectivity map: 5.00679e-06
Process 12: Time to build ghost dual graph: : 2.86102e-06
Process 12: Time to build local boundary vertex-cell connectivity map: 5.00679e-06
Process 12: Time to build ghost boundary vertex-cell connectivity map: 5.00679e-06
Process 12: Time to build ghost dual graph: : 2.14577e-06
Process 12: Time to build local boundary vertex-cell connectivity map: 5.00679e-06
Process 12: Time to build ghost boundary vertex-cell connectivity map: 0
Process 12: Time to build ghost dual graph: : 1.19209e-06
Process 12: Time to build local boundary vertex-cell connectivity map: 5.00679e-06
Process 12: Time to build ghost boundary vertex-cell connectivity map: 5.96046e-06
Process 12: Time to build ghost dual graph: : 2.14577e-06
Process 12: Time to build local boundary vertex-cell connectivity map: 5.00679e-06
Process 12: Time to build ghost boundary vertex-cell connectivity map: 5.00679e-06
Process 12: Time to build ghost dual graph: : 3.8147e-06
Process 12: Time to build local boundary vertex-cell connectivity map: 5.96046e-06
Process 12: Time to build ghost boundary vertex-cell connectivity map: 5.00679e-06
Process 12: Time to build ghost dual graph: : 3.09944e-06
Process 12: Finish compute graph ghost edges.
Process 12: End compute dual graph.
Process 12: Start to compute partitions using SCOTCH
Process 10: Start SCOTCH graph building.
Process 6: Start SCOTCH graph building.
Process 8: Start SCOTCH graph building.
Process 12: Start SCOTCH graph building.
Process 2: Start SCOTCH graph building.
Process 14: Start SCOTCH graph building.
Process 0: Start SCOTCH graph building.
Process 4: Start SCOTCH graph building.
Process 9: Start SCOTCH graph building.
Process 11: Start SCOTCH graph building.
Process 15: Start SCOTCH graph building.
Process 3: Start SCOTCH graph building.
Process 5: Start SCOTCH graph building.
Process 7: Start SCOTCH graph building.
Process 1: Start SCOTCH graph building.
Process 13: Start SCOTCH graph building.
Process 8: End SCOTCH graph building.
Process 2: End SCOTCH graph building.
Process 10: End SCOTCH graph building.
Process 14: End SCOTCH graph building.
Process 0: End SCOTCH graph building.
Process 6: End SCOTCH graph building.
Process 12: End SCOTCH graph building.
Process 4: End SCOTCH graph building.
Process 15: End SCOTCH graph building.
Process 3: End SCOTCH graph building.
Process 7: End SCOTCH graph building.
Process 13: End SCOTCH graph building.
Process 1: End SCOTCH graph building.
Process 9: End SCOTCH graph building.
Process 11: End SCOTCH graph building.
Process 5: End SCOTCH graph building.
Process 12: Start SCOTCH partitioning.
Process 6: Start SCOTCH partitioning.
Process 4: Start SCOTCH partitioning.
Process 14: Start SCOTCH partitioning.
Process 10: Start SCOTCH partitioning.
Process 8: Start SCOTCH partitioning.
Process 7: Start SCOTCH partitioning.
Process 2: Start SCOTCH partitioning.
Process 11: Start SCOTCH partitioning.
Process 3: Start SCOTCH partitioning.
Process 13: Start SCOTCH partitioning.
Process 5: Start SCOTCH partitioning.
Process 0: Start SCOTCH partitioning.
Process 1: Start SCOTCH partitioning.
Process 9: Start SCOTCH partitioning.
Process 15: Start SCOTCH partitioning.
Process 6: End SCOTCH partitioning.
Process 6: Finished computing partitions using SCOTCH
Process 10: End SCOTCH partitioning.
Process 10: Finished computing partitions using SCOTCH
Process 2: End SCOTCH partitioning.
Process 2: Finished computing partitions using SCOTCH
Process 14: End SCOTCH partitioning.
Process 14: Finished computing partitions using SCOTCH
Process 4: End SCOTCH partitioning.
Process 4: Finished computing partitions using SCOTCH
Process 8: End SCOTCH partitioning.
Process 8: Finished computing partitions using SCOTCH
Process 12: End SCOTCH partitioning.
Process 12: Finished computing partitions using SCOTCH
Process 0: End SCOTCH partitioning.
Process 0: Finished computing partitions using SCOTCH
Process 3: End SCOTCH partitioning.
Process 3: Finished computing partitions using SCOTCH
Process 5: End SCOTCH partitioning.
Process 5: Finished computing partitions using SCOTCH
Process 1: End SCOTCH partitioning.
Process 1: Finished computing partitions using SCOTCH
Process 13: End SCOTCH partitioning.
Process 13: Finished computing partitions using SCOTCH
Process 9: End SCOTCH partitioning.
Process 9: Finished computing partitions using SCOTCH
Process 7: End SCOTCH partitioning.
Process 7: Finished computing partitions using SCOTCH
Process 11: End SCOTCH partitioning.
Process 11: Finished computing partitions using SCOTCH
Process 15: End SCOTCH partitioning.
Process 15: Finished computing partitions using SCOTCH
Making solver...
Problem: Channel
Solver: IPCS_opt_seg
NB! Directory already exists: 'results/channel_ipcs_opt_refinement_level_0'.
Computing time step according to stability criteria

Number of timesteps: 13
Size of timestep: 0.0384615384615

[gpc-f138n061:14107] *** Process received signal ***
[gpc-f138n061:14107] Signal: Segmentation fault (11)
[gpc-f138n061:14107] Signal code: Address not mapped (1)
[gpc-f138n061:14107] Failing at address: 0x1ddb750
mpirun: killing job...

--------------------------------------------------------------------------
mpirun noticed that process rank 0 with PID 14083 on node gpc-f138n061 exited on signal 0 (Unknown signal 0).
--------------------------------------------------------------------------
mpirun: clean termination accomplished

16 total processes killed (some possibly by mpirun during cleanup)

Revision history for this message
Damiaan (dhabets) said : 		#3

Here's what I get running the navier-stokes cxx demo:

got dolfin
got dolfin
got dolfin
got dolfin
got dolfin
got dolfin
got dolfin
got dolfin
got dolfin
got dolfin
got dolfin
got dolfin
got dolfin
got dolfin
got dolfin
got dolfin
Process 0: Number of global vertices: 1331
Process 0: Number of global cells: 6000
Process 2: Compute dual graph.
Process 4: Compute dual graph.
Process 6: Compute dual graph.
Process 8: Compute dual graph.
Process 10: Compute dual graph.
Process 14: Compute dual graph.
Process 0: Compute dual graph.
Process 12: Compute dual graph.
Process 1: Compute dual graph.
Process 3: Compute dual graph.
Process 5: Compute dual graph.
Process 7: Compute dual graph.
Process 9: Compute dual graph.
Process 11: Compute dual graph.
Process 13: Compute dual graph.
Process 15: Compute dual graph.
Process 10: Compute local cell-cell connections
Process 10: Time to build vertex-cell connectivity map: 0.000246048
Process 14: Compute local cell-cell connections
Process 14: Time to build vertex-cell connectivity map: 0.000159025
Process 2: Compute local cell-cell connections
Process 4: Compute local cell-cell connections
Process 4: Time to build vertex-cell connectivity map: 0.000207901
Process 6: Compute local cell-cell connections
Process 8: Compute local cell-cell connections
Process 0: Compute local cell-cell connections
Process 12: Compute local cell-cell connections
Process 0: Time to build vertex-cell connectivity map: 0.000173092
Process 13: Compute local cell-cell connections
Process 13: Time to build vertex-cell connectivity map: 0.000232935
Process 12: Time to build vertex-cell connectivity map: 0.00024581
Process 6: Time to build vertex-cell connectivity map: 0.000245094
Process 8: Time to build vertex-cell connectivity map: 0.000239134
Process 1: Compute local cell-cell connections
Process 1: Time to build vertex-cell connectivity map: 0.000215054
Process 3: Compute local cell-cell connections
Process 3: Time to build vertex-cell connectivity map: 0.000239849
Process 5: Compute local cell-cell connections
Process 5: Time to build vertex-cell connectivity map: 0.000234127
Process 7: Compute local cell-cell connections
Process 7: Time to build vertex-cell connectivity map: 0.000231981
Process 9: Compute local cell-cell connections
Process 9: Time to build vertex-cell connectivity map: 0.000207901
Process 11: Compute local cell-cell connections
Process 11: Time to build vertex-cell connectivity map: 0.000233889
Process 14: Time to build local dual graph: : 0.00052309
Process 14: Finished computing local cell-cell connections
Process 14: Preparing data to to send off-process.
Process 15: Compute local cell-cell connections
Process 15: Time to build vertex-cell connectivity map: 0.000231028
Process 10: Time to build local dual graph: : 0.000895023
Process 10: Finished computing local cell-cell connections
Process 10: Preparing data to to send off-process.
Process 2: Time to build vertex-cell connectivity map: 0.000774145
Process 4: Time to build local dual graph: : 0.000821114
Process 4: Finished computing local cell-cell connections
Process 4: Preparing data to to send off-process.
Process 0: Time to build local dual graph: : 0.000548124
Process 0: Finished computing local cell-cell connections
Process 0: Preparing data to to send off-process.
Process 6: Time to build local dual graph: : 0.000813961
Process 12: Time to build local dual graph: : 0.00082016
Process 12: Finished computing local cell-cell connections
Process 12: Preparing data to to send off-process.
Process 6: Finished computing local cell-cell connections
Process 6: Preparing data to to send off-process.
Process 8: Time to build local dual graph: : 0.000876188
Process 8: Finished computing local cell-cell connections
Process 8: Preparing data to to send off-process.
Process 11: Time to build local dual graph: : 0.000823975
Process 11: Finished computing local cell-cell connections
Process 11: Preparing data to to send off-process.
Process 13: Time to build local dual graph: : 0.000886917
Process 13: Finished computing local cell-cell connections
Process 13: Preparing data to to send off-process.
Process 1: Time to build local dual graph: : 0.000813961
Process 1: Finished computing local cell-cell connections
Process 1: Preparing data to to send off-process.
Process 3: Time to build local dual graph: : 0.000815868
Process 3: Finished computing local cell-cell connections
Process 3: Preparing data to to send off-process.
Process 5: Time to build local dual graph: : 0.000873089
Process 5: Finished computing local cell-cell connections
Process 5: Preparing data to to send off-process.
Process 7: Time to build local dual graph: : 0.000871181
Process 7: Finished computing local cell-cell connections
Process 7: Preparing data to to send off-process.
Process 9: Time to build local dual graph: : 0.00104618
Process 9: Finished computing local cell-cell connections
Process 9: Preparing data to to send off-process.
Process 15: Time to build local dual graph: : 0.000874043
Process 15: Finished computing local cell-cell connections
Process 15: Preparing data to to send off-process.
Process 2: Time to build local dual graph: : 0.00136209
Process 2: Finished computing local cell-cell connections
Process 2: Preparing data to to send off-process.
Process 2: Compute graph ghost edges.
Process 2: Time to build local boundary vertex-cell connectivity map: 0.000192881
Process 2: Time to build ghost boundary vertex-cell connectivity map: 0.000159979
Process 3: Compute graph ghost edges.
Process 3: Time to build local boundary vertex-cell connectivity map: 0.000201941
Process 3: Time to build ghost boundary vertex-cell connectivity map: 0.000139952
Process 3: Time to build ghost dual graph: : 5.79357e-05
Process 2: Time to build ghost dual graph: : 0.000263929
Process 2: Time to build local boundary vertex-cell connectivity map: 0.00014782
Process 2: Time to build ghost boundary vertex-cell connectivity map: 0.000151873
Process 2: Time to build ghost dual graph: : 0.000221014
Process 2: Time to build local boundary vertex-cell connectivity map: 0.000160933
Process 2: Time to build ghost boundary vertex-cell connectivity map: 0
Process 2: Time to build ghost dual graph: : 3.79086e-05
Process 3: Time to build local boundary vertex-cell connectivity map: 0.000177145
Process 3: Time to build ghost boundary vertex-cell connectivity map: 0.00014782
Process 3: Time to build ghost dual graph: : 0.000252008
Process 3: Time to build local boundary vertex-cell connectivity map: 0.000160933
Process 3: Time to build ghost boundary vertex-cell connectivity map: 0.000136852
Process 14: Compute graph ghost edges.
Process 2: Time to build local boundary vertex-cell connectivity map: 0.000144005
Process 2: Time to build ghost boundary vertex-cell connectivity map: 0.000147104
Process 2: Time to build ghost dual graph: : 0.000215054
Process 2: Time to build local boundary vertex-cell connectivity map: 0.000149012
Process 10: Compute graph ghost edges.
Process 10: Time to build local boundary vertex-cell connectivity map: 0.00016284
Process 12: Compute graph ghost edges.
Process 2: Time to build ghost boundary vertex-cell connectivity map: 0.000159025
Process 0: Compute graph ghost edges.
Process 1: Compute graph ghost edges.
Process 1: Time to build local boundary vertex-cell connectivity map: 0.000273943
Process 3: Time to build ghost dual graph: : 0.000252008
Process 3: Time to build local boundary vertex-cell connectivity map: 0.000162125
Process 3: Time to build ghost boundary vertex-cell connectivity map: 0
Process 3: Time to build ghost dual graph: : 3.91006e-05
Process 3: Time to build local boundary vertex-cell connectivity map: 0.00015521
Process 3: Time to build ghost boundary vertex-cell connectivity map: 0.000145912
Process 3: Time to build ghost dual graph: : 0.000225067
Process 2: Time to build ghost dual graph: : 0.000242949
Process 13: Compute graph ghost edges.
Process 1: Time to build ghost boundary vertex-cell connectivity map: 0.00022912
Process 10: Time to build ghost boundary vertex-cell connectivity map: 0.000154018
Process 11: Compute graph ghost edges.
Process 10: Time to build ghost dual graph: : 4.00543e-05
Process 4: Compute graph ghost edges.
Process 12: Time to build local boundary vertex-cell connectivity map: 0.000257969
Process 12: Time to build ghost boundary vertex-cell connectivity map: 0.000157118
Process 12: Time to build ghost dual graph: : 4.19617e-05
Process 12: Time to build local boundary vertex-cell connectivity map: 0.000163078
Process 12: Time to build ghost boundary vertex-cell connectivity map: 0.000164032
Process 8: Compute graph ghost edges.
Process 10: Time to build local boundary vertex-cell connectivity map: 0.000150919
Process 10: Time to build ghost boundary vertex-cell connectivity map: 0.000151873
Process 10: Time to build ghost dual graph: : 3.8147e-05
Process 10: Time to build local boundary vertex-cell connectivity map: 0.000153065
Process 2: Time to build local boundary vertex-cell connectivity map: 0.000149965
Process 2: Time to build ghost boundary vertex-cell connectivity map: 0.000143051
Process 2: Time to build ghost dual graph: : 3.48091e-05
Process 2: Time to build local boundary vertex-cell connectivity map: 0.000149012
Process 2: Time to build ghost boundary vertex-cell connectivity map: 0.000164032
Process 2: Time to build ghost dual graph: : 3.60012e-05
Process 0: Time to build local boundary vertex-cell connectivity map: 0.000257969
Process 0: Time to build ghost boundary vertex-cell connectivity map: 0
Process 0: Time to build ghost dual graph: : 4.98295e-05
Process 0: Time to build local boundary vertex-cell connectivity map: 0.000241041
Process 12: Time to build ghost dual graph: : 5.79357e-05
Process 10: Time to build ghost boundary vertex-cell connectivity map: 0.000154972
Process 0: Time to build ghost boundary vertex-cell connectivity map: 0.000263929
Process 9: Compute graph ghost edges.
Process 9: Time to build local boundary vertex-cell connectivity map: 0.000262976
Process 9: Time to build ghost boundary vertex-cell connectivity map: 0.000233889
Process 9: Time to build ghost dual graph: : 5.6982e-05
Process 10: Time to build ghost dual graph: : 3.91006e-05
Process 8: Time to build local boundary vertex-cell connectivity map: 0.000279903
Process 1: Time to build ghost dual graph: : 0.000338078
Process 1: Time to build local boundary vertex-cell connectivity map: 0.000253916
Process 1: Time to build ghost boundary vertex-cell connectivity map: 0
Process 1: Time to build ghost dual graph: : 5.4121e-05
Process 3: Time to build local boundary vertex-cell connectivity map: 0.00019002
Process 3: Time to build ghost boundary vertex-cell connectivity map: 0.000143051
Process 3: Time to build ghost dual graph: : 0.000241995
Process 3: Time to build local boundary vertex-cell connectivity map: 0.000159979
Process 2: Time to build local boundary vertex-cell connectivity map: 0.000150919
Process 13: Time to build local boundary vertex-cell connectivity map: 0.000256062
Process 13: Time to build ghost boundary vertex-cell connectivity map: 0.000236034
Process 13: Time to build ghost dual graph: : 5.10216e-05
Process 4: Time to build local boundary vertex-cell connectivity map: 0.00026989
Process 11: Time to build local boundary vertex-cell connectivity map: 0.000252008
Process 11: Time to build ghost boundary vertex-cell connectivity map: 0.000225067
Process 11: Time to build ghost dual graph: : 5.50747e-05
Process 5: Compute graph ghost edges.
Process 10: Time to build local boundary vertex-cell connectivity map: 0.000152111
Process 2: Time to build ghost boundary vertex-cell connectivity map: 0.000140905
Process 8: Time to build ghost boundary vertex-cell connectivity map: 0.000232935
Process 12: Time to build local boundary vertex-cell connectivity map: 0.000247002
Process 2: Time to build ghost dual graph: : 3.69549e-05
Process 0: Time to build ghost dual graph: : 0.00032711
Process 8: Time to build ghost dual graph: : 5.10216e-05
Process 10: Time to build ghost boundary vertex-cell connectivity map: 0.000150919
Process 4: Time to build ghost boundary vertex-cell connectivity map: 0.000230789
Process 10: Time to build ghost dual graph: : 3.88622e-05
Process 12: Time to build ghost boundary vertex-cell connectivity map: 0.000210047
Process 12: Time to build ghost dual graph: : 4.1008e-05
Process 12: Time to build local boundary vertex-cell connectivity map: 0.000161886
Process 12: Time to build ghost boundary vertex-cell connectivity map: 0.000158072
Process 4: Time to build ghost dual graph: : 5.79357e-05
Process 4: Time to build local boundary vertex-cell connectivity map: 0.000250101
Process 2: Time to build local boundary vertex-cell connectivity map: 0.000149012
Process 2: Time to build ghost boundary vertex-cell connectivity map: 0.000139952
Process 2: Time to build ghost dual graph: : 3.50475e-05
Process 2: Time to build local boundary vertex-cell connectivity map: 0.000149012
Process 8: Time to build local boundary vertex-cell connectivity map: 0.000243187
Process 8: Time to build ghost boundary vertex-cell connectivity map: 0.000236034
Process 0: Time to build local boundary vertex-cell connectivity map: 0.000247955
Process 0: Time to build ghost boundary vertex-cell connectivity map: 0.000230074
Process 10: Time to build local boundary vertex-cell connectivity map: 0.000151873
Process 10: Time to build ghost boundary vertex-cell connectivity map: 0.000150919
Process 10: Time to build ghost dual graph: : 3.79086e-05
Process 8: Time to build ghost dual graph: : 6.69956e-05
Process 10: Time to build local boundary vertex-cell connectivity map: 0.000164032
Process 12: Time to build ghost dual graph: : 5.79357e-05
Process 4: Time to build ghost boundary vertex-cell connectivity map: 0.000209093
Process 2: Time to build ghost boundary vertex-cell connectivity map: 0.00016284
Process 4: Time to build ghost dual graph: : 4.31538e-05
Process 2: Time to build ghost dual graph: : 3.60012e-05
Process 9: Time to build local boundary vertex-cell connectivity map: 0.000259161
Process 9: Time to build ghost boundary vertex-cell connectivity map: 0.000241041
Process 9: Time to build ghost dual graph: : 5.91278e-05
Process 9: Time to build local boundary vertex-cell connectivity map: 0.000263214
Process 9: Time to build ghost boundary vertex-cell connectivity map: 0.000218868
Process 9: Time to build ghost dual graph: : 5.6982e-05
Process 10: Time to build ghost boundary vertex-cell connectivity map: 0.000140905
Process 8: Time to build local boundary vertex-cell connectivity map: 0.000154972
Process 1: Time to build local boundary vertex-cell connectivity map: 0.000247002
Process 1: Time to build ghost boundary vertex-cell connectivity map: 0.000226974
Process 1: Time to build ghost dual graph: : 0.00033617
Process 1: Time to build local boundary vertex-cell connectivity map: 0.000267982
Process 10: Time to build ghost dual graph: : 3.69549e-05
Process 3: Time to build ghost boundary vertex-cell connectivity map: 0.000205994
Process 3: Time to build ghost dual graph: : 4.19617e-05
Process 3: Time to build local boundary vertex-cell connectivity map: 0.000163078
Process 3: Time to build ghost boundary vertex-cell connectivity map: 0.000141859
Process 3: Time to build ghost dual graph: : 3.98159e-05
Process 3: Time to build local boundary vertex-cell connectivity map: 0.000160933
Process 3: Time to build ghost boundary vertex-cell connectivity map: 0.000140905
Process 3: Time to build ghost dual graph: : 3.79086e-05
Process 2: Time to build local boundary vertex-cell connectivity map: 0.000149965
Process 11: Time to build local boundary vertex-cell connectivity map: 0.000261068
Process 11: Time to build ghost boundary vertex-cell connectivity map: 0.000251055
Process 11: Time to build ghost dual graph: : 5.29289e-05
Process 11: Time to build local boundary vertex-cell connectivity map: 0.000264883
Process 11: Time to build ghost boundary vertex-cell connectivity map: 0.000228882
Process 11: Time to build ghost dual graph: : 5.19753e-05
Process 0: Time to build ghost dual graph: : 0.000372171
Process 4: Time to build local boundary vertex-cell connectivity map: 0.00016284
Process 5: Time to build local boundary vertex-cell connectivity map: 0.000267029
Process 5: Time to build ghost boundary vertex-cell connectivity map: 0.000241995
Process 5: Time to build ghost dual graph: : 5.10216e-05
Process 5: Time to build local boundary vertex-cell connectivity map: 0.000245094
Process 5: Time to build ghost boundary vertex-cell connectivity map: 0.000269175
Process 12: Time to build local boundary vertex-cell connectivity map: 0.000247002
Process 8: Time to build ghost boundary vertex-cell connectivity map: 0.000141144
Process 8: Time to build ghost dual graph: : 3.60012e-05
Process 10: Time to build local boundary vertex-cell connectivity map: 0.000151873
Process 2: Time to build ghost boundary vertex-cell connectivity map: 0.000141144
Process 4: Time to build ghost boundary vertex-cell connectivity map: 0.000138998
Process 2: Time to build ghost dual graph: : 3.69549e-05
Process 4: Time to build ghost dual graph: : 0.000259876
Process 4: Time to build local boundary vertex-cell connectivity map: 0.000159979
Process 10: Time to build ghost boundary vertex-cell connectivity map: 0.000153065
Process 10: Time to build ghost dual graph: : 3.8147e-05
Process 10: Time to build local boundary vertex-cell connectivity map: 0.000151873
Process 10: Time to build ghost boundary vertex-cell connectivity map: 0.000140905
Process 8: Time to build local boundary vertex-cell connectivity map: 0.000151157
Process 8: Time to build ghost boundary vertex-cell connectivity map: 0.000150919
Process 8: Time to build ghost dual graph: : 3.60012e-05
Process 8: Time to build local boundary vertex-cell connectivity map: 0.000150919
Process 0: Time to build local boundary vertex-cell connectivity map: 0.000241995
Process 0: Time to build ghost boundary vertex-cell connectivity map: 0.000255108
Process 0: Time to build ghost dual graph: : 4.81606e-05
Process 12: Time to build ghost boundary vertex-cell connectivity map: 0.000221014
Process 12: Time to build ghost dual graph: : 4.22001e-05
Process 12: Time to build local boundary vertex-cell connectivity map: 0.000154018
Process 12: Time to build ghost boundary vertex-cell connectivity map: 0.000145912
Process 2: Time to build local boundary vertex-cell connectivity map: 0.000149012
Process 2: Time to build ghost boundary vertex-cell connectivity map: 0.000152111
Process 2: Time to build ghost dual graph: : 3.60012e-05
Process 12: Time to build ghost dual graph: : 5.50747e-05
Process 10: Time to build ghost dual graph: : 4.91142e-05
Process 2: Time to build local boundary vertex-cell connectivity map: 0.000162125
Process 4: Time to build ghost boundary vertex-cell connectivity map: 0.000157118
Process 8: Time to build ghost boundary vertex-cell connectivity map: 0.000162125
Process 8: Time to build ghost dual graph: : 3.60012e-05
Process 10: Time to build local boundary vertex-cell connectivity map: 0.000153065
Process 11: Time to build local boundary vertex-cell connectivity map: 0.000274897
Process 11: Time to build ghost boundary vertex-cell connectivity map: 0.000239134
Process 11: Time to build ghost dual graph: : 5.38826e-05
Process 11: Time to build local boundary vertex-cell connectivity map: 0.000257015
Process 2: Time to build ghost boundary vertex-cell connectivity map: 0.000151873
Process 0: Time to build local boundary vertex-cell connectivity map: 0.000243187
Process 2: Time to build ghost dual graph: : 3.69549e-05
Process 9: Time to build local boundary vertex-cell connectivity map: 0.000255108
Process 9: Time to build ghost boundary vertex-cell connectivity map: 0.000237226
Process 9: Time to build ghost dual graph: : 5.88894e-05
Process 9: Time to build local boundary vertex-cell connectivity map: 0.000252008
Process 1: Time to build ghost boundary vertex-cell connectivity map: 0.000255108
Process 1: Time to build ghost dual graph: : 0.000386
Process 1: Time to build local boundary vertex-cell connectivity map: 0.000264883
Process 4: Time to build ghost dual graph: : 0.000227928
Process 3: Time to build local boundary vertex-cell connectivity map: 0.000208139
Process 3: Time to build ghost boundary vertex-cell connectivity map: 0.000156164
Process 3: Time to build ghost dual graph: : 4.00543e-05
Process 3: Time to build local boundary vertex-cell connectivity map: 0.000161886
Process 3: Time to build ghost boundary vertex-cell connectivity map: 0.000142097
Process 3: Time to build ghost dual graph: : 3.91006e-05
Process 3: Time to build local boundary vertex-cell connectivity map: 0.000160933
Process 8: Time to build local boundary vertex-cell connectivity map: 0.000143051
Process 12: Time to build local boundary vertex-cell connectivity map: 0.000244856
Process 10: Time to build ghost boundary vertex-cell connectivity map: 0.000138044
Process 7: Compute graph ghost edges.
Process 7: Time to build local boundary vertex-cell connectivity map: 0.000249147
Process 2: Time to build local boundary vertex-cell connectivity map: 0.000149965
Process 5: Time to build ghost dual graph: : 4.91142e-05
Process 5: Time to build local boundary vertex-cell connectivity map: 0.000252008
Process 5: Time to build ghost boundary vertex-cell connectivity map: 0.000223875
Process 5: Time to build ghost dual graph: : 4.88758e-05
Process 5: Time to build local boundary vertex-cell connectivity map: 0.000235081
Process 6: Compute graph ghost edges.
Process 13: Time to build local boundary vertex-cell connectivity map: 0.00240016
Process 0: Time to build ghost boundary vertex-cell connectivity map: 0.000234127
Process 8: Time to build ghost boundary vertex-cell connectivity map: 0.000137806
Process 8: Time to build ghost dual graph: : 3.60012e-05
Process 4: Time to build local boundary vertex-cell connectivity map: 0.000159979
Process 4: Time to build ghost boundary vertex-cell connectivity map: 0
Process 11: Time to build ghost boundary vertex-cell connectivity map: 0.000255108
Process 4: Time to build ghost dual graph: : 3.79086e-05
Process 0: Time to build ghost dual graph: : 4.91142e-05
Process 2: Time to build ghost boundary vertex-cell connectivity map: 0.000139952
Process 7: Time to build ghost boundary vertex-cell connectivity map: 0.000253201
Process 10: Time to build ghost dual graph: : 0.000254154
Process 2: Time to build ghost dual graph: : 3.60012e-05
Process 12: Time to build ghost boundary vertex-cell connectivity map: 0.000232935
Process 9: Time to build ghost boundary vertex-cell connectivity map: 0.000240088
Process 12: Time to build ghost dual graph: : 5.50747e-05
Process 3: Time to build ghost boundary vertex-cell connectivity map: 0.000203848
Process 6: Time to build local boundary vertex-cell connectivity map: 0.000274897
Process 4: Time to build local boundary vertex-cell connectivity map: 0.000154972
Process 8: Time to build local boundary vertex-cell connectivity map: 0.000149965
Process 11: Time to build ghost dual graph: : 5.60284e-05
Process 10: Time to build local boundary vertex-cell connectivity map: 0.000150919
Process 2: Time to build local boundary vertex-cell connectivity map: 0.000150204
Process 3: Time to build ghost dual graph: : 5.6982e-05
Process 0: Time to build local boundary vertex-cell connectivity map: 0.000238895
Process 7: Time to build ghost dual graph: : 5.07832e-05
Process 8: Time to build ghost boundary vertex-cell connectivity map: 0.000149012
Process 4: Time to build ghost boundary vertex-cell connectivity map: 0.000135899
Process 9: Time to build ghost dual graph: : 5.79357e-05
Process 5: Time to build ghost boundary vertex-cell connectivity map: 0.000246048
Process 2: Time to build ghost boundary vertex-cell connectivity map: 0.000151873
Process 10: Time to build ghost boundary vertex-cell connectivity map: 0.000149965
Process 13: Time to build ghost boundary vertex-cell connectivity map: 0.000174999
Process 12: Time to build local boundary vertex-cell connectivity map: 0.000246048
Process 6: Time to build ghost boundary vertex-cell connectivity map: 0.000252962
Process 2: Time to build ghost dual graph: : 3.88622e-05
Process 1: Time to build ghost boundary vertex-cell connectivity map: 0.000268936
Process 1: Time to build ghost dual graph: : 5.60284e-05
Process 6: Time to build ghost dual graph: : 5.60284e-05
Process 0: Time to build ghost boundary vertex-cell connectivity map: 0.000225067
Process 13: Time to build ghost dual graph: : 4.50611e-05
Process 4: Time to build ghost dual graph: : 0.000228167
Process 8: Time to build ghost dual graph: : 0.000256062
Process 0: Time to build ghost dual graph: : 4.88758e-05
Process 2: Time to build local boundary vertex-cell connectivity map: 0.000152111
Process 10: Time to build ghost dual graph: : 0.000235081
Process 12: Time to build ghost boundary vertex-cell connectivity map: 0.000218153
Process 9: Time to build local boundary vertex-cell connectivity map: 0.000257969
Process 9: Time to build ghost boundary vertex-cell connectivity map: 0.000225067
Process 9: Time to build ghost dual graph: : 5.81741e-05
Process 12: Time to build ghost dual graph: : 5.29289e-05
Process 11: Time to build local boundary vertex-cell connectivity map: 0.000264883
Process 11: Time to build ghost boundary vertex-cell connectivity map: 0.000226021
Process 11: Time to build ghost dual graph: : 5.50747e-05
Process 4: Time to build local boundary vertex-cell connectivity map: 0.000159979
Process 8: Time to build local boundary vertex-cell connectivity map: 0.000150919
Process 5: Time to build ghost dual graph: : 0.000386953
Process 5: Time to build local boundary vertex-cell connectivity map: 0.000235796
Process 2: Time to build ghost boundary vertex-cell connectivity map: 0.000139952
Process 2: Time to build ghost dual graph: : 3.60012e-05
Process 3: Time to build local boundary vertex-cell connectivity map: 0.000204086
Process 3: Time to build ghost boundary vertex-cell connectivity map: 0.000156164
Process 3: Time to build ghost dual graph: : 4.1008e-05
Process 3: Time to build local boundary vertex-cell connectivity map: 0.000153065
Process 3: Time to build ghost boundary vertex-cell connectivity map: 0.000154972
Process 10: Time to build local boundary vertex-cell connectivity map: 0.000152111
Process 10: Time to build ghost boundary vertex-cell connectivity map: 9.53674e-07
Process 10: Time to build ghost dual graph: : 3.91006e-05
Process 1: Time to build local boundary vertex-cell connectivity map: 0.000267029
Process 1: Time to build ghost boundary vertex-cell connectivity map: 0.000231028
Process 1: Time to build ghost dual graph: : 5.50747e-05
Process 0: Time to build local boundary vertex-cell connectivity map: 0.000221014
Process 2: Finish compute graph ghost edges.
Process 6: Time to build local boundary vertex-cell connectivity map: 0.000297785
Process 4: Time to build ghost boundary vertex-cell connectivity map: 0.000153065
Process 8: Time to build ghost boundary vertex-cell connectivity map: 0.000144005
Process 7: Time to build local boundary vertex-cell connectivity map: 0.000228167
Process 7: Time to build ghost boundary vertex-cell connectivity map: 0.000241041
Process 7: Time to build ghost dual graph: : 5.19753e-05
Process 7: Time to build local boundary vertex-cell connectivity map: 0.00023818
Process 12: Time to build local boundary vertex-cell connectivity map: 0.000229836
Process 13: Time to build local boundary vertex-cell connectivity map: 0.000151157
Process 13: Time to build ghost boundary vertex-cell connectivity map: 0.000142097
Process 13: Time to build ghost dual graph: : 3.69549e-05
Process 13: Time to build local boundary vertex-cell connectivity map: 0.000149012
Process 13: Time to build ghost boundary vertex-cell connectivity map: 0.000169992
Process 13: Time to build ghost dual graph: : 3.91006e-05
Process 10: Time to build local boundary vertex-cell connectivity map: 0.000168085
Process 2: End compute dual graph.
Process 2: Start to compute partitions using SCOTCH
Process 9: Time to build local boundary vertex-cell connectivity map: 0.000260115
Process 8: Time to build ghost dual graph: : 0.000174999
Process 0: Time to build ghost boundary vertex-cell connectivity map: 0.000236988
Process 6: Time to build ghost boundary vertex-cell connectivity map: 0.000237942
Process 11: Time to build local boundary vertex-cell connectivity map: 0.000279188
Process 0: Time to build ghost dual graph: : 5.10216e-05
Process 4: Time to build ghost dual graph: : 0.000252962
Process 3: Time to build ghost dual graph: : 5.38826e-05
Process 6: Time to build ghost dual graph: : 6.19888e-05
Process 10: Time to build ghost boundary vertex-cell connectivity map: 0.000146151
Process 12: Time to build ghost boundary vertex-cell connectivity map: 0.000214815
Process 8: Time to build local boundary vertex-cell connectivity map: 0.000144005
Process 8: Time to build ghost boundary vertex-cell connectivity map: 0
Process 12: Time to build ghost dual graph: : 5.29289e-05
Process 8: Time to build ghost dual graph: : 3.91006e-05
Process 4: Time to build local boundary vertex-cell connectivity map: 0.000159979
Process 0: Time to build local boundary vertex-cell connectivity map: 0.000235081
Process 10: Time to build ghost dual graph: : 0.000226021
Process 8: Time to build local boundary vertex-cell connectivity map: 0.000148058
Process 4: Time to build ghost boundary vertex-cell connectivity map: 0.000147104
Process 6: Time to build local boundary vertex-cell connectivity map: 0.000267029
Process 7: Time to build ghost boundary vertex-cell connectivity map: 0.000218153
Process 7: Time to build ghost dual graph: : 4.91142e-05
Process 7: Time to build local boundary vertex-cell connectivity map: 0.000233889
Process 7: Time to build ghost boundary vertex-cell connectivity map: 0.000240088
Process 12: Time to build local boundary vertex-cell connectivity map: 0.000217199
Process 4: Time to build ghost dual graph: : 3.88622e-05
Process 10: Time to build local boundary vertex-cell connectivity map: 0.000152111
Process 13: Time to build local boundary vertex-cell connectivity map: 0.000148058
Process 13: Time to build ghost boundary vertex-cell connectivity map: 0.000154018
Process 13: Time to build ghost dual graph: : 3.69549e-05
Process 13: Time to build local boundary vertex-cell connectivity map: 0.000148058
Process 13: Time to build ghost boundary vertex-cell connectivity map: 0.000143051
Process 13: Time to build ghost dual graph: : 3.60012e-05
Process 8: Time to build ghost boundary vertex-cell connectivity map: 0.000146866
Process 9: Time to build ghost boundary vertex-cell connectivity map: 0.000241995
Process 9: Time to build ghost dual graph: : 6.10352e-05
Process 9: Time to build local boundary vertex-cell connectivity map: 0.000255108
Process 0: Time to build ghost boundary vertex-cell connectivity map: 0.000217915
Process 5: Time to build ghost boundary vertex-cell connectivity map: 0.000247955
Process 5: Time to build ghost dual graph: : 0.000341892
Process 5: Time to build local boundary vertex-cell connectivity map: 0.000241041
Process 5: Time to build ghost boundary vertex-cell connectivity map: 0
Process 5: Time to build ghost dual graph: : 4.91142e-05
Process 15: Compute graph ghost edges.
Process 15: Time to build local boundary vertex-cell connectivity map: 0.000257969
Process 0: Time to build ghost dual graph: : 4.81606e-05
Process 4: Time to build local boundary vertex-cell connectivity map: 0.000160933
Process 11: Time to build ghost boundary vertex-cell connectivity map: 0.000244141
Process 11: Time to build ghost dual graph: : 5.81741e-05
Process 11: Time to build local boundary vertex-cell connectivity map: 0.000259161
Process 10: Time to build ghost boundary vertex-cell connectivity map: 0.000151157
Process 12: Time to build ghost boundary vertex-cell connectivity map: 0.000234842
Process 6: Time to build ghost boundary vertex-cell connectivity map: 0.00027585
Process 1: Time to build local boundary vertex-cell connectivity map: 0.000277996
Process 1: Time to build ghost boundary vertex-cell connectivity map: 0.000246048
Process 1: Time to build ghost dual graph: : 5.4121e-05
Process 1: Time to build local boundary vertex-cell connectivity map: 0.000248909
Process 6: Time to build ghost dual graph: : 5.48363e-05
Process 8: Time to build ghost dual graph: : 0.000221968
Process 3: Time to build local boundary vertex-cell connectivity map: 0.000181913
Process 3: Time to build ghost boundary vertex-cell connectivity map: 0.000152826
Process 3: Time to build ghost dual graph: : 3.98159e-05
Process 3: Time to build local boundary vertex-cell connectivity map: 0.000162125
Process 3: Time to build ghost boundary vertex-cell connectivity map: 0.000139952
Process 3: Time to build ghost dual graph: : 5.19753e-05
Process 4: Time to build ghost boundary vertex-cell connectivity map: 0.000139952
Process 4: Time to build ghost dual graph: : 3.8147e-05
Process 12: Time to build ghost dual graph: : 5.57899e-05
Process 7: Time to build ghost dual graph: : 5.00679e-05
Process 9: Time to build ghost boundary vertex-cell connectivity map: 0.000217199
Process 15: Time to build ghost boundary vertex-cell connectivity map: 0.000255823
Process 15: Time to build ghost dual graph: : 5.50747e-05
Process 11: Time to build ghost boundary vertex-cell connectivity map: 0.000225067
Process 11: Time to build ghost dual graph: : 5.79357e-05
Process 8: Time to build local boundary vertex-cell connectivity map: 0.000149965
Process 0: Time to build local boundary vertex-cell connectivity map: 0.000249147
Process 10: Time to build ghost dual graph: : 0.000242949
Process 13: Time to build local boundary vertex-cell connectivity map: 0.000148058
Process 4: Time to build local boundary vertex-cell connectivity map: 0.000159979
Process 10: Time to build local boundary vertex-cell connectivity map: 0.000152111
Process 12: Time to build local boundary vertex-cell connectivity map: 0.000249147
Process 12: Time to build ghost boundary vertex-cell connectivity map: 0.000146866
Process 8: Time to build ghost boundary vertex-cell connectivity map: 0.000138998
Process 0: Time to build ghost boundary vertex-cell connectivity map: 0.000221014
Process 0: Time to build ghost dual graph: : 4.60148e-05
Process 6: Time to build local boundary vertex-cell connectivity map: 0.000249863
Process 4: Time to build ghost boundary vertex-cell connectivity map: 0.000153065
Process 4: Time to build ghost dual graph: : 3.88622e-05
Process 10: Time to build ghost boundary vertex-cell connectivity map: 0.000139952
Process 10: Time to build ghost dual graph: : 3.79086e-05
Process 8: Time to build ghost dual graph: : 0.000247002
Process 14: Time to build local boundary vertex-cell connectivity map: 0.00630808
Process 1: Time to build ghost boundary vertex-cell connectivity map: 0.000228167
Process 1: Time to build ghost dual graph: : 5.60284e-05
Process 1: Time to build local boundary vertex-cell connectivity map: 0.000261068
Process 1: Time to build ghost boundary vertex-cell connectivity map: 0.000224113
Process 1: Time to build ghost dual graph: : 5.4121e-05
Process 4: Time to build local boundary vertex-cell connectivity map: 0.000165939
Process 11: Time to build local boundary vertex-cell connectivity map: 0.000250816
Process 11: Time to build ghost boundary vertex-cell connectivity map: 0.000226021
Process 11: Time to build ghost dual graph: : 5.19753e-05
Process 7: Time to build local boundary vertex-cell connectivity map: 0.000239849
Process 7: Time to build ghost boundary vertex-cell connectivity map: 0.000233889
Process 7: Time to build ghost dual graph: : 5.00679e-05
Process 7: Time to build local boundary vertex-cell connectivity map: 0.000221014
Process 10: Time to build local boundary vertex-cell connectivity map: 0.000157833
Process 0: Time to build local boundary vertex-cell connectivity map: 0.000231028
Process 5: Time to build local boundary vertex-cell connectivity map: 0.000239134
Process 5: Time to build ghost boundary vertex-cell connectivity map: 0.000235796
Process 5: Time to build ghost dual graph: : 0.000323057
Process 8: Time to build local boundary vertex-cell connectivity map: 0.000152111
Process 9: Time to build ghost dual graph: : 0.000375032
Process 9: Time to build local boundary vertex-cell connectivity map: 0.000253916
Process 9: Time to build ghost boundary vertex-cell connectivity map: 0.000230074
Process 4: Time to build ghost boundary vertex-cell connectivity map: 0.000140905
Process 15: Time to build local boundary vertex-cell connectivity map: 0.000273943
Process 15: Time to build ghost boundary vertex-cell connectivity map: 0.000259876
Process 15: Time to build ghost dual graph: : 5.19753e-05
Process 4: Time to build ghost dual graph: : 3.79086e-05
Process 13: Time to build ghost boundary vertex-cell connectivity map: 0.000160933
Process 13: Time to build ghost dual graph: : 3.48091e-05
Process 13: Time to build local boundary vertex-cell connectivity map: 0.000149012
Process 13: Time to build ghost boundary vertex-cell connectivity map: 0.000142097
Process 13: Time to build ghost dual graph: : 3.38554e-05
Process 13: Time to build local boundary vertex-cell connectivity map: 0.000149012
Process 13: Time to build ghost boundary vertex-cell connectivity map: 0.000142813
Process 13: Time to build ghost dual graph: : 3.60012e-05
Process 14: Time to build ghost boundary vertex-cell connectivity map: 0.000261068
Process 12: Time to build ghost dual graph: : 0.000372887
Process 10: Time to build ghost boundary vertex-cell connectivity map: 0.000151157
Process 11: Time to build local boundary vertex-cell connectivity map: 0.000256062
Process 10: Time to build ghost dual graph: : 3.79086e-05
Process 8: Time to build ghost boundary vertex-cell connectivity map: 0.00015378
Process 8: Time to build ghost dual graph: : 3.69549e-05
Process 14: Time to build ghost dual graph: : 5.48363e-05
Process 0: Time to build ghost boundary vertex-cell connectivity map: 0.000241995
Process 4: Time to build local boundary vertex-cell connectivity map: 0.000159979
Process 0: Time to build ghost dual graph: : 4.91142e-05
Process 10: Time to build local boundary vertex-cell connectivity map: 0.000152826
Process 4: Time to build ghost boundary vertex-cell connectivity map: 0.000150919
Process 4: Time to build ghost dual graph: : 4.00543e-05
Process 4: Time to build local boundary vertex-cell connectivity map: 0.000161886
Process 0: Time to build local boundary vertex-cell connectivity map: 0.000233889
Process 8: Time to build local boundary vertex-cell connectivity map: 0.000150919
Process 8: Time to build ghost boundary vertex-cell connectivity map: 0.000152111
Process 8: Time to build ghost dual graph: : 3.60012e-05
Process 14: Time to build local boundary vertex-cell connectivity map: 0.000258923
Process 12: Time to build local boundary vertex-cell connectivity map: 0.000235081
Process 12: Time to build ghost boundary vertex-cell connectivity map: 0.000149012
Process 10: Time to build ghost boundary vertex-cell connectivity map: 0.000139952
Process 10: Time to build ghost dual graph: : 3.69549e-05
Process 10: Finish compute graph ghost edges.
Process 8: Time to build local boundary vertex-cell connectivity map: 0.000151157
Process 14: Time to build ghost boundary vertex-cell connectivity map: 0.000257969
Process 10: End compute dual graph.
Process 10: Start to compute partitions using SCOTCH
Process 12: Time to build ghost dual graph: : 0.000258923
Process 4: Time to build ghost boundary vertex-cell connectivity map: 0.000152826
Process 0: Time to build ghost boundary vertex-cell connectivity map: 0.000224829
Process 14: Time to build ghost dual graph: : 5.60284e-05
Process 4: Time to build ghost dual graph: : 4.1008e-05
Process 0: Time to build ghost dual graph: : 5.19753e-05
Process 15: Time to build local boundary vertex-cell connectivity map: 0.000258923
Process 15: Time to build ghost boundary vertex-cell connectivity map: 0.000231028
Process 15: Time to build ghost dual graph: : 4.88758e-05
Process 15: Time to build local boundary vertex-cell connectivity map: 0.000231981
Process 15: Time to build ghost boundary vertex-cell connectivity map: 0.000239134
Process 8: Time to build ghost boundary vertex-cell connectivity map: 0.000146151
Process 1: Time to build local boundary vertex-cell connectivity map: 0.000252008
Process 1: Time to build ghost boundary vertex-cell connectivity map: 0.000236034
Process 1: Time to build ghost dual graph: : 5.50747e-05
Process 1: Time to build local boundary vertex-cell connectivity map: 0.000257015
Process 8: Time to build ghost dual graph: : 3.60012e-05
Process 13: Time to build local boundary vertex-cell connectivity map: 0.000154972
Process 13: Time to build ghost boundary vertex-cell connectivity map: 0.000157118
Process 13: Time to build ghost dual graph: : 3.60012e-05
Process 13: Time to build local boundary vertex-cell connectivity map: 0.000149012
Process 13: Time to build ghost boundary vertex-cell connectivity map: 0.000144005
Process 13: Time to build ghost dual graph: : 3.60012e-05
Process 13: Time to build local boundary vertex-cell connectivity map: 0.00014782
Process 7: Time to build ghost boundary vertex-cell connectivity map: 0.000218153
Process 7: Time to build ghost dual graph: : 0.0003829
Process 7: Time to build local boundary vertex-cell connectivity map: 0.000230074
Process 5: Time to build local boundary vertex-cell connectivity map: 0.000241995
Process 5: Time to build ghost boundary vertex-cell connectivity map: 0.000215054
Process 5: Time to build ghost dual graph: : 0.000371933
Process 5: Time to build local boundary vertex-cell connectivity map: 0.000236034
Process 4: Time to build local boundary vertex-cell connectivity map: 0.000161886
Process 9: Time to build ghost dual graph: : 0.000332117
Process 9: Time to build local boundary vertex-cell connectivity map: 0.000246048
Process 9: Time to build ghost boundary vertex-cell connectivity map: 0
Process 9: Time to build ghost dual graph: : 5.29289e-05
Process 9: Time to build local boundary vertex-cell connectivity map: 0.000247955
Process 11: Time to build ghost boundary vertex-cell connectivity map: 0.000237942
Process 11: Time to build ghost dual graph: : 0.000383854
Process 11: Time to build local boundary vertex-cell connectivity map: 0.000258923
Process 12: Time to build local boundary vertex-cell connectivity map: 0.000247002
Process 12: Time to build ghost boundary vertex-cell connectivity map: 0
Process 8: Time to build local boundary vertex-cell connectivity map: 0.000144005
Process 15: Time to build ghost dual graph: : 6.48499e-05
Process 0: Time to build local boundary vertex-cell connectivity map: 0.000224829
Process 1: Time to build ghost boundary vertex-cell connectivity map: 0.000219107
Process 14: Time to build local boundary vertex-cell connectivity map: 0.000265837
Process 4: Time to build ghost boundary vertex-cell connectivity map: 0.000139952
Process 4: Time to build ghost dual graph: : 4.00543e-05
Process 12: Time to build ghost dual graph: : 5.29289e-05
Process 7: Time to build ghost boundary vertex-cell connectivity map: 0.000237942
Process 8: Time to build ghost boundary vertex-cell connectivity map: 0.000150919
Process 8: Time to build ghost dual graph: : 3.79086e-05
Process 4: Time to build local boundary vertex-cell connectivity map: 0.000161171
Process 4: Time to build ghost boundary vertex-cell connectivity map: 0.000150919
Process 4: Time to build ghost dual graph: : 3.98159e-05
Process 12: Time to build local boundary vertex-cell connectivity map: 0.000219107
Process 12: Time to build ghost boundary vertex-cell connectivity map: 0.000136852
Process 14: Time to build ghost boundary vertex-cell connectivity map: 0.000222921
Process 14: Time to build ghost dual graph: : 5.50747e-05
Process 0: Time to build ghost boundary vertex-cell connectivity map: 0.000236988
Process 0: Time to build ghost dual graph: : 5.19753e-05
Process 8: Time to build local boundary vertex-cell connectivity map: 0.000152111
Process 8: Time to build ghost boundary vertex-cell connectivity map: 0.000138998
Process 8: Time to build ghost dual graph: : 3.60012e-05
Process 0: Time to build local boundary vertex-cell connectivity map: 0.000236034
Process 8: Finish compute graph ghost edges.
Process 14: Time to build local boundary vertex-cell connectivity map: 0.000264883
Process 4: Time to build local boundary vertex-cell connectivity map: 0.000163078
Process 12: Time to build ghost dual graph: : 0.000277042
Process 9: Time to build ghost boundary vertex-cell connectivity map: 0.00022006
Process 9: Time to build ghost dual graph: : 0.000331879
Process 9: Time to build local boundary vertex-cell connectivity map: 0.000257969
Process 8: End compute dual graph.
Process 8: Start to compute partitions using SCOTCH
Process 11: Time to build ghost boundary vertex-cell connectivity map: 0.000224113
Process 11: Time to build ghost dual graph: : 0.000337839
Process 11: Time to build local boundary vertex-cell connectivity map: 0.000259161
Process 11: Time to build ghost boundary vertex-cell connectivity map: 0
Process 11: Time to build ghost dual graph: : 5.19753e-05
Process 15: Time to build local boundary vertex-cell connectivity map: 0.000264883
Process 15: Time to build ghost boundary vertex-cell connectivity map: 0.000256062
Process 15: Time to build ghost dual graph: : 5.07832e-05
Process 15: Time to build local boundary vertex-cell connectivity map: 0.000230789
Process 4: Time to build ghost boundary vertex-cell connectivity map: 0.000144005
Process 13: Time to build ghost boundary vertex-cell connectivity map: 0.000152111
Process 13: Time to build ghost dual graph: : 0.000261068
Process 13: Time to build local boundary vertex-cell connectivity map: 0.000149012
Process 13: Time to build ghost boundary vertex-cell connectivity map: 0.000155926
Process 13: Time to build ghost dual graph: : 0.000229836
Process 0: Time to build ghost boundary vertex-cell connectivity map: 0.000240088
Process 4: Time to build ghost dual graph: : 3.91006e-05
Process 1: Time to build ghost dual graph: : 5.60284e-05
Process 1: Time to build local boundary vertex-cell connectivity map: 0.000260115
Process 1: Time to build ghost boundary vertex-cell connectivity map: 0.000237942
Process 1: Time to build ghost dual graph: : 5.6982e-05
Process 4: Finish compute graph ghost edges.
Process 14: Time to build ghost boundary vertex-cell connectivity map: 0.000267029
Process 0: Time to build ghost dual graph: : 4.88758e-05
Process 5: Time to build ghost boundary vertex-cell connectivity map: 0.000222921
Process 5: Time to build ghost dual graph: : 4.79221e-05
Process 5: Time to build local boundary vertex-cell connectivity map: 0.000240088
Process 5: Time to build ghost boundary vertex-cell connectivity map: 0.000243187
Process 5: Time to build ghost dual graph: : 5.10216e-05
Process 7: Time to build ghost dual graph: : 0.000330925
Process 7: Time to build local boundary vertex-cell connectivity map: 0.000241041
Process 7: Time to build ghost boundary vertex-cell connectivity map: 0
Process 7: Time to build ghost dual graph: : 5.00679e-05
Process 7: Time to build local boundary vertex-cell connectivity map: 0.000230074
Process 14: Time to build ghost dual graph: : 5.60284e-05
Process 1: Time to build local boundary vertex-cell connectivity map: 0.000256062
Process 12: Time to build local boundary vertex-cell connectivity map: 0.000244856
Process 0: Time to build local boundary vertex-cell connectivity map: 0.000242949
Process 0: Time to build ghost boundary vertex-cell connectivity map: 0.000224113
Process 0: Time to build ghost dual graph: : 4.60148e-05
Process 4: End compute dual graph.
Process 4: Start to compute partitions using SCOTCH
Process 14: Time to build local boundary vertex-cell connectivity map: 0.000257969
Process 12: Time to build ghost boundary vertex-cell connectivity map: 0.000178814
Process 12: Time to build ghost dual graph: : 0.000251055
Process 6: Time to build ghost boundary vertex-cell connectivity map: 0.00325203
Process 12: Time to build local boundary vertex-cell connectivity map: 0.000192881
Process 5: Time to build local boundary vertex-cell connectivity map: 0.000243902
Process 5: Time to build ghost boundary vertex-cell connectivity map: 0.000217915
Process 5: Time to build ghost dual graph: : 4.88758e-05
Process 5: Time to build local boundary vertex-cell connectivity map: 0.000241995
Process 6: Time to build ghost dual graph: : 7.9155e-05
Process 0: Time to build local boundary vertex-cell connectivity map: 0.000233889
Process 9: Time to build ghost boundary vertex-cell connectivity map: 0.000242233
Process 9: Time to build ghost dual graph: : 0.0003829
Process 9: Time to build local boundary vertex-cell connectivity map: 0.000248909
Process 11: Time to build local boundary vertex-cell connectivity map: 0.000251055
Process 11: Time to build ghost boundary vertex-cell connectivity map: 0.000224829
Process 11: Time to build ghost dual graph: : 0.00031805
Process 15: Time to build ghost boundary vertex-cell connectivity map: 0.000255108
Process 15: Time to build ghost dual graph: : 4.91142e-05
Process 15: Time to build local boundary vertex-cell connectivity map: 0.000249863
Process 15: Time to build ghost boundary vertex-cell connectivity map: 0.000241995
Process 15: Time to build ghost dual graph: : 4.88758e-05
Process 13: Time to build local boundary vertex-cell connectivity map: 0.000150919
Process 13: Time to build ghost boundary vertex-cell connectivity map: 0
Process 13: Time to build ghost dual graph: : 3.8147e-05
Process 13: Time to build local boundary vertex-cell connectivity map: 0.000143051
Process 13: Time to build ghost boundary vertex-cell connectivity map: 0.000152111
Process 13: Time to build ghost dual graph: : 0.000218153
Process 13: Time to build local boundary vertex-cell connectivity map: 0.000146866
Process 7: Time to build ghost boundary vertex-cell connectivity map: 0.00022316
Process 7: Time to build ghost dual graph: : 0.000251055
Process 7: Time to build local boundary vertex-cell connectivity map: 0.000235081
Process 1: Time to build ghost boundary vertex-cell connectivity map: 0.000241041
Process 1: Time to build ghost dual graph: : 5.6982e-05
Process 1: Time to build local boundary vertex-cell connectivity map: 0.000257015
Process 1: Time to build ghost boundary vertex-cell connectivity map: 0.000218153
Process 1: Time to build ghost dual graph: : 5.38826e-05
Process 12: Time to build ghost boundary vertex-cell connectivity map: 0.000218868
Process 7: Time to build ghost boundary vertex-cell connectivity map: 0.000242949
Process 5: Time to build ghost boundary vertex-cell connectivity map: 0.000244141
Process 5: Time to build ghost dual graph: : 5.10216e-05
Process 12: Time to build ghost dual graph: : 5.48363e-05
Process 13: Time to build ghost boundary vertex-cell connectivity map: 0.00014019
Process 0: Time to build ghost boundary vertex-cell connectivity map: 0.000245094
Process 9: Time to build ghost boundary vertex-cell connectivity map: 0.000252962
Process 0: Time to build ghost dual graph: : 4.81606e-05
Process 6: Time to build local boundary vertex-cell connectivity map: 0.000262976
Process 12: Finish compute graph ghost edges.
Process 11: Time to build local boundary vertex-cell connectivity map: 0.000259876
Process 12: End compute dual graph.
Process 12: Start to compute partitions using SCOTCH
Process 0: Time to build local boundary vertex-cell connectivity map: 0.000232935
Process 6: Time to build ghost boundary vertex-cell connectivity map: 0.000232935
Process 0: Time to build ghost boundary vertex-cell connectivity map: 0.000212908
Process 14: Time to build ghost boundary vertex-cell connectivity map: 0.00125885
Process 0: Time to build ghost dual graph: : 5.38826e-05
Process 14: Time to build ghost dual graph: : 5.98431e-05
Process 0: Finish compute graph ghost edges.
Process 1: Time to build local boundary vertex-cell connectivity map: 0.000254869
Process 1: Time to build ghost boundary vertex-cell connectivity map: 0.00022912
Process 1: Time to build ghost dual graph: : 5.6982e-05
Process 1: Time to build local boundary vertex-cell connectivity map: 0.00026083
Process 3: Finish compute graph ghost edges.
Process 7: Time to build ghost dual graph: : 0.000366926
Process 7: Time to build local boundary vertex-cell connectivity map: 0.000237942
Process 7: Time to build ghost boundary vertex-cell connectivity map: 0.000230074
Process 7: Time to build ghost dual graph: : 4.91142e-05
Process 13: Time to build ghost dual graph: : 0.000252962
Process 13: Finish compute graph ghost edges.
Process 9: Time to build ghost dual graph: : 5.6982e-05
Process 9: Time to build local boundary vertex-cell connectivity map: 0.000254869
Process 9: Time to build ghost boundary vertex-cell connectivity map: 0.000213861
Process 9: Time to build ghost dual graph: : 5.31673e-05
Process 9: Time to build local boundary vertex-cell connectivity map: 0.000242949
Process 15: Time to build local boundary vertex-cell connectivity map: 0.000267029
Process 15: Time to build ghost boundary vertex-cell connectivity map: 0.000305891
Process 15: Time to build ghost dual graph: : 6.29425e-05
Process 15: Time to build local boundary vertex-cell connectivity map: 0.000240088
Process 5: Time to build local boundary vertex-cell connectivity map: 0.000230074
Process 5: Time to build ghost boundary vertex-cell connectivity map: 0.000241041
Process 5: Time to build ghost dual graph: : 5.00679e-05
Process 5: Time to build local boundary vertex-cell connectivity map: 0.000236988
Process 11: Time to build ghost boundary vertex-cell connectivity map: 0.000233889
Process 11: Time to build ghost dual graph: : 0.000374079
Process 11: Time to build local boundary vertex-cell connectivity map: 0.000253916
Process 14: Time to build local boundary vertex-cell connectivity map: 0.000303984
Process 14: Time to build ghost boundary vertex-cell connectivity map: 0.000223875
Process 14: Time to build ghost dual graph: : 5.50747e-05
Process 0: End compute dual graph.
Process 0: Start to compute partitions using SCOTCH
Process 5: Time to build ghost boundary vertex-cell connectivity map: 0.000231028
Process 5: Time to build ghost dual graph: : 4.79221e-05
Process 5: Time to build local boundary vertex-cell connectivity map: 0.000241041
Process 5: Time to build ghost boundary vertex-cell connectivity map: 0.000241995
Process 5: Time to build ghost dual graph: : 5.10216e-05
Process 1: Time to build ghost boundary vertex-cell connectivity map: 0.000219107
Process 1: Time to build ghost dual graph: : 5.98431e-05
Process 7: Time to build local boundary vertex-cell connectivity map: 0.00022912
Process 7: Time to build ghost boundary vertex-cell connectivity map: 0.000240088
Process 7: Time to build ghost dual graph: : 5.10216e-05
Process 7: Time to build local boundary vertex-cell connectivity map: 0.000233173
Process 9: Time to build ghost boundary vertex-cell connectivity map: 0.000244856
Process 9: Time to build ghost dual graph: : 6.10352e-05
Process 9: Time to build local boundary vertex-cell connectivity map: 0.000262022
Process 9: Time to build ghost boundary vertex-cell connectivity map: 0.000221968
Process 9: Time to build ghost dual graph: : 5.98431e-05
Process 6: Time to build ghost dual graph: : 0.00144696
Process 15: Time to build ghost boundary vertex-cell connectivity map: 0.000258207
Process 15: Time to build ghost dual graph: : 4.79221e-05
Process 15: Time to build local boundary vertex-cell connectivity map: 0.000150204
Process 15: Time to build ghost boundary vertex-cell connectivity map: 0.000152111
Process 15: Time to build ghost dual graph: : 3.69549e-05
Process 15: Time to build local boundary vertex-cell connectivity map: 0.000149965
Process 15: Time to build ghost boundary vertex-cell connectivity map: 0.000144005
Process 15: Time to build ghost dual graph: : 4.81606e-05
Process 11: Time to build ghost boundary vertex-cell connectivity map: 0.000256062
Process 11: Time to build ghost dual graph: : 5.91278e-05
Process 11: Time to build local boundary vertex-cell connectivity map: 0.000263214
Process 11: Time to build ghost boundary vertex-cell connectivity map: 0.000231028
Process 11: Time to build ghost dual graph: : 5.6982e-05
Process 6: Time to build local boundary vertex-cell connectivity map: 0.000254869
Process 6: Time to build ghost boundary vertex-cell connectivity map: 0.000213146
Process 14: Time to build local boundary vertex-cell connectivity map: 0.0012579
Process 15: Time to build local boundary vertex-cell connectivity map: 0.000159979
Process 15: Time to build ghost boundary vertex-cell connectivity map: 0.000149965
Process 15: Time to build ghost dual graph: : 3.91006e-05
Process 15: Time to build local boundary vertex-cell connectivity map: 0.000151157
Process 15: Time to build ghost boundary vertex-cell connectivity map: 0.000152111
Process 15: Time to build ghost dual graph: : 3.79086e-05
Process 15: Time to build local boundary vertex-cell connectivity map: 0.000149012
Process 5: Time to build local boundary vertex-cell connectivity map: 0.000247002
Process 5: Time to build ghost boundary vertex-cell connectivity map: 0.000229836
Process 5: Time to build ghost dual graph: : 4.98295e-05
Process 7: Time to build ghost boundary vertex-cell connectivity map: 0.000241995
Process 7: Time to build ghost dual graph: : 5.38826e-05
Process 7: Time to build local boundary vertex-cell connectivity map: 0.000241041
Process 7: Time to build ghost boundary vertex-cell connectivity map: 0.000146151
Process 7: Time to build ghost dual graph: : 3.50475e-05
Process 7: Time to build local boundary vertex-cell connectivity map: 0.000141859
Process 7: Time to build ghost boundary vertex-cell connectivity map: 0.000149965
Process 14: Time to build ghost boundary vertex-cell connectivity map: 0.000257969
Process 14: Time to build ghost dual graph: : 5.79357e-05
Process 14: Time to build local boundary vertex-cell connectivity map: 0.000254869
Process 14: Time to build ghost boundary vertex-cell connectivity map: 0.000216961
Process 15: Time to build ghost boundary vertex-cell connectivity map: 0.000160933
Process 15: Time to build ghost dual graph: : 0.000249147
Process 15: Time to build local boundary vertex-cell connectivity map: 0.000141859
Process 15: Time to build ghost boundary vertex-cell connectivity map: 0.000149012
Process 15: Time to build ghost dual graph: : 0.000218868
Process 6: Time to build ghost dual graph: : 0.0013752
Process 7: Time to build ghost dual graph: : 3.60012e-05
Process 7: Time to build local boundary vertex-cell connectivity map: 0.000155926
Process 7: Time to build ghost boundary vertex-cell connectivity map: 0.000139952
Process 7: Time to build ghost dual graph: : 3.8147e-05
Process 6: Time to build local boundary vertex-cell connectivity map: 0.000253916
Process 6: Time to build ghost boundary vertex-cell connectivity map: 0
Process 6: Time to build ghost dual graph: : 5.29289e-05
Process 6: Time to build local boundary vertex-cell connectivity map: 0.000245094
Process 14: Time to build ghost dual graph: : 0.00106597
Process 15: Time to build local boundary vertex-cell connectivity map: 0.000174046
Process 15: Time to build ghost boundary vertex-cell connectivity map: 0
Process 15: Time to build ghost dual graph: : 3.60012e-05
Process 14: Time to build local boundary vertex-cell connectivity map: 0.000264883
Process 14: Time to build ghost boundary vertex-cell connectivity map: 0.000218868
Process 14: Time to build ghost dual graph: : 5.50747e-05
Process 14: Time to build local boundary vertex-cell connectivity map: 0.000252008
Process 6: Time to build ghost boundary vertex-cell connectivity map: 0.00122499
Process 6: Time to build ghost dual graph: : 0.000331163
Process 6: Time to build local boundary vertex-cell connectivity map: 0.000252008
Process 6: Time to build ghost boundary vertex-cell connectivity map: 0.000222206
Process 14: Time to build ghost boundary vertex-cell connectivity map: 0.00125289
Process 14: Time to build ghost dual graph: : 5.6982e-05
Process 14: Time to build local boundary vertex-cell connectivity map: 0.000253916
Process 14: Time to build ghost boundary vertex-cell connectivity map: 0.000221014
Process 14: Time to build ghost dual graph: : 5.38826e-05
Process 6: Time to build ghost dual graph: : 0.00137806
Process 6: Time to build local boundary vertex-cell connectivity map: 0.000252008
Process 6: Time to build ghost boundary vertex-cell connectivity map: 0.000244856
Process 6: Time to build ghost dual graph: : 5.50747e-05
Process 14: Time to build local boundary vertex-cell connectivity map: 0.00125813
Process 14: Time to build ghost boundary vertex-cell connectivity map: 0.00023818
Process 14: Time to build ghost dual graph: : 5.6982e-05
Process 14: Time to build local boundary vertex-cell connectivity map: 0.000254154
Process 6: Time to build local boundary vertex-cell connectivity map: 0.00127292
Process 6: Time to build ghost boundary vertex-cell connectivity map: 0.00022006
Process 6: Time to build ghost dual graph: : 5.4121e-05
Process 6: Time to build local boundary vertex-cell connectivity map: 0.000253916
Process 14: Time to build ghost boundary vertex-cell connectivity map: 0.00124383
Process 14: Time to build ghost dual graph: : 0.000403881
Process 14: Time to build local boundary vertex-cell connectivity map: 0.000242949
Process 14: Time to build ghost boundary vertex-cell connectivity map: 0.000211954
Process 6: Time to build ghost boundary vertex-cell connectivity map: 0.00125098
Process 6: Time to build ghost dual graph: : 5.60284e-05
Process 6: Time to build local boundary vertex-cell connectivity map: 0.000256062
Process 6: Time to build ghost boundary vertex-cell connectivity map: 0.000242949
Process 6: Time to build ghost dual graph: : 5.6982e-05
Process 14: Time to build ghost dual graph: : 0.00136209
Process 14: Time to build local boundary vertex-cell connectivity map: 0.000254154
Process 14: Time to build ghost boundary vertex-cell connectivity map: 0
Process 14: Time to build ghost dual graph: : 5.4121e-05
Process 14: Time to build local boundary vertex-cell connectivity map: 0.000246048
Process 6: Time to build local boundary vertex-cell connectivity map: 0.00125599
Process 6: Time to build ghost boundary vertex-cell connectivity map: 0.000217915
Process 6: Time to build ghost dual graph: : 5.31673e-05
Process 6: Time to build local boundary vertex-cell connectivity map: 0.000253916
Process 6: Time to build ghost boundary vertex-cell connectivity map: 0.000236988
Process 6: Time to build ghost dual graph: : 5.48363e-05
Process 14: Time to build ghost boundary vertex-cell connectivity map: 0.00123191
Process 14: Time to build ghost dual graph: : 0.000330925
Process 14: Finish compute graph ghost edges.
Process 14: End compute dual graph.
Process 14: Start to compute partitions using SCOTCH
Process 6: Time to build local boundary vertex-cell connectivity map: 0.000259161
Process 6: Time to build ghost boundary vertex-cell connectivity map: 0.00022006
Process 6: Time to build ghost dual graph: : 5.38826e-05
Process 6: Finish compute graph ghost edges.
Process 6: End compute dual graph.
Process 6: Start to compute partitions using SCOTCH
Process 11: Finish compute graph ghost edges.
Process 7: Finish compute graph ghost edges.
Process 1: Finish compute graph ghost edges.
Process 13: End compute dual graph.
Process 13: Start to compute partitions using SCOTCH
Process 5: Finish compute graph ghost edges.
Process 9: Finish compute graph ghost edges.
Process 15: Finish compute graph ghost edges.
Process 3: End compute dual graph.
Process 3: Start to compute partitions using SCOTCH
Process 9: End compute dual graph.
Process 9: Start to compute partitions using SCOTCH
Process 11: End compute dual graph.
Process 11: Start to compute partitions using SCOTCH
Process 15: End compute dual graph.
Process 15: Start to compute partitions using SCOTCH
Process 7: End compute dual graph.
Process 7: Start to compute partitions using SCOTCH
Process 1: End compute dual graph.
Process 1: Start to compute partitions using SCOTCH
Process 5: End compute dual graph.
Process 5: Start to compute partitions using SCOTCH
Process 12: Start SCOTCH graph building.
Process 2: Start SCOTCH graph building.
Process 4: Start SCOTCH graph building.
Process 14: Start SCOTCH graph building.
Process 6: Start SCOTCH graph building.
Process 0: Start SCOTCH graph building.
Process 10: Start SCOTCH graph building.
Process 8: Start SCOTCH graph building.
Process 3: Start SCOTCH graph building.
Process 9: Start SCOTCH graph building.
Process 11: Start SCOTCH graph building.
Process 15: Start SCOTCH graph building.
Process 7: Start SCOTCH graph building.
Process 13: Start SCOTCH graph building.
Process 1: Start SCOTCH graph building.
Process 5: Start SCOTCH graph building.
Process 0: End SCOTCH graph building.
Process 14: End SCOTCH graph building.
Process 2: End SCOTCH graph building.
Process 4: End SCOTCH graph building.
Process 10: End SCOTCH graph building.
Process 8: End SCOTCH graph building.
Process 6: End SCOTCH graph building.
Process 12: End SCOTCH graph building.
Process 1: End SCOTCH graph building.
Process 3: End SCOTCH graph building.
Process 9: End SCOTCH graph building.
Process 11: End SCOTCH graph building.
Process 15: End SCOTCH graph building.
Process 7: End SCOTCH graph building.
Process 13: End SCOTCH graph building.
Process 5: End SCOTCH graph building.
Process 8: Start SCOTCH partitioning.
Process 14: Start SCOTCH partitioning.
Process 2: Start SCOTCH partitioning.
Process 12: Start SCOTCH partitioning.
Process 0: Start SCOTCH partitioning.
Process 13: Start SCOTCH partitioning.
Process 9: Start SCOTCH partitioning.
Process 11: Start SCOTCH partitioning.
Process 15: Start SCOTCH partitioning.
Process 7: Start SCOTCH partitioning.
Process 5: Start SCOTCH partitioning.
Process 4: Start SCOTCH partitioning.
Process 6: Start SCOTCH partitioning.
Process 3: Start SCOTCH partitioning.
Process 10: Start SCOTCH partitioning.
Process 1: Start SCOTCH partitioning.
Process 2: End SCOTCH partitioning.
Process 2: Finished computing partitions using SCOTCH
Process 0: End SCOTCH partitioning.
Process 0: Finished computing partitions using SCOTCH
Process 6: End SCOTCH partitioning.
Process 6: Finished computing partitions using SCOTCH
Process 12: End SCOTCH partitioning.
Process 12: Finished computing partitions using SCOTCH
Process 4: End SCOTCH partitioning.
Process 4: Finished computing partitions using SCOTCH
Process 8: End SCOTCH partitioning.
Process 8: Finished computing partitions using SCOTCH
Process 14: End SCOTCH partitioning.
Process 14: Finished computing partitions using SCOTCH
Process 10: End SCOTCH partitioning.
Process 10: Finished computing partitions using SCOTCH
Process 7: End SCOTCH partitioning.
Process 7: Finished computing partitions using SCOTCH
Process 3: End SCOTCH partitioning.
Process 3: Finished computing partitions using SCOTCH
Process 5: End SCOTCH partitioning.
Process 5: Finished computing partitions using SCOTCH
Process 13: End SCOTCH partitioning.
Process 13: Finished computing partitions using SCOTCH
Process 9: End SCOTCH partitioning.
Process 9: Finished computing partitions using SCOTCH
Process 11: End SCOTCH partitioning.
Process 11: Finished computing partitions using SCOTCH
Process 15: End SCOTCH partitioning.
Process 15: Finished computing partitions using SCOTCH
Process 1: End SCOTCH partitioning.
Process 1: Finished computing partitions using SCOTCH
mesh ok
mesh ok
mesh ok
mesh ok
mesh ok
mesh ok
mesh ok
mesh ok
mesh ok
mesh ok
mesh ok
mesh ok
mesh ok
mesh ok
mesh ok
mesh ok
[gpc-f128n057:01153] *** Process received signal ***
[gpc-f128n057:01153] Signal: Segmentation fault (11)
[gpc-f128n057:01153] Signal code: Address not mapped (1)
[gpc-f128n057:01153] Failing at address: 0x32d2650

Revision history for this message
Damiaan (dhabets) said : 		#4

Correction, the above log is for this python script:

from dolfin import *
print 'got dolfin'
mesh = UnitCube(10,10,10)
print 'mesh ok'
V = VectorFunctionSpace(mesh, "CG", 1)
print 'Vector fcn space ok'
v = Function(V)
print 'Made it :) KVS was wrong...'

Revision history for this message
Damiaan (dhabets) said : 		#5

This is for the navier-stokes demo:

[gpc-f105n024:21879] *** Process received signal ***
[gpc-f105n020:22093] *** Process received signal ***
[gpc-f105n020:22093] Signal: Segmentation fault (11)
[gpc-f105n020:22093] Signal code: Address not mapped (1)
[gpc-f105n020:22093] Failing at address: 0x7fff00000000
[gpc-f105n020:22097] *** Process received signal ***
[gpc-f105n020:22097] Signal: Segmentation fault (11)
[gpc-f105n020:22097] Signal code: Address not mapped (1)
[gpc-f105n020:22097] Failing at address: 0x7fff00000000
[gpc-f105n020:22098] *** Process received signal ***
[gpc-f105n020:22098] Signal: Segmentation fault (11)
[gpc-f105n020:22098] Signal code: Address not mapped (1)
[gpc-f105n020:22098] Failing at address: 0x7fff00000000
[gpc-f105n024:21880] *** Process received signal ***
[gpc-f105n024:21880] Signal: Segmentation fault (11)
[gpc-f105n024:21880] Signal code: Address not mapped (1)
[gpc-f105n024:21880] Failing at address: 0x7fff00000000
[gpc-f105n020:22096] *** Process received signal ***
[gpc-f105n020:22096] Signal: Segmentation fault (11)
[gpc-f105n020:22096] Signal code: Address not mapped (1)
[gpc-f105n020:22096] Failing at address: 0x7fff00000000
[gpc-f105n024:21881] *** Process received signal ***
[gpc-f105n024:21881] Signal: Segmentation fault (11)
[gpc-f105n024:21881] Signal code: Address not mapped (1)
[gpc-f105n024:21881] Failing at address: 0x7fff00000000
[gpc-f105n020:22091] *** Process received signal ***
[gpc-f105n020:22091] Signal: Segmentation fault (11)
[gpc-f105n020:22091] Signal code: Address not mapped (1)
[gpc-f105n020:22091] Failing at address: 0x7fff00000000
[gpc-f105n020:22092] *** Process received signal ***
[gpc-f105n020:22092] Signal: Segmentation fault (11)
[gpc-f105n020:22092] Signal code: Address not mapped (1)
[gpc-f105n020:22092] Failing at address: 0x7fff00000000
[gpc-f105n020:22095] *** Process received signal ***
[gpc-f105n020:22095] Signal: Segmentation fault (11)
[gpc-f105n020:22095] Signal code: Address not mapped (1)
[gpc-f105n020:22095] Failing at address: 0x7fff00000000
[gpc-f105n024:21883] *** Process received signal ***
[gpc-f105n024:21883] Signal: Segmentation fault (11)
[gpc-f105n024:21883] Signal code: Address not mapped (1)
[gpc-f105n024:21883] Failing at address: 0x7fff00000000
[gpc-f105n020:22094] *** Process received signal ***
[gpc-f105n020:22094] Signal: Segmentation fault (11)
[gpc-f105n020:22094] Signal code: Address not mapped (1)
[gpc-f105n020:22094] Failing at address: 0x7fff00000000
[gpc-f105n024:21879] Signal: Segmentation fault (11)
[gpc-f105n024:21879] Signal code: Address not mapped (1)
[gpc-f105n024:21879] Failing at address: 0x7fff00000000
[gpc-f105n024:21882] *** Process received signal ***
[gpc-f105n024:21884] *** Process received signal ***
[gpc-f105n024:21884] Signal: Segmentation fault (11)
[gpc-f105n024:21884] Signal code: Address not mapped (1)
[gpc-f105n024:21884] Failing at address: 0x7fff00000000
[gpc-f105n024:21886] *** Process received signal ***
[gpc-f105n024:21886] Signal: Segmentation fault (11)
[gpc-f105n024:21886] Signal code: Address not mapped (1)
[gpc-f105n024:21886] Failing at address: 0x7fff00000000
[gpc-f105n024:21882] Signal: Segmentation fault (11)
[gpc-f105n024:21882] Signal code: Address not mapped (1)
[gpc-f105n024:21882] Failing at address: 0x7fff00000000
[gpc-f105n024:21885] *** Process received signal ***
[gpc-f105n024:21885] Signal: Segmentation fault (11)
[gpc-f105n024:21885] Signal code: Address not mapped (1)
[gpc-f105n024:21885] Failing at address: 0x7fff00000000
[gpc-f105n024:21880] [ 0] /lib64/libpthread.so.0(+0xf4a0) [0x7fffedfc04a0]
[gpc-f105n024:21882] [ 0] /lib64/libpthread.so.0(+0xf4a0) [0x7fffedfc04a0]
[gpc-f105n024:21882] [ 1] /lib64/libc.so.6(+0x125b70) [0x7fffedd46b70]
[gpc-f105n024:21882] [ 2] /home/s/steinman/dhabets/Root/FEniCS/lib/libxml2.so.2(xmlParserInputBufferCreateFilename+0x137) [0x7ffff73e9a67]
[gpc-f105n024:21882] [ 3] /home/s/steinman/dhabets/Root/FEniCS/lib/libxml2.so.2(xmlNewInputFromFile+0x42) [0x7ffff73c4872]
[gpc-f105n024:21882] [ 4] /home/s/steinman/dhabets/Root/FEniCS/lib/libxml2.so.2(xmlCreateURLParserCtxt+0x3c) [0x7ffff73d9dfc]
[gpc-f105n024:21882] [ 5] /home/s/steinman/dhabets/Root/FEniCS/lib/libxml2.so.2(xmlSAXUserParseFile+0x26) [0x7ffff73d9eb6]
[gpc-f105n024:21882] [ 6] /home/s/steinman/dhabets/Root/FEniCS/lib/libdolfin.so.1.0(_ZN6dolfin15XMLLocalMeshSAX4readEv+0x8f) [0x7ffff798256f]
[gpc-f105n024:21882] [ 7] /home/s/steinman/dhabets/Root/FEniCS/lib/libdolfin.so.1.0(_ZN6dolfin7XMLFilersERNS_4MeshE+0x64) [0x7ffff795e784]
[gpc-f105n024:21882] [ 8] /home/s/steinman/dhabets/Root/FEniCS/lib/libdolfin.so.1.0(_ZN6dolfin4MeshC2ESs+0x235) [0x7ffff7a269d5]
[gpc-f105n024:21882] [ 9] demo_navier-stokes(main+0x7f) [0x45172f]
[gpc-f105n024:21882] [10] /lib64/libc.so.6(__libc_start_main+0xfd) [0x7fffedc3fcdd]
[gpc-f105n024:21882] [11] demo_navier-stokes() [0x42ae85]
[gpc-f105n024:21882] *** End of error message ***
[gpc-f105n024:21880] [ 1] /lib64/libc.so.6(+0x125b70) [0x7fffedd46b70]
[gpc-f105n024:21883] [ 0] /lib64/libpthread.so.0(+0xf4a0) [0x7fffedfc04a0]
[gpc-f105n024:21883] [ 1] /lib64/libc.so.6(+0x125b70) [0x7fffedd46b70]
[gpc-f105n024:21883] [ 2] /home/s/steinman/dhabets/Root/FEniCS/lib/libxml2.so.2(xmlParserInputBufferCreateFilename+0x137) [0x7ffff73e9a67]
[gpc-f105n024:21883] [ 3] /home/s/steinman/dhabets/Root/FEniCS/lib/libxml2.so.2(xmlNewInputFromFile+0x42) [0x7ffff73c4872]
[gpc-f105n024:21883] [ 4] /home/s/steinman/dhabets/Root/FEniCS/lib/libxml2.so.2(xmlCreateURLParserCtxt+0x3c) [0x7ffff73d9dfc]
[gpc-f105n024:21883] [ 5] /home/s/steinman/dhabets/Root/FEniCS/lib/libxml2.so.2(xmlSAXUserParseFile+0x26) [0x7ffff73d9eb6]
[gpc-f105n024:21883] [ 6] /home/s/steinman/dhabets/Root/FEniCS/lib/libdolfin.so.1.0(_ZN6dolfin15XMLLocalMeshSAX4readEv+0x8f) [0x7ffff798256f]
[gpc-f105n024:21883] [ 7] /home/s/steinman/dhabets/Root/FEniCS/lib/libdolfin.so.1.0(_ZN6dolfin7XMLFilersERNS_4MeshE+0x64) [0x7ffff795e784]
[gpc-f105n024:21883] [ 8] /home/s/steinman/dhabets/Root/FEniCS/lib/libdolfin.so.1.0(_ZN6dolfin4MeshC2ESs+0x235) [0x7ffff7a269d5]
[gpc-f105n024:21883] [ 9] demo_navier-stokes(main+0x7f) [0x45172f]
[gpc-f105n024:21883] [10] /lib64/libc.so.6(__libc_start_main+0xfd) [0x7fffedc3fcdd]
[gpc-f105n024:21883] [11] demo_navier-stokes() [0x42ae85]
[gpc-f105n024:21883] *** End of error message ***
[gpc-f105n024:21885] [ 0] /lib64/libpthread.so.0(+0xf4a0) [0x7fffedfc04a0]
[gpc-f105n024:21885] [ 1] /lib64/libc.so.6(+0x125b70) [0x7fffedd46b70]
[gpc-f105n024:21885] [ 2] /home/s/steinman/dhabets/Root/FEniCS/lib/libxml2.so.2(xmlParserInputBufferCreateFilename+0x137) [0x7ffff73e9a67]
[gpc-f105n024:21885] [ 3] /home/s/steinman/dhabets/Root/FEniCS/lib/libxml2.so.2(xmlNewInputFromFile+0x42) [0x7ffff73c4872]
[gpc-f105n024:21885] [ 4] /home/s/steinman/dhabets/Root/FEniCS/lib/libxml2.so.2(xmlCreateURLParserCtxt+0x3c) [0x7ffff73d9dfc]
[gpc-f105n024:21885] [ 5] /home/s/steinman/dhabets/Root/FEniCS/lib/libxml2.so.2(xmlSAXUserParseFile+0x26) [0x7ffff73d9eb6]
[gpc-f105n024:21885] [ 6] /home/s/steinman/dhabets/Root/FEniCS/lib/libdolfin.so.1.0(_ZN6dolfin15XMLLocalMeshSAX4readEv+0x8f) [0x7ffff798256f]
[gpc-f105n024:21885] [ 7] /home/s/steinman/dhabets/Root/FEniCS/lib/libdolfin.so.1.0(_ZN6dolfin7XMLFilersERNS_4MeshE+0x64) [0x7ffff795e784]
[gpc-f105n024:21885] [ 8] /home/s/steinman/dhabets/Root/FEniCS/lib/libdolfin.so.1.0(_ZN6dolfin4MeshC2ESs+0x235) [0x7ffff7a269d5]
[gpc-f105n024:21885] [ 9] demo_navier-stokes(main+0x7f) [0x45172f]
[gpc-f105n024:21885] [10] /lib64/libc.so.6(__libc_start_main+0xfd) [0x7fffedc3fcdd]
[gpc-f105n024:21885] [11] demo_navier-stokes() [0x42ae85]
[gpc-f105n024:21885] *** End of error message ***
[gpc-f105n024:21879] [ 0] /lib64/libpthread.so.0(+0xf4a0) [0x7fffedfc04a0]
[gpc-f105n024:21879] [ 1] /lib64/libc.so.6(+0x125b70) [0x7fffedd46b70]
[gpc-f105n024:21879] [ 2] /home/s/steinman/dhabets/Root/FEniCS/lib/libxml2.so.2(xmlParserInputBufferCreateFilename+0x137) [0x7ffff73e9a67]
[gpc-f105n024:21879] [ 3] /home/s/steinman/dhabets/Root/FEniCS/lib/libxml2.so.2(xmlNewInputFromFile+0x42) [0x7ffff73c4872]
[gpc-f105n024:21879] [ 4] /home/s/steinman/dhabets/Root/FEniCS/lib/libxml2.so.2(xmlCreateURLParserCtxt+0x3c) [0x7ffff73d9dfc]
[gpc-f105n024:21879] [ 5] /home/s/steinman/dhabets/Root/FEniCS/lib/libxml2.so.2(xmlSAXUserParseFile+0x26) [0x7ffff73d9eb6]
[gpc-f105n024:21879] [ 6] /home/s/steinman/dhabets/Root/FEniCS/lib/libdolfin.so.1.0(_ZN6dolfin15XMLLocalMeshSAX4readEv+0x8f) [0x7ffff798256f]
[gpc-f105n024:21879] [ 7] /home/s/steinman/dhabets/Root/FEniCS/lib/libdolfin.so.1.0(_ZN6dolfin7XMLFilersERNS_4MeshE+0x64) [0x7ffff795e784]
[gpc-f105n024:21879] [ 8] /home/s/steinman/dhabets/Root/FEniCS/lib/libdolfin.so.1.0(_ZN6dolfin4MeshC2ESs+0x235) [0x7ffff7a269d5]
[gpc-f105n024:21879] [ 9] demo_navier-stokes(main+0x7f) [0x45172f]
[gpc-f105n024:21879] [10] /lib64/libc.so.6(__libc_start_main+0xfd) [0x7fffedc3fcdd]
[gpc-f105n024:21879] [11] demo_navier-stokes() [0x42ae85]
[gpc-f105n024:21879] *** End of error message ***
[gpc-f105n024:21881] [ 0] /lib64/libpthread.so.0(+0xf4a0) [0x7fffedfc04a0]
[gpc-f105n024:21881] [ 1] /lib64/libc.so.6(+0x125b70) [0x7fffedd46b70]
[gpc-f105n024:21881] [ 2] /home/s/steinman/dhabets/Root/FEniCS/lib/libxml2.so.2(xmlParserInputBufferCreateFilename+0x137) [0x7ffff73e9a67]
[gpc-f105n024:21881] [ 3] /home/s/steinman/dhabets/Root/FEniCS/lib/libxml2.so.2(xmlNewInputFromFile+0x42) [0x7ffff73c4872]
[gpc-f105n024:21881] [ 4] /home/s/steinman/dhabets/Root/FEniCS/lib/libxml2.so.2(xmlCreateURLParserCtxt+0x3c) [0x7ffff73d9dfc]
[gpc-f105n024:21881] [ 5] /home/s/steinman/dhabets/Root/FEniCS/lib/libxml2.so.2(xmlSAXUserParseFile+0x26) [0x7ffff73d9eb6]
[gpc-f105n024:21881] [ 6] /home/s/steinman/dhabets/Root/FEniCS/lib/libdolfin.so.1.0(_ZN6dolfin15XMLLocalMeshSAX4readEv+0x8f) [0x7ffff798256f]
[gpc-f105n024:21881] [ 7] /home/s/steinman/dhabets/Root/FEniCS/lib/libdolfin.so.1.0(_ZN6dolfin7XMLFilersERNS_4MeshE+0x64) [0x7ffff795e784]
[gpc-f105n024:21881] [ 8] /home/s/steinman/dhabets/Root/FEniCS/lib/libdolfin.so.1.0(_ZN6dolfin4MeshC2ESs+0x235) [0x7ffff7a269d5]
[gpc-f105n024:21881] [ 9] demo_navier-stokes(main+0x7f) [0x45172f]
[gpc-f105n024:21881] [10] /lib64/libc.so.6(__libc_start_main+0xfd) [0x7fffedc3fcdd]
[gpc-f105n024:21881] [11] demo_navier-stokes() [0x42ae85]
[gpc-f105n024:21881] *** End of error message ***
[gpc-f105n024:21886] [ 0] /lib64/libpthread.so.0(+0xf4a0) [0x7fffedfc04a0]
[gpc-f105n024:21886] [ 1] /lib64/libc.so.6(+0x125b70) [0x7fffedd46b70]
[gpc-f105n024:21886] [ 2] /home/s/steinman/dhabets/Root/FEniCS/lib/libxml2.so.2(xmlParserInputBufferCreateFilename+0x137) [0x7ffff73e9a67]
[gpc-f105n024:21886] [ 3] /home/s/steinman/dhabets/Root/FEniCS/lib/libxml2.so.2(xmlNewInputFromFile+0x42) [0x7ffff73c4872]
[gpc-f105n024:21886] [ 4] /home/s/steinman/dhabets/Root/FEniCS/lib/libxml2.so.2(xmlCreateURLParserCtxt+0x3c) [0x7ffff73d9dfc]
[gpc-f105n024:21886] [ 5] /home/s/steinman/dhabets/Root/FEniCS/lib/libxml2.so.2(xmlSAXUserParseFile+0x26) [0x7ffff73d9eb6]
[gpc-f105n024:21886] [ 6] /home/s/steinman/dhabets/Root/FEniCS/lib/libdolfin.so.1.0(_ZN6dolfin15XMLLocalMeshSAX4readEv+0x8f) [0x7ffff798256f]
[gpc-f105n024:21886] [ 7] /home/s/steinman/dhabets/Root/FEniCS/lib/libdolfin.so.1.0(_ZN6dolfin7XMLFilersERNS_4MeshE+0x64) [0x7ffff795e784]
[gpc-f105n024:21886] [ 8] /home/s/steinman/dhabets/Root/FEniCS/lib/libdolfin.so.1.0(_ZN6dolfin4MeshC2ESs+0x235) [0x7ffff7a269d5]
[gpc-f105n024:21886] [ 9] demo_navier-stokes(main+0x7f) [0x45172f]
[gpc-f105n024:21886] [10] /lib64/libc.so.6(__libc_start_main+0xfd) [0x7fffedc3fcdd]
[gpc-f105n024:21886] [11] demo_navier-stokes() [0x42ae85]
[gpc-f105n024:21886] *** End of error message ***
[gpc-f105n024:21884] [ 0] /lib64/libpthread.so.0(+0xf4a0) [0x7fffedfc04a0]
[gpc-f105n024:21884] [ 1] /lib64/libc.so.6(+0x125b70) [0x7fffedd46b70]
[gpc-f105n024:21884] [ 2] /home/s/steinman/dhabets/Root/FEniCS/lib/libxml2.so.2(xmlParserInputBufferCreateFilename+0x137) [0x7ffff73e9a67]
[gpc-f105n024:21884] [ 3] /home/s/steinman/dhabets/Root/FEniCS/lib/libxml2.so.2(xmlNewInputFromFile+0x42) [0x7ffff73c4872]
[gpc-f105n024:21884] [ 4] /home/s/steinman/dhabets/Root/FEniCS/lib/libxml2.so.2(xmlCreateURLParserCtxt+0x3c) [0x7ffff73d9dfc]
[gpc-f105n024:21884] [ 5] /home/s/steinman/dhabets/Root/FEniCS/lib/libxml2.so.2(xmlSAXUserParseFile+0x26) [0x7ffff73d9eb6]
[gpc-f105n024:21884] [ 6] /home/s/steinman/dhabets/Root/FEniCS/lib/libdolfin.so.1.0(_ZN6dolfin15XMLLocalMeshSAX4readEv+0x8f) [0x7ffff798256f]
[gpc-f105n024:21884] [ 7] /home/s/steinman/dhabets/Root/FEniCS/lib/libdolfin.so.1.0(_ZN6dolfin7XMLFilersERNS_4MeshE+0x64) [0x7ffff795e784]
[gpc-f105n024:21884] [ 8] /home/s/steinman/dhabets/Root/FEniCS/lib/libdolfin.so.1.0(_ZN6dolfin4MeshC2ESs+0x235) [0x7ffff7a269d5]
[gpc-f105n024:21884] [ 9] demo_navier-stokes(main+0x7f) [0x45172f]
[gpc-f105n024:21884] [10] /lib64/libc.so.6(__libc_start_main+0xfd) [0x7fffedc3fcdd]
[gpc-f105n024:21884] [11] demo_navier-stokes() [0x42ae85]
[gpc-f105n024:21884] *** End of error message ***
[gpc-f105n024:21880] [ 2] /home/s/steinman/dhabets/Root/FEniCS/lib/libxml2.so.2(xmlParserInputBufferCreateFilename+0x137) [0x7ffff73e9a67]
[gpc-f105n024:21880] [ 3] /home/s/steinman/dhabets/Root/FEniCS/lib/libxml2.so.2(xmlNewInputFromFile+0x42) [0x7ffff73c4872]
[gpc-f105n024:21880] [ 4] /home/s/steinman/dhabets/Root/FEniCS/lib/libxml2.so.2(xmlCreateURLParserCtxt+0x3c) [0x7ffff73d9dfc]
[gpc-f105n024:21880] [ 5] /home/s/steinman/dhabets/Root/FEniCS/lib/libxml2.so.2(xmlSAXUserParseFile+0x26) [0x7ffff73d9eb6]
[gpc-f105n024:21880] [ 6] /home/s/steinman/dhabets/Root/FEniCS/lib/libdolfin.so.1.0(_ZN6dolfin15XMLLocalMeshSAX4readEv+0x8f) [0x7ffff798256f]
[gpc-f105n024:21880] [ 7] /home/s/steinman/dhabets/Root/FEniCS/lib/libdolfin.so.1.0(_ZN6dolfin7XMLFilersERNS_4MeshE+0x64) [0x7ffff795e784]
[gpc-f105n024:21880] [ 8] /home/s/steinman/dhabets/Root/FEniCS/lib/libdolfin.so.1.0(_ZN6dolfin4MeshC2ESs+0x235) [0x7ffff7a269d5]
[gpc-f105n024:21880] [ 9] demo_navier-stokes(main+0x7f) [0x45172f]
[gpc-f105n024:21880] [10] /lib64/libc.so.6(__libc_start_main+0xfd) [0x7fffedc3fcdd]
[gpc-f105n024:21880] [11] demo_navier-stokes() [0x42ae85]
[gpc-f105n024:21880] *** End of error message ***
[gpc-f105n020:22091] [ 0] /lib64/libpthread.so.0(+0xf4a0) [0x7fffedfc04a0]
[gpc-f105n020:22091] [ 1] /lib64/libc.so.6(+0x125b70) [0x7fffedd46b70]
[gpc-f105n020:22091] [ 2] /home/s/steinman/dhabets/Root/FEniCS/lib/libxml2.so.2(xmlParserInputBufferCreateFilename+0x137) [0x7ffff73e9a67]
[gpc-f105n020:22091] [ 3] /home/s/steinman/dhabets/Root/FEniCS/lib/libxml2.so.2(xmlNewInputFromFile+0x42) [0x7ffff73c4872]
[gpc-f105n020:22091] [ 4] /home/s/steinman/dhabets/Root/FEniCS/lib/libxml2.so.2(xmlCreateURLParserCtxt+0x3c) [0x7ffff73d9dfc]
[gpc-f105n020:22091] [ 5] /home/s/steinman/dhabets/Root/FEniCS/lib/libxml2.so.2(xmlSAXUserParseFile+0x26) [0x7ffff73d9eb6]
[gpc-f105n020:22091] [ 6] /home/s/steinman/dhabets/Root/FEniCS/lib/libdolfin.so.1.0(_ZN6dolfin15XMLLocalMeshSAX4readEv+0x8f) [0x7ffff798256f]
[gpc-f105n020:22091] [ 7] /home/s/steinman/dhabets/Root/FEniCS/lib/libdolfin.so.1.0(_ZN6dolfin7XMLFilersERNS_4MeshE+0x64) [0x7ffff795e784]
[gpc-f105n020:22091] [ 8] /home/s/steinman/dhabets/Root/FEniCS/lib/libdolfin.so.1.0(_ZN6dolfin4MeshC2ESs+0x235) [0x7ffff7a269d5]
[gpc-f105n020:22091] [ 9] demo_navier-stokes(main+0x7f) [0x45172f]
[gpc-f105n020:22091] [10] /lib64/libc.so.6(__libc_start_main+0xfd) [0x7fffedc3fcdd]
[gpc-f105n020:22091] [11] demo_navier-stokes() [0x42ae85]
[gpc-f105n020:22091] *** End of error message ***
[gpc-f105n020:22098] [ 0] /lib64/libpthread.so.0(+0xf4a0) [0x7fffedfc04a0]
[gpc-f105n020:22098] [ 1] /lib64/libc.so.6(+0x125b70) [0x7fffedd46b70]
[gpc-f105n020:22098] [ 2] /home/s/steinman/dhabets/Root/FEniCS/lib/libxml2.so.2(xmlParserInputBufferCreateFilename+0x137) [0x7ffff73e9a67]
[gpc-f105n020:22098] [ 3] /home/s/steinman/dhabets/Root/FEniCS/lib/libxml2.so.2(xmlNewInputFromFile+0x42) [0x7ffff73c4872]
[gpc-f105n020:22098] [ 4] /home/s/steinman/dhabets/Root/FEniCS/lib/libxml2.so.2(xmlCreateURLParserCtxt+0x3c) [0x7ffff73d9dfc]
[gpc-f105n020:22098] [ 5] /home/s/steinman/dhabets/Root/FEniCS/lib/libxml2.so.2(xmlSAXUserParseFile+0x26) [0x7ffff73d9eb6]
[gpc-f105n020:22098] [ 6] /home/s/steinman/dhabets/Root/FEniCS/lib/libdolfin.so.1.0(_ZN6dolfin15XMLLocalMeshSAX4readEv+0x8f) [0x7ffff798256f]
[gpc-f105n020:22098] [ 7] /home/s/steinman/dhabets/Root/FEniCS/lib/libdolfin.so.1.0(_ZN6dolfin7XMLFilersERNS_4MeshE+0x64) [0x7ffff795e784]
[gpc-f105n020:22098] [ 8] /home/s/steinman/dhabets/Root/FEniCS/lib/libdolfin.so.1.0(_ZN6dolfin4MeshC2ESs+0x235) [0x7ffff7a269d5]
[gpc-f105n020:22098] [ 9] demo_navier-stokes(main+0x7f) [0x45172f]
[gpc-f105n020:22098] [10] /lib64/libc.so.6(__libc_start_main+0xfd) [0x7fffedc3fcdd]
[gpc-f105n020:22098] [11] demo_navier-stokes() [0x42ae85]
[gpc-f105n020:22098] *** End of error message ***
[gpc-f105n020:22097] [ 0] /lib64/libpthread.so.0(+0xf4a0) [0x7fffedfc04a0]
[gpc-f105n020:22097] [ 1] /lib64/libc.so.6(+0x125b70) [0x7fffedd46b70]
[gpc-f105n020:22097] [ 2] /home/s/steinman/dhabets/Root/FEniCS/lib/libxml2.so.2(xmlParserInputBufferCreateFilename+0x137) [0x7ffff73e9a67]
[gpc-f105n020:22097] [ 3] /home/s/steinman/dhabets/Root/FEniCS/lib/libxml2.so.2(xmlNewInputFromFile+0x42) [0x7ffff73c4872]
[gpc-f105n020:22097] [ 4] /home/s/steinman/dhabets/Root/FEniCS/lib/libxml2.so.2(xmlCreateURLParserCtxt+0x3c) [0x7ffff73d9dfc]
[gpc-f105n020:22097] [ 5] /home/s/steinman/dhabets/Root/FEniCS/lib/libxml2.so.2(xmlSAXUserParseFile+0x26) [0x7ffff73d9eb6]
[gpc-f105n020:22097] [ 6] /home/s/steinman/dhabets/Root/FEniCS/lib/libdolfin.so.1.0(_ZN6dolfin15XMLLocalMeshSAX4readEv+0x8f) [0x7ffff798256f]
[gpc-f105n020:22097] [ 7] /home/s/steinman/dhabets/Root/FEniCS/lib/libdolfin.so.1.0(_ZN6dolfin7XMLFilersERNS_4MeshE+0x64) [0x7ffff795e784]
[gpc-f105n020:22097] [ 8] /home/s/steinman/dhabets/Root/FEniCS/lib/libdolfin.so.1.0(_ZN6dolfin4MeshC2ESs+0x235) [0x7ffff7a269d5]
[gpc-f105n020:22097] [ 9] demo_navier-stokes(main+0x7f) [0x45172f]
[gpc-f105n020:22097] [10] /lib64/libc.so.6(__libc_start_main+0xfd) [0x7fffedc3fcdd]
[gpc-f105n020:22097] [11] demo_navier-stokes() [0x42ae85]
[gpc-f105n020:22097] *** End of error message ***
[gpc-f105n020:22095] [ 0] /lib64/libpthread.so.0(+0xf4a0) [0x7fffedfc04a0]
[gpc-f105n020:22095] [ 1] /lib64/libc.so.6(+0x125b70) [0x7fffedd46b70]
[gpc-f105n020:22095] [ 2] /home/s/steinman/dhabets/Root/FEniCS/lib/libxml2.so.2(xmlParserInputBufferCreateFilename+0x137) [0x7ffff73e9a67]
[gpc-f105n020:22095] [ 3] /home/s/steinman/dhabets/Root/FEniCS/lib/libxml2.so.2(xmlNewInputFromFile+0x42) [0x7ffff73c4872]
[gpc-f105n020:22095] [ 4] /home/s/steinman/dhabets/Root/FEniCS/lib/libxml2.so.2(xmlCreateURLParserCtxt+0x3c) [0x7ffff73d9dfc]
[gpc-f105n020:22095] [ 5] /home/s/steinman/dhabets/Root/FEniCS/lib/libxml2.so.2(xmlSAXUserParseFile+0x26) [0x7ffff73d9eb6]
[gpc-f105n020:22095] [ 6] /home/s/steinman/dhabets/Root/FEniCS/lib/libdolfin.so.1.0(_ZN6dolfin15XMLLocalMeshSAX4readEv+0x8f) [0x7ffff798256f]
[gpc-f105n020:22095] [ 7] /home/s/steinman/dhabets/Root/FEniCS/lib/libdolfin.so.1.0(_ZN6dolfin7XMLFilersERNS_4MeshE+0x64) [0x7ffff795e784]
[gpc-f105n020:22095] [ 8] /home/s/steinman/dhabets/Root/FEniCS/lib/libdolfin.so.1.0(_ZN6dolfin4MeshC2ESs+0x235) [0x7ffff7a269d5]
[gpc-f105n020:22095] [ 9] demo_navier-stokes(main+0x7f) [0x45172f]
[gpc-f105n020:22095] [10] /lib64/libc.so.6(__libc_start_main+0xfd) [0x7fffedc3fcdd]
[gpc-f105n020:22095] [11] demo_navier-stokes() [0x42ae85]
[gpc-f105n020:22095] *** End of error message ***
[gpc-f105n020:22093] [ 0] /lib64/libpthread.so.0(+0xf4a0) [0x7fffedfc04a0]
[gpc-f105n020:22093] [ 1] /lib64/libc.so.6(+0x125b70) [0x7fffedd46b70]
[gpc-f105n020:22093] [ 2] /home/s/steinman/dhabets/Root/FEniCS/lib/libxml2.so.2(xmlParserInputBufferCreateFilename+0x137) [0x7ffff73e9a67]
[gpc-f105n020:22093] [ 3] /home/s/steinman/dhabets/Root/FEniCS/lib/libxml2.so.2(xmlNewInputFromFile+0x42) [0x7ffff73c4872]
[gpc-f105n020:22093] [ 4] /home/s/steinman/dhabets/Root/FEniCS/lib/libxml2.so.2(xmlCreateURLParserCtxt+0x3c) [0x7ffff73d9dfc]
[gpc-f105n020:22093] [ 5] /home/s/steinman/dhabets/Root/FEniCS/lib/libxml2.so.2(xmlSAXUserParseFile+0x26) [0x7ffff73d9eb6]
[gpc-f105n020:22093] [ 6] /home/s/steinman/dhabets/Root/FEniCS/lib/libdolfin.so.1.0(_ZN6dolfin15XMLLocalMeshSAX4readEv+0x8f) [0x7ffff798256f]
[gpc-f105n020:22096] [ 0] /lib64/libpthread.so.0(+0xf4a0) [0x7fffedfc04a0]
[gpc-f105n020:22096] [ 1] /lib64/libc.so.6(+0x125b70) [0x7fffedd46b70]
[gpc-f105n020:22096] [ 2] /home/s/steinman/dhabets/Root/FEniCS/lib/libxml2.so.2(xmlParserInputBufferCreateFilename+0x137) [0x7ffff73e9a67]
[gpc-f105n020:22096] [ 3] /home/s/steinman/dhabets/Root/FEniCS/lib/libxml2.so.2(xmlNewInputFromFile+0x42) [0x7ffff73c4872]
[gpc-f105n020:22096] [ 4] /home/s/steinman/dhabets/Root/FEniCS/lib/libxml2.so.2(xmlCreateURLParserCtxt+0x3c) [0x7ffff73d9dfc]
[gpc-f105n020:22096] [ 5] /home/s/steinman/dhabets/Root/FEniCS/lib/libxml2.so.2(xmlSAXUserParseFile+0x26) [0x7ffff73d9eb6]
[gpc-f105n020:22096] [ 6] /home/s/steinman/dhabets/Root/FEniCS/lib/libdolfin.so.1.0(_ZN6dolfin15XMLLocalMeshSAX4readEv+0x8f) [0x7ffff798256f]
[gpc-f105n020:22096] [ 7] /home/s/steinman/dhabets/Root/FEniCS/lib/libdolfin.so.1.0(_ZN6dolfin7XMLFilersERNS_4MeshE+0x64) [0x7ffff795e784]
[gpc-f105n020:22096] [ 8] /home/s/steinman/dhabets/Root/FEniCS/lib/libdolfin.so.1.0(_ZN6dolfin4MeshC2ESs+0x235) [0x7ffff7a269d5]
[gpc-f105n020:22093] [ 7] /home/s/steinman/dhabets/Root/FEniCS/lib/libdolfin.so.1.0(_ZN6dolfin7XMLFilersERNS_4MeshE+0x64) [0x7ffff795e784]
[gpc-f105n020:22093] [ 8] /home/s/steinman/dhabets/Root/FEniCS/lib/libdolfin.so.1.0(_ZN6dolfin4MeshC2ESs+0x235) [0x7ffff7a269d5]
[gpc-f105n020:22093] [ 9] demo_navier-stokes(main+0x7f) [0x45172f]
[gpc-f105n020:22093] [10] /lib64/libc.so.6(__libc_start_main+0xfd) [0x7fffedc3fcdd]
[gpc-f105n020:22093] [11] demo_navier-stokes() [0x42ae85]
[gpc-f105n020:22093] *** End of error message ***
[gpc-f105n020:22096] [ 9] demo_navier-stokes(main+0x7f) [0x45172f]
[gpc-f105n020:22096] [10] /lib64/libc.so.6(__libc_start_main+0xfd) [0x7fffedc3fcdd]
[gpc-f105n020:22096] [11] demo_navier-stokes() [0x42ae85]
[gpc-f105n020:22096] *** End of error message ***
[gpc-f105n020:22094] [ 0] /lib64/libpthread.so.0(+0xf4a0) [0x7fffedfc04a0]
[gpc-f105n020:22094] [ 1] /lib64/libc.so.6(+0x125b70) [0x7fffedd46b70]
[gpc-f105n020:22094] [ 2] /home/s/steinman/dhabets/Root/FEniCS/lib/libxml2.so.2(xmlParserInputBufferCreateFilename+0x137) [0x7ffff73e9a67]
[gpc-f105n020:22094] [ 3] /home/s/steinman/dhabets/Root/FEniCS/lib/libxml2.so.2(xmlNewInputFromFile+0x42) [0x7ffff73c4872]
[gpc-f105n020:22094] [ 4] /home/s/steinman/dhabets/Root/FEniCS/lib/libxml2.so.2(xmlCreateURLParserCtxt+0x3c) [0x7ffff73d9dfc]
[gpc-f105n020:22094] [ 5] /home/s/steinman/dhabets/Root/FEniCS/lib/libxml2.so.2(xmlSAXUserParseFile+0x26) [0x7ffff73d9eb6]
[gpc-f105n020:22094] [ 6] /home/s/steinman/dhabets/Root/FEniCS/lib/libdolfin.so.1.0(_ZN6dolfin15XMLLocalMeshSAX4readEv+0x8f) [0x7ffff798256f]
[gpc-f105n020:22094] [ 7] /home/s/steinman/dhabets/Root/FEniCS/lib/libdolfin.so.1.0(_ZN6dolfin7XMLFilersERNS_4MeshE+0x64) [0x7ffff795e784]
[gpc-f105n020:22094] [ 8] /home/s/steinman/dhabets/Root/FEniCS/lib/libdolfin.so.1.0(_ZN6dolfin4MeshC2ESs+0x235) [0x7ffff7a269d5]
[gpc-f105n020:22094] [ 9] demo_navier-stokes(main+0x7f) [0x45172f]
[gpc-f105n020:22094] [10] /lib64/libc.so.6(__libc_start_main+0xfd) [0x7fffedc3fcdd]
[gpc-f105n020:22094] [11] demo_navier-stokes() [0x42ae85]
[gpc-f105n020:22094] *** End of error message ***
[gpc-f105n020:22092] [ 0] /lib64/libpthread.so.0(+0xf4a0) [0x7fffedfc04a0]
[gpc-f105n020:22092] [ 1] /lib64/libc.so.6(+0x125b70) [0x7fffedd46b70]
[gpc-f105n020:22092] [ 2] /home/s/steinman/dhabets/Root/FEniCS/lib/libxml2.so.2(xmlParserInputBufferCreateFilename+0x137) [0x7ffff73e9a67]
[gpc-f105n020:22092] [ 3] /home/s/steinman/dhabets/Root/FEniCS/lib/libxml2.so.2(xmlNewInputFromFile+0x42) [0x7ffff73c4872]
[gpc-f105n020:22092] [ 4] /home/s/steinman/dhabets/Root/FEniCS/lib/libxml2.so.2(xmlCreateURLParserCtxt+0x3c) [0x7ffff73d9dfc]
[gpc-f105n020:22092] [ 5] /home/s/steinman/dhabets/Root/FEniCS/lib/libxml2.so.2(xmlSAXUserParseFile+0x26) [0x7ffff73d9eb6]
[gpc-f105n020:22092] [ 6] /home/s/steinman/dhabets/Root/FEniCS/lib/libdolfin.so.1.0(_ZN6dolfin15XMLLocalMeshSAX4readEv+0x8f) [0x7ffff798256f]
[gpc-f105n020:22092] [ 7] /home/s/steinman/dhabets/Root/FEniCS/lib/libdolfin.so.1.0(_ZN6dolfin7XMLFilersERNS_4MeshE+0x64) [0x7ffff795e784]
[gpc-f105n020:22092] [ 8] /home/s/steinman/dhabets/Root/FEniCS/lib/libdolfin.so.1.0(_ZN6dolfin4MeshC2ESs+0x235) [0x7ffff7a269d5]
[gpc-f105n020:22092] [ 9] demo_navier-stokes(main+0x7f) [0x45172f]
[gpc-f105n020:22092] [10] /lib64/libc.so.6(__libc_start_main+0xfd) [0x7fffedc3fcdd]
[gpc-f105n020:22092] [11] demo_navier-stokes() [0x42ae85]
[gpc-f105n020:22092] *** End of error message ***
--------------------------------------------------------------------------
mpirun noticed that process rank 13 with PID 21885 on node gpc-f105n024-ib0 exited on signal 11 (Segmentation fault).
--------------------------------------------------------------------------

Revision history for this message
Damiaan (dhabets) said : 		#6

Ignore the navier-stokes demo for now; that may be something else.

But this script:

from dolfin import *
print 'got dolfin'
mesh = UnitCube(10,10,10)
print 'mesh ok'
V = VectorFunctionSpace(mesh, "CG", 1)
print 'Vector fcn space ok'
v = Function(V)
print 'Made it :) KVS was wrong...'

runs fine on 8 cores (ie. 1 node) but as soon as we got to two nodes (9 or more cores) then it fails as shown in the log in comment #3 above.

Any ideas?

Revision history for this message
Kent-Andre Mardal (kent-and) said : 		#7

On 14 January 2013 16:45, Damiaan <email address hidden>wrote:

> Question #218957 on DOLFIN changed:
> https://answers.launchpad.net/dolfin/+question/218957
>
> Damiaan gave more information on the question:
> Ignore the navier-stokes demo for now; that may be something else.
>
> But this script:
>
> from dolfin import *
> print 'got dolfin'
> mesh = UnitCube(10,10,10)
> print 'mesh ok'
> V = VectorFunctionSpace(mesh, "CG", 1)
> print 'Vector fcn space ok'
> v = Function(V)
> print 'Made it :) KVS was wrong...'
>
>
> runs fine on 8 cores (ie. 1 node) but as soon as we got to two nodes (9 or
> more cores) then it fails as shown in the log in comment #3 above.
>
> Any ideas?
>
>
>
So you are able to construct FunctionSpaces on 1-8 cores on one node, but
not anything else?
Can you attach a debugger to the process' to get some more info?

The function space construction
V = VectorFunctionSpace(mesh, "CG", 1)
requires jit-compilation, and there will be modules under $HOME/.instant.

The jit-compilation will just be performed on one core, but the produced
code is assumed to
be accessible and usable as shared libraries on all the other cores. Is
this ok in your case?
I guess you can log into the nodes and look at the shared libraries under
$HOME/.instant/cache.
(You may also set another cachedir than $HOME/.instant if that helps)

I have not seen this kind of error before before, but it seems that there
is something wrong with the basic
communication between the nodes, I guess either filesystem or MPI.

Kent

Revision history for this message
Garth Wells (garth-wells) said : 		#8

Set CXX=mpicxx and CC=mpicc

I have seen with the Intel MPI wrappers that I get a crash when not
using the MPI wrappers for the JIT code and running jobs that span
compute nodes.

Garth

On 14 January 2013 17:45, Damiaan <email address hidden> wrote:
> Question #218957 on DOLFIN changed:
> https://answers.launchpad.net/dolfin/+question/218957
>
> Damiaan gave more information on the question:
> Ignore the navier-stokes demo for now; that may be something else.
>
> But this script:
>
> from dolfin import *
> print 'got dolfin'
> mesh = UnitCube(10,10,10)
> print 'mesh ok'
> V = VectorFunctionSpace(mesh, "CG", 1)
> print 'Vector fcn space ok'
> v = Function(V)
> print 'Made it :) KVS was wrong...'
>
>
> runs fine on 8 cores (ie. 1 node) but as soon as we got to two nodes (9 or more cores) then it fails as shown in the log in comment #3 above.
>
> Any ideas?
>
> --
> You received this question notification because you are a member of
> DOLFIN Team, which is an answer contact for DOLFIN.

Revision history for this message
Damiaan (dhabets) said : 		#9

Garth & Kent-Andre, thank you.

Re. the filesystem, yes, I can't write to home directories from the nodes, so I had to point the cachedir elsewhere using:
declare -x INSTANT_CACHE_DIR="/scratch/s/steinman/dhabets/.instant"

Re. setting CXX and CC, I'm using gcc and both are set, but I just noticed that mpicc wraps around the intel compiler...

Will post back what I find out.

Revision history for this message
Damiaan (dhabets) said : 		#10

 Ok, tried both suggestions and still receive the same error:

[gpc-f131n045:06978] *** Process received signal ***
[gpc-f131n045:06978] Signal: Segmentation fault (11)
[gpc-f131n045:06978] Signal code: Address not mapped (1)
[gpc-f131n045:06978] Failing at address: 0x3328310

Any suggestions on how to debug this?

Revision history for this message
Damiaan (dhabets) said : 		#11

This may not be related at all, but when running it through valgrind I get the seg 11 as well, but then at the end:

    FunctionSpaceBase.__init__(self, mesh, element)
Traceback (most recent call last):
    FunctionSpaceBase.__init__(self, mesh, element)
    FunctionSpaceBase.__init__(self, mesh, element)
  File "/home/s/steinman/dhabets/Root/FEniCS/lib/python2.7/site-packages/dolfin/functions/functionspace.py", line 77, in __init__
  File "/home/s/steinman/dhabets/Root/FEniCS/lib/python2.7/site-packages/dolfin/functions/functionspace.py", line 77, in __init__
  File "/home/s/steinman/dhabets/Root/FEniCS/lib/python2.7/site-packages/dolfin/functions/functionspace.py", line 77, in __init__
    ufc_element, ufc_dofmap = jit(self._ufl_element)
    ufc_element, ufc_dofmap = jit(self._ufl_element)
    FunctionSpaceBase.__init__(self, mesh, element)
    ufc_element, ufc_dofmap = jit(self._ufl_element)
  File "/home/s/steinman/dhabets/Root/FEniCS/lib/python2.7/site-packages/dolfin/compilemodules/jit.py", line 77, in mpi_jit
  File "/home/s/steinman/dhabets/Root/FEniCS/lib/python2.7/site-packages/dolfin/compilemodules/jit.py", line 77, in mpi_jit
  File "/home/s/steinman/dhabets/Root/FEniCS/lib/python2.7/site-packages/dolfin/compilemodules/jit.py", line 77, in mpi_jit
  File "/home/s/steinman/dhabets/Root/FEniCS/lib/python2.7/site-packages/dolfin/functions/functionspace.py", line 77, in __init__
  File "./test.py", line 6, in <module>
    ufc_element, ufc_dofmap = jit(self._ufl_element)
  File "/home/s/steinman/dhabets/Root/FEniCS/lib/python2.7/site-packages/dolfin/compilemodules/jit.py", line 77, in mpi_jit
    output = local_jit(*args,**kwargs)
    output = local_jit(*args,**kwargs)
  File "/home/s/steinman/dhabets/Root/FEniCS/lib/python2.7/site-packages/dolfin/compilemodules/jit.py", line 102, in jit
    output = local_jit(*args,**kwargs)
    output = local_jit(*args,**kwargs)
  File "/home/s/steinman/dhabets/Root/FEniCS/lib/python2.7/site-packages/dolfin/compilemodules/jit.py", line 102, in jit
  File "/home/s/steinman/dhabets/Root/FEniCS/lib/python2.7/site-packages/dolfin/compilemodules/jit.py", line 102, in jit
  File "/home/s/steinman/dhabets/Root/FEniCS/lib/python2.7/site-packages/dolfin/compilemodules/jit.py", line 102, in jit
    raise OSError, "Could not find swig installation. Pass an existing "\
    raise OSError, "Could not find swig installation. Pass an existing "\
    raise OSError, "Could not find swig installation. Pass an existing "\
    raise OSError, "Could not find swig installation. Pass an existing "\
OSErrorOSError: Could not find swig installation. Pass an existing swig binary or install SWIG version 2.0 or higher.
OSErrorOSError
V = VectorFunctionSpace(mesh, "CG", 1)
: Could not find swig installation. Pass an existing swig binary or install SWIG version 2.0 or higher.

: Could not find swig installation. Pass an existing swig binary or install SWIG version 2.0 or higher.

: Could not find swig installation. Pass an existing swig binary or install SWIG version 2.0 or higher.
  File "/home/s/steinman/dhabets/Root/FEniCS/lib/python2.7/site-packages/dolfin/functions/functionspace.py", line 505, in __init__

    FunctionSpaceBase.__init__(self, mesh, element)
  File "/home/s/steinman/dhabets/Root/FEniCS/lib/python2.7/site-packages/dolfin/functions/functionspace.py", line 77, in __init__
    ufc_element, ufc_dofmap = jit(self._ufl_element)
  File "/home/s/steinman/dhabets/Root/FEniCS/lib/python2.7/site-packages/dolfin/compilemodules/jit.py", line 77, in mpi_jit
    output = local_jit(*args,**kwargs)
  File "/home/s/steinman/dhabets/Root/FEniCS/lib/python2.7/site-packages/dolfin/compilemodules/jit.py", line 108, in jit
    check_swig_version(cpp.__swigversion__, same=True)
  File "/home/s/steinman/dhabets/Root/FEniCS/lib/python2.7/site-packages/instant/config.py", line 71, in check_swig_version
    installed_version = map(int, get_swig_version().split('.'))
  File "/home/s/steinman/dhabets/Root/FEniCS/lib/python2.7/site-packages/instant/config.py", line 47, in get_swig_version
    raise OSError("SWIG is not installed on the system.")
OSError: SWIG is not installed on the system.

SWIG is installed, how do I set the path to swig? I tried setting swig_binary/swig_path in bash.

Revision history for this message
Garth Wells (garth-wells) said : 		#12

On 14 January 2013 21:20, Damiaan <email address hidden> wrote:
> Question #218957 on DOLFIN changed:
> https://answers.launchpad.net/dolfin/+question/218957
>
> Damiaan posted a new comment:
> Ok, tried both suggestions and still receive the same error:
>
> [gpc-f131n045:06978] *** Process received signal ***
> [gpc-f131n045:06978] Signal: Segmentation fault (11)
> [gpc-f131n045:06978] Signal code: Address not mapped (1)
> [gpc-f131n045:06978] Failing at address: 0x3328310
>
> Any suggestions on how to debug this?
>

Start by testing a C++ demo rather than Python, and make sure to
build the demo using the MPI wrappers.

Garth

> --
> You received this question notification because you are a member of
> DOLFIN Team, which is an answer contact for DOLFIN.

Revision history for this message
Damiaan (dhabets) said : 		#13

Thanks Garth; I found a library that was linking to intel related libraries and was compiled with icc. I replaced it with a gcc compiled version and got further (to exactly where the debugger ended).

I'll close this one as the swig issue is a separate issue.