DistributedMesh Class Reference

The mesh during setup, spread over the ranks. Each rank starts from its block of the mesh (cells, nodes and boundary faces by global id) and keeps it as the directory that serves those cells and nodes to other ranks: More...

#include <distributed_mesh.h>

Public Types
using	PeriodicMap = std::unordered_map< uint64_t, std::pair< uint64_t, std::array< int8_t, 3 > > >
	Periodic node -> (key, lattice offset), as in PeriodicNodes but by global id.

Public Member Functions
	DistributedMesh (MeshBlock block, const std::vector< Mesh::PeriodicPair > &periodic={})
	Match faces across ranks, joining the zones of the periodic pairs (collective).
uint64_t	n_global_cells () const
uint64_t	first_cell () const
uint32_t	n_block_cells () const
const std::vector< uint64_t > &	cell_distribution () const
	First block cell of every rank, and the total (ParMETIS vtxdist).
const std::vector< uint64_t > &	graph_offsets () const
	Dual graph of the block cells (cells sharing a face), by global id (CSR); until distribute().
const std::vector< uint64_t > &	graph_neighbors () const
const PeriodicMap &	block_periodic_classes () const
	Periodic classes of the periodic nodes of the block's cells and boundary faces, and no others.
const MeshBlock &	mesh_block () const
	This rank's block of the mesh.
std::vector< std::array< double, N_DIM > >	block_cell_centers () const
	Vertex average of every block cell (collective).
void	distribute (const std::vector< int > &owner)
	Send every block cell to its owner rank, and drop the dual graph (collective).
std::shared_ptr< Mesh >	build_local_mesh (int halo_layers, Distribution &dist)
	This rank's mesh: its owned cells in global order, then halo_layers layers of vertex neighbors, each in global order. Faces on the global boundary keep their zones; faces towards cells not on this rank go to PARTITION_ZONE. Fills dist, including the exchange plan (collective; after distribute()).

Detailed Description

The mesh during setup, spread over the ranks. Each rank starts from its block of the mesh (cells, nodes and boundary faces by global id) and keeps it as the directory that serves those cells and nodes to other ranks:

1. Construction matches faces across ranks: each cell face and boundary face goes to a rank chosen by hashing its sorted node ids, which pairs them into the dual graph and tags each cell's boundary faces with their zones. With periodic pairs, nodes are first replaced by their periodic keys (see docs/design/periodic.md), so the faces of paired zones join into interior faces.
2. A partitioner assigns an owner to every block cell (partition_hilbert, partition_graph), and distribute() sends each cell to its owner.
3. build_local_mesh() adds halo layers by a distributed breadth-first search over vertex neighbors and builds the rank's Mesh and Distribution. Asking for more layers later grows the existing ones.

All memory is O(cells / ranks) per rank, plus the halo.

Member Typedef Documentation

PeriodicMap

using DistributedMesh::PeriodicMap = std::unordered_map<uint64_t, std::pair<uint64_t, std::array<int8_t, 3> >>

Periodic node -> (key, lattice offset), as in PeriodicNodes but by global id.

Constructor & Destructor Documentation

DistributedMesh()

DistributedMesh::DistributedMesh ( MeshBlock  block, const std::vector< Mesh::PeriodicPair > &  periodic = {}  )

explicit

Match faces across ranks, joining the zones of the periodic pairs (collective).

Member Function Documentation

block_cell_centers()

std::vector< std::array< double, N_DIM > > DistributedMesh::block_cell_centers

(

)

const

Vertex average of every block cell (collective).

block_periodic_classes()

const PeriodicMap & DistributedMesh::block_periodic_classes ( ) const

inline

Periodic classes of the periodic nodes of the block's cells and boundary faces, and no others.

build_local_mesh()

std::shared_ptr< Mesh > DistributedMesh::build_local_mesh	(	int	halo_layers,
		Distribution &	dist
	)

This rank's mesh: its owned cells in global order, then halo_layers layers of vertex neighbors, each in global order. Faces on the global boundary keep their zones; faces towards cells not on this rank go to PARTITION_ZONE. Fills dist, including the exchange plan (collective; after distribute()).

cell_distribution()

const std::vector< uint64_t > & DistributedMesh::cell_distribution ( ) const

inline

First block cell of every rank, and the total (ParMETIS vtxdist).

distribute()

void DistributedMesh::distribute

(

const std::vector< int > &

owner

)

Send every block cell to its owner rank, and drop the dual graph (collective).

first_cell()

uint64_t DistributedMesh::first_cell ( ) const

inline

graph_neighbors()

const std::vector< uint64_t > & DistributedMesh::graph_neighbors ( ) const

inline

graph_offsets()

const std::vector< uint64_t > & DistributedMesh::graph_offsets ( ) const

inline

Dual graph of the block cells (cells sharing a face), by global id (CSR); until distribute().

mesh_block()

const MeshBlock & DistributedMesh::mesh_block ( ) const

inline

This rank's block of the mesh.

n_block_cells()

uint32_t DistributedMesh::n_block_cells ( ) const

inline

n_global_cells()

uint64_t DistributedMesh::n_global_cells ( ) const

inline

---

The documentation for this class was generated from the following files:

• /home/runner/work/mallard/mallard/mallard/src/parallel/distributed_mesh.h
• /home/runner/work/mallard/mallard/mallard/src/parallel/distributed_mesh.cpp