Halo exchange

A simulation is being run on 27 processors on a distributed-memory system. The simulation is run on a network of cells: each processor handles 27 cells. However, at certain intervals, each processor must find out the values of the cells adjacent to those in its own domain. The process of each processor finding these values out is called halo exchange.

When writing and debugging the code to perform this exchange, it can be difficult to determine whether or not it has been performed correctly, and if not, to determine what went wrong: it's very easy to get code the wrong way round so that a processor acquires its right-hand halo from its left-hand neighbour. The use of a visualization tool like AVS simplifies this.

Initially, no processor has a halo. Below can be seen 27 coloured blocks - each block represents one processor, and the smaller sub-blocks represent the simulation cells for that processor. Each processor's original blocks are given a unique colour.



During the first phase of the halo exchange, each processor informs the processors above and below of the contents of its subdomain. Note that the lattice is periodic.



Then, they swap left and right.



Finally, they swap forwards and backwards.



Note that this algorithm results in the correct orientation of the corner cells.