Design Patterns and Performance Analysis of Polymorphism in Multiphysics FE Assembly on GPU

The computation of elemental system matrices and right-hand-side vectors and their assembly into sparse linear algebra data structures is a key component of FE codes. This computation naturally exposes parallelism (loops over regions of PDEs with different coefficients or of different type, loops over elements, loops over integration points, etc). This parallelism can be exploited by implementing the computation in terms of functors that are executed in Kokkos parallel regions.

In this talk, we focus on such a Kokkos implementation in the context of a multiphysics FE code. Specifically, we investigate polymorphic design patterns to isolate commonality across physical applications and provide interfaces to accommodate physical application specificities (differences in data structures used for scalar-valued versus vector-valued FE basis functions, differences in actions of differential operators of different types, etc). The use of polymorphism on device can have performance impacts and is challenging in terms of code generation.

We will discuss how we used and compared static and dynamic polymorphic design patterns, and analyzed the performance impact, especially on GPU (virtual function call overhead, compiler optimisations, etc). The design patterns will be illustrated and the performance will be evaluated in the context of a computational electromagnetism simulation relevant to diffraction gratings.