SC 2017 Bird of a Feather: Cross-Layer Allocation and Management of Hardware Resources in Shared Memory Node

Abstract

The question of efficiently reserving cores by a subset of an application, different independent libraries or by runtime systems, can be a nightmare. With more complex applications, runtime system and OS the question of intelligently cooperating in allocating cores in an isolated fashion to the different parts of an application is a key problem. A more integrated solution would allow a decreased coordination cost via a cohabitation of the different elements of the software stack with collaborative access to the available hardware resources instead of a usually time-based competitive one.
The goal of the BoF is to gather the community working in the area of thread allocation (from runtime system designers to compilers) to discuss these issues, share visions, propose solutions, and coordinate a worldwide effort.

Problematic

Even if supercomputers are composed of distributed memory nodes, managing efficiently each node is a key issue for performance. As we are facing an important increase of the number of cores on each node with a deep hierarchy, allocating and managing the threads that are executed on the cores is a challenge that requires cooperation and coordination between the different components of the software stack.
Figure 1 show a typical example of this issue.



swstack
Fig 1.Software stack with different parts using threads


At the application level some part of the application can use pthreads to program its concurrency. It can also rely on computational libraries (e.g. MKL) that can also be multithreaded. Moreover, other parts of the application can use OpenMP parallel section (which are also implement with threads). Furthermore, the runtime system can implement its own parallelism (e.g. progress thread for communication) with threads. As for now, each part of the application is most of the time unaware that some other parts are also using threads. Therefore, these threads can be bound to the same subset of cores and therefore step on each other leading to a slowdown of their respective execution. Moreover, currently, no mechanisms is to query for unused cores, to reserve some of them or to check which part of the application is using them.
Tools like hwloc provide a portable and static view of the node topology but does not provide any intelligent strategy to share resources between different application part. Therefore, even if most of the element of the HPC software stack that query topology information use hwloc, we are lacking a mechanism to allocate and manage core across the whole stack.

Why a BoF?

Many research team have identified the above problem as part of their agenda. This may be called application composition, dynamic topology management or topology-aware core selection, etc. In any case, the basic problem is the same. However, proposed and envision solutions may differ. We think it would be of great interest for the HPC community to meet and discuss the issue and the envisioned solution. The goal is to share the visions, the ideas and coordinate a worldwide effort. The ultimate goal would be to see if we can come-up with some come way to address the problem and standardize across the software stack how information related to core management is expressed.

We think that SC is the perfect venue for hosting the BoF as many interested researchers will be attending the conference. Moreover, the HPC community concerned with the problematic will be there. This encompasses: runtime-system, MPI, topology modeling, compilers, operating systems, OpenMP, hardware, etc.