SC 2017 Bird of a Feather

Cross-Layer Allocation and Management of Hardware Resources in Shared Memory Node

Abstract

The question of efficiently reserving cores by a subset of an application, different independent libraries or by runtime systems, can be a nightmare. With more complex applications, runtime systems and OS, the question of intelligently cooperating in allocating cores in an isolated fashion to the different parts of an application is a key problem. A more integrated solution would allow a decreased coordination cost via a cohabitation of the different elements of the software stack with collaborative access to the available hardware resources, instead of a usually time-based competitive one.

The goal of the BoF is to gather the community working in the area of thread allocation (from runtime system designers to compilers) to discuss these issues, share visions, propose solutions, and coordinate a worldwide effort.

Problematic

Even if supercomputers are composed of distributed memory nodes, managing each node efficiently is a key issue for performance. As we are facing an important increase in the number of cores on each node with a deep hierarchy, allocating and managing the threads that are executed on the cores is a challenge that requires cooperation and coordination between the different components of the software stack.

Figure 1 shows a typical example of this issue.

Fig 1. Software stack with different parts using threads

At the application level some part of the application can use pthreads to program its concurrency. It can also rely on computational libraries (e.g. MKL) that can also be multithreaded. Moreover, other parts of the application can use OpenMP parallel sections (which are also implemented with threads). Furthermore, the runtime system can implement its own parallelism (e.g. progress thread for communication) with threads. As of now, each part of the application is most of the time unaware that some other parts are also using threads. Therefore, these threads can be bound to the same subset of cores and step on each other, leading to a slowdown of their respective execution. Moreover, currently, no mechanism exists to query for unused cores, to reserve some of them, or to check which part of the application is using them.

Tools like hwloc provide a portable and static view of the node topology but do not provide any intelligent strategy to share resources between different application parts. Therefore, even if most elements of the HPC software stack that query topology information use hwloc, we are lacking a mechanism to allocate and manage cores across the whole stack.

Why a BoF?

Many research teams have identified the above problem as part of their agenda. This may be called application composition, dynamic topology management or topology-aware core selection, etc. In any case, the basic problem is the same. However, proposed and envisioned solutions may differ. We think it would be of great interest for the HPC community to meet and discuss the issue and the envisioned solutions. The goal is to share the visions, the ideas, and coordinate a worldwide effort. The ultimate goal would be to see if we can come up with some way to address the problem and standardize across the software stack how information related to core management is expressed.

We think that SC is the perfect venue for hosting the BoF as many interested researchers will be attending the conference. Moreover, the HPC community concerned with the problematic will be there. This encompasses: runtime systems, MPI, topology modeling, compilers, operating systems, OpenMP, hardware, etc.