NAMD on the clusters
In order to use NAMD one must first load a compiler, MPI implementation and linear algebra library. NAMD is currently only built for the Intel stack so this implies:
NAMD is parallelised using the Charm++ framework which is somewhat different to the more commonly used MPI and OpenMP models. For this reason we briefly describe Charm++ and its options.
"Charm++ is a C++-based parallel programming system, founded on the migratable-objects programming model, and supported by a novel and powerful adaptive runtime system." (http://charm.cs.illinois.edu/manuals/html/charm++/1.html)
Charm++ is available as a module (
module load intel intel-mpi charm). Note that there is no need to load the Charm++ module to run NAMD
SMP mode in Charm++ spawns one OS process per logical node. Within this process there are two types of threads:
- Worker Threads that have objects mapped to them and execute entry methods
- Communication Thread that sends and receives data (depending on the network layer)
Charm++ always spawns one communication thread per process when using SMP mode and as many worker threads as the user specifies (see the options below). In general, the worker threads produce messages and hand them to the communication thread, which receives messages and schedules them on worker threads.
As we use SMP mode the most important option is the number of worker threads per compute node. This needs to be given explicitly as Charm++ cannot deduce it from the run-time environment.
This value should be one less than the number of CPU cores per node so that one is left free for the communications process.
Due to the SMP mode of Charm++ we need to specify 1 task per node and assign all the available CPUs to this task. The number depends on the cluster so you will need one script per processor type.
|Cluster||CPUs per node||Notes|
|Deneb phase 1||16||E5v2|
|Deneb phase 2||24||E5v3|
|Fidis||28||E5v4 and s6g1|
Example job script
We assume that you already know how to prepare input for NAMD - if not please contact your local specialist.
A sample script that asks for 2 nodes on Fidis and sets the
++ppn option based on the number of CPU cores you have asked for is given below:
This should then give an output starting with:
Please check that there are no warning messages given at the start which are often due to not correctly setting the number of tasks per node to be 1. An example of a "bad" run is as follows with the warning being "Warning: the number of SMP threads (256) is greater than the number of physical cores (24)"