How can I fix "cannot find cred plugin for cred/munge" or (auth/munge) Plugin when starting slurmctld?

edit

If the munge plugin is missing from your system, you can build it from the Slurm source code. Here's how to build and install the munge plugin:

Download the latest Slurm source code from the official website (https://www.schedmd.com/downloads.php).

Uzip the slurm source. Now install the dependencies:

sudo yum install -y gcc make munge munge-devel munge-libs openssl-devel pam-devel mariadb-devel readline-devel

For Debian based systems use apt-get instead.

cd slurm-20.11.9
./configure

Build the Slurm plugins:

make

Once the build is complete, locate the munge plugin cred_munge.so in the src/plugins/cred/munge/.libs directory: bash

cd src/plugins/cred/munge/.libs

Copy the cred_munge.so plugin to the /usr/local/lib/slurm directory:

sudo cp cred_munge.so /usr/local/lib/slurm

Also copy the auth_munge.so plugin the same way:

cp ./src/plugins/auth/munge/.libs/auth_munge.so /usr/local/lib/slurm 

Make sure your slurm.conf configuration file has the following line:

CredType=cred/munge

Restart the Slurm services for the changes to take effect:

sudo systemctl restart slurmctld
sudo systemctl restart slurmd

Now the munge plugin should be properly installed and available for Slurm to use.