Release notes for Slurm versions in AWS PCS

Changes implemented in AWS PCS

• You can configure gres.conf records for a compute node group with the gresCustomSettings property, and declare the schedulable GRES with the Gres custom Slurm setting. This lets you configure GPU topology, MIG devices, MPS, shards, and your own generic resources. For more information, see Configuring custom GRES settings in AWS PCS.

• AWS PCS enables Slurm GRES autodetection by default on compute node groups that have GPUs. AWS PCS enables autodetection only when the socket layout of the node is known. For more information, see Configuring custom GRES settings in AWS PCS.

• AWS PCS configures the CPU topology of a compute node group by default. It determines the number of NUMA domains from the instance type and sets Sockets, CoresPerSocket, and Parameters=numa_node_as_socket on each node. For more information, see Configuring hardware topology in AWS PCS.

• CommitDelay=1 is set by default in slurmdbd.conf. CommitDelay controls how long slurmdbd batches commits to the accounting database, and setting it improves slurmdbd performance. To use a different value, set CommitDelay in the slurmdbd.conf custom settings for your cluster.

• The Slurm REST API reads the username from the name field within the id claim of a JWT. The top-level sun, username, and custom userclaimfield claims are deprecated. Existing tokens continue to authenticate. For more information, see Authenticating with Slurm REST API in AWS PCS.

• A single sample AMI now supports every Slurm version that is not EOL, and the AMI name no longer contains a Slurm version. For more information, see Using sample AMIs.

• Clusters that use SPANK plugins support rolling updates when you update to 26.05 or later. Clusters that use Slurm CLI Filter Plugins support rolling updates when you update from 25.11 or later. For more information, see Updating the scheduler version of a cluster in AWS PCS.

• New custom settings are available: the Exclusive queue setting, the PurgeJobEnvAfter and PurgeJobScriptAfter custom slurmdbd settings, the PreserveCaseResource value of the slurmdbd Parameters setting, and the REBOOT_ONLY value of HealthCheckNodeState.

Deprecations and removals

• Slurm deprecates the ExclusiveUser queue setting and the OverSubscribe=EXCLUSIVE value in favor of the new Exclusive queue setting. AWS PCS continues to support ExclusiveUser and OverSubscribe=EXCLUSIVE on 26.05, but we recommend that you migrate to Exclusive.

• Slurm removes SchedulerParameters=enable_job_state_cache. Remove it from the custom Slurm settings of your cluster before you update to 26.05.

• SchedMD release notes: https://github.com/SchedMD/slurm/blob/slurm-26.05/RELEASE_NOTES.md

• SchedMD changelog: https://github.com/SchedMD/slurm/blob/master/CHANGELOG/slurm-26.05.md

Changes implemented in AWS PCS

• Scheduler audit logs are now delivered separately through the PCS_SCHEDULER_AUDIT_LOGS log type, simplifying troubleshooting and auditing with independent control over log delivery. For more information, see Scheduler audit logs in AWS PCS.

• Expedited requeue is enabled by default. Jobs that fail due to node issues (such as insufficient capacity errors) can be requeued with the highest scheduling priority using sbatch --requeue=expedite. This is controlled by the SchedulerParameters=enable_expedited_requeue setting.

• The requeue_delay parameter is available as a custom cluster setting with a default of 5 seconds. Previously, requeue delay was tied to credential expiration (70 seconds). Administrators can now configure this independently via SchedulerParameters=requeue_delay=<seconds>.

• HealthCheckNodeState now supports the START_ONLY value, which runs the health check program only at node startup (slurmd start).

• CommunicationParameters=disable_http is set by default to disable HTTP endpoints (metrics and health probes) introduced in Slurm 25.11. To re-enable these endpoints, set CommunicationParameters=enable_http. For more information, see Slurm metrics in AWS PCS.

Known issues

• Slurm 25.11 validates AllowQOS and DenyQOS partition settings even when AccountingStorageEnforce=QOS is not set. If a QOS referenced in AllowQOS or DenyQOS doesn't exist in the Slurm accounting database, slurmctld exits with a fatal error. Ensure that all QOS values listed in partition AllowQOS and DenyQOS settings exist in the accounting database before upgrading to or restarting Slurm 25.11.

• The slurmd log may show the error message error: cannot create url_parser context for http_parser/libhttp_parser. This is a known Slurm issue that occurs even when CommunicationParameters=disable_http is set. The error can be safely ignored and doesn't affect cluster operation.

• SchedMD release announcement: https://www.schedmd.com/slurm-version-25-11-0-is-now-available/

• SchedMD release notes: https://github.com/SchedMD/slurm/blob/slurm-25.11/RELEASE_NOTES.md

Changes implemented in AWS PCS

• The Slurm requeue_on_resume_failure SchedulerParameter is now Enabled by default.

• "stderr" was removed as an option for LogTimeFormat, as it was disabled in Slurm 25.05.

• AWS PCS supports Multi-cluster sackd configuration: login node can access multiple clusters.

• SchedMD release announcement: https://www.schedmd.com/slurm-version-25-05-0-is-now-available/

• SchedMD release notes: https://github.com/SchedMD/slurm/blob/slurm-25-05-0-1/RELEASE_NOTES.md

Changes implemented in AWS PCS

• AWS PCS supports Slurm accounting. For more information, see Slurm accounting in AWS PCS.

• SchedMD release announcement

• SchedMD release notes

Changes implemented in AWS PCS

• The new Slurm Step Manager module is now enabled by default in AWS PCS. This module provides significant benefits by offloading step management from the central controller to compute nodes, substantially improving system concurrency in environments with heavy step usage. To support this configuration and better isolate Prolog and Epilog process execution, new prolog flags (Contain, Alloc) are enabled.

• Hierarchical communication from controller to compute nodes is enabled to optimize Slurm intra-node communication, which improves scalability and performance. Additionally, the routing configuration now uses partition node lists for communications from the controller, instead of the plugin's default routing algorithm, enhancing system resiliency.

• A new hash plugin HashPlugin=hash/sha3 replaces the previous hash/k12 plugin. This is now enabled by default in AWS PCS clusters.

• Slurm controller logs now include enhanced auditing capabilities for all inbound remote procedure calls (RPC) to slurmctld. The logs include the source address, authenticated user, and RPC type before connection processing.

• SchedMD release announcement

• SchedMD release notes

Slurm settings you can change in AWS PCS

• The SuspendTime defaults to 60. Use the AWS PCS scaleDownIdleTimeInSeconds configuration parameter to set it. For more information, see the scaleDownIdleTimeInSeconds parameter of the ClusterSlurmConfiguration data type in the AWS PCS API Reference.

• The MaxJobCount and MaxArraySize is based on the size you choose for the cluster. For more information, see the size parameter of the CreateCluster API action in the AWS PCS API Reference.

• The SelectTypeParameters Slurm setting defaults to CR_CPU. You can provide it as a value for slurmCustomSettings to set it when you create a cluster. For more information, see the slurmCustomSettings parameter of the CreateCluster API action and SlurmCustomSetting in the AWS PCS API Reference.

• You can set Prolog and Epilog at the cluster level. You can provide it as a value for slurmCustomSettings to set it when you create a cluster. For more information, see CreateCluster and SlurmCustomSetting in the AWS PCS API Reference.

• You can set Weight and RealMemory at the compute node group level. You can provide it as a value for slurmCustomSettings to set it when you create a compute node group. For more information, see CreateComputeNodeGroup and SlurmCustomSetting in the AWS PCS API Reference.