Overview

Palmetto is Clemson University's primary high-performance computing (HPC) resource; heavily utilized by researchers, students, faculty, and staff from a broad range of disciplines.

Currently, Palmetto is comprised of 2021 compute nodes (totalling 23072 CPU cores), and features:

  • 2079 compute nodes, totalling 28832 cores
  • 595 nodes are equipped with 2x NVIDIA Tesla GPUs, with a total of 1194 GPUs in the cluster; 103 nodes each have 2x NVIDIA Tesla V100 GPUs
  • 4 nodes with Intel Phi co-processors (2 per node)
  • 17 large-memory nodes (with 0.5 TB - 2.0 TB of memory); in addition, 480 nodes have at least 128 GB of RAM
  • 100 GB of personal space (backed up daily for 42 days) for each user
  • 726 TB of scratch storage space for computing and a burst-buffer
  • 10 and 25 Gbps Ethernet, and 56 and 100 Gbps Infiniband networks
  • ranked 9th among the public academic institutions in the US (and 392nd overall among all world-wide supercomputers) on Top500 list, as of November 2019
  • benchmarked at 1.4 PFlops (44,016 cores from Infiniband part of Palmetto)
  • the cluster is 100% battery-backed

Compute Hardware

The cluster is divided into several "phases"; the basic hardware configuration (node count, cores, RAM) is given below. For more detailed and up-to-date information, you can view the file /etc/hardware-table after logging in.

Ethernet phases

Phases 0 through 6 of the cluster consist of older hardware with 10 Gbps Ethernet interconnect. Maximum run time for a single task is limited to 168 hours.

Phase Node count Cores RAM
1a 116 8 31 GB
1b 40 12 92 GB
2a 30 16 382 GB
2b 162 8 15 GB
3 224 8 15 GB
4 319 8 15 GB
5a 309 8 31 GB
5b 9 8 31 GB
5c 32 8 22 GB
5d 23 12 46 GB
6 65 24 46 GB

Infiniband phases

Phases 7-19 of the cluster consist of newer hardware with 56 Gbps Infiniband interconnect (Phases 7-17) and 100 Gbps Infiniband interconnect (Phases 18-19). Maximum run time for a single task is limited to 72 hours.

Phase Node count Cores RAM
7a 42 16 62 GB
7b 12 16 62 GB
8a 71 16 62 GB
8b 57 16 62 GB
8c 88 16 62 GB
9 72 16 126 GB
10 80 20 126 GB
11a 40 20 126 GB
11b 4 20 126 GB
12 30 24 126 GB
13 24 24 126 GB
14 12 24 126 GB
15 32 24 126 GB
16 40 28 126 GB
17 20 28 126 GB
18a 2 40 372 GB
18b 65 40 372 GB
18c 10 40 748 GB
19a 28 40 372 GB
19b 4 48 372 GB

GPUs

There are 595 nodes (phases 7a-8b, 9-11a, 12-19a) on Palmetto equipped with NVIDIA Tesla GPUs (M2075, K20m, M2070q, K40m, P100, V100).

Intel Xeon Phi accelerators

4 nodes (phase 11b) are equipped with Intel Phi co-processors (2 per node).

Big-memory nodes

Phase 0 consists of 6 "bigmem" machines with large core count and RAM (505 GB to 2 TB).

Phase Node count Cores RAM
0 6 24 505 GB
0 1 64 2 TB
0 5 32 750 GB
0 1 40 1.5 TB
0 1 40 1 TB
0 3 80 1.5 TB

Storage

Various options for storing data (on temporary and permanent basis) are available to researchers using Palmetto:

Location Available space Notes
/home 100 GB per user Backed-up nightly, permanent storage space accessible from all nodes
/scratch1 233 TB shared by all users Not backed up, temporary work space accessible from all nodes, BeeGFS Parallel File Sytem
/scratch2 160 TB shared by all users Not backed up, temporary work space accessible from all nodes, XFS
/scratch3 129 TB shared by all users Not backed up, temporary work space accessible from all nodes, ZFS
/scratch4 174 TB shared by all users Not backed up, temporary work space accessible from all nodes, BeeGFS Parallel File Sytem
/local_scratch Varies between nodes (99GB-800GB) Per-node temporary work space, accessible only for the lifetime of job

Additional high-performance and backed-up storage may be purchased for group usage. Please visit http://citi.clemson.edu/infrastructure for details and pricing.

Job scheduling

The Palmetto cluster uses the Portable Batch Scheduling system (PBS) to schedule jobs.

Condominium model

Palmetto cluster operates in a condominium model which allows faculty to purchase immediate access to compute nodes on the cluster. More information can be found in the Owner's Guide.

Acknowledging Palmetto Cluster

We would appreciate if all publications that include results generated using the Palmetto cluster include a short statement in the Acknowledgment section. As an example, the acknowledgment may look like this:

Clemson University is acknowledged for generous allotment of compute time on Palmetto cluster.