A Storage Solution for a High-Performance Computing (HPC) cluster

Description

The current storage appliances are nearing end of life and there is a requirement for a new storage system that meets ICBF’s needs. At present, there are eight compute nodes in the cluster, consisting of • 2 Dell R930 servers (Quad Intel Xeon E7-8893v3 with 6TB) • 2 Dell R940’s (Quad Intel Xeon Gold 6242 with 3TB RAM) • 4 Dell R960’s (Quad Intel Xeon Platinum 8444H with 3TB RAM) Going forward, there is a requirement to be able to run production, development and test jobs with the dev/test jobs having minimal impact on production. Rather than having the eight nodes working together in one cluster, the intention is to set up three independent environments, development, test and production and allocate certain nodes to each of these environments. For example, in the diagram below, nodes 1 to 3 run production jobs, nodes 4 and 5 run test jobs and nodes 6 to 8 run development jobs. A requirement of the new solution is that storage can be allocated independently to each environment. We need to ensure that any jobs that run on the development and test partitions have a minimal impact on production. The workload requires a minimum Input-Output per second (IOPS) of 50,000 for the production workloads. The IOPS required for the development and test environments do not necessarily need to be at this level, a minimum requirement of 30,000. There is a requirement for a minimum of 300TB of usable storage, to be split between production, development and test. All this 300TB must be available to the end user. Please see diagram in Tender Document The current storage appliances are nearing end of life and there is a requirement for a new storage system that meets ICBF’s needs. At present, there are eight compute nodes in the cluster, consisting of • 2 Dell R930 servers (Quad Intel Xeon E7-8893v3 with 6TB) • 2 Dell R940’s (Quad Intel Xeon Gold 6242 with 3TB RAM) • 4 Dell R960’s (Quad Intel Xeon Platinum 8444H with 3TB RAM) Going forward, there is a requirement to be able to run production, development and test jobs with the dev/test jobs having minimal impact on production. Rather than having the eight nodes working together in one cluster, the intention is to set up three independent environments, development, test and production and allocate certain nodes to each of these environments. For example, in the diagram below, nodes 1 to 3 run production jobs, nodes 4 and 5 run test jobs and nodes 6 to 8 run development jobs. A requirement of the new solution is that storage can be allocated independently to each environment. We need to ensure that any jobs that run on the development and test partitions have a minimal impact on production. The workload requires a minimum Input-Output per second (IOPS) of 50,000 for the production workloads. The IOPS required for the development and test environments do not necessarily need to be at this level, a minimum requirement of 30,000. There is a requirement for a minimum of 300TB of usable storage, to be split between production, development and test. All this 300TB must be available to the end user. Please see diagram in Tender Document