LARGE-SCALE PARALLEL COMPUTATIONS USING NSCC SUPERCOMPUTER
About NSCC
The NSCC supercomputer, provided by the National Supercomputing Centre Singapore (NSCC), is a state-of-the-art national facility with petascale-standard computing resource to support high performance computing (HPC) for scientific and engineering computing needs in Singapore.
The NSCC supercomputer was opened for alpha test in March 2016 and the beta test will be launched in June 2016. As one of the founding institutions, all NUS researchers are eligible to access and run larger-scale parallel computations using thousands of CPU cores on the NSCC supercomputer.
Key Technologies of the NSCC Supercomputer
The NSCC supercomputer is built with the following key technologies and features:
• Total computing power of 1 PFlops
• Total storage capacity of 10 PB
• Large shared-memory computing server with 1TB, 4TB and up to 6TB
• 128 GPU compute nodes for acceleration
• Very High bandwidth interconnect with EDR Infiniband, 100 Gbps
• High bandwidth Infiniband (40 Gbps) connection from the login nodes in NUS
• The fastest intercontinental links (100G to US Internet2 and 10G to EU GÉANT via TEIN4), as anchor of the global InfiniCortex project
Enrollment for NSCC Supercomputer
NUS staff and students can access the NSCC supercomputer by registering at the enrollment portal http://user.nscc.sg/saml/. The enrollment user guide is available here.
Access to NSCC Supercomputer
You can access NSCC supercomputer either via SSH login-based Linux command terminal or via the portal login-based Display Manager (DM). HPC related tasks such as program compilation, job submission and management etc. can be performed on the SSH login terminal; if you need to visualise large data set generated and stored on the NSCC system, you can use Display Manager by logging into the portal from your office computer.
The SSH login address of the login nodes is: nus.nscc.sg. Four login nodes are installed in NUS campus facilitating direct access of the NSCC supercomputer for users working in NUS premise. The login nodes are connected with the NSCC system via a 40 Gpbs Infiniband network, which provides a seamless and high performance access of the NSCC supercomputer’s home directory from NUS premise.
The Display Manager portal address is https://nusweb.nscc.sg -> Display Manager. It enables you to use the remote high-end graphics server for large data set visualization directly on your office computer, without downloading those data back to your local computer. The detailed instructions are available here.
Parallel Computations Supported on NSCC Supercomputer
The PBS Pro job scheduler is used to manage all the computational jobs in batch queues. The table below explains the major batch computing queues, computing resources allocated for the queues and targeted application/jobs.
| Queue * | Description of the Queue | Computing Resource | Suitable for: |
|---|---|---|---|
| large | Large Memory queue | 1TB to 6TB shared-memory per job,24 hours max walltime. | Large computational jobs that demands huge shared-memory system. |
| long | Long Computing queue | Up to 24 cores per job,120 hours max walltime. | Small jobs running on a few cores but need days to complete. |
| dev | Development and Test queue | Up to 48 cores per job,1 hour max walltime. | Development, test and debug of program/application, with short waiting time. |
| normal | Standard Computing queue | Up to 3000 cores per job,24 hours max walltime. | Large scale parallel computational jobs that can perform well on many cores. |
| gpu | GPU Computing queue | One GPU node per job,24 hours max walltime. | GPU parallel application/simulations. |
* Names and configuration of queues may update by NSCC. Please refer to http://help.nscc.sg/user-guide/ for detailed instructions.
There are a number of open-source software tools/packages installed on the system. Check out the list here. You can submit a request for the open-source software that you would like to run on the NSCC supercomputer.
About the NUS HPC System
As a national supercomputer, NSCC system is designed to support and promote large scale parallel computations. You are encouraged to use the powerful system running your computations on thousands of CPU cores or even more, provided your application is well-parallelized and can scale up very well on large number of cores.
In NUS, Computer Centre continues to provide local HPC resources to the NUS research community. This includes the standard HPC computing cluster, GPU computing cluster, parallel filesystem for scratch space, and also a few common and heavily used licensed software packages such as Abaqus, ANSYS/Fluent, Gaussian and Matlab, etc. It is unclear if these licensed software packages will be available on the NSCC supercomputer and when, as there are known issues about licensing and pricing.
You can continue using the local NUS HPC resources, taking into consideration the following benefits and offerings:
• Your application works optimally in existing HPC cluster
• Your computations take weeks or up to a few months uninterrupted to complete
• Your computations need to run on the specific licensed software available in NUS
• Your analyses involve huge number of serial or small scale computational tasks
