High Performance Workspace for Computational Clusters
Two High Performance Workspaces are available on all HPC systems: /hpctmp and /hpctmp2. Each user is allocated 500GB space by default in both /hpctmp and /hpctmp2. Users can access these two workspace from any login nodes. The High Performance Workspace can help to improve the overall data processing performance of your jobs, therefore, you are always encourage submit your jobs from these two workspace:
HPC Workspace | Disk Quota | File System Type |
---|---|---|
/hpctmp2 | 500 GB | Lustre File System |
/hpctmp | 500 GB | NAS File system |
The two workspaces are designed to be used like a normal Linux directory. Hence, all Linux file management commands (like ls, cd, mv, cp, rm) and C and Fortran file I/O functions calls are applicable.
For the first time to use the workspace, you shall create your own folder/directory (recommended to use your username as the folder/directory name) under the workspace using the Linux commands illustrated below.
Linux Command | Description |
---|---|
# cd /hpctmp2 |
change directory to /hpctmp2 |
# mkdir a0123456 |
create a sub-folder using your username “a0123456” |
# ls -ld a0123456 |
list and check the sub-folder permission |
# cd a0123456 |
change directory to the sub-folder |
# pwd a0123456 |
print and show the absolute location of this sub-folder |
Once you created your own folder in /hpctmp or /hpctmp2, you can access and use it by changing to this directory as:
# cd /hpctmp2/a0123456 |
change directory to your folder in /hpctmp2 |
You can copy files between your HPC home directory and workspace as below:
Linux Command | Description |
---|---|
# cp ~/file1 /hpctmp2/a0123456/ |
Copy file1 from HPC home directory to /hpctmp2/a0123456. |
# cp /hpctmp2/a0123456/file2 ~/file2 |
Copy file2 from /hpctmp2/a0123456 to HPC home directory. |
Please note that these two workspaces are designed as temporary workspace, it is not recommended to use them for long term mass data archiving purpose. Files that are not in use should be copied back to your own PCs or local storage media for archiving, and then to be deleted from the workspace.
To transfer files/folders between the local disk on your Windows computer and workspace /hpctmp2, please follow this instruction (PDF) using the Secure Shell file transfer utility or other equivalent tools. For Max/Linux users, you can use the equivalent secure file transfer utilities or use scp command from the terminal.
Automatic Purging of Older Files
To achieve optimum performance, backup (including snapshots) of data and files is not enabled in the workspaces. You are recommended to backup your data/files as-and-when it is necessary to avoid data loss.
To maintain the performance and maximize the usage, automatic purging of files older than 60 days is applied to both workspaces.
Disk Quota
Each user is allocated 500 GB disk space in both /hpctmp and /hpctmp2, therefore, the total disk quota for each user is 1TB. Users are encourage to do regular house-keeping to prevent the disk quota being exceeded and your running jobs are affected. You can check the usage and disk quota via the following customized command:
# hpc space -------------------------------------------------------------- Disk space for HPC home directory (quota -v) -------------------------------------------------------------- HPC Home Dir Usage Quota Limit(KB) Status /home/svu/ccev711 1221623, 3984384, 4194304, 30.66% ============================================================== -------------------------------------------------------------- Disk space for HPC workspace at /hpctmp2 ( lfs quota -u username /hpctmp2 ) -------------------------------------------------------------- HPC WorkSpace Usage Quota(KB) Limit(KB) Status /hpctmp2 0, 512000000, 563200000, 0.00% ============================================================== ----------------------------------------------------------------- Disk space for HPC WorkSpace ----------------------------------------------------------------- HPC WorkSpace Usage, Limit, Status /hpctmp/ccev711 8.86G, 500.00G, 1.77% =================================================================
If any user has a justifiable reason for more disk quota for the High Performance Workspace, you can request for more disk space in one of them. As /hpctmp is currently less used and easier to expand, users are encourage to put up the request for /hpctmp.
To submit your request, please follow the steps below:
• Login to the Service Portal
• Choose IT Service Catalogue
• Scroll down and choose HPC Service Request
• Scroll down and choose HPC Workspace. Click on Quota Increase Request. Choose either /hpctmp or /hpctmp2, fill in the online request form and Submit.