The Evolving Roles of HPC in Research and Enterprise Computing Support

Tan Chee Chiang, Research Computing (REC), NUS IT

The last issue I shared about greater enterprise adoption of HPC resources and technologies due to the emergence of AI as a common application. This round we will look into how the NUS IT Research Computing (REC) team has evolved to stay relevant in this new HPC-AI era.

Adding intelligent and cloud solutions development to the role

 

 

As shown in the middle column above, both the Research SW Engineer and HPC Specialist are our traditional support roles of research computing and will remain key in the foreseeable future. Over the past few years, the Data Engineer, AI Engineer and Cloud Specialist roles have been added to support not just the research computing but also the enterprise computing community.

Resources at the right-hand side are the backend resources that the team has to access to deliver both the HPC and AI solutions. While NSCC and SingAREN resources are used mainly by researchers, the On-Prem Infrastructure and Cloud resources are accessed by both research and enterprise applications.

Next, we will look into what you, as users or project collaborators, can expect from each of the roles above.

The different roles of REC

Each team member holds multiple roles. For example, it is necessary for a single person to be able to advise on data and AI engineering issues as they are inter-related, and on Cloud as more AI applications are expected to be Cloud-based.

Common services for all roles

General research computing technical support, programming training, software maintenance and support, and computing system design/deployment support.

Research SW Engineer

Working with researchers in application performance tuning and code parallelisation.

HPC Specialist

HPC system design, deployment, operation and support. HPC software tools and application software deployment, maintenance and support.

Data Engineer

Develop and automate data pipeline for data extract, transform and load (ETL). Design, deploy, and maintain data processing platform.

AI Engineer

Perform model design, training and tuning. Design, deploy and maintain AI model development platform.

Cloud Specialist

Advise and execute cloud deployment or migration of application software and services.

How do these evolving roles fit into the history of HPC at NUS?

Before 2000, we had the era of specialised hardware where custom-built supercomputers such as vector supercomputers were used. Around 2001 to 2010, was a period where commodity hardware such as x86 cluster and grid computing were deployed for HPC. From 2011 to 2020, we focused on catering for data centric HPC with new storage solutions, data management platform and high-speed research network development, during which we introduced the Data Engineering role.

Up to 2020, the roles of Research Software Engineer, HPC Specialist and Data Engineer were sufficient to support the various applications of HPC in research. Moving forward into the next era of AI-centric cloud-based HPC, the new AI and Cloud expertise we are developing will help to expand the use of HPC not just in research but also in enterprise applications.

How can you engage the REC team?

If you are already our HPC user, you can approach us through our service portal nTouch (“HPC Enquiries” under Catalog). If not, you can register for an HPC account here.

If you are not from the research community, you may approach your department IT contact if you like to explore AI project with us. They should be able to link you up with the right people. Most AI related projects are likely to involve various IT expertise across NUS IT and your IT contact will be the common contact point.