RUNNING OPEN-SOURCE HPC APPLICATIONS IN THE CLOUD WITH ALCES FLIGHT GRIDWARE
Before the advent of cloud computing, high performance computing environments only existed within the confines of organisations’ data centres. Traditionally, such an environment would have been costly to implement and run. Organisations spend millions building and maintaining their HPC environments, and this usually involves various departments such as infrastructure, mechanical and electrical facilities as well as the IT department.
In recent years, cloud resources have grown in terms of capacity and capabilities, to such an extent that many organisations are now considering the cloud to handle workload that extends to high performance computing. The cloud’s highly scalable nature and its pay-as-you-use scheme makes it attractive to organisations that either do not have space or are in need of an agile solution for their computational workloads.
WHAT IS ALCES FLIGHT COMPUTE?
Alces Flight Compute provides a fully-featured, scalable High Performance Computing (HPC) environment for research and scientific computing.Compatible with both on-demand and spot instances, Flight rapidly delivers a whole HPC cluster, ready to go and complete with job scheduler and applications.
Clusters are deployed in a Virtual Private Cluster (VPC) environment for security, with SSH and graphical-desktop connectivity for users. Data management tools for POSIX and S3 object storage are also included to help users transfer files and manage storage resources.
AWS and Alces Flight Compute
Since the beginning of 2017, NUS IT has been working with AWS[1] to implement an HPC environment using products from their partners such as Alces Flight[2], to enable quick access to HPC applications in the cloud. The pay-as-you-use nature of the AWS cloud offering means that there is virtually no upfront infrastructure to invest in, and no need to worry about utility costs, service level maintenance etc. Any researcher can now spin up an AWS compute-optimized instance that they can login to, in virtually no time at all.
To enhance security, NUS IT and AWS has established a secure VPN connectivity to their cloud infrastructure, and researchers can safely and quickly access HPC compute clusters in their virtual private cloud, without worrying about Internet exposure.
Software installation is also made easier with partners like Alces Flight. Alces Flight Gridware has simplified the installation of over 1,000 Linux open-sourced HPC applications, libraries and compilers, and these are integrated with AWS[3] – meaning that Alces Flight HPC clusters can make use of features such as autoscaling and spot bidding.
What are the applications available in Alces Flight Gridware? Their application repository[4] lists open-sourced software from many categories, including: Biochemistry (Amber, Beast, etc), BioInformatics, Bio-Physics, Chemistry, Engineering (ANSYS Workbench, OpenFOAM), Medicine, Physics, Statistics, Machine Learning, etc. The list is long, and includes tools and libraries such as MPI libraries (mvapich2, openmpi) and compilers that researchers will find useful. The advantage of using such a service is that the software repository is kept up-to-date by the vendor, and there is no need to manage the software versions ourselves.
Conclusion
In a nutshell, the Alces Flight Compute, running in AWS cloud, together with Alces Flight Gridware provide a quick, flexible and highly scalable HPC environment to run popular open-source HPC applications for NUS researchers.
All the above are now available via NUS IT’s HPC Cloud Portal. If you are interested in trying this out, please contact us at: -cloudhpc@nus.edu.sg nusit – we will be happy to brief you on its usage.
References
[1] See: http://aws.amazon.com/
[2] See: http://www.alces-flight.com/
[3] Alces Flight is also available in other public clouds such as Microsoft Azure.
[4] See: http://docs.alces-flight.com/en/stable/apps/gridware.html and https://gridware.alces-flight.com/software
