hpc-tips-and-tricks

By Kuang Hao, Research Computing, NUS Information Technology, on 24 September 2020

NLP (Natural Language Processing) is not easy. When you are getting interested in it, immediately you realize that raw texts cannot be fed to our machine who only understands 0s and 1s. To make things harder, meaning of words can be ambiguous; sentences’ interpretation varies depending on the context; and there are hundreds of languages with different syntax & grammar rules. Fortunately, thanks to recent pioneers in NLP, there are standardized pipelines to follow and easy-to-use tools at hand.

This article introduces the most popular tools in NLTK, how to set up and use them, as well as its availability on our HPC clusters.

By Yeo Eng Hee, Research Computing, NUS Information Technology, on 24 September 2020

The use of the Cloud to store data and to run HPC analysis jobs has become more pervasive, and one important question users of Cloud storage should ask is: “How secure is my data in the Cloud?”.  While some cloud storage providers give users the assurance that their storage in the Cloud is encrypted, and some even provide encryption services and, key management services.  It is always good to have an additional layer of assurance. We explore one such open-source tool for encrypting files and folders in the Cloud: Cryptomator, an open-source encryption tool that allows us to safely store the key to our data in our own computer, instead of in the Cloud.

By Ku Wee Kiat, Research Computing, NUS Information Technology, on 24 September 2020

As HPC users, we often work in the Linux terminal. We need to be productive in a pure text environment where the only interface you can use to control it is the keyboard. What you enter is what you get, if you do not execute the command to list a directory’s contents, you will not know what is inside the directory. So how can we do the things in the most productive way? Yes, by using shortcuts!

By Kumar Sambhav, Research Computing, NUS Information Technology, on 15 May 2020

Managing Big Data on Hadoop clusters has seen a lot of paradigm shift in the recent times. From Sysadmin managed clusters at the command line level to on-prem centrally managed platforms like Cloudera, Hortonworks and MapR. All of these platforms have a primary problem of being dependent on physical hardware resources. Read on to discover how EMR addresses this shortcoming in the cloud.

By Vamshidhar Gangu, Research Computing, NUS Information Technology, on 15 May 2020

Tips on how to run MATLAB Parallel Computing Toolbox jobs properly in our HPC cluster. Potential conflict between multiple concurrent jobs is addressed in this article.

By Ku Wee Kiat, Research Computing, NUS Information Technology, on 15 May 2020

Ever needed to have a directory of files renamed to a certain format? Extract lines with certain keywords from log files? Even create csv files from semi-structured logs? There is no need to bust out the custom python or R scripts or install any software when most simple tasks can be solved at a much faster speed using Bash tools.