Serial/Parallel Computing of Fluent/CFX Solver
The most efficient and convenient way to run Fluent solver for your CFD simulations, which need hours, days or weeks to finish, is to run the solver in batch/parallel mode. First, you shall setup the CFD problem, including mesh, models, boundary conditions etc., on an interactive Fluent interface. Next save the initial problem to Fluent case and data file pair. Then you can choose a suitable batch queue to submit your case for computing.
Batch Serial Fluent/CFX Job
Submit Batch Fluent Job
Small or medium sized jobs, which may need about one hour or slightly longer to complete, are recommended submitting to run in serial-batch mode (on single CPU processor/core). The small case is recommended to be submitted to queue “serial” and large case is recommended to be submitted to queues “parallel8” or “parallel” for parallel computation.
For example, the following is the command used to submit a serial-batch Fluent job to queue serial:
fluent-job -q serial -o myjob.o "fluent 3d -gu -i myjob_script"
where:
-q serial -o myjob.o fluent 3d -gu -i myjob_script |
define to submit Fluent job to queue serial specify the filename to keep the standard output message for the job run Fluent 3D solver without graphics display (required for batch) specify the Fluent command script file in text. The file contains the Fluent commands for the execution, you may use Unix text editor, such as vi, emacs or pico, to edit this file. |
Here is an example of the script inside the file myjob_script (for steady-state case):
rcd test.cas it 1000 wd test_1000.dat exit y |
# read test.cas and test.dat into Fluent solver # do 1000 iterations # store (write data) the solution to test_1000.dat # exit Fluent solver # confirm the exit with “y” or “yes“ |
There have many ways to create/edit the text-based script file. To learn those ways, please check at the article on editing files on Linux from the HPC@NUS Issue No. 7 in 2005 March.
When the job is completed, you should find the final data file “test_1000.dat” at the working directory where you submitted the job. To do the post-processing and visualise the results, launch a Fluent interface interactively from the command line and read the case and data files into it to do the visualisation.
Here is another example of the script for unsteady flow simulation, or time-dependent flow:
/file/rcd myfile.cas /file/autosave/data-frequency 100 /solve/set/time-step 6 /solve/dual-time-iterate 1000 20 wd mydata_new.dat exit y |
# read myfile.cas and myfile.dat into Fluent solver # define auto-save frequency of result data as per 100 iterations # set time step as 6 seconds # start to do iterations of 1000 iterations in total 20 sub-iterations within one time step # store (write data) the final solution to mydata_new.dat # exit Fluent solver # confirm the exit with “y” or “yes“ |
When the job is completed, you should find 10 (=1000/100) sets of intermediate data file and the final data file “mydata_new.dat“. You can use those data files to visualise the results.
Submit Batch CFX Job
CFX batch job can be submitted when you have the CFX .def file created correctly. The job submission is as below.
cfx-job -q serial -o myjob.o "cfx5solve -batch -def case1.def -name case1_new"
where:
-q serial -o myjob.o cfx5solve -batch -def case1.def -name case1_new |
define to submit CFX job to queue serial specify the filename to keep the standard output message for the job run CFX solver in batch mode to process the problem defined in cas1.def, and save the results into case1_new. |
Batch Parallel Fluent/CFX Job
For long hour Fluent simulations, parallel computing with Fluent solver is encouraged. By running simulations on parallel model, or using multiple CPUs, the computing time will be shortened and hence improve the production. Fluent solver is well and ready parallelised using MPI and is able to run in parallel manner using multiple CPUs on all HPC clusters. The parallel queues supported Fluent solver include parallel8 and parallel.
To use parallel computing for a Fluent simulation, similarly as the serial-batch job process you need to setup your Fluent model first and save the model in Fluent case and data files. Initial debug and/or trial is necessary to ensure the model is properly defined.
At the same time, prepare the Fluent script file that is used for the job submission (same as above). To submit a parallel Fluent job, run command “fluent-job” and follow the 6 steps:
fluent-job 1. Fluent version [16, 18, default=18]: 18 2. Solver type [2d, 2ddp, 3d, 3ddp]: 3d <Enter> 3. Fluent Inputput/Script file Name: myjob_script <Enter> 4. Output file Name [e.g. stdout_fluent.o]: <Enter>
---------------------------------------- Your Fluent batch job is: ---------------------------------------- 1. FLUENT VERSION..........> 18 2. SOLVER..................> 3d 3. Input File..............> myjob_script 4. Stderr Output File......> STDOUT_fluent 5. Num of CPU..............> 12 6. Run Queue...............> parallel12
Enter <1,2,3,4,5> to change the above inputs, or Enter <y,n> to submit/discard this job. <1,2,3,4,5,y,n,q(quit)>: Y <Enter>
There have two mandatory inputs for the parallel Fluent job: 1) the solver type and 2) the Fluent input script file. The solver type is related to your Fluent model, in either 3d or 2d.
For Steps 3, 4 and 5, you can use the default settings (just press Enter to accept) which was tuned to choose the most suitable parameters and resources for the job for you. You can also input your own parameters for any specific reasons or purpose.
A summary of the batch job will be listed after all the necessary information is provided. Modifications to the batch job are allowed, or enter “Y” to confirm the job submission.
The post-processing is the same as mentioned above.
CFX solver can be submitted to run in parallel mode as below with the .def is ready to process.
cfx-job -q parallel12 -o myjob.o "cfx5solve -batch -def case1.def -initial case1_001.res –par-local –partition 12"
Job Monitor
You can monitor the progress of your job with the following PBS commands:
qstat |
to list all your batch jobs |
qstat -f <jobid> | to list a particular batch jobs in detail |
qstat -ans <jobid> | to find out the pending reason of job |
qcat -j <jobid> | to display the standard output of a particular job |
qdel <jobid> | to terminate a particular job |