It definitely seems using module purge to unload openmpi system module worked, since I can start Paraview Binary. However it seems I do need the openmpi module to be loaded for openfoam and other software which I plan on building (SU2, Elmer). For now I just using a bashrc function as a workaround:
function paraview {
module purge;
/home/CentOS-VM/.root/opt/ParaView-5.8.1-MPI-Linux-Python3.7-64bit/bin/paraview
module load mpi/openmpi-x86_64;
};
As this could call paraview using this function name, and when I quit, it automatically loads mpi module again. Only downside is I cant use paraFoam, gotta use touch .foam to create a file and then run paraview.