Are print statements disabled on non zero ranks in python pipelines?

In Situ Support
1

I have the following code in my python script:

if(rank == 0):
    print("print from rank 0")
    os.system("echo echo rank 0")
comm.Barrier()
if(rank == 1):
    print("print from rank 1")
    os.system("echo echo rank 1")
comm.Barrier()
print("hello again from rank %d"%rank)

Upon execution by the ParaView implementation of Catalyst 2 through an application running on 2 cores, I get the following output:

print from rank 0
echo rank 0
echo rank 1
hello again from rank 0

Note the absence of print from rank 1. I am quite confused by this. Are print statements disabled on non zero ranks? If yes, how is this done? How do I print on non zero ranks, for instance for debugging? I could not find any reference to this in the documentation. Are there other python methods that are disabled on non zero ranks?

Thanks,
Alexandre

2

Hi @Alexandre_Minot ,

this is indeed strange. As far as I know we do not do anything special regarding MPI and limiting any kind of screen output from ranks . In fact, there is an example catalyst script that prints per rank information.

In this particular example it could be that the output is buffered and you never get to see it. Try to add flush=True as argument in the print command or add sys.stdout.flush() before each barrier.

3

Thanks for your help. I’m having trouble building the fortran example. It get:

CMake Error at Catalyst2/Fortran90FullExample/CMakeLists.txt:19 (add_executable):
  Target "Fortran90FullExampleV2" links to target
  "catalyst::catalyst_fortran" but the target was not found.  Perhaps a
  find_package() call is missing for an IMPORTED target, or an ALIAS target
  is missing?

However, if I run the CxxPolyhedra example, I also get outputs only on rank 0. Could you add the rank number to the prints in the CxxPolyhedra example and post the output you get please?

4

I’m having trouble building the fortran example.

You will need to update catalyst. Try the v2.0.0-rc4 tag.

Could you add the rank number to the prints in the CxxPolyhedra example and post the output you get please?

Sure ! catalyst_pipeline_2.py (1.1 KB) is the new file
and below what I get using catalyst v2.0.0-rc4 and a recent ParaView build using commit e423f2cee420dd744027ad226c5f730759d367e1 from master

$  ./bin/CxxPolyhedraV2 ../catalyst_pipeline_2.py                           
executing catalyst_pipeline
(   0.357s) [pvbatch         ]        v2_internals.py:171   WARN| Module 'catalyst_pipeline_2' missing Catalyst 'options', will use a default options object
====================================================
------------------- rank 0/1 ----------------------                                                                                                                   
executing (cycle=0, time=0.0)
bounds: (0.0, 69.0, 0.0, 64.9, 0.0, 55.9)
velocity-magnitude-range: (0.0, 0.0)
pressure-range: (1.0, 1.0) 
-------------------------------------------------------------

====================================================
------------------- rank 0/1 ----------------------
executing (cycle=1, time=0.1)
bounds: (0.0, 69.0, 0.0, 64.9, 0.0, 55.9)
velocity-magnitude-range: (0.0, 6.490000000000001)
pressure-range: (1.0, 1.0) 
-------------------------------------------------------------

====================================================
...

$ mpirun -np 4 ./bin/CxxPolyhedraV2 ../catalyst_pipeline_2.py 
executing catalyst_pipeline                                  
executing catalyst_pipeline                                                        
executing catalyst_pipeline                        
(   0.417s) [pvbatch         ]        v2_internals.py:171   WARN| Module 'catalyst_pipeline_2' missing Catalyst 'options', will use a default options object
(   0.418s) [pvbatch         ]        v2_internals.py:171   WARN| Module 'catalyst_pipeline_2' missing Catalyst 'options', will use a default options object
(   0.420s) [pvbatch         ]        v2_internals.py:171   WARN| Module 'catalyst_pipeline_2' missing Catalyst 'options', will use a default options object
executing catalyst_pipeline              
(   0.426s) [pvbatch         ]        v2_internals.py:171   WARN| Module 'catalyst_pipeline_2' missing Catalyst 'options', will use a default options object
====================================================                               
------------------- rank 0/4 ----------------------                                
executing (cycle=0, time=0.0)                      
bounds: (0.0, 17.0, 0.0, 64.9, 0.0, 55.9)          
velocity-magnitude-range: (0.0, 0.0)               
pressure-range: (1.0, 1.0)                
-------------------------------------------------------------
                                                                                   
------------------- rank 2/4 ----------------------                                
executing (cycle=0, time=0.0)                      
bounds: (35.0, 52.0, 0.0, 64.9, 0.0, 55.9)         
velocity-magnitude-range: (0.0, 0.0)      
pressure-range: (1.0, 1.0)                         
-------------------------------------------------------------
                                                                                   
------------------- rank 3/4 ----------------------                                
executing (cycle=0, time=0.0)                       
bounds: (52.0, 69.0, 0.0, 64.9, 0.0, 55.9)         
velocity-magnitude-range: (0.0, 0.0)        
pressure-range: (1.0, 1.0)                         
-------------------------------------------------------------
                                                                                   
------------------- rank 1/4 ----------------------                                
executing (cycle=0, time=0.0)                      
bounds: (17.0, 35.0, 0.0, 64.9, 0.0, 55.9)          
velocity-magnitude-range: (0.0, 0.0)               
pressure-range: (1.0, 1.0)                         
-------------------------------------------------------------
                                                                                   
====================================================                               
------------------- rank 0/4 ----------------------                                
executing (cycle=1, time=0.1)                      
bounds: (0.0, 17.0, 0.0, 64.9, 0.0, 55.9)          
velocity-magnitude-range: (0.0, 6.490000000000001) 
pressure-range: (1.0, 1.0)                        
-------------------------------------------------------------
                                                                                   
------------------- rank 2/4 ----------------------                                
executing (cycle=1, time=0.1)                      
bounds: (35.0, 52.0, 0.0, 64.9, 0.0, 55.9)         
velocity-magnitude-range: (0.0, 6.490000000000001)
pressure-range: (1.0, 1.0)                         
-------------------------------------------------------------
...
5

I’m getting outputs for rank 0 only with your pipeline. Weird. Must be something on my end. I’ll keep digging, thanks!