pvbatch on Windows uses many CPU cores by default?

ParaView Support
,
1

Hi,
I have ParaView 5.13.2 (I downloaded ParaView-5.13.2-MPI-Windows-Python3.10-msvc2017-AMD64.msi from the official website) installed on Windows 11.

When I try to use pvbatch to render an orbit around a box (.py state file attached here:
orbiting_around_a_box.py (7.4 KB)):

pvbatch orbiting_around_a_box.py

I see that pvbatch is using almost all CPU core as shown below:

image
image1182×763 47.6 KB

Question:

How do I tell pvbatch to use only some specific number of processes?

I have tried with the two versions of mpiexec (see below the output of where.exe command) with different -np X where X is an integer between [2, 10], but on the tasks manager as shown in the screenshot above, the CPU usage always looks the same.

# for mpiexec command I tried: mpiexec, the first absolute path of mpiexec, and the second absolute path as well
# X is an integer
mpiexec -np X pvbatch orbiting_around_a_box.py

Other relevant information:

❯ where.exe pvbatch
C:\Program Files\ParaView 5.13.2\bin\pvbatch.exe

 where.exe mpiexec
C:\Program Files (x86)\Common Files\Intel\Shared Libraries\intel64\bin\mpiexec.exe
C:\Program Files\Microsoft MPI\Bin\mpiexec.exe
2

Its not about MPI, its about multithreading.

You should be able to control it by positioning the correct env var: VTK_SMP_MAX_THREADS.
More info here: https://www.kitware.com/vtk-shared-memory-parallelism-tools-2021-updates/

3

Could you please elaborate? I don’t understand your reply.

4

You say that pvbatch is using all your CPU cores even when you run it without mpi, it means that its not mpi using all your core, but its the native multithreading support of VTK that is using all your cores.

The number of cores used by the native multithreading support can be controlled using the environment variable, VTK_SMP_MAX_THREADS.

5

Thanks for the clarification. But this is still confusing because the documentation (7. Remote and parallel visualization — ParaView Documentation 5.12.0 documentation) clearly states that to run pvbatch in parallel you can use mpi.

So I think my question still holds: I don’t want to use multithreading or whatever, just want to use MPI with many processes to run pvbatch as linked in the documentation.

Thanks

6

Of course you can, but that is not your question.

I don’t want to use multithreading or whatever, just want to use MPI with many processes to run pvbatch as linked in the documentation.

Sure, use it as the documentation states, with mpirun or mpiexec.

But it will still uses all your cores because of multithreading.

7

Sorry, But this is so confusing. What is this SMP thing? Why do I need it? What has this to do with ParaView? Why it is not documented? … many questions.

I do believe that this deserve some explanation, or some context (I tried to read the blog post you linked, but it is too technical and not clear).

Thank you

8

symmetric multiprocessing, aka multithreading

Why do I need it? What has this to do with ParaView?

Its a feature of VTK, which is the engine of ParaView

Why it is not documented?

https://www.paraview.org/ParaView/index.php/Multi_Core_Processing#Threading_Methods

https://www.kitware.com/paraview-hpc-scalability-guide/

But I agree, this doc should be ported to the ParaView guide and updated.

1 Like
9

@Kenneth_Moreland: I do really appreciate your input here.
Thank you very much.

11

I created an environment variable on Windows:

image
image622×583 15.5 KB

I restarted Windows, and opened a new terminal (Powershell) and confirmed this env variable is correctly set:

❯ $env:VTK_SMP_MAX_THREADS
1

The problem still persist. When I run pvbatch it uses all the CPUs as shown in the first screenshot from my first post

1182×763 47.6 KB

12

This is unexpected, I would have expected this to work.

Edit: This is caused by TBB that ignore this env var. You should be able to set VTK_SMP_BACKEND_IN_USE to Sequential instead.

1 Like
13

I added another environment variable as shown below:

image
image610×581 15.2 KB

I restarted Windows, but that has no effect. :thinking:

14

Then I’m afraid I have no idea and more investigation is needed.