# Improving speed for thresholding of list of values (custom selection+extractSelect)
**URL:** https://discourse.paraview.org/t/improving-speed-for-thresholding-of-list-of-values-custom-selection-extractselect/17625
**Category:** ParaView Support
**Created:** [June 17, 2026, 11:45am UTC](https://discourse.paraview.org/t/improving-speed-for-thresholding-of-list-of-values-custom-selection-extractselect/17625 "2026-06-17T11:45:34Z")
**Posts on this page:** 4
**Page:** 1
### Author:  [@otaolafr](https://discourse.paraview.org/u/otaolafr)
#### Post date: [June 17, 2026, 11:45am UTC](https://discourse.paraview.org/t/improving-speed-for-thresholding-of-list-of-values-custom-selection-extractselect/17625/1 "2026-06-17T11:45:34Z")
Hello,
I need to extract cells with specific values of a specific field. For this I am using the following lines of code:
```auto
SetActiveSource(filter)
QuerySelect(QueryString=f'(isin({fieldName}, {listOfValues}))', FieldType=FieldType, InsideOut=complementary)
extracted = ExtractSelection(Input=filter,registrationName=registrationName)
SetActiveSource(filter)
ClearSelection()
```
the listOfValues can be millons therefore this process can be slow, but it works for float, ints and whatever kind of elements (cells or points). I had been trying to improve the speed of this with programmable filters but to whatever i test using numpy and vtk i never get around the same speed as the previous cited lines of code.
So I assume there is no fastest way to do so? If I need to assume that this IS the fastest way (anyone welcome to tell me if there is another fastest way). when doing QuerySelect there is a GUI/visualization part (where the selected elements becomes pink, and then the `ClearSelection()` will clean it and continue the script, my question is there any way to setoff the visual feedback of the QuerySelect? I am looking to improve further this speed and I execute all the lines at same time, therefore I dont care about ‘showing’ the pink selected cells.
---
### Author:  [@otaolafr](https://discourse.paraview.org/u/otaolafr)
#### Post date: [June 18, 2026, 12:53pm UTC](https://discourse.paraview.org/t/improving-speed-for-thresholding-of-list-of-values-custom-selection-extractselect/17625/2 "2026-06-18T12:53:28Z")
hey @Kenichiro-Yoshimi,
sorry for bothering with a direct tag, from the post you made on my other post, I tried implementing something based on it that would use the same logic, but I am not achieving a speed up (actually it takes the double time), would you mind giving me your oppinion?
what I came up to:
```py
def extractSelectionByListOfValues_test(filter=[], listOfValues=[]):
if filter==[]:
filter= GetActiveSource()
if isinstance(filter,str):
filter=FindSource(filter)
# create a new 'Programmable Filter'
programmableFilter1 = ProgrammableFilter(registrationName='ProgrammableFilter1', Input=filter)
# find source
# Properties modified on programmableFilter1
programmableFilter1.Set(
RequestInformationScript='',
RequestUpdateExtentScript='',
PythonPath='',
Script=f"""import numpy as np
from vtkmodules.vtkCommonCore import vtkIdList
from vtkmodules.vtkCommonDataModel import vtkSelectionNode, vtkSelection
from vtkmodules.vtkFiltersExtraction import vtkExtractSelection
from vtk.numpy_interface import dataset_adapter as dsa
from vtk.util.numpy_support import numpy_to_vtk, vtk_to_numpy
from vtkmodules.util.numpy_support import numpy_to_vtkIdTypeArray
# --- Get inputs ---
def get_vtk_dataset(wrapped_input):
vtk_obj = wrapped_input.VTKObject
if vtk_obj.IsA("vtkMultiBlockDataSet"):
return vtk_obj.GetBlock(0)
return vtk_obj
typeOfIDs='CellIds'
#get the input object
vtk_ref = get_vtk_dataset(inputs[0]) # removedLayer
wrap_ref = dsa.WrapDataObject(vtk_ref)
#get the list of IDs in a np.array for typeOfIDs
gid_ref = np.asarray(wrap_ref.CellData[typeOfIDs]).ravel() #typeOfIDs="CellIds"
cell_array = np.asarray(wrap_ref.CellData["CellIds"])
listOfValues={listOfValues}
if listOfValues:
if isinstance(listOfValues,list):
gid_toExtract=np.array(listOfValues)
elif isinstance(listOfValues,numpy.ndarray):
gid_toExtract=listOfValues
else:
error
else:
vtk_tgt = get_vtk_dataset(inputs[1]) # finalChannel
nCells = vtk_tgt.GetNumberOfCells()
wrap_tgt = dsa.WrapDataObject(vtk_tgt)
gid_toExtract = np.asarray(wrap_tgt.CellData[typeOfIDs]).ravel() #typeOfIDs="CellIds"
out = self.GetOutputDataObject(0)
# Sort ref, then searchsorted to find which target points exist in ref
gid_ref_sorted = np.sort(gid_ref)
pos = np.searchsorted(gid_ref_sorted, gid_toExtract)
pos_clipped = np.clip(pos, 0, len(gid_ref_sorted) - 1)
mask = gid_ref_sorted[pos_clipped] == gid_toExtract
shared_local_ids = np.nonzero(mask)[0]
selection_ids = numpy_to_vtkIdTypeArray(pos_clipped[shared_local_ids].astype(np.int64), deep=1)
node = vtkSelectionNode()
node.SetFieldType(vtkSelectionNode.CELL)
node.SetContentType(vtkSelectionNode.INDICES)
node.SetSelectionList(selection_ids)
sel = vtkSelection()
sel.AddNode(node)
extract = vtkExtractSelection()
extract.SetInputDataObject(0, vtk_ref)
extract.SetInputDataObject(1, sel)
extract.Update()
out.ShallowCopy(extract.GetOutputDataObject(0))
""",
)
return programmableFilter1
```
vs
```py
def extractSelectionByListOfValues_old(filter=[], listOfValues=[]):
SetActiveSource(filter)
QuerySelect(QueryString=f'(isin({fieldName}, {listOfValues}))', FieldType=FieldType, InsideOut=complementary)
extracted = ExtractSelection(Input=filter,registrationName=registrationName)
SetActiveSource(filter)
ClearSelection()
return extracted
```
I was hopping a speedup due to paraview creating the ‘pink cells’ when using QuerySelect, but for my current test, extractSelectionByListOfValues\_old is running in 14 s, where extractSelectionByListOfValues\_test is taking 32 s.
the filter is a unstructured grid of `630 209 cells` and for the test I using the function several times for different lengths of listOfValues (`[1766, 367694, 2, 86990, 4, 27714, 20387, 11144, 26251, 8742, 27682, 10872, 18895, 19929, 1642, 4, 5, 4, 480, 2] cells` each test) so rader small, as in your test case the speed was crazy even for larger examples, i am afraid i am missing something.
thanks (and sorry for the tag, but since I was basing the funciton on your post, would appreciate your oppinion)
---
### Author:  [@Kenichiro-Yoshimi](https://discourse.paraview.org/u/Kenichiro-Yoshimi)
#### Post date: [June 19, 2026, 1:05am UTC](https://discourse.paraview.org/t/improving-speed-for-thresholding-of-list-of-values-custom-selection-extractselect/17625/3 "2026-06-19T01:05:07Z")
Hi @otaolafr,
A couple of things explain what you’re seeing:
Your 20 lengths sum to exactly 630,209 — you’re partitioning the entire grid into 20 groups. So the per-call overhead is paid 20×, and that’s what dominates — not the extraction itself.
That per-call overhead is two things your \_test adds and QuerySelect doesn’t:
1. Creating a fresh ProgrammableFilter every call.
2. Compiling the value list into the script string (listOfValues={listOfValues}). For the 367,694-value call, just compile()-ing that literal is ~0.85 s — paid every time.
I reproduced both paths with plain VTK (same vtkExtractSelection engine), and the extraction kernel itself is sub-second even for the large lists — so the bottleneck was never the extraction, it’s the per-call pipeline-creation + source-compilation overhead. You’re not missing anything.
So: don’t bake the values into the script (pass them as real data), and reuse one filter instead of rebuilding it 20×.
Also note your searchsorted→INDICES step is actually incorrect unless CellIds is already 0..N-1 (it feeds sorted positions as original indices); np.isin(gid\_ref, values) → np.nonzero(…)[0] gives the correct original indices and was the fastest path in my test.
---
### Author:  [@otaolafr](https://discourse.paraview.org/u/otaolafr)
#### Post date: [June 19, 2026, 5:49am UTC](https://discourse.paraview.org/t/improving-speed-for-thresholding-of-list-of-values-custom-selection-extractselect/17625/4 "2026-06-19T05:49:16Z")
Thanks for your answer and taking the time.
my issue is that i need to do different things with each of the 20 groups, for each of them i will apply another function that will do a lot of checks and extract its external surface, so I need to be able to differentiate them.
So maybe create a mask, so each of the 20s have a different value and after filter using the mask, to get ‘separate’ intermediate outputs where i will call the function? do you think there would be a big improvement by doing this? I am asking as i am jumping a little bit to the void, not sure that it will be outperform and i will encounter a lot of bottle necks as i dont know vtk library to the level of doing what i am doing in paraview with the filters themselves…
> 1. Compiling the value list into the script string (listOfValues={listOfValues}). For the 367,694-value call, just compile()-ing that literal is ~0.85 s — paid every time.
if i pass a filter and recover its values instead as i do with `vtk_tgt` i should get a speed up then? but not so much as the 367694 is the largest one by far, therefore the other ones wouls be less than 0.85 s so the speed up should be \>\>20s, no?
> Also note your searchsorted→INDICES step is actually incorrect unless CellIds is already 0..N-1 (it feeds sorted positions as original indices); np.isin(gid\_ref, values) → np.nonzero(…)[0] gives the correct original indices and was the fastest path in my test.
the cellIds beging in 0, but yes you are right, this part on your original code, was a little bit confusing for me to follow up. thanks again.
EDIT: also i just noticed that in pvbatch the overhead is crazy reduced! which is quite middle ground as i can check in paraview and if it works then scale up in pvpython. still appreciate your opinion but just posting it in case of others need.