Generate CSV of complete pipeline in Custom Application

Hi,

Is there a way to generate a CSV of complete pipeline (data, multiple filters e.g calculator, glyphs, extract selection) in a custom application code?

I want to generate the CSV from within the code and not from the GUI. In other terms, once I click a button (let’s say RunSim) then in my code I want to generate the CSV which I would send to my other code.

Yes, use python macros or custom C++ logic.

Thanks for the response. What do you mean by python macros?

And is there a function/functions already in paraview that I can use in C++?

Python macros: 12. Python & Batch: ParaView & Python — ParaView Documentation 5.11.0 documentation

And is there a function/functions already in paraview that I can use in C++?

Yes of course, but they are quite low level. create any filters you need using the pqObjectBuilder.

(As far as I know) Because there is no such command or option where I can just use it to generate a full CSV at the end so I was thinking of somehow traversing through my pipeline and convert the each filter’s output into CSV and then at the end merge all the CSVs.

But I am not sure which function to use to access the pipeline and then traverse. Secondly, how to convert each filter’s output to csv.

What do you mean by creating every filter using pqObjectBuilder?

The user will already apply the filter, in the code I want to generate CSV which has all the data related to those filters.

I’m afraid this is not the right approach at all.

Do not convert anything to CSV

Use ParaView filters to process your data and write it to csv at the end.

I am not sure how exactly I would do that.

So once the user has applied the filter, they will click my custom built button (RunSim) and that is when I need to create the csv at the end.

For this which filter are you referring to?

Sorry for my confusion

Which filter would i use at the end to process the data? And do you mean to do this after the user has applied their filters?

If I create a python macro:

  1. Will I be able to load it into my custom application as I load a filter or any other plugin during build time?

  2. Will I be able to call that macro from C++ code? I have a function in C++ which is called when I click a custom button on my custom application’s GUI and I want that function to call the macro.

No, you will be able to load at as a python macros, it’s not a plugin. If you are writing a custom app, it may be better to write a custom C++ logic instead.

Will I be able to call that macro from C++ code? I have a function in C++ which is called when I click a custom button on my custom application’s GUI and I want that function to call the macro.

Yes, this is possible.

In that case, would you be able to tell me the workflow of that?

Well, just add a bouton and write the C++ logic behind it. Please give it a try and let us know if you have issues.

Well i have created the button and it works fine, I am just not sure about the right way to approach the C++ logic to create the CSVs for all filters output.

Is there a way to just generate the CSV once which has all filters output? (I could not find that way)

I believe once the button is clicked I would have to access all the filters in the pipeline browser through C++ code (this is the part where I am stuck) and then create the CSVs one by one for each filler.

maybe you are looking for pqSaveDataReaction::SaveActiveData ?

Not exactly. So this will only save data for the active filter, right? But i want data for all the filters.
So maybe a function which gives access to all the filters which I can iterate over and save the data for each filter?

Or maybe a function where i can make all filters active and then save all their data in CSV

So, with this code I am able to generate CSV files for each filter but is there a way to somehow to just generate 1 file where the data for all the filters are added.

Note that each filter and the source has different data columns (attributes)

void myMainWindow::on_menuRunSimulation_aboutToShow()
{
    pqServerManagerModel* smModel = pqApplicationCore::instance()->getServerManagerModel();
    QList<pqPipelineSource*> sources = smModel->findItems<pqPipelineSource*>();
    vtkSMSession* activeSession = vtkSMSession::SafeDownCast(vtkSMProxyManager::GetProxyManager()->GetActiveSession());

    int sourceIndex = 0;
    foreach(pqPipelineSource* source, sources)
    {
      logFile << "Source class: " << source->getProxy()->GetClassName() << std::endl;
    logFile << "Writing file for source: " << source->getProxy()->GetClassName() << std::endl;
        vtkSMSourceProxy* sourceProxy = vtkSMSourceProxy::SafeDownCast(source->getProxy());
        sourceProxy->UpdatePipeline();

        // Get the client-side object's class name
        vtkObjectBase* clientSideObjectBase = sourceProxy->GetClientSideObject();
        vtkObject* clientSideObject = dynamic_cast<vtkObject*>(clientSideObjectBase);
        if (clientSideObject != nullptr)
        {
            std::string className = clientSideObject->GetClassName();

            std::string filename = className + std::to_string(sourceIndex) + ".csv";
            sourceIndex++;
            logFile << "FileName: " << filename << std::endl;

            vtkSMSessionProxyManager* proxyManager = activeSession->GetSessionProxyManager();
            vtkSmartPointer<vtkSMProxy> writerProxy = proxyManager->NewProxy("writers", "CSVWriter");

            vtkSMSourceProxy* writer = vtkSMSourceProxy::SafeDownCast(writerProxy);
            vtkSMPropertyHelper(writer, "Input").Set(sourceProxy);
            vtkSMPropertyHelper(writer, "FileName").Set(filename.c_str());

            writer->UpdateVTKObjects();
            writer->UpdatePipeline();

            writerProxy->Delete();
        }
    }
}

You could use “AppendDataSet” or “GroupDataSet” first.

Where exactly would I do that? Before iterating over the sources/filters?

And will it be able to group data with number of columns? I mean if filter A has data columns a,b,c and filler B has a,b,e,f would they both be merged?

Just try it in ParaView.

I tried doing that, and after applying the AppendDataSet filter I saved the file as CSV but that CSV has only 3 columns Points:0, Points:1, Points:2 I have other arrays in my pipeline, e.g. vectors created by calculator which are not there in the CVS.