ParaView 5.9 can't load python 3.8 libraries on MacOS Catalina

That’s possible to load the library from Python in Terminal and I added the key “com.apple.security.cs.disable-library-validation” to the plist but without success:

    import geopandas as gpd
  File "/usr/local/lib/python3.8/site-packages/geopandas/__init__.py", line 3, in <module>
    from geopandas.geoseries import GeoSeries  # noqa
  File "/usr/local/lib/python3.8/site-packages/geopandas/geoseries.py", line 9, in <module>
    from pyproj import CRS, Transformer
  File "/usr/local/lib/python3.8/site-packages/pyproj/__init__.py", line 49, in <module>
    import pyproj.network
  File "/usr/local/lib/python3.8/site-packages/pyproj/network.py", line 10, in <module>
    from pyproj._network import (  # noqa: F401
ImportError: dlopen(/usr/local/lib/python3.8/site-packages/pyproj/_network.cpython-38-darwin.so, 2): no suitable image found.  Did find:
	/usr/local/lib/python3.8/site-packages/pyproj/_network.cpython-38-darwin.so: code signature in (/usr/local/lib/python3.8/site-packages/pyproj/_network.cpython-38-darwin.so) not valid for use in process using Library Validation: mapped file has no Team ID and is not a platform binary (signed with custom identity or adhoc?)

The library file can be signed by trusted self-signed code-signing certificate without any changes (the Python library still doesn’t work in ParaView):

codesign -vvvv /usr/local/lib/python3.8/site-packages/pyproj/_network.cpython-38-darwin.so
/usr/local/lib/python3.8/site-packages/pyproj/_network.cpython-38-darwin.so: valid on disk
/usr/local/lib/python3.8/site-packages/pyproj/_network.cpython-38-darwin.so: satisfies its Designated Requirement

How did you build/obtain ParaView? The official packages or a custom build?

@ben.boeckel The original build is used with this launch script:

#!/bin/sh
PYTHONPATH=/usr/local/lib/python3.8/site-packages
GDAL_DATA=/usr/local/lib/python3.8/site-packages/rasterio/gdal_data
APPPATH=/Applications/ParaView-5.9.0-RC3.app

PYTHONPATH="$PYTHONPATH" GDAL_DATA="$GDAL_DATA" "$APPPATH/Contents/MacOS/paraview"

It would seem that some Apple library got into your /usr/local (Homebrew?) software somehow. I really don’t know how Library Validation works or how we could fix it beyond signing and noterizing ParaView itself (which I believe we do? @cory.quammen). Not signing ParaView isn’t really a practical option though…

We do sign the ParaView binaries. I have no idea whether what you are trying to do (pull in a external Python package) is possible, but I strongly suspect it is not. It’s challenging enough to get the known executables and libraries signed and working in the ParaView Mac binaries in the first place.

@ben.boeckel @cory.quammen ParaView 5.8 doesn’t work now:
“ParaView-5.8.1-1379.app” cannot be opened because the developer cannot be verified.
ParaView 5.7 with python 2.7 and the same Python modules works fine. Looks as your recent macOS builds are broken…

This ParaView plugin is used:

I wrote it for ParaView 5.7 with Python 2.7 and it’s works fine. It WAS ok for ParaView 5.8 some time ago. Also, it was usable for ParaView 5.9 on MacOS Mojave but it doesn’t work on MacOS Catalina. And, probably, the situation is the worst on recent MacOS BigSur.

ParaView-5.8.1-1379.app is a nightly build, and we don’t sign nightlies.

You can work around the message “cannot be opened because the developer cannot be verified” by allowing the executable to run by going to System Preferences -> Security & Privacy and under “Allow apps downloaded from:” click the radio button that says “App Store and identified developers.” When you run ParaView-5.8.1-1379.app again, it might deny you, so you’ll need to explicitly allow that to run.

Keep in mind that starting with ParaView 5.9, we are including Python 3.8 in the ParaView binaries on macos. Previously, we relied on the system Python installed on macos.

@cory.quammen Hmm, I already have the option enabled to launch applications from identified developers. But on MacOS Catalina I can’t run this one.
Thanks, I remember about this great change… it was very hard to maintain ParaView plugins with multiple dependencies for obsolete Python 2.7. Actually, I don’t understand why Homebrew python 2.7 libraries work with old ParaView but Homebrew python 3.8 libraries produce system validation error even signed by trust self-signed certificate.

Some updates? We still not able to use Python extensions in ParaView…

@cory.quammen I think I found the issue. You’ve enabled hardened runtime validation for the ParaView build and now we can’t use 3rd party libraries in it. See this example of valid 3rd party library:

codesign -v -v /usr/local/lib/python3.8/site-packages/pyproj/_network.cpython-38-darwin.so
/usr/local/lib/python3.8/site-packages/pyproj/_network.cpython-38-darwin.so: valid on disk
/usr/local/lib/python3.8/site-packages/pyproj/_network.cpython-38-darwin.so: satisfies its Designated Requirement

otool -l /usr/local/lib/python3.8/site-packages/pyproj/_network.cpython-38-darwin.so | grep -B 1 -A 3 LC_VERSION_MIN

It’s self-signed but it can’t be loaded probably because LC_VERSION_MIN missed (non Apple compiler used). Obviously, we can’t be limited by software compiled by Apple compiler only! How about multithreaded (OpenMP) Fortran and C? Lots of science libraries requires GCC + OpenMP.

We do this to make it easier for most Mac users to install ParaView without having to explicitly override security warnings the operating system raises. We have to no plans to stop doing this.

However, we could post the unsigned/unhardened packages on the downloads page. Would that work for your use case?

@cory.quammen Thanks, I think the additional build and brief explanation on the download page would be great for ParaView Python extension developers and users.

Okay, the unsigned .dmg of 5.9.0 for macOS has been posted on www.paraview.org/download. I hope that works for your use case.

@cory.quammen Hmm, I can’t see the new one:

Ctrl+F5 should do the trick.

Sure, I tried it :slight_smile: I’ve downloaded the build by other browser (Google Chrome instead of Safari) and it’s works with external Python 3.8 modules. Thanks!