Running a pvpython server in Google Kubernetes engine environment

I’m trying to run a pvpython server in a GKE container with NVidia Tesla T4, and having trouble getting everything to work. The server crashes with a message

(1426.516s) [paraview        ]   vtkShaderProgram.cxx:438    ERR| vtkShaderProgram (0x6f9e9f0): 

Loguru caught a signal: SIGSEGV
Stack trace:
...

4       0x7fae93ab279e vtkCompositeMapperHelper2::RenderPiece(vtkRenderer*, vtkActor*) + 94
3       0x7fae93ab2d60 vtkCompositeMapperHelper2::RenderPieceDraw(vtkRenderer*, vtkActor*) + 368
2       0x7fae93ab282f vtkCompositeMapperHelper2::DrawIBO(vtkRenderer*, vtkActor*, int, vtkOpenGLHelper&, unsigned int, int) + 111
1       0x7fae93ab1ef1 vtkCompositeMapperHelper2::UpdateShaders(vtkOpenGLHelper&, vtkRenderer*, vtkActor*) + 97
0       0x7fae93b25e49 vtkOpenGLPolyDataMapper::UpdateShaders(vtkOpenGLHelper&, vtkRenderer*, vtkActor*) + 1081
(1426.516s) [paraview        ]                       :0     FATL| Signal: SIGSEGV

It looks as if Paraview cannot find the GPU, then falling back on a CPU-based rendering, but it’s also crippled and crashes eventually. I can’t figure out what’s going on. Can anyone give me hints for debugging the problem?

  • I compiled paraview 5.8.1 in EGL mode:
-DVTK_USE_X=OFF
-DPARAVIEW_USE_QT=OFF
-DVTK_OPENGL_HAS_OSMESA=OFF
-DVTK_OPENGL_HAS_EGL=ON
  • The pvpython server is running a paraviewweb protocol to talk to a desktop client.

  • I ran the server on an Ubuntu 20.04 + Tesla T4 running in GCE (a raw VM, without Docker or Kubernetes), and it runs fine. I confirmed that it’s rendering using the GPU.

  • When I run the same server on an virtually identical Ubuntu 20.04 + T4 running in GKE, Paraview doesn’t seem to find the GPU and proceeds with messages like below:

( 462.382s) [paraview        ]   vtkShaderProgram.cxx:452    ERR| vtkShaderProgram (0x429e4f0): 1: #version 140
2: #ifdef GL_ES
3: #ifdef GL_FRAGMENT_PRECISION_HIGH
4: precision highp float;
5: precision highp sampler2D;
6: precision highp sampler3D;
7: #else
8: precision mediump float;
9: precision mediump sampler2D;
10: precision mediump sampler3D;
11: #endif
12: #define texelFetchBuffer texelFetch
13: #define texture1D texture
14: #define texture2D texture
15: #define texture3D texture
16: #else // GL_ES
17: #define highp
18: #define mediump
19: #define lowp
20: #if __VERSION__ == 150
21: #define texelFetchBuffer texelFetch
22: #define texture1D texture
23: #define texture2D texture
24: #define texture3D texture
25: #endif
26: #endif // GL_ES
27: #define varying in
28: 
29: out vec4 fragOutput0;
30: 
31: void main(void) {
32:   fragOutput0 = vec4(float(gl_PrimitiveID)/100.0,1.0,1.0,1.0);
33: }

( 462.382s) [paraview        ]   vtkShaderProgram.cxx:453    ERR| vtkShaderProgram (0x429e4f0): 0:32(27): error: `gl_PrimitiveID' undeclared
0:32(21): error: cannot construct `float' from a non-numeric data type
0:32(21): error: operands to arithmetic operators must be numeric
0:32(16): error: cannot construct `vec4' from a non-numeric data type

I see a several similar messages, but the server eventually crashes (see the above).

Tue Aug 18 18:25:27 2020       
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 418.67       Driver Version: 418.67       CUDA Version: 10.1     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|===============================+======================+======================|
|   0  Tesla T4            Off  | 00000000:00:04.0 Off |                    0 |
| N/A   40C    P8    10W /  70W |      0MiB / 15079MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+
                                                                               
+-----------------------------------------------------------------------------+
| Processes:                                                       GPU Memory |
|  GPU       PID   Type   Process name                             Usage      |
|=============================================================================|
|  No running processes found                                                 |
+-----------------------------------------------------------------------------+
  • I also ran a couple of simple CUDA applications (no OpenGL though) and they ran fine inside the same container that runs the pvserver.

  • Between a GKE pod and a raw VM, the pvpython is loading basically the same set of shared object files from the same locations. I do notice that on GKE, pvpython is loading LLVM-related libraries, but I guess it’s somehow falling back on CPU-based rendering.

I can give more details as necessary.

Hum, it is hard to say what may be going wrong in your setup.
I don’t necessary have much experience with GKE but I know other have managed to get it working under Kubernetes.

Is your image working in docker with the nvidia-runtime? (Just making sure you have all the pieces required)

You mean the nvidia device plugin (https://github.com/NVIDIA/k8s-device-plugin)?

No, we are using GCE plugin, which comes bundled in GKE nodes.


(see section "NVIDIA GPU device plugin used by GCE)

Is there a way have paraview print how its choosing the backend renderer? I tried “-v=9” and it didn’t show anything interesting.

What is the output of the following script when running it via pvpython?

from paraview.simple import *
print(GetOpenGLInformation().GetCapabilities())

Ok, on GKE Docker, I get:

>>> from paraview.simple import *
from paraview.simple import *
>>> print(GetOpenGLInformation().GetCapabilities())
print(GetOpenGLInformation().GetCapabilities())
OpenGL vendor string:  VMware, Inc.
OpenGL renderer string:  llvmpipe (LLVM 10.0.0, 256 bits)
OpenGL version string:  3.1 Mesa 20.0.8
OpenGL extensions:
  GL_AMD_conservative_depth
  GL_AMD_draw_buffers_blend
  GL_AMD_multi_draw_indirect
  GL_AMD_query_buffer_object
  GL_AMD_seamless_cubemap_per_texture
  GL_AMD_shader_stencil_export
  GL_AMD_shader_trinary_minmax
  GL_AMD_texture_texture4
  GL_ANGLE_texture_compression_dxt3
  GL_ANGLE_texture_compression_dxt5
  GL_APPLE_packed_pixels
  GL_ARB_ES2_compatibility
  GL_ARB_ES3_compatibility
  GL_ARB_arrays_of_arrays
  GL_ARB_base_instance
  GL_ARB_blend_func_extended
  GL_ARB_buffer_storage
  GL_ARB_clear_buffer_object
  GL_ARB_clear_texture
  GL_ARB_clip_control
  GL_ARB_color_buffer_float
  GL_ARB_compatibility
  GL_ARB_compressed_texture_pixel_storage
  GL_ARB_compute_shader
  GL_ARB_conditional_render_inverted
  GL_ARB_conservative_depth
  GL_ARB_copy_buffer
  GL_ARB_copy_image
  GL_ARB_cull_distance
  GL_ARB_debug_output
  GL_ARB_depth_buffer_float
  GL_ARB_depth_clamp
  GL_ARB_depth_texture
  GL_ARB_derivative_control
  GL_ARB_direct_state_access
  GL_ARB_draw_buffers
  GL_ARB_draw_buffers_blend
  GL_ARB_draw_elements_base_vertex
  GL_ARB_draw_indirect
  GL_ARB_draw_instanced
  GL_ARB_enhanced_layouts
  GL_ARB_explicit_attrib_location
  GL_ARB_explicit_uniform_location
  GL_ARB_fragment_coord_conventions
  GL_ARB_fragment_layer_viewport
  GL_ARB_fragment_program
  GL_ARB_fragment_program_shadow
  GL_ARB_fragment_shader
  GL_ARB_framebuffer_no_attachments
  GL_ARB_framebuffer_object
  GL_ARB_framebuffer_sRGB
  GL_ARB_get_program_binary
  GL_ARB_get_texture_sub_image
  GL_ARB_half_float_pixel
  GL_ARB_half_float_vertex
  GL_ARB_indirect_parameters
  GL_ARB_instanced_arrays
  GL_ARB_internalformat_query
  GL_ARB_internalformat_query2
  GL_ARB_invalidate_subdata
  GL_ARB_map_buffer_alignment
  GL_ARB_map_buffer_range
  GL_ARB_multi_bind
  GL_ARB_multi_draw_indirect
  GL_ARB_multisample
  GL_ARB_multitexture
  GL_ARB_occlusion_query
  GL_ARB_occlusion_query2
  GL_ARB_parallel_shader_compile
  GL_ARB_pipeline_statistics_query
  GL_ARB_pixel_buffer_object
  GL_ARB_point_parameters
  GL_ARB_point_sprite
  GL_ARB_polygon_offset_clamp
  GL_ARB_program_interface_query
  GL_ARB_provoking_vertex
  GL_ARB_query_buffer_object
  GL_ARB_robustness
  GL_ARB_sampler_objects
  GL_ARB_seamless_cube_map
  GL_ARB_seamless_cubemap_per_texture
  GL_ARB_separate_shader_objects
  GL_ARB_shader_atomic_counter_ops
  GL_ARB_shader_atomic_counters
  GL_ARB_shader_bit_encoding
  GL_ARB_shader_draw_parameters
  GL_ARB_shader_group_vote
  GL_ARB_shader_image_load_store
  GL_ARB_shader_image_size
  GL_ARB_shader_objects
  GL_ARB_shader_stencil_export
  GL_ARB_shader_storage_buffer_object
  GL_ARB_shader_subroutine
  GL_ARB_shader_texture_lod
  GL_ARB_shading_language_100
  GL_ARB_shading_language_420pack
  GL_ARB_shading_language_include
  GL_ARB_shading_language_packing
  GL_ARB_shadow
  GL_ARB_stencil_texturing
  GL_ARB_sync
  GL_ARB_texture_border_clamp
  GL_ARB_texture_buffer_object
  GL_ARB_texture_buffer_object_rgb32
  GL_ARB_texture_buffer_range
  GL_ARB_texture_compression
  GL_ARB_texture_compression_bptc
  GL_ARB_texture_compression_rgtc
  GL_ARB_texture_cube_map
  GL_ARB_texture_cube_map_array
  GL_ARB_texture_env_add
  GL_ARB_texture_env_combine
  GL_ARB_texture_env_crossbar
  GL_ARB_texture_env_dot3
  GL_ARB_texture_float
  GL_ARB_texture_gather
  GL_ARB_texture_mirror_clamp_to_edge
  GL_ARB_texture_mirrored_repeat
  GL_ARB_texture_multisample
  GL_ARB_texture_non_power_of_two
  GL_ARB_texture_query_levels
  GL_ARB_texture_query_lod
  GL_ARB_texture_rectangle
  GL_ARB_texture_rg
  GL_ARB_texture_rgb10_a2ui
  GL_ARB_texture_stencil8
  GL_ARB_texture_storage
  GL_ARB_texture_storage_multisample
  GL_ARB_texture_swizzle
  GL_ARB_texture_view
  GL_ARB_timer_query
  GL_ARB_transform_feedback2
  GL_ARB_transform_feedback3
  GL_ARB_transform_feedback_instanced
  GL_ARB_transform_feedback_overflow_query
  GL_ARB_transpose_matrix
  GL_ARB_uniform_buffer_object
  GL_ARB_vertex_array_bgra
  GL_ARB_vertex_array_object
  GL_ARB_vertex_attrib_binding
  GL_ARB_vertex_buffer_object
  GL_ARB_vertex_program
  GL_ARB_vertex_shader
  GL_ARB_vertex_type_10f_11f_11f_rev
  GL_ARB_vertex_type_2_10_10_10_rev
  GL_ARB_viewport_array
  GL_ARB_window_pos
  GL_ATI_blend_equation_separate
  GL_ATI_draw_buffers
  GL_ATI_fragment_shader
  GL_ATI_separate_stencil
  GL_ATI_texture_compression_3dc
  GL_ATI_texture_env_combine3
  GL_ATI_texture_float
  GL_ATI_texture_mirror_once
  GL_EXT_EGL_image_storage
  GL_EXT_EGL_sync
  GL_EXT_abgr
  GL_EXT_bgra
  GL_EXT_blend_color
  GL_EXT_blend_equation_separate
  GL_EXT_blend_func_separate
  GL_EXT_blend_minmax
  GL_EXT_blend_subtract
  GL_EXT_compiled_vertex_array
  GL_EXT_copy_texture
  GL_EXT_direct_state_access
  GL_EXT_draw_buffers2
  GL_EXT_draw_instanced
  GL_EXT_draw_range_elements
  GL_EXT_fog_coord
  GL_EXT_framebuffer_blit
  GL_EXT_framebuffer_multisample
  GL_EXT_framebuffer_multisample_blit_scaled
  GL_EXT_framebuffer_object
  GL_EXT_framebuffer_sRGB
  GL_EXT_gpu_program_parameters
  GL_EXT_gpu_shader4
  GL_EXT_multi_draw_arrays
  GL_EXT_packed_depth_stencil
  GL_EXT_packed_float
  GL_EXT_packed_pixels
  GL_EXT_pixel_buffer_object
  GL_EXT_point_parameters
  GL_EXT_polygon_offset_clamp
  GL_EXT_provoking_vertex
  GL_EXT_rescale_normal
  GL_EXT_secondary_color
  GL_EXT_separate_specular_color
  GL_EXT_shader_integer_mix
  GL_EXT_shadow_funcs
  GL_EXT_stencil_two_side
  GL_EXT_stencil_wrap
  GL_EXT_subtexture
  GL_EXT_texture
  GL_EXT_texture3D
  GL_EXT_texture_array
  GL_EXT_texture_buffer_object
  GL_EXT_texture_compression_dxt1
  GL_EXT_texture_compression_latc
  GL_EXT_texture_compression_rgtc
  GL_EXT_texture_compression_s3tc
  GL_EXT_texture_cube_map
  GL_EXT_texture_edge_clamp
  GL_EXT_texture_env_add
  GL_EXT_texture_env_combine
  GL_EXT_texture_env_dot3
  GL_EXT_texture_integer
  GL_EXT_texture_lod_bias
  GL_EXT_texture_mirror_clamp
  GL_EXT_texture_object
  GL_EXT_texture_rectangle
  GL_EXT_texture_sRGB
  GL_EXT_texture_sRGB_R8
  GL_EXT_texture_sRGB_decode
  GL_EXT_texture_shared_exponent
  GL_EXT_texture_snorm
  GL_EXT_texture_swizzle
  GL_EXT_timer_query
  GL_EXT_transform_feedback
  GL_EXT_vertex_array
  GL_EXT_vertex_array_bgra
  GL_IBM_multimode_draw_arrays
  GL_IBM_rasterpos_clip
  GL_IBM_texture_mirrored_repeat
  GL_INGR_blend_func_separate
  GL_KHR_context_flush_control
  GL_KHR_debug
  GL_KHR_no_error
  GL_KHR_parallel_shader_compile
  GL_KHR_texture_compression_astc_ldr
  GL_KHR_texture_compression_astc_sliced_3d
  GL_MESA_pack_invert
  GL_MESA_shader_integer_functions
  GL_MESA_texture_signed_rgba
  GL_MESA_window_pos
  GL_MESA_ycbcr_texture
  GL_NV_blend_square
  GL_NV_conditional_render
  GL_NV_depth_clamp
  GL_NV_fog_distance
  GL_NV_light_max_exponent
  GL_NV_packed_depth_stencil
  GL_NV_primitive_restart
  GL_NV_texgen_reflection
  GL_NV_texture_env_combine4
  GL_NV_texture_rectangle
  GL_OES_EGL_image
  GL_OES_read_format
  GL_S3_s3tc
  GL_SGIS_generate_mipmap
  GL_SGIS_texture_border_clamp
  GL_SGIS_texture_edge_clamp
  GL_SGIS_texture_lod
  GL_SUN_multi_draw_arrays

Running the same binary on a native VM gets:

>>> from paraview.simple import *
>>> print(GetOpenGLInformation().GetCapabilities())

OpenGL vendor string:  NVIDIA Corporation
OpenGL renderer string:  Tesla T4/PCIe/SSE2
OpenGL version string:  4.6.0 NVIDIA 440.100
OpenGL extensions:
  GL_AMD_multi_draw_indirect
  GL_AMD_seamless_cubemap_per_texture
  GL_AMD_vertex_shader_viewport_index
  GL_AMD_vertex_shader_layer
  GL_ARB_arrays_of_arrays
  GL_ARB_base_instance
  GL_ARB_bindless_texture
  GL_ARB_blend_func_extended
  GL_ARB_buffer_storage
  GL_ARB_clear_buffer_object
  GL_ARB_clear_texture
  GL_ARB_clip_control
  GL_ARB_color_buffer_float
  GL_ARB_compatibility
  GL_ARB_compressed_texture_pixel_storage
  GL_ARB_conservative_depth
  GL_ARB_compute_shader
  GL_ARB_compute_variable_group_size
  GL_ARB_conditional_render_inverted
  GL_ARB_copy_buffer
  GL_ARB_copy_image
  GL_ARB_cull_distance
  GL_ARB_debug_output
  GL_ARB_depth_buffer_float
  GL_ARB_depth_clamp
  GL_ARB_depth_texture
  GL_ARB_derivative_control
  GL_ARB_direct_state_access
  GL_ARB_draw_buffers
  GL_ARB_draw_buffers_blend
  GL_ARB_draw_indirect
  GL_ARB_draw_elements_base_vertex
  GL_ARB_draw_instanced
  GL_ARB_enhanced_layouts
  GL_ARB_ES2_compatibility
  GL_ARB_ES3_compatibility
  GL_ARB_ES3_1_compatibility
  GL_ARB_ES3_2_compatibility
  GL_ARB_explicit_attrib_location
  GL_ARB_explicit_uniform_location
  GL_ARB_fragment_coord_conventions
  GL_ARB_fragment_layer_viewport
  GL_ARB_fragment_program
  GL_ARB_fragment_program_shadow
  GL_ARB_fragment_shader
  GL_ARB_fragment_shader_interlock
  GL_ARB_framebuffer_no_attachments
  GL_ARB_framebuffer_object
  GL_ARB_framebuffer_sRGB
  GL_ARB_geometry_shader4
  GL_ARB_get_program_binary
  GL_ARB_get_texture_sub_image
  GL_ARB_gl_spirv
  GL_ARB_gpu_shader5
  GL_ARB_gpu_shader_fp64
  GL_ARB_gpu_shader_int64
  GL_ARB_half_float_pixel
  GL_ARB_half_float_vertex
  GL_ARB_imaging
  GL_ARB_indirect_parameters
  GL_ARB_instanced_arrays
  GL_ARB_internalformat_query
  GL_ARB_internalformat_query2
  GL_ARB_invalidate_subdata
  GL_ARB_map_buffer_alignment
  GL_ARB_map_buffer_range
  GL_ARB_multi_bind
  GL_ARB_multi_draw_indirect
  GL_ARB_multisample
  GL_ARB_multitexture
  GL_ARB_occlusion_query
  GL_ARB_occlusion_query2
  GL_ARB_parallel_shader_compile
  GL_ARB_pipeline_statistics_query
  GL_ARB_pixel_buffer_object
  GL_ARB_point_parameters
  GL_ARB_point_sprite
  GL_ARB_polygon_offset_clamp
  GL_ARB_post_depth_coverage
  GL_ARB_program_interface_query
  GL_ARB_provoking_vertex
  GL_ARB_query_buffer_object
  GL_ARB_robust_buffer_access_behavior
  GL_ARB_robustness
  GL_ARB_sample_locations
  GL_ARB_sample_shading
  GL_ARB_sampler_objects
  GL_ARB_seamless_cube_map
  GL_ARB_seamless_cubemap_per_texture
  GL_ARB_separate_shader_objects
  GL_ARB_shader_atomic_counter_ops
  GL_ARB_shader_atomic_counters
  GL_ARB_shader_ballot
  GL_ARB_shader_bit_encoding
  GL_ARB_shader_clock
  GL_ARB_shader_draw_parameters
  GL_ARB_shader_group_vote
  GL_ARB_shader_image_load_store
  GL_ARB_shader_image_size
  GL_ARB_shader_objects
  GL_ARB_shader_precision
  GL_ARB_shader_storage_buffer_object
  GL_ARB_shader_subroutine
  GL_ARB_shader_texture_image_samples
  GL_ARB_shader_texture_lod
  GL_ARB_shading_language_100
  GL_ARB_shader_viewport_layer_array
  GL_ARB_shading_language_420pack
  GL_ARB_shading_language_include
  GL_ARB_shading_language_packing
  GL_ARB_shadow
  GL_ARB_sparse_buffer
  GL_ARB_sparse_texture
  GL_ARB_sparse_texture2
  GL_ARB_sparse_texture_clamp
  GL_ARB_spirv_extensions
  GL_ARB_stencil_texturing
  GL_ARB_sync
  GL_ARB_tessellation_shader
  GL_ARB_texture_barrier
  GL_ARB_texture_border_clamp
  GL_ARB_texture_buffer_object
  GL_ARB_texture_buffer_object_rgb32
  GL_ARB_texture_buffer_range
  GL_ARB_texture_compression
  GL_ARB_texture_compression_bptc
  GL_ARB_texture_compression_rgtc
  GL_ARB_texture_cube_map
  GL_ARB_texture_cube_map_array
  GL_ARB_texture_env_add
  GL_ARB_texture_env_combine
  GL_ARB_texture_env_crossbar
  GL_ARB_texture_env_dot3
  GL_ARB_texture_filter_anisotropic
  GL_ARB_texture_filter_minmax
  GL_ARB_texture_float
  GL_ARB_texture_gather
  GL_ARB_texture_mirror_clamp_to_edge
  GL_ARB_texture_mirrored_repeat
  GL_ARB_texture_multisample
  GL_ARB_texture_non_power_of_two
  GL_ARB_texture_query_levels
  GL_ARB_texture_query_lod
  GL_ARB_texture_rectangle
  GL_ARB_texture_rg
  GL_ARB_texture_rgb10_a2ui
  GL_ARB_texture_stencil8
  GL_ARB_texture_storage
  GL_ARB_texture_storage_multisample
  GL_ARB_texture_swizzle
  GL_ARB_texture_view
  GL_ARB_timer_query
  GL_ARB_transform_feedback2
  GL_ARB_transform_feedback3
  GL_ARB_transform_feedback_instanced
  GL_ARB_transform_feedback_overflow_query
  GL_ARB_transpose_matrix
  GL_ARB_uniform_buffer_object
  GL_ARB_vertex_array_bgra
  GL_ARB_vertex_array_object
  GL_ARB_vertex_attrib_64bit
  GL_ARB_vertex_attrib_binding
  GL_ARB_vertex_buffer_object
  GL_ARB_vertex_program
  GL_ARB_vertex_shader
  GL_ARB_vertex_type_10f_11f_11f_rev
  GL_ARB_vertex_type_2_10_10_10_rev
  GL_ARB_viewport_array
  GL_ARB_window_pos
  GL_ATI_draw_buffers
  GL_ATI_texture_float
  GL_ATI_texture_mirror_once
  GL_S3_s3tc
  GL_EXT_texture_env_add
  GL_EXT_abgr
  GL_EXT_bgra
  GL_EXT_bindable_uniform
  GL_EXT_blend_color
  GL_EXT_blend_equation_separate
  GL_EXT_blend_func_separate
  GL_EXT_blend_minmax
  GL_EXT_blend_subtract
  GL_EXT_compiled_vertex_array
  GL_EXT_Cg_shader
  GL_EXT_depth_bounds_test
  GL_EXT_direct_state_access
  GL_EXT_draw_buffers2
  GL_EXT_draw_instanced
  GL_EXT_draw_range_elements
  GL_EXT_EGL_image_storage
  GL_EXT_fog_coord
  GL_EXT_framebuffer_blit
  GL_EXT_framebuffer_multisample
  GL_EXTX_framebuffer_mixed_formats
  GL_EXT_framebuffer_multisample_blit_scaled
  GL_EXT_framebuffer_object
  GL_EXT_framebuffer_sRGB
  GL_EXT_geometry_shader4
  GL_EXT_gpu_program_parameters
  GL_EXT_gpu_shader4
  GL_EXT_multi_draw_arrays
  GL_EXT_multiview_texture_multisample
  GL_EXT_multiview_timer_query
  GL_EXT_packed_depth_stencil
  GL_EXT_packed_float
  GL_EXT_packed_pixels
  GL_EXT_pixel_buffer_object
  GL_EXT_point_parameters
  GL_EXT_polygon_offset_clamp
  GL_EXT_post_depth_coverage
  GL_EXT_provoking_vertex
  GL_EXT_raster_multisample
  GL_EXT_rescale_normal
  GL_EXT_secondary_color
  GL_EXT_separate_shader_objects
  GL_EXT_separate_specular_color
  GL_EXT_shader_image_load_formatted
  GL_EXT_shader_image_load_store
  GL_EXT_shader_integer_mix
  GL_EXT_shadow_funcs
  GL_EXT_sparse_texture2
  GL_EXT_stencil_two_side
  GL_EXT_stencil_wrap
  GL_EXT_texture3D
  GL_EXT_texture_array
  GL_EXT_texture_buffer_object
  GL_EXT_texture_compression_dxt1
  GL_EXT_texture_compression_latc
  GL_EXT_texture_compression_rgtc
  GL_EXT_texture_compression_s3tc
  GL_EXT_texture_cube_map
  GL_EXT_texture_edge_clamp
  GL_EXT_texture_env_combine
  GL_EXT_texture_env_dot3
  GL_EXT_texture_filter_anisotropic
  GL_EXT_texture_filter_minmax
  GL_EXT_texture_integer
  GL_EXT_texture_lod
  GL_EXT_texture_lod_bias
  GL_EXT_texture_mirror_clamp
  GL_EXT_texture_object
  GL_EXT_texture_shadow_lod
  GL_EXT_texture_shared_exponent
  GL_EXT_texture_sRGB
  GL_EXT_texture_sRGB_R8
  GL_EXT_texture_sRGB_decode
  GL_EXT_texture_storage
  GL_EXT_texture_swizzle
  GL_EXT_timer_query
  GL_EXT_transform_feedback2
  GL_EXT_vertex_array
  GL_EXT_vertex_array_bgra
  GL_EXT_vertex_attrib_64bit
  GL_EXT_window_rectangles
  GL_EXT_import_sync_object
  GL_NV_robustness_video_memory_purge
  GL_NVX_shared_sync_object
  GL_IBM_rasterpos_clip
  GL_IBM_texture_mirrored_repeat
  GL_KHR_context_flush_control
  GL_KHR_debug
  GL_EXT_memory_object
  GL_EXT_memory_object_fd
  GL_KHR_parallel_shader_compile
  GL_KHR_no_error
  GL_KHR_robust_buffer_access_behavior
  GL_KHR_robustness
  GL_EXT_semaphore
  GL_EXT_semaphore_fd
  GL_KHR_shader_subgroup
  GL_KTX_buffer_region
  GL_NV_alpha_to_coverage_dither_control
  GL_NV_bindless_multi_draw_indirect
  GL_NV_bindless_multi_draw_indirect_count
  GL_NV_bindless_texture
  GL_NV_blend_equation_advanced
  GL_NV_blend_equation_advanced_coherent
  GL_NVX_blend_equation_advanced_multi_draw_buffers
  GL_NV_blend_minmax_factor
  GL_NV_blend_square
  GL_NV_clip_space_w_scaling
  GL_NV_command_list
  GL_NV_compute_program5
  GL_NV_compute_shader_derivatives
  GL_NV_conditional_render
  GL_NV_conservative_raster
  GL_NV_conservative_raster_dilate
  GL_NV_conservative_raster_pre_snap
  GL_NV_conservative_raster_pre_snap_triangles
  GL_NV_conservative_raster_underestimation
  GL_NV_copy_depth_to_color
  GL_NV_copy_image
  GL_NV_depth_buffer_float
  GL_NV_depth_clamp
  GL_NV_draw_texture
  GL_NV_draw_vulkan_image
  GL_NV_ES1_1_compatibility
  GL_NV_ES3_1_compatibility
  GL_NV_explicit_multisample
  GL_NV_feature_query
  GL_NV_fence
  GL_NV_fill_rectangle
  GL_NV_float_buffer
  GL_NV_fog_distance
  GL_NV_fragment_coverage_to_color
  GL_NV_fragment_program
  GL_NV_fragment_program_option
  GL_NV_fragment_program2
  GL_NV_fragment_shader_barycentric
  GL_NV_fragment_shader_interlock
  GL_NV_framebuffer_mixed_samples
  GL_NV_framebuffer_multisample_coverage
  GL_NV_geometry_shader4
  GL_NV_geometry_shader_passthrough
  GL_NV_gpu_program4
  GL_NV_internalformat_sample_query
  GL_NV_gpu_program4_1
  GL_NV_gpu_program5
  GL_NV_gpu_program5_mem_extended
  GL_NV_gpu_program_fp64
  GL_NV_gpu_shader5
  GL_NV_half_float
  GL_NV_light_max_exponent
  GL_NV_memory_attachment
  GL_NV_mesh_shader
  GL_NV_multisample_coverage
  GL_NV_multisample_filter_hint
  GL_NV_occlusion_query
  GL_NV_packed_depth_stencil
  GL_NV_parameter_buffer_object
  GL_NV_parameter_buffer_object2
  GL_NV_path_rendering
  GL_NV_path_rendering_shared_edge
  GL_NV_point_sprite
  GL_NV_primitive_restart
  GL_NV_query_resource
  GL_NV_query_resource_tag
  GL_NV_register_combiners
  GL_NV_register_combiners2
  GL_NV_representative_fragment_test
  GL_NV_sample_locations
  GL_NV_sample_mask_override_coverage
  GL_NV_scissor_exclusive
  GL_NV_shader_atomic_counters
  GL_NV_shader_atomic_float
  GL_NV_shader_atomic_float64
  GL_NV_shader_atomic_fp16_vector
  GL_NV_shader_atomic_int64
  GL_NV_shader_buffer_load
  GL_NV_shader_storage_buffer_object
  GL_NV_shader_subgroup_partitioned
  GL_NV_shader_texture_footprint
  GL_NV_shading_rate_image
  GL_NV_stereo_view_rendering
  GL_NV_texgen_reflection
  GL_NV_texture_barrier
  GL_NV_texture_compression_vtc
  GL_NV_texture_env_combine4
  GL_NV_texture_multisample
  GL_NV_texture_rectangle
  GL_NV_texture_rectangle_compressed
  GL_NV_texture_shader
  GL_NV_texture_shader2
  GL_NV_texture_shader3
  GL_NV_transform_feedback
  GL_NV_transform_feedback2
  GL_NV_uniform_buffer_unified_memory
  GL_NV_vertex_attrib_integer_64bit
  GL_NV_vertex_buffer_unified_memory
  GL_NV_vertex_program
  GL_NV_vertex_program1_1
  GL_NV_vertex_program2
  GL_NV_vertex_program2_option
  GL_NV_vertex_program3
  GL_NV_viewport_array2
  GL_NV_viewport_swizzle
  GL_NVX_sysmem_buffer
  GL_NVX_conditional_render
  GL_NV_gpu_multicast
  GL_NVX_progress_fence
  GL_NVX_gpu_memory_info
  GL_NV_shader_thread_group
  GL_NV_shader_thread_shuffle
  GL_KHR_blend_equation_advanced
  GL_KHR_blend_equation_advanced_coherent
  GL_OVR_multiview
  GL_OVR_multiview2
  GL_SGIS_generate_mipmap
  GL_SGIS_texture_lod
  GL_SGIX_depth_texture
  GL_SGIX_shadow
  GL_SUN_slice_accum

>>> >>>

Is libnvidia-gl-xxx installed, where xxx is a driver version?

Thanks for the tip. I figured it out.

In GKE, the basic CUDA and OpenGL libraries are provided through Docker volume mounts in /usr/local/nvidia, not through apt. But the container forgets to mount /usr/share/glvnd/egl_vendor.d/10_nvidia.json. So the binary only finds the mesa.json file in the same directory and loads llvmpipe instead. So a workaround is to create file

/usr/share/glvnd/egl_vendor.d/10_nvidia.json

with contents

{
    "file_format_version" : "1.0.0",
    "ICD" : {
        "library_path" : "libEGL_nvidia.so.0"
    }
}

in the Dockerfile. I’ll follow the issue up with the Google cloud team.

1 Like