Fixing the Apple OpenGL driver crash on glBufferSubData uploads larger than 2GB in VTK

TL;DR

Fix: split the upload in vtkOpenGLBufferObject::UploadRangeInternal into chunks (1GB is a safe chunk size). All VBO and IBO uploads from the polydata mapper go through this function, so one change covers both vertex and index buffers. Diagnosis: Apple's OpenGL-on-Metal driver crashes when a single glBufferSubData call transfers more than 2^31 bytes.

The error

Fixing the Apple OpenGL driver crash on glBufferSubData uploads larger than 2GB in VTK

Diagnosis: Apple's OpenGL-on-Metal driver crashes when a single glBufferSubData call transfers more than 2^31 bytes. The driver reports no GL error; it just crashes in memmove. This is a driver bug, not a hardware limit: the same buffer uploads fine in chunks and reads back exactly.

Repro notes worth keeping: glBufferData of the full size succeeds and GLBUFFERSIZE reports the full size, so allocation is fine. The failure is specific to the sub-data upload path above 2^31 bytes, confirmed with a standalone CGL program with no VTK involved.

Fix: split the upload in vtkOpenGLBufferObject::UploadRangeInternal into chunks (1GB is a safe chunk size). All VBO and IBO uploads from the polydata mapper go through this function, so one change covers both vertex and index buffers. For buffers under the chunk size the behavior is a single call, identical to before, which makes it safe on every platform. In code terms: loop over the range, and per chunk call glBufferSubData with the chunk offset, the chunk size, and the data pointer advanced by the chunk offset.

Where to submit: the Slicer VTK fork first, since that is the fastest path into Slicer builds, then upstream VTK as a merge request so every VTK-based application benefits. The patch is small, platform-safe, and fixes a hard crash with no error reporting, which is exactly the kind of change upstream takes.

Context: Thread: Crash rendering very large meshes on macOS: Apple's OpenGL driver fails on glBufferSubData uploads over 2 GB. On Apple Silicon (M5 Max, 128GB RAM, macOS 26.5), Slicer crashes with EXCBADACCESS in platformmemmove inside AppleMetalOpenGLRenderer when rendering a mesh whose vertex or index buffer exceeds 2GB. A 234M-triangle segmentation builds a 2.8GB index buffer uploaded in a single glBufferSubData call from vtkOpenGLBufferObject::UploadRangeInternal. Isolation: glBufferData of the full size succeeds and GLBUFFERSIZE reports the full size; a single glBufferSubData of exactly 2^31 bytes works but anything larger crashes; chunked uploads of 1GB or less work and read back correctly via glMapBufferRange. Not a hardware limit: MTLDevice.maxBufferLength is about 86GB, and the driver reports no GL error, it just crashes. The reporter proposes chunking the upload in vtkOpenGLBufferObject::UploadRangeInternal and asks whether to submit to upstream VTK, the Slicer VTK fork, or both.

When to use

• You hit this exact issue.
• The symptom matches: Fixing the Apple OpenGL driver crash on glBufferSubData uploads larger than 2GB in VTK.

When NOT to use

• A different error or symptom; the cause here is specific to this one.

Compatibility

Per the linked source (mentions 26.5).

Edge cases

• If your error message differs even slightly, this is probably a different issue; search the exact text.
• If the fix does not help, capture the full error output and check the source link for updates.

Thread: Crash rendering very large meshes on macOS: Apple's OpenGL driver fails on glBufferSubData uploads over 2 GB. On Apple Silicon (M5 Max, 128GB RAM, macOS 26.5), Slicer crashes with EXCBADACCESS in platformmemmove inside AppleMetalOpenGLRenderer when rendering a mesh whose vertex or index buffer exceeds 2GB. A 234M-triangle segmentation builds a 2.8GB index buffer uploaded in a single glBufferSubData call from vtkOpenGLBufferObject::UploadRangeInternal. Isolation: glBufferData of the full size succeeds and GLBUFFERSIZE reports the full size; a single glBufferSubData of exactly 2^31 bytes works but anything larger crashes; chunked uploads of 1GB or less work and read back correctly via glMapBufferRange. Not a hardware limit: MTLDevice.maxBufferLength is about 86GB, and the driver reports no GL error, it just crashes. The reporter proposes chunking the upload in vtkOpenGLBufferObject::UploadRangeInternal and asks whether to submit to upstream VTK, the Slicer VTK fork, or both. Resolved: Diagnosis: Apple's OpenGL-on-Metal driver crashes when a single glBufferSubData call transfers more than 2^31 bytes. The driver reports no GL error; it just crashes in memmove. This is a driver bug, not a hardware limit: the same buffer uploads fine in chunks and reads back exactly.

Repro notes worth keeping: glBufferData of the full size succeeds and GLBUFFERSIZE reports the full size, so allocation is fine. The failure is specific to the sub-data upload path above 2^31 bytes, confirmed with a standalone CGL program with no VTK involved.

Fix: split the upload in vtkOpenGLBufferObject::UploadRangeInternal into chunks (1GB is a safe chunk size). All VBO and IBO uploads from the polydata mapper go through this function, so one change covers both vertex and index buffers. For buffers under the chunk size the behavior is a single call, identical to before, which makes it safe on every platform. In code terms: loop over the range, and per chunk call glBufferSubData with the chunk offset, the chunk size, and the data pointer advanced by the chunk offset.

Where to submit: the Slicer VTK fork first, since that is the fastest path into Slicer builds, then upstream VTK as a merge request so every VTK-based application benefits. The patch is small, platform-safe, and fixes a hard crash with no error reporting, which is exactly the kind of change upstream takes.