It looks like you are using GridMap. There is a bug with that using Bullet Physics. You can switch the 3D physics engine to Godot Physics in the project settings, I think that fixes it. However, it seems like everything is moving to Godot Physics for Godot 4.0, so I'm not sure the bug will ever be fixed.