There are two solutions I've used in the past:
You can either use MeshDataTool to get information about a mesh, then give the triangles to some functions in the Geometry singleton to find out which one is hovered.
The second approach is more complicated, but more performant: You create a new mesh where every triangle has a different vertex color. Then you put it into a Viewport with the same transform as the real thing. If you sample the pixel where the cursor is, you get the triangle ID.
To actually render the highlight, you could dynamically create a new mesh or give the triangle position / ID to a shader and let it handle the rest.
Some code examples are here, but I think understanding is more future-proof than copy-pasting in this case. I have different selection types in the linked addon. There is also one for quads.
