I think I've finally cracked the height issue for good! However it does look like it will need some backend dev intervention in order to have some proper fixes to deal with this kind of problem unless somebody knows something that I don't.
If the cell height is too low, the calculations that the baking does will leave massive gaps in the mesh you've provided if the slopes are too steep on the mesh being baked to. However if you bake within the provided parameters that will give you the most accurate results and give you a proper navmesh your agent can walk on without too much tweaking.
I've managed to think of workarounds but they're a bit clunky and I think that if the devs removed some of the limits they have on the baking these workaround wouldn't be needed which is why I'm trying to kind of chase people up on this issue. You can increase the cell height to make the navmesh bake 'correctly' however this creates and extremely frustrating height offset which breaks the agent's movement if you place it on the ground as you'd expect because the collider is technically underneath the navmesh so no wonder it's not moving but at least I can explain that in detail now.
In order to fix this, you can move the collider up and it seems as long as it's above the navmesh the pathfinding will calculate correctly and is actually quite accurate. It's just that because of the previously mentioned height offset with the navmesh baking above the ground that ruins the height accuracy that you would otherwise get with normal settings. I didn't know this was what was happening, so I was getting extremely frustrated as none of the tweaks I was previously trying was working because I was incorrectly expecting the navmesh to just bake itself to the ground fairly accurately.
The next logical thing I was thinking of would be to simply have the villager mesh itself be moved further down either through a raycast or by moving the mesh down to account for the height offset but leaving the collision shape in the same place. This did actually work, to a certain extend and simply dragging the villager mesh further down it did go to the ground, however because of the height offset and the way the collision was following navmesh above the ground it resulted in the mesh frequently clipping through the ground and losing it's height accuracy.
In terms of keeping everything accurate, the best thing I can think of is maybe re-modelling the mesh entirely so it's not at such an angle that the baking works fine. However I feel like that takes away from the sort of aesthetic I'm looking for with regards to how I've designed my island and I would like to see if the back end devs could have a look at this in detail. I know they're pushing ahead with various features for 4.0 which I am looking forward to but it would be nice because especially for noobs who want to try what I'm doing it would prevent a lot of headaches in the future.
By the way, moving both the navmesh and collision shape way above the baked navmesh has the same result so again, the height of the baked navmesh cells seem to have been causing the majority of my problems.
I'm just posting again to explain in proper detail now that I understand what it is that I've been tweaking so much to get the agents to work correctly. I may try redesigning my island since that's a solution I can do myself and seeing how much of a difference that makes and if it does then that will confirm what I've been testing.
