I was able to find out why my models would often look blurry like.
First, I am using quad remesher for the sculpting. Nothing wrong with using the remesher, just need to put a triangulate modifier on it.
Then the sculpting itself needs sculpted more sharpness.
After this the bake cage and also the lowpoly target needs triangulate modifier on it. This makes the texture maps look better when baking. Such as the normal map.
And as always, use 32bit for the normal map. Because it will not look right without it.
