Maths by its nature is a bit abstract, but I think that the primary issue is that it is sequentially learned for the most part in primary and secondary education. If there is ever a point that a student is struggling with a concept and the teacher/parents don't identify it in time, the student is then faced with not understanding new concepts afterwards, and may just be left behind.

This is starkly different to other fields a student will be presented, they generally will have multiple topics that are not strictly reliant on parts learned before, and that can be relatively easily co-developed in everyday life.