Kind of not true.
While it's a progressive subject, there are independent trees. For example, you can learn logic, geometry and algebra completely independently - and you can fail at one and succeed at the others.
However, I agree with you that within each tree, if you miss some fundamental detail, you will continue to get stuck.
I studied mathematics in university almost 40 years ago and it changed my life and how I thought. I recommend developing a love of mathematics to anyone, as it will make you a happier and more competent person.