My father learned to speak, read and write Chinese — just as a hobby! He spoke a dozen other languages already, but it took him many years — he said it was the hardest thing he ever did.
Some years ago I read a fascinating article by a linguist on this subject, but assiduous searching did not find it even though I remember it very clearly. He pointed out that learning written Chinese is even hard for Chinese — for example, it takes Chinese kids almost twice as long to become literate enough to read a newspaper as western kids.
The article noted that using a Chinese dictionary is hard enough that China has dictionary search competitions in high school — something like debate club. He mentioned also that it’s not uncommon for educated Chinese people to temporarily forget the character for fairly common words (“tin can”), and it’s not like English, where you can spell out the word and see how it looks (“Prezident”, hmm, that looks wrong): if you don’t remember the character, you’re stuck without further information.
Now, when it comes to spoken Chinese, I get different stories as to the difficulty. Pretty well everyone agrees that the grammar is quite easy and logical, but opinions seem divided on how hard the tonal system is for foreigners to pick up — whether it’s just “hard” or “very very hard”. Sorry, “easy” isn’t one of the choices.