Following up on that last introductory post, here's my original list – drawn from a quarter-century of study! – of what's hard about learning Japanese.
The three hard things
Here's a little good news to go with the bad: My list only goes as far as three items. That can't be so bad, right? Well, decide for yourself:
Let me assume for the moment that you, dear Reader, hail from a background speaking English or another European language.
Japanese isn't related to English (well, not until we go back to some proto-Ur-language), or the family of Indo-European languages. So if you're coming from one of those, you don't get the little "freebies" – familiar structures here and there – that you get when tackling a language from somewhere within your tongue's family tree.
So the initial hurdle is big: Many points of structure will seem new to you, maybe even "bizarre", right from Lesson 1. Obviously, that's not something remotely unique to Japanese; you'll find a similar gap any time you go to learn a language unrelated to your own, with large differences in structure. This factor says nothing about the language being difficult in absolute terms; it only makes it difficult relative to tongues with more similarity to what you already know. (Naturally, it works in reverse, too: Japanese school kids all suffer through the initial hurdles of acquiring English, a language unrelated to their own.)
If you know Korean, on the other hand, you're in luck: You get to cross this difficulty off the list. You'll find the structure of Korean nicely similar to its Japanese relative. (That alone hardly guarantees fast fluency, though I have met Koreans who picked up excellent Japanese with a speed I could only envy.)
2. Formal speech
This is more properly referred to as "honorifics", and technically breaks down into respectful language, humble language, and polite language. I'll call it all "formal speech" for now.
Wherever you hail from, your native tongue probably has little interlocutions to pretty-up an expression, to display respect, and so on. So there's nothing new here in concept. But there's no getting around it: In Japanese, formal speech gets complex – possibly more so than in any other major language. (I believe Korean gets pretty complex as well; I'm not qualified to say whether it gets as complex.) There's a lot to learn – and regardless of what a slim phrase book or some expat English teacher may tell you, if you really want to learn the language, you do need to dive into formal speech.
3. Reading and writing
This one you've probably expected. I'd call it the biggest difficulty, and it's an absolute, not a relative, one. That is, it doesn't matter what your native language is; if you want to learn to read and write Japanese, you have a lot of study ahead of you. (There's no magic in how the Japanese do it, either; school kids learn it through a lot of study.)
If you know Chinese characters, you get a big head start over someone who wouldn't know a 大 from a 太. But that will really only take you so far; fluency in written Chinese will let you pick out lots of key words and names from Japanese text, yet not necessarily yield much context or understanding. You've still got a long path ahead. (Incidentally: Yes, that works in reverse too. Fluency in written Japanese will let you pick out plenty of useful bits from Chinese text, though not let you actually read typical material.)
This isn't to downplay the difficulty of learning to read and write other languages. In fact, comparisons of written Japanese and (for example) English tend to vastly underrate the difficulty of the latter, a topic on which I'll have much to say later. But at the same time, I have no reservation in saying that achieving a given level of literacy in Japanese takes more time, in absolute terms, than reaching that same level in English, due to the demands of Chinese characters.
In summary, there are only three big hurdles for the learner of Japanese:
1. Unfamiliarity as it's not related to your native language (unless that's Korean)
2. Formal speech
3. Reading and writing (thanks to Chinese characters – though obviously, literacy in Chinese gives you a leg up)
Not so bad, really. And here's the good news: there's plenty more that's easy about learning Japanese!