One thing I've noticed about this forum is that there are often extremely enthusiastic people who are new at Japanese but study very often, and make very strong claims about their language ability. For example, they've been studying for only one year, but they claim that they can understanding 90% of native media like dramas or books. Most of the time they talk about this very quick progress at the early stage, but then eventually talk about their progress less and less.

So, my hunch is that this is related to the Dunning-Kruger effect. I'm basically taking this from Wikipedia, but the Dunning-Kruger effect is that people who are incompetent at a certain task think that they are very good at the task. Because they are not good at the task, they cannot adequately judge their level, so they have a tendency to rate themselves as being very good. That is, they suffer from illusory superiority.

(This is also probably related to how fast people think they will learn. Someone who just passed N3 might say that they think they can attain N1 in a year. But, actually, going from N3 to N1 in a year is very very difficult. And, what's more, someone who is at N3 level has absolutely no good way to assess the difficulty of N1. If you showed someone at N3 level some questions from N2, N1, and various tests made for Japanese people, they probably couldn't distinguish which ones are more difficult. It would all be Greek to them. So, there's no way they can estimate their level in relation to N1. Therefore, there's no way they can estimate how long it will be until they can pass N1)

What's more, the other side of the Dunning-Kruger effect is that people who are pretty good at a certain skill will generally rate themselves more poorly than those who are bad at the skill. They have illusory inferiority. So they might think that there are not so good at Japanese, even though they are really at quite a high level. So, someone who only understands only 30% of a native media source might say that he understands 90% (illusory superiority). But someone who understands 80% of a media source might say that he only understands 60% (illusory inferiority).

As for the hypothesis, it was that for a given skill incompetent people will:
1. tend to overestimate their own level of skill;
2. fail to recognize genuine skill in others;
3. fail to recognize the extremity of their inadequacy;
4. recognize and acknowledge their own previous lack of skill, if they can be trained to substantially improve.

Anyone think there is any merit to this way of thinking about it?

Personally I think it has more to do with people wanting to brag on the Internet than anything else.

i'm not sure it has much to do with either of those things.

Though probably there is some Dunning-Kruger effect, i very much doubt it's anywhere near as pronounced as that suggests.

i think you've got to consider both the types of things that people say they understand around 90% of, and the way people are learning on this forum.

it actually doesn't take very long at all to understand 85%+ of most of the common materials that beginners use, like drama, certain anime, etc, because they use a relatively small vocabulary set, and repeat set phrases often. i remember actually going through some scripts and counting the words i had never come across before after doing the core6K + subs2srs for a while, and in the easiest it was 8 words. That's all! A more average one may have been around 35 or so, and only the hardest, like medical dramas had any significant amount.

If you switch to academic subjects in Japanese, you see the same kind of thing. Another few thousand or so words and things become accessible in that subject really quickly. I'm noticing the same thing now, when i've started reading fiction... at first there are many new descriptive words to learn, but they are used widely, across different books. Another few thousand, and you're only annoyed occasionally.

i think the reason lots of people report their massive progress in the first year, and not so much afterwards is mostly that that's a huuuuuge jump, bigger than you'll ever really get again. After a year, sure, you get a biggish jump whenever you turn to something new, but it never adds as much to your understanding as that 1st year. In the 1st year, you've progressed so far that you assume that last whatever% isn't going to take as long as it does. But when you're still annoyed by constant new words that you haven't come across before another year later, even when they don't account for much % wise, of course, you don't feel like you're making that much progress...

If think you bring up some interesting points IceCream, but I also think that Tzadeck is right. It's just a case of "the more you know the more you know you don't know", I can take see it in my own example.
People usually don't concentrate on what they have (at least for too long), they concentrate on what they don't have. And for an advanced learner it might seem harder to progress because as a beginner you have lots of tutorials, establishes strategies etc., and when you get to advanced level you may get somewhat lost... you are constantly encountering some new words and phrases although you thought you learned everything as a beginner, and you wonder how much is still there till real fluency? And it's also harder to get apparent results when you are an advanced learner, at beginner stages you can learn some grammar structure and you are instantly encountering it all the time and you are happy, while at advanced level you learn something and it never appears since then.

And on top of that I think that it is somewhat of a psychological defence mechanism that when you're a beginner you think that learning the language isn't so hard, you just do that, learn that and you will be good - it gives you motivation and doesn't make you overwhelmed. When you are advanced you can drop that defence mechanism a bit and you are starting to see the enormity of the language and task still in front of you.

I actually think that some of those who are making the stronger claims regarding their progress and or level are also the people who I personally believe are also using the soundest methods. People like ta12121, mezbup, jarvik and probably a few others. There may be some exaggeration but for the most part I don't doubt their reported levels because their results and methodology agree with my own experience. Though it is certainly true that the better you start to understand your true level, I think unless someones claims appear irrational I don't see the need for excessive nitpicking about reported results. It is usually quite easy to tell from a persons writing style and personality as it comes across online if they are making ridiculous claims or not.

Pretty much only skills assessments I take notice of from here are actual test results and answering questions in the questions thread...

I think a both what Tzadeck and Ice Cream said can be true. Some people definitely get the idea that if they can follow conversations in dramas they will be able to follow real life conversations which isn't always the case. It must be pretty difficult to judge where you're at if you're not in Japan.

@Tzadeck

Pretty much. I think it's mostly online ego, though, and perhaps the belief that lying to oneself for motivation will benefit and autocorrect in the long term rather than result in less efficient learning as errors and knowledge gaps are glossed over rather than corrected and filled. People are quite resistant to the idea, however, so I tend to just focus on stuff related to metacognition, which is also often touched on by folks like Kornell, Carpenter, Rohrer, Roediger, Karpicke, et al. And also the idea that personal testimony over the web is much less important than sound logic and scientific fact. (If I actually succeeded at that, I suspect certain gurus would be out of work, and there'd be a lot less language blogs online.)

This is also why I keep an eye on how and how quickly bad advice is politely ignored on this forum. ^_^ It's a good sign when new learners start things off with a keen sense of what nonsense to skip over, while appreciating the helpful spirit in which it was presented.

I think it's a bit misleading to just refer to the "dunning-kruger effect" like it's a Thing, looking at wikipedia (http://en.wikipedia.org/wiki/Dunning%E2%...ger_effect) it doesn't seem that well established and I think you're making it sound much more general than it has been proven to be (so far). The most obvious difference between how you're using the term and what I understand to be the chief findings related to the effect is that the Dunning&Kruger aren't making any strong claims about how people change over time.

That is, D&K are saying that persons who are poor performers fail at seeing their own flaws and overestimate their ability while high-performers seek them out and have a down-to-earth estimation of their ability. It is not saying that a person will fail to see its flaws as it starts learning something and then, later on once it has learned a lot, it will learn to see its flaws. It's not that specific. For all we know, it might be that the majority of "good performers" are effective at evaluating themselves from the get-go when they start learning something and that's part of why they become good performers. Meanwhile, a minority of the good performers are people who are still bad at estimating their skill but because they've been at it so long or because of some other beneficial trait they had from the get-go, they made it anyway.

Of course, this is merely speculation to show how there might be different explanations (and they're very clearcut and simplifying our complex reality, this in order to make my point more easily understood) for the effect that don't harmonize so well with the OP. I'm not saying that the OP's claims are wrong, just that they aren't strongly backed by its references. I think another important factor in all this is the learner's age; many start learning Japanese in their early adulthood and may be a bit inexperienced and rash.

I'd also like to say that while there IS too much babbling about levels left and right I really don't think it's so unreasonable to think that a full-time (as in, devoting ~35 hours/week) Japanese student could make it to "JLPT1 level" from a "JLPT 3 level" understanding in one year. Without any special abilities. As long as they pour the time in and the way they learn is pretty good, I'd say it's doable. Not everyone wants to use that much of their time to learn the language but I think that for the people who do decide to go to university/college to study Japanese, setting the bar at somewhere between JLPT2 and JLPT1 in one year is a sound idea. Assuming they have no diagnosed learning impairment etc etc. IMO, when it comes to languages, it is better to talk about what amount of time it will take to learn things instead of using the diffuse and anxiety-evoking word "difficulty".

Edit: Also I totally agree with Icecream.

Editedit: When I said "setting the bar at JLPT2-1ish" I didn't really mean "aim at", I think it's a terrible idea to shape your studies based on what the JLPT tests look like. What I meant is that the students should aim to achieve a level of Japanese fluency where among other things they can pass JLPT2 and at least not be too far from passing JLPT1. Hope that makes sense.

I believe ta121212 said he'd need 3-5 more years till he's satisified and [I believe] he's at JLPT1 or better.

Damn, I still have a long way to go...

@IceCream
Useful information, I now know what to expect

Another possible factor is that it's easy to over-assess your proficiency. It seems like every year I think "OK, now I actually understand most of what I'm reading".

One more thing that influences over-estimation at the beginner level of language study is that usually native speakers are really generous in praising your language skills at that level. As you advance in a language, praises are less and less excessive, while on a really advanced stage native speakers will more often point to your mistakes than praise how 上手 you are.

nestor Wrote:It's a good sign when new learners start things off with a keen sense of what nonsense to skip over, while appreciating the helpful spirit in which it was presented.

But new learners don't have the relevant experience to assess what is nonsense or not. Beginners (and people in general) should start off with an open mind and stay that way, listening to various opinions while gaining their own experience and knowledge which will allow them to judge the credibility of people's claims and advice with more accuracy.

It seems to me that a lot of beginners start off very enthusiastically sticking to some new method x that they read about somewhere, and are somewhat sold on that method from the get go (the dominant paradigm here seeming to be SRS+Sentence+Media). Some of these people then reserve their healthy sense of skepticism to everything but what they themselves are doing. Because beginners have only just started, they tend to favour methods that offer results in the short term and they can't accurately assess what would have been best to do two years ago because that time hasn't passed yet. In some longterm endeavours like language learning, with the majority seeming to be beginners that often results in the same bad advice getting reinforced. I suspect the same thing happens in plenty of other contexts to, such as dieting and exercise.

If anything this problem persists because people aren't reporting their results enough, not that reported results are being exaggerated. The problem is a lot of people are echoing the the same methods and advice without offering up any results (be it anecdotal or not).

nadiatims Wrote:If anything this problem persists because people aren't reporting their results enough, not that reported results are being exaggerated. The problem is a lot of people are echoing the the same methods and advice without offering up any results (be it anecdotal or not).

I'd just like to add that I think that even if more people reported their results, there wouldn't be that much of a difference unless the way they filter their reports changes. A lot of people only report results when they are good and skipping it when they're doing badly. I've certainly had my ups and downs and I'm not ashamed to say that I'm far from being at any kind of peak right now as I know from experience I'll get back up there, but many learners have very low confidence. There's also the matter of learners who give up - a lot of people who were really bold and told everyone how well they are doing just disappear after a while and we never hear what it was that didn't work out for them and why they quit, even though that's some of the most important information we as learners could have. Instead we mostly get people who succeeded telling everyone what they think made other people fail (far too often, "they didn't do what I did").

It's possible that this is linked to the Dunning&Kruger thing, ie a subgroup of the people giving up could be "poor performers" who become devastated and give up once reality rears its ugly face at them and their flaws start to become too obvious for them to ignore. Of course, it's bound to be mostly a question of lack of time in a great deal of cases. It's likely that many of these really enthusiastic learners aren't planning ahead and forget to manage their lives, putting too much time into Japanese for a period and then finding they have no time at all for Japanese because there's so much they need to do after neglecting it all.

nadiatims Wrote:It is usually quite easy to tell from a persons writing style and personality as it comes across online if they are making ridiculous claims or not.

nadiatims Wrote:But new learners don't have the relevant experience to assess what is nonsense or not. Beginners (and people in general) should start off with an open mind and stay that way, listening to various opinions while gaining their own experience and knowledge which will allow them to judge the credibility of people's claims and advice with more accuracy.

It's not always easy to assess a member's credibility or the quality of their advice. A community learning site like this necessarily involves a kind of honour system. As I see it, we each have a responsibility to offer some honest assessment of the accuracy of any information we provide. I think people are generally pretty good about that.

Members who can be relied on for high quality answers (pm215 comes immediately to mind, but there are many others) generally offer some indication of their level of certainty [and/or use references.] They've earned the trust of regular members, but even newcomers would be able to safely gauge the reliability of their information. It's clear they do their homework. They also invite alternative views where appropriate.

Some other members, unfortunately, sometimes offer uninformed opinion with such certainty, authoritative tone and exaggerated 'evidence' that newcomers are more likely to be misled by it. (They'd wouldn't expect anyone to fake certainty or stats about language learning.) Such voices of absolute authority often reject alternative views and ignore references.

Exaggerated or incorrect information also wastes heaps of time. People often feel compelled to counterbalance potentially misleading information or bald assertions that others are 'wrong'. This, in turn, can trigger needless debate and time-consuming research just to persuade someone to stop making false or unsubstantiated claims. (Research that should have been done by the person who made the claim!)

Perhaps this misplaced certainty also stems from insufficient awareness of the limitations of their own knowledge? Or maybe it's just a matter of style of expression? (I can't imagine they actually believe statistics comes in only two flavours: 90% or 95%. lol) Or perhaps it's somehow related to ego, as someone mentioned earlier? I mean, how can one have a clearly demonstrated lack of knowledge about a language issue, yet still insist that native speakers, Japanese instructors, graduate students and academics are all wrong? gah

Surely the goal is improved community knowledge, not winning some individual prize for having The Best Method, or The Biggest Vocab Pile or The Only Correct View. So, yeah, beginners beware. 95% of really authoritative posts are definitely crap and anyone who thinks otherwise is totally wrong. ;p

Quote:In some longterm endeavours like language learning, with the majority seeming to be beginners that often results in the same bad advice getting reinforced.

Who gets to be the arbiter of "bad"? Based on what? The opinion of another beginner? :-)

[edit] fixed words

lol. truly masterful post that one.

thora Wrote:They've earned the trust of regular members, but even newcomers would be able to safely gauge the reliability of their information. It's clear they do their homework.

Why is it important to have earned the trust of regular members? Can't we just judge people on the solidity of their arguments irrespective of post counts?

thora Wrote:They also invite alternative views where appropriate.

In other words, views that don't disagree with the majority opinion. Who gets to decide which alternative views are appropriate or not?

thora Wrote:This, in turn, can trigger needless debate and time-consuming research just to persuade someone to stop making false or unsubstantiated claims.

No-one is forcing you to waste your time trying to persuade others.

thora Wrote:I mean, how can one have a clearly demonstrated lack of knowledge about a language issue, yet still insist that native speakers, Japanese instructors, graduate students and academics are all wrong? gah

So lemme get this straight, in order to be taken seriously one needs to either be a native speaker, graduate student or an academic? Not everyone cares to enter graduate school and or pursue a career in academics. Does knowledge and experience in a subject count for nothing if gained in the real world rather than a university campus?

thora Wrote:Who gets to be the arbiter of "bad"? Based on what? The opinion of another beginner? :-)

No one does, that's my point. It's up to the individual to judge the credibility of anything that's written on this forum. People should be skeptical but also open to all opinions and judge them against their own experience. I was just describing a phenomenon whereby beginners inevitably lack the experience by which to judge the quality of advice and may tend to choose and promote popular methods over effective ones. If people would at least provide some indication of their results (anecdotal or not), everyone would have bit more to go by. A lot of the new threads that pop up are people asking for advice. They'll then get differing advice from a bunch of different posters, which is fine and to be expected, but how can the OP have any idea of what will work best or even just well? Some advice could well be coming from someone who has be doggedly sticking to that method for years without making serious progress. Certainly there will be some overestimation and underestimation here and there. Most like none of it will be 100% accurate, but I don't see the need to immediately call bullshit on the people that are actually reporting decent progress.

@Thora

Good comment. ^_^

Overtime you'll realize that your level of skill in a language will never reach "Mastery" level. What I mean by that is: you can't know everything, nor should you. I'm more relaxed now about my skill then when I started. When I began, all I cared about was "fluency!!!". Now it's more about heading into new areas that I still find hard.

I could have done things differently if I had the knowledge back then but you can't go back in time, so better make due now. My srsing has slowed down nowadays(due to not having an active computer to use. But I plan by next month to buy a powerhouse laptop, so my "extreme" Japanese learning will continue. Why did I put quotes there? I've gotten used to srsing vocab at a good pace and adding new sentences from context. I can basically blow through 50-100 new vocab a day and not feel a sweat coming down. That intense first year of studying has taught me: "Go at your own pace". Some people love learning fast/a lot while others prefer taking there time and gradually gaining the skills. So choose what's most important to you.

And one last note: this is what I feel I can do with Japanese, if I continue(not literally):

TheKorv Wrote:I believe ta121212 said he'd need 3-5 more years till he's satisified and [I believe] he's at JLPT1 or better.

Damn, I still have a long way to go...

@IceCream
Useful information, I now know what to expect

What I've come to figure out is: I believe if you do it right(learning methods,immersion). You could gain fluent(advanced) skill in reading,listening and speaking. But I believe the writing will take another 1-2 years. I'm talking about physical writing as well as other ones. So 3-5 years doesn't sound too off. But then again, putting in the time will prove you can reach fluency faster.

Boom! This thread just got ta12-ified.

nest0r Wrote:Boom! This thread just got ta12-ified.

凄いよ　[すご]いよ

Tzadeck Wrote:Most of the time they talk about this very quick progress at the early stage, but then eventually talk about their progress less and less.

So, my hunch is that this is related to the Dunning-Kruger effect.

Or it could simply be diminishing return. From my experience the way people rate themselves at language use is based mostly on personality. Mostly people just think their level is shit. I've met people fluent in English who either thought their English was shit, or pretended to convey this idea. I've met people whose English was shit and they admitted it.

Bragging about progress and rating one's ability aren't necessarily the same thing. After my first year I made a somewhat satirical post about fluency to try to convey this idea. I made a lot of quick progress during that first year, and yet I still thought my level was shit. I still think my level is shit in my 3rd year (mostly because I've almost completely stopped studying). And yet strangely, I became fluent after 1 year.

IceCream Wrote:i'm not sure it has much to do with either of those things.

Though probably there is some Dunning-Kruger effect, i very much doubt it's anywhere near as pronounced as that suggests.

i think you've got to consider both the types of things that people say they understand around 90% of, and the way people are learning on this forum.

it actually doesn't take very long at all to understand 85%+ of most of the common materials that beginners use, like drama, certain anime, etc, because they use a relatively small vocabulary set, and repeat set phrases often. i remember actually going through some scripts and counting the words i had never come across before after doing the core6K + subs2srs for a while, and in the easiest it was 8 words. That's all! A more average one may have been around 35 or so, and only the hardest, like medical dramas had any significant amount.

If you switch to academic subjects in Japanese, you see the same kind of thing. Another few thousand or so words and things become accessible in that subject really quickly. I'm noticing the same thing now, when i've started reading fiction... at first there are many new descriptive words to learn, but they are used widely, across different books. Another few thousand, and you're only annoyed occasionally.

i think the reason lots of people report their massive progress in the first year, and not so much afterwards is mostly that that's a huuuuuge jump, bigger than you'll ever really get again. After a year, sure, you get a biggish jump whenever you turn to something new, but it never adds as much to your understanding as that 1st year. In the 1st year, you've progressed so far that you assume that last whatever% isn't going to take as long as it does. But when you're still annoyed by constant new words that you haven't come across before another year later, even when they don't account for much % wise, of course, you don't feel like you're making that much progress...

Whenever I watch news+subtitles(jp). I notice that I can read/understand pretty much all of what it says(95%). There is occasional new vocab+names that I wouldn't know how to read. I've learned you just gotta keep learning and learning. There isn't that 100% mark in languages. One should only aim for 90%-99%. But never 100%.

It seems to me (and this is speaking from little experience) that this phenomenon is related to diminishing returns, as kazelee said. When you start learning a language, your goals are bountiful, small, and easily defined; things like learning how to form conditional sentences or write hiragana et cetera. Later on, the things you want to achieve are amorphous and ill-defined. This makes it harder to say "I didn't understand X, now I do."
Also, the large majority of language comes from a large collection of those small, achievable goals. So it's natural for someone who has never set out on a long-term learning experience before to think they will achieve fluency faster than it will probably take them in reality. I think it has less to do with overestimating than assuming the rate at which you attain knowledge at the beginning will continue forever. I taught guitar for a while and saw the same thing with my students then. It's easy to think you'll be the next Hedrix when you memorize a few new scales but, as you get better, you realize that being great requires tons of time spent on tiny details and solid goals like "learn the dorian mode" are replaced with loose goals like "sound better". For most people just being able to play a few songs and jam with friends is enough and, so, we live in a world with a bunch of mediocre guitar players (including myself). The same is true for language learning. For a lot of people, sounding perfectly native just isn't worth the effort. I think a lot of people making early claims of greatness are just not familiar with long-term learning and the ever-decreasing rate at which one tends to improve.

On a side note, this post reminds me of why I love science so much. The more you learn, the more you realize how little you know. But if you like learning, realizing you know less than you thought just means you've got more room play and get to do more of what you love.

nadiatims Wrote:Why is it important to have earned the trust of regular members? Can't we just judge people on the solidity of their arguments irrespective of post counts?

Nothing to do with post count. Regular members can comfortably rely on information posted by members who have earned a reputation for consistently providing quality information and for demonstrating personality traits such as thoroughness, willingness to admit error, reasonableness, willingness to seriously consider alternatives, etc. Someone with a high post count but consistently shoddy information wouldn't have much credibility. As you said earlier, you can tell from people's personality and style whether they're claims are likely to be exaggerations.

Newcomers have no historical knowledge, so obviously they can only make an assessment based on the content of an individual post. But as you say, beginners often don't have the experience or knowledge to gauge the quality or reasonableness of the content. (Nonsense reasoning is something different.) Including information regarding opinion vs fact, references, certainty, experience level, etc. can obviously help them determine how much weight to give it. Newcomers can't easily detect exaggeration, so we shouldn't exaggerate. :-)

thora Wrote:They also invite alternative views where appropriate.

nadiatims Wrote:In other words, views that don't disagree with the majority opinion. Who gets to decide which alternative views are appropriate or not?

No. They invite other answers when they aren't 100% certain and they acknowledge when alternative views exist (and might be able to discuss their relative merits.) The topic determines whether or not alternative views exists, not the person. In other words, they don't ignore views that don't jibe with their personal opinions.

I haven't noticed any suppression of minority opinions. Many people are able to offer opposing or unpopular opinions on various topics without triggering any big debate or potentially misleading beginners. I believe that's a result of the way in which they present and are able to justify their views.

Is it possible to fool yourself into thinking you know more than you actually know? This guy certainly did:

http://forum.koohii.com/showthread.php?tid=7248

He genuinely believed he could understand Chinese, until he was tested; where it was revealed he couldn't speak or understand the most basic conversation.

I wonder if just "getting used to" the sounds and grammar patterns of a language is part of assuming you're close to mastering a language?