aphasiac Wrote:http://forum.koohii.com/showthread.php?tid=7248

He genuinely believed he could understand Chinese, until he was tested; where it was revealed he couldn't speak or understand the most basic conversation.

Ouch, that is painful to watch...

Also, I think tzadeck is pretty much spot on.

kazelee Wrote:

Tzadeck Wrote:Most of the time they talk about this very quick progress at the early stage, but then eventually talk about their progress less and less.
...(stuff edited out)...
So, my hunch is that this is related to the Dunning-Kruger effect.

Or it could simply be diminishing return.

I was saying that the Dunning-Kruger effect was related to their false assessment of their level. It's not directly related to how much they talk about their progress, or how much progress they continue to make over time.

The reason I mentioned the way they talk about their progress is because it usually gives away that their self-assessment was off. Although they said that they understood 90% of any standard drama after a year of study, it takes them much longer than that to actually pass one of the JLPT tests that would show that level of comprehension, or to start answering questions on the forum that any person with that kind of comprehension should be able to answer (for example, in the "What's this word/phrase?" thread). This becomes obvious when they talk about their progress towards the relevant JLPT level less and less as months go by.

I realize JLPT isn't a perfect indicator here, since the skill set is quite a bit different from understanding television, but it's the closest thing we have to an objective level tester that is often talked about in the forum.

ta12121 Wrote:Whenever I watch news+subtitles(jp). I notice that I can read/understand pretty much all of what it says(95%). There is occasional new vocab+names that I wouldn't know how to read.

I think this is what Tzadeck is talking about. 95% is a huge amount and more than what a native child would understand when watching the news. It can be easy to think you understand that much when youre not actively being tested or questioned on the material, so if there was something you misinterpreted you wouldnt know it necessarily. Plus there`s the visuals that tell half the story.

I`ve noticed that this effect is strongest with people who havent yet traveled to Japan or spent much time there. Back home in the States, I was the go-to guy for Japanese questions, but was rarely in situations where it was vital that I know it or the situations were something I could pick and choose, like video games or anime.

But once I went to Japan and was being put in new environments that werent covered in any textbook I had read (trying to fix an internet connectivity problem over the phone, an unexpected meeting with some higher-ups at the office, etc.), I realized that shit just got real and I speak Japanese like a caveman who has been trampled by a herd of wooly mammoths.

julianjalapeno Wrote:I`ve noticed that this effect is strongest with people who havent yet traveled to Japan or spent much time there. Back home in the States, I was the go-to guy for Japanese questions, but was rarely in situations where it was vital that I know it or the situations were something I could pick and choose, like video games or anime.

But once I went to Japan and was being put in new environments that werent covered in any textbook I had read (trying to fix an internet connectivity problem over the phone, an unexpected meeting with some higher-ups at the office, etc.), I realized that shit just got real and I speak Japanese like a caveman who has been trampled by a herd of wooly mammoths.

This raises two interesting points:

1) Self-assessments should be taken with a (large) grain of salt. We are humans after all, and often when we thought we understood something, it turns out we didn't actually understand fully (or correctly).

2) It is one thing to have a feeling of mastery over things like anime or dramas. But these are just forms of media, and there's a whole facet of real-world communication that only living and being in Japan will expose you to -- something which is often overlooked when we assess our comprehension abilities.

Tzadeck Wrote:

kazelee Wrote:

Tzadeck Wrote:Most of the time they talk about this very quick progress at the early stage, but then eventually talk about their progress less and less.
...(stuff edited out)...
So, my hunch is that this is related to the Dunning-Kruger effect.

Or it could simply be diminishing return.

I was saying that the Dunning-Kruger effect was related to their false assessment of their level. It's not directly related to how much they talk about their progress, or how much progress they continue to make over time.

But saying that assumes that it IS a false assessment in the first place. Which it may not be. And the reasons you gave to suggest it was (not mentioning progress after a year, etc) can just as easily be explained by diminishing returns.

As for the JLPT, i'm not sure JLPT1 is a perfect indicator, though JLPT2 might be a good one. It seems like you have to actually study for JLPT1 specifically in order to pass it in most cases, whereas JLPT2 seems like something you can pass just by reading & listening. Of course, if you'd studied a TON of different materials, you might pass JLPT1 on your own. But there's plenty of grammar points and words that you just don't come across that often, so...

Anyway, i'm not sure why it should be a false assessment, really. You could probably run the numbers for yourself and figure out how much of program X someone could understand as a percentage based on their vocab level.

...of course, that isn't the full story, because there would be some known words that people miss, other words you can understand through context, etc. But it would give you a rough estimate...

But, before people start Ta-bashing again, he did do that JCAT thing ages ago didn't he? And did really well, so...

I guess one point is the context thing. With more visual and body language cues in television, it does make it easier to understand than say, a radio program. And probably that also feeds into the self assessment. But language learning isn't a static thing, also, like Surreal said. There's ups and downs, and backwards steps, and understanding even through context is still understanding, so...

There's also plenty of english words that i wouldn't be able to give a definition of, and could only understand through context... perhaps in some contexts, i wouldn't get it at all. But language isn't a vacuum, either. In that sense, there isn't any % you can give that's accurate. But thinking that people say 90% when they really mean 30% is a bit much, i think.

ugh, there was some point i wanted to make but i'm tired and rambling. sleeeeeep...

EDIT: i guess i was trying to say, it depends how you test it. If it's by vocabulary level, or general understanding, i doubt you would see any massive "dunning-kruger" effect from people's self assessments. If you test in some other way, like pulling everything out of context and testing, or something similar, you would probably see more of one.
How much would also vary from person to person, i guess...

I can't understand why you worry about that "100%", especially you ta12.. There are times I watch German news and wouldn't understand 100% what they say either, perhaps 'only' 97%. I guess the same counts for any other natives of other languages, i.e. Japanese, too.
I also don't think that a person certified with JLPT1/2 necessarily is fluent in the language, i.e. both speaking and writing. It's not a guarantee and by now means an "automatism" or anything.

Concerning what you wrote, Tzadeck, by this Dunning-Krüger (ü?) effect some can boost theirselves in such communities like this one and by being totally anonymous I can't see a problem in such actions, as long there are people who *really* are good and have skill and can correct those people when they pretend to 'know it' and help others - in a wrong way and matter...

Tzadeck Wrote:

kazelee Wrote:

Tzadeck Wrote:Most of the time they talk about this very quick progress at the early stage, but then eventually talk about their progress less and less.
...(stuff edited out)...
So, my hunch is that this is related to the Dunning-Kruger effect.

Or it could simply be diminishing return.

I was saying that the Dunning-Kruger effect was related to their false assessment of their level. It's not directly related to how much they talk about their progress, or how much progress they continue to make over time.

I didn't edit anything between those lines.

Icecream touched on what my thoughts were after reading your next post. Another way to know for sure how falsely one perceives their ability would be to present a source, have a person rate their ability to comprehend the source, and then test their knowledge on the source.

Also, the 90% being referenced only encompasses the media being focused on. A person who understands 90+% of a war drama isn't going to understand 90+% of a legal drama unless the drama has a lack of specialized terms. For example, I understand virtually all technical media put in front of me, but wasn't until I took anatomy & physiology that half the stuff on medical shows made sense. Its the same for any other language.

To me it seems like this Dunning-Kruger effects is a drawn out version of "the more I know, the more I realize I don't know" with a strange emphasis on superiority and inferiority.

I've been thinking about this, and I'm going to side with icecream.

No one has made any claims that are actually unbelievable. It is totally possible to have 80-90% fluency in anime, dramas or manga within a year if you study a hell of alot. The same words and grammar patterns pop up repeatedly in similar media; so if you really work and master one show, you'll master most of them in that genre (except maybe 10-20% of new vocab / patterns).

Now combine that with people who are capable of studying for long periods per day (and I'm talking an ungodly amount - more than 99% of posters on here are capable of) and those have a natural talent for memorisation and learning languages.

The most unbelievable story was of course AJATT's Khatz, who claims 90% fluency in all 4 disciplines - reading / writing / speaking / listening - in 1.5 years. Yet most people on here believe him, myself included - simply for the fact that his story sounds genuine, and he has no real reason to lie about it.

kazelee Wrote:Another way to know for sure how falsely one perceives their ability would be to present a source, have a person rate their ability to comprehend the source, and then test their knowledge on the source.

My goodness that almost sounds like....a test?! Like...of fluency?! A Test of Japanese Language Proficiency??
Or J-CAT, J-Biz, or what have you...

I've seen a sense of JLPT and test scores =/= fluency around here, and I agree - it doesn't. But what it does give you is an unbiased way of measuring your ability. I'd say test scores are way more accurate than people's own self-reported scores.

You understood 90%...of what exactly? The vocab used? The grammar points? The dialogue? The general idea of what was going on? I was watching some variety thing on youtube a little while ago, and I understood everything that was going on. There were parts that I couldn't understand even 30% of what somebody was saying -- but it didn't hinder my understanding of the show.

That's why I'm always skeptical of people's self-reports here. Unless you have some sort of unbiased way to determine what's actually being understood, your scale of "understanding" could be completely different than mine.

Asriel Wrote:

kazelee Wrote:Another way to know for sure how falsely one perceives their ability would be to present a source, have a person rate their ability to comprehend the source, and then test their knowledge on the source.

My goodness that almost sounds like....a test?! Like...of fluency?! A Test of Japanese Language Proficiency??
Or J-CAT, J-Biz, or what have you....

You missed the part where the person is rated against their comprehension of that source in comparison to their perceive comprehension. Like if someone rate themselves as knowing 90% of the JLPT and the score reflects it. We're talking about perceived ability.

And I agree, knowledge != fluency.

Asriel Wrote:That's why I'm always skeptical of people's self-reports here. Unless you have some sort of unbiased way to determine what's actually being understood, your scale of "understanding" could be completely different than mine.

The majority of self reports here are statistical. IE studying 1000 words in 10 days. Finishing RTK after 3 months. Etc.

Also 90% isn't a lot.

It is hard to assess one's own level of knowledge in a language. If someone would come and ask me how good I think my Japanese is by now, the honest answer is: not good at all. This is because i still have many things to learn.

I can't measure my knowledge against that of a GoddessCarlie, IceCream, etc. because they know things that I don't know yet. So they are ahead of me, they are way better, and they might have passed JLPT Level 1, or been to Japan already.

Having passed a JLPT test does also mean nothing at all. You can learn for the test, you can memorize vocabulary, grammar, and pass the test. What does this say about the own level? You passed a test. Others failed it. But you have been working hard, and you've been learning for this test, acquiring knowledge, and working on your language skills.

And now what would I answer in addition to: not good at all? I know that the material i work with is comprehensible to me. This is enough to say about my knowledge. This does not allow anyone to test my ability to use the language. Nor can anyone come and say you are beginner, you are intermediate, advanced.

The only way to measure my knowledge, and assess how far I come is my own understanding of such materials as books, media, chatroom talk. Can I understand what is going on 100% - no. Can i get the general gist of what's going on? It depends. Can I follow a longer storyline without looking things up? Definitely no! Do i think i have to say i understand 30% of something, or 100% of one thing? Of course not. Because I can understand something up to a 100% when I read it, listen to it, and know what is going on.

What I know is that a level does mean little to me, and it means little to others, how good or bad my level is at this very moment. Another thing that can not be assessed. You can be well prepared [for a test], you know, that you know, what you have to know. But you get nervous and fail. Another time, you learned nothing at all, but you passed the test with ease. The result of this test taken at that moment tells everyone that you are a winner or a looser, but it does not tell anything about your real abilities.

Sometimes it's bragging. I know more than you do, 20.000 sentences, 40.000 words, the word of our lord backwards, in Japanese, i have read 400 books, watched tons of movies so I know everything there is to know. At the end of the day it only matters how much you know, to be able to function in a language, no matter what you tell others about your abilities.

I think many people who use this site believe that if they learn X amount of words, mine Y amount of sentences and can write Z amount of kanji they will become fluent. Learning a language, well more specifically becoming fluent in a language, is not something that you can put set targets on, so that by learning XYZ amount I will understand 90% or 95% of everything I hear or read.

Yes you might be able to read a full news article and only find 2 or 3 words you don't understand, or you might be able to understand a full dialogue in a drama/anime, but can you actually maintain a spoken conversation in Japanese at an adult-like level, while sounding natural without too many errors? If you haven't gone to Japan and lived there for some length of time then I would say probably not.

Obviously the harder you study the easier it is, because you have a greater range of words to choose from, but do you think you are going to use 5000+ different words over the course of a day, or even a week? Probably not. Becoming fluent in a language takes a lot of time and it is a natural process, which you can assist along the way. In my opinion, you almost need to create a second mind within your brain - one that functions fully in Japanese be it thinking, listening, speaking, reading, that you essentially 'fall into' when you switch from speaking, reading, listening, thinking in your native language. And all of this takes a long time.

But gearing yourself up to having X amount of understanding once you have done Y amount of study is just setting yourself up for disappointment in the long run.

Slightly of topic, but I think it reflects the attitude of quite a few on this website.

kazelee Wrote:You missed the part where the person is rated against their comprehension of that source in comparison to their perceive comprehension. Like if someone rate themselves as knowing 90% of the JLPT and the score reflects it. We're talking about perceived ability.

Well then you ask them how they did right after they take it, and then find out their score a few months later when it comes in. Just like coming out of a test in real life, you've got people talking how it was.
Here's some examples of what I'm talking about (I use myself as an example, just because I know that I posted in both places)
Right after: http://forum.koohii.com/showthread.php?p...#pid121438
Actual Results: http://forum.koohii.com/showthread.php?p...3#pid66923
Just saying that it happens.

kazelee Wrote:The majority of self reports here are statistical. IE studying 1000 words in 10 days. Finishing RTK after 3 months. Etc.

This is a very good point. In terms of speed/quantity, I would hope that the percentage correct of mature cards would also be reported, though. It's really the more "generic" type of "I understood 95% of it" that gets me. You're quantifying something that can't accurately be quantified. Even if you could, everyone's perception of comprehension is different.

If person A says that he only understands about 75% (of all the words spoken), but person B says that he understood 95% (of what was going on), then what? There's definitely a bias. I think this is partly where the 'effect' comes in. Person B actually thinks he's understanding everything, but Person A realizes that he doesn't.

Nagareboshi Wrote:Having passed a JLPT test does also mean nothing at all. You can learn for the test, you can memorize vocabulary, grammar, and pass the test. What does this say about the own level? You passed a test. Others failed it. But you have been working hard, and you've been learning for this test, acquiring knowledge, and working on your language skills.

Yes, but there's a big difference between studying Japanese and then passing a test, and studying FOR a test and then passing it. I would hope that most serious language learners would go for the former.

Nagareboshi Wrote:I know that the material i work with is comprehensible to me. This is enough to say about my knowledge...The only way to measure my knowledge, and assess how far I come is my own understanding of such materials as books, media, chatroom talk.
...
At the end of the day it only matters how much you know, to be able to function in a language, no matter what you tell others about your abilities.

I agree with you, all that really matters is your actual abilities, regardless of what you tell other people. You don't have to tell anyone anything. The effect is more about where you believe that your skills actually are. "I understand xx% of xx manga, so I'm pretty much fluent" etc...


So yeah...I think what I'm trying to say is that no matter what you tell people your level is, no matter how much you say you understand, you can't really be sure without an unbiased way to tell. I think the post-test self-reports followed up by real results is a good way to judge both real comprehension (assuming one isn't studying just for the test) and their perceived comprehension, and their relation to each other.

You can say that it doesn't matter, as long as you keep progressing -- but how can you be sure you're progressing if there's no gauge? It can be a test, or it can be "Now I can read this book that I couldn't before " measuring progress is good.

kazelee Wrote:To me it seems like this Dunning-Kruger effects is a drawn out version of "the more I know, the more I realize I don't know" with a strange emphasis on superiority and inferiority.

Please, please, people read the wikipedia article: http://en.wikipedia.org/wiki/Dunning%E2%...ger_effect

The Dunning-Kruger hypothesis ("effect" is too powerful of a word imo) isn't saying what people are confusing it for! I wrote about this in my earlier post but I guess it was skipped over by most members... At least read the wikipedia article yourself and think about what D&K are actually claiming instead of making it much more than it really is. The way it is now, it's just being used as a buzzword and puzzling people. In fact I'd say it would have been better not to use the term "Dunning-Kruger effect" to begin with.

julianjalapeno Wrote:

ta12121 Wrote:Whenever I watch news+subtitles(jp). I notice that I can read/understand pretty much all of what it says(95%). There is occasional new vocab+names that I wouldn't know how to read.

I think this is what Tzadeck is talking about. 95% is a huge amount and more than what a native child would understand when watching the news. It can be easy to think you understand that much when youre not actively being tested or questioned on the material, so if there was something you misinterpreted you wouldnt know it necessarily. Plus there`s the visuals that tell half the story.

I`ve noticed that this effect is strongest with people who havent yet traveled to Japan or spent much time there. Back home in the States, I was the go-to guy for Japanese questions, but was rarely in situations where it was vital that I know it or the situations were something I could pick and choose, like video games or anime.

But once I went to Japan and was being put in new environments that werent covered in any textbook I had read (trying to fix an internet connectivity problem over the phone, an unexpected meeting with some higher-ups at the office, etc.), I realized that shit just got real and I speak Japanese like a caveman who has been trampled by a herd of wooly mammoths.

that makes sense but I plan to combat this by getting as much real-life practice as I can. Although from what I hear, a lot of people didn't have much trouble in Japan, after studying it overseas for a few years. Most people noted they couldn't read certain things(brands,products,food names,place names,etc). Which makes prefect sense, even I don't know how to read a lot of those things(yet). But with the srs, this will become easy. What I've noticed is, once you get to a good point in certain skills, diving into the other ones will get people far. It will comes down to: you suck now but later on you'll say "Hard to believe I was once there"

Tori-kun Wrote:I can't understand why you worry about that "100%", especially you ta12.. There are times I watch German news and wouldn't understand 100% what they say either, perhaps 'only' 97%. I guess the same counts for any other natives of other languages, i.e. Japanese, too.
I also don't think that a person certified with JLPT1/2 necessarily is fluent in the language, i.e. both speaking and writing. It's not a guarantee and by now means an "automatism" or anything.

Concerning what you wrote, Tzadeck, by this Dunning-Krüger (ü?) effect some can boost theirselves in such communities like this one and by being totally anonymous I can't see a problem in such actions, as long there are people who *really* are good and have skill and can correct those people when they pretend to 'know it' and help others - in a wrong way and matter...

I don't say 100% because most people think that's impossible. Which it is, but it really comes down to what your watching,reading,listening,etc. That's the thing about languages, there will always be something you don't understand. But that's what I've grown to love. I start out with 0% knowledge and get to 95% knowledge. So it really comes down to constant learning.

SendaiDan Wrote:I think many people who use this site believe that if they learn X amount of words, mine Y amount of sentences and can write Z amount of kanji they will become fluent. Learning a language, well more specifically becoming fluent in a language, is not something that you can put set targets on, so that by learning XYZ amount I will understand 90% or 95% of everything I hear or read.

Yes you might be able to read a full news article and only find 2 or 3 words you don't understand, or you might be able to understand a full dialogue in a drama/anime, but can you actually maintain a spoken conversation in Japanese at an adult-like level, while sounding natural without too many errors? If you haven't gone to Japan and lived there for some length of time then I would say probably not.

Obviously the harder you study the easier it is, because you have a greater range of words to choose from, but do you think you are going to use 5000+ different words over the course of a day, or even a week? Probably not. Becoming fluent in a language takes a lot of time and it is a natural process, which you can assist along the way. In my opinion, you almost need to create a second mind within your brain - one that functions fully in Japanese be it thinking, listening, speaking, reading, that you essentially 'fall into' when you switch from speaking, reading, listening, thinking in your native language. And all of this takes a long time.

But gearing yourself up to having X amount of understanding once you have done Y amount of study is just setting yourself up for disappointment in the long run.

Slightly of topic, but I think it reflects the attitude of quite a few on this website.

This is another reason why you must dive into areas you struggle in. Some people are fluent speakers but can't read. Some people can read/listen to a fluent level but can't speak well or write. So it really comes down to conquering what you have trouble with. So basically, keep going and keep learning.

As for studying set amount of words/sentences,etc. It's just a way of keeping me going on my goal of fluency. I'm not saying studying all those sentences/vocab=fluency. But it will give me a greater knowledge for listening,reading and overall the kanji. That's my goal for now. Don't worry, I'm not neglecting my speaking but the main goal is just never stop learning(then I'll succeed no problem)

Nagareboshi Wrote:It is hard to assess one's own level of knowledge in a language. If someone would come and ask me how good I think my Japanese is by now, the honest answer is: not good at all. This is because i still have many things to learn.

I can't measure my knowledge against that of a GoddessCarlie, IceCream, etc. because they know things that I don't know yet. So they are ahead of me, they are way better, and they might have passed JLPT Level 1, or been to Japan already.

Having passed a JLPT test does also mean nothing at all. You can learn for the test, you can memorize vocabulary, grammar, and pass the test. What does this say about the own level? You passed a test. Others failed it. But you have been working hard, and you've been learning for this test, acquiring knowledge, and working on your language skills.

And now what would I answer in addition to: not good at all? I know that the material i work with is comprehensible to me. This is enough to say about my knowledge. This does not allow anyone to test my ability to use the language. Nor can anyone come and say you are beginner, you are intermediate, advanced.

The only way to measure my knowledge, and assess how far I come is my own understanding of such materials as books, media, chatroom talk. Can I understand what is going on 100% - no. Can i get the general gist of what's going on? It depends. Can I follow a longer storyline without looking things up? Definitely no! Do i think i have to say i understand 30% of something, or 100% of one thing? Of course not. Because I can understand something up to a 100% when I read it, listen to it, and know what is going on.

What I know is that a level does mean little to me, and it means little to others, how good or bad my level is at this very moment. Another thing that can not be assessed. You can be well prepared [for a test], you know, that you know, what you have to know. But you get nervous and fail. Another time, you learned nothing at all, but you passed the test with ease. The result of this test taken at that moment tells everyone that you are a winner or a looser, but it does not tell anything about your real abilities.

Sometimes it's bragging. I know more than you do, 20.000 sentences, 40.000 words, the word of our lord backwards, in Japanese, i have read 400 books, watched tons of movies so I know everything there is to know. At the end of the day it only matters how much you know, to be able to function in a language, no matter what you tell others about your abilities.

Agree 100%. I think the best way is to get a third party(different people). Plus tests which give a non-bias way of ranking. I know when I say 90% I refer to vocab,context,grammar and overal understanding of the sentence. Nowadays it will goes to a 100%. But what does that mean? Example: I watch a news show(those clips). It lists all the texts, I have no problem reading, expect for names/place names or rare vocab or vocab I don't know. So my understand is usual at 90%. But let' say I read classical Japanese, then my understanding will go down a lot. But I bet if you'd asked japanese people to do the same. They'd have trouble understanding it, this is basically what a language is: you never stop learning. You keep improving your skills until a high level of fluency. You will know, the people around you will know and lastly,you'll keep learning.