Ok, this is a bit ambiguous, but a few weeks ago, I thought I read on here somewhere that either someone had made some software that could scan some text and determine the difficulty of it, or they were just tossing the idea of such a piece of software out there. It was supposed to look at the frequency of kanji, or something along those lines.

My main question is, does this software exist?

http://forum.koohii.com/showthread.php?tid=7072

While counting Kanji is 1 way of indicating difficulty, actually rating the kanji and vocabulary (and grammar?) used would be better.

wccrawford Wrote:While counting Kanji is 1 way of indicating difficulty, actually rating the kanji and vocabulary (and grammar?) used would be better.

cb's word frequency generator:
http://forum.koohii.com/showthread.php?p...2#pid69302　

While it's not necessarily a "difficulty rater," it'll find the most common vocab and you'll be able to kind of self-gauge whether or not you're ready for a given text

Oh, thanks, just what I wanted

Though it would be cool if something had a counter similar to anki's kanji counter, which showed how many kanji were from each grade level.

Zarxrax, are you familiar with this? http://language.tiu.ac.jp/tools_e.html

All this time when I talked about incorporating level-checking tools, I kept thinking of JLPT but it was actually that above link that was my mental model, from when Nukemarine posted it back when cb4960 first gave us subs2srs. I kept thinking how great it'd be to have this kind of thing integrated with our Anki cards.

nest0r Wrote:Zarxrax, are you familiar with this? http://language.tiu.ac.jp/tools_e.html

All this time when I talked about incorporating level-checking tools, I kept thinking of JLPT but it was actually that above link that was my mental model, from when Nukemarine posted it back when cb4960 first gave us subs2srs. I kept thinking how great it'd be to have this kind of thing integrated with our Anki cards.

Do you think this gives an accurate enough answer?
As you can only use a part of that text I wonder whether it will be enough...it's expected that it is...but you never know.

Actually this 少し難しいand 難しいdoesn't give an accurate enough answer...like what can I compare those levels to?
If the rating is this vague then I can probably tell the difficulty much better just by reading the first page by myself...although it's slower


Edit: I actually checked the first 3 pages...it showed ふつう then I checked the first 6 pages and it shows 少し難しい。。。so I guess it isn't just well enough
EDIT2: Lol, pretty much any book that I have from your smart thing is either 少し難しいor難しい！
A 1300 kanji novel and a 2300 kanji novel are both rated 難しい....it sucks

jwpce seems like it might be the best solution for me, for now.
Ratio of unique kanji to total characters in the text seems like a fairly good measure of a text's difficulty level.
But it justs gives me lots of ???? marks when I try to paste in certain texts.

Zarxrax Wrote:jwpce seems like it might be the best solution for me, for now.
Ratio of unique kanji to total characters in the text seems like a fairly good measure of a text's difficulty level.
But it justs gives me lots of ???? marks when I try to paste in certain texts.

Copy the text to paint and save as UTF-8 I think. then open the file. Try!

jettyke Wrote:

nest0r Wrote:Zarxrax, are you familiar with this? http://language.tiu.ac.jp/tools_e.html

All this time when I talked about incorporating level-checking tools, I kept thinking of JLPT but it was actually that above link that was my mental model, from when Nukemarine posted it back when cb4960 first gave us subs2srs. I kept thinking how great it'd be to have this kind of thing integrated with our Anki cards.

Do you think this gives an accurate enough answer?
As you can only use a part of that text I wonder whether it will be enough...it's expected that it is...but you never know.

Actually this 少し難しいand 難しいdoesn't give an accurate enough answer...like what can I compare those levels to?
If the rating is this vague then I can probably tell the difficulty much better just by reading the first page by myself...although it's slower


Edit: I actually checked the first 3 pages...it showed ふつう then I checked the first 6 pages and it shows 少し難しい。。。so I guess it isn't just well enough
EDIT2: Lol, pretty much any book that I have from your smart thing is either 少し難しいor難しい！
A 1300 kanji novel and a 2300 kanji novel are both rated 難しい....it sucks

I would imagine people focused on the JLPT breakdown in some way. Total words from which levels, etc.

But I don't really care about that, in case you were trying to persuade me or something. That site's from years ago. It also uses Chasen. I don't know much about that but I think Mecab is better? At any rate, I thought I was clear that I was only conveying an idea, a model. Hence my post in the cbJisho thread.

Personally I think that most texts are sufficiently consistent that you can make a reasonable estimate by just reading the first paragraph or two and judging from that. I guess if you had a huge stack of books you might want to sort them automatically by difficulty, but wouldn't it be more fun to read them in order of how interesting the contents were instead? :-)