I've been thinking about this recently.

My Core6k deck has about 1300 cards matured at the moment. I figure a good chunk of those are loan words but I still have some that are compound words. My fail rate on mature cards is 90% +/-1%.

I believe I remember seeing that Damien mentioned that 10% is a pretty decent number, for failing mature cards, but why do we fail/forget them though. Given that the SM2 algorithm is pretty robust I would think the rate would be somewhere closer to 5% or even less. However there is the human factor in this. Do we tend to fail these cards because we grade our self wrong when they come around again, and end up pushing the interval back further than we should resulting in us forgetting it?

Thoughts?

It sounds like you're not encountering these words enough in real life.

You won't encounter many of the Core2k+ words unless you are reading business or political articles. There are a ton of words that are just kind of unused. However this is suppose to be the strength of Anki. It lets us keep the words we may not see too often, by reviewing them before we forget them.

vix86 Wrote:You won't encounter many of the Core2k+ words unless you are reading business or political articles. There are a ton of words that are just kind of unused. However this is suppose to be the strength of Anki. It lets us keep the words we may not see too often, by reviewing them before we forget them.

You're assuming all cards are the same difficulty. Some cards are just more difficult than others. Anki tries to adjust for this by letting you tell it which cards are difficult, but it's not perfect. When you think about it, 90% retention with these kinds of algorithms is pretty stunning.

This site has a similar system of failed/normal/hard, but it's not as granular. The thing about the cards you're failing is that the algorithm is having trouble scheduling them based on your input. They aren't necessarily harder because if they truly were obviously harder then you would have marked them as such. Maybe at the time you hit the button marked easy it felt easy, but it turned out to be normal. Or in Anki maybe you hit normal, but in actuality it was hard. Maybe your finger slipped.

Your criticism also assumes that your brain is the same every single day, and that's just not the case. There's some days where my brain is just better at Japanese. There's other days where I'm drunk or sick that things don't come as easily. And yet on the rarest of days I have studied for 8 hours straight, and then forgotten how to read "今".

So once again, taking all these things into account, I'd say 90% retention is damn good. The best thing about that statistic you have is that it's provable. You have the records of your reviews. Those 10%, you know you don't know them well enough. In a brute force/rote learning scenario you'd never realize in the first place that those cards were harder or you didn't know them. You might suspect something like that was happening, but without any way of knowing what you really know.

Any SMx algorithm is based on probability and 10% Forgetting Index is what is a default, I believe, in Anki. This 10% means that on any day your success rate should be around 90% regardless whether the card is "mature" or "not mature" (BTW, can somebody explain to me what's the meaning of "mature" in Anki? I'm using SuperMemo so I'm not familiar with the term.) If you want a higher success rate then you need to decrease the FI (which will result in a higher burden, of course) - again, not sure how you do it in Anki.

Inny Jan Wrote:BTW, can somebody explain to me what's the meaning of "mature" in Anki? I'm using SuperMemo so I'm not familiar with the term.

It simply means that it's been more than a month since your first review of the card.

Isbilenper Wrote:

Inny Jan Wrote:BTW, can somebody explain to me what's the meaning of "mature" in Anki? I'm using SuperMemo so I'm not familiar with the term.

It simply means that it's been more than a month since your first review of the card.

Actually it's a card whose interval is over 21 days (which, at default settings, should take around 1 month to achieve if no fails occur).

i always thought that the mature interval was just arbitrarily defined as over 30 days; there is no reason why you can't forget them.

on top of that, the intervals themselves are based on personal judgment which can be flawed, and the algorithm is only an approximation of what actually happens in our brain. i'm not sure our brain works on a resolution of 3 judgments only.

just a bunch of random hypotheses...

Has any work been done around extending the forgetting index?

I've always wondered if a 90 percent retention rate was in fact sub-optimal as it leads to a lot of unnecessary reviews. I also recall reading somewhere that maximum learning occurs when something is somewhat difficult to recall. Although a 90 percent pass rate feels satisfying, I am wondering if I wouldn't be better off aiming for 75 percent; this would make cards more difficult to recall BUT I would remember them more quickly if this theory is correct.

Obviously this depends on what exactly one is learning and why one is learning it. If it is essential to recall everything than a 90 percent pass rate, or better, should be the aim, but if I want to quickly amass a large amount of material then maybe a higher forgetting index would be more appropriate.

The specific value of FI=90% is not what is built in into the SM algorithm - it's is an arbitrary value that reflects your personal preference for retention of the acquired material. If you can change it to, let's say, 75% - it's your pick. The calculated intervals will be longer than in case of FI=90%, which:
1) will allow for adding more items to the review process; and
2) will lead to lower success rate during the reviews.

I guess you would need to experiment yourself with different values of FI to find the optimal one.

The other thing is that, in my experience at least, recollection is not something that as simple as: "I remember" vs. "I don't remember". It seems that recollection is somehow related to the time your brain needs to formulate the answer for the stimuli. You say "I don't remember/hard" because in the time you spend on trying to recall nothing has happened - hence you mark it as a failure. But I do notice that if you allow longer time for the recollection then something that is not instantly/in short time recalled eventually is recalled - it's only that it takes a long time to happen. Long enough for me to mark the item/card "failed".

well I too would like to extend the forgetting index... problem is there is no way to modify the way anki's buttons affect a given card's interval. someone would need to write a plug-in to effect this sort of change, and if you can believe it one does not exist yet...!

where's cb4960 when you need him??

It might be possible to write a plugin to do what you want but it might be hard. The problem is that the code for setting the interval after you answer a card, is in the libanki part of Anki. Most plugins modify the GUI in some fashion which is separate. I believe I've heard Damien mention on a few different occasions that you should avoid trying to directly modify the functioning of the library/interval handling stuff.

What did you have in mind for modifying the buttons though. You couldn't just set the buttons to some specified amount because the time scales by a factor over time. Do you mean to change the factor by a certain percent for each button?

Inny Jan Wrote:But I do notice that if you allow longer time for the recollection then something that is not instantly/in short time recalled eventually is recalled - it's only that it takes a long time to happen. Long enough for me to mark the item/card "failed".

I used to fail these too, but now I pass them. Sometimes I can practically feel the neurons growing with the strain of recall. The interesting thing is that I still have an approx. 92 percent pass rate and I get instant recall on many of the kanji I previously struggled with. All this makes me wonder what the effects of bumping up the FI would be. It would be interesting to take two groups of students, set different FI for each group, and run a little experiment over a semester or two.

If I recognize a card immediately then I mark it easy and try to make it go away.

Disclaimer: This is only for the RevTK site. Sad to say, a friend actually lent me a computer so I could start using it for Anki but I haven't got round to setting it up 怠け者だネ

Long unreviewed kanji are just starting to emerge from box 6 so I may be eating my words, if not my characters, in a month's time.

Edit: Here is the quote I was trying to find:

Quote:SuperMemo...(SRS)... is based on the insight that there is an ideal moment to practice what you've learned. Practice too soon and you waste your time. Practice too late and you've forgotten the material and have to relearn it. The right time to practice is just at the moment you're about to forget.

Want to Remember everything You'll Ever Learn: Surrender to the Algorithm

A hilarious article about the founder of SuperMemo from Wired magazine.

vix86 Wrote:It might be possible to write a plugin to do what you want but it might be hard. The problem is that the code for setting the interval after you answer a card, is in the libanki part of Anki. Most plugins modify the GUI in some fashion which is separate. I believe I've heard Damien mention on a few different occasions that you should avoid trying to directly modify the functioning of the library/interval handling stuff.

What did you have in mind for modifying the buttons though. You couldn't just set the buttons to some specified amount because the time scales by a factor over time. Do you mean to change the factor by a certain percent for each button?

Well I would specifically like to turn the number 2 'hard' button into a soft fail, rather than how gradually increases the interval now. At the moment there are no good options if you fail a card just slightly... other than starting the whole review cycle over again.

I kinda got halfway by basically nerfing my fail button... I set it to a 10% button 1 multiplier, which means that failing a card takes its interval down to 90% of what it was, rather than zero. When you see it again you have options for (if there were a 1 year interval) approx 1.5 months for the hard button, through 3 months or so for the easy. This kinda accomplishes the task (unless I totally blank on an old card, in which case I miss my old fail button) but I don't want to be thinking that hard about how to grade cards... I just want to think about the japanese and review naturally...

The thing that I'm seeing in the code though is that 1 is obvious fail and 3 & 4 are passes based on a number calculated by factoring in prior eases (I think). A 2 though is a hardset constant though:

Code:

if ease == 2:
    interval = (card.ivl + delay/4) * 1.2

with card.ivl being the card's old interval.

So basically an ease of 2 is always going to increase at some constant rate compared to an ease of 3 or 4. If you want a soft fail, then it should be possible (once I figure out how) to simply rewrite this to set it to some arbitrary interval like 18 hours later; every time you fail answer 2.

Ya a constant multiplier of the current interval would be ideal...

for example, if a soft fail is desired... the result would be the current interval * .65 (or thereabouts)

ideally in the plugin one would be able to modify the multiplier to whatever you want... to try out different values and see what works best.

That would be an amazing gift to the community Vix if you can do it...

Correct me if I'm wrong, but if your fail ratio for Mature cards is 0%, that probably isn't a good thing. That probably means your grading yourself too easy and spending more time on the cards then you need to be.