[ファブリス]

I posted some numbers yesterday, but I'll admit I'm not a mathematician at heart. I used a simple sum(pass)/sum(failed) to get an average pass rate for all flashcards, all users.

As I went to bed I realised I could split this by number of reviews, so we can see the pass rate over time :

Code:

+--------------+----------+-------------------------------------+
| totalreviews | count(*) | SUM(successcount)/SUM(totalreviews) |
+--------------+----------+-------------------------------------+
|            0 |   508099 |                                NULL |
|            1 |   229976 |                                0.49 |
|            2 |   115018 |                                0.80 |
|            3 |   112413 |                                0.84 |
|            4 |   141928 |                                0.89 |
|            5 |   158553 |                                0.90 |
|            6 |   130305 |                                0.89 |
|            7 |    90775 |                                0.87 |
|            8 |    47403 |                                0.80 |
|            9 |    36060 |                                0.79 |
|           10 |    28967 |                                0.79 |
|           11 |    22147 |                                0.78 |
|           12 |    17172 |                                0.78 |
|           13 |    11714 |                                0.75 |
|           14 |     9507 |                                0.76 |
|           15 |     7260 |                                0.74 |
|           16 |     5524 |                                0.74 |
|           17 |     4344 |                                0.73 |
|           18 |     3402 |                                0.72 |
|           19 |     2900 |                                0.74 |
+--------------+----------+-------------------------------------+

Keep in mind those are not successive positive reviews, just total number of reviews.

Rates drop down slowly after 5 reviews, but still at 74% after 19 reviews.

Interestingly there is a big drop of 7% between 7 and 8 reviews, this could be due to one of the intervals being too long.

7th revision is after a 120 day interval and 8th revision is after a 240 day interval, assuming the flashcard was never failed, and not using the "easy" answer, which would push the intervals even more.

Lately I've been pondering about that and I'm leaning towards using new intervals for the next area, it may result in more reviews, but I've found vocab and sentence reviews to be less taxing than the kanji reviews.

These are the base values, without the amount of variance added in :

0, 3, 7, 14, 30, 60, 120, 240 (i.e. a card going to the 2nd stack gets a 3 day interval)

This is the numbers I'm using for the vocab/sentence reviews :

From many reports on this forum I'm thinking people need more early reviews. So I've changed the values for the vocab/sentence area :

0, 1, 3, 10, 30, 60, 120, 180

- after the first review from the blue stack, get a review one day later
- get another review 3 days later as usual
- one week interval instead of 4 days
- the rest is similar...
- the last interval reduced to 6 months, instead of 8 months

I don't think we need that initial review. Looking at the 49%, it seems most just breeze through the initial review to get the card in the system. It probably waits in the missed stack until it's in line to be reviewed for real (ok, that's the way I was doing).

It might be justified to instead treat "added" cards like "missed" cards in that you click a button saying you've reviewed it to your satisfaction. Missed cards keep the "learned" button while added cards have a "reviewed" button. Clicking on the "learned" button brings up the next missed card while clicking on the "added" button brings up the next added card. This is important as you may only want to review so many missed cards before going on to reviewing added cards.

For example: I had built up almost 400 cards in my missed stack (I was adding a chapter a day, reviewing only 30 to 40 missed cards a day). Most of those were missed the first time up. Now I'm weeding through the missed stack, since I'm no longer adding cards. It's like I'm learning these Kanji for the first time, but I'm only clicking learned on those I reviewed, which get reviewed in 3 days at a 70 to 80 percent pass rate.

With that change, I think you can remove that review scheduled for the next day.

For the review spacing, Anki is cool in that you get it put over a range of days, and the system selects the day with the least amount of scheduled reviews. Pretty cool cause over time you're reviewing consistent number per day. Not sure if such a thing is easy to implement in RevTK.

Anyway, I like the chart. The 49% on the first review does kind of validate my theory and probably calls for some type of change.

The one day after review is a great idea! After puting cards in the blue stack, I would wait 1 day before adding them. But if you added the 1 day inerval it would make my initial reviews much easier.

[ファブリス]

Nukemarine may be right.

I forgot that I've put on my to-do list : a way to review new cards, or any lesson, without entering the SRS, or affecting the flashcard status. If I add this mode for users who need more early reviews, then the intervals don't need to be affected.

[ファブリス]

Ack, yes... 1 day is too short. I added 150 or so compounds yesterday and reviewing them again the next day is too early, at least for vocab/sentences. Besides, 1 day interval doesn't allow for variance.

I'm going to trial a 1 day variance to the first interval of 3 days though. Currently the site does not add variance to the first 3 day-review. With 1 day interval it could be 2 days, 3 days or 4 days. (this is to help spread reviews over days).

ファブリス Wrote:I'm going to trial a 1 day variance to the first interval of 3 days though. Currently the site does not add variance to the first 3 day-review. With 1 day interval it could be 2 days, 3 days or 4 days. (this is to help spread reviews over days).

I like the sound of that. Anything that spreads the reviews more evenly is a win in my book.

-- Daniel

The variant sounds good. If it works well with the 3 day review, perhaps it can be added to the other blocks as well.

For example

Block 2: 2-4 days
Block 3: 6-8 days
Block 4: 13-15 days
Block 5: 29-31 days

Etc.

Now, is it random or will the system put the card on a day with the least number of cards scheduled? In my opinion, its better to go for scheduling cards on days with least scheduled for review.

[ファブリス]

There is a variance added to the intervals, ever since the last update I did on the Leitner system, when I increased cardboxes to eight, and added the "Easy" answer. But back then I didn't update everybody's cards with the variance (a mistake maybe), so it took a while to see any effect on the existing flashcards.

I was also maybe a little too conservative with the variance :

Intervals: 3, 7, 14, 30, 60, 120, 240
Variance: 0, 2, 3, 5, 10, 15, 30

So a card scheduled for review in 14 days could expire anywhere from 11 days to 17 days. That's how it works now.

[ファブリス]

Oh... I just came accross this, testing on the pass rate of cards only tested once for the period 2007-10-16 > 2007-11-16 :

Code:

+--------+--------+--------------+----------+
| Failed | Passed | TotalReviews | PassRate |
+--------+--------+--------------+----------+
|  17274 |  11631 |        28905 |     0.40 |
+--------+--------+--------------+----------+

Then the period 2007-11-16 to today 2007-12-16 :

Code:

+--------+--------+--------------+----------+
| Failed | Passed | TotalReviews | PassRate |
+--------+--------+--------------+----------+
|  11008 |  17509 |        28517 |     0.61 |
+--------+--------+--------------+----------+

EDIT: Ahhhh number get complicated fast, lol. Ok so to summarize I think it goes like this : if all failed cards were cleared by everyone, the global pass rate for cards tested just once would be closer to that 61%. But many users also leave cards tested once and failed in the red stack, while other users clear them. If users never cleared their failed cards, the global pass rate for cards tested once would keep getting lower and lower. Instead many users clear the failed cards and so each month, new cards pile up in the red stack that were never tested again, while other cards were cleared (and thus tested more than once)... hence the 1st time pass rate is closer to 61%, but declines over time to ~50% for the last two months, and then remains between 45%-50% if we go back further in time (always with cards tested just once). In other words the pass rate reduces due to red cards piling up, but not so much as 40% due to users also clearing their cards over time

EDIT EDIT: so... to get the actual pass rate for 1st time review I have to look at results only for today : 84%

2007-12-16 pass rate :

Code:

+--------+--------+--------------+----------+
| Failed | Passed | TotalReviews | PassRate |
+--------+--------+--------------+----------+
|    169 |    867 |         1036 |     0.84 |
+--------+--------+--------------+----------+

Going back one week, red cards not cleared start adding up and lowering the pass rate to 74% :

2007-12-09 > 2007-12-16 pass rate :

Code:

+--------+--------+--------------+----------+
| Failed | Passed | TotalReviews | PassRate |
+--------+--------+--------------+----------+
|   3067 |   8888 |        11955 |     0.74 |
+--------+--------+--------------+----------+

So hopefully the conclusion is the actuall 1st review passrate is well over 80%.

Quite surprising actually, nice going!