+- kanji koohii FORUM (http://forum.koohii.com)
+-- Forum: Learning Japanese (http://forum.koohii.com/forum-4.html)
+--- Forum: Learning resources (http://forum.koohii.com/forum-9.html)
+--- Thread: Literally grow your vocabulary (/thread-8019.html)
Literally grow your vocabulary - wccrawford - 2011-06-25
http://www.memrise.com/welcome/
The concept is that each word is a seed you plant and tend to, and you watch it grow.
Sounds really corny, but I thought I'd try it and see. Japanese (and many others) are in Beta right now, which apparently means they don't have official word lists... So they're kinda sketchy.
Edit: Haha, well, it appears to be really glitchy. YMMV.
Literally grow your vocabulary - Splatted - 2011-06-25
Yep, crashed firefox.
I tried a bit of the Mandarin one; I can only assume that it was incomplete because there was absolutely no sound and no attempt at phonetic writing, but the picture and mnemonic were quite good.
Literally grow your vocabulary - shaggadelyc - 2011-06-25
same here, got shown 2 cards to study, then asked me to write a third unknown one which i obviously didnt know and then decided to crash...
but on the other hand, it looks promising, nice interface and the mnemonics looked very helpful.
smart.fm 2?
Literally grow your vocabulary - nest0r - 2011-06-25
Looking at the Science section and browsing through Google Scholar, this Detre fellow seems like the real deal (I think I already have a paper or two by them somewhere), their research seems focused on the computational simulations as opposed to behavioural tests; between that and the nature of these sorts of startups, I have some fear about how much they'll try and emphasize the superiority blah blah of their particular propietary algorithms to magically spoonfeed the users and such, at a time when research by Karpicke, et al., is showing the precise scheduling, including whether the intervals expand, isn't as important as simply spacing, period, (Edit: And the mechanism of retrieval) with adaptiveness guided by learners' metacognitive awareness.
(On the expanding thing, regardless of whether it's intrinsically superior is now in question or not [I was reading a recent response by Bjork that seemed to acknowledge it's less important than they thought, but still important in many ways], I still think it's superior in that it's a natural product of strengthening memories over lengthy retention intervals and also in terms of efficiency, i.e. you can decrease study time without loss of superiority to non-spaced scheduling, and without falling behind other forms of spaced retrieval.)
My impression is that the general idea remains that the inter-session intervals naturally increase as the RI increases, also, even if they haven't pinned down why (so I think the statement of reviewing just before forgetting might be premature or oversimplified), so there's no reason not to expand. Plus their definition of expanding seems to require an immediate retrieval rather than a delay, when with Spaced Retrieval Systems there's usually a considerable post-study retrieval delay.
The other stuff reads like Medina's Brain Rules, and is also solid.
I'm really not a fan of gamification as a web trend, either, but that doesn't mean it's all bad...
Literally grow your vocabulary - iSoron - 2011-06-25
shaggadelyc Wrote:same here, got shown 2 cards to study, then asked me to write a third unknown one which i obviously didnt know and then decided to crash...Exactly the same here.
@nest0r tl;dr
Literally grow your vocabulary - nest0r - 2011-06-25
iSoron Wrote:heheshaggadelyc Wrote:same here, got shown 2 cards to study, then asked me to write a third unknown one which i obviously didnt know and then decided to crash...Exactly the same here.
@nest0r tl;dr
Short version (I'm experimenting with these, so bear with me):
The science of that site, based on my experience of having read very many of the papers on spaced retrieval and memorization, looks solid, but I fear they might end up too focused on ‘the algorithm’ as a kind of gimmick, when recent research has begun to suggest that the scheduling isn't as important as spacing, period; the expanding aspect might just be more of a side-effect (but a very preferable and efficient one).
Finding the sweet spot, the desirable difficulty, is about spacing, sure, but also about a bevy of other factors that connect to it, such as the user's feedback and self-study skills, and the card design (if we're talking about cards).
I feel like that awareness of retrieval and metacognition tends to get lost or backgrounded when talking about SRS-type stuff.
___
I was just reading this about adjusting cue informativeness as an alternative for modulating difficulty to the scheduling focus; seems interesting: http://0-www.eric.ed.gov.novacat.nova.edu/ERICWebPortal/search/recordDetails.jsp?ERICExtSearch_SearchValue_0=EJ539241&ERICExtSearch_SearchType_0=no&_pageLabel=RecordDetails&accno=EJ920806&_nfls=false - I've been toying with this idea myself, thinking about weakening cues over time to offset the decreasing i+N-ness, and also strengthening cues but multimodally so it strengthens the memory peripherally (and making it more natural to authentic language usage) without compromising the retrieval benefits, by being more precise with cue/target relations... don't mind me, still experimenting.
They referenced this model as being good, schedule-wise: http://www.ncbi.nlm.nih.gov/pubmed/18590367
Here's the type of stuff that downplays spacing scheduls that I'm referring to:
http://www.ncbi.nlm.nih.gov/pubmed/21574747
http://www.ncbi.nlm.nih.gov/pubmed/19966244
And here's a response from Bjork (sort of the godfather of spaced retrieval, as I understand it): http://www.ncbi.nlm.nih.gov/pubmed/20173196
Literally grow your vocabulary - vosmiura - 2011-06-25
nest0r, so expanding intervals are not the key to memorization - but rather expanding intervals are a side effect of memorization we can take advantage of to reduce review times and learn more stuff?
Literally grow your vocabulary - Glacial - 2011-06-25
From the abstracts, it sounds like increasing intervals are important - but the most important thing is that the intervals are increasing at all, not by the degree to which they increase. I'm not a huge devotee of Khatz, but to me this sort of sounds like his "just showing up" principle.
Literally grow your vocabulary - nest0r - 2011-06-25
vosmiura Wrote:nest0r, so expanding intervals are not the key to memorization - but rather expanding intervals are a side effect of memorization we can take advantage of to reduce review times and learn more stuff?That's my impression, but I'm still working this stuff out, since I only discovered the studies questioning the expanded thing a few months ago (Edit: Actually nevermind, I saw it last year but ironically dismissed it while agreeing with them: “As for expanding vs. equal, I think neither of those is as important as 'spaced', just as I try not to focus on the algorithms--I think it should ultimately be a spacing that's semi-automated in accordance with the user's self-feedback, and the main reason for expanding the spacing is simply efficiency, maintaining recall with the least amount of review over time.”).
All this time I thought it was just me (thinking that the algorithmic fine tuning of the schedule wasn't as important as the user feedback that let you push reviews away as they got easier [in conjunction with something that was simply algorithmic]).
Here's an interesting bit from Karpicke, et al.:
“The theory about why expanding retrieval should be su-
perior to equally spaced retrieval practice is twofold. First,
an early initial retrieval attempt increases the likelihood
of success before forgetting has a chance to occur. Sec-
ond, gradually increasing the interval between repeated
retrieval attempts increases retrieval difficulty on those at-
tempts and thereby improves later retention (Bjork, 1999).
However, at least under our conditions, it is not clear that
retrieval actually grows increasingly difficult across re-
peated tests regardless of whether the tests are expanding
or equally spaced. Assuming that recall latency is a viable
index of retrieval difficulty (Benjamin, Bjork, & Schwartz,
1998; Koriat & Ma’ayan, 2005), in prior research we found
that recall times grew faster across repeated tests, indicat-
ing that repeated retrieval grew easier, not more difficult
(Karpicke & Roediger, 2007a). If retrieval grew more dif-
ficult, we might also expect recall performances to decline
across tests, but the present data show that recall remained
relatively constant or even exhibited a modest degree of
hypermnesia (see too Logan & Balota, 2008). Therefore
it may be true that increasing the difficulty of retrieval at-
tempts does in fact promote long-term retention (Pyc &
Rawson, 2009), but the data imply that current procedures
for implementing expanding retrieval do not successfully
induce increasingly difficult retrieval across repeated tests.”
I never experienced cards getting harder with increasing intervals, only easier, so it does seem wrong if the logic is that increasing intervals makes them more difficult, because they ought to feel the same as the retrieval makes the memory stronger. And when you grade Hard, it should be to increase the interval rather than shorten it. ^_^ Instead it's more like you're telling Anki that it was easy and thus it's okay to spread it out further so you can focus on other stuff.
Oh and to add a bit, I think for Karpicke and Roediger, the desirable difficulty through retrieval practice is the priority, and the spacing comes into play by ensuring that when retrieval occurs, they're not recalling from immediate memory. It's enough for them or through their research results that you generically maintain ‘a delay’ to increase difficulty. And to apply that to SRS, the longer the period of time you're spacing, and the more items you're spacing, naturally results in lengthier intervals that generically need ‘an algorithm’ to manage in conjunction with grading.
Here's another bit:
“But what about the mechanism for spacing of retrieval? Our
data reviewed above suggest that the critical ingredient is encouraging fairly
difficult retrieval, especially on an initial test. Beyond that point, it probably does
not matter whether students test themselves using expanding or equal interval
conditions. What matters is repeated spaced retrieval (with feedback if an error is
made).” Edit: They suggest two days there for initial test delay.
(Not just retrieval errors, though: http://www.ncbi.nlm.nih.gov/pubmed/18605878)
Apparently that idea of modulating cues and even shifting the modality was tested to good effect in the 80s and called multiplexing. Interesting.
Another idea of delayed feedback for desirable difficulties is interesting; I've been tinkering with small delays in feedback both for subvocalization/listening, but also when handwriting new words without consulting the screen in tiny moments (seconds? milliseconds?) between initially writing+consulting, and listening/speaking/encoding meaning. Simple but effective. (Yes, now I'm just riffing.)
Edit: Oops, I didn't notice this the first time I read the paper, but they found in a later study that retrieval difficulty did increase with expanding intervals, but it didn't result in superior recall, and “If anything, patterns of increasing retrieval ease, not difficulty, tended to be associated with greater levels of final recall.” The latter for the equal/contracting schedules. So natural and ideal subjective preference is, perhaps, for easier and easier cards, not cards which always feel hard. ^_^ Initial difficulty allowed to develop through spaced learning into strongly internalized knowledge. In that sense, rather than ‘just before forgetting’, i.e. maximal distance between retrieval and subsequent retrieval, any point beyond a substantive delay is fine for increasing performance, and the expanded aspect arises from the increasingly strong memories that allows for efficiency as inter-session intervals increase with very long retention intervals. Maximal distance through highly calibrated mathematics thus not optimizing recall. If so, perhaps it's better for the user to determine maximum approximations based on post-retrieval evaluations of expected difficulty, and the SRS to ensure there's an algorithmically increasing baseline delay that's not dropped through.
Literally grow your vocabulary - nadiatims - 2011-06-25
I see no compelling reason to use this site. About the only good thing I can see is the fact they have audio on the cards.
While I agree that reviewing over long intervals helps retention, I no longer consider it necessary or desirable to force every item you want to learn through this 1 minute, 1 hour, 1 day, 1 week, 1 month (or whatever) gauntlet or spend a lot of time on mnemonics or other elaborate encoding.
Such finely tuned closely spaced review is not necessary for encoding to the longterm memory. I no longer agree with this notion that an item needs to be firmly held onto and kept in mind from first contact through to it's eventual storage in the long term memory. At best it's unnecessary work (four or so redundant reviews), and at worst it's actually undermining the the first semi-longterm review. Rather just review an item over longer intervals a few times until you judge it learned, and the word will stick.
A rather telling point is that only one of the five member team is actually reported to have learned a second language. The CEO is apparently some kind of memory guru who can memorise a pack of shuffled cards in 45 seconds. That's fantastic and all, but seems to be more a test of short term/working memory and learning ability than long-term memory. I'd be more interested in whether he can still remember the order of that deck and 10 others after a spacing of one month and how efficient his methods are in the longterm. I imagine not very because forcing yourself to remember something in the short term is always more work that just allowing yourself to remember it in the longterm.
Then there's the fact that this service doesn't seem to let you make your own decks, it's still very buggy and they're obviously going to try and monetise it at some point. I predict that the site won't get enough repeat visitors to get good advertising revenue and will switch to paid memberships soon enough.
Also I dislike how it uses multiple choice and forces you to click the answer within a time limit.
All in all a very poor service imo.
Literally grow your vocabulary - vosmiura - 2011-06-26
nadiatims Wrote:At best it's unnecessary work (four or so redundant reviews), and at worst it's actually undermining the the first semi-longterm review. Rather just review an item over longer intervals a few times until you judge it learned, and the word will stick.How many times do you usually fail your long term reviews before a new word sticks?
Literally grow your vocabulary - nadiatims - 2011-06-26
@vosmiura
depends on the word. The way I do this is:
I write out a list of 30 or so words on paper, which I then ignore for a decent chunk of time (2 weeks or more). When I check my comprehension after that time (first review after initial learning), I find I can usually correctly remember about 10 to 50% of the words, usually averaging about 30%. Then I rewrite the words I forgot and rinse and repeat. So the list gets shorter and shorter. I've usually learned them all after 3 or 4 reviews (about 2 months).
Literally grow your vocabulary - gregdetre - 2011-06-26
Hey gang,
I'm Greg Detre, the CTO and co-founder of Memrise.
I've really enjoyed reading this discussion thread. There are a couple of references here that are new to me (thank you!).
I also want to apologize for how the site has performed for you. We were lucky enough to get hit by a triple whammy of good press on the same day, and I'm afraid we didn't stand up to the traffic as well as we should have. Hopefully, we'll have things shipshape very soon.
I'm going to do some reading of the articles here now! But if any of you have any questions, I'll try and remember to check back here, or feel free to email me at greg@memrise.com.
Yours,
Greg
Literally grow your vocabulary - nest0r - 2011-06-26
Good luck with your site. I'm not into such sites myself as a learner, but I'll be interested in the kinds of choices you make for exercises and suchlike (and how it might related to inhibition/activation/resonance or retrieval-induced-facilitation/forgetting, &c.), i.e. inter-item relatedness, design of cues and targets and whathaveyou.
Here's a study that suggests you should add negative pictures to your site. ^_^
Negative Emotional Arousal Following Retrieval Enhances Later Recall
For the activation/resonance and retrieval-induced-facilitation thing (as a kind of mirror to inhibition and retrieval-induced-forgetting), I was thinking of MacWhinney and Chan, respectively:
The Unified Model
Long-term effects of testing on the recall of nontested materials
Literally grow your vocabulary - jettyke - 2011-06-26
@nest0r
Do I have to pay to read those ( most of your linked) studies?
Literally grow your vocabulary - nest0r - 2011-06-26
I would say 60-80% of the links I post can be found in .pdf format via Google; try Googling the titles and skimming for the Quick View links. The .edu publications pages of the scholars also often contain an overwhelming number of .pdf linked studies. I've begun alternating the amount of direct links to .pdfs I post, as I imagine some prefer to read the abstracts before downloading the full papers, and also I think Zotero usage has gotten more prevalent, and it's preferable to process the information and save stuff through Zotero (though occasionally I have to find the .pdf from another link and attach to the Zotero entry).
Literally grow your vocabulary - gregdetre - 2011-06-26
Hey Nest0r,
I just re-read your posts in detail.
Frustratingly, I'm on a bus so i can't easily download the pdfs. Reading papers from an applied perspective nowadays, I pay much more attention to the effect size. I spent years on behavioral, imaging and modeling work around phenomena like retrieval-induced forgetting, and I believe these are real, interesting and important effects, but they're noisy and relatively small in the grand scheme of things (relative to, say mnemonics, testing and repetitions). Plus, we have to think about the overall experience - so it might be a little while before start flashing negative images after each learning event
)I take your point about chest-beating over 'proprietary algorithms'. Algorithms do matter, and I obsess pretty hard over them behind the scenes. But we believe that our community, our content and the experience are more important, so we spend a huge amount of energy on those aspects. [It looks like we need to add scalability to that list!]
I'm still in close contact with a lot of my colleagues from Princeton and elsewhere, and it's my dream to work with them over the long-term to make sure that the very best scientific ideas filter into the educational mainstream. Keep in touch by email (greg@memrise.com) if you spot anything else that you think we should be incorporating, or if you'd like to maybe get involved with in some way,
Thanks for your thoughts,
Yours,
Greg
Literally grow your vocabulary - gregdetre - 2011-06-26
Oh, and P.S. we have experimented with hints as a means of adaptively altering test difficulty. This was a big part of my PhD, but it's a tricky, tricky business to get right. I'd welcome ideas.
Literally grow your vocabulary - nest0r - 2011-06-26
Mnemonics, testing, and repetitions is the perfect trifecta, so I think you're set! Hopefully your site ends up contributing to a more comprehensive, large scale understanding of spaced retrieval processes and relational/explicitly enhanced vocabulary acquisition.
Literally grow your vocabulary - nest0r - 2011-06-26
@vos
By the way, I'm starting to come around on the microspacing thing, combined with mnemonics (and of course my usual encoding strategies, but those are given, I just mention the mnemonics despite it also being given due to the thread context and recent discourses on relational strategies, and the similar suggestion by Roediger, et al., below). Because I'm beginning to think there's no such thing as microspacing, only overlearning (studying beyond reaching a criterial level of a successful recall of all items from the session, overlearning being studying immediately beyond that without spacing subsequent retrievals). (By this definition, which I think fits the notion as we've been using it over the years: http://thesciencenetwork.org/docs/BrainsRUs/Increasing%20Retention_Pashler.pdf - Note also the coarse variability of the optimal ISI there.)
Rather than trying to translate Karpicke & Roediger's spacing scale of seconds/minutes/hours that they use to evaluate the long-term memory retention after a day or week, rather than expanding that to an infinite retention interval wholesale, I will experiment with actually using the scales they use in Anki more gradually. I'm trying to condense and extract such a schedule from a couple of their papers at the moment. (Edit: The Absolute vs. Relative paper [long spacing experiment which resulted in 200% increase in long-term retention of a week] and this paper's Practical Advice section being the sources I'm extrapolating from.)
The result ought to be something akin to, for example: a study session using mnemonics of ~20-25 words, followed by, a few minutes later, retrieval practice (Question→Answer in Anki), failing cards to be reviewed right at cycle's end (when all new cards have been passed/failed), with the criterion to begin spacing being one successful recall (thus it's removed from the study/restudy/test cycle - Though technically the spacing begins with that few minutes between study and first test). Successful recall results in 2-3 more spaced repetitions that day, I haven't decided on the scale there, but the maximum final spacing before a substantive delay would be 4-6 hours. Then that delay after the first day's study + initial spaced repetitions would be a wait of ~24-48 hours to do the third or fourth repetitions, then a week after that, etc., and I'll take the ‘good enough’ expanded approach from there.
Edit: Hmm, still working it out. The most interesting thing for me is, my usual process is to study it then grade it hard without retrieval, so it comes up in .75-.99 days for the actual first retrieval. Instead, actually performing 3 successful recalls spaced by trials occurring in minutes or even seconds during the initial learning phase is superior for long-term retention as evaluated 10 minutes, 2 days, or even a week later, if the experiments since 1978 are any indication. So I wonder if implementing that initial phase with a time scale intended to play out over weeks and months and years will smooth things out.
I'll probably just end up studying and retrieving new cards a total of 3 times on the first day, with no real ideal spacing, it seems seconds to hours doesn't matter much. Thus I may simply fail them to avoid my initial button intervals, then reset them and proceed as normal (though I may select an initial interval of 36-48 hours rather than the usual 24).
If I find success with the initial learning phase this way, perhaps I'll be interested in spacing by trial (~10 seconds max duration per trial) on the first day rather than seconds/minutes/hours.
Which reminds me that I need to play with n-backing more.
Edit: Now I'm veering towards 2 retrievals spread over hours the initial day, since it re-occurs to me we're factoring in mnemonic strategizing and multisensory integration and such also, rather than 8 second word pair study trials... I do like the 14 second limit on test trials, though.