Back

Mighty Morphin Morphology

(2017-03-06, 6:42 am)kevsestrella Wrote:  
Thanks for the info about CJK Morphemizer Kaegi, I haven't read previous post, I will now.

To add onto this, the behavior also works a bit like this:
Ex. You learn how to write the 池 kanji in RTK, but you haven't learned the word 池[いけ] meaning pond. You have a large vocab deck, maybe the Core10k or some vocab you saved from your fun sources of content.
If you're currently learning how to write 池 in your Kanji deck and you have a morphemizer turned on for it, it will increase the priority of the 池 word.
If you have 池 in your Kanji deck as Mature and it's being tracked with the morphemizer, the card will have no New Morphemes, and so the priority will be at the highest, assuming you keep these options: http://i.imgur.com/H5EHhvs.png

I having kanji being tracked is probably a good thing, but I haven't re-enabled it.
I basically can read words like 旅行[りょこう] but don't know what the 旅 kanji is or how to write it, but MorphMan will show the word 旅[たび] at a (much) higher priority because I know 旅行. It sorts the words with no new morphemes first.

tldr; I think having it track kanji is good, because it'll put all of the kanji that exist as their own word and put them to the front. A short-term issue is that MorphMan will be assuming you know all of the words that are one kanji-only, ex. 話, 電, 気、意, etc., but if you have a core10k deck or something, it'll give help you fill in those gaps so that you actually do know those single-kanji words. For example, if you're a beginner and you learned 話す and learned how to write 話 in RTK, but didn't learn the word 話, you may find a sentence in sub2SRS sentence with the word 話. Maybe the sentence will be 「いい話だ。」 and will have no new words. It'll be listed as a high priority sentence, since it won't be highlighted as Unknown (or in Red color if you set up colors for unknown). Since 話 won't contain no new morphemes, though, so you'll see them earlier with MorphMan and fix your knowledge.

Gonna set my deck up so that it tracks my kanji now, I think. I think it will function the way I want to if I have it tracking my kanji along with my words.

(2017-03-06, 6:42 am)kevsestrella Wrote:  
Thanks for the info about CJK Morphemizer Kaegi, I haven't read previous post, I will now.

To add onto this, the behavior also works a bit like this:
Ex. You learn how to write the 池 kanji in RTK, but you haven't learned the word 池[いけ] meaning pond. You have a large vocab deck, maybe the Core10k or some vocab you saved from your fun sources of content.
If you're currently learning how to write 池 in your Kanji deck and you have a morphemizer turned on for it, it will increase the priority of the 池 word.
If you have 池 in your Kanji deck as Mature and it's being tracked with the morphemizer, the card will have no New Morphemes, and so the priority will be at the highest, assuming you keep these options: http://i.imgur.com/H5EHhvs.png

I having kanji being tracked is probably a good thing, but I haven't re-enabled it.
I basically can read words like 旅行[りょこう] but don't know what the 旅 kanji is or how to write it, but MorphMan will show the word 旅[たび] at a (much) higher priority because I know 旅行. It sorts the words with no new morphemes first.

tldr; I think having it track kanji is good, because it'll put all of the kanji that exist as their own word and put them to the front. A short-term issue is that MorphMan will be assuming you know all of the words that are one kanji-only, ex. 話, 電, 気、意, etc., but if you have a core10k deck or something, it'll give help you fill in those gaps so that you actually do know those single-kanji words. For example, if you're a beginner and you learned 話す and learned how to write 話 in RTK, but didn't learn the word 話, you may find a sentence in sub2SRS sentence with the word 話. Maybe the sentence will be 「いい話だ。」 and will have no new words. It'll be listed as a high priority sentence, since it won't be highlighted as Unknown (or in Red color if you set up colors for unknown). Since 話 won't contain no new morphemes, though, so you'll see them earlier with MorphMan and fix your knowledge
Edited: 2017-03-06, 12:00 pm
Reply
(2017-02-23, 10:15 am)NinKenDo Wrote: I dunno. I've reached the end of my tether. Not trying to get jieba working with MM for a couple weeks until I can come at this problem with fresh eyes. I've just been winding myself up more and more every day. Haha. Time better spent studying and letting my brain process the problem in the background.

I've finally found the time to experiment a bit. The results of it can now be found in morphemizer-jieba branch on https://github.com/kaegi/MorphMan. Everything works as expected, except using the 'cut' method triggers (somehow) a 'check your database'-error the first on recalc, which you can ignore. I tried to investigate this error, but I don't even know where to start searching a connection. This is as far as I'm willing to go with this branch - the rest is up to you.

(2017-02-19, 4:49 pm)vladz0r Wrote: There are a few peculiar things/interactions with the "Learn Now."
If I have a JP->Eng Card1 and an Eng->JP Card2, and hit "Learn Now" on the Card1, it'll also try and show me the Card2. If I hit "Learn Now" on the Card2 for a word, it also tries to show me the Card1 again for the word, messing with the scheduling. The "Learn Now" returning multiple cards for a single note can also cause Anki errors, since Anki normally buried the card2 for a word if you saw the card1 for the word today.

Thank you for your bug report, but this particular feature can be considered a "convenient hack". It tampers with Anki's internal scheduling variables, which (I'm very sure) shouldn't be touched outside of the Anki codebase. The last time I fixed a seemingly trivial bug for that feature, took me about 5-7 hours to find the cause and the fix only partially improved the situation. Moreover, this feature isn't really related to the MorphMan plugin (it can be realized as an extra, separately maintained plugin without much effort) and you can approximate the behavior with repositioning new cards. Because the function is handy and improves the workflow a little, I left it inside MorphMan, but I don't think I will spent time "fixing" the feature/hack.

EDIT: Though I still have the same attitude towards this feature, it seems like I was to hasty in my assessment of this special bug. The fix was simpler than I expected (on second glace) and it is now on the "fix-learn-now-card-selection" branch. After some using/testing it in my Anki studies the next days, I will include it in the master branch. Again, thanks vladz0r, for reporting the bug.
Edited: 2017-03-09, 10:50 am
Reply
Sweet Smile
Having the bug fixed is useful for me, because sometimes I select a bunch of cards to [study now] and I'd have to bury my card2 and card3 versions of the card that arise, and sometimes I would mess up and not be able to undo the burying, and have to re-do the search in order to see the card now. The [study now] feature is amazing to have, and really makes Anki more effective and personalized.
Reply
EPIC SALE: Get 30% OFF Premium & Premium PLUS! (Sept 18 - 29)
JapanesePod101
Hey kaegi, can you confirm that for you the morphemizer-jieba branch gives you focus morphs of my than one character? I still have the same problem as before, so it might be a Windows thing.
Reply
(2017-03-11, 7:30 pm)NinKenDo Wrote: Hey kaegi, can you confirm that for you the morphemizer-jieba branch gives you focus morphs of my than one character? I still have the same problem as before, so it might be a Windows thing.

With activated morphHighlight one of the next vocab cards (the tenth or so) are [focus morph is written to Morpheme Readings/didn't want to create a new field]:

[Image: Bildschirmfoto%20von%20%C2%BB2017-03-12%...B.png?dl=1]

Or:

[Image: Bildschirmfoto%20von%20%C2%BB2017-03-12%...B.png?dl=1]

If I had to guess, I'd think the problem most probably is somewhere in your MorphMan configuration.
Reply
Okay, deleting my databases did the trick. Which is weird, because I did that before and it didn't fix the issue. But obviously something else changed as well, that was way back when I was still trying to use jieba.cut(). Thanks for all your help kaegi, super pumped to test this over the coming week and hopefully not destroy my collection.

I'll try and somewhat maintain the jieba branch until I can figure out the best way to modularise it.

Exciting, exciting, exciting!

Zài jiàn.

EDIT: I finally managed to fork after deleting a fork of MorphMan's parent project. Here's the link:

https://github.com/NinKenDo64/Jieba-Morph/
Edited: 2017-03-12, 7:02 pm
Reply
(2017-03-09, 8:45 am)kaegi Wrote: EDIT: Though I still have the same attitude towards this feature, it seems like I was to hasty in my assessment of this special bug. The fix was simpler than I expected (on second glace) and it is now on the "fix-learn-now-card-selection" branch. After some using/testing it in my Anki studies the next days, I will include it in the master branch. Again, thanks vladz0r, for reporting the bug.

I got around to trying to use this branch, and it gives me an error on my cards regarding the dictionaries and stuff.
I copied it over my old Morphman 4.0.
Here's the error - http://i.imgur.com/dHIekkP.png
Reply
(2017-03-22, 3:05 pm)vladz0r Wrote:
(2017-03-09, 8:45 am)kaegi Wrote: EDIT: Though I still have the same attitude towards this feature, it seems like I was to hasty in my assessment of this special bug. The fix was simpler than I expected (on second glace) and it is now on the "fix-learn-now-card-selection" branch. After some using/testing it in my Anki studies the next days, I will include it in the master branch. Again, thanks vladz0r, for reporting the bug.

I got around to trying to use this branch, and it gives me an error on my cards regarding the dictionaries and stuff.
I copied it over my old Morphman 4.0.
Here's the error - http://i.imgur.com/dHIekkP.png

I already fixed that in master, but didn't rebase fix-learn-now-card-selection on master. I've done that now. So please try again.
Reply
Is it possible to use MorphMan for two languages at once, and keep the word count separate?
Ex. using the morphemizer for Japanese, and the other morphemizer for something like Spanish? I'm trying to dive into Spanish, and I like using Morphman to roughly keep track of my words, and with subs2srs I think it'll be a fun journey.

It doesn't really have any issues other than the word count being combined between my Spanish and Japanese, but in a real-world scenario the world count is kinda useless anyway, but still cool to see the number go up for each.
Edited: 2017-04-06, 6:06 pm
Reply
(2017-04-06, 4:04 pm)vladz0r Wrote: Is it possible to use MorphMan for two languages at once, and keep the word count separate?
Ex. using the morphemizer for Japanese, and the other morphemizer for something like Spanish? I'm trying to dive into Spanish, and I like using Morphman to roughly keep track of my words, and with subs2srs I think it'll be a fun journey.

It doesn't really have any issues other than the word count being combined between my Spanish and Japanese, but in a real-world scenario the world count is kinda useless anyway, but still cool to see the number go up for each.

No this is not possible. Also, different counters would be very hard to implement, because all words are saved into the same database without information which word is which language.
Reply
After seeing that it's being developed again and it has a GUI now I decided to give Morphman another try after many failed attempts over the years. I set everything up and did a Recalc.

I checked a few hundred cards that come up when using the search parameter 'MorphMan_UnknownMorphCount:1' and among the unknown morphemes I've found 体, 上, 時, 区, 気, ところ. These words are part of my Core cards and they are definitely mature. I checked and they are review cards with huge intervals. Other words that are part of Core are being counted as mature normally.
What's more, if I put all these morphemes in the external.db they still show up as unknowns in the card field.
(I'm also finding particles like の. Should they be blacklisted by default?)

Any idea what's causing this?

Thank you.
Reply
(2017-04-12, 3:28 pm)thedoc Wrote: After seeing that it's being developed again and it has a GUI now I decided to give Morphman another try after many failed attempts over the years. I set everything up and did a Recalc.

I checked a few hundred cards that come up when using the search parameter 'MorphMan_UnknownMorphCount:1' and among the unknown morphemes I've found 体, 上, 時, 区, 気, ところ. These words are part of my Core cards and they are definitely mature. I checked and they are review cards with huge intervals. Other words that are part of Core are being counted as mature normally.
What's more, if I put all these morphemes in the external.db they still show up as unknowns in the card field.
(I'm also finding particles like の. Should they be blacklisted by default?)

Any idea what's causing this?

Thank you.

Glad to see it's finally working for you. The issue might be related to the part-of-speech tagging.

The morphemizer will assign a category like 動詞 to words, which might be context dependent. So if a word appears in a full sentence it might have a different part-of-speech category compared to a word in a single-word-sentence. To print all information of a morpheme including part-of-speech information, you can select a card in the Anki browser and click on "Edit -> MorphMan: View Morpheme". Try comparing the full information of ところ in your Core deck card vs your second deck.
Reply
Is there a way to disable morph-highlighting? It's a little buggy for me so I've turned off my css for it and but it still slows down opening a deck to review the first time significantly for me.
Reply
(2017-08-09, 9:35 pm)txtreme1 Wrote: Is there a way to disable morph-highlighting? It's a little buggy for me so I've turned off my css for it and but it still slows down opening a deck to review the first time significantly for me.

The slow down the first time you open a deck a day comes from skipping "too easy" cards. The plugin asks Anki for the next new card, assesses its difficulty and buries it if to easy (going to the next card). The next day all the cards are unburied again, so the process starts again if you review a "new" card. It takes a couple of seconds for about 10000 "too easy" cards on my computer.

Deleting or moving the buried cards from the deck fixes the problem.
Edited: 2017-08-10, 4:57 am
Reply
(2017-08-10, 4:56 am)kaegi Wrote:
(2017-08-09, 9:35 pm)txtreme1 Wrote: Is there a way to disable morph-highlighting? It's a little buggy for me so I've turned off my css for it and but it still slows down opening a deck to review the first time significantly for me.

The slow down the first time you open a deck a day comes from skipping "too easy" cards. The plugin asks Anki for the next new card, assesses its difficulty and buries it if to easy (going to the next card). The next day all the cards are unburied again, so the process starts again if you review a "new" card. It takes a couple of seconds for about 10000 "too easy" cards on my computer.

Deleting or moving the buried cards from the deck fixes the problem.

Are those tags marked with mm_alreadyKnown? Because I don't have any for whatever reason. I'm having difficulty finding which cards you're talking about also because I don't have any cards when I search for is:buried, after the long pause.
Reply
(2017-08-13, 1:39 pm)xtreme1 Wrote:
(2017-08-10, 4:56 am)kaegi Wrote:
(2017-08-09, 9:35 pm)txtreme1 Wrote: Is there a way to disable morph-highlighting? It's a little buggy for me so I've turned off my css for it and but it still slows down opening a deck to review the first time significantly for me.

The slow down the first time you open a deck a day comes from skipping "too easy" cards. The plugin asks Anki for the next new card, assesses its difficulty and buries it if to easy (going to the next card). The next day all the cards are unburied again, so the process starts again if you review a "new" card. It takes a couple of seconds for about 10000 "too easy" cards on my computer.

Deleting or moving the buried cards from the deck fixes the problem.

Are those tags marked with mm_alreadyKnown? Because I don't have any for whatever reason. I'm having difficulty finding which cards you're talking about also because I don't have any cards when I search for is:buried, after the long pause.

That's strange... No not the "mm_alreadyKnown" cards, just regularly buried cards. They typically have a low due number, the first few ones start with a "due" number around 999. Buried cards have parentheses around their due number. You should have some, if you have "skipping" enabled + already learned some (many) words + recalculated your database at some point.

What are your skip settings (settings in the "MorphMan Preferences -> General" Tab)? Does the slowdown occur every time you open a deck (multiple times a day) or only with your first "new" card a day? I don't think the slowdown has anything to do with MorphMan highlight, but just in case: what makes you assume there is a connection with the highlighting feature?
Reply
(2017-08-14, 12:37 pm)kaegi Wrote:
(2017-08-13, 1:39 pm)xtreme1 Wrote:
(2017-08-10, 4:56 am)kaegi Wrote:
(2017-08-09, 9:35 pm)txtreme1 Wrote: Is there a way to disable morph-highlighting? It's a little buggy for me so I've turned off my css for it and but it still slows down opening a deck to review the first time significantly for me.

The slow down the first time you open a deck a day comes from skipping "too easy" cards. The plugin asks Anki for the next new card, assesses its difficulty and buries it if to easy (going to the next card). The next day all the cards are unburied again, so the process starts again if you review a "new" card. It takes a couple of seconds for about 10000 "too easy" cards on my computer.

Deleting or moving the buried cards from the deck fixes the problem.

Are those tags marked with mm_alreadyKnown? Because I don't have any for whatever reason. I'm having difficulty finding which cards you're talking about also because I don't have any cards when I search for is:buried, after the long pause.

That's strange... No not the "mm_alreadyKnown" cards, just regularly buried cards. They typically have a low due number, the first few ones start with a "due" number around 999. Buried cards have parentheses around their due number. You should have some, if you have "skipping" enabled + already learned some (many) words + recalculated your database at some point.

What are your skip settings (settings in the "MorphMan Preferences -> General" Tab)? Does the slowdown occur every time you open a deck (multiple times a day) or only with your first "new" card a day? I don't think the slowdown has anything to do with MorphMan highlight, but just in case: what makes you assume there is a connection with the highlighting feature?

I assumed highlight was the only action that happens at runtime, so seems like that assumption is wrong based on your last comment. I don't have a general tab, which means it's time to update it seems! Is there a way to know what version I'm currently on?

Side question: Let's say I unsuspend a card, and I've encountered it in the wild enough times that I want to move it to the top of my learning queue. How can I get morphman to still "recognize" the card, but not muck around with it's order. Or should I just remove the tag associated with morphman for that card? But then once I do learn it, I would have to add it back, so that morphman knows my progress with it?
Reply
(2017-08-14, 10:08 pm)xtreme1 Wrote:
(2017-08-14, 12:37 pm)kaegi Wrote:
(2017-08-13, 1:39 pm)xtreme1 Wrote:
(2017-08-10, 4:56 am)kaegi Wrote:
(2017-08-09, 9:35 pm)txtreme1 Wrote: Is there a way to disable morph-highlighting? It's a little buggy for me so I've turned off my css for it and but it still slows down opening a deck to review the first time significantly for me.

The slow down the first time you open a deck a day comes from skipping "too easy" cards. The plugin asks Anki for the next new card, assesses its difficulty and buries it if to easy (going to the next card). The next day all the cards are unburied again, so the process starts again if you review a "new" card. It takes a couple of seconds for about 10000 "too easy" cards on my computer.

Deleting or moving the buried cards from the deck fixes the problem.

Are those tags marked with mm_alreadyKnown? Because I don't have any for whatever reason. I'm having difficulty finding which cards you're talking about also because I don't have any cards when I search for is:buried, after the long pause.

That's strange... No not the "mm_alreadyKnown" cards, just regularly buried cards. They typically have a low due number, the first few ones start with a "due" number around 999. Buried cards have parentheses around their due number. You should have some, if you have "skipping" enabled + already learned some (many) words + recalculated your database at some point.

What are your skip settings (settings in the "MorphMan Preferences -> General" Tab)? Does the slowdown occur every time you open a deck (multiple times a day) or only with your first "new" card a day? I don't think the slowdown has anything to do with MorphMan highlight, but just in case: what makes you assume there is a connection with the highlighting feature?

I assumed highlight was the only action that happens at runtime, so seems like that assumption is wrong based on your last comment. I don't have a general tab, which means it's time to update it seems! Is there a way to know what version I'm currently on?

Side question: Let's say I unsuspend a card, and I've encountered it in the wild enough times that I want to move it to the top of my learning queue. How can I get morphman to still "recognize" the card, but not muck around with it's order. Or should I just remove the tag associated with morphman for that card? But then once I do learn it, I would have to add it back, so that morphman knows my progress with it?

There is no way to display the current version, but that seems like a useful feature for debugging. I added that to my todo list.

I'm not sure I understand your side question... If you want to analyze your cards but not modify them you only have to turn off "MorphMan Preferences -> Note Filter -> Modify" on a filter that applies only your "analyze-only" cards. You can put a more specialized filter for a note type in front (e.g. with a "no_modify" tag) and disable modify for it. Only the first matching filter/rule is applied for a note/card.
If that't not what you meant... There is a "Learn Now" feature in the Anki browser. This will put the currently selected cards in the temporary learning queue (so they will be the next new/reviewed cards no matter what) - but the drawback is that it won't persist after an Anki restart.
Edited: 2017-08-15, 3:04 am
Reply
I always see this thread but never know what it means. Could you maybe add a short subtitle? I had to read in quite a ways to figure out what this was.
Reply
(2017-08-15, 3:04 am)kaegi Wrote:
(2017-08-14, 10:08 pm)xtreme1 Wrote:
(2017-08-14, 12:37 pm)kaegi Wrote:
(2017-08-13, 1:39 pm)xtreme1 Wrote:
(2017-08-10, 4:56 am)kaegi Wrote: The slow down the first time you open a deck a day comes from skipping "too easy" cards. The plugin asks Anki for the next new card, assesses its difficulty and buries it if to easy (going to the next card). The next day all the cards are unburied again, so the process starts again if you review a "new" card. It takes a couple of seconds for about 10000 "too easy" cards on my computer.

Deleting or moving the buried cards from the deck fixes the problem.

Are those tags marked with mm_alreadyKnown? Because I don't have any for whatever reason. I'm having difficulty finding which cards you're talking about also because I don't have any cards when I search for is:buried, after the long pause.

That's strange... No not the "mm_alreadyKnown" cards, just regularly buried cards. They typically have a low due number, the first few ones start with a "due" number around 999. Buried cards have parentheses around their due number. You should have some, if you have "skipping" enabled + already learned some (many) words + recalculated your database at some point.

What are your skip settings (settings in the "MorphMan Preferences -> General" Tab)? Does the slowdown occur every time you open a deck (multiple times a day) or only with your first "new" card a day? I don't think the slowdown has anything to do with MorphMan highlight, but just in case: what makes you assume there is a connection with the highlighting feature?

I assumed highlight was the only action that happens at runtime, so seems like that assumption is wrong based on your last comment. I don't have a general tab, which means it's time to update it seems! Is there a way to know what version I'm currently on?

Side question: Let's say I unsuspend a card, and I've encountered it in the wild enough times that I want to move it to the top of my learning queue. How can I get morphman to still "recognize" the card, but not muck around with it's order. Or should I just remove the tag associated with morphman for that card? But then once I do learn it, I would have to add it back, so that morphman knows my progress with it?

There is no way to display the current version, but that seems like a useful feature for debugging. I added that to my todo list.

I'm not sure I understand your side question... If you want to analyze your cards but not modify them you only have to turn off "MorphMan Preferences -> Note Filter -> Modify" on a filter that applies only your "analyze-only" cards. You can put a more specialized filter for a note type in front (e.g. with a "no_modify" tag) and disable modify for it. Only the first matching filter/rule is applied for a note/card.
If that't not what you meant... There is a "Learn Now" feature in the Anki browser. This will put the currently selected cards in the temporary learning queue (so they will be the next new/reviewed cards no matter what) - but the drawback is that it won't persist after an Anki restart.

That did answer both my side questions, thanks!

Upon update to 4.0 and disabling the all the options in the general tab, it still gives me a slow down whenever I restart anki.
Reply
In that case I don't know what could be the cause. I do not experience this problem and can't remember any other code that would produce a noticeable slowdown.

Just to get a feel for the size of your database... What is your known morpheme count (the number in the top bar "K ???") or how many MorphMan cards do you have?


Did you get the slowdown without MorphMan? I'm at a loss here...
Reply
(2017-08-16, 7:29 am)kaegi Wrote: In that case I don't know what could be the cause. I do not experience this problem and can't remember any other code that would produce a noticeable slowdown.

Just to get a feel for the size of your database... What is your known morpheme count (the number in the top bar "K ???") or how many MorphMan cards do you have?


Did you get the slowdown without MorphMan? I'm at a loss here...

I have 205504 morphman cards, and more every day. It's basically every deck I've been able to get hands on, and I sentence mine out of it, unsuspended them when encountered, and taking the one with the lowest morphman count. I'm not sure where the known morpheme count is or what top bar you mean, unless you are talking about the cards in the browser that have my morphman tag. That's the 200k number I just posted above. 

Yeah, I still get the slowdown after updating to the latest 4.0 morphman, no change there. Without morphman the lag goes away.

SideQuestion: With my uncompressed deck size now above the anki sync limit of 250 MB, I'm trying to reduce the number of fields I have. Morphman has a lot of fields, is it possible to trim some of the 7 and still have full functionality?
Edited: 2017-08-19, 5:51 pm
Reply
(2017-08-19, 5:50 pm)xtreme1 Wrote:
(2017-08-16, 7:29 am)kaegi Wrote: In that case I don't know what could be the cause. I do not experience this problem and can't remember any other code that would produce a noticeable slowdown.

Just to get a feel for the size of your database... What is your known morpheme count (the number in the top bar "K ???") or how many MorphMan cards do you have?


Did you get the slowdown without MorphMan? I'm at a loss here...

I have 205504 morphman cards, and more every day. It's basically every deck I've been able to get hands on, and I sentence mine out of it, unsuspended them when encountered, and taking the one with the lowest morphman count. I'm not sure where the known morpheme count is or what top bar you mean, unless you are talking about the cards in the browser that have my morphman tag. That's the 200k number I just posted above. 

Yeah, I still get the slowdown after updating to the latest 4.0 morphman, no change there. Without morphman the lag goes away.

SideQuestion: With my uncompressed deck size now above the anki sync limit of 250 MB, I'm trying to reduce the number of fields I have. Morphman has a lot of fields, is it possible to trim some of the 7 and still have full functionality?

That's what I anticipated... The slowdown might just be the time to load and parse the MorphMan databases (which in theory should be reasonably fast). There is no simple fix for that unfortunately. If I get back to more "extreme" changes in MorphMan, I will look into lazy loading/chunking of the database - that might be possible or not.

SideQuestion answer: Only the focus morph field is needed. See "Extra Fields" tab in "MorphMan Preferences".
Reply
(2017-08-20, 3:26 am)kaegi Wrote:
(2017-08-19, 5:50 pm)xtreme1 Wrote:
(2017-08-16, 7:29 am)kaegi Wrote: In that case I don't know what could be the cause. I do not experience this problem and can't remember any other code that would produce a noticeable slowdown.

Just to get a feel for the size of your database... What is your known morpheme count (the number in the top bar "K ???") or how many MorphMan cards do you have?


Did you get the slowdown without MorphMan? I'm at a loss here...

I have 205504 morphman cards, and more every day. It's basically every deck I've been able to get hands on, and I sentence mine out of it, unsuspended them when encountered, and taking the one with the lowest morphman count. I'm not sure where the known morpheme count is or what top bar you mean, unless you are talking about the cards in the browser that have my morphman tag. That's the 200k number I just posted above. 

Yeah, I still get the slowdown after updating to the latest 4.0 morphman, no change there. Without morphman the lag goes away.

SideQuestion: With my uncompressed deck size now above the anki sync limit of 250 MB, I'm trying to reduce the number of fields I have. Morphman has a lot of fields, is it possible to trim some of the 7 and still have full functionality?

That's what I anticipated... The slowdown might just be the time to load and parse the MorphMan databases (which in theory should be reasonably fast). There is no simple fix for that unfortunately. If I get back to more "extreme" changes in MorphMan, I will look into lazy loading/chunking of the database - that might be possible or not.

SideQuestion answer: Only the focus morph field is needed. See "Extra Fields" tab in "MorphMan Preferences".

Ok cool, thanks for the prompt responses.


SideQuestion: Oh right, the info was right where I should have expected it to be.
Reply
(2017-08-20, 1:43 pm)xtreme1 Wrote:
(2017-08-20, 3:26 am)kaegi Wrote:
(2017-08-19, 5:50 pm)xtreme1 Wrote:
(2017-08-16, 7:29 am)kaegi Wrote: In that case I don't know what could be the cause. I do not experience this problem and can't remember any other code that would produce a noticeable slowdown.

Just to get a feel for the size of your database... What is your known morpheme count (the number in the top bar "K ???") or how many MorphMan cards do you have?


Did you get the slowdown without MorphMan? I'm at a loss here...

I have 205504 morphman cards, and more every day. It's basically every deck I've been able to get hands on, and I sentence mine out of it, unsuspended them when encountered, and taking the one with the lowest morphman count. I'm not sure where the known morpheme count is or what top bar you mean, unless you are talking about the cards in the browser that have my morphman tag. That's the 200k number I just posted above. 

Yeah, I still get the slowdown after updating to the latest 4.0 morphman, no change there. Without morphman the lag goes away.

SideQuestion: With my uncompressed deck size now above the anki sync limit of 250 MB, I'm trying to reduce the number of fields I have. Morphman has a lot of fields, is it possible to trim some of the 7 and still have full functionality?

That's what I anticipated... The slowdown might just be the time to load and parse the MorphMan databases (which in theory should be reasonably fast). There is no simple fix for that unfortunately. If I get back to more "extreme" changes in MorphMan, I will look into lazy loading/chunking of the database - that might be possible or not.

SideQuestion answer: Only the focus morph field is needed. See "Extra Fields" tab in "MorphMan Preferences".

Ok cool, thanks for the prompt responses.


SideQuestion: Oh right, the info was right where I should have expected it to be.

To verify the theory:
  • Does the slowdown still occur after you move .../Anki/<profile>/dbs to another location (so your database is emtpy/temporarily "deleted"), and does it occur again if you regenerate it from scratch?
Just to have some additional information for debugging later:
  • What's the disk size of your databases in .../Anki/<profile>/dbs?
  • How many seconds slowdown do you experience?
FYI: Thanks to gnprice MorphMan will have some command-line tools for importing vocabulary and frequencies in the database from texts without needing to import Anki cards. That way you can separate sentence mining and your actual Anki learning cards.
Reply