overture2112 Wrote:

MonjaIsshin Wrote:According to your instructions on page 1 of this thread...Actions>It's Morphin Time>Set i+N.

Needing to manually do that is only applicable to Morph Man 1. In MM2 there's a thread in the background that constantly updates any decks that aren't open (thus if you only have one deck, you need to sit at the deck selection page while it updates). In MM3 it returned back to a manual process because it was made over 100x faster and the workflow ends up nicer that way.

MonjaIsshin Wrote:Can you take me step-by-step through this? I'm trying to run this on the Tanuki-Ultima, where I have redefine the {{Japanese - Definition (Kanji)}} as {{Expression}} and added a field named iPlusN and set it to be numerically ordered

1) Change deck to have Expression field and iPlusN field.
2) Go to morph man gui -> config auto and enable the deck to be auto analyzed.
3) Go to deck selection window and wait for at least a minute. If it's still going Anki will warn you / prevent you from using that deck, saying the database is already open. You can also check the morphman.log file to get a better idea of what's going on and if it's done or errored out etc instead of just waiting and hoping for the best
4) Open the deck and it should be ready to go.

For more detail, the wiki should cover things like how to check the log (although note to use the Morph Man 2 wiki page, as "Morph Man" now points to instructions for MM3)

I use a mac so maybe there are some differences or something, but I can seem to figure this out.

Step 1) Change deck to have Expression field and iPlusN field. ---> done

Step 2) Go to morph man gui -> config auto and enable the deck to be auto analyzed. ---> How do you go to the morph man gui? The only reference to morphman that I have in anki is Morphman Recalc and Morphman Manager?

Also I was reading in another page of this thread that the iPlusN field has to be sorted numerically. I can't seem to find any options to allow me to do that either. Anki allows me to sort by that field, but there are now other options that I can see.

Not really sure where to go from here..

Some of these are *quite* late, but posting responses in case others come across similar issues:

lesson17lesson17 Wrote:...Why is the [Japanese Support] plug-in required?...

The Japanese support plugin helps locate mecab. Likely not relevant if you're not using mecab and have rewritten that portion of the code, so you should be okay.

JusBlueboy Wrote:if not MECAB_ENCODING: MECAB_ENCODING = getMecabEncoding()
File "C:\Documents and Settings\Acer\My Documents\Anki\addons\morph\morphemes.py", line 54, in getMecabEncoding
return runMecabCmd( [ '-D' ] ).stdout.readlines()[2].lstrip( 'charset:' ).lstrip()
IndexError: list index out of range

It seems like Mecab isn't installed or working correctly. Perhaps you don't have the Japanese Support plugin installed?

johndoge Wrote:[...]line 175, in updateNotes
TAG.register( tagNames )
UnboundLocalError: local variable 'tagNames' referenced before assignment

I believe this occurs only when zero cards are processed. Try changing your config to enable some models. Maybe it's possible that when you reinstalled Anki the decks were updated and model names changed somehow?

That said, I've corrected MorphMan (locally; haven't updated Anki plugin's site) to avoid this issue in the first place and better handle the case where you don't have anything enabled.

aa666 Wrote:each access to card browser (on either 'B' or 'L') is accompanied by a memory leak of 8-10 mb

This is (sort of) intended. Morphman caches *super* aggressively with no limit. If I recall, there should be an option to disable memoization entirely, but I ought to modify that particular case to behave better.

vosmiura Wrote:@overture2112, I'm not sure if this feature is already built in somehow, but I wanted the ability to include some decks as input to MM3 without having them be re-scheduled by MM3, so I added a "readonly" option....

Yes, I've had this feature locally for awhile but never released it. I really like it for certain grammar decks.

SeanMartin66 Wrote:In short, I just want to use it for one deck and ignore the others.

See http://rtkwiki.koohii.com/wiki/Morph_Man#Configuration

Nayr182 Wrote:Just a quick question, do I have to recalculate after I finish my reviews each day? Or is the initial calculation enough for the entire life of the deck provided I don't add new cards to the deck?

You have to manually recalculate whenever you want it to reflect your new knowledge. Generally this means once right before doing reviews is sufficient, though when learning new material I actually tend to do it every few minutes. This is because of the feature to reinforce recently learned vocab.

Ex: You learn words A, B, and C. A few minutes later you recalc. Then MorphMan will prioritize sentences for new words D, E, and F that contain ones you just learned (ie. A, B, and C).

I *really* like the reinforcement of "seen but not yet mature" morphemes, as getting more example uses of new words helps understand the real range of its meanings, so on my desktop I tend to recalc any time I take a break for a few seconds, but you'd probably still get helpful results even if you only did it once a month.

Note MorphMan keeps a temporary list of new words you just learned this session and auto skips new sentences focused on that word (if you have the "auto skip alternatives" feature on) which is lost when you close Anki; so you need to recalc after re-opening Anki to avoid getting "new" sentences which are actually for words you already learned since the last recalc.

Nayr182 Wrote:Also is it possible to use morphman filter the adding of bulk cards to a deck?

For example, if I wanted to add the core6k deck or another shared deck to my own self created deck, but I don't want review things that I already know.

So when I add core6k (for example) to my own deck, could I use morphman to tag or delete cards that haven't been reviewed yet, but introduce no new material?

Yes. You can just enable the deck and then suspend anything with "k+N" of 0, a low "morphManIndex", no entry for "focusMorph", etc.

Alternatively, playing with MorphMan Manager and Mass Tagger would work too.

Nayr182 Wrote:Basically I just want to have the ability tag cards that introduce no new morpheme and haven't been learned yet.

This would enable people to add large amounts of cards from subs2srs etc, and easily be able to deleted cards that don't teach new material.

You can already do this. For example, filter cards where "k+N" field is 0 (ie. have zero unknowns). Alternatively, notes are automatically tagged with "comprehension" if all the words are already known and mature.

I actually just subs2srs entire seasons of shows and vn2srs entire novels and then just let MorphMan run on all of them. With the "next new card feature" enabled, any sentences that only contain words that are already known are automatically skipped. Alternatively (and useful for AnkiDroid which has no MorphMan), you can use a Filtered Deck that just contains k+N:1 cards.

Nayr182 Wrote:Is there an official place where one can request new features etc?

Sadly not. Though I'm looking to restart studying and update my tools, so I'm tempted to look for a solution for this.

jessem Wrote:Is there no real start-up guide for this?...I haven't edited the config file at all...but I don't know if that's necessary for it to work or if that's just customization...

There's some youtube videos which might help, but editing the config file is definitely required. The settings 99% of people want require irreversible modification to the notes, but that's not a safe default behavior, so by default MorphMan doesn't touch anything unless you explicitly enable it to.

choubatsu Wrote:Just a suggestion based on all the posts here but, it seems like there's a need for very basic instructions on how to edit the config file (with a sample). I'm having the same problems with errors because I have no idea how to edit the config file according to my own decks.

I'll try to create a more in-depth guide on modifying the config, as well as upload one I use as a sample (and explain why).

choubatsu Wrote:How do you get Morphman to ignore certain decks?

Create an entry in the deck overrides with enabled set to False. See http://rtkwiki.koohii.com/wiki/Morph_Man#Configuration

choubatsu Wrote:The total morphemes listed in Morpheme Manager is 1996. Can this be right?

It's possible it's not looking at the right field(s). By default it only looks at the "Expression" field, but you'll need to modify that if your notes are different.

Nayr182 Wrote:Step 2) Go to morph man gui -> config auto and enable the deck to be auto analyzed

I think you may be running and/or looking at instructions for an old version of MorphMan?
Version 2 had an auto mode where it constantly recalculated in the background. Version 3 is manual-only (but like 1000x faster and safer).

overture2112 Wrote:

aa666 Wrote:each access to card browser (on either 'B' or 'L') is accompanied by a memory leak of 8-10 mb

This is (sort of) intended. Morphman caches *super* aggressively with no limit. If I recall, there should be an option to disable memoization entirely, but I ought to modify that particular case to behave better.

I'm wrong; there's no memoization occurring here.

I actually get the same leak without MorphMan installed by just clicking Anki's Browse over and over (v2.0.31).

overture2112 Wrote:I actually get the same leak without MorphMan installed by just clicking Anki's Browse over and over (v2.0.31).

it's japanese support (jp.py), not morphman itself i suppose
with no plugin installed i don't see any leak

Whenever I try to do something with morphman, I get this error message:

An error occurred in an add-on.
Please post on the add-on forum:
https://anki.tenderapp.com/discussions/add-ons

Traceback (most recent call last):
File "C:\Users\Jenny\Documents\Anki\addons\morphman.py", line 6, in onMorphManRecalc
morph.main.main()
File "C:\Users\Jenny\Documents\Anki\addons\morph\main.py", line 199, in main
allDb = mkAllDb( cur )
File "C:\Users\Jenny\Documents\Anki\addons\morph\main.py", line 57, in mkAllDb
ms = getMorphemes( fieldValue )
File "C:\Users\Jenny\Documents\Anki\addons\morph\util.py", line 119, in __call__
value = self.func(*args)
File "C:\Users\Jenny\Documents\Anki\addons\morph\morphemes.py", line 82, in getMorphemes
ms = [ tuple( m.split('\t') ) for m in interact( e ).split('\r') ] # morphemes
File "C:\Users\Jenny\Documents\Anki\addons\morph\util.py", line 119, in __call__
value = self.func(*args)
File "C:\Users\Jenny\Documents\Anki\addons\morph\morphemes.py", line 66, in interact
p = mecab()
File "C:\Users\Jenny\Documents\Anki\addons\morph\util.py", line 119, in __call__
value = self.func(*args)
File "C:\Users\Jenny\Documents\Anki\addons\morph\morphemes.py", line 59, in mecab
if not MECAB_ENCODING: MECAB_ENCODING = getMecabEncoding()
File "C:\Users\Jenny\Documents\Anki\addons\morph\morphemes.py", line 54, in getMecabEncoding
return runMecabCmd( [ '-D' ] ).stdout.readlines()[2].lstrip( 'charset:' ).lstrip()
File "C:\Users\Jenny\Documents\Anki\addons\morph\morphemes.py", line 50, in runMecabCmd
s = subprocess.Popen( cmd + args, bufsize=-1, stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.STDOUT, startupinfo=si )
File "C:\cygwin\home\dae\win\build\pyi.win32\anki\outPYZ1.pyz/subprocess", line 679, in __init__
File "C:\cygwin\home\dae\win\build\pyi.win32\anki\outPYZ1.pyz/subprocess", line 896, in _execute_child
WindowsError: [Error 2] The system cannot find the file specified

and it confuses the heck out of me. This happens when I recalc morphman, extract morphenes, or view morphenes.

what should I do to fix it?

Thanks

My guess is you don't have the Japanese Support plugin installed (the official plugin that adds the `mecab` program).

I can't make this thing work. I don't understand how to go about it.

Does it only work with japanese or does it work with other languages as well? How can I make it work with another language?

ok, finally got it working.

You have to edit the config.py file and set the note types that will be overridden and the ones that will be used.
Also the same process for decks and profiles, which ones are overridden and the ones that are not.

I had parentheses in my note types, it wasn't working with those in the names, so I had to remove them.

Once you do that, you have to add these fields to your cards:

k+N
m+N
unknowns
unmatures
unknownFreq
morphManIndex
focusMorph
vocabRank
iPlusN

After all this stuff, RESTART ANKI (because you changed the config.py). Took me a while to realize this.

Then and only then can you recalc.

Had to battle with the add on for 3 days to achieve this hahaha

cheers

errtu Wrote:Once you do that, you have to add these fields to your cards:

This step is completely optional.

vosmiura Wrote:@overture2112, I'm not sure if this feature is already built in somehow, but I wanted the ability to include some decks as input to MM3 without having them be re-scheduled by MM3, so I added a "readonly" option.

e.g.

model_overrides = {
'subs2srs': { 'enabled':True },
'Japanese': { 'readonly':True },
'k2k Japanese': { 'readonly':True },
}

Then I changed the logic so decks with 'readonly' == True participate in the morpheme input phase but not in the output phase.

Could you please specify how and where I can change the participation of the input and output? I am very interested in this added "readonly" feature and would like to implement it myself, but I can not figure out how to make this work. Do I have to make the modifications in the config file, or somewhere else?

It's not called readonly. In config.py just mark the deck as analyze with no enable flag:

Code:

# Model overrides can only override the entries marked above. 2nd priority
model_overrides = {
    'subs2srs': { 'enabled':True },
    'Japanese Sentences': { 'analyze':True },
    'Japanese Legacy MCD': { 'analyze':True },
    'Japanese MCD': { 'analyze':True, 'morph_fields': [u'Text'] },
}

@tokyostyle
I did not know this was possible, I will give this a try. Thanks!

It was a happy day when I realized its power. If you read between the lines of my configuration I basically schedule all of my "sentence" cards normally using Anki but add their contents to morphman. That way my subs2srs deck takes into account more of my passive vocabulary when ordering those cards.

I did some experimenting to get it working, but so far without success...

I have modified the config file like this:

Code:

# Model overrides can only override the entries marked above. 2nd priority
model_overrides = {
    'Japanese subs2srs': { 'enabled':True },
    'Japanese Recognition': { 'analyze':True },
}

I assumed that "model" is the used note type.

So far, the deck that uses the subs2srs note type is reordered, however, it appears that morphemes are not extracted from the Japanese Recognition cards (the database is still empty). Therefore, all the cards in the subs2srs deck are all on "notready". I did check that the Japanese Recognition type has an expression field to be analyzed, so that should be ok.

Any ideas?

Ｔｏｋｙｏｓｔｙｌｅ，　便利な情報をありがとう！My setup is pretty similar, so like others, I was wondering how to do this. That will save me from having to export morphemes from other decks when I want to recalculate.

Rivvie Wrote:Any ideas?

This is kind of embarrassing. I apparently hacked that feature in.

In morph/main.py on or around line 44 you need to change:

Code:

if not C('enabled'): continue

to

Code:

if not C('enabled') and not C('analyze'): continue

Do not change the one later on in the file.

[RawToast]

Quote:In morph/main.py on or around line 44 you need to change:

If it helps anyone, for me that was on line 38.

I then had to go to util.py and amend line 48:

From: return cfgMod.default[key]
To: return cfgMod.default.get( key )

So the cfg method look like this:

Code:

def cfg( modelId, deckId, key ):
    assert cfgMod, 'Tried to use cfgMods before profile loaded'
    profile = mw.pm.name
    model = mw.col.models.get( modelId )[ 'name' ] if modelId else None
    deck = mw.col.decks.get( deckId )[ 'name' ] if deckId else None
    if key in cfgMod.deck_overrides.get( deck, [] ):
        return cfgMod.deck_overrides[ deck ][ key ]
    elif key in cfgMod.model_overrides.get( model, [] ):
        return cfgMod.model_overrides[ model ][ key ]
    elif key in cfgMod.profile_overrides.get( profile, [] ):
        return cfgMod.profile_overrides[ profile ][ key ]
    else:
        return cfgMod.default.get( key )

This should satisfy "and not C('analyze')"

After these steps my "known morphenes" increased by ~200. I had about ~100 reviewed cards of the type I added the 'analyze' key to.

Thanks tokyostyle and RawToast! Thanks to both of your comments, I was able to get it working. I have zero programming knowledge whatsoever, so couldn't have done it without your help ^^.

Good to hear! I made all of those hacks years and years ago now and have never had to touch them again so hopefully you'll get the same utility out of it!

Edit: spelling

I dropped the ball here. I had an ankiweb issue that was preventing me from updating MorphMan and then totally forgot after it was resolved. The MorphMan available via Anki is quite out of date!

Amongst other things, I added a "set due based on mmi" feature, which you can optionally disable to analyze notes but not modify the due date of their cards. Thankfully tokyostyle provided an alternative that achieves a similar effect.

I'll eventually upload it to ankiweb, but in the meantime, for anyone that's curious, you can get the latest version (ver 3.7) here: https://github.com/jre2/JapaneseStudy/bl...phman3.zip

-- Big picture
Controlling analysis
● Config option to have MorphMan analyze notes but not modify their due times ("set due based on mmi").
● Config option to have MorphMan analyze notes and modify their due times, but consider them unlearnt. This is useful if you have a grammar deck where you only test yourself on the grammar rule, not the vocabulary ("ignore maturity").

Prioritizing sentences better
● MorphMan reinforces vocab you recently learnt by prioritizing sentences that contain unmature words. Can be tweaked with "reinforce new vocab weight"
● MorphMan will prioritize sentences that contain words in priority.db. Highly useful if your goal is to read a particular book or watch a particular episode of a show, as this will make sure you learn any morphemes in priority.db first. See "priority.db weight"

Stats
● Morphman now displays the number of morphemes you know on the toolbar. If you hover over it, you can also see your progress towards your "goal" databases (both known words and percentage when weighted by frequency).
● To create a goal database simply prepend the name with "Goal."

For example, create a database of all the morphemes in volume 1 of Yotsubato! and save it as "Goal.Yotsubato.db" and "priority.db" (or union it with your current priority.db if you already have one). Now all the new words you learn will be words found in Yotsubato! (if possible) and you can easily measure your progress in terms of what percentage of words you know and how many unique new words you still need to learn.

Bugs
● MorphMan *should* be able to prioritize certain parts of speech, but it doesn't seem to work. See "verb bonus" option.

-- Changelog
Version 3.4
● mmi (MorphMan Index) algorithm changes (reinforce new vocab, part of speech emphasis, etc) and more configurability
● option to not set due based on mmi

Version 3.5
● better progress information
● large performance boost by caching better
● don't modify notes/cards that haven't changed
● add priority.db feature

Version 3.6
● change batch play keybind to ctrl-alt-p (since ctrl-shift-p is now taken by preview)
● add tags for too long and non-optimal sentence length
● add ability to not count notes for morpheme maturity (ie. treat cards as 0 interval)
● next new card algorithim now skips cards tagged as already known, which helps in the advent that Anki was restarted but Morphman wasn't recalc'd

Version 3.7
● add stats tracker for morphemes known and compares against "goal" dbs
● tracker also shows percentage of morphemes in goal database that are known, weighted by frequency
● brought back Mass Tagger feature

Do you want my Cloze type hack in the main branch? All it does is check to see the field being analyzed is a close type then strips it of the extra close markup before sending it though the parsing process.

Hi, I'm using the Dec 31 version from GitHub.

I had to add the "#-*- coding: utf-8 -*-" to main.py to make it run.

Before the first run, I changed the tags in config.py (for example I changed "comprehension" to "mm_comprehension"). But it seems to be sometimes using the changed tag and sometimes using the default one. Some cards have one, some cards have the other, and some have both. It's not untagging consistently either.
Searching for "comprehension" in the code, I don't see how this is possible, so maybe I did something silly...
The deck already had someone else's MorphMan tags in it before I started :o

I added new Requirements section on the MM wiki page with Japanese support plugin. It was mentioned in small note in How to use it section, but was IMO too easy to overlook (I did). Resulting error message does not give a clue what the problem actually is, so it is extra important to have this info highlighted.

I also suggest to add it in the OP.

Cheers and big thanks for the plugin, it is epic!

Looks like the old version on AnkiWeb disappeared sometime in the last month, or has it changed name?

(I was going to test my new plugin with it, since that's the version most people will be using.)

Great plugin

I've noticed a lot of my cards are listing particles in the "focusMorph", "unknowns" and "unmatures" fields. How can I inform MorphMan that these items are known?

I have tried adding a list of particles to my known.db but this doesn't seem to have an effect.

Thanks!