How I compare them:

I usually play with the data structures in a python interpreter.  In this case I just created a list of all the expressions, calculated the length, and then converted it to a set (thus remove duplicates) and calculated the length again.  Should be correct, albeit extremely strict (ie, a sentence with an extra space or punctuation would be considered different- it's also case sensitive).

>>> from mach4 import *
[ snipped output for brevity ]
>>> len( [ d[i]['sentExpr'] for i in d ] )
9669
>>> len(set( [ d[i]['sentExpr'] for i in d ] ))
9604
>>>

Last edited by overture2112 (2011 January 30, 12:48 am)

Since it seems a number of the words are marginally out of phonetic index order and the sentences' sub-order seems arbitrary, I think the best call at this point is to manually correct them.

I've made a publically editable spreadsheet here.  I've added a few fields, but it should be easy enough to update with any existing corrections anyone has made (the PackIndex and PackName uniquely identify an entry, so just sort your spreadsheet and this one by those and copy/paste updated columns).

The Core6k index and word/sentence audio files are based on guesses. Effectively, the word expression and reading must match exactly and in the case of multiple potential matches in Core, it first tries the first one that has exactly the same meaning, then tries the first one whose meaning contains one of the entry's meanings, and failing that just grabs the first match (ignoring meaning).

If anyone can think of useful fields to add or has a better suggestion on how to collaboratively correct the issues, please suggest them.

Edit: My matching code mentioned above identifies 5792/9669 as in core6k and 3877/9699 as new (with 4 possibly duplicates; same expression, reading, and english meaning).

Last edited by overture2112 (2011 January 26, 2:21 am)

I've also started doing a few. How do we match the leftover sound files to sentences... transcribe and search the spreadsheet?

And can someone tell me that the guy in js01115a and js01164a really is a native speaker?

Last edited by Irixmark (2011 February 13, 8:54 am)

Irixmark wrote:

And can someone tell me that the guy in js01115a and js01164a really is a native speaker?

Native speaking wife say no.

Plodding away, but with only brainobush and me working on it, we'll take ages... anyone care to help out with a few sentences/audio matches?

I've dragged the sentence audio files into iTunes and just play them in order (left half of screen) while inserting a "Y" for every match. Works fine, but it's tiring.

I wasn't planning to do more than those either... if I understand correctly, we have the other sentences from Core2k6k anyway?

Did someone already find a way to match it all? I noticed this deck shared on Anki. (Japanese Core 10000 or something.)