Back

New manga OCR reader (capture2text alternative)

#1
Hi all, 
capture2text was not very suitable for manga reading, so I made  better alternative:

Advantages over capture2text:

Automatic text extraction, just one click to read. No precise and annoying selections needed.
Can read bubbles of complex shape.
Significant text processing to remove common OCR mistakes and adjust uppercase/lowercase characters where other OCR software fails.
Image binarization includes upscaling.
It can sometimes read images that cannot be read manually.
Text output is usually good enough for automated translation.


Software is currently in very early version so memory leaks are likely and interface is messy.


Usage:

Just unzip anywhere, run and right click in the bubble you want to read.
You have to click on white space inside the text bubble somewhere close to the center !
within a second you will have text in the clipboard for further processing 
Application will press ESC button to cancel right mouse click menu which pops when you click, so you cant use right button for original purpose.
If you are reading bad quality source you may have to adjust threshold slider to get better binarization results.
If characters are improperly detected you can try to shrink or zoom source and click again  zooming improves recognition accuracy, shrinking improves character detection


Application can be downloaded here:
http://d-h.st/aJev

Since it is in early development I need feedback and suggestions what can be improved.
Also I need better name for my application.

Further development will be mostly focused on more advanced text extraction to make it possible to read text which is embedded directly into image.
Also this is starting point for manga editing application to automate manga translation which will automatically clean and replace Japanese text with English
Edited: 2016-01-15, 12:42 pm
Reply
#2
Why does your application need SSL? Where is it connecting to and for what purpose?
Reply
#3
(2015-11-18, 9:30 am)tokyostyle Wrote: Why does your application need SSL? Where is it connecting to and for what purpose?

SSl is used to access translation webpage when you click translate button:
https://www.babelfish.com
Reply
MONSTER Sale Get 28% OFF Basic, Premium & Premium PLUS! (Oct 16 - 27)
JapanesePod101
#4
(2015-11-18, 9:09 am)Digix Wrote: Hi all, 
capture2text was not very suitable for manga reading, so I made  better alternative:

Advantages over capture2text:

Automatic text extraction, just one click to read. No precise and annoying selections needed.
Can read bubbles of complex shape.
Significant text processing to remove common OCR mistakes and adjust uppercase/lowercase characters where other OCR software fails.
Image binarization includes upscaling.
It can sometimes read images that cannot be read manually.
Text output is usually good enough for automated translation.


Software is currently in very early version so memory leaks are likely and interface is messy.


Usage:

Just unzip anywhere, run and right click in the bubble you want to read.
You have to click on white space inside the text bubble somewhere close to the center !
within a second you will have text in the clipboard for further processing 
Application will press ESC button to cancel right mouse click menu which pops when you click, so you cant use right button for original purpose.
If you are reading bad quality source you may have to adjust threshold slider to get better binarization results.
If characters are improperly detected you can try to shrink or zoom source and click again  zooming improves recognition accuracy, shrinking improves character detection


Application can be downloaded here:
http://www.filedropper.com/reader

Since it is in early development I need feedback and suggestions what can be improved.
Also I need better name for my application.

Further development will be mostly focused on more advanced text extraction to make it possible to read text which is embedded directly into image.
Also this is starting point for manga editing application to automate manga translation which will automatically clean and replace Japanese text with English
Your link is down
Edited: 2015-12-28, 1:10 am
Reply
#5
new link
http://www.megafileupload.com/io8v/Reader.rar
Reply
#6
(2016-01-04, 5:27 pm)Digix Wrote: new link
http://www.megafileupload.com/io8v/Reader.rar

Wow there are some pretty cool ideas in this. I like that you can just click the box of the manga panel to get text, though sometimes it would be useful to manually select. It was still too buggy to use (mouse would freeze then come in laggy after a minute, executing the exe opened like 4 windows every time, and random crashes, and issues with furigana recognition), but I'm really looking forward to using this in the future.
Reply
#7
Manual selection will be in the next version
as well as some interface improvements.

Lags will usually happen if you click outside of bubble as it will capture too much of area and try to read it all.
It does not crash on my computer so it is quite hard to fix that problem


furigana is causing real problems here and I still dont know how to remove it easily
I have some cleanup ideas, to extract text from background, maybe it will also work with furigana as well
Reply
#8
latest version
http://d-h.st/aJev
or
http://www.filehosting.org/file/details/...Reader.rar

Custom selections are possible now -- Click or select desired area, center of selection must not touch any symbol
Horizontal text handling added -- If selection width is twice as big as height horizontal text mode will be activated
White on black text handling -- if selection center is on black area, image will be inverted
Furigana removal  as well as function to increase spacing between columns for better recognition accuracy (may not work on skewed text)

some bug fixes , so application should be crashing less often
Edited: 2016-01-07, 3:49 pm
Reply
#9
You sir, are unbelievable!

Quote:Also this is starting point for manga editing application to automate manga translation which will automatically clean and replace Japanese text with English

This is actually EXACTLY the program I was hoping for ever to exist!
Please continue the developement, just ask for my money and I will give it to you.

However, the download-links aren't working. Could you replace them, please?

EDIT: How about RawMe/RAWME/RAWme as a name, because its purposed usage for translating RAW's?
I'll come up with more - promised. I'll even design you a logo.
Edited: 2017-02-27, 1:48 pm
Reply
#10
I temporarily stopped development of this application because it turned a bit too complex, there does not seem to be any significant interest and I ran into problems with integrating spelling checker and text overlays on the pictures.
Basic reading part is pretty much complete and finished.

I think I will resume its development later, but now I have some other work to do.

If you are interested in this application we can discuss it over email, so give your email to me via pm any kind of help will be useful.
Reply
#11
I'm an avid user of Capture2Text and I've been wishing for something like this for a loooong time!!!

Thank you sooo much for creating this tool. I really want to use it and have been trying to download the file for a few days but the site you uploaded to (Dev-host) seems to be having problems.

Can you please upload an alternative link?
Reply
#12
Here's the mirror download of the tool.
Reply
#13
I don't understand why this is not popular. I think not much people know (or think) about posibility of this. Best and easiest OCR i ever seen. Just one button, no need to paint windows on bubbles. Simple and amaizing. However i whould prefer google translator instead babelfish (it realy suck).
Edited: 2017-09-24, 12:01 am
Reply
#14
(2017-09-23, 11:59 pm)Psion Wrote: I don't understand why this is not popular. I think not much people know (or think) about posibility of this. Best and easiest OCR i ever seen. Just one button, no need to paint windows on bubbles. Simple and amaizing. However i whould prefer google translator instead babelfish (it realy suck).

True, I was using KanjiTomo but this one is even better. The UI could be improved though.
Reply
#15
This program is pretty cool, I don't know why I never tried it before.  Thanks for this. 

Since this copies the string into the clipboard, you can pair it with something like Jglossator.
Reply
#16
I wonder what you think can be improved in the UI?
I notice more problems with OCR itself which needs considerable postprocessing.
Reply
#17
If you're using tesseract you can improve OCR quality by pre-scaling the text (as in characters) to a specific size in pixels. With the tesseract 4 "best" LSTM traineddata the best scale I've found is somewhere between 30 and 40 pixels wide, 32 works well. I let the user set the approximate size of the text in the manga before OCRing it in nezuyomi.
Edited: 2017-09-25, 5:57 pm
Reply
#18
(2017-09-25, 3:27 pm)Digix Wrote: I wonder what you think can be improved in the UI?
I notice more problems with OCR itself which needs considerable postprocessing.

Well, OCR is far better than most other solutions I tried to read manga.

The program needs an option to have it always on top so it's easier to access to it.
Also a mini-mode would be useful when you don't want it to show real OCR characters, just the text that is pasted in the field and maybe the translation, similar to Babylon.

Anyway, good job as I said Smile
Reply
#19
My software does text upscaling to improve small font recognisability but it may be a good idea to resize text to the specific constant size. if it happens to be too big. as now I am pretty puzzled why OCR accuracy depends on character size so much

always on top function is not that useful because you are not supposed to use interface in any way besides checking for obvious OCR failures.
The main purpose of this program is to place text into clipboard where it can be translated with some other tool

Also for those who want to see internal image processing sequence, you can resize the window and uncover some more stuff.
Reply
#20
(2017-10-03, 7:53 am)Digix Wrote: as now I am pretty puzzled why OCR accuracy depends on character size so much
OCRs are typically machine learning algorithms based on convolutions of images of characters. Presumibly the convolutions would be slightly different at different resolutions unless the OCR standardized resolutions internally. I would suspect that sending the OCR charactor images closer to the resolution the neural networks were trained on would result in higher accuracy. Just a guess based on general understanding of OCR, but no practical experience.
Reply
#21
(2017-10-03, 3:24 pm)yogert909 Wrote:
(2017-10-03, 7:53 am)Digix Wrote: as now I am pretty puzzled why OCR accuracy depends on character size so much
OCRs are typically machine learning algorithms based on convolutions of images of characters.  Presumibly the convolutions would be slightly different at different resolutions unless the OCR standardized resolutions internally.  I would suspect that sending the OCR charactor images closer to the resolution the neural networks were trained on would result in higher accuracy.  Just a guess based on general understanding of OCR, but no practical experience.

In this situation, I would assume that OCR engine should do all resizing work itself to get optimal character size.
In that situation, i can see some problems with upscaling small fonts because resizing them will introduce severe distortions.
But strangely enough OCR accuracy fails when the picture is high resolution and crystal clear and just could not be any better.
Reply
#22
It needs DPI information to have a good first guess of how big the characters actually are. But in tesseract's case, even then it doesn't actually do resizing of its own. It wants to take advantage of the extra clarity given by having high res text, so it can't just downscale it at will. But if it wasn't trained on text that large it's going to make mistakes.
Edited: 2017-10-06, 8:35 am
Reply