MathProf
In my project prototype, I will have a JD head that provides interaction with a student who wants (needs?) to learn some remedial math. My design approach is that the bot interface will serve two purposes:
a friendly, engaging interface with the student
essentially a data collection device observing student understanding through
- spoken responses from student to questions and prompts posed by JD
- written responses from student as problem solutions are attempted by the student
So, from a prototype perspective, we probably have "friendly and engaging" covered (I have to admit being excited about learning more about EZ-AI). However, I believe that the handwriting recognition (real-time) will be tough. I have taken a look at open source tesseract, but don't see it as a practical solution. I believe that product compares written characters to some existing database of alphabetic character representations and does so offline. What's needed here is a software tool that real time determines the character written by the student.
Part of the solution is that, when a student is first introduced to a JD head in the lab, part of the get-to-know-you interchange will be JD asking the student to write the numerals 0 through 9, and the letters x, y, a, b, and c. That might be all we need in remedial math. So we can store, for each student, a mini-database that contains both written and verbal expressions of each of the numerals and the selected letters.
After a bunch of research, it looks like the only (real time) solutions is to use a neural network application running on the associated PC. I've found some research done in 2000 involving methods and coding for "on-line" recognition of handwritten signatures that would see to be more than enough processing power for this prototype
My question: Are you aware of any software tool that could use digital camera input to recognize in real time handwritten characters?
Thanks for any suggestions!
Ron
Look into roborelm. It might be able to do it. If not I think he is working on handwriting recognition.
WOOF! Thanks, Dave.
Handing writing recognition is really, really, really ify! As d.cochran said, take a look at RoboRealm, it does many vision processing tasks including character recognition (still ify in my humble opinion), has an interface pipe directly to ARC and has a 30day trial. My advice is not to download it until you are ready to spend time experimenting to see if you want to buy it.
ARC with the custom object might recognize trained hand written characters but my gut feeling is that might be pushing the limit of capability, unless the student writes the character and also does the training. If you were going to use base set of training with hopes students will make characters similar to what was trained as a custom object that might be a bridge too far.
Justin, regarding "If you were going to use base set of training with hopes students will make characters similar to what was trained as a custom object that might be a bridge too far.", I found a research paper published some years ago reporting on the use of simple neural networks to recognize numeric digits in zip codes written on items processed by the US Postal Service. I've just skimmed the paper, but that approach might work. After implementing NNs for just digits, I'll add characters "x", "y", "a", "b", "+", "-", "/", and "=" to see how far that gets me.
It looks like, once the digital image is received from the EZ-B, the question to ask is "which of the special characters/digits does this image best match" rather than "what is this character". This alternative approach will not require training by each student.
Ron