thechamp Posted May 18, 2017 at 10:41 AM Report Share Posted May 18, 2017 at 10:41 AM I just found out I've got onto a start-up incubator, so am leaving my current job to take up the funding and design/build a product for a year. If my future team mates think it's a good idea, I'm quite interested in a Chinese studying app for children, with perhaps a gaming element to it. Does anyone have any experience of studying Chinese that they think would be particularly fun/engaging for children? Has anyone taught Chinese to Western children and noticed something novel about how children approach tones/characters? I have a few ideas but I thought it might be an interesting discussion, and a year and a team is quite a decent amount of time to make something nice, if things go to plan. 1 Quote Link to comment Share on other sites More sharing options...
roddy Posted May 18, 2017 at 11:34 AM Report Share Posted May 18, 2017 at 11:34 AM Boring. Use one of the new AI as a service offerings (Google's new one looks fun) to teach computers how to give meaningful feedback on learner pronunciation. * * you owe me $$$$$$$$$$$$$$$$$$$$ 1 Quote Link to comment Share on other sites More sharing options...
thechamp Posted May 18, 2017 at 12:07 PM Author Report Share Posted May 18, 2017 at 12:07 PM The technical test for it actually involved hooking up to Microsoft Azure or AWS's image recognition services. I chose AWS and am generally quite interested in these services but unless they expose a lot of the code (which from what I've seen, they don't) then I don't see how you can give them these kinds of specific training sets. Also all my exposure to that kind of statistical analysis is in excel rather than doing it in code so would be a big ask, but we have a year so maybe it can be done! Sort of like a phone app where you talk into it and it assigns you a score, based on how close you are to 'putonghua' pronunciation, based on your natural pitch? Quote Link to comment Share on other sites More sharing options...
imron Posted May 18, 2017 at 12:17 PM Report Share Posted May 18, 2017 at 12:17 PM See here. 1 Quote Link to comment Share on other sites More sharing options...
roddy Posted May 18, 2017 at 12:18 PM Report Share Posted May 18, 2017 at 12:18 PM Not just a score. What I'm imagining (and I'm so far out of my depth...) Say you give it umpteen thousand examples of 'correct' speech. Then you give it umpteen thousand examples of 'incorrect' speech, but coded by actual humans*. Initially maybe for 'problem with initial / final / tone' but perhaps getting more specific with common errors. 'Pronounced pinyin x as English sh' for example. Then magic happens. Then the AI can take an example of learner speech and say 'you're getting q and ch mixed up and your first tone is dipping. Try again... ok, that's better...' Maybe that's impossible, or trivial. But it'd be great. PS. GPUs, Imron? Old. *You have funding, yeah? Quote Link to comment Share on other sites More sharing options...
imron Posted May 18, 2017 at 12:26 PM Report Share Posted May 18, 2017 at 12:26 PM 6 minutes ago, roddy said: Then magic happens See here 9 minutes ago, roddy said: Maybe that's impossible, or trivial. See here. Quote Link to comment Share on other sites More sharing options...
thechamp Posted May 18, 2017 at 12:28 PM Author Report Share Posted May 18, 2017 at 12:28 PM Imron based on the test I am assuming that the stacks we'll be using we won't actually be building out specific AI but relying on these cloud services. There are a couple of AWS ones that look like they could be re-purposed for this kind of product but that's the thing - I think they actually provide only extremely specific services and you're not given access to any of the underlying software. You basically hook up to the API and pass values for processing. Also, I have friends who experimented with Tensorflow on my MSc and I'm a little bit scared tbh. Quote Link to comment Share on other sites More sharing options...
imron Posted May 18, 2017 at 12:30 PM Report Share Posted May 18, 2017 at 12:30 PM Tensorflow can be embedded on the device and work without access to a network. Quote Link to comment Share on other sites More sharing options...
roddy Posted May 18, 2017 at 12:31 PM Report Share Posted May 18, 2017 at 12:31 PM Yeah, you're going to struggle find an xkcd cartoon I haven't already seen. What do you reckon, Imron? Plausible? Speech recognition has obviously come on in leaps and bounds. How long till feedback goes beyond a tone graph? Quote Link to comment Share on other sites More sharing options...
imron Posted May 18, 2017 at 12:32 PM Report Share Posted May 18, 2017 at 12:32 PM Anyway, in general, I agree with Roddy. The market is saturated with the 'learning apps for children'. The pronunciation feedback App is a much better idea. 1 minute ago, roddy said: What do you reckon, Imron? Plausible? I reckon both plausible and a good idea. Building the training set would likely be the most difficult part. Quote Link to comment Share on other sites More sharing options...
roddy Posted May 18, 2017 at 12:35 PM Report Share Posted May 18, 2017 at 12:35 PM Thanks Imron, I needed a new signature. Building the training set is difficult, but feasible, no? Coding schema, several dozen BLCU CSL students on their summer break, lots of Chinese students around the world recording phrases in exchange for the coded feedback... 1 Quote Link to comment Share on other sites More sharing options...
thechamp Posted May 18, 2017 at 12:39 PM Author Report Share Posted May 18, 2017 at 12:39 PM Hahaha, I like those XKCD comics too. I have done enough stats in the past to have be aware of linear regression and classifications etc. Most of Machine Learning seems to pretty much be statistical analysis and classification anyway, and then coding up the application. Hmmm, maybe it is doable?! I've never had a year to program anything end to end, and we're in teams of 4 (although 1 person will be a business person) so I actually think perhaps it is plausible? Quote Link to comment Share on other sites More sharing options...
thechamp Posted May 18, 2017 at 12:44 PM Author Report Share Posted May 18, 2017 at 12:44 PM Also even if it fails miserably - awesome transferable skills! Quote Link to comment Share on other sites More sharing options...
roddy Posted May 18, 2017 at 12:45 PM Report Share Posted May 18, 2017 at 12:45 PM I actually wonder if it's a viable business model *WITHOUT* the AI. Think how many students would like reliable, actionable feedback on their pronunciation. Now think what someone in the US will pay for that, and how much you'd have to pay someone in China for something they can do on the bus with headphones in. Either way your first step, a comprehensive review of Chinese pronunciation and what goes wrong with it, is the same. Much smaller moat without the AI though. And *surely* - think of how many phonemes Alexa and Siri and Google whateveritis are processing every day. I find it hard to believe they cant be taught to tell you that you just said zh instead of ch. In fact this'd work great as an Alexa Skill. Quote Link to comment Share on other sites More sharing options...
thechamp Posted May 18, 2017 at 12:54 PM Author Report Share Posted May 18, 2017 at 12:54 PM That's what I mean though - you don't have access to the code when you use these cloud services. Hooking up to them involves a lot of command line fiddling and then you pass files (in the case of the test it was JPEGs....and the image recognition on for AWS is the only one I've really used (except a Microsoft cloud service we use in our finance system)). You can't really 'teach' eg Amazon Lex....you have to essentially rebuild Amazon Lex, to do what you want it to do. It's quite a big ask. I've been told though that they're not expecting the product to be market ready after a year, just MVP as something to show to investors. Maybe it is plausible but it's really ambitious. It would basically involve taking one of the tensorflow templates and re-purposing it. Quote Link to comment Share on other sites More sharing options...
roddy Posted May 18, 2017 at 12:54 PM Report Share Posted May 18, 2017 at 12:54 PM Come on man! Think big! Be the change I want to see in the world! People were trying this in 2004. Extra credit - wearables Quote Link to comment Share on other sites More sharing options...
thechamp Posted May 18, 2017 at 12:55 PM Author Report Share Posted May 18, 2017 at 12:55 PM Hahaha, I don't want to go anywhere near trying to set up a kind of skype service from England to China thanks! Quote Link to comment Share on other sites More sharing options...
thechamp Posted May 18, 2017 at 01:03 PM Author Report Share Posted May 18, 2017 at 01:03 PM Wow, a lot to think about! I didn't expect the discussion to go in this direction(!) Yes building the training set would be very time intensive, but with this kind of thing I'm expecting there'll be a lot of time when the business person isn't marketing etc....and I think that would probably be a good time for them to do this. Quote Link to comment Share on other sites More sharing options...
roddy Posted May 18, 2017 at 01:05 PM Report Share Posted May 18, 2017 at 01:05 PM Oooh, hang on. Tell users if their Chinese characters are well-formed or not, and why... Quote Link to comment Share on other sites More sharing options...
thechamp Posted May 18, 2017 at 01:08 PM Author Report Share Posted May 18, 2017 at 01:08 PM I've realised you could actually also cheat with this, a lot, because you could use the speech recognition services as they are (they return text of the input audio) and then do your pronunciation checks on how accurate the text was at writing the input audio. I really don't know how that would solve the tones issue though. I'm not a machine learning researcher I'm a semi-competent software engineer trying to get a product ready in 1 year! Quote Link to comment Share on other sites More sharing options...
Recommended Posts
Join the conversation
You can post now and select your username and password later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.