Using AI (ChatGPT) for learning Chinese

[Jan Finster]

On 1/25/2023 at 6:43 PM, phills said:

When you use it as a chatbot, how well does it respond to you if you speak ungrammatically to it?  Can you get it to correct your speech?  I imagine it'd be a terrible at
"correcting" you, but maybe you can prompt it to engage in that behavior.

 

 It is actually pretty good, but then I am not a native speaker. So, I guess I should test this in German to really scrutinise it. You can test it in English yourself.

 

Here are a few more things I tried:

a) let us have a discussion in mandarin and correct every mistake I make (grammar or choice of words) [happy as far as I can tell]

b) Let me practice sentence patterns and correct any grammar mistakes or if I express myself in a grammatically correct but uncommon way [I was happy with it]

c) Please explain in simple mandarin the process of XYZ [helps me generate content]

d) Please explain why it is 就会变得 here and not just 变得 or 会变 [and similar grammar Qs; I thought it was helpful]

e) please rewrite the following paragraph substituting all verbs with other verbs that express the same thing [to learn how to say things differently...]

f) list all medical terms in this text and provide a simple definition in mandarin... [vocabulary mining]

g) List 5 other ways to say XYZ differently. Provide example sentences for each.

 

The list goes on....

 

I am super happy with it. I am just afraid it will not stay free for long and if I had to pay for every question, this would be cost prohibitive since I ask tons of small questions.

 

 

[Jan Finster]

Sometimes ChatGpt sounds like a child with ADHD ?

 

I cannot remember the second character in 经济. Please provide other words that contain this character and explain its meaning

 

The second character of "经济" (jīng jì) is "济" (jì), and it means to help or aid.

Here are some other words that contain the character "济":

救济 (jiù jì) - relief, aid

救援 (jiù yuán) - rescue, aid

救助 (jiù zhù) - aid, assistance

救急 (jiù jí) - emergency aid

救生 (jiù shēng) - lifesaving

As you can see, the character "济" is often used in words related to providing help or assistance. In the word "经济" (jīng jì), it refers to the economy as a system that helps to sustain and support the needs of society through the production, distribution and consumption of goods and services.

 

me: but your examples contain 救 not 济???? 

 

You are correct, my apologies for the mistake. I apologize for any confusion caused.

Here are some examples of words that contain "济" and not "救" ......

 

 

[phills]

On 1/26/2023 at 4:19 AM, Jan Finster said:

Here are a few more things I tried:

a) let us have a discussion in mandarin and correct every mistake I make (grammar or choice of words) [happy as far as I can tell]

b) Let me practice sentence patterns and correct any grammar mistakes or if I express myself in a grammatically correct but uncommon way [I was happy with it]

c) Please explain in simple mandarin the process of XYZ [helps me generate content]

d) Please explain why it is 就会变得 here and not just 变得 or 会变 [and similar grammar Qs; I thought it was helpful]

e) please rewrite the following paragraph substituting all verbs with other verbs that express the same thing [to learn how to say things differently...]

f) list all medical terms in this text and provide a simple definition in mandarin... [vocabulary mining]

g) List 5 other ways to say XYZ differently. Provide example sentences for each.

 

These are all great prompts!  Perfect for polishing intermediate / advanced level language skills.  The language learning websites should be jumping on this, or someone new is going to leave them in their dust.  I think there are some language learning software devs around on this forum, and I'm curious as to their take.

 

As someone who's been reading up on AI tech, the second character thing is a neat example of the bot's behavior.  Most of the training examples probably involve changing the first character and not the second.  I'm guessing statistically, the chance of a typo/mistake:

 

(mistyping "济" instead of "救" in the fragment:

"Here are some other words that contain the character "济": 救济 (jiù jì) - relief, aid"...)

 

is about the same order of magnitude as text asking to vary the second character in the training set.  Plus both 济 and 救 means help, which makes it even easier to confuse. 

 

Of course, once it goes wrong, it'll keep going in that wrong direction, confidently.

 

Did it correct itself properly afterwards, after you corrected it?

 

[Jan Finster]

On 1/26/2023 at 10:27 AM, phills said:

AI tech, the second character thing is a neat example of the bot's behavior. 

 

I realised it does not do things like:

"please rewrite the following paragraph but use different verbs and nouns while keeping the orginal meaning. "

 

On 1/26/2023 at 10:27 AM, phills said:

Perfect for polishing intermediate / advanced level language skills.  The language learning websites should be jumping on this, or someone new is going to leave them in their dust.  I think there are some language learning software devs around on this forum, and I'm curious as to their take.

 

It helps whem massively to create new content, but I guess they also lose some ground to ChatGPT. Once ChatGPT accepts audio input and can provide audio output, this will be groundbreaking. Apart from the TTS voice, it would make many low-level teachers obsolete. Overall however, I am optimistic that every new tech makes "the pie enlarge", so there will always be a market for language teachers. I am getting excited about the future of language learning.

 

 

[phills]

 

On 1/26/2023 at 7:20 PM, Jan Finster said:

It helps whem massively to create new content, but I guess they also lose some ground to ChatGPT.

 

You can use ChatGPT in API mode though, so they can actually use ChatGPT in their current sites to provide exercises & tests through clever prompting.  OpenAI is a pure tech company, they're not going to chase the language learning market.  They want someone to do it for them.

 

And you can easily pipe together voice recog & chat gpt & text-to-voice.  You just need to redo lesson plans in prompt-script form (a script of a series of prompts).

 

I remember seeing someone else's calculations (which I didn't check myself), who claimed that, at current prices, the cost of using ChatGPT through the API isn't much more than the cost of electricity of running ChatGPT locally.

[Kenny同志]

SkyNet is becoming a reality. 

[大块头]

@Kenny同志 Reading through the examples posted so far, are there any instances where the Chinese wasn't idiomatic? I don't think we've had a native-speaker chime in yet.

[Kenny同志]

There are some expressions that do not sound very natural. But as I'm using my phone, it's kind of hard for me to give you any examples. Personally, I wouldn't use it for learning Chinese before I become an advanced learner. 

[大块头]

https://www.hackingchinese.com/can-chatgpt-pass-the-hsk/

 

Quote

So can ChatGPT pass the HSK?
Well, if we mean if it can give the correct answers to multiple-choice reading questions, the answer is a resounding “yes”, at least if the questions are straight up, honest reading questions not deliberately designed to trick you.

This is quite amazing! Going into this project, I expected it to do very well on matching and fill-in-the-blanks types of questions, which it did. I also expected it to struggle with trickier questions, which it also did. However, I e expected it to struggle a lot more than it did!

I was also blown away by the accuracy of most of the reading comprehension questions. Sure, it’s one thing to be able to fill in the blanks or match words, but actually, content-based reading questions also went surprisingly well!

I should have predicted that, though, because ChatGPT also does pretty well on most take-home exam style questions, as shown in multiple YouTube videos (search for your area of expertise and ChatGPT and you’ll probably find something).

It should be pointed out that it would be premature to say that ChatGPT could pass HSK, even ignoring the fact that it needs human guidance to format the questions properly, because the HSK is a test you either pass or fail the whole exam, not just the reading part. However, if we assume that the percentage of correct answers required for the whole exam were required for each part, ChatGPT would breeze through HSK 3-5. It would also pass HSK 6, although not with as safe a margin.

 

[phills]

 

On 2/14/2023 at 1:53 AM, 大块头 said:

However, I expected it to struggle a lot more than it did!

I was also blown away by the accuracy of most of the reading comprehension questions. Sure, it’s one thing to be able to fill in the blanks or match words, but actually, content-based reading questions also went surprisingly well!

 

Interesting blog post!  As I kind of expected, it passed even HSK 6.  Sentence completion is what these chat bots are designed to do.  And the standardized AI test suites tend to include reading comprehension type tests (in English). So that's also a skill that's well polished, and seems to have transferred to Chinese mostly intact. 

 

The Chinese AI companies have native chinese chatbots that'd probably do even better on the "tricky" questions that ChatGPT stumbled on, since they're trained natively on gobs of chinese text.  Just like I bet ChatGPT would do better on the tricky questions if they were phrased in English. 

 

The hardest part seems to be getting the chatbot to understand the test format.  I noticed the author translated all the instructions into English, I wonder how it'd do if you just fed it to them in Chinese. 

 

As a pure measure of its language ability (vs its thinking ability), it seems like you should feed the chatbot the exact same test in English & Chinese, and compare their results. 

[Insectosaurus]

Based on my experiences its Chinese ability is way worse than for English. I ask it to define English words for me when I encounter new ones, and it does so really well. Most Chinese words I want it to define it gets completely wrong. Perhaps for common words it's equally good for English and Chinese (thought I doubt it) but for less common ones it's more often than not clueless.

 

苍头是什么意思？

"苍头"在中国语境中一般指的是白发苍苍的老人，是一种形容老年人头发花白的俚语。也有一些地方用“苍头”来形容年约较大的人，即不一定要是老年人，而是指相对年轻的人来说已经有些年纪的人。

 

For Swedish:

Vad är en färist? (What is a "färist" [cattle grid])

Ordet "färist" är inte ett vanligt begrepp eller ett ord som är känt för mig. Kan du beskriva vad du menar med det? Kanske kan jag hjälpa dig bättre om jag får mer information.

(It's not a word known to me. Please tell me what you mean. Perhaps I can help you if I receive more information.)

[Jan Finster]

On 2/13/2023 at 7:57 PM, Insectosaurus said:

Based on my experiences its Chinese ability is way worse than for English

Actually it sometimes even tries to wiggle itself out of helping me with Chinese saying it is an English language model and it was not trained for Chinese. When I call the bluff and "remind" it that we practiced Mandarin before together, it complies... ?

[alantin]

It has clearly eaten a lot more English content than content in other languages. I know it's Japanese is a bit on the formal side and a bit unnatural. It makes a lot of grammatical errors in Finnish, but overall it's quite ok, two native speakers have told me It's Chinese is pretty good, and a Vietnamese laughed at it and said it is still a bit weird.

 

But it is still really amazing and already quite a regular tool for me!

[markhavemann]

On 1/21/2023 at 12:04 PM, Johnny-5 said:

hmm, not in China 

I get the same, even with a VPN it somehow knows that I'm not in the US or wherever.

[Johnny-5]

On 2/18/2023 at 2:28 PM, markhavemann said:

I get the same, even with a VPN it somehow knows that I'm not in the US or wherever.

Now that I've changed my VPN to one that actually works, I've gotten to this step but I don't have a non-chinese phone number.

Don't think it's important, it seems like a neat thing to play around with, but I don't really see how it fits into my learning "process".

[malazann]

have you seen the really really creative ways people have used prompts to circumvent chatGPT self-censorship? They convince chatGPT that it's actually someone else (like "Dan") and get it to do all the things that the normal one would block (like writing a poem praising Hitler). 
 

There is an unofficial mantra on this forum (i might be making this up), that the way to get better at "X" is to do more of "X". It's really really simple, yet my mind is always trying to do Y to get better at X. 

Anyways, I am setting up a chatgpt environment where I will provide a scenario and basic outline, then it will improvise a conversation with me. I've been adamant about it using the "method acting" method in order to become the characters. I've told it to add randomness into the scenarios, and made sure it knew to take up multiple characters in a scenario (if need). I also have tried to stop it from softening any rough language/attitudes, but we'll see how that goes. Maybe I'll set up a Shanghai-er and Beijing-er convo and get real dirty

 

[becky82]

On 2/14/2023 at 2:32 AM, phills said:

Interesting blog post!  As I kind of expected, it passed even HSK 6.

 

The HSK6 exam (the reading section) it took seems to be H61000 (https://archive.org/details/elsey_HSK6), which I understand is an exam script from 2010 when HSK 2.0 was first being implemented.  I don't mean to be a stickler here, but this is significantly easier than the current HSK6.  Also, it's quite possible this exam script (or parts of it) is part of ChatGPT's training data; this would get a student disqualified.  And it only scored 60%, which is equal to (what people usually consider as) the pass mark.

 

That being said, it's still quite amazing.

[becky82]

There's a new 李永乐 video where ChatGPT takes the Gao Kao 高考 (college entrance exam).  Here are its marks:

 

 

English 英语: 86%.  Literature 语文: 67%.  Mathematics 数学: 67%.  History 历史: 62%.  Physics 物理: 60%.  Biology 生物: 60%.  Politics 政治: 54%.  (He left off the details of the politics section, perhaps to avoid controversy.)

[phills]

Interesting!  What's a passing score on the 高考? 

 

For comparison, Chat-GPT scores 1050 on the SAT, according to this guy.  That's about 52% percentile. 

 

https://twitter.com/teddynpc/status/1598767389390573569

 

[realmayo]

I think it's interesting that (if I understand it right) these things are properly called Large Language Models rather than AI. Someone said recently: the entire point of LLMs is to use language in a plausible way - that's all ... there's no 'intelligence', so they're not 'artificial intelligence', they're not trying to solve anything apart from how to respond to your words with plausible sounding words of their own. As such, they ought to do well in these kinds of tests! But they are fundamentally stupid! (Good  fun though, friends of mine admired its ability to write classical poems on contemporary topics....)