A warning on using ChatGPT and other AI technologies for foreign language learning

Recently I have started being more active on Threads, and the other day I came across someone who was talking about maintaining a daily journal in Japanese. This is a great thing to do and I have learned much myself from writing various forms of content in Japanese. However, I was a bit shocked when I saw that they off-handedly mentioned that they always used ChatGPT during writing the journal.

This made a part of me want to start lecturing them about why ChatGPT wasn’t a great idea to use when writing a block in Japanese, but I decided to calm down and instead use this as material for a longish blog article. This is actually something I touched upon a bit in one of my podcast episodes, but I wanted to drill down into some related topics here.

Just to give some context, I am a software developer by trade and have used AI-related technologies including ChatGPT and some image generation tools as well. I don’t consider myself an expert on AI, but I’ve used these sorts of tools enough to be comfortable giving my  opinion on what they can and can’t do, especially in the area of language learning. Though perhaps I should rephrase that: it’s not about what they can or can’t do, it’s about whether these tools doing something for you is beneficial to your education as a language learner. I haven’t used AI much for language learning, but I did try a few quick queries before writing this article just to see what ChatGPT would return on a Japanese grammar topic, and what I saw reinforced my opinion.

To be clear, if your goal is just being able to communicate (or understand) something in a foreign language for the moment, then things like digital translation tools (which overlaps with AI to a certain degree) are generally very useful, and I think using them is great. Even for Japanese-English, they have improved greatly in the last few years, and while I think for some areas like literary works a human translator can still do a much better job, for basic everyday stuff I think translation tools can get you pretty far.

But the context here is someone who is learning a foreign language, and writing a journal each day, presumably in order to improve their skills in that language, so that they can eventually communicate without any forms of technology, in other words by using their own brain.

Broadly speaking, AI tools can serve two purposes related to languages. First, they can translate from one language to another. Second, they can provide explanations about aspects of grammar, words, or try to correct some text. The first use is more about raw communication, and the second more about making the AI a learning resource, or even something approaching a language coach.

So I’m imagining how that person on Threads is putting prompts into ChatGPT, possibly doing a little of both of these two objectives. 

My problem with the former (using a translation tool) can be summed up by giving the parallel of watching a foreign language show with English subtitles. By doing this, you can get the feeling you are actually learning, and may frequently nod, saying to yourself “Yeah, I get it. That’s what that means…” But the problem is that this is all after the fact, and there is little motivation for you to actually be following along and understanding at a deep level. In other words, after doing this for hundreds of hours, there is no guarantee that you can understand the spoken language in a foreign language show if the subtitles were shut off.

While I think there is some value in watching with subtitles on (especially in terms of learning new expressions), if you want to practice comprehension of spoken Japanese, you need to actually practice listening and trying to understand spoken Japanese. Reading subtitles actually occupies a big part of your brain just to read those and parse them (even for your native language), bandwidth that would be better spent just focusing on the spoken language.

Similarly, for writing text, if you are using ChatGPT or a similar AI for every journal you are writing in some foreign language, you are drastically reducing the amount of time you need to actually think about forming your own sentences. Instead, your mind is now in the weird mode of trying to check the translated text, looking for errors, and maybe trying to extract meaning from the parts so you can learn something from the experience.

Normally, when you have to form a sentence on your own (whether written or spoken), you think about what you want to say (possibly in your native language, which is OK), and then you think about related expressions you know in the target language, and also maybe about grammar, and vocab, and then try to make sentences using all that material. This process of integrating all these disparate forms of knowledge is really one of the most important skills you are training yourself to do. Eventually, as you get better you learn what types of words or concepts need to be said first, and you start speaking (or writing) those first, all while your brain pipelines things and thinks about the rest of the sentence. But if you are using a tool to give you a rough draft of a sentence, you are short cutting things and missing out most of the critical pieces here.

If you are simply just looking up words or grammar (not trying to write a whole sentence or paragraph), then sure, you can use ChatGPT to try and explain something to you. But the problem is with both words and grammar, there are already a bunch of sites out there that give good information. Sure, it may require more reading and time to comprehend and integrate all the info, but that is the point––you have to actually think for yourself and figure out how to use all this information.

But what I’ve said so far doesn’t factor out one of the biggest issues with ChatGPT and other similar AI tools: that they tend to make mistakes frequently, and often when you least expect it. Even if you assume it is correct 90% of the time, the other 10% you have no idea what it will get wrong, and the more you rely on it, the more likely you are to pick up those same mistakes.

As I mentioned earlier, I tried a few Japanese grammar questions on ChatGPT, and while the answers looked very nice and neat, when I read through it in detail in each case I found at least one or two points that either was blatantly wrong, or didn’t make sense to me. (If you want me to analyze a specific language-related query that is wrong, let me know in the comments and I’ll be glad to make an article focusing on that.)

Even though I have talked many times about how a focus on grammar is important with language learning (I even published a book about grammar recently), I will admit that it isn’t the most important thing. The most important thing is putting yourself in contact with native-level language, and learning from that.

Having a foundation in grammar is important since it gives you a leg up in understanding native-level language, and also in creating your own sentences. But ultimately, if you are (for example) writing a journal in Japanese, you should be referring to similar content written by natives, like a journal written by a Japanese person. Reading these things before writing your own journal entries can be helpful, or you can look through a native’s journal entries when searching for something similar to what you want to say. But asking ChatGPT to translate or explain something is very different from using a true human native speaker for reference. 

Now I realize that being of an older generation, there may be a tendency for me to feel negatively about new technologies (though I’m not that old), and try to argue in favor of the “old and good” ways. That’s why I want to emphasize that I don’t think there is absolutely no value to using ChatGPT, but it must be used with extreme caution, and ideally only a very small part of your study routine. So if you spend most of your time on grammar, vocabulary, reading/listening to native language, and forming your own sentences, I don’t think it’s going to hurt to ask ChatGPT once in a while for its opinion. But any such queries should be backed by followup research to verify any claims, and whenever possible you should search for native-level sentences similar to what you are dealing with. While it is unlikely entire sentences said by a tool will hit a lot of sites on a web search engine, even if they are correct and natural, you can pick out little parts here and there and see how frequently they are actually used. (This trick works much better for written language, though.)

Similarly, I sometimes use AI tools for software development. So far, it seems to give useless answers over 90% of the time, but once in a while it will give me a good idea. However, having a strong foundation in software development principles allows me to easily see if it is just hallucinating, or giving me real actionable advice.

While I have heard some interesting research results about using how ChatGPT to assist a task tends to make people think less about certain things, this sort of research is relatively new, and research results are often twisted to suit the personal opinions of people in power. Even though I do think there is likely some truth to these things, my points in the article are not at all based on research results. Rather, I am basing this on what I consider a common sense idea about how a tool designed to help you think (or to think for you) will necessarily require less of your thought, and hence your ability to remember and understand things will be reduced. 

Then there is the topic of how AI will improve into the future. We have seen AI making rapid advances in the last decade, and I don’t see any reason this won’t continue in the future (though it is not a smooth curve of progress, but a bumpy and irregular one).

Some readers may think I’m crossing a line here (especially if you believe in a soul), but I personally think it’s fair to assume that eventually AI will become smarter than humans (and more accurate) in most, if not all domains. When that happens, yes I do think having an AI language teacher would be a great thing that can drastically speed up the learning curve.

However, when that happens, I think the human race will have much bigger problems to deal with, or at least much larger societal changes occurring such that the very nature of human life is shifting to something completely different. Indeed, if AI is that great, there is no reason that computer translation can’t be made to be better than human translation, and potentially even faster. 

To summarize, while I think ChatGPT and other similar AI tools can have potential value for language learners, using those tools is a slippery slope that can distract from learners spending their time actually understand the language at a fundamental level, so I would suggest to either stay away from it completely, or exercise proper caution when trying to learn from any AI tool. At least for the time being, I would focus more on talking with humans, and consuming content that was created by humans.

And when producing your own sentences (whether spoken or written), I feel that it is infinitely more valuable to produce those sentences using your own brain (regardless of how imperfect they may be), as opposed to relying on a tool to produce them, and trying to edit that output.

For scenarios where the point is not to learn something or to train a specific skill, but rather to produce some output (like a program that serves some purpose), using AI tools makes much more sense. And industries like the software development industry are gradually changing to shift the role of humans from writing code to more advanced things like deciding what the code should do. 

In any case, it will be fun to see where all this AI stuff goes. But for the time being, I would recommend doing language learning the “old fashioned” way.