Experiment: How good are Google Translate’s English to Japanese translations?

Back when I first started studying Japanese, I wasn’t aware of any translation programs that worked between English and Japanese, and if there was I’m pretty sure they were very expensive and not very commonly used.

Nowadays we have things like Google Translate, a free service that translates between over 50 world languages, including Japanese.

As someone who has spend a great deal of time learning Japanese, part of me gets a bit scared at the prospect of a machine translator that is even remotely close to that of a human. Could all my efforts go to waste?

I’m curious about both directions of translation (English->Japanese and Japanese->English), and although choosing my native English as the target language would be easier to evaluate, for this post I’ve decided to just focus on English->Japanese this time.

I’ve picked a single paragraph of my recent essay on conversation in a foreign language, as it is a somewhat complex passage that was written naturally, without any bias of writing it for the purposes of a translation experiment.

Let’s start with the English text. Here it is:

Of all of the activities we do on a daily basis, I feel that the act of communicating with another person using spoken language and a series of back and forth exchanges, what we call “conversation”, holds a very unique place. If you think about all the things involved in this process, and how closely it is linked with who we are, it’s pretty amazing.

Now for the Japanese text returned by Google Translate:

私たちが日常的に行う活動の全てを、私は話し言葉を使って別の人と私たちは “会話”と呼んでいるものを前後に取引所、一連の通信の行為は、非常にユニークな場所を保持していることを感じる。あなたがこのプロセスに関与に関するすべてのことを考えると、我々が誰であるか、それがとリンクされているどのように密接に、それはかなり驚くべきことだ場合。

If you are reasonably fluent in Japanese feel free to read through the Japanese text above and you’ll realize soon where this post is going.

For the rest of us, let’s start by breaking down the first sentence into a few phrases and see how they match up with the original English text. I’m not going to look for perfection, but instead focus on the overall grammar and see if Google Translate was able to comprehend the sentence.

Of all of the activities we do on a daily basis ====> 私たちが日常的に行う活動の全てを

All of the words used in this translated portion match roughly to the original English sentence, although some of them are quite stiff. For example, I don’t think 行うwould be the best choice here, though that’s a minor quibble.

However, there is a much bigger problem – A critical aspect of the grammar in the source text is missing. The first word “of” represents treating all the “activities we do on a daily basis” as a category from which something is chosen from, and this is not captured anywhere in the Japanese translation.

I would expect something like: “〜の中で”、”の内(で)”、or at least a ”〜から” somewhere, but nothing like this is present.

The translator clearly failed here to comprehend the English grammar. Let’s check the next phrase.

the act of communicating with another person using spoken language ====> 私は話し言葉を使って別の人と

The parts “using spoken language” and “another person” are translated in a way that is understandable. However the part “the act of communicating” got shifted to later in the sentence for some reason. Let’s look at that phrase next.

and a series of back and forth exchanges ====> を前後に取引所,一連の通信の行為は

We can see where these words came from in the original text, but the ordering and grammar is all messed up.

“back and forth exchanges” was translated to a semi-literal but awkward “を前後に取引”, and the second half of the above phrase says something like “the act of a series of communications”. This is very different from what I said in the English text because it is mixing up two phrases.

Also, the “and” from “and a series of…” is not represented anywhere. I would expect a に or と somewhere.

what we call “conversation” ====> 私たちは “会話”と呼んでいるもの

Though there are times “call” can be translated as “呼ぶ” in Japanese, but in this case the phrase “という” more appropriate. The resultant phrase here sounds decidedly like translated Japanese.

Grammatically I think が would be more natural here than は, and although the translated phrase itself is somewhat understandable, the もの here is followed by “を前後に取引所。。。” which makes absolutely no sense grammatically.

holds a very unique place ====> 非常にユニークな場所を保持している

The grammar here is relatively intact, but as usual the word choice is quite unnatural and the resultant phrase hard to understand. I would expect something like “位置づけ” to be used in a proper translation.

I feel that ====> ことを感じる

This part is also a bit unnatural but literally correct. I think using “と思う” or “ような気がする” would be more appropriate.
I’m not going to go over the entire second sentence, but like the first it’s a real train reck. First there is the odd phrase “関与に関する”, and the fact the phrase “and how closely it is linked to who we are” is translated into the wrong place (after the “if you think…” instead of before it). It ends with the extremely odd “だ場合”, and I have no idea where that came from.

I think this experiment clearly shows that Google Translate’s English->Japanese translation has a long way to go in terms of word choice, grammar comprehension (especially understanding how commas work), and odd bugs.

You could argue that the English used in the sample passage is a bit advanced (and arguably not that well written), and it’s true that Google Translate works a bit better with simpler grammar. However for a translate tool to be much use it must handle at least this level of complexity, and maintain at least the critical connections between the phrases involved.

So Japanese learners, you can relax knowing that least for the near future, your effort will not be for nothing. (:

Also, let this be a lesson that you should never use Google Translate to try and validate your own Japanese, or use it to get suggestions for phrasing or grammar.

Notes

1) Though Google is a massive company with a huge amount of funding, you could make the point they have put less effort into it because it is being given away for free. If anyone knows of any better translation services that I can try for free I would be curious to experiment with those. But given the complexity of understanding and generating written language I’m not expecting much progress anytime soon. I did try a few brief tests with Bing which was equally horrible, if not worse.

References

http://selftaughtjapanese.com/2015/01/28/the-art-of-conversation-in-a-foreign-language/

https://translate.google.com