KIT724-Assignment3-AR

KIT724-A3-AR » Devlog

2022-09-20 by weite

Share this post:

Share on Twitter Share on Facebook

1. Introduction

When learning a second language, the logographic system is a very difficult part, such as Chinese characters in Chinese and Japanese. This Augmented Reality (AR) application can help children learning logogram language as a second language easier.

2. Description of the application

The purpose of this application is to create a connection between the writing system and the real world so the language learner can understand the characters better.

Chinese characters are developed logograms used to write Chinese. They have also been modified to write various East Asian languages, and they continue to be an important part of the Japanese writing system under the name kanji(Wikipedia 2022). Hanja, or Chinese characters, are widely used in South Korean academics to examine the country's records, documents, history, and literature. Before switching to a Romanised alphabet, Vietnam first wrote Vietnamese using Chinese characters and then their own version.

Traditional learning methods could be boring because logograms are characterized by being unrelated to the pronunciation of the word they represent.

Figure 1 - Chinese characters(Arch Chinese 2022)

To help learn Chinese characters, some methods use pictures to enhance fun and efficiency. However, this brings another problem. The pictures are always present next to the text so the learners do not need to actively think and recall, instead, they passively receive this information.

Figure 2 - Chinese character with picture (Hanbridge Mandarin 2017)

To help improve the above problem, the traditional method is difficult to do an interaction to toggle the picture display. With the help of AR technology, such interaction can be easily made. It has been shown that using technology to enhance Chinese character learning and teaching is effective (Zhan & Cheng 2014). While an AR system having effectiveness for improving Chinese character learning performance has also been reported (Wen 2018).

Therefore, this application using AR technology presents a game-like method to learners especially for kids. It provides more attractive and effective way in learning Chinese characters.

3. Interaction Design

As mentioned above, traditional method can only display character only or with picture. If learners choose to learn with picture, the picture will always display next to the character, so learners have no space to actively practice their memory. This reduces the efficiency of learning as learner always able to see the answers.

With this AR application, learners can decide when to display the character’s corresponding 3D model, so before this model popup, they have time to actively recall what does the character mean and this enhance the learning efficiency.

After this AR app launched on mobile phone, users can point their mobile phone camera to the target if they cannot recall the meaning of the character. Then this AR app will pop up the 3D model of that character so learners can memorize the meaning.

This AR app also provide bilingual pronunciation of Chinese characters for learns to practice reading and listening. Apart from reading the character, the pronunciation is a difficult problem in traditional learning methods. In this app, when users checking the 3D model of character, they can click the audio button to play the pronunciation recording of the character.

Figure 3 - 3D model pops up when character captured

Storyboard

Upside: Traditionlly

Downside: With AR

With character only, leaner does not know the meaning if he is struggling	With character and picture, learner knows it too easy so no thinking and memorizing process
Using AR, learner can think first	Use AR app to get the answer

Figure 3 Storyboard

4. Technical Development

This AR application is developed with Unity(3D) with Vuforia and targeted to Android mobile phone. With Vuforia we can easily track the pre-provided images and display 3D object in real time. In addition, there is virtual button inside the application for interaction and this is implemented by extra script.

5. Descriptions of 3D Models

Three Chinese characters are chosen in the demo AR app and each character has a corresponding 3D model.

These three characters and models are ‘car’, ‘cat’ and ‘tree’.


car	cat	tree

Figure 4 Character 3D models

6. References

6.1 Academic articles

Zhan, H & Cheng, HJ 2014, ‘The Role of Technology in Teaching and Learning Chinese Characters’, International Journal of Technology in Teaching and Learning, vol. 10, no. 2, pp. 147-162.

Wen, Y, ‘Chinese Character Composition Game with the Augment Paper’, International Forum of Educational Technology & Society, vol. 21, no. 3, pp. 132-145.