A Foreign Language Learning Application Using Mobile .
Informatica Economică vol. 20, no. 4/201676A Foreign Language Learning Applicationusing Mobile Augmented RealityFlorentin - Alexandru DIȚĂBucharest University of Economic Studies, Bucharest, Romaniaalexandru.dita@csie.ase.roIn this paper is described a foreign language learning application using mobile augmentedreality based on gamification method and text recognition. The mobile augmented reality is atechnology that extends the real world elements with 2D or 3D computer generated objects andlets the users interact with them. A Gamification system is based on different mechanisms thatincrease the motivation of students, due to the impact that videogames have in their emotional,cognitive and social areas. The proposed solution applies Optical Character Recognition technique, using the camera of the mobile device, in order to identify the text written on a card. Theimplementation combines the features of gamification system and mobile augmented reality inorder to make the learning process more easy and fun. This paper aims to present the resultsafter testing the foreign language learning application in different scenarios.Keywords: Gamification, M-Learning, Mobile Augmented Reality, Optical Character Recognition, Virtual Object1IntroductionNowadays, the education institutes already use in the process of learning the advantages of mobile learning technology, suchas portability (the students have easy access atany time to information), the use of digital assets (audio and video files) and huge amountof textual information. This study proposes anew system approach for improving the learning process by using Mobile Augmented Reality. The features of this technology stimulatethe learning environment and consider that thestudent's motivation is the most important factor in learning.The existing literature [1] highlights two different approaches for augmented reality (AR).Firstly, the authors consider that AR is a system which has the following characteristics:Combines real and virtual; Interactive in realtime; Registered in 3-D. Secondly [1] considers Augmented Reality part of Virtual Continuum. This context is composed of two elements: Real Environment - which contains allrepresentations from real life and Virtual Environment - which combines different elements from virtual life. ARs are consideredcloser to reality than virtual environment because this type of systems are supposed to extend the real objects with virtual elements.According to [2] Augmented Reality is a technology that extends the real world elementswith 2D or 3D computer generated objectsand lets the users interact with them. Also, theauthors present three types of devices whichcan be used to view the result of AR technology: a Head Mounted Display (HMD), a mobile device - with at least one back camera anda desktop or a laptop computer.Similarly, [2] considers mobile learning technology as an alternative teaching resource because the users have easy access in real timeto information. Also, using mobile devices inthe learning process two main problems of traditional learning are solved: the location andtime. Likewise, [1] considers that mobilelearning is a type of eLearning without restrictions of time and space and this technology improves the quality of classical learningenvironments, but does not replace them.Other researches, such as [3] and [4] highlightthe benefits of using a Mobile Augmented Reality system based on tracking objects. In [3]is proposed a mobile augmented reality system based on visual recognition. According tothe authors the visual recognition consists intotracking objects identified in real world withthe camera of the mobile device and augmenting the scene with virtual objects created byMAR technology. Similarly, [4] presents aDOI: 10.12948/issn14531305/20.4.2016.07
Informatica Economică vol. 20, no. 4/2016Mobile Augmented Reality system bases on3D registration technology. According to theauthors the 3D registration represents thetracking of the position and pose of the mobiledevice in the real time and real life and finallyuse these information to insert in the realworld the virtual scene created by MAR. The3D registration technology is based on twotype of approaches: markers and natural features. Recently, the 3D registration for MARuses GPS and sensors from mobile devices toprovide different information about positionand pose of the camera and combine themwith computer vision to improve the speedand accuracy of the system.Other studies, such as [5] propose a makerbased Mobile Augmented Reality solution.The application is used by a Vocational Educational and Training (VET) program for supporting the maintenance process of a car. Theauthors define VET as "education programsthat are designed for learners to acquire theknowledge, skills and competencies specificto a particular occupation, trade or class of occupations or trades"(p. 49-50). Moreover, in[5] is presented an AR application used byVET students for basic mathematics skills,which is based on videogames.More studies, such as [6] present a MobileAugmented Reality solution for learningFrench language which is based on gamification didactic method. The author, BernadettePerry, describes gamification as “using gamebased mechanics, aesthetics and game thinking to engage people, motivate action, andpromote learning” (p. 2309). [6] suggests thatthe features of MAR technology, such as 3Dregistration or visual recognition, improve thebenefits of gamification method.Moreover, the authors of [7] consider that“gamification is the use of the game design elements in non-game contexts” in order to improve the children's study motivation and givethem the opportunity for experimenting insafe environments.In this paper is proposed a mobile augmentedreality system for learning a foreign language,based on text recognition and gamification didactic method. The first step of the implemented solution is to identify the text writtenDOI: 10.12948/issn14531305/20.4.2016.0777on a card, applying Optical Character Recognition technique, using the camera of the mobile device. Afterwards, the identified wordwill be displayed on the device's screen, augmented with virtual 2D objects. Finally, theapplication allows the pronunciation of theobject spelling in a language which is selectedby the user. The target users of the prototypepresented in this paper are the children between 5 and 10 years old that want to learnnew words from one of the available foreignlanguages. From an economical point of view,the proposed solution offers the benefit of reducing the additional costs, which werecaused by the buying of different books orlearning materials, in order to start the learning process.The paper is organized as follows. Section 2analyzes the architecture of the proposed application. The implementation of the prototype is shown in Section 3. Section 4 presentsthe results obtained by using the solution anddescribes the future work.2 Application ArchitectureThe purpose of this paper is to present a foreign language learning application using mobile augmented reality based on gamificationdidactic method and text recognition. According to [6], 35 is the average age of the gamersand 97% of adolescents play video games.Moreover, as stated in [8] a gamification system increases the motivation of pupils, due tothe impact that videogames have in their emotional, cognitive and social areas. Consequently, learning a foreign language becomesmore easy and fun using a gamification approach.The text recognition system is needed, considering that the inputs of the application consistin cards containing words from different categories. This approach is based on applying asimilarly Optical Character Recognition(OCR) technique, such as [9] to detect thewritten text, and an algorithm to match the results of OCR with available words.The proposed architecture for foreign language learning application using mobile augmented reality, is shown in Figure 1.
78Informatica Economică vol. 20, no. 4/2016Fig. 1. Architecture of foreign language learning application using mobile augmented realityThe input of the application consists in already prepared cards containing words fromdifferent categories, such as: animals, fruits,vegetables. The current solution supportstranslations from Romanian in one of the following languages: English, French and Italian. The mobile solution accesses the devicecamera to get frames with the cards. For everyidentified frame the text recognition process isapplied in order to detect the written word. After the detection process is successfully finished, the results are sent to the gamificationprocess. Firstly, in this stage, the applicationchecks if the corresponding virtual objects ofthe previous identified words are alreadysaved in the local database, before augmenting them on the device's screen. In case theyare not found, the application calls the RESTService in order to get the virtual objects fromthe global database. According to [3] a RESTsolution is a service or an application that canbe used by any device which executes HTTPcalls. The virtual objects, which are returnedafter querying the REST Service, are thensaved by the application in the local database.Secondly, the gamification process uses thecorresponding virtual objects that have beenstored in the local database in order to drawthem on the device's display.The REST Service contains methods that allow to retrieve the following needed information: the available categories and the storeddata on the existing words. This data represents the virtual objects that are used in thegamification system and consist of: the corresponding 2D images of the available wordsand the translations of the Romanian texts inEnglish, French and Italian. The global database contains five tables in order to managethe functionalities of the presented mobileaugmented reality application. On the otherhand, the local database defines two tablesthat store the virtual objects of the selectedcategory. The records from this table are replaced every time when the selected categoryis changed in order to reduce the needed spaceon the mobile device.3 ImplementationThe mobile augmented reality application isdeveloped with Java programming languageusing Android SDK and tess-two library.Tess-two is an open source library based onTesseract OCR Engine and Leptonica Engineprocessing image which permits words identification from different languages. The currentapplication is composed of two systems: TextRecognition and Gamification.3.1 Text Recognition SystemText Recognition system is based on an engine class which uses the tess-two library inorder to get the written text and an algorithmfor matching them with available words fromthe selected category. The matching algorithmDOI: 10.12948/issn14531305/20.4.2016.07
Informatica Economică vol. 20, no. 4/2016consists in checking if a least 80% of the characters from a text returned by the OCR engineare identically and in the same order withcharacters of an available word from the selected category.In the OCR engine method, which communicates with the tess-two library, two types ofvariables are defined in order to speed up theprocessing done with the OCR technique. Thefirst type, called VAR CHAR WHITELIST,79contains the available characters of the detection process from the used library. The secondtype of variable is the VAR CHAR BLACKLIST which defines the unavailable charactersof the detection process. Also, the applicationchecks if the difference between the length ofthe text detected by the recognition engine andthe length of an available word from a selectedcategory is -1, 0 or 1 before calling the checkWord() method. In Figure 2, a workflow of theText Recognition System is presented.Fig. 2. Workflow of the Text Recognition SystemThe next listing shows a part of the checkWord() method that implements the matchingalgorithm.Listing 1. Matching algorithm methodprivate boolean checkWord(String response, String word) {.for (int i 0; i response.length(); i ) {for (int j 0; j word.length(); j ) {if (response.charAt(i) word.charAt(j) &&(positionResponse -1 positionResponse i)&& (positionWord -1 positionWord j)&& frequency[j] 0) {positionResponse i;positionWord j;no ;frequency[j] 1;break;}}}int flag (int) (0.8 * word.length());return (no flag);}DOI: 10.12948/issn14531305/20.4.2016.07
Informatica Economică vol. 20, no. 4/201680The checkWord() method uses the variablespositionResponse, positionWord and frequency in order to check if the characters ofthe detected text are in the same order as thecharacters of an available word from the system.3.2 The Gamification SystemGamification System is based on a controllerclass which uploads the virtual objects corresponding to the written words detected by theText Recognition System from the database,an engine class which draws them on the mobile screen and a translation system whichuses Text-To-Speech (TTS) and the languagechosen by the user.In the first section of the Gamification Systemthe user has to select one from the followingthree types of foreign languages in which thedetected word is translated: English, French orItalian.The uploading process of the virtual objectsconsists in two steps: firstly, the system searchthem in the local database, which is installedon the mobile device. Secondly, if the resultof previous querying is empty then the systemgets the virtual objects from the global database using a REST Service.In the next step, if the uploading process issuccessfully finished, the Gamification System uses methods of the drawing engine in order to augment the virtual objects on the mobile device. This part includes the creation ofthe 2D image, the calculation of its coordinates in the drawing area and the visual components which are used for starting the translation (microphone image) and changing theobjects between them (left right arrows).Finally, the translation system consists inmethods that are responsible for taking overthe word from the displayed virtual object,based on the language that is chosen by theuser before starting the uploading process, anda method that uses the Google Text-to-SpeechAPI functionality in order to pronounce thetranslated word.In Figure 3, is presented a workflow of theGamification System.Fig. 3. Workflow of the Gamification SystemThe implementation of the Gamification System is composed of the following three components: the interaction with the local database; the drawing of the stored virtual objects;the communication with the REST Service;the translation process with TTS.DOI: 10.12948/issn14531305/20.4.2016.07
Informatica Economică vol. 20, no. 4/2016The first part consists in using a SQLite Database in order to store and retrieve the virtualobjects from the REST Service API. The component implements a controller class, whichhas the necessary methods for creating theSQLite Database. This is derived from the abstract class SQLiteOpenHelper. According to[10], the SQLiteOpenHelper contains methods that allow to create the database and obtains an instance of the SQLiteDatabase class.The previous class includes specialized methods for adding, deleting and updating records.Also, the SQLiteDatabase exposes methodsthat permit direct execution of SQL commands such as: execSQL(), used for the insertcommands and rawQuery(), used for selectcommands. Moreover, the component contains a second class that defines the methodsfor data manipulation using the presented controller class and an instance of theSQLiteDatabase class.81The component that does the drawing of thestored virtual objects is based on a game viewengine. This extends SurfaceView class whichallows to augment on the mobile device'sscreen different elements, such as: pictures,texts, buttons, lines, rectangles etc. The virtualobjects are encapsulated in a class that has thefollowing fields: a Bitmap that contains asprite image, the width and the height of theimage, the number of rows, the number of columns and the position where the virtual objectmust be drawn. A sprite is a static image or ananimated graphic that is used to create 2D videogames. The game view engine overrides theonDraw() method from SurfaceView in orderto draw on the mobile screen the virtual objects based on the previous class. The Listing2 shows the implementation of onDraw()method.Listing 2. Drawing of a virtual objectprivate void onDraw(Canvas canvas) {canvas.drawColor(Color.TRANSPARENT, Mode.CLEAR);Paint paint new etTextSize((float) (canvas.getHeight() * 0.15));paint.setTextSkewX((float) -0.1);canvas.drawText(word, (float) (canvas.getWidth() * 0.42),(float) (canvas.getHeight() * 0.8), paint);}//onDraw() method of the sprite elementpublic void onDraw(Canvas canvas) {if (BMP COLUMNS 1) {update();int srcX currentFrame * width;int srcY sourceY * height;Rect src new Rect(srcX, srcY, srcX width, srcY height);Rect dst new Rect((int) x, (int) y, (int) x width, (int) y height);canvas.drawBitmap(bmp, src, dst, null);} else {canvas.drawBitmap(bmp, x, y, null);}}private void update() {.currentFrame currentFrame % BMP COLUMNS;}The third component of the system is represented by an AsyncTask class which executesDOI: 10.12948/issn14531305/20.4.2016.07HTTP calls over the REST Service in order to
Informatica Economică vol. 20, no. 4/201682get the required virtual objects. The AsyncTask class implements the doInBackground()method that interacts with the REST Servicefunctionalities. Firstly, this method executes aHTTP request to the service in order to get thevirtual objects from a specified category. Theresponse of the REST Service call is a JSONobject that contains the needed information.Secondly, the doInBackground() method usesthe GSON library in order to transform theJSON object into a corresponding Java class.Listing 3 presents a part of the REST Serviceengine.Listing 3. The communication with the REST Servicepublic class HttpRequestTask extends AsyncTask String, Void,ArrayList ObjectARJson {URL url null;HttpURLConnection connection null;@Overrideprotected ArrayList ObjectARJson doInBackground(String. params) {StringBuilder sb new StringBuilder();ArrayList ObjectARJson list new ArrayList ObjectARJson ();try {url new URL(params[0]);connection (HttpURLConnection) url.openConnection();InputStream is connection.getInputStream();InputStreamReader reader new InputStreamReader(is);BufferedReader br new BufferedReader (reader);String linie null;while((linie br.readLine())! null){sb.append(linie);}GsonBuilder gsonBuilder new GsonBuilder();Gson gson gsonBuilder.create();ObjectARJson[] vectorAR for(int i 0;i vectorAR.length;i )list.add(vectorAR[i]);} catch (MalformedURLException e) {e.printStackTrace();} catch (IOException e) {e.printStackTrace();}return list;}}The REST Service is developed with Javaprogramming language using SpringBootframework in order to create a Web application based on a REST architecture. The RESTService is composed of methods that allowcommunication with an Oracle Database, thatstores information about the virtual objects.The last element of the Gamification Systemis a translation process based on Text-ToSpeech (TTS) functionality. According to[11], the core functionality of TTS is "textprocessing that functions as the producer ofsyllabic speech units to be used in the generation of human-like speech". This process contains two methods for translating the identified words by Text Recognition OI: 10.12948/issn14531305/20.4.2016.07
Informatica Economică vol. 20, no. 4/2016The first method creates a HashMap objectwhich contains the translations of the displayed vir
A Foreign Language Learning Application using Mobile Augmented Reality Florentin - Alexandru DIȚĂ Bucharest University of Economic Studies, Bucharest, Romania alexandru.dita@csie.ase.ro In this paper is described a foreign language learning application using mobile augmented reality based on gamification method and text recognition.
Sparks, R. (2016). Myths about foreign language learning and learning disabilities. Foreign Language Annals, 49 (2), 252-270. Sparks, R. (2009). If you don’t know where you’re going, you’ll wind up somewhere else: The case of “foreign language learning disability.” Foreign Language Annals, 42, 7-26.
American Sign Language is offered and accepted as a foreign language to fulfill high school and post-secondary language requirements. New Hampshire American Sign Language as a foreign language has never been proposed to the state legislature. However, some high schools and colleges offer American Sign Language and give academic credit.
I. Foreign Exchange Rate Determination The foreign exchange rate is the price of a foreign currency. As any other price, it is determined by the interaction of demand and supply for the foreign currency (FX). -FX is demanded to buy foreign goods and services (imports), and to buy foreign financial assets (capital outflows).
Media Articles Business Week (19 Feb 2007 issue) Doctrine: Journal of General Military Review, Issue 3 Foreign Policy in Focus, “Prospects for al-Qaeda” (24 Jan 2003) Foreign Policy (Jan-Feb 2008) Foreign Policy (March-Apr 2008) Foreign Policy (May-June 2008) Foreign Policy (Nov-Dec 2008) Foreign Policy (Sept-Oct 2008)
Contents—Continued E. Command Language Program, page 37 F. Guidelines for a Successful Command Language Program, page 39 G. Training Resources, page 41 Table List Table 2–1: Language proficiency indicator, page 2 Table 6–1: Foreign language proficiency bonus payment levels, page 15 Table 6–2: Department of the Army Civilian foreign language proficiency pay payment table, page 18
The foreign tax credit can also be claimed by taxpayers living in the United States for taxes paid to foreign countries on income earned from foreign sources. In 1987, over 661,000 taxpayers claimed foreign tax credits of about 1 billion. IRS could not tell us how many persons taking foreign tax credits lived in foreign
Foreign investors should familiarise themselves with Australia's foreign investment framework and ensure they comply with the law. Failure to do so may result in the imposition of penalties. The foreign investment review framework The foreign investment review framework is set by the . Foreign Acquisitions and Takeovers Act 1975 (the Act) and the
Zoo Animal Nutrition III (2006) was edited by A. Fidgett, M. Clauss, K. Eulenberger, J.-M. Hatt, I. Hume, G. Janssens, J. Nijboer. Filander Verlag, Fürth ISBN-10: 3-930831-57-0 ISBN-13: 978-3-930831-57-9 To obtain a copy of the book, contact Filander Verlag at info@filander.de BIRDS Schoemaker, N.J. Some diet-related problems seen in birds 1 Ghysels, P. Transferring birds to pellet feeding 1 .
