Developing English Vocabulary Software

Abstract

One approach to the problem of fostering English vocabulary expansion in non-native students is the development of "user-friendly," game-like computer software. This paper traces the three-year development process of VOC software through its two major phases, from the original "fill in the blank" style, where the user must provide a Japanese equivalent for a given target English word in a natural example sentence, to an easier to use, multiple choice format, in which many words in a given sentence are potential targets. Various functions of the user version of the program are described, as well as features of the authoring system, which allows users to expand the basic data set. Problems of development and aspects of user feedback are presented. The development of such software provides an example of a technological solution to a language-learning problem.

Contents

1. Introduction
2. VOC I

2.1 Design
2.2 The development of the software
2.3 The development of the data
2.4 Authoring System
2.5 Note and Help screens
2.6 Data Problems
2.7 User Testing
3. VOC II

3.1 Design
3.2 The development of the software
3.3 The development of the data
3.4 User Testing
3.5 Authoring Functions
4. Conclusion

Research and development funding for the VOC software was provided by a Research Grant from Sanno Junior College for the years 1991-1993. The authors gratefully acknowledge this assistance, without which the progress of our project would have been severely restricted.

1. Introduction

2. VOC I

2.1 Design

We began with the idea of making a vocabulary game. We reasoned that the more enjoyable the experience, and the less textbook-like the software, the greater the potential for vocabulary development. In simplest terms, the core idea of the program is that the user sees an English sentence and is required to give the meaning of the selected word in that sentence.

The target vocabulary can be selected according to difficulty level using various published frequency lists. We present the words in the context of natural spoken English sentences, gotten by transcribing television news shows and similar sources.

The sentences are grouped into categories by subject matter. The user selects a category and the first sentence appears on the screen. The target word is in a different color from the rest of the sentence, and there are "hints," in a third color. Hints are words which the user can see the meaning of by "paying points". The point value of the target word goes down each time a hint is used.

When the user feels s/he knows the meaning, s/he types it in Japanese. If the answer is correct, points are earned. After answering, a Japanese translation of the sentence appears, along with an expanded meaning of the target word, including grammatical comments, and the meanings of all the hints.

One "round" consists of 10 words, in 10 different sentences. At the end of a round, the user gets the score. Other functions let a user see all the previous words missed, a summary of previous rounds, etc.

In tandem with the game development, we developed an authoring system to allow addition of new target words, sentences, and hints, as well as various additional ways of monitoring user progress, viewing the data, and analyzing the system as a whole. We refer to this authoring system as the "Teachers' Version," and the user game as the "Student Version."

2.2 The development of the Software

Once we had our design criteria, we began development of the software. The target machine was the NEC 9800 series DOS computer. The programming language, Clipper, has the advantage of permitting simultaneous development for both IBM compatible machines and the NEC 9800 series machines - the same program can be compiled to run on either machine. This permitted testing the program on a faster IBM-type machine during development. It also means that the program can be compiled to run on IBM compatible machines operating under the DOS/V operating system, which permits kanji character display and input.

Clipper is one of the so-called xBASE dialects of programming languages which developed from the dBASE programming language. The original dBASE language, however, was an interpreter, while Clipper was designed from the outset as a compiled language, resulting in acceptable speed and functionality for application development. An advantage of using such a language for application development dependent on large data sets is the capability of retaining compatibility with the dBASE database structure. As a result, we were able to use dBASE III to directly modify data outside of program functions, a feature which speeded and facilitated development.

In simplest terms, the navigation system we implemented for the program is via a set of moving light bar menus. This type of menu allows the user to navigate the menu by means of arrow keys, and then to select by hitting the  <Return> key. Optionally the first letter of any menu item can be selected. We made all menus, help messages, and on-screen comments bilingually: the user can select from a setup menu which mode, Japanese or English, the menus and comments will be displayed in. The moving light bar menu system requires almost no learning curve on the part of the user, but we attached help screens to all parts of the program to remind the user of the result of any of the choices made.

The major programming problem for the core of the program, the sentence with its hints and target words, was to make an easy user interface. We present the user with a sentence at the top of the screen, with hint words highlighted in one color, target word in another, and the remainder of the sentence in a neutral color. Using the left or right arrow keys, the user can then move through the sentence easily, jumping from hint to hint to target, to the word of their choice. If a hint word is selected, and the key pressed, the meaning of that word in Japanese is displayed beneath the target sentence. If the target word is selected, the user is presented with an input box for the correct answer. At this point the user must switch the keyboard to kanji input mode. One negative result of this feature was that the control of the kanji display and conversion was outside program control, sometimes resulting in unaesthetic screens.

The program required the development and storage of large quantities of various types of data. Each sentence had to be input with its associated hints and target words marked in such a way that the program could display and respond to them appropriately. Every sentence had an associated Japanese translation, and of course each hint and target word had a mass of associated data: definitions, parts of speech, difficulty level, source, grammatical notes, etc. We used a set of relational databases to store this data, including a user database for storing passwords, user-selectable modification choices, score-keeping, etc. The quantity of data is essentially limited only by the capacity of the hard drive on the user machine.

To facilitate the input and development to this data, various functions were designed, many of which eventually became part of the authoring system. Often features which we initially considered to be development tools turned out to be useful authoring functions as well, and so we included them in the Teacher's Version.

For example, as the number of target sentences increased, naturally the number of target words increased correspondingly. We found it necessary to include a function for displaying all the target words as a scrolling list which could be arranged alphabetically or according to the semantic groups of the sentences they belonged to. From this list we could jump directly to the editing screen for the associated sentence, where all of the input is done (hints, translations, etc.), by scrolling to the word and pressing the key.

Similarly, as the number of sentences in the database increased, it became necessary to have a convenient way of verifying that all the words were spelled correctly. We developed a function to permit output of all the data to an ASCII file, on which a standard English spelling checker could be run. The corrected file could than be imported back into the program with all the corrections intact. The development of this function led us to incorporate a modified version into the Teachers' Version, allowing input of sentences directly from a user file which had been modified to signal target words, hints, etc., using fairly simple formatting code symbols.

An additional user function which developed from a development tool was the capability of inserting a note or memo anywhere in the program without exiting. These notes could be viewed or edited at any time, and sorted alphabetically or by date for searching. We used this function for communication during the development cycles, but found that it was useful not only as a personal memo system, but as a potential communication tool between teacher and student. We used it extensively during the testing cycles with student users.

2.3 The development of the Data

While the input of large quantities of data is a tedious process, this turned out to be one of the most interesting and exciting parts of the development. As English teachers, selecting target sentences, target words, hint words, making notes and adding other example sentences is a normal activity for class preparation, something we are well familiar with.

For VOC, the target sentences were selected from newspaper articles (The Japan Times, Asahi Evening News), and transcriptions of TV news (ABC World News Tonight) or FEN news. Selection of the target words was based on the word frequency analysis lists in Shirano (1989) and the Genius English-Japanese Dictionary (1988). Shirano recorded about 11 hours of FEN news, transcribed it, entered the data into a computer, and found the most frequently recurring words. He then excluded from his list the "fundamental" words, which are expected to be learned from Junior High School textbooks (1628 words), and proper nouns which are considered familiar to students (1000 words). His resulting list contains some 1800 words and their derivatives, and he claims that these words, together with the "fundamental" words and familiar proper names, should cover approximately 94% of the vocabulary used in the news. The 1800 words in his list range from "Genius" Level B (High School fundamental words) and Level B' (essential for University entrance exams), to Level C (University level). The words found in Shirano's list were selected as "target" words, and arranged according the "Genius" levels. Other words considered to be difficult or important for college students were made into hints. Included with the target words and hints were derivatives, antonyms, synonyms, example sentences, and grammatical explanations, wherever we felt they were necessary or useful.

We grouped the sentences into categories based on their subject matter. When the user elects to "Play a Game" from the opening menu, a second menu appears containing the categories. The following categories were used:

1. Politics and Elections
2. World Affairs
3. Business and Economy
4. Justice and Crime
5. Science and Medicine
6. Society and Culture
7. Accidents and Disasters
8. Sports
9. Miscellaneous

Once the user selects a category, the opening game screen with the target sentence, hints, and target words, appears.

Since VOC I requires the user to keyboard in the meaning of the English target word, it was necessary to input all the possible correct responses that a user might make. This was a troublesome and rather unrewarding phase of the data development, and one whose total success was questionable. It was necessary to try to imagine all the possible responses that a user might make, and input them accurately as a "search base" for each target word. Even a good answer would be regarded as wrong by the program if the word were not registered in the search base, or if it were recorded slightly differently. For example, a correct answer typed in hiragana would be considered wrong if only the kanji were listed in the search base. "Close" answers are as wrong as completely off-base answers. Consider the following target sentence (#71), in which "expect" is the target word:

"On the other hand, more than half the respondents said they would expect change in the government."

We initially made a search base consisting of "[various Japanese words more or less meaning 'expect']" But on deeper reflection, one must consider as well, "[less common words for 'expect', etc.]" And all the above in hiragana are possible. The prediction of all possible correct answers proved to be one of the major drawbacks in developing a system where the user response was typed in directly at the keyboard. One device we developed to help with this problem was to record all the answers that users made. Afterwards, a manual review of user responses could reveal additional "correct" words which could be added to the search base.

2.4 Authoring System

The opening menu for the Teacher's Version (Authoring System) differs from that of the normal Student Version in the inclusion of additional items. It contains the following selections:

R-un a game
A-dd / edit questions
L-ist questions
N-[ihongo (in Japanese)]
M-aintenance
R-eports
E-xit

Add / Edit Questions

The primary authoring selection is the "Add/Edit Question" selection. Selection of this item brings up an input screen on which the author can input a new question. Items for hints and target words are marked by the use of special symbols, (such as "< >" to enclose an intended hint word), and a unique number is automatically assigned to each question to provide a link for the related databases, indicate the question type, level of difficulty, and the question group. The screen allows the input of all possible correct user answers, as well as the space for translation.

Items on the screen appear in small data windows, which can be edited using minimal word-processing functions. Data within each window can be scrolled, and a word-wrap feature is implemented. Variations conventions are employed to allow the author to control certain aspects of the appearance of the question on the game screen, such as forcing words to a new line.

A set of navigation and function tools are available to the user from the Add/Edit Screen. Pressing the  <Roll-Up> or  <Roll-Down> key moves the user to the next or previous (numerically ordered) question. The  <Esc> key calls a function menu:

RE-EDIT question
EXIT to top menu
TEST display question
SUMMARY display
WRONG answers entered
LIST question words
GROUP count display
MAKE a copy of question
CREATE new question
DELETE this question
NEXT question
PREVIOUS question
JUMP to question #:

As with the other menus, display can be in either English or Japanese characters, navigated by moving a highlight bar up and down the menu, or pressing the first letter of the line.

List questions

Selection of this item on the opening menu (or the List question words item on the authoring menu), presents the user a screen which provides an overview of all the target words, the vocabulary used for all the questions. The screen is an alphabetical list, which can be rearranged numerically at the press of a key. From this scrolling menu, selected words can be displayed in their full sentence forms, or the Add/Edit screen appropriate to the word can be selected to provide editing capability. Question numbers can be changed from this screen as well.

Maintenance

Selection of "Maintenance" from the opening menu of the teacher edition brings up a secondary menu with the following items:

A-dd / Edit Messages

This is a development function, used to facilitate the addition of Japanese message translations into the program. All messages and menus are "data-driven". That is, they are stored as data in a database, permitting them to be easily added and edited. After the development phase, this function is no longer used.

I-nput from File

This function allows data for new questions to be imported from an ASCII file, written with a normal word processor. The user is prompted for a file name, and the process is automatic. Questions with existing numbers will be updated, those with new numbers will be appended. A backup file is automatically created, allowing the reinstallation of the previous data set in the event of an error.

W-rite VocNotes.DOC

This function creates an ASCII file containing all the user added notes, arranged in chronological order.

P-rint to VOC.DAT

This function creates an ASCII file containing all the current questions. The file can be edited, checked with a spelling checker, etc., and then reimported using the above Input function.

R-estore pre-input file

This is the safety feature to prevent accidental destruction of data from a badly constructed input data file. It automatically restores the pre-input database of questions.

S-peed

Allows selection of an option for displaying the target sentence as if it were being typed onto the screen. The speed at which letters appear can be user-selected, allowing a kind of "speed-reading" functionality. Also, the speed at which the dropping hints fall after they are selected can be user-modified..

C-olors

This allows user customization of the Game Screen colors. Background, Hints, Selected Hints, Target Words, etc., are all indicated by different color combinations, which can be tested when this item is selected.

E-xit

Returns to the Opening Menu.

2.5 Note and Help Screens

Various function keys call up windows from any screen. The  <F2> key calls up a window in which the user can enter notes with minimal word-processing capabilities.

<F3> brings up a scrolling note list menu containing the first part of the text of all the current notes. These can be displayed fully by pressing  <Return>, at which point they can be read or edited. Notes can be searched alphabetically by pressing the first few letters, or arranged chronologically. They can be deleted by pressing the  <Delete> key. Exiting the note menu from a note which begins with a game question number exits to the corresponding question on the Add/Edit screen or List Screen. Note questions beginning with that number are automatically changed as well.

<F4> allows the author to search for any occurrences of a word in the questions or hints. The result is a list of all question numbers containing the word. This list of numbers can be automatically returned to the screen by pressing  <Shift+F4> to facilitate searching.

<F1> provides context sensitive help. Depending on where the user is when  <F1> is pressed, a help screen appropriate to that function will appear. Additionally, minimal help for usual functions appears at the bottom of most screens.

2.6 Data Problems

During the process of input and debugging we encountered many problems. Perhaps the most serious one was the recurrence of bad data: Data which had been input correctly was displaying in scrambled form. When we checked the database, we found that the data had somehow been changed. Our development took place on two, and sometimes three different types of machines, IBM-compatible, Epson, and NEC. Data was input under direct program control, via the Add/Edit screen, as well as by direct manipulation of the database using dBASE III+. Sometimes the data was manipulated under a non-kanji operating system, or in non-Japanese mode. All of these variable features had to be sorted out and examined in the search for the solution to the data-scrambling problem.

In fact we discovered more than a single source for the dilemma. However, the underlying consistent feature was the storage of kanji characters. The fact that kanji characters take up two spaces on the screen is not aesthetically motivated. Rather, kanji characters are represented by the codes for two characters together, and stored as a sequence of code numbers. This is one of the reasons why most word-processors have difficulty displaying half-space English characters and double-space kanji characters together on the same line. It turned out to be one of the major sources of our problem. We discovered that when we combined English and Japanese characters in a sentence, as in notes, or translations, the spacing was a critical factor. That is, all Japanese input had to be stored in exactly even-numbered quantities, with the character beginning at the start of the sequence of two spaces. In addition, we found that the English version of dBASE was unreliable for saving or copying Japanese data, although it initially presented it correctly on-screen. This meant that we had to devise and program "safe" data-input systems which automatically compensated for the varying lengths of Japanese and English input, and successfully modified the spacing.

Another difficulty we encountered dealt with "two-word" targets, such as "according to". There were not only aesthetic considerations of screen display, but the more difficult problem of how to deal with such words when they were separated by non-target words, like "give... a hand". Our solutions required inputting such hints and targets with additional special coding on the Add/input screen, so that the program could recognize them and display/handle them appropriately.

Long sentences with many hints and explanations provided another problem. There was not enough room on the 24-line screen to display the sentence and all the additional information. We had to devise a system to allow the user to scroll through the hints, so that they could be used as an aid to solving the question.

2.7 User Testing

As development reached its final stages, we began testing VOC I on Sanno work-study students. Up to the point when the system operation was stable, we had exclusively tested it ourselves. The feedback we received was interesting and useful, but at the same time, some of the results were disturbing. There were basically three types of user response:

1. "In question #208, I answered [in Japanese], but it was wrong. Was it really wrong?" "In #001, my answer was [in Japanese]. Why was it wrong?" "In #062, why was [in Japanese] wrong?" These kinds of questions/complaints were frequent and recurring. Actually, most of the answers were acceptable. Just as we had feared, the number of possible Japanese "equivalents" for a given English word is forbiddingly large. Whenever students' "good" answers were rejected by the program, they felt frustration, and eventually found the software uninteresting.

2. Some students wrote: "When I don't know the answer, isn't there a way to see the answer?" "When I don't know what to answer it takes too long, because I don't know what to do." What they could have done, what we intended for them to do, was to draw hints, take a guess, and give any answers at all until their points were exhausted. But the students didn't want to lose a game, nor to make any stupid answers. So they felt they were stuck.

3. Many students mentioned that if they could see the pronunciation marks for a word, it would be very helpful. We had been thinking about that even before the students' comments, and made even greater efforts to find a solution. Unfortunately, this problem remains unresolved. Because we use the half-width character set for English letters, and would like to have similarly attractive phonetic symbols, we are temporarily stymied. We have not found any way to create half-width characters which we can display on the screen.

As a result of the user feedback, we reluctantly decided to redesign the game. The major problem was that users faced with typing an answer sometimes "just didn't know what to say," and so the experience became stressful rather than enjoyable. An additional problem was that they tended not to use the hints. Since use of a hint incurred a penalty, and took time, the user was not encouraged to learn the additional words the hints provided. Typing the answers in kanji required some knowledge of word-processor-like skills, and sometimes resulted in unattractive screen displays. The user's often typed answers that were essentially correct, but which hadn't been included in our database. VOC I was put aside, and we began development of VOC II.

3. VOC II

3.1 Design

We decided to implement VOC II as a multiple-choice version. In this way a user who didn't have any idea of the right answer could simply guess at one of the four choices. We solved the hint problem by having the hints become target words themselves. Now a single sentence could have many target words highlighted, and the user could arrow through them and view the various choices before selecting which one to respond to. Eliminating kanji user-input had the additional advantages of simplifying the use, eliminating the unpredictable screen displays, and, to some extent, simplifying our development data input process. On the other hand, it involved devising a new multiple-choice system, and selecting and inputting a substantial quantity of new data.

3.2 The Development of the Software

A major problem resulted from the implementation of the multiple-choice format of VOC II. The program grew larger, and required greater memory to run. We tried a variety of technical solutions, basically numerous overlay schemes during compilation, but machines with small (640K) memories still experienced run-time errors when certain sequences of the program were called. This situation has still not been satisfactorily resolved, but we found that by upgrading the operating system to DOS 5.0 (or higher), the advanced memory handling facility of that operating system solved most of the problems when the user machine had at least 1 megabyte of memory. A different way of dealing with the problem is the separation of the user (student) version from the teacher (authoring) version. When we tried this, we found virtually no problems in running the (smaller) student version. However, since the authoring version includes a full-fledged student version within it, the problem remained. A similar path may be to remove some of the functions, and in fact, some of the development functions may not be particularly valuable in the final authoring version, and so we can save some memory in that area as well.

3.3 The Development of the Data

Conversion of the program from single target words to multiple target resulted in an increase in the number of target words from our original 315 to 1,230. In other words, words which had been hints in VOC I, became targets in VOC II. For the most part, translations, parts of speech, comments and notes, etc., were already included in our VOC I dataset, and so the conversion was somewhat facilitated. However, the new nature of the program functions required the development of numerous new authoring routines.

After deciding to make multiple-choice questions, our main concern was with how to make multiple-choice distracters easily. For each target word, we needed three distracters, since we were offering a four-item multiple choice selection. We developed a new input routine for the inputting of distracters and the creation of the multiple-choice questions.

When we make multiple choice questions, it is often difficult to come up with good second and third choice distracters. By using all the words in the question database, we generated a long list, complete with Japanese translations and parts of speech, which could be browsed (scanned forwards and backwards) and arranged alphabetically or by part of speech. We can jump back and forth through the list by typing just the first few letters of a word or part of speech. Usually, we input the correct choice first, by merely highlighting the target word and typing the number, from 1 to 4, which we want the correct word to be offered at. Then, scrolling through the list until we find a suitable distracter, we use the same process, highlighting the word (usually with the same part of speech as the target), and typing in the number where the distracter will appear on the screen.

The Japanese translations are moved in the question base along with the selected distracter word, and after choices are made, they can be edited. Also added automatically are the Genius level of the target word, and the information as to whether the word is in the FEN frequency list. We made a new system of giving points for correct answers, based on the Genius level of the word as follows: