-->

Augmentative and Alternative Communication Devices

Modeling a simple symbol based AAC device that has two levels of images

Assignment - Nested Version


Starter Code

Github Starter Code

Details

There are many disabilities that affect a person's ability to speak. For instance certain disabilities, such as ALS - a progressive neurodegenerative disease [1], affect the muscles of the mouth, causing slurred speech or the inability to speak. Other disabilities may include developmental delays that affect a person's speech. As such, alternatives to verbal communication are often sought. These alternatives are referred to as Augmented and Alternative Communication Systems (AACs). An AAC can be either high-tech or low/no-tech. An example of low/no-tech options might be a pointing to printed pictures to communicate an idea or writing on paper to communicate.

However, technology can provide different options for those with speech impairments to communicate. One of the most famous people to use an AAC is Stephen Hawking. His AAC would speak aloud what he typed and was extremely high tech, combining sensors to use a facial muscle to control an interface and artificial intelligence to aid text prediction, decreasing the amount that he needed to type. Below is a short video highlighting the technology behind his AAC. 

While many AACs require a user to type out what they want spoken, there are also AACs that rely on symbols for communication. An example of these types of AACs is shown in the picture below.

The main page of an AAC device layout for English speakers

Various, CC BY-SA 4.0, via Wikimedia Commons

 

With symbol based AACs, when the user clicks on the image, the device speaks aloud the text that represents that image. For this assignment, we will work to create a simple version of an AAC device like the one shown above.

 

Getting Started

First, you will want to make sure that the text to speech library, FreeTTS, is working so that we can create a working simple AAC. 

Run the TextSpeech Driver class to ensure the text to speech engine is working. Code from: https://www.geeksforgeeks.org/converting-text-speech-java/

Note: This section is only needed if you are not using Github to share your assignments with the students. If they are pulling from Github, the library should be set up in Eclipse.

To Install Library in Eclipse:

  1. Click Build Path -> Add Libraries
  2. Select User Library
  3. Check FreeTTS

Programming Details

You have been given the following classes to represent the the logic of the AAC. The class should use HashMaps to store the mappings. You have been provided a GUI Class, AAC, which creates a graphical interface which displays the images and speaks the text aloud. The AAC will have a main page that initially shows all the categories that are on the AAC. When a user clicks on one of the category images, it will load a new page of images. When a user clicks on one of these images, the text associated with that image is spoken aloud.  In order for the AAC class to work, you will need to implement two classes, AACCategory and AACMappings which are described below.

Part 1: AACCategory

This portion of the AAC should represent a single category of items in the AAC. It stores the mapping between the image location and the text that should be spoken and the name of the category. You are to implement the methods specified in the documentation.

Part 2: AACMapping

This is the class the keeps track of the complete set of AAC mappings. It will store the mapping of the images on the home page to the AACCategories. The class should keep track of the current category that is being displayed. Then given the current category ("" if the home screen or the category name for any other category), it should respond appropriately for each of the methods.

The AACMapping constructor takes a file that will be in the following format. In the file, the first line will represent the category. It will have the file location first (will be one "word") and then the category name. Then each of the items that is in the category will follow and the line will start with a ">". The line will then have the file location (will be one "word") and then the text to speak. A sample file with two categories (food and clothing) and 2 and 3 items in each category is shown below.

img/food/plate.png food
>img/food/icons8-french-fries-96.png french fries
>img/food/icons8-watermelon-96.png watermelon
img/clothing/hanger.png clothing
>img/clothing/collaredshirt.png collared shirt
>img/clothing/sweater.png sweater
>img/clothing/skirt.png skirt

In the file, the first line will represent the category. It will have the file location first (will be one "word") and then the category name. Then each of the items that is in the category will follow and the line will start with a ">". The line will then have the file location (will be one "word") and then the text to speak. 

  • constructor - this method should read in the file and create the relevant mappings from images to categories and add all the items to each category. It should also start the AAC on the home screen
  • addItem - should add the image and text to the category that is currently selected. If it is on the home screen, it should create a new category with the given image and name. If it is in a category, it should add the image and text to speak to the currently shown category
  • getCategory - it should return which is the current category the page is on or the empty string if the page is empty
  • getImageLocs - it should return an array of the images for the currently selected category or the home page
  • select - given an image, it will either return the name of the category associated with that image if on the home screen and update the current category to that category or if in a category, it will return the text to be spoken that is associated with that image. If the image is not found in the expected area, it will throw the ElementNotFoundException
  • isCategory - given an image, it will return true if it is a category image and false otherwise
  • reset - resets the current category to the home screen ("")
  • writeToFile - writes the AACMappings in the same format that they are read in

Make sure you include the comments from the Javadoc and use good style.