program程序辅导、Java编程设计调试、讲解Java 讲解R语言编程|解析Haskell程序

项目预算：开发周期：发布时间：要求地区：

Lab 11: Natural Language Processing
This lab is bonus and is not required.
Goals for this lab:
 Use HashMaps to quickly store and access data
 Use HashSets to maintain a duplicate free collection of data
 Determine the term frequency of words in a file
 Prevent stopwords from being included in the term frequency
 Write, compile, and execute Java programs using the given instructions
General Guidelines:
 Use meaningful variable names
 Use appropriate indentation
 Use comments, especially for the header, variables, and blocks of code.
Example Header:
Date:
Instructor:
Description: A small description in your own words
that describe what the program does. Any
additional flags needed to allow the
program to compile should also be placed
here.
*/
1. Term Frequency
The area of natural language processing is typically an intersection between linguistics, computer science, and
artificial intelligence. Algorithms and software are developed in order to allow the computer to infer meaning
and make decisions based on human languages. This could be as complex as teaching a computer to carry on a
reasonable conversation with a human, or as “simple” as trying to determine if people liked a movie based on
their social media posts. In both cases, sentences must be broken down into individual words or phrases so
that the computer can process them. One of the first steps is often term frequency analyses, where the
frequency of words are counted, so that the most important or significant words can be determined.
For this exercise you will be writing a program to count the number of words in chunk of text. The input file
will contain multiple lines, and punctuation has already been removed, however not all of the words are
completely lowercase. You will need to break up each line into individual words, and maintain a count of how
many times each word appears in the overall text chunk using a HashMap. Finally, you will need to output the
list of words and their frequencies to the user
To complete this program:
 In a file named FreqCountV1.java, define a class named FreqCountV1 which will perform the text
analysis. The class should contain:
o A private, static method named readTextFile, which takes in a HashMap and returns
nothing and should:
 Prompt the user for the name of the file containing the text, and repeatedly reprompt
the user if the filename they enter does not exist
 Read in the file one word at a time and for each word
 Convert it to all lowercase letters
 If the word already exists in the HashMap, increase its count by one in the
HashMap
 Otherwise, add it to the HashMap and set its count to one
o A private, static method named outputFreq, which takes in a HashMap and returns nothing
and should:
 Output each word and its associated count from the HashMap
 Because words are different lengths, make sure that you determine the length of the
longest word and use that in your formatting to produce a cleaner output (see
outputV1.txt)
o A main method, which should:
 Contain a HashMap that indexes counts using Strings
 Call readTextFile using the HashMap
 Call outputFreq using the HashMap
Complete the program, making sure to compile and run it to verify that it produces the correct results. Note
that you will submit the FreqCountV1.java file to Canvas.
2. Stopwords
One of the initial challenges facing proper term frequency analysis is the occurrence of stopwords, or words
that provide little to no information regarding the text. These are typically common words such as “the”, “a”,
“and”, “have”, or single letters. Thus, to focus on the more important words, we want to perform stopword
removal (or at least prevent their inclusion) in our analysis.
For this exercise you will be improving upon your previous code from part 1 of this lab. You will perform all of
the same steps from last time, but before you read in the text file, you will need to read in a list of stopwords
and store them in a HashSet. Then, as you are reading in your text file, use the HashSet to prevent the
adding of stopwords to the HashMap containing your term frequencies.
To complete this program:
 In a file named FreqCountV2.java, define a class named FreqCountV2 which will perform the text
analysis. The class should contain:
o A private, static method named readStopWordsFile, which takes in a HashSet and
returns nothing and should:
 Prompt the user for the name of the file containing the list of stopwords, and repeatedly
reprompt the user if the filename they enter does not exist
 Read in the file one word at a time and for each word add it to the HashSet
o A private, static method named readTextFile, which takes in a HashMap, a HashSet,
and returns nothing and should:
 Prompt the user for the name of the file containing the text, and repeatedly reprompt
the user if the filename they enter does not exist
 Read in the file one word at a time and for each word
 Convert it to all lowercase letters
 If the word exists in the HashSet, ignore it and do not add it to the HashMap
 If the word already exists in the HashMap, increase its count by one in the
HashMap
 Otherwise, add it to the HashMap and set its count to one
o A private, static method named outputFreq, which takes in a HashMap and returns nothing
and should:
 Output each word and its associated count from the HashMap
 Because words are different lengths, make sure that you determine the length of the
longest word and use that in your formatting to produce a cleaner output (see
outputV2.txt)
o A main method, which should:
 Contain a HashMap that indexes counts using Strings
 Contain a HashSet that stores stopwords as Strings
 Call readStopWordsFile using the HashSet
 Call readTextFile using the HashMap and HashSet
 Call outputFreq using the HashMap
Complete the program, making sure to compile and run it to verify that it produces the correct results. Note
that you will submit the FreqCountV2.java file to Canvas.
Submission
Once you have completed this lab it is time to turn in your work. Please submit the following files to the Lab
11 assignment in Canvas:
 FreqCountV1.java
 FreqCountV2.java
After you have submitted your files, you are welcome to practice some additional coding.
Rubric
Part 1 60 pts:
 5 points: Program compiles successfully
 20 points: Correctly implementing the readTextFile method
o 5 points for obtaining the filename from the user and opening the file
o 5 points for converting all words to all lowercase letters
o 5 points for increasing a words frequency count if it does exist
o 5 points for adding the word and setting its frequency count to one if it does not exist
 20 points: Correctly implementing the outputFreq method
o 5 points for correctly outputting all of the words in the text file
o 10 points for correctly outputting all of the word frequencies in the text file
o 5 points outputting all of the words and their frequencies with proper formatting (See
outputV1.txt)
 10 points: Correctly implementing the main method
o 4 points for correctly creating the HashMap
o 3 points for calling the readTextFile method with the appropriate input
o 3 points for calling the outputFreq method with the appropriate input
 5 points: Appropriate commenting is included
Part 2 40 pts:
 5 points: Program compiles successfully
 10 points: Correctly implementing the readStopWordsFile method
o 5 points for obtaining the filename from the user and opening the file
o 5 points for adding all of the stopwords from the file to the HashSet
 20 points: Correctly implementing the outputFreq method
o 5 points for correctly outputting all of the words in the text file, excluding stopwords
o 10 points for correctly outputting all of the word frequencies in the text file, excluding
stopwords
o 5 points outputting all of the words and their frequencies, excluding stopwords, with proper
formatting (See outputV2.txt)
 5 points: Appropriate commenting is included