Computers and Technology, 27.02.2020 16:34, jackie36390
Modify the WordCount program so it outputs the wordcount for each distinct word in each file. So the output of this DocWordCount program should be of the form ‘wordfilename count’, where ‘’ serves as a delimiter between word and filename and tab serves as a delimiter between filename and count. Submit your source code in a file named DocWordCount. java.
Explanation: Consider two simple files file1.txt and file2.txt. $ echo "Hadoop is yellow Hadoop" > file1.txt $ echo "yellow Hadoop is an elephant" > file2.txt Running ‘DocWordCount. java’ on these two files will give an output similar to that below, where is a delimiter.
Output of DocWordCount. java
yellowfile2.txt 1
Hadoopfile2.txt 1
isfile2.txt 1
elephantfile2.txt 1
yellowfile1.txt 1
Hadoopfile1.txt 2
isfile1.txt 1
anfile2.txt 1
Initial code that needs to be modified:
package org. myorg;
import java. io. IOException;
import java. util. regex. Pattern;
import org. apache. hadoop. conf. Configured;
import org. apache. hadoop. util. Tool;
import org. apache. hadoop. util. ToolRunner;
import org. apache. log4j. Logger;
import org. apache. hadoop. mapreduce. Job;
import org. apache. hadoop. mapreduce. Mapper;
import org. apache. hadoop. mapreduce. Reducer;
import org. apache. hadoop. fs. Path;
import org. apache. hadoop. mapreduce. lib. input. FileInputFormat;
import org. apache. hadoop. mapreduce. lib. output. FileOutputFormat;
import org. apache. hadoop. io. IntWritable;
import org. apache. hadoop. io. LongWritable;
import org. apache. hadoop. io. Text;
public class WordCount extends Configured implements Tool {
private static final Logger LOG = Logger .getLogger( WordCount. class);
public static void main( String[] args) throws Exception {
int res = ToolRunner .run( new WordCount(), args);
System .exit(res);
}
public int run( String[] args) throws Exception {
Job job = Job .getInstance(getConf(), " wordcount ");
job. setJarByClass( this .getClass());
FileInputFormat. addInputPaths(job, args[0]);
FileOutputFormat. setOutputPath(job, new Path(args[ 1]));
job. setMapperClass( Map .class);
job. setReducerClass( Reduce .class);
job. setOutputKeyClass( Text .class);
job. setOutputValueClass( IntWritable .class);
return job. waitForCompletion( true) ? 0 : 1;
}
public static class Map extends Mapper {
private final static IntWritable one = new IntWritable( 1);
private Text word = new Text();
private static final Pattern WORD_BOUNDARY = Pattern .compile("\\s*\\b\\s*");
public void map( LongWritable offset, Text lineText, Context context)
throws IOException, InterruptedException {
String line = lineText. toString();
Text currentWord = new Text();
for ( String word : WORD_BOUNDARY .split(line)) {
if (word. isEmpty()) {
continue;
}
currentWord = new Text(word);
context. write(currentWord, one);
}
}
}
public static class Reduce extends Reducer {
@Override
public void reduce( Text word, Iterable counts, Context context)
throws IOException, InterruptedException {
int sum = 0;
for ( IntWritable count : counts) {
sum += count. get();
}
context. write(word, new IntWritable(sum));
}
}
}
Answers: 2
Computers and Technology, 24.06.2019 04:30, BobBball9126
How do you share someone else’s tweet with your own twitter followers?
Answers: 1
Computers and Technology, 24.06.2019 11:00, zeesharpe05
These statements describe lists in presentation programs: a. bullets can be turned off and on. b. bullets cannot be turned off. c. bullet styles, colors, and sizes can be changed. d. lists don't have to use bullets or numbers. e. numbering styles, colors, and sizes can be changed. f. numbers can be turned off and on. g. numbers cannot be turned off. select all that apply
Answers: 2
Computers and Technology, 24.06.2019 14:30, SmartScholar4094
Two students are discussing electricity that has a frequency of 60 hz. student a says that this type of electricity is referred to as ac. student b says that in this type of electricity, the electrons flow in only one direction. which of the following statements is correct? a. only student a is correct b. only student b is correct c. both of the two students are correct d. neither of the two students is correct
Answers: 1
Modify the WordCount program so it outputs the wordcount for each distinct word in each file. So the...
English, 30.08.2019 04:30
Chemistry, 30.08.2019 04:30