Skip to content
GitLab
Commit 60decd4b authored by ringo's avatar ringo
Browse files

version3.6

parent fef2d386
Hide whitespace changes
Inline Side-by-side
README.md
View file @ 60decd4b
......@@ -46,7 +46,15 @@
### 分布式运行：
![](/Users/ringo/Desktop/截屏2021-11-03 下午3.37.32.png)
![截屏2021-11-03 下午3.37.44](/Users/ringo/Desktop/截屏2021-11-03 下午3.37.44.png)
![截屏2021-11-03 下午3.34.15](/Users/ringo/Desktop/截屏2021-11-03 下午3.34.15.png)
![截屏2021-11-03 下午3.34.15](/Users/ringo/Desktop/截屏2021-11-03 下午3.34.15.png)
除了将文件输出到hdfs系统外，其余与本地运行时相同。
## 遇到的问题：
......@@ -62,5 +70,21 @@
3. 分布式运行找不到文件路径的问题：
（有待解决）
![截屏2021-10-30 下午10.27.51](/Users/ringo/Desktop/截屏2021-10-30 下午10.27.51.png)
一开始怀疑是文件的默认读取路径有问题，遂编写脚本使用dfs里的test方法进行检查，发现文件是存在的，说明不是文件系统的问题。
![截屏2021-11-03 上午10.19.42](/Users/ringo/Desktop/截屏2021-11-03 上午10.19.42.png)
![截屏2021-11-03 上午10.20.37](/Users/ringo/Desktop/截屏2021-11-03 上午10.20.37.png)
后来发现是因为hadoop里面的addcachefile方法默认从本地上传文件，在将输入路径改为filesystem里面的读入方法之后，问题解决。
4. resourcemanager无法连接8088端口的问题：
![截屏2021-11-03 下午2.29.31](/Users/ringo/Desktop/截屏2021-11-03 下午2.29.31.png)
首先检查系统的8088端口，发现端口是畅通的，后来修改了yarn-site.xml，指定了resourcemanager的连接方法之后，运行正常
![截屏2021-11-03 下午3.57.14](/Users/ringo/Desktop/截屏2021-11-03 下午3.57.14.png)
README.pdf
View file @ 60decd4b
No preview for this file type
output/shakespearealls11-r-00000 0 → 100644
View file @ 60decd4b
1: lord, 213
2: parolles, 177
3: bertram, 137
4: king, 136
5: helena, 125
6: lafeu, 117
7: shall, 115
8: thou, 100
9: countess, 99
10: sir, 97
11: good, 90
12: know, 87
13: thy, 84
14: thee, 79
15: second, 70
16: clown, 67
17: love, 65
18: say, 62
19: diana, 58
20: enter, 57
21: tis, 53
22: hath, 51
23: let, 50
24: soldier, 49
25: make, 49
26: come, 47
27: great, 47
28: man, 46
29: madam, 45
30: honour, 45
31: speak, 43
32: count, 40
33: think, 40
34: like, 38
35: did, 35
36: widow, 35
37: ring, 35
38: son, 34
39: gentleman, 32
40: mother, 32
41: wife, 32
42: exit, 30
43: time, 30
44: duke, 30
45: away, 29
46: ends, 28
47: act, 28
48: poor, 28
49: life, 27
50: knave, 27
51: pray, 26
52: exeunt, 26
53: rousillon, 26
54: nature, 25
55: leave, 25
56: young, 24
57: heaven, 24
58: scene, 24
59: does, 24
60: drum, 23
61: father, 23
62: tell, 23
63: god, 23
64: hear, 22
65: marry, 22
66: thine, 22
67: hand, 22
68: florence, 22
69: maid, 21
70: captain, 21
71: night, 21
72: serve, 21
73: lady, 20
74: live, 20
75: hope, 20
76: nay, 20
77: truth, 19
78: fortune, 19
79: virginity, 19
80: daughter, 19
81: answer, 19
82: france, 19
83: farewell, 18
84: old, 18
85: fair, 18
86: death, 18
87: noble, 18
88: art, 18
89: comes, 18
90: gone, 18
91: heart, 18
92: sweet, 18
93: majesty, 17
94: master, 17
95: himself, 17
96: court, 17
97: business, 17
98: bring, 17
99: lordship, 17
100: till, 17
output/shakespeareantony23-r-00000 0 → 100644
View file @ 60decd4b
1: antony, 428
2: cleopatra, 315
3: caesar, 292
4: mark, 264
5: thou, 184
6: enobarbus, 153
7: domitius, 140
8: octavius, 133
9: shall, 127
10: enter, 112
11: thee, 110
12: thy, 110
13: charmian, 105
14: let, 99
15: good, 95
16: come, 93
17: pompey, 81
18: sir, 79
19: soldier, 75
20: make, 74
21: did, 69
22: messenger, 65
23: eros, 64
24: lepidus, 62
25: say, 59
26: tis, 59
27: like, 56
28: exeunt, 55
29: lord, 54
30: menas, 54
31: madam, 52
32: agrippa, 52
33: act, 49
34: know, 49
35: great, 48
36: queen, 48
37: egypt, 46
38: man, 45
39: world, 45
40: hath, 44
41: scene, 44
42: octavia, 43
43: iras, 42
44: guard, 42
45: love, 40
46: speak, 39
47: dolabella, 38
48: heart, 38
49: hear, 38
50: exit, 37
51: alexas, 35
52: gods, 35
53: rome, 34
54: death, 34
55: time, 33
56: noble, 33
57: friends, 32
58: way, 31
59: think, 31
60: sea, 31
61: tell, 30
62: night, 29
63: second, 29
64: does, 28
65: look, 28
66: mecaenas, 28
67: hand, 28
68: fortune, 27
69: aside, 26
70: nay, 26
71: men, 26
72: fight, 26
73: till, 26
74: hast, 26
75: dead, 25
76: day, 24
77: best, 24
78: follow, 24
79: sword, 24
80: bear, 23
81: land, 23
82: women, 23
83: eyes, 22
84: proculeius, 22
85: honour, 22
86: war, 22
87: thyreus, 21
88: farewell, 21
89: soothsayer, 21
90: better, 20
91: myself, 20
92: leave, 20
93: bring, 20
94: gone, 20
95: scarus, 20
96: canidius, 19
97: pray, 19
98: dear, 19
99: art, 19
100: mardian, 19
output/shakespeareas12-r-00000 0 → 100644
View file @ 60decd4b
1: rosalind, 275
2: orlando, 164
3: thou, 142
4: celia, 126
5: love, 119
6: good, 115
7: duke, 105
8: man, 95
9: thee, 90
10: touchstone, 88
11: like, 84
12: jaques, 80
13: sir, 76
14: thy, 75
15: come, 68
16: shall, 68
17: oliver, 60
18: enter, 54
19: did, 53
20: hath, 52
21: phebe, 47
22: father, 46
23: say, 44
24: know, 43
25: time, 43
26: silvius, 42
27: brother, 42
28: forest, 41
29: let, 41
30: senior, 38
31: fool, 37
32: old, 35
33: corin, 34
34: heart, 34
35: frederick, 34
36: make, 33
37: audrey, 33
38: tis, 33
39: tell, 30
40: better, 30
41: lord, 30
42: world, 29
43: youth, 29
44: young, 29
45: woman, 29
46: art, 28
47: shepherd, 28
48: speak, 28
49: exeunt, 27
50: look, 27
51: pray, 26
52: think, 26
53: doth, 25
54: fair, 25
55: comes, 25
56: scene, 25
57: charles, 24
58: life, 24
59: god, 24
60: bear, 24
61: true, 23
62: little, 23
63: adam, 23
64: marry, 22
65: beau, 22
66: act, 22
67: poor, 22
68: daughter, 21
69: sweet, 21
70: wit, 21
71: faith, 20
72: thing, 20
73: master, 20
74: eyes, 19
75: lie, 19
76: fortune, 19
77: day, 18
78: court, 18
79: nay, 18
80: men, 18
81: till, 18
82: morrow, 18
83: mistress, 17
84: away, 17
85: thank, 17
86: nature, 17
87: william, 17
88: amiens, 17
89: prithee, 17
90: exit, 16
91: live, 16
92: truly, 16
93: place, 16
94: word, 16
95: gentle, 16
96: hand, 16
97: hour, 16
98: matter, 16
99: leave, 15
100: house, 14
output/shakespearecomedy7-r-00000 0 → 100644
View file @ 60decd4b
1: syracuse, 232
2: dromio, 221
3: antipholus, 219
4: ephesus, 169
5: thou, 134
6: sir, 111
7: adriana, 90
8: thee, 65
9: thy, 59
10: man, 57
11: luciana, 56
12: did, 55
13: come, 48
14: master, 47
15: chain, 47
16: know, 45
17: duke, 43
18: home, 42
19: angelo, 41
20: enter, 40
21: wife, 37
22: time, 36
23: let, 36
24: hath, 34
25: say, 33
26: day, 30
27: husband, 29
28: merchant, 29
29: solinus, 28
30: tell, 28
31: mistress, 28
32: good, 28
33: mad, 27
34: aegeon, 27
35: sister, 27
36: shall, 26
37: officer, 26
38: money, 26
39: make, 25
40: house, 25
41: till, 25
42: dinner, 24
43: think, 24
44: art, 22
45: like, 22
46: bear, 21
47: gold, 20
48: love, 19
49: second, 19
50: life, 18
51: sent, 18
52: villain, 17
53: end, 17
54: nay, 17
55: long, 17
56: courtezan, 17
57: pray, 17
58: tis, 17
59: came, 16
60: doth, 16
61: aemelia, 16
62: speak, 15
63: bound, 15
64: god, 15
65: exit, 15
66: gave, 14
67: away, 14
68: errors, 14
69: comes, 14
70: exeunt, 14
71: hear, 14
72: fetch, 14
73: saw, 14
74: break, 14
75: hand, 13
76: sweet, 13
77: fair, 13
78: hour, 13
79: hast, 13
80: face, 13
81: sure, 13
82: look, 12
83: scene, 12
84: heart, 12
85: quoth, 12
86: comedy, 12
87: gone, 12
88: wrong, 12
89: town, 12
90: didst, 12
91: door, 12
92: rope, 12
93: eye, 12
94: pinch, 12
95: luce, 11
96: merry, 11
97: myself, 11
98: men, 11
99: act, 11
100: woman, 11
output/shakespearecoriolanus24-r-00000 0 → 100644
View file @ 60decd4b
1: coriolanus, 245
2: menenius, 196
3: marcius, 142
4: sicinius, 138
5: shall, 123
6: brutus, 111
7: cominius, 105
8: rome, 103
9: thou, 100
10: enter, 99
11: aufidius, 98
12: thy, 91
13: good, 90
14: let, 89
15: come, 86
16: citizen, 85
17: second, 83
18: people, 83
19: sir, 83
20: say, 73
21: like, 69
22: volumnia, 68
23: thee, 68
24: servingman, 66
25: make, 62
26: did, 62
27: know, 61
28: noble, 59
29: speak, 59
30: man, 56
31: hear, 55
32: tis, 52
33: hath, 49
34: gods, 46
35: senator, 46
36: exeunt, 45
37: citizens, 44
38: city, 42
39: pray, 42
40: tribunes, 41
41: power, 41
42: think, 41
43: mother, 40
44: lartius, 40
45: peace, 38
46: virgilia, 38
47: country, 37
48: home, 37
49: voices, 36
50: general, 36
51: tell, 36
52: friends, 35
53: great, 34
54: time, 34
55: roman, 34
56: act, 33
57: way, 33
58: scene, 32
59: heart, 31
60: consul, 31
61: true, 30
62: honour, 29
63: away, 29
64: love, 29
65: messenger, 28
66: nay, 28
67: house, 27
68: son, 27
69: senators, 27
70: men, 27
71: bear, 27
72: stand, 27
73: worthy, 27
74: news, 25
75: lord, 24
76: state, 24
77: soldier, 23
78: corioli, 23
79: blood, 23
80: valeria, 23
81: wife, 23
82: war, 23
83: volsces, 22
84: place, 22
85: death, 21
86: world, 21
87: caius, 21
88: titus, 21
89: said, 20
90: poor, 20
91: look, 20
92: set, 20
93: leave, 19
94: gone, 19
95: wars, 19
96: common, 19
97: aedile, 19
98: forth, 19
99: answer, 19
100: gates, 19
output/shakespearecymbeline17-r-00000 0 → 100644
View file @ 60decd4b
1: thou, 171
2: imogen, 165
3: cymbeline, 147
4: thy, 132
5: posthumus, 129
6: leonatus, 120
7: cloten, 111
8: lord, 106
9: pisanio, 105
10: iachimo, 96
11: sir, 92
12: shall, 89
13: thee, 87
14: good, 83
15: belarius, 81
16: guiderius, 79
17: hath, 78
18: enter, 72
19: queen, 69
20: arviragus, 66
21: make, 65
22: let, 60
23: like, 60
24: did, 58
25: man, 56
26: come, 55
27: tis, 55
28: lady, 55
29: know, 52
30: lucius, 46
31: time, 45
32: second, 45
33: king, 41
34: think, 41
35: father, 37
36: master, 37
37: gentleman, 36
38: gods, 36
39: say, 36
40: britain, 35
41: act, 35
42: speak, 34
43: exeunt, 34
44: exit, 34
45: caius, 33
46: fear, 33
47: thing, 33
48: life, 33
49: great, 32
50: honour, 32
51: true, 32
52: look, 31
53: art, 31
54: love, 31
55: roman, 31
56: aside, 30
57: son, 30
58: scene, 29
59: court, 28
60: leave, 28
61: heart, 28
62: mother, 27
63: mistress, 27
64: pray, 27
65: death, 27
66: hand, 27
67: away, 26
68: false, 26
69: madam, 25
70: brother, 25
71: poor, 24
72: long, 24
73: men, 23
74: dead, 23
75: philario, 23
76: hear, 23
77: nay, 22
78: world, 22
79: stand, 22
80: die, 22
81: best, 22
82: myself, 22
83: way, 21
84: nature, 21
85: report, 20
86: doth, 20
87: noble, 20
88: gone, 19
89: day, 19
90: bed, 19
91: place, 19
92: ring, 19
93: worthy, 19