-
Notifications
You must be signed in to change notification settings - Fork 0
/
MapReduce Transpose
48 lines (31 loc) · 1.04 KB
/
MapReduce Transpose
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
In this example I will do a MapReduce Algorithm to pivot a table.
the input of the algorithm is :
| a | b | c |
| d | e | f |
then the output is :
| a | d |
| b | e |
| c | f |
The key is the row index and the value is the values associated with that row.
The mapper goles is to generate a key_value pair.
{ 0 : a} { 1 : b} { 2 : c} { 0 : d} { 1 : e} { 2 : f}
map(row_index, values):
for (column_index = 1 to length values):
k = column_index
v ={ row_index :values[column_index]}
return{ k : v }
After we do a shuffle and sort phase we get :
{0 , [(0 , a) , (0 , d)]}
{1 , [(1 , b) , (1 , d)]}
{2 , [(2 , c) , (2 , f)]}
then the reducer will return keys will be transpose row indexs and the
values will be the elements in that row which we refer to as transposed Valuess.
(0 , [a, d])
(1 , [b, e])
(2 , [c, f])
reduce(k, v) :
Pivotvalues = []
for ( a in v[]) :
Pivotvalues[a.k] = a.value
Pivot_index = k
return {Pivot_index ,Pivotvalues }