Data science Software Course Training in Ameerpet Hyderabad

Data science Software Course Training in Ameerpet Hyderabad

Wednesday, 3 May 2017

Spark : Conditional Transformations


 Conditions Transformations:

val trans = emp.map{ x =>
       val w = x.split(",");
       val sal = w(2).toInt
     
      val grade = if(sal>=70000) "A" else
                    if(sal>=50000) "B" else
                     if(sal>=30000) "C" else "D"
      val tax = sal*10/100
     
      val dno = w(4).toInt
      val dname = dno match{
                  case 11 => "Marketing"
                  case 12 => "Hr"
                  case 13 => "Finance"
                  case _  => "Other"
           }
      var sex = w(3).toLowerCase
      sex = if(sex=="m") "Male" else "Female"

     val res = Array(w(0), w(1),
                   w(2),tax.toString, grade, sex, dname).mkString(",")
      res
  }

 trans.saveAsTextFile("/user/cloudera/spLab/results4")
-----------------------------------------
   


               

3 comments: