Data science Software Course Training in Ameerpet Hyderabad

Data science Software Course Training in Ameerpet Hyderabad

Wednesday, 3 May 2017

Spark : Conditional Transformations


 Conditions Transformations:

val trans = emp.map{ x =>
       val w = x.split(",");
       val sal = w(2).toInt
     
      val grade = if(sal>=70000) "A" else
                    if(sal>=50000) "B" else
                     if(sal>=30000) "C" else "D"
      val tax = sal*10/100
     
      val dno = w(4).toInt
      val dname = dno match{
                  case 11 => "Marketing"
                  case 12 => "Hr"
                  case 13 => "Finance"
                  case _  => "Other"
           }
      var sex = w(3).toLowerCase
      sex = if(sex=="m") "Male" else "Female"

     val res = Array(w(0), w(1),
                   w(2),tax.toString, grade, sex, dname).mkString(",")
      res
  }

 trans.saveAsTextFile("/user/cloudera/spLab/results4")
-----------------------------------------
   


               

5 comments:

  1. Good Post! Thank you so much for sharing this pretty post, it was so good to read and useful to improve my knowledge as updated one, keep blogging.

    https://www.emexotechnologies.com/online-courses/big-data-hadoop-training-in-electronic-city/

    ReplyDelete
  2. Good Post! Thank you so much for sharing this pretty post, it was so good to read and useful to improve my knowledge as updated one, keep blogging.
    Big Data Hadoop Training in electronic city

    ReplyDelete