Removing header and trailer of the File using Scala might not be real-time use case since you will be using Spark when dealing with large datasets. This post helpful mainly for interview purpose, An Interviewer might ask to write code for this using scala instead Unix/Spark.
Here is the code snippet to achieve the same using Scala –
[code lang=”scala”]
import scala.io.Source
object RemoveHeaderTrailer{
def main(args: Array[String]){
println("start")
val input = Source.fromFile("C:/Users/Sai/input.txt")
//input.getLines().drop(1).foreach(println)//This is for removing Header alone
val lines = input.getLines().toList
val required_data = lines.slice(1,lines.size-1).mkString("\n")
import java.io._
val pw = new PrintWriter(new File("C:/Users/Sai/output.txt"))
pw.write(required_data)
pw.close()
println("end")
}
}
[/code]
