Apache Spark에서 debug 기능을 사용해 보았습니다.

hisashinoMacBook-Pro:apache-spark-csv-sample hisashi$ sudo $SPARK_HOME/bin/spark-submit --class com.sample.csv.spark.SparkCSVSample --master "local[2]" --driver-java-options "-agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=5005" target/scala-2.11/spark_csv_sample_2.11-1.0.0.jar logs/ tmp/spark_csv_sample.csv /hoge,/piyo

↓コンソール上にこんなメッセージがでてきた
Listening for transport dt_socket at address: 5005

$SPARK_HOME/bin/spark-submit 
  --class com.sample.csv.spark.SparkCSVSample 
  --master "local[2]" 
  --driver-java-options "-agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=5005" ←(ここを　IntelliJのCommand line arguments for running remote JVM の定義に合わせるらしい)
  target/scala-2.11/spark_csv_sample_2.11-1.0.0.jar 
  logs/ tmp/spark_csv_sample.csv /hoge,/piyo