Intellj 로컬에서 Scala Compile을 할 때 SBT compile을 사용한다. SBT Console SBT Action
에서 compile
커맨드로 컴파일을 하지만 다음과 같은 Out Of Memory (OOM) 오류가 발생했다.
[debug] Forcing garbage collection...
java.util.concurrent.ExecutionException: java.lang.OutOfMemoryError: GC overhead limit exceeded
at java.util.concurrent.FutureTask.report(FutureTask.java:122)
at java.util.concurrent.FutureTask.get(FutureTask.java:192)
at sbt.ConcurrentRestrictions$$anon$4.take(ConcurrentRestrictions.scala:188)
at sbt.Execute.next$1(Execute.scala:85)
at sbt.Execute.processAll(Execute.scala:88)
at sbt.Execute.runKeep(Execute.scala:68)
at sbt.EvaluateTask$.liftedTree1$1(EvaluateTask.scala:359)
at sbt.EvaluateTask$.run$1(EvaluateTask.scala:358)
at sbt.EvaluateTask$.runTask(EvaluateTask.scala:378)
at sbt.Aggregation$$anonfun$3.apply(Aggregation.scala:69)
at sbt.Aggregation$$anonfun$3.apply(Aggregation.scala:67)
at sbt.EvaluateTask$.withStreams(EvaluateTask.scala:314)
at sbt.Aggregation$.timedRun(Aggregation.scala:67)
at sbt.Aggregation$.runTasks(Aggregation.scala:76)
at sbt.Aggregation$$anonfun$applyTasks$1.apply(Aggregation.scala:37)
at sbt.Aggregation$$anonfun$applyTasks$1.apply(Aggregation.scala:36)
at sbt.Command$$anonfun$applyEffect$2$$anonfun$apply$3.apply(Command.scala:61)
at sbt.Command$$anonfun$applyEffect$2$$anonfun$apply$3.apply(Command.scala:61)
at sbt.Aggregation$$anonfun$evaluatingParser$4$$anonfun$apply$5.apply(Aggregation.scala:158)
at sbt.Aggregation$$anonfun$evaluatingParser$4$$anonfun$apply$5.apply(Aggregation.scala:157)
at sbt.Act$$anonfun$sbt$Act$$actParser0$1$$anonfun$sbt$Act$$anonfun$$evaluate$1$1$$anonfun$apply$10.apply(Act.scala:253)
at sbt.Act$$anonfun$sbt$Act$$actParser0$1$$anonfun$sbt$Act$$anonfun$$evaluate$1$1$$anonfun$apply$10.apply(Act.scala:250)
at sbt.Command$.process(Command.scala:93)
at sbt.MainLoop$$anonfun$1$$anonfun$apply$1.apply(MainLoop.scala:96)
at sbt.MainLoop$$anonfun$1$$anonfun$apply$1.apply(MainLoop.scala:96)
at sbt.State$$anon$1.doX$1(State.scala:183)
at sbt.State$$anon$1.process(State.scala:190)
at sbt.MainLoop$$anonfun$1.apply(MainLoop.scala:96)
at sbt.MainLoop$$anonfun$1.apply(MainLoop.scala:96)
at sbt.ErrorHandling$.wideConvert(ErrorHandling.scala:17)
at sbt.MainLoop$.next(MainLoop.scala:96)
at sbt.MainLoop$.run(MainLoop.scala:89)
at sbt.MainLoop$$anonfun$runWithNewLog$1.apply(MainLoop.scala:68)
at sbt.MainLoop$$anonfun$runWithNewLog$1.apply(MainLoop.scala:63)
at sbt.Using.apply(Using.scala:24)
at sbt.MainLoop$.runWithNewLog(MainLoop.scala:63)
at sbt.MainLoop$.runAndClearLast(MainLoop.scala:46)
at sbt.MainLoop$.runLoggedLoop(MainLoop.scala:30)
at sbt.MainLoop$.runLogged(MainLoop.scala:22)
at sbt.StandardMain$.runManaged(Main.scala:109)
at sbt.xMain.run(Main.scala:38)
at xsbt.boot.Launch$$anonfun$run$1.apply(Launch.scala:57)
at xsbt.boot.Launch$.withContextLoader(Launch.scala:77)
at xsbt.boot.Launch$.run(Launch.scala:57)
at xsbt.boot.Launch$$anonfun$explicit$1.apply(Launch.scala:45)
at xsbt.boot.Launch$.launch(Launch.scala:65)
at xsbt.boot.Launch$.apply(Launch.scala:16)
at xsbt.boot.Boot$.runImpl(Boot.scala:32)
at xsbt.boot.Boot$.main(Boot.scala:21)
at xsbt.boot.Boot.main(Boot.scala)
Caused by: java.lang.OutOfMemoryError: GC overhead limit exceeded
[error] java.util.concurrent.ExecutionException: java.lang.OutOfMemoryError: GC overhead limit exceeded
[error] Use 'last' for the full log.
위 처럼 last
커맨드를 입력하여 디테일한 full log를 볼 수 있다.
SBT Console에서 SBT start를 하면 아래와 같이 뜨면서 Memory Head size가 얼마나 되는지 보여준다. 이를 설정에서 변경할 수 있다.
java -Dsbt.log.noformat=true -Djline.terminal=jline.UnsupportedTerminal -Xmx1G -XX:MaxPermSize=1G -jar /Users/naver/Library/Caches/IntelliJIdea2016.1/sbt/sbt-launch.jar
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=1G; support was removed in 8.0
[info] Loading project definition from /Users/naver/Documents/improvement/academic-spark-import/project
[info] Set current project to academic-ncr-spark (in build file:/Users/naver/Documents/improvement/academic-spark-import/)
> [info] ans: String = <modified system property 'jline.terminal' for Scala console compatibility>
해결책
IntelliJ
- Preferences
- Language & Frameworks
- Scala Compile Server
로 가서 Compile Server JVM의 heap size를 조정한다.
JVM maximum head size, MB
를 늘려준다.
'Programming > Scala' 카테고리의 다른 글
[Spark] RDD에서 DataFrame 변환 시 NullPointerException 문제 (0) | 2021.01.26 |
---|---|
[Scala/Spark] XML 파싱 에러 (SAXParseException) 해결하고 HTML 파싱해보기 (0) | 2020.05.28 |
댓글