前景提要
HDC调试需求开发(15万预算),能者速来!>>>
INFO memory.MemoryStore: Block broadcast_2 stored as values in memory (estimated size 432.0 B, free 912.3 MB)
19/12/02 15:28:09 INFO memory.MemoryStore: Block broadcast_2_piece0 stored as bytes in memory (estimated size 29.4 KB, free 912.3 MB)
19/12/02 15:28:09 INFO storage.BlockManagerInfo: Added broadcast_2_piece0 in memory on slave2:38988 (size: 29.4 KB, free: 912.3 MB)
19/12/02 15:28:09 INFO spark.SparkContext: Created broadcast 2 from broadcast at NativeMethodAccessorImpl.java:0
19/12/02 15:28:09 INFO yarn.ApplicationMaster: Final app status: FAILED, exitCode: 13
19/12/02 15:28:09 ERROR yarn.ApplicationMaster: Uncaught exception:
java.lang.IllegalStateException: User did not initialize spark context!
at org.apache.spark.deploy.yarn.ApplicationMaster.runDriver(ApplicationMaster.scala:464)
at org.apache.spark.deploy.yarn.ApplicationMaster.run(ApplicationMaster.scala:275)
at org.apache.spark.deploy.yarn.ApplicationMaster$$anon$3.run(ApplicationMaster.scala:805)
at org.apache.spark.deploy.yarn.ApplicationMaster$$anon$3.run(ApplicationMaster.scala:804)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1875)
at org.apache.spark.deploy.yarn.ApplicationMaster$.main(ApplicationMaster.scala:804)
at org.apache.spark.deploy.yarn.ApplicationMaster.main(ApplicationMaster.scala)
19/12/02 15:28:09 INFO spark.SparkContext: Invoking stop() from shutdown hook
19/12/02 15:28:09 INFO server.AbstractConnector: Stopped Spark@639046fc{HTTP/1.1,[http/1.1]}{0.0.0.0:0}
19/12/02 15:28:09 INFO ui.SparkUI: Stopped Spark web UI at http://slave2:46166
19/12/02 15:28:09 INFO spark.MapOutputTrackerMasterEndpoint: MapOutputTrackerMasterEndpoint stopped!
19/12/02 15:28:09 INFO memory.MemoryStore: MemoryStore cleared
19/12/02 15:28:09 INFO storage.BlockManager: BlockManager stopped
19/12/02 15:28:09 INFO storage.BlockManagerMaster: BlockManagerMaster stopped
19/12/02 15:28:09 INFO scheduler.OutputCommitCoordinator$OutputCommitCoordinatorEndpoint: OutputCommitCoordinator stopped!
19/12/02 15:28:09 INFO spark.SparkContext: Successfully stopped SparkContext
19/12/02 15:28:09 INFO yarn.ApplicationMaster: Deleting staging directory hdfs://rongan/user/hbase/.sparkStaging/application_1574702307465_5363
19/12/02 15:28:09 INFO util.ShutdownHookManager: Shutdown hook called
19/12/02 15:28:09 INFO util.ShutdownHookManager: Deleting directory /data/yarn/nm/usercache/hbase/appcache/application_1574702307465_5363/spark-bd608f8d-3c63-4f31-947c-b425df71ae14
19/12/02 15:28:09 INFO util.ShutdownHookManager: Deleting directory /data/yarn/nm/usercache/hbase/appcache/application_1574702307465_5363/spark-bd608f8d-3c63-4f31-947c-b425df71ae14/pyspark-d1a9900b-2ebb-41e6-9a4f-b4c0c3badf62
For more detailed output, check the application tracking page: http://master:8088/cluster/app/application_1574702307465_5363 Then click on links to logs of each attempt.
. Failing the application.
Exception in thread "main" org.apache.spark.SparkException: Application application_1574702307465_5363 finished with failed status
at org.apache.spark.deploy.yarn.Client.run(Client.scala:1158)
at org.apache.spark.deploy.yarn.YarnClusterApplication.start(Client.scala:1606)
at org.apache.spark.deploy.SparkSubmit.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:851)
at org.apache.spark.deploy.SparkSubmit.doRunMain$1(SparkSubmit.scala:167)
at org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:195)
at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:86)
at org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:926)
at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:935)
at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
19/12/02 15:28:11 INFO util.ShutdownHookManager: Shutdown hook called
19/12/02 15:28:11 INFO util.ShutdownHookManager: Deleting directory /tmp/spark-e2302e8e-ab54-4b4a-879b-9c1b8930fd46
19/12/02 15:28:11 INFO util.ShutdownHookManager: Deleting directory /tmp/spark-a6b79ec2-4e04-478a-b498-8bb67c579422
You have new mail in /var/spool/mail/root