��h��׶9�A@���P VERSIONAPPLICATION_ACLVIEW_APP MODIFY_APP APPLICATION_OWNERhadoop/-container_e17_1533196506314_4460157_01_000004��stderr750SLF4J: Class path contains multiple SLF4J bindings. SLF4J: Found binding in [jar:file:/opt/hadoop-2.7.3/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: Found binding in [jar:file:/data1/hadoop/yarn/data/usercache/hadoop/filecache/2042090/log4j-slf4j-impl-2.4.1.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation. SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory] log4j:WARN No appenders could be found for logger (org.apache.hadoop.metrics2.impl.MetricsSystemImpl). log4j:WARN Please initialize the log4j system properly. log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info. stdout0syslog57472018-12-10 00:40:54,182 INFO [main] org.apache.hadoop.metrics2.impl.MetricsConfig: loaded properties from hadoop-metrics2.properties 2018-12-10 00:40:54,253 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Scheduled snapshot period at 10 second(s). 2018-12-10 00:40:54,254 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: MapTask metrics system started 2018-12-10 00:40:54,264 INFO [main] org.apache.hadoop.mapred.YarnChild: Executing with tokens: 2018-12-10 00:40:54,264 INFO [main] org.apache.hadoop.mapred.YarnChild: Kind: mapreduce.job, Service: job_1533196506314_4460157, Ident: (org.apache.hadoop.mapreduce.security.token.JobTokenIdentifier@3932c79a) 2018-12-10 00:40:54,487 INFO [main] org.apache.hadoop.mapred.YarnChild: Sleeping for 0ms before retrying again. Got null now. 2018-12-10 00:40:54,749 INFO [main] org.apache.hadoop.mapred.YarnChild: mapreduce.cluster.local.dir for child: /data0/hadoop/yarn/data/usercache/hadoop/appcache/application_1533196506314_4460157,/data1/hadoop/yarn/data/usercache/hadoop/appcache/application_1533196506314_4460157,/data2/hadoop/yarn/data/usercache/hadoop/appcache/application_1533196506314_4460157,/data4/hadoop/yarn/data/usercache/hadoop/appcache/application_1533196506314_4460157,/data5/hadoop/yarn/data/usercache/hadoop/appcache/application_1533196506314_4460157,/data6/hadoop/yarn/data/usercache/hadoop/appcache/application_1533196506314_4460157,/data7/hadoop/yarn/data/usercache/hadoop/appcache/application_1533196506314_4460157,/data8/hadoop/yarn/data/usercache/hadoop/appcache/application_1533196506314_4460157,/data9/hadoop/yarn/data/usercache/hadoop/appcache/application_1533196506314_4460157,/dataa/hadoop/yarn/data/usercache/hadoop/appcache/application_1533196506314_4460157,/datab/hadoop/yarn/data/usercache/hadoop/appcache/application_1533196506314_4460157,/data3/hadoop/yarn/data/usercache/hadoop/appcache/application_1533196506314_4460157 2018-12-10 00:40:55,043 INFO [main] org.apache.hadoop.conf.Configuration.deprecation: session.id is deprecated. Instead, use dfs.metrics.session-id 2018-12-10 00:40:55,763 INFO [main] org.apache.hadoop.conf.Configuration.deprecation: mapred.output.dir is deprecated. Instead, use mapreduce.output.fileoutputformat.outputdir 2018-12-10 00:40:55,772 INFO [main] org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter: File Output Committer Algorithm version is 1 2018-12-10 00:40:55,773 INFO [main] org.apache.hadoop.conf.Configuration.deprecation: mapred.work.output.dir is deprecated. Instead, use mapreduce.task.output.dir 2018-12-10 00:40:55,793 INFO [main] org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter: File Output Committer Algorithm version is 1 2018-12-10 00:40:55,804 INFO [main] org.apache.hadoop.mapred.Task: Using ResourceCalculatorProcessTree : [ ] 2018-12-10 00:40:56,401 INFO [main] org.apache.sqoop.mapreduce.db.DBInputFormat: Using read commited transaction isolation 2018-12-10 00:40:56,571 INFO [main] org.apache.hadoop.mapred.MapTask: Processing split: `id` >= 5467237 AND `id` < 8200708 2018-12-10 00:40:56,610 INFO [main] org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter: File Output Committer Algorithm version is 1 2018-12-10 00:40:56,611 INFO [main] org.apache.hadoop.conf.Configuration.deprecation: mapred.output.key.class is deprecated. Instead, use mapreduce.job.output.key.class 2018-12-10 00:40:56,615 INFO [main] org.apache.hadoop.conf.Configuration.deprecation: mapred.output.value.class is deprecated. Instead, use mapreduce.job.output.value.class 2018-12-10 00:40:56,741 INFO [main] org.apache.sqoop.mapreduce.hcat.SqoopHCatImportHelper: HCatalog Storer Info : Handler = null Input format class = org.apache.hadoop.hive.ql.io.orc.OrcInputFormat Output format class = org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat Serde class = org.apache.hadoop.hive.ql.io.orc.OrcSerde Storer properties serialization.format=1 transient_lastDdlTime=1529984981 2018-12-10 00:40:56,746 INFO [main] org.apache.sqoop.mapreduce.db.DBRecordReader: Working on split: `id` >= 5467237 AND `id` < 8200708 2018-12-10 00:40:56,762 INFO [main] org.apache.sqoop.mapreduce.db.DBRecordReader: Executing query: SELECT `id`, `code`, `user_id`, `state`, `device_code`, `is_lock`, `is_sync`, `cust_id`, `start_date`, `end_date`, `activate_date`, `bind_date`, `service_stop`, `cancelled_date`, `create_date`, `update_date`, `user_type`, `password`, `product_ids`, `uid` FROM `bss_customer` AS `bss_customer` WHERE ( `id` >= 5467237 ) AND ( `id` < 8200708 ) 2018-12-10 00:40:56,930 INFO [main] org.apache.orc.impl.WriterImpl: ORC writer created for path: hdfs://ycluster/user/hive/warehouse/ods.db/bss_customer_fj/_SCRATCH0.3130352759450352/dt=20181209/_temporary/1/_temporary/attempt_1533196506314_4460157_m_000002_0/part-m-00002 with stripeSize: 67108864 blockSize: 268435456 compression: ZLIB bufferSize: 262144 2018-12-10 00:41:41,901 INFO [main] org.apache.hadoop.mapred.Task: Task:attempt_1533196506314_4460157_m_000002_0 is done. And is in the process of committing 2018-12-10 00:41:41,937 INFO [main] org.apache.hadoop.mapred.Task: Task attempt_1533196506314_4460157_m_000002_0 is allowed to commit now 2018-12-10 00:41:41,937 INFO [main] org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter: File Output Committer Algorithm version is 1 2018-12-10 00:41:41,944 INFO [main] org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter: Saved output of task 'attempt_1533196506314_4460157_m_000002_0' to hdfs://ycluster/user/hive/warehouse/ods.db/bss_customer_fj/_SCRATCH0.3130352759450352/dt=20181209/_temporary/1/task_1533196506314_4460157_m_000002 2018-12-10 00:41:41,973 INFO [main] org.apache.hadoop.mapred.Task: Task 'attempt_1533196506314_4460157_m_000002_0' done. /-container_e17_1533196506314_4460157_01_000001�� stderr2301SLF4J: Class path contains multiple SLF4J bindings. SLF4J: Found binding in [jar:file:/opt/hadoop-2.7.3/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: Found binding in [jar:file:/data1/hadoop/yarn/data/usercache/hadoop/filecache/2042090/log4j-slf4j-impl-2.4.1.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation. SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory] 十二月 10, 2018 12:40:48 上午 com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register 信息: Registering org.apache.hadoop.mapreduce.v2.app.webapp.JAXBContextResolver as a provider class 十二月 10, 2018 12:40:48 上午 com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register 信息: Registering org.apache.hadoop.yarn.webapp.GenericExceptionHandler as a provider class 十二月 10, 2018 12:40:48 上午 com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register 信息: Registering org.apache.hadoop.mapreduce.v2.app.webapp.AMWebServices as a root resource class 十二月 10, 2018 12:40:48 上午 com.sun.jersey.server.impl.application.WebApplicationImpl _initiate 信息: Initiating Jersey application, version 'Jersey: 1.9 09/02/2011 11:17 AM' 十二月 10, 2018 12:40:49 上午 com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider 信息: Binding org.apache.hadoop.mapreduce.v2.app.webapp.JAXBContextResolver to GuiceManagedComponentProvider with the scope "Singleton" 十二月 10, 2018 12:40:49 上午 com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider 信息: Binding org.apache.hadoop.yarn.webapp.GenericExceptionHandler to GuiceManagedComponentProvider with the scope "Singleton" 十二月 10, 2018 12:40:49 上午 com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider 信息: Binding org.apache.hadoop.mapreduce.v2.app.webapp.AMWebServices to GuiceManagedComponentProvider with the scope "PerRequest" log4j:WARN No appenders could be found for logger (org.apache.hadoop.ipc.Server). log4j:WARN Please initialize the log4j system properly. log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info. stdout0syslog1617662018-12-10 00:40:45,688 INFO [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Created MRAppMaster for application appattempt_1533196506314_4460157_000001 2018-12-10 00:40:45,929 INFO [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Executing with tokens: 2018-12-10 00:40:45,930 INFO [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Kind: YARN_AM_RM_TOKEN, Service: , Ident: (appAttemptId { application_id { id: 4460157 cluster_timestamp: 1533196506314 } attemptId: 1 } keyId: 458218143) 2018-12-10 00:40:46,202 INFO [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Using mapred newApiCommitter. 2018-12-10 00:40:46,204 INFO [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: OutputCommitter set in config null 2018-12-10 00:40:46,466 INFO [main] org.apache.hadoop.conf.Configuration.deprecation: mapred.output.dir is deprecated. Instead, use mapreduce.output.fileoutputformat.outputdir 2018-12-10 00:40:46,476 INFO [main] org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter: File Output Committer Algorithm version is 1 2018-12-10 00:40:47,082 INFO [main] org.apache.hadoop.conf.Configuration.deprecation: mapred.work.output.dir is deprecated. Instead, use mapreduce.task.output.dir 2018-12-10 00:40:47,100 INFO [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: OutputCommitter is org.apache.hive.hcatalog.mapreduce.FileOutputCommitterContainer 2018-12-10 00:40:47,274 INFO [main] org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.mapreduce.jobhistory.EventType for class org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler 2018-12-10 00:40:47,275 INFO [main] org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.mapreduce.v2.app.job.event.JobEventType for class org.apache.hadoop.mapreduce.v2.app.MRAppMaster$JobEventDispatcher 2018-12-10 00:40:47,276 INFO [main] org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.mapreduce.v2.app.job.event.TaskEventType for class org.apache.hadoop.mapreduce.v2.app.MRAppMaster$TaskEventDispatcher 2018-12-10 00:40:47,277 INFO [main] org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.mapreduce.v2.app.job.event.TaskAttemptEventType for class org.apache.hadoop.mapreduce.v2.app.MRAppMaster$TaskAttemptEventDispatcher 2018-12-10 00:40:47,277 INFO [main] org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.mapreduce.v2.app.commit.CommitterEventType for class org.apache.hadoop.mapreduce.v2.app.commit.CommitterEventHandler 2018-12-10 00:40:47,282 INFO [main] org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.mapreduce.v2.app.speculate.Speculator$EventType for class org.apache.hadoop.mapreduce.v2.app.MRAppMaster$SpeculatorEventDispatcher 2018-12-10 00:40:47,282 INFO [main] org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.mapreduce.v2.app.rm.ContainerAllocator$EventType for class org.apache.hadoop.mapreduce.v2.app.MRAppMaster$ContainerAllocatorRouter 2018-12-10 00:40:47,283 INFO [main] org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncher$EventType for class org.apache.hadoop.mapreduce.v2.app.MRAppMaster$ContainerLauncherRouter 2018-12-10 00:40:47,320 INFO [main] org.apache.hadoop.mapreduce.v2.jobhistory.JobHistoryUtils: Default file system [hdfs://ycluster:8020] 2018-12-10 00:40:47,354 INFO [main] org.apache.hadoop.mapreduce.v2.jobhistory.JobHistoryUtils: Default file system [hdfs://ycluster:8020] 2018-12-10 00:40:47,382 INFO [main] org.apache.hadoop.mapreduce.v2.jobhistory.JobHistoryUtils: Default file system [hdfs://ycluster:8020] 2018-12-10 00:40:47,393 INFO [main] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Emitting job history data to the timeline server is not enabled 2018-12-10 00:40:47,433 INFO [main] org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.mapreduce.v2.app.job.event.JobFinishEvent$Type for class org.apache.hadoop.mapreduce.v2.app.MRAppMaster$JobFinishEventHandler 2018-12-10 00:40:47,519 INFO [main] org.apache.hadoop.metrics2.impl.MetricsConfig: loaded properties from hadoop-metrics2.properties 2018-12-10 00:40:47,588 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Scheduled snapshot period at 10 second(s). 2018-12-10 00:40:47,589 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: MRAppMaster metrics system started 2018-12-10 00:40:47,596 INFO [main] org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: Adding job token for job_1533196506314_4460157 to jobTokenSecretManager 2018-12-10 00:40:47,750 INFO [main] org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: Not uberizing job_1533196506314_4460157 because: not enabled; too much RAM; 2018-12-10 00:40:47,772 INFO [main] org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: Input size for job job_1533196506314_4460157 = 0. Number of splits = 4 2018-12-10 00:40:47,772 INFO [main] org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: Number of reduces for job job_1533196506314_4460157 = 0 2018-12-10 00:40:47,772 INFO [main] org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: job_1533196506314_4460157Job Transitioned from NEW to INITED 2018-12-10 00:40:47,774 INFO [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: MRAppMaster launching normal, non-uberized, multi-container job job_1533196506314_4460157. 2018-12-10 00:40:47,829 INFO [main] org.apache.hadoop.ipc.CallQueueManager: Using callQueue class java.util.concurrent.LinkedBlockingQueue 2018-12-10 00:40:47,845 INFO [Socket Reader #1 for port 19346] org.apache.hadoop.ipc.Server: Starting Socket Reader #1 for port 19346 2018-12-10 00:40:47,877 INFO [main] org.apache.hadoop.yarn.factories.impl.pb.RpcServerFactoryPBImpl: Adding protocol org.apache.hadoop.mapreduce.v2.api.MRClientProtocolPB to the server 2018-12-10 00:40:47,878 INFO [IPC Server Responder] org.apache.hadoop.ipc.Server: IPC Server Responder: starting 2018-12-10 00:40:47,878 INFO [IPC Server listener on 19346] org.apache.hadoop.ipc.Server: IPC Server listener on 19346: starting 2018-12-10 00:40:47,880 INFO [main] org.apache.hadoop.mapreduce.v2.app.client.MRClientService: Instantiated MRClientService at 192-168-72-93/192.168.72.93:19346 2018-12-10 00:40:47,962 INFO [main] org.mortbay.log: Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog 2018-12-10 00:40:47,971 INFO [main] org.apache.hadoop.security.authentication.server.AuthenticationFilter: Unable to initialize FileSignerSecretProvider, falling back to use random secrets. 2018-12-10 00:40:47,976 INFO [main] org.apache.hadoop.http.HttpRequestLog: Http request log for http.requests.mapreduce is not defined 2018-12-10 00:40:47,984 INFO [main] org.apache.hadoop.http.HttpServer2: Added global filter 'safety' (class=org.apache.hadoop.http.HttpServer2$QuotingInputFilter) 2018-12-10 00:40:48,036 INFO [main] org.apache.hadoop.http.HttpServer2: Added filter AM_PROXY_FILTER (class=org.apache.hadoop.yarn.server.webproxy.amfilter.AmIpFilter) to context mapreduce 2018-12-10 00:40:48,036 INFO [main] org.apache.hadoop.http.HttpServer2: Added filter AM_PROXY_FILTER (class=org.apache.hadoop.yarn.server.webproxy.amfilter.AmIpFilter) to context static 2018-12-10 00:40:48,039 INFO [main] org.apache.hadoop.http.HttpServer2: adding path spec: /mapreduce/* 2018-12-10 00:40:48,039 INFO [main] org.apache.hadoop.http.HttpServer2: adding path spec: /ws/* 2018-12-10 00:40:48,432 INFO [main] org.apache.hadoop.yarn.webapp.WebApps: Registered webapp guice modules 2018-12-10 00:40:48,434 INFO [main] org.apache.hadoop.http.HttpServer2: Jetty bound to port 31953 2018-12-10 00:40:48,434 INFO [main] org.mortbay.log: jetty-6.1.26 2018-12-10 00:40:48,472 INFO [main] org.mortbay.log: Extract jar:file:/opt/hadoop-2.7.3/share/hadoop/yarn/hadoop-yarn-common-2.7.3.jar!/webapps/mapreduce to /data0/hadoop/yarn/data/usercache/hadoop/appcache/application_1533196506314_4460157/container_e17_1533196506314_4460157_01_000001/tmp/Jetty_0_0_0_0_31953_mapreduce____.w5c3kw/webapp 2018-12-10 00:40:49,737 INFO [main] org.mortbay.log: Started HttpServer2$SelectChannelConnectorWithSafeStartup@0.0.0.0:31953 2018-12-10 00:40:49,738 INFO [main] org.apache.hadoop.yarn.webapp.WebApps: Web app mapreduce started at 31953 2018-12-10 00:40:49,741 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.speculate.DefaultSpeculator: JOB_CREATE job_1533196506314_4460157 2018-12-10 00:40:49,743 INFO [main] org.apache.hadoop.ipc.CallQueueManager: Using callQueue class java.util.concurrent.LinkedBlockingQueue 2018-12-10 00:40:49,744 INFO [Socket Reader #1 for port 8046] org.apache.hadoop.ipc.Server: Starting Socket Reader #1 for port 8046 2018-12-10 00:40:49,747 INFO [IPC Server Responder] org.apache.hadoop.ipc.Server: IPC Server Responder: starting 2018-12-10 00:40:49,747 INFO [IPC Server listener on 8046] org.apache.hadoop.ipc.Server: IPC Server listener on 8046: starting 2018-12-10 00:40:49,773 INFO [main] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerRequestor: nodeBlacklistingEnabled:true 2018-12-10 00:40:49,773 INFO [main] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerRequestor: maxTaskFailuresPerNode is 3 2018-12-10 00:40:49,773 INFO [main] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerRequestor: blacklistDisablePercent is 33 2018-12-10 00:40:49,863 INFO [main] org.apache.hadoop.yarn.client.ConfiguredRMFailoverProxyProvider: Failing over to ycluster-resourcemanager2 2018-12-10 00:40:49,893 INFO [main] org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator: maxContainerCapability: 2018-12-10 00:40:49,893 INFO [main] org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator: queue: root.hadoop 2018-12-10 00:40:49,897 INFO [main] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Upper limit on the thread pool size is 500 2018-12-10 00:40:49,897 INFO [main] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: The thread pool initial size is 10 2018-12-10 00:40:49,900 INFO [main] org.apache.hadoop.yarn.client.api.impl.ContainerManagementProtocolProxy: yarn.client.max-cached-nodemanagers-proxies : 0 2018-12-10 00:40:49,906 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: job_1533196506314_4460157Job Transitioned from INITED to SETUP 2018-12-10 00:40:49,909 INFO [CommitterEvent Processor #0] org.apache.hadoop.mapreduce.v2.app.commit.CommitterEventHandler: Processing the event EventType: JOB_SETUP 2018-12-10 00:40:49,915 INFO [CommitterEvent Processor #0] org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter: File Output Committer Algorithm version is 1 2018-12-10 00:40:49,925 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: job_1533196506314_4460157Job Transitioned from SETUP to RUNNING 2018-12-10 00:40:49,944 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: task_1533196506314_4460157_m_000000 Task Transitioned from NEW to SCHEDULED 2018-12-10 00:40:49,944 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: task_1533196506314_4460157_m_000001 Task Transitioned from NEW to SCHEDULED 2018-12-10 00:40:49,944 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: task_1533196506314_4460157_m_000002 Task Transitioned from NEW to SCHEDULED 2018-12-10 00:40:49,945 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: task_1533196506314_4460157_m_000003 Task Transitioned from NEW to SCHEDULED 2018-12-10 00:40:49,946 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1533196506314_4460157_m_000000_0 TaskAttempt Transitioned from NEW to UNASSIGNED 2018-12-10 00:40:49,946 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1533196506314_4460157_m_000001_0 TaskAttempt Transitioned from NEW to UNASSIGNED 2018-12-10 00:40:49,946 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1533196506314_4460157_m_000002_0 TaskAttempt Transitioned from NEW to UNASSIGNED 2018-12-10 00:40:49,946 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1533196506314_4460157_m_000003_0 TaskAttempt Transitioned from NEW to UNASSIGNED 2018-12-10 00:40:49,948 INFO [Thread-53] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: mapResourceRequest: 2018-12-10 00:40:50,000 INFO [eventHandlingThread] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Event Writer setup for JobId: job_1533196506314_4460157, File: hdfs://ycluster:8020/tmp/hadoop-yarn/staging/hadoop/.staging/job_1533196506314_4460157/job_1533196506314_4460157_1.jhist 2018-12-10 00:40:50,007 INFO [eventHandlingThread] org.apache.hadoop.conf.Configuration.deprecation: mapred.output.dir is deprecated. Instead, use mapreduce.output.fileoutputformat.outputdir 2018-12-10 00:40:50,895 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Before Scheduling: PendingReds:0 ScheduledMaps:4 ScheduledReds:0 AssignedMaps:0 AssignedReds:0 CompletedMaps:0 CompletedReds:0 ContAlloc:0 ContRel:0 HostLocal:0 RackLocal:0 2018-12-10 00:40:50,928 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerRequestor: getResources() for application_1533196506314_4460157: ask=1 release= 0 newContainers=0 finishedContainers=0 resourcelimit= knownNMs=39 2018-12-10 00:40:51,936 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Got allocated containers 4 2018-12-10 00:40:51,943 INFO [RMCommunicator Allocator] org.apache.hadoop.yarn.util.RackResolver: Resolved 192-168-72-24 to /S6700-XG0-XG1-XG2/K04 2018-12-10 00:40:51,949 INFO [RMCommunicator Allocator] org.apache.hadoop.yarn.util.RackResolver: Resolved 192-168-72-94 to /S6700-XG0-XG1-XG2/K09 2018-12-10 00:40:51,954 INFO [RMCommunicator Allocator] org.apache.hadoop.yarn.util.RackResolver: Resolved 192-168-72-93 to /S6700-XG0-XG1-XG2/K09 2018-12-10 00:40:51,959 INFO [RMCommunicator Allocator] org.apache.hadoop.yarn.util.RackResolver: Resolved 192-168-72-12 to /S6700-XG0-XG1-XG2/K02 2018-12-10 00:40:51,960 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Assigned container container_e17_1533196506314_4460157_01_000002 to attempt_1533196506314_4460157_m_000000_0 2018-12-10 00:40:51,962 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Assigned container container_e17_1533196506314_4460157_01_000003 to attempt_1533196506314_4460157_m_000001_0 2018-12-10 00:40:51,962 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Assigned container container_e17_1533196506314_4460157_01_000004 to attempt_1533196506314_4460157_m_000002_0 2018-12-10 00:40:51,962 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Assigned container container_e17_1533196506314_4460157_01_000005 to attempt_1533196506314_4460157_m_000003_0 2018-12-10 00:40:51,962 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: After Scheduling: PendingReds:0 ScheduledMaps:0 ScheduledReds:0 AssignedMaps:4 AssignedReds:0 CompletedMaps:0 CompletedReds:0 ContAlloc:4 ContRel:0 HostLocal:0 RackLocal:0 2018-12-10 00:40:52,010 INFO [AsyncDispatcher event handler] org.apache.hadoop.yarn.util.RackResolver: Resolved 192-168-72-24 to /S6700-XG0-XG1-XG2/K04 2018-12-10 00:40:52,025 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: The job-jar file on the remote FS is hdfs://ycluster/tmp/hadoop-yarn/staging/hadoop/.staging/job_1533196506314_4460157/job.jar 2018-12-10 00:40:52,026 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: The job-conf file on the remote FS is /tmp/hadoop-yarn/staging/hadoop/.staging/job_1533196506314_4460157/job.xml 2018-12-10 00:40:52,179 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: Adding #0 tokens and #2 secret keys for NM use for launching container 2018-12-10 00:40:52,180 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: Size of containertokens_dob is 1 2018-12-10 00:40:52,180 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: Putting shuffle token in serviceData 2018-12-10 00:40:52,583 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1533196506314_4460157_m_000000_0 TaskAttempt Transitioned from UNASSIGNED to ASSIGNED 2018-12-10 00:40:52,588 INFO [AsyncDispatcher event handler] org.apache.hadoop.yarn.util.RackResolver: Resolved 192-168-72-94 to /S6700-XG0-XG1-XG2/K09 2018-12-10 00:40:52,588 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1533196506314_4460157_m_000001_0 TaskAttempt Transitioned from UNASSIGNED to ASSIGNED 2018-12-10 00:40:52,589 INFO [AsyncDispatcher event handler] org.apache.hadoop.yarn.util.RackResolver: Resolved 192-168-72-93 to /S6700-XG0-XG1-XG2/K09 2018-12-10 00:40:52,589 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1533196506314_4460157_m_000002_0 TaskAttempt Transitioned from UNASSIGNED to ASSIGNED 2018-12-10 00:40:52,589 INFO [AsyncDispatcher event handler] org.apache.hadoop.yarn.util.RackResolver: Resolved 192-168-72-12 to /S6700-XG0-XG1-XG2/K02 2018-12-10 00:40:52,590 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1533196506314_4460157_m_000003_0 TaskAttempt Transitioned from UNASSIGNED to ASSIGNED 2018-12-10 00:40:52,591 INFO [ContainerLauncher #1] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Processing the event EventType: CONTAINER_REMOTE_LAUNCH for container container_e17_1533196506314_4460157_01_000003 taskAttempt attempt_1533196506314_4460157_m_000001_0 2018-12-10 00:40:52,591 INFO [ContainerLauncher #0] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Processing the event EventType: CONTAINER_REMOTE_LAUNCH for container container_e17_1533196506314_4460157_01_000002 taskAttempt attempt_1533196506314_4460157_m_000000_0 2018-12-10 00:40:52,591 INFO [ContainerLauncher #2] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Processing the event EventType: CONTAINER_REMOTE_LAUNCH for container container_e17_1533196506314_4460157_01_000004 taskAttempt attempt_1533196506314_4460157_m_000002_0 2018-12-10 00:40:52,591 INFO [ContainerLauncher #3] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Processing the event EventType: CONTAINER_REMOTE_LAUNCH for container container_e17_1533196506314_4460157_01_000005 taskAttempt attempt_1533196506314_4460157_m_000003_0 2018-12-10 00:40:52,594 INFO [ContainerLauncher #3] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Launching attempt_1533196506314_4460157_m_000003_0 2018-12-10 00:40:52,594 INFO [ContainerLauncher #0] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Launching attempt_1533196506314_4460157_m_000000_0 2018-12-10 00:40:52,594 INFO [ContainerLauncher #1] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Launching attempt_1533196506314_4460157_m_000001_0 2018-12-10 00:40:52,594 INFO [ContainerLauncher #2] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Launching attempt_1533196506314_4460157_m_000002_0 2018-12-10 00:40:52,595 INFO [ContainerLauncher #3] org.apache.hadoop.yarn.client.api.impl.ContainerManagementProtocolProxy: Opening proxy : 192-168-72-12:27463 2018-12-10 00:40:52,616 INFO [ContainerLauncher #2] org.apache.hadoop.yarn.client.api.impl.ContainerManagementProtocolProxy: Opening proxy : 192-168-72-93:53778 2018-12-10 00:40:52,617 INFO [ContainerLauncher #0] org.apache.hadoop.yarn.client.api.impl.ContainerManagementProtocolProxy: Opening proxy : 192-168-72-24:16310 2018-12-10 00:40:52,619 INFO [ContainerLauncher #1] org.apache.hadoop.yarn.client.api.impl.ContainerManagementProtocolProxy: Opening proxy : 192-168-72-94:54353 2018-12-10 00:40:52,666 INFO [ContainerLauncher #1] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Shuffle port returned by ContainerManager for attempt_1533196506314_4460157_m_000001_0 : 13562 2018-12-10 00:40:52,666 INFO [ContainerLauncher #2] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Shuffle port returned by ContainerManager for attempt_1533196506314_4460157_m_000002_0 : 13562 2018-12-10 00:40:52,666 INFO [ContainerLauncher #3] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Shuffle port returned by ContainerManager for attempt_1533196506314_4460157_m_000003_0 : 13562 2018-12-10 00:40:52,666 INFO [ContainerLauncher #0] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Shuffle port returned by ContainerManager for attempt_1533196506314_4460157_m_000000_0 : 13562 2018-12-10 00:40:52,668 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: TaskAttempt: [attempt_1533196506314_4460157_m_000003_0] using containerId: [container_e17_1533196506314_4460157_01_000005 on NM: [192-168-72-12:27463] 2018-12-10 00:40:52,671 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1533196506314_4460157_m_000003_0 TaskAttempt Transitioned from ASSIGNED to RUNNING 2018-12-10 00:40:52,671 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: TaskAttempt: [attempt_1533196506314_4460157_m_000001_0] using containerId: [container_e17_1533196506314_4460157_01_000003 on NM: [192-168-72-94:54353] 2018-12-10 00:40:52,671 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1533196506314_4460157_m_000001_0 TaskAttempt Transitioned from ASSIGNED to RUNNING 2018-12-10 00:40:52,671 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: TaskAttempt: [attempt_1533196506314_4460157_m_000002_0] using containerId: [container_e17_1533196506314_4460157_01_000004 on NM: [192-168-72-93:53778] 2018-12-10 00:40:52,672 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1533196506314_4460157_m_000002_0 TaskAttempt Transitioned from ASSIGNED to RUNNING 2018-12-10 00:40:52,672 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: TaskAttempt: [attempt_1533196506314_4460157_m_000000_0] using containerId: [container_e17_1533196506314_4460157_01_000002 on NM: [192-168-72-24:16310] 2018-12-10 00:40:52,672 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1533196506314_4460157_m_000000_0 TaskAttempt Transitioned from ASSIGNED to RUNNING 2018-12-10 00:40:52,672 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.speculate.DefaultSpeculator: ATTEMPT_START task_1533196506314_4460157_m_000003 2018-12-10 00:40:52,672 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: task_1533196506314_4460157_m_000003 Task Transitioned from SCHEDULED to RUNNING 2018-12-10 00:40:52,672 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.speculate.DefaultSpeculator: ATTEMPT_START task_1533196506314_4460157_m_000001 2018-12-10 00:40:52,672 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: task_1533196506314_4460157_m_000001 Task Transitioned from SCHEDULED to RUNNING 2018-12-10 00:40:52,673 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.speculate.DefaultSpeculator: ATTEMPT_START task_1533196506314_4460157_m_000002 2018-12-10 00:40:52,673 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: task_1533196506314_4460157_m_000002 Task Transitioned from SCHEDULED to RUNNING 2018-12-10 00:40:52,673 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.speculate.DefaultSpeculator: ATTEMPT_START task_1533196506314_4460157_m_000000 2018-12-10 00:40:52,673 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: task_1533196506314_4460157_m_000000 Task Transitioned from SCHEDULED to RUNNING 2018-12-10 00:40:52,963 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerRequestor: getResources() for application_1533196506314_4460157: ask=1 release= 0 newContainers=0 finishedContainers=0 resourcelimit= knownNMs=39 2018-12-10 00:40:54,643 INFO [Socket Reader #1 for port 8046] SecurityLogger.org.apache.hadoop.ipc.Server: Auth successful for job_1533196506314_4460157 (auth:SIMPLE) 2018-12-10 00:40:54,661 INFO [IPC Server handler 1 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: JVM with ID : jvm_1533196506314_4460157_m_18691697672196 asked for a task 2018-12-10 00:40:54,661 INFO [IPC Server handler 1 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: JVM with ID: jvm_1533196506314_4460157_m_18691697672196 given task: attempt_1533196506314_4460157_m_000002_0 2018-12-10 00:40:59,550 INFO [Socket Reader #1 for port 8046] SecurityLogger.org.apache.hadoop.ipc.Server: Auth successful for job_1533196506314_4460157 (auth:SIMPLE) 2018-12-10 00:40:59,563 INFO [IPC Server handler 7 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: JVM with ID : jvm_1533196506314_4460157_m_18691697672195 asked for a task 2018-12-10 00:40:59,563 INFO [IPC Server handler 7 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: JVM with ID: jvm_1533196506314_4460157_m_18691697672195 given task: attempt_1533196506314_4460157_m_000001_0 2018-12-10 00:40:59,909 INFO [Socket Reader #1 for port 8046] SecurityLogger.org.apache.hadoop.ipc.Server: Auth successful for job_1533196506314_4460157 (auth:SIMPLE) 2018-12-10 00:40:59,920 INFO [IPC Server handler 1 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: JVM with ID : jvm_1533196506314_4460157_m_18691697672194 asked for a task 2018-12-10 00:40:59,920 INFO [IPC Server handler 1 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: JVM with ID: jvm_1533196506314_4460157_m_18691697672194 given task: attempt_1533196506314_4460157_m_000000_0 2018-12-10 00:40:59,993 INFO [Socket Reader #1 for port 8046] SecurityLogger.org.apache.hadoop.ipc.Server: Auth successful for job_1533196506314_4460157 (auth:SIMPLE) 2018-12-10 00:41:00,004 INFO [IPC Server handler 7 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: JVM with ID : jvm_1533196506314_4460157_m_18691697672197 asked for a task 2018-12-10 00:41:00,005 INFO [IPC Server handler 7 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: JVM with ID: jvm_1533196506314_4460157_m_18691697672197 given task: attempt_1533196506314_4460157_m_000003_0 2018-12-10 00:41:01,632 INFO [IPC Server handler 4 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000002_0 is : 0.0 2018-12-10 00:41:04,681 INFO [IPC Server handler 6 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000002_0 is : 0.0 2018-12-10 00:41:06,468 INFO [IPC Server handler 6 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000001_0 is : 0.0 2018-12-10 00:41:06,749 INFO [IPC Server handler 2 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000003_0 is : 0.0 2018-12-10 00:41:06,755 INFO [IPC Server handler 1 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000000_0 is : 0.0 2018-12-10 00:41:07,732 INFO [IPC Server handler 15 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000002_0 is : 0.0 2018-12-10 00:41:09,518 INFO [IPC Server handler 3 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000001_0 is : 0.0 2018-12-10 00:41:09,789 INFO [IPC Server handler 7 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000003_0 is : 0.0 2018-12-10 00:41:09,791 INFO [IPC Server handler 4 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000000_0 is : 0.0 2018-12-10 00:41:10,794 INFO [IPC Server handler 6 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000002_0 is : 0.0 2018-12-10 00:41:12,569 INFO [IPC Server handler 15 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000001_0 is : 0.0 2018-12-10 00:41:12,820 INFO [IPC Server handler 3 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000003_0 is : 0.0 2018-12-10 00:41:12,844 INFO [IPC Server handler 15 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000000_0 is : 0.0 2018-12-10 00:41:13,855 INFO [IPC Server handler 12 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000002_0 is : 0.0 2018-12-10 00:41:15,629 INFO [IPC Server handler 5 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000001_0 is : 0.0 2018-12-10 00:41:15,870 INFO [IPC Server handler 5 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000003_0 is : 0.0 2018-12-10 00:41:15,873 INFO [IPC Server handler 8 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000000_0 is : 0.0 2018-12-10 00:41:16,889 INFO [IPC Server handler 14 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000002_0 is : 0.0 2018-12-10 00:41:18,684 INFO [IPC Server handler 19 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000001_0 is : 0.0 2018-12-10 00:41:18,914 INFO [IPC Server handler 19 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000003_0 is : 0.0 2018-12-10 00:41:18,917 INFO [IPC Server handler 11 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000000_0 is : 0.0 2018-12-10 00:41:19,922 INFO [IPC Server handler 10 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000002_0 is : 0.0 2018-12-10 00:41:21,724 INFO [IPC Server handler 2 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000001_0 is : 0.0 2018-12-10 00:41:21,946 INFO [IPC Server handler 2 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000000_0 is : 0.0 2018-12-10 00:41:21,948 INFO [IPC Server handler 22 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000003_0 is : 0.0 2018-12-10 00:41:22,963 INFO [IPC Server handler 13 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000002_0 is : 0.0 2018-12-10 00:41:24,774 INFO [IPC Server handler 7 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000001_0 is : 0.0 2018-12-10 00:41:24,973 INFO [IPC Server handler 17 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000003_0 is : 0.0 2018-12-10 00:41:24,975 INFO [IPC Server handler 0 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000000_0 is : 0.0 2018-12-10 00:41:25,992 INFO [IPC Server handler 26 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000002_0 is : 0.0 2018-12-10 00:41:27,810 INFO [IPC Server handler 3 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000001_0 is : 0.0 2018-12-10 00:41:28,001 INFO [IPC Server handler 21 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000003_0 is : 0.0 2018-12-10 00:41:28,005 INFO [IPC Server handler 25 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000000_0 is : 0.0 2018-12-10 00:41:29,037 INFO [IPC Server handler 20 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000002_0 is : 0.0 2018-12-10 00:41:30,858 INFO [IPC Server handler 12 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000001_0 is : 0.0 2018-12-10 00:41:31,041 INFO [IPC Server handler 9 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000003_0 is : 0.0 2018-12-10 00:41:31,058 INFO [IPC Server handler 29 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000000_0 is : 0.0 2018-12-10 00:41:32,065 INFO [IPC Server handler 24 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000002_0 is : 0.0 2018-12-10 00:41:33,894 INFO [IPC Server handler 19 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000001_0 is : 0.0 2018-12-10 00:41:34,081 INFO [IPC Server handler 27 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000003_0 is : 0.0 2018-12-10 00:41:34,090 INFO [IPC Server handler 28 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000000_0 is : 0.0 2018-12-10 00:41:35,094 INFO [IPC Server handler 23 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000002_0 is : 0.0 2018-12-10 00:41:36,939 INFO [IPC Server handler 2 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000001_0 is : 0.0 2018-12-10 00:41:37,118 INFO [IPC Server handler 18 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000003_0 is : 0.0 2018-12-10 00:41:37,120 INFO [IPC Server handler 16 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000000_0 is : 0.0 2018-12-10 00:41:38,123 INFO [IPC Server handler 1 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000002_0 is : 0.0 2018-12-10 00:41:39,970 INFO [IPC Server handler 17 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000001_0 is : 0.0 2018-12-10 00:41:40,143 INFO [IPC Server handler 7 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000003_0 is : 0.0 2018-12-10 00:41:40,160 INFO [IPC Server handler 4 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000000_0 is : 0.0 2018-12-10 00:41:40,383 INFO [IPC Server handler 6 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000003_0 is : 0.0 2018-12-10 00:41:40,672 INFO [IPC Server handler 3 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Commit-pending state update from attempt_1533196506314_4460157_m_000003_0 2018-12-10 00:41:40,672 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1533196506314_4460157_m_000003_0 TaskAttempt Transitioned from RUNNING to COMMIT_PENDING 2018-12-10 00:41:40,673 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: attempt_1533196506314_4460157_m_000003_0 given a go for committing the task output. 2018-12-10 00:41:40,674 INFO [IPC Server handler 15 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Commit go/no-go request from attempt_1533196506314_4460157_m_000003_0 2018-12-10 00:41:40,675 INFO [IPC Server handler 15 on 8046] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: Result of canCommit for attempt_1533196506314_4460157_m_000003_0:true 2018-12-10 00:41:40,712 INFO [IPC Server handler 12 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000003_0 is : 1.0 2018-12-10 00:41:40,714 INFO [IPC Server handler 5 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Done acknowledgement from attempt_1533196506314_4460157_m_000003_0 2018-12-10 00:41:40,716 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1533196506314_4460157_m_000003_0 TaskAttempt Transitioned from COMMIT_PENDING to SUCCESS_CONTAINER_CLEANUP 2018-12-10 00:41:40,717 INFO [ContainerLauncher #4] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Processing the event EventType: CONTAINER_REMOTE_CLEANUP for container container_e17_1533196506314_4460157_01_000005 taskAttempt attempt_1533196506314_4460157_m_000003_0 2018-12-10 00:41:40,717 INFO [ContainerLauncher #4] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: KILLING attempt_1533196506314_4460157_m_000003_0 2018-12-10 00:41:40,718 INFO [ContainerLauncher #4] org.apache.hadoop.yarn.client.api.impl.ContainerManagementProtocolProxy: Opening proxy : 192-168-72-12:27463 2018-12-10 00:41:40,734 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1533196506314_4460157_m_000003_0 TaskAttempt Transitioned from SUCCESS_CONTAINER_CLEANUP to SUCCEEDED 2018-12-10 00:41:40,743 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: Task succeeded with attempt attempt_1533196506314_4460157_m_000003_0 2018-12-10 00:41:40,744 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: task_1533196506314_4460157_m_000003 Task Transitioned from RUNNING to SUCCEEDED 2018-12-10 00:41:40,746 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: Num completed Tasks: 1 2018-12-10 00:41:40,764 INFO [DefaultSpeculator background processing] org.apache.hadoop.mapreduce.v2.app.speculate.DefaultSpeculator: DefaultSpeculator.addSpeculativeAttempt -- we are speculating task_1533196506314_4460157_m_000000 2018-12-10 00:41:40,764 INFO [DefaultSpeculator background processing] org.apache.hadoop.mapreduce.v2.app.speculate.DefaultSpeculator: We launched 1 speculations. Sleeping 15000 milliseconds. 2018-12-10 00:41:40,764 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: Scheduling a redundant attempt for task task_1533196506314_4460157_m_000000 2018-12-10 00:41:40,765 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1533196506314_4460157_m_000000_1 TaskAttempt Transitioned from NEW to UNASSIGNED 2018-12-10 00:41:41,013 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Before Scheduling: PendingReds:0 ScheduledMaps:1 ScheduledReds:0 AssignedMaps:4 AssignedReds:0 CompletedMaps:1 CompletedReds:0 ContAlloc:4 ContRel:0 HostLocal:0 RackLocal:0 2018-12-10 00:41:41,018 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerRequestor: getResources() for application_1533196506314_4460157: ask=1 release= 0 newContainers=0 finishedContainers=1 resourcelimit= knownNMs=39 2018-12-10 00:41:41,018 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Received completed container container_e17_1533196506314_4460157_01_000005 2018-12-10 00:41:41,019 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: After Scheduling: PendingReds:0 ScheduledMaps:1 ScheduledReds:0 AssignedMaps:3 AssignedReds:0 CompletedMaps:1 CompletedReds:0 ContAlloc:4 ContRel:0 HostLocal:0 RackLocal:0 2018-12-10 00:41:41,019 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: Diagnostics report from attempt_1533196506314_4460157_m_000003_0: Container killed by the ApplicationMaster. Container killed on request. Exit code is 143 Container exited with a non-zero exit code 143 2018-12-10 00:41:41,152 INFO [IPC Server handler 4 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000002_0 is : 0.0 2018-12-10 00:41:41,313 INFO [IPC Server handler 6 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000002_0 is : 0.0 2018-12-10 00:41:41,935 INFO [IPC Server handler 2 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Commit-pending state update from attempt_1533196506314_4460157_m_000002_0 2018-12-10 00:41:41,936 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1533196506314_4460157_m_000002_0 TaskAttempt Transitioned from RUNNING to COMMIT_PENDING 2018-12-10 00:41:41,936 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: attempt_1533196506314_4460157_m_000002_0 given a go for committing the task output. 2018-12-10 00:41:41,937 INFO [IPC Server handler 22 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Commit go/no-go request from attempt_1533196506314_4460157_m_000002_0 2018-12-10 00:41:41,937 INFO [IPC Server handler 22 on 8046] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: Result of canCommit for attempt_1533196506314_4460157_m_000002_0:true 2018-12-10 00:41:41,971 INFO [IPC Server handler 0 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000002_0 is : 1.0 2018-12-10 00:41:41,972 INFO [IPC Server handler 26 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Done acknowledgement from attempt_1533196506314_4460157_m_000002_0 2018-12-10 00:41:41,973 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1533196506314_4460157_m_000002_0 TaskAttempt Transitioned from COMMIT_PENDING to SUCCESS_CONTAINER_CLEANUP 2018-12-10 00:41:41,974 INFO [ContainerLauncher #5] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Processing the event EventType: CONTAINER_REMOTE_CLEANUP for container container_e17_1533196506314_4460157_01_000004 taskAttempt attempt_1533196506314_4460157_m_000002_0 2018-12-10 00:41:41,974 INFO [ContainerLauncher #5] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: KILLING attempt_1533196506314_4460157_m_000002_0 2018-12-10 00:41:41,974 INFO [ContainerLauncher #5] org.apache.hadoop.yarn.client.api.impl.ContainerManagementProtocolProxy: Opening proxy : 192-168-72-93:53778 2018-12-10 00:41:41,979 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1533196506314_4460157_m_000002_0 TaskAttempt Transitioned from SUCCESS_CONTAINER_CLEANUP to SUCCEEDED 2018-12-10 00:41:41,979 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: Task succeeded with attempt attempt_1533196506314_4460157_m_000002_0 2018-12-10 00:41:41,979 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: task_1533196506314_4460157_m_000002 Task Transitioned from RUNNING to SUCCEEDED 2018-12-10 00:41:41,980 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: Num completed Tasks: 2 2018-12-10 00:41:42,019 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Before Scheduling: PendingReds:0 ScheduledMaps:1 ScheduledReds:0 AssignedMaps:3 AssignedReds:0 CompletedMaps:2 CompletedReds:0 ContAlloc:4 ContRel:0 HostLocal:0 RackLocal:0 2018-12-10 00:41:42,020 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Received completed container container_e17_1533196506314_4460157_01_000004 2018-12-10 00:41:42,020 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Got allocated containers 1 2018-12-10 00:41:42,020 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: Diagnostics report from attempt_1533196506314_4460157_m_000002_0: Container killed by the ApplicationMaster. Container killed on request. Exit code is 143 Container exited with a non-zero exit code 143 2018-12-10 00:41:42,026 INFO [RMCommunicator Allocator] org.apache.hadoop.yarn.util.RackResolver: Resolved 192-168-72-23 to /S6700-XG0-XG1-XG2/K04 2018-12-10 00:41:42,026 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Assigned container container_e17_1533196506314_4460157_01_000006 to attempt_1533196506314_4460157_m_000000_1 2018-12-10 00:41:42,026 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: After Scheduling: PendingReds:0 ScheduledMaps:0 ScheduledReds:0 AssignedMaps:3 AssignedReds:0 CompletedMaps:2 CompletedReds:0 ContAlloc:5 ContRel:0 HostLocal:0 RackLocal:0 2018-12-10 00:41:42,027 INFO [AsyncDispatcher event handler] org.apache.hadoop.yarn.util.RackResolver: Resolved 192-168-72-23 to /S6700-XG0-XG1-XG2/K04 2018-12-10 00:41:42,028 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1533196506314_4460157_m_000000_1 TaskAttempt Transitioned from UNASSIGNED to ASSIGNED 2018-12-10 00:41:42,028 INFO [ContainerLauncher #6] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Processing the event EventType: CONTAINER_REMOTE_LAUNCH for container container_e17_1533196506314_4460157_01_000006 taskAttempt attempt_1533196506314_4460157_m_000000_1 2018-12-10 00:41:42,028 INFO [ContainerLauncher #6] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Launching attempt_1533196506314_4460157_m_000000_1 2018-12-10 00:41:42,028 INFO [ContainerLauncher #6] org.apache.hadoop.yarn.client.api.impl.ContainerManagementProtocolProxy: Opening proxy : 192-168-72-23:18284 2018-12-10 00:41:42,036 INFO [ContainerLauncher #6] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Shuffle port returned by ContainerManager for attempt_1533196506314_4460157_m_000000_1 : 13562 2018-12-10 00:41:42,036 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: TaskAttempt: [attempt_1533196506314_4460157_m_000000_1] using containerId: [container_e17_1533196506314_4460157_01_000006 on NM: [192-168-72-23:18284] 2018-12-10 00:41:42,036 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1533196506314_4460157_m_000000_1 TaskAttempt Transitioned from ASSIGNED to RUNNING 2018-12-10 00:41:42,036 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.speculate.DefaultSpeculator: ATTEMPT_START task_1533196506314_4460157_m_000000 2018-12-10 00:41:43,005 INFO [IPC Server handler 25 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000001_0 is : 0.0 2018-12-10 00:41:43,027 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerRequestor: getResources() for application_1533196506314_4460157: ask=1 release= 0 newContainers=0 finishedContainers=0 resourcelimit= knownNMs=39 2018-12-10 00:41:43,208 INFO [IPC Server handler 6 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000000_0 is : 0.0 2018-12-10 00:41:46,035 INFO [IPC Server handler 20 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000001_0 is : 0.0 2018-12-10 00:41:46,243 INFO [IPC Server handler 3 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000000_0 is : 0.0 2018-12-10 00:41:49,064 INFO [IPC Server handler 24 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000001_0 is : 0.0 2018-12-10 00:41:49,284 INFO [IPC Server handler 15 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000000_0 is : 0.0 2018-12-10 00:41:51,144 INFO [Socket Reader #1 for port 8046] SecurityLogger.org.apache.hadoop.ipc.Server: Auth successful for job_1533196506314_4460157 (auth:SIMPLE) 2018-12-10 00:41:51,154 INFO [IPC Server handler 6 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: JVM with ID : jvm_1533196506314_4460157_m_18691697672198 asked for a task 2018-12-10 00:41:51,155 INFO [IPC Server handler 6 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: JVM with ID: jvm_1533196506314_4460157_m_18691697672198 given task: attempt_1533196506314_4460157_m_000000_1 2018-12-10 00:41:52,096 INFO [IPC Server handler 23 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000001_0 is : 0.0 2018-12-10 00:41:52,315 INFO [IPC Server handler 12 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000000_0 is : 0.0 2018-12-10 00:41:55,124 INFO [IPC Server handler 1 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000001_0 is : 0.0 2018-12-10 00:41:55,359 INFO [IPC Server handler 5 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000000_0 is : 0.0 2018-12-10 00:41:55,765 INFO [DefaultSpeculator background processing] org.apache.hadoop.mapreduce.v2.app.speculate.DefaultSpeculator: DefaultSpeculator.addSpeculativeAttempt -- we are speculating task_1533196506314_4460157_m_000001 2018-12-10 00:41:55,765 INFO [DefaultSpeculator background processing] org.apache.hadoop.mapreduce.v2.app.speculate.DefaultSpeculator: We launched 1 speculations. Sleeping 15000 milliseconds. 2018-12-10 00:41:55,765 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: Scheduling a redundant attempt for task task_1533196506314_4460157_m_000001 2018-12-10 00:41:55,765 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1533196506314_4460157_m_000001_1 TaskAttempt Transitioned from NEW to UNASSIGNED 2018-12-10 00:41:56,045 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Before Scheduling: PendingReds:0 ScheduledMaps:1 ScheduledReds:0 AssignedMaps:3 AssignedReds:0 CompletedMaps:2 CompletedReds:0 ContAlloc:5 ContRel:0 HostLocal:0 RackLocal:0 2018-12-10 00:41:56,047 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerRequestor: getResources() for application_1533196506314_4460157: ask=1 release= 0 newContainers=0 finishedContainers=0 resourcelimit= knownNMs=39 2018-12-10 00:41:57,048 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Got allocated containers 1 2018-12-10 00:41:57,054 INFO [RMCommunicator Allocator] org.apache.hadoop.yarn.util.RackResolver: Resolved 192-168-72-73 to /S6700-XG0-XG1-XG2/K08 2018-12-10 00:41:57,055 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Assigned container container_e17_1533196506314_4460157_01_000007 to attempt_1533196506314_4460157_m_000001_1 2018-12-10 00:41:57,055 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: After Scheduling: PendingReds:0 ScheduledMaps:0 ScheduledReds:0 AssignedMaps:4 AssignedReds:0 CompletedMaps:2 CompletedReds:0 ContAlloc:6 ContRel:0 HostLocal:0 RackLocal:0 2018-12-10 00:41:57,055 INFO [AsyncDispatcher event handler] org.apache.hadoop.yarn.util.RackResolver: Resolved 192-168-72-73 to /S6700-XG0-XG1-XG2/K08 2018-12-10 00:41:57,056 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1533196506314_4460157_m_000001_1 TaskAttempt Transitioned from UNASSIGNED to ASSIGNED 2018-12-10 00:41:57,057 INFO [ContainerLauncher #7] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Processing the event EventType: CONTAINER_REMOTE_LAUNCH for container container_e17_1533196506314_4460157_01_000007 taskAttempt attempt_1533196506314_4460157_m_000001_1 2018-12-10 00:41:57,057 INFO [ContainerLauncher #7] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Launching attempt_1533196506314_4460157_m_000001_1 2018-12-10 00:41:57,057 INFO [ContainerLauncher #7] org.apache.hadoop.yarn.client.api.impl.ContainerManagementProtocolProxy: Opening proxy : 192-168-72-73:2363 2018-12-10 00:41:57,064 INFO [ContainerLauncher #7] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Shuffle port returned by ContainerManager for attempt_1533196506314_4460157_m_000001_1 : 13562 2018-12-10 00:41:57,064 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: TaskAttempt: [attempt_1533196506314_4460157_m_000001_1] using containerId: [container_e17_1533196506314_4460157_01_000007 on NM: [192-168-72-73:2363] 2018-12-10 00:41:57,064 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1533196506314_4460157_m_000001_1 TaskAttempt Transitioned from ASSIGNED to RUNNING 2018-12-10 00:41:57,064 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.speculate.DefaultSpeculator: ATTEMPT_START task_1533196506314_4460157_m_000001 2018-12-10 00:41:57,909 INFO [IPC Server handler 11 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000000_1 is : 0.0 2018-12-10 00:41:58,056 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerRequestor: getResources() for application_1533196506314_4460157: ask=1 release= 0 newContainers=0 finishedContainers=0 resourcelimit= knownNMs=39 2018-12-10 00:41:58,163 INFO [IPC Server handler 3 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000001_0 is : 0.0 2018-12-10 00:41:58,401 INFO [IPC Server handler 8 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000000_0 is : 0.0 2018-12-10 00:42:00,955 INFO [IPC Server handler 13 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000000_1 is : 0.0 2018-12-10 00:42:01,190 INFO [IPC Server handler 15 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000001_0 is : 0.0 2018-12-10 00:42:01,460 INFO [IPC Server handler 14 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000000_0 is : 0.0 2018-12-10 00:42:04,001 INFO [IPC Server handler 21 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000000_1 is : 0.0 2018-12-10 00:42:04,217 INFO [IPC Server handler 12 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000001_0 is : 0.0 2018-12-10 00:42:04,505 INFO [IPC Server handler 19 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000000_0 is : 0.0 2018-12-10 00:42:06,973 INFO [Socket Reader #1 for port 8046] SecurityLogger.org.apache.hadoop.ipc.Server: Auth successful for job_1533196506314_4460157 (auth:SIMPLE) 2018-12-10 00:42:06,983 INFO [IPC Server handler 21 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: JVM with ID : jvm_1533196506314_4460157_m_18691697672199 asked for a task 2018-12-10 00:42:06,983 INFO [IPC Server handler 21 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: JVM with ID: jvm_1533196506314_4460157_m_18691697672199 given task: attempt_1533196506314_4460157_m_000001_1 2018-12-10 00:42:07,045 INFO [IPC Server handler 9 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000000_1 is : 0.0 2018-12-10 00:42:07,245 INFO [IPC Server handler 5 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000001_0 is : 0.0 2018-12-10 00:42:07,544 INFO [IPC Server handler 11 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000000_0 is : 0.0 2018-12-10 00:42:08,803 INFO [IPC Server handler 10 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000001_1 is : 0.0 2018-12-10 00:42:08,843 FATAL [IPC Server handler 2 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Task: attempt_1533196506314_4460157_m_000001_1 - exited : java.lang.RuntimeException: java.lang.RuntimeException: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server. at org.apache.sqoop.mapreduce.db.DBInputFormat.setConf(DBInputFormat.java:167) at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:76) at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:136) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:751) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341) at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1698) at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158) Caused by: java.lang.RuntimeException: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server. at org.apache.sqoop.mapreduce.db.DBInputFormat.getConnection(DBInputFormat.java:220) at org.apache.sqoop.mapreduce.db.DBInputFormat.setConf(DBInputFormat.java:165) ... 9 more Caused by: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.mysql.jdbc.Util.handleNewInstance(Util.java:425) at com.mysql.jdbc.SQLError.createCommunicationsException(SQLError.java:989) at com.mysql.jdbc.MysqlIO.(MysqlIO.java:341) at com.mysql.jdbc.ConnectionImpl.coreConnect(ConnectionImpl.java:2192) at com.mysql.jdbc.ConnectionImpl.connectOneTryOnly(ConnectionImpl.java:2225) at com.mysql.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:2024) at com.mysql.jdbc.ConnectionImpl.(ConnectionImpl.java:779) at com.mysql.jdbc.JDBC4Connection.(JDBC4Connection.java:47) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.mysql.jdbc.Util.handleNewInstance(Util.java:425) at com.mysql.jdbc.ConnectionImpl.getInstance(ConnectionImpl.java:389) at com.mysql.jdbc.NonRegisteringDriver.connect(NonRegisteringDriver.java:330) at java.sql.DriverManager.getConnection(DriverManager.java:664) at java.sql.DriverManager.getConnection(DriverManager.java:247) at org.apache.sqoop.mapreduce.db.DBConfiguration.getConnection(DBConfiguration.java:302) at org.apache.sqoop.mapreduce.db.DBInputFormat.getConnection(DBInputFormat.java:213) ... 10 more Caused by: java.net.NoRouteToHostException: 没有到主机的路由 (Host unreachable) at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) at java.net.Socket.connect(Socket.java:589) at com.mysql.jdbc.StandardSocketFactory.connect(StandardSocketFactory.java:211) at com.mysql.jdbc.MysqlIO.(MysqlIO.java:300) ... 26 more 2018-12-10 00:42:08,843 INFO [IPC Server handler 2 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Diagnostics report from attempt_1533196506314_4460157_m_000001_1: Error: java.lang.RuntimeException: java.lang.RuntimeException: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server. at org.apache.sqoop.mapreduce.db.DBInputFormat.setConf(DBInputFormat.java:167) at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:76) at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:136) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:751) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341) at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1698) at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158) Caused by: java.lang.RuntimeException: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server. at org.apache.sqoop.mapreduce.db.DBInputFormat.getConnection(DBInputFormat.java:220) at org.apache.sqoop.mapreduce.db.DBInputFormat.setConf(DBInputFormat.java:165) ... 9 more Caused by: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.mysql.jdbc.Util.handleNewInstance(Util.java:425) at com.mysql.jdbc.SQLError.createCommunicationsException(SQLError.java:989) at com.mysql.jdbc.MysqlIO.(MysqlIO.java:341) at com.mysql.jdbc.ConnectionImpl.coreConnect(ConnectionImpl.java:2192) at com.mysql.jdbc.ConnectionImpl.connectOneTryOnly(ConnectionImpl.java:2225) at com.mysql.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:2024) at com.mysql.jdbc.ConnectionImpl.(ConnectionImpl.java:779) at com.mysql.jdbc.JDBC4Connection.(JDBC4Connection.java:47) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.mysql.jdbc.Util.handleNewInstance(Util.java:425) at com.mysql.jdbc.ConnectionImpl.getInstance(ConnectionImpl.java:389) at com.mysql.jdbc.NonRegisteringDriver.connect(NonRegisteringDriver.java:330) at java.sql.DriverManager.getConnection(DriverManager.java:664) at java.sql.DriverManager.getConnection(DriverManager.java:247) at org.apache.sqoop.mapreduce.db.DBConfiguration.getConnection(DBConfiguration.java:302) at org.apache.sqoop.mapreduce.db.DBInputFormat.getConnection(DBInputFormat.java:213) ... 10 more Caused by: java.net.NoRouteToHostException: 没有到主机的路由 (Host unreachable) at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) at java.net.Socket.connect(Socket.java:589) at com.mysql.jdbc.StandardSocketFactory.connect(StandardSocketFactory.java:211) at com.mysql.jdbc.MysqlIO.(MysqlIO.java:300) ... 26 more 2018-12-10 00:42:08,843 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: Diagnostics report from attempt_1533196506314_4460157_m_000001_1: Error: java.lang.RuntimeException: java.lang.RuntimeException: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server. at org.apache.sqoop.mapreduce.db.DBInputFormat.setConf(DBInputFormat.java:167) at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:76) at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:136) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:751) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341) at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1698) at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158) Caused by: java.lang.RuntimeException: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server. at org.apache.sqoop.mapreduce.db.DBInputFormat.getConnection(DBInputFormat.java:220) at org.apache.sqoop.mapreduce.db.DBInputFormat.setConf(DBInputFormat.java:165) ... 9 more Caused by: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.mysql.jdbc.Util.handleNewInstance(Util.java:425) at com.mysql.jdbc.SQLError.createCommunicationsException(SQLError.java:989) at com.mysql.jdbc.MysqlIO.(MysqlIO.java:341) at com.mysql.jdbc.ConnectionImpl.coreConnect(ConnectionImpl.java:2192) at com.mysql.jdbc.ConnectionImpl.connectOneTryOnly(ConnectionImpl.java:2225) at com.mysql.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:2024) at com.mysql.jdbc.ConnectionImpl.(ConnectionImpl.java:779) at com.mysql.jdbc.JDBC4Connection.(JDBC4Connection.java:47) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.mysql.jdbc.Util.handleNewInstance(Util.java:425) at com.mysql.jdbc.ConnectionImpl.getInstance(ConnectionImpl.java:389) at com.mysql.jdbc.NonRegisteringDriver.connect(NonRegisteringDriver.java:330) at java.sql.DriverManager.getConnection(DriverManager.java:664) at java.sql.DriverManager.getConnection(DriverManager.java:247) at org.apache.sqoop.mapreduce.db.DBConfiguration.getConnection(DBConfiguration.java:302) at org.apache.sqoop.mapreduce.db.DBInputFormat.getConnection(DBInputFormat.java:213) ... 10 more Caused by: java.net.NoRouteToHostException: 没有到主机的路由 (Host unreachable) at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) at java.net.Socket.connect(Socket.java:589) at com.mysql.jdbc.StandardSocketFactory.connect(StandardSocketFactory.java:211) at com.mysql.jdbc.MysqlIO.(MysqlIO.java:300) ... 26 more 2018-12-10 00:42:08,843 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1533196506314_4460157_m_000001_1 TaskAttempt Transitioned from RUNNING to FAIL_CONTAINER_CLEANUP 2018-12-10 00:42:08,844 INFO [ContainerLauncher #8] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Processing the event EventType: CONTAINER_REMOTE_CLEANUP for container container_e17_1533196506314_4460157_01_000007 taskAttempt attempt_1533196506314_4460157_m_000001_1 2018-12-10 00:42:08,845 INFO [ContainerLauncher #8] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: KILLING attempt_1533196506314_4460157_m_000001_1 2018-12-10 00:42:08,845 INFO [ContainerLauncher #8] org.apache.hadoop.yarn.client.api.impl.ContainerManagementProtocolProxy: Opening proxy : 192-168-72-73:2363 2018-12-10 00:42:08,852 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1533196506314_4460157_m_000001_1 TaskAttempt Transitioned from FAIL_CONTAINER_CLEANUP to FAIL_TASK_CLEANUP 2018-12-10 00:42:08,853 INFO [CommitterEvent Processor #1] org.apache.hadoop.mapreduce.v2.app.commit.CommitterEventHandler: Processing the event EventType: TASK_ABORT 2018-12-10 00:42:08,860 WARN [CommitterEvent Processor #1] org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter: Could not delete hdfs://ycluster/user/hive/warehouse/ods.db/bss_customer_fj/_SCRATCH0.3130352759450352/dt=20181209/_temporary/1/_temporary/attempt_1533196506314_4460157_m_000000_0 2018-12-10 00:42:08,862 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1533196506314_4460157_m_000001_1 TaskAttempt Transitioned from FAIL_TASK_CLEANUP to FAILED 2018-12-10 00:42:08,863 INFO [Thread-53] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerRequestor: 1 failures on node 192-168-72-73 2018-12-10 00:42:09,067 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Received completed container container_e17_1533196506314_4460157_01_000007 2018-12-10 00:42:09,067 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: After Scheduling: PendingReds:0 ScheduledMaps:0 ScheduledReds:0 AssignedMaps:3 AssignedReds:0 CompletedMaps:2 CompletedReds:0 ContAlloc:6 ContRel:0 HostLocal:0 RackLocal:0 2018-12-10 00:42:09,067 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: Diagnostics report from attempt_1533196506314_4460157_m_000001_1: Container killed by the ApplicationMaster. Container killed on request. Exit code is 143 Container exited with a non-zero exit code 143 2018-12-10 00:42:10,082 INFO [IPC Server handler 27 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000000_1 is : 0.0 2018-12-10 00:42:10,273 INFO [IPC Server handler 8 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000001_0 is : 0.0 2018-12-10 00:42:10,569 INFO [IPC Server handler 10 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000000_0 is : 0.0 2018-12-10 00:42:10,765 INFO [DefaultSpeculator background processing] org.apache.hadoop.mapreduce.v2.app.speculate.DefaultSpeculator: DefaultSpeculator.addSpeculativeAttempt -- we are speculating task_1533196506314_4460157_m_000001 2018-12-10 00:42:10,765 INFO [DefaultSpeculator background processing] org.apache.hadoop.mapreduce.v2.app.speculate.DefaultSpeculator: We launched 1 speculations. Sleeping 15000 milliseconds. 2018-12-10 00:42:10,765 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: Scheduling a redundant attempt for task task_1533196506314_4460157_m_000001 2018-12-10 00:42:10,766 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1533196506314_4460157_m_000001_2 TaskAttempt Transitioned from NEW to UNASSIGNED 2018-12-10 00:42:10,766 INFO [Thread-53] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Added attempt_1533196506314_4460157_m_000001_2 to list of failed maps 2018-12-10 00:42:11,068 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Before Scheduling: PendingReds:0 ScheduledMaps:1 ScheduledReds:0 AssignedMaps:3 AssignedReds:0 CompletedMaps:2 CompletedReds:0 ContAlloc:6 ContRel:0 HostLocal:0 RackLocal:0 2018-12-10 00:42:11,069 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerRequestor: getResources() for application_1533196506314_4460157: ask=1 release= 0 newContainers=0 finishedContainers=0 resourcelimit= knownNMs=39 2018-12-10 00:42:12,070 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Got allocated containers 1 2018-12-10 00:42:12,071 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Assigning container Container: [ContainerId: container_e17_1533196506314_4460157_01_000008, NodeId: 192-168-72-94:54353, NodeHttpAddress: 192-168-72-94:8042, Resource: , Priority: 5, Token: Token { kind: ContainerToken, service: 192.168.72.94:54353 }, ] to fast fail map 2018-12-10 00:42:12,071 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Assigned from earlierFailedMaps 2018-12-10 00:42:12,071 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Assigned container container_e17_1533196506314_4460157_01_000008 to attempt_1533196506314_4460157_m_000001_2 2018-12-10 00:42:12,071 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: After Scheduling: PendingReds:0 ScheduledMaps:0 ScheduledReds:0 AssignedMaps:4 AssignedReds:0 CompletedMaps:2 CompletedReds:0 ContAlloc:7 ContRel:0 HostLocal:0 RackLocal:0 2018-12-10 00:42:12,072 INFO [AsyncDispatcher event handler] org.apache.hadoop.yarn.util.RackResolver: Resolved 192-168-72-94 to /S6700-XG0-XG1-XG2/K09 2018-12-10 00:42:12,072 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1533196506314_4460157_m_000001_2 TaskAttempt Transitioned from UNASSIGNED to ASSIGNED 2018-12-10 00:42:12,073 INFO [ContainerLauncher #9] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Processing the event EventType: CONTAINER_REMOTE_LAUNCH for container container_e17_1533196506314_4460157_01_000008 taskAttempt attempt_1533196506314_4460157_m_000001_2 2018-12-10 00:42:12,073 INFO [ContainerLauncher #9] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Launching attempt_1533196506314_4460157_m_000001_2 2018-12-10 00:42:12,073 INFO [ContainerLauncher #9] org.apache.hadoop.yarn.client.api.impl.ContainerManagementProtocolProxy: Opening proxy : 192-168-72-94:54353 2018-12-10 00:42:12,080 INFO [ContainerLauncher #9] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Shuffle port returned by ContainerManager for attempt_1533196506314_4460157_m_000001_2 : 13562 2018-12-10 00:42:12,080 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: TaskAttempt: [attempt_1533196506314_4460157_m_000001_2] using containerId: [container_e17_1533196506314_4460157_01_000008 on NM: [192-168-72-94:54353] 2018-12-10 00:42:12,080 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1533196506314_4460157_m_000001_2 TaskAttempt Transitioned from ASSIGNED to RUNNING 2018-12-10 00:42:12,080 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.speculate.DefaultSpeculator: ATTEMPT_START task_1533196506314_4460157_m_000001 2018-12-10 00:42:13,072 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerRequestor: getResources() for application_1533196506314_4460157: ask=1 release= 0 newContainers=0 finishedContainers=0 resourcelimit= knownNMs=39 2018-12-10 00:42:13,111 INFO [IPC Server handler 18 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000000_1 is : 0.0 2018-12-10 00:42:13,323 INFO [IPC Server handler 14 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000001_0 is : 0.0 2018-12-10 00:42:13,608 INFO [IPC Server handler 2 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000000_0 is : 0.0 2018-12-10 00:42:13,663 INFO [Socket Reader #1 for port 8046] SecurityLogger.org.apache.hadoop.ipc.Server: Auth successful for job_1533196506314_4460157 (auth:SIMPLE) 2018-12-10 00:42:13,674 INFO [IPC Server handler 22 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: JVM with ID : jvm_1533196506314_4460157_m_18691697672200 asked for a task 2018-12-10 00:42:13,674 INFO [IPC Server handler 22 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: JVM with ID: jvm_1533196506314_4460157_m_18691697672200 given task: attempt_1533196506314_4460157_m_000001_2 2018-12-10 00:42:16,150 INFO [IPC Server handler 4 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000000_1 is : 0.0 2018-12-10 00:42:16,356 INFO [IPC Server handler 19 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000001_0 is : 0.0 2018-12-10 00:42:16,645 INFO [IPC Server handler 22 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000000_0 is : 0.0 2018-12-10 00:42:19,199 INFO [IPC Server handler 12 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000000_1 is : 0.0 2018-12-10 00:42:19,397 INFO [IPC Server handler 11 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000001_0 is : 0.0 2018-12-10 00:42:19,674 INFO [IPC Server handler 13 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000000_0 is : 0.0 2018-12-10 00:42:20,565 INFO [IPC Server handler 10 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000001_2 is : 0.0 2018-12-10 00:42:22,239 INFO [IPC Server handler 5 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000000_1 is : 0.0 2018-12-10 00:42:22,425 INFO [IPC Server handler 10 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000001_0 is : 0.0 2018-12-10 00:42:22,713 INFO [IPC Server handler 17 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000000_0 is : 0.0 2018-12-10 00:42:23,622 INFO [IPC Server handler 22 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000001_2 is : 0.0 2018-12-10 00:42:25,267 INFO [IPC Server handler 8 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000000_1 is : 0.0 2018-12-10 00:42:25,459 INFO [IPC Server handler 2 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000001_0 is : 0.0 2018-12-10 00:42:25,749 INFO [IPC Server handler 0 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000000_0 is : 0.0 2018-12-10 00:42:26,661 INFO [IPC Server handler 13 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000001_2 is : 0.0 2018-12-10 00:42:28,305 INFO [IPC Server handler 14 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000000_1 is : 0.0 2018-12-10 00:42:28,487 INFO [IPC Server handler 22 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000001_0 is : 0.0 2018-12-10 00:42:28,773 INFO [IPC Server handler 26 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000000_0 is : 0.0 2018-12-10 00:42:29,702 INFO [IPC Server handler 17 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000001_2 is : 0.0 2018-12-10 00:42:30,025 INFO [IPC Server handler 20 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000001_0 is : 0.0 2018-12-10 00:42:30,588 INFO [IPC Server handler 13 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000001_0 is : 0.0 2018-12-10 00:42:30,595 FATAL [IPC Server handler 17 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Task: attempt_1533196506314_4460157_m_000001_0 - exited : org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.hdfs.server.namenode.LeaseExpiredException): No lease on /user/hive/warehouse/ods.db/bss_customer_fj/_SCRATCH0.3130352759450352/dt=20181209/_temporary/1/_temporary/attempt_1533196506314_4460157_m_000001_0/part-m-00001 (inode 761544109): File does not exist. Holder DFSClient_attempt_1533196506314_4460157_m_000001_0_-1729942809_1 does not have any open files. at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkLease(FSNamesystem.java:3432) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.completeFileInternal(FSNamesystem.java:3520) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.completeFile(FSNamesystem.java:3487) at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.complete(NameNodeRpcServer.java:787) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.complete(ClientNamenodeProtocolServerSideTranslatorPB.java:537) at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java) at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:616) at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:982) at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2049) at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2045) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1698) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2045) at org.apache.hadoop.ipc.Client.call(Client.java:1475) at org.apache.hadoop.ipc.Client.call(Client.java:1412) at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:229) at com.sun.proxy.$Proxy14.complete(Unknown Source) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.complete(ClientNamenodeProtocolTranslatorPB.java:462) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:191) at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102) at com.sun.proxy.$Proxy15.complete(Unknown Source) at org.apache.hadoop.hdfs.DFSOutputStream.completeFile(DFSOutputStream.java:2291) at org.apache.hadoop.hdfs.DFSOutputStream.closeImpl(DFSOutputStream.java:2267) at org.apache.hadoop.hdfs.DFSOutputStream.close(DFSOutputStream.java:2232) at org.apache.hadoop.fs.FSDataOutputStream$PositionCache.close(FSDataOutputStream.java:72) at org.apache.hadoop.fs.FSDataOutputStream.close(FSDataOutputStream.java:106) at org.apache.orc.impl.WriterImpl.close(WriterImpl.java:2791) at org.apache.hadoop.hive.ql.io.orc.WriterImpl.close(WriterImpl.java:313) at org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat$OrcRecordWriter.close(OrcOutputFormat.java:120) at org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat$OrcRecordWriter.close(OrcOutputFormat.java:108) at org.apache.hive.hcatalog.mapreduce.StaticPartitionFileRecordWriterContainer.close(StaticPartitionFileRecordWriterContainer.java:53) at org.apache.hadoop.mapred.MapTask$NewDirectOutputCollector.close(MapTask.java:670) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:793) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341) at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1698) at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158) 2018-12-10 00:42:30,596 INFO [IPC Server handler 17 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Diagnostics report from attempt_1533196506314_4460157_m_000001_0: Error: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.hdfs.server.namenode.LeaseExpiredException): No lease on /user/hive/warehouse/ods.db/bss_customer_fj/_SCRATCH0.3130352759450352/dt=20181209/_temporary/1/_temporary/attempt_1533196506314_4460157_m_000001_0/part-m-00001 (inode 761544109): File does not exist. Holder DFSClient_attempt_1533196506314_4460157_m_000001_0_-1729942809_1 does not have any open files. at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkLease(FSNamesystem.java:3432) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.completeFileInternal(FSNamesystem.java:3520) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.completeFile(FSNamesystem.java:3487) at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.complete(NameNodeRpcServer.java:787) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.complete(ClientNamenodeProtocolServerSideTranslatorPB.java:537) at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java) at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:616) at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:982) at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2049) at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2045) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1698) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2045) at org.apache.hadoop.ipc.Client.call(Client.java:1475) at org.apache.hadoop.ipc.Client.call(Client.java:1412) at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:229) at com.sun.proxy.$Proxy14.complete(Unknown Source) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.complete(ClientNamenodeProtocolTranslatorPB.java:462) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:191) at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102) at com.sun.proxy.$Proxy15.complete(Unknown Source) at org.apache.hadoop.hdfs.DFSOutputStream.completeFile(DFSOutputStream.java:2291) at org.apache.hadoop.hdfs.DFSOutputStream.closeImpl(DFSOutputStream.java:2267) at org.apache.hadoop.hdfs.DFSOutputStream.close(DFSOutputStream.java:2232) at org.apache.hadoop.fs.FSDataOutputStream$PositionCache.close(FSDataOutputStream.java:72) at org.apache.hadoop.fs.FSDataOutputStream.close(FSDataOutputStream.java:106) at org.apache.orc.impl.WriterImpl.close(WriterImpl.java:2791) at org.apache.hadoop.hive.ql.io.orc.WriterImpl.close(WriterImpl.java:313) at org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat$OrcRecordWriter.close(OrcOutputFormat.java:120) at org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat$OrcRecordWriter.close(OrcOutputFormat.java:108) at org.apache.hive.hcatalog.mapreduce.StaticPartitionFileRecordWriterContainer.close(StaticPartitionFileRecordWriterContainer.java:53) at org.apache.hadoop.mapred.MapTask$NewDirectOutputCollector.close(MapTask.java:670) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:793) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341) at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1698) at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158) 2018-12-10 00:42:30,596 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: Diagnostics report from attempt_1533196506314_4460157_m_000001_0: Error: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.hdfs.server.namenode.LeaseExpiredException): No lease on /user/hive/warehouse/ods.db/bss_customer_fj/_SCRATCH0.3130352759450352/dt=20181209/_temporary/1/_temporary/attempt_1533196506314_4460157_m_000001_0/part-m-00001 (inode 761544109): File does not exist. Holder DFSClient_attempt_1533196506314_4460157_m_000001_0_-1729942809_1 does not have any open files. at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkLease(FSNamesystem.java:3432) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.completeFileInternal(FSNamesystem.java:3520) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.completeFile(FSNamesystem.java:3487) at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.complete(NameNodeRpcServer.java:787) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.complete(ClientNamenodeProtocolServerSideTranslatorPB.java:537) at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java) at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:616) at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:982) at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2049) at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2045) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1698) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2045) at org.apache.hadoop.ipc.Client.call(Client.java:1475) at org.apache.hadoop.ipc.Client.call(Client.java:1412) at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:229) at com.sun.proxy.$Proxy14.complete(Unknown Source) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.complete(ClientNamenodeProtocolTranslatorPB.java:462) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:191) at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102) at com.sun.proxy.$Proxy15.complete(Unknown Source) at org.apache.hadoop.hdfs.DFSOutputStream.completeFile(DFSOutputStream.java:2291) at org.apache.hadoop.hdfs.DFSOutputStream.closeImpl(DFSOutputStream.java:2267) at org.apache.hadoop.hdfs.DFSOutputStream.close(DFSOutputStream.java:2232) at org.apache.hadoop.fs.FSDataOutputStream$PositionCache.close(FSDataOutputStream.java:72) at org.apache.hadoop.fs.FSDataOutputStream.close(FSDataOutputStream.java:106) at org.apache.orc.impl.WriterImpl.close(WriterImpl.java:2791) at org.apache.hadoop.hive.ql.io.orc.WriterImpl.close(WriterImpl.java:313) at org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat$OrcRecordWriter.close(OrcOutputFormat.java:120) at org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat$OrcRecordWriter.close(OrcOutputFormat.java:108) at org.apache.hive.hcatalog.mapreduce.StaticPartitionFileRecordWriterContainer.close(StaticPartitionFileRecordWriterContainer.java:53) at org.apache.hadoop.mapred.MapTask$NewDirectOutputCollector.close(MapTask.java:670) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:793) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341) at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1698) at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158) 2018-12-10 00:42:30,597 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1533196506314_4460157_m_000001_0 TaskAttempt Transitioned from RUNNING to FAIL_CONTAINER_CLEANUP 2018-12-10 00:42:30,597 INFO [ContainerLauncher #3] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Processing the event EventType: CONTAINER_REMOTE_CLEANUP for container container_e17_1533196506314_4460157_01_000003 taskAttempt attempt_1533196506314_4460157_m_000001_0 2018-12-10 00:42:30,597 INFO [ContainerLauncher #3] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: KILLING attempt_1533196506314_4460157_m_000001_0 2018-12-10 00:42:30,597 INFO [ContainerLauncher #3] org.apache.hadoop.yarn.client.api.impl.ContainerManagementProtocolProxy: Opening proxy : 192-168-72-94:54353 2018-12-10 00:42:30,603 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1533196506314_4460157_m_000001_0 TaskAttempt Transitioned from FAIL_CONTAINER_CLEANUP to FAIL_TASK_CLEANUP 2018-12-10 00:42:30,633 INFO [CommitterEvent Processor #2] org.apache.hadoop.mapreduce.v2.app.commit.CommitterEventHandler: Processing the event EventType: TASK_ABORT 2018-12-10 00:42:30,641 WARN [CommitterEvent Processor #2] org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter: Could not delete hdfs://ycluster/user/hive/warehouse/ods.db/bss_customer_fj/_SCRATCH0.3130352759450352/dt=20181209/_temporary/1/_temporary/attempt_1533196506314_4460157_m_000000_0 2018-12-10 00:42:30,642 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1533196506314_4460157_m_000001_0 TaskAttempt Transitioned from FAIL_TASK_CLEANUP to FAILED 2018-12-10 00:42:30,644 INFO [Thread-53] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerRequestor: 1 failures on node 192-168-72-94 2018-12-10 00:42:30,767 INFO [DefaultSpeculator background processing] org.apache.hadoop.mapreduce.v2.app.speculate.DefaultSpeculator: DefaultSpeculator.addSpeculativeAttempt -- we are speculating task_1533196506314_4460157_m_000001 2018-12-10 00:42:30,767 INFO [DefaultSpeculator background processing] org.apache.hadoop.mapreduce.v2.app.speculate.DefaultSpeculator: We launched 1 speculations. Sleeping 15000 milliseconds. 2018-12-10 00:42:30,768 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: Scheduling a redundant attempt for task task_1533196506314_4460157_m_000001 2018-12-10 00:42:30,768 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1533196506314_4460157_m_000001_3 TaskAttempt Transitioned from NEW to UNASSIGNED 2018-12-10 00:42:30,768 INFO [Thread-53] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Added attempt_1533196506314_4460157_m_000001_3 to list of failed maps 2018-12-10 00:42:31,089 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Before Scheduling: PendingReds:0 ScheduledMaps:1 ScheduledReds:0 AssignedMaps:4 AssignedReds:0 CompletedMaps:2 CompletedReds:0 ContAlloc:7 ContRel:0 HostLocal:0 RackLocal:0 2018-12-10 00:42:31,092 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerRequestor: getResources() for application_1533196506314_4460157: ask=1 release= 0 newContainers=0 finishedContainers=1 resourcelimit= knownNMs=39 2018-12-10 00:42:31,092 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Received completed container container_e17_1533196506314_4460157_01_000003 2018-12-10 00:42:31,092 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: After Scheduling: PendingReds:0 ScheduledMaps:1 ScheduledReds:0 AssignedMaps:3 AssignedReds:0 CompletedMaps:2 CompletedReds:0 ContAlloc:7 ContRel:0 HostLocal:0 RackLocal:0 2018-12-10 00:42:31,092 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: Diagnostics report from attempt_1533196506314_4460157_m_000001_0: Container killed by the ApplicationMaster. Container killed on request. Exit code is 143 Container exited with a non-zero exit code 143 2018-12-10 00:42:31,344 INFO [IPC Server handler 19 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000000_1 is : 0.0 2018-12-10 00:42:31,800 INFO [IPC Server handler 21 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000000_0 is : 0.0 2018-12-10 00:42:32,093 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Got allocated containers 1 2018-12-10 00:42:32,093 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Assigning container Container: [ContainerId: container_e17_1533196506314_4460157_01_000009, NodeId: 192-168-72-84:13498, NodeHttpAddress: 192-168-72-84:8042, Resource: , Priority: 5, Token: Token { kind: ContainerToken, service: 192.168.72.84:13498 }, ] to fast fail map 2018-12-10 00:42:32,093 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Assigned from earlierFailedMaps 2018-12-10 00:42:32,094 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Assigned container container_e17_1533196506314_4460157_01_000009 to attempt_1533196506314_4460157_m_000001_3 2018-12-10 00:42:32,094 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: After Scheduling: PendingReds:0 ScheduledMaps:0 ScheduledReds:0 AssignedMaps:4 AssignedReds:0 CompletedMaps:2 CompletedReds:0 ContAlloc:8 ContRel:0 HostLocal:0 RackLocal:0 2018-12-10 00:42:32,100 INFO [AsyncDispatcher event handler] org.apache.hadoop.yarn.util.RackResolver: Resolved 192-168-72-84 to /S6700-XG0-XG1-XG2/K10 2018-12-10 00:42:32,101 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1533196506314_4460157_m_000001_3 TaskAttempt Transitioned from UNASSIGNED to ASSIGNED 2018-12-10 00:42:32,101 INFO [ContainerLauncher #0] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Processing the event EventType: CONTAINER_REMOTE_LAUNCH for container container_e17_1533196506314_4460157_01_000009 taskAttempt attempt_1533196506314_4460157_m_000001_3 2018-12-10 00:42:32,101 INFO [ContainerLauncher #0] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Launching attempt_1533196506314_4460157_m_000001_3 2018-12-10 00:42:32,101 INFO [ContainerLauncher #0] org.apache.hadoop.yarn.client.api.impl.ContainerManagementProtocolProxy: Opening proxy : 192-168-72-84:13498 2018-12-10 00:42:32,113 INFO [ContainerLauncher #0] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Shuffle port returned by ContainerManager for attempt_1533196506314_4460157_m_000001_3 : 13562 2018-12-10 00:42:32,113 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: TaskAttempt: [attempt_1533196506314_4460157_m_000001_3] using containerId: [container_e17_1533196506314_4460157_01_000009 on NM: [192-168-72-84:13498] 2018-12-10 00:42:32,113 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1533196506314_4460157_m_000001_3 TaskAttempt Transitioned from ASSIGNED to RUNNING 2018-12-10 00:42:32,114 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.speculate.DefaultSpeculator: ATTEMPT_START task_1533196506314_4460157_m_000001 2018-12-10 00:42:32,758 INFO [IPC Server handler 26 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000001_2 is : 0.0 2018-12-10 00:42:33,095 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerRequestor: getResources() for application_1533196506314_4460157: ask=1 release= 0 newContainers=0 finishedContainers=0 resourcelimit= knownNMs=39 2018-12-10 00:42:33,829 INFO [IPC Server handler 25 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000000_0 is : 0.0 2018-12-10 00:42:34,385 INFO [IPC Server handler 11 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000000_1 is : 0.0 2018-12-10 00:42:34,414 INFO [IPC Server handler 10 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000000_0 is : 0.0 2018-12-10 00:42:34,424 FATAL [IPC Server handler 2 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Task: attempt_1533196506314_4460157_m_000000_0 - exited : org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.hdfs.server.namenode.LeaseExpiredException): No lease on /user/hive/warehouse/ods.db/bss_customer_fj/_SCRATCH0.3130352759450352/dt=20181209/_temporary/1/_temporary/attempt_1533196506314_4460157_m_000000_0/part-m-00000 (inode 761544157): File does not exist. Holder DFSClient_attempt_1533196506314_4460157_m_000000_0_798513081_1 does not have any open files. at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkLease(FSNamesystem.java:3432) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.completeFileInternal(FSNamesystem.java:3520) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.completeFile(FSNamesystem.java:3487) at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.complete(NameNodeRpcServer.java:787) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.complete(ClientNamenodeProtocolServerSideTranslatorPB.java:537) at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java) at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:616) at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:982) at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2049) at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2045) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1698) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2045) at org.apache.hadoop.ipc.Client.call(Client.java:1475) at org.apache.hadoop.ipc.Client.call(Client.java:1412) at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:229) at com.sun.proxy.$Proxy14.complete(Unknown Source) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.complete(ClientNamenodeProtocolTranslatorPB.java:462) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:191) at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102) at com.sun.proxy.$Proxy15.complete(Unknown Source) at org.apache.hadoop.hdfs.DFSOutputStream.completeFile(DFSOutputStream.java:2291) at org.apache.hadoop.hdfs.DFSOutputStream.closeImpl(DFSOutputStream.java:2267) at org.apache.hadoop.hdfs.DFSOutputStream.close(DFSOutputStream.java:2232) at org.apache.hadoop.fs.FSDataOutputStream$PositionCache.close(FSDataOutputStream.java:72) at org.apache.hadoop.fs.FSDataOutputStream.close(FSDataOutputStream.java:106) at org.apache.orc.impl.WriterImpl.close(WriterImpl.java:2791) at org.apache.hadoop.hive.ql.io.orc.WriterImpl.close(WriterImpl.java:313) at org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat$OrcRecordWriter.close(OrcOutputFormat.java:120) at org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat$OrcRecordWriter.close(OrcOutputFormat.java:108) at org.apache.hive.hcatalog.mapreduce.StaticPartitionFileRecordWriterContainer.close(StaticPartitionFileRecordWriterContainer.java:53) at org.apache.hadoop.mapred.MapTask$NewDirectOutputCollector.close(MapTask.java:670) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:793) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341) at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1698) at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158) 2018-12-10 00:42:34,424 INFO [IPC Server handler 2 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Diagnostics report from attempt_1533196506314_4460157_m_000000_0: Error: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.hdfs.server.namenode.LeaseExpiredException): No lease on /user/hive/warehouse/ods.db/bss_customer_fj/_SCRATCH0.3130352759450352/dt=20181209/_temporary/1/_temporary/attempt_1533196506314_4460157_m_000000_0/part-m-00000 (inode 761544157): File does not exist. Holder DFSClient_attempt_1533196506314_4460157_m_000000_0_798513081_1 does not have any open files. at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkLease(FSNamesystem.java:3432) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.completeFileInternal(FSNamesystem.java:3520) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.completeFile(FSNamesystem.java:3487) at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.complete(NameNodeRpcServer.java:787) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.complete(ClientNamenodeProtocolServerSideTranslatorPB.java:537) at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java) at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:616) at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:982) at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2049) at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2045) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1698) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2045) at org.apache.hadoop.ipc.Client.call(Client.java:1475) at org.apache.hadoop.ipc.Client.call(Client.java:1412) at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:229) at com.sun.proxy.$Proxy14.complete(Unknown Source) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.complete(ClientNamenodeProtocolTranslatorPB.java:462) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:191) at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102) at com.sun.proxy.$Proxy15.complete(Unknown Source) at org.apache.hadoop.hdfs.DFSOutputStream.completeFile(DFSOutputStream.java:2291) at org.apache.hadoop.hdfs.DFSOutputStream.closeImpl(DFSOutputStream.java:2267) at org.apache.hadoop.hdfs.DFSOutputStream.close(DFSOutputStream.java:2232) at org.apache.hadoop.fs.FSDataOutputStream$PositionCache.close(FSDataOutputStream.java:72) at org.apache.hadoop.fs.FSDataOutputStream.close(FSDataOutputStream.java:106) at org.apache.orc.impl.WriterImpl.close(WriterImpl.java:2791) at org.apache.hadoop.hive.ql.io.orc.WriterImpl.close(WriterImpl.java:313) at org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat$OrcRecordWriter.close(OrcOutputFormat.java:120) at org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat$OrcRecordWriter.close(OrcOutputFormat.java:108) at org.apache.hive.hcatalog.mapreduce.StaticPartitionFileRecordWriterContainer.close(StaticPartitionFileRecordWriterContainer.java:53) at org.apache.hadoop.mapred.MapTask$NewDirectOutputCollector.close(MapTask.java:670) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:793) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341) at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1698) at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158) 2018-12-10 00:42:34,425 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: Diagnostics report from attempt_1533196506314_4460157_m_000000_0: Error: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.hdfs.server.namenode.LeaseExpiredException): No lease on /user/hive/warehouse/ods.db/bss_customer_fj/_SCRATCH0.3130352759450352/dt=20181209/_temporary/1/_temporary/attempt_1533196506314_4460157_m_000000_0/part-m-00000 (inode 761544157): File does not exist. Holder DFSClient_attempt_1533196506314_4460157_m_000000_0_798513081_1 does not have any open files. at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkLease(FSNamesystem.java:3432) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.completeFileInternal(FSNamesystem.java:3520) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.completeFile(FSNamesystem.java:3487) at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.complete(NameNodeRpcServer.java:787) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.complete(ClientNamenodeProtocolServerSideTranslatorPB.java:537) at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java) at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:616) at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:982) at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2049) at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2045) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1698) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2045) at org.apache.hadoop.ipc.Client.call(Client.java:1475) at org.apache.hadoop.ipc.Client.call(Client.java:1412) at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:229) at com.sun.proxy.$Proxy14.complete(Unknown Source) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.complete(ClientNamenodeProtocolTranslatorPB.java:462) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:191) at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102) at com.sun.proxy.$Proxy15.complete(Unknown Source) at org.apache.hadoop.hdfs.DFSOutputStream.completeFile(DFSOutputStream.java:2291) at org.apache.hadoop.hdfs.DFSOutputStream.closeImpl(DFSOutputStream.java:2267) at org.apache.hadoop.hdfs.DFSOutputStream.close(DFSOutputStream.java:2232) at org.apache.hadoop.fs.FSDataOutputStream$PositionCache.close(FSDataOutputStream.java:72) at org.apache.hadoop.fs.FSDataOutputStream.close(FSDataOutputStream.java:106) at org.apache.orc.impl.WriterImpl.close(WriterImpl.java:2791) at org.apache.hadoop.hive.ql.io.orc.WriterImpl.close(WriterImpl.java:313) at org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat$OrcRecordWriter.close(OrcOutputFormat.java:120) at org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat$OrcRecordWriter.close(OrcOutputFormat.java:108) at org.apache.hive.hcatalog.mapreduce.StaticPartitionFileRecordWriterContainer.close(StaticPartitionFileRecordWriterContainer.java:53) at org.apache.hadoop.mapred.MapTask$NewDirectOutputCollector.close(MapTask.java:670) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:793) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341) at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1698) at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158) 2018-12-10 00:42:34,425 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1533196506314_4460157_m_000000_0 TaskAttempt Transitioned from RUNNING to FAIL_CONTAINER_CLEANUP 2018-12-10 00:42:34,425 INFO [ContainerLauncher #2] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Processing the event EventType: CONTAINER_REMOTE_CLEANUP for container container_e17_1533196506314_4460157_01_000002 taskAttempt attempt_1533196506314_4460157_m_000000_0 2018-12-10 00:42:34,426 INFO [ContainerLauncher #2] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: KILLING attempt_1533196506314_4460157_m_000000_0 2018-12-10 00:42:34,426 INFO [ContainerLauncher #2] org.apache.hadoop.yarn.client.api.impl.ContainerManagementProtocolProxy: Opening proxy : 192-168-72-24:16310 2018-12-10 00:42:34,434 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1533196506314_4460157_m_000000_0 TaskAttempt Transitioned from FAIL_CONTAINER_CLEANUP to FAIL_TASK_CLEANUP 2018-12-10 00:42:34,434 INFO [CommitterEvent Processor #3] org.apache.hadoop.mapreduce.v2.app.commit.CommitterEventHandler: Processing the event EventType: TASK_ABORT 2018-12-10 00:42:34,436 WARN [CommitterEvent Processor #3] org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter: Could not delete hdfs://ycluster/user/hive/warehouse/ods.db/bss_customer_fj/_SCRATCH0.3130352759450352/dt=20181209/_temporary/1/_temporary/attempt_1533196506314_4460157_m_000000_0 2018-12-10 00:42:34,437 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1533196506314_4460157_m_000000_0 TaskAttempt Transitioned from FAIL_TASK_CLEANUP to FAILED 2018-12-10 00:42:34,437 INFO [Thread-53] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerRequestor: 1 failures on node 192-168-72-24 2018-12-10 00:42:35,097 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Received completed container container_e17_1533196506314_4460157_01_000002 2018-12-10 00:42:35,097 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: After Scheduling: PendingReds:0 ScheduledMaps:0 ScheduledReds:0 AssignedMaps:3 AssignedReds:0 CompletedMaps:2 CompletedReds:0 ContAlloc:8 ContRel:0 HostLocal:0 RackLocal:0 2018-12-10 00:42:35,097 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: Diagnostics report from attempt_1533196506314_4460157_m_000000_0: Container killed by the ApplicationMaster. Container killed on request. Exit code is 143 Container exited with a non-zero exit code 143 2018-12-10 00:42:35,796 INFO [IPC Server handler 21 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000001_2 is : 0.0 2018-12-10 00:42:37,414 INFO [IPC Server handler 10 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000000_1 is : 0.0 2018-12-10 00:42:38,828 INFO [IPC Server handler 25 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000001_2 is : 0.0 2018-12-10 00:42:39,223 INFO [Socket Reader #1 for port 8046] SecurityLogger.org.apache.hadoop.ipc.Server: Auth successful for job_1533196506314_4460157 (auth:SIMPLE) 2018-12-10 00:42:39,239 INFO [IPC Server handler 5 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: JVM with ID : jvm_1533196506314_4460157_m_18691697672201 asked for a task 2018-12-10 00:42:39,239 INFO [IPC Server handler 5 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: JVM with ID: jvm_1533196506314_4460157_m_18691697672201 given task: attempt_1533196506314_4460157_m_000001_3 2018-12-10 00:42:40,455 INFO [IPC Server handler 22 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000000_1 is : 0.0 2018-12-10 00:42:41,860 INFO [IPC Server handler 20 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000001_2 is : 0.0 2018-12-10 00:42:43,495 INFO [IPC Server handler 13 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000000_1 is : 0.0 2018-12-10 00:42:44,893 INFO [IPC Server handler 9 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000001_2 is : 0.0 2018-12-10 00:42:45,768 INFO [DefaultSpeculator background processing] org.apache.hadoop.mapreduce.v2.app.speculate.DefaultSpeculator: DefaultSpeculator.addSpeculativeAttempt -- we are speculating task_1533196506314_4460157_m_000000 2018-12-10 00:42:45,768 INFO [DefaultSpeculator background processing] org.apache.hadoop.mapreduce.v2.app.speculate.DefaultSpeculator: We launched 1 speculations. Sleeping 15000 milliseconds. 2018-12-10 00:42:45,768 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: Scheduling a redundant attempt for task task_1533196506314_4460157_m_000000 2018-12-10 00:42:45,768 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1533196506314_4460157_m_000000_2 TaskAttempt Transitioned from NEW to UNASSIGNED 2018-12-10 00:42:45,768 INFO [Thread-53] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Added attempt_1533196506314_4460157_m_000000_2 to list of failed maps 2018-12-10 00:42:46,111 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Before Scheduling: PendingReds:0 ScheduledMaps:1 ScheduledReds:0 AssignedMaps:3 AssignedReds:0 CompletedMaps:2 CompletedReds:0 ContAlloc:8 ContRel:0 HostLocal:0 RackLocal:0 2018-12-10 00:42:46,112 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerRequestor: getResources() for application_1533196506314_4460157: ask=1 release= 0 newContainers=0 finishedContainers=0 resourcelimit= knownNMs=39 2018-12-10 00:42:46,245 INFO [IPC Server handler 8 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000001_3 is : 0.0 2018-12-10 00:42:46,524 INFO [IPC Server handler 17 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000000_1 is : 0.0 2018-12-10 00:42:47,114 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Got allocated containers 1 2018-12-10 00:42:47,114 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Assigning container Container: [ContainerId: container_e17_1533196506314_4460157_01_000010, NodeId: 192-168-72-88:24481, NodeHttpAddress: 192-168-72-88:8042, Resource: , Priority: 5, Token: Token { kind: ContainerToken, service: 192.168.72.88:24481 }, ] to fast fail map 2018-12-10 00:42:47,114 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Assigned from earlierFailedMaps 2018-12-10 00:42:47,115 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Assigned container container_e17_1533196506314_4460157_01_000010 to attempt_1533196506314_4460157_m_000000_2 2018-12-10 00:42:47,115 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: After Scheduling: PendingReds:0 ScheduledMaps:0 ScheduledReds:0 AssignedMaps:4 AssignedReds:0 CompletedMaps:2 CompletedReds:0 ContAlloc:9 ContRel:0 HostLocal:0 RackLocal:0 2018-12-10 00:42:47,122 INFO [AsyncDispatcher event handler] org.apache.hadoop.yarn.util.RackResolver: Resolved 192-168-72-88 to /S6700-XG0-XG1-XG2/K09 2018-12-10 00:42:47,123 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1533196506314_4460157_m_000000_2 TaskAttempt Transitioned from UNASSIGNED to ASSIGNED 2018-12-10 00:42:47,123 INFO [ContainerLauncher #1] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Processing the event EventType: CONTAINER_REMOTE_LAUNCH for container container_e17_1533196506314_4460157_01_000010 taskAttempt attempt_1533196506314_4460157_m_000000_2 2018-12-10 00:42:47,123 INFO [ContainerLauncher #1] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Launching attempt_1533196506314_4460157_m_000000_2 2018-12-10 00:42:47,123 INFO [ContainerLauncher #1] org.apache.hadoop.yarn.client.api.impl.ContainerManagementProtocolProxy: Opening proxy : 192-168-72-88:24481 2018-12-10 00:42:47,134 INFO [ContainerLauncher #1] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Shuffle port returned by ContainerManager for attempt_1533196506314_4460157_m_000000_2 : 13562 2018-12-10 00:42:47,135 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: TaskAttempt: [attempt_1533196506314_4460157_m_000000_2] using containerId: [container_e17_1533196506314_4460157_01_000010 on NM: [192-168-72-88:24481] 2018-12-10 00:42:47,136 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1533196506314_4460157_m_000000_2 TaskAttempt Transitioned from ASSIGNED to RUNNING 2018-12-10 00:42:47,136 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.speculate.DefaultSpeculator: ATTEMPT_START task_1533196506314_4460157_m_000000 2018-12-10 00:42:47,923 INFO [IPC Server handler 29 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000001_2 is : 0.0 2018-12-10 00:42:48,116 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerRequestor: getResources() for application_1533196506314_4460157: ask=1 release= 0 newContainers=0 finishedContainers=0 resourcelimit= knownNMs=39 2018-12-10 00:42:49,282 INFO [IPC Server handler 14 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000001_3 is : 0.0 2018-12-10 00:42:49,557 INFO [IPC Server handler 0 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000000_1 is : 0.0 2018-12-10 00:42:50,957 INFO [IPC Server handler 24 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000001_2 is : 0.0 2018-12-10 00:42:52,321 INFO [IPC Server handler 19 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000001_3 is : 0.0 2018-12-10 00:42:52,594 INFO [IPC Server handler 26 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000000_1 is : 0.0 2018-12-10 00:42:53,995 INFO [IPC Server handler 27 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000001_2 is : 0.0 2018-12-10 00:42:55,397 INFO [Socket Reader #1 for port 8046] SecurityLogger.org.apache.hadoop.ipc.Server: Auth successful for job_1533196506314_4460157 (auth:SIMPLE) 2018-12-10 00:42:55,403 INFO [IPC Server handler 10 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000001_3 is : 0.0 2018-12-10 00:42:55,409 INFO [IPC Server handler 2 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: JVM with ID : jvm_1533196506314_4460157_m_18691697672202 asked for a task 2018-12-10 00:42:55,409 INFO [IPC Server handler 2 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: JVM with ID: jvm_1533196506314_4460157_m_18691697672202 given task: attempt_1533196506314_4460157_m_000000_2 2018-12-10 00:42:55,621 INFO [IPC Server handler 21 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000000_1 is : 0.0 2018-12-10 00:42:57,025 INFO [IPC Server handler 28 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000001_2 is : 0.0 2018-12-10 00:42:58,456 INFO [IPC Server handler 22 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000001_3 is : 0.0 2018-12-10 00:42:58,658 INFO [IPC Server handler 25 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000000_1 is : 0.0 2018-12-10 00:43:00,055 INFO [IPC Server handler 23 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000001_2 is : 0.0 2018-12-10 00:43:01,515 INFO [IPC Server handler 17 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000001_3 is : 0.0 2018-12-10 00:43:01,691 INFO [IPC Server handler 20 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000000_1 is : 0.0 2018-12-10 00:43:02,312 INFO [IPC Server handler 19 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000000_2 is : 0.0 2018-12-10 00:43:03,085 INFO [IPC Server handler 18 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000001_2 is : 0.0 2018-12-10 00:43:04,560 INFO [IPC Server handler 26 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000001_3 is : 0.0 2018-12-10 00:43:04,725 INFO [IPC Server handler 9 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000000_1 is : 0.0 2018-12-10 00:43:05,364 INFO [IPC Server handler 11 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000000_2 is : 0.0 2018-12-10 00:43:06,114 INFO [IPC Server handler 16 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000001_2 is : 0.0 2018-12-10 00:43:07,601 INFO [IPC Server handler 21 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000001_3 is : 0.0 2018-12-10 00:43:07,763 INFO [IPC Server handler 29 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000000_1 is : 0.0 2018-12-10 00:43:08,402 INFO [IPC Server handler 10 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000000_2 is : 0.0 2018-12-10 00:43:09,143 INFO [IPC Server handler 7 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000001_2 is : 0.0 2018-12-10 00:43:10,661 INFO [IPC Server handler 20 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000001_3 is : 0.0 2018-12-10 00:43:10,804 INFO [IPC Server handler 24 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000000_1 is : 0.0 2018-12-10 00:43:11,451 INFO [IPC Server handler 22 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000000_2 is : 0.0 2018-12-10 00:43:12,173 INFO [IPC Server handler 15 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000001_2 is : 0.0 2018-12-10 00:43:13,715 INFO [IPC Server handler 9 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000001_3 is : 0.0 2018-12-10 00:43:13,827 INFO [IPC Server handler 27 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000000_1 is : 0.0 2018-12-10 00:43:14,490 INFO [IPC Server handler 13 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000000_2 is : 0.0 2018-12-10 00:43:15,214 INFO [IPC Server handler 5 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000001_2 is : 0.0 2018-12-10 00:43:16,765 INFO [IPC Server handler 24 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000001_3 is : 0.0 2018-12-10 00:43:16,860 INFO [IPC Server handler 28 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000000_1 is : 0.0 2018-12-10 00:43:16,970 INFO [IPC Server handler 23 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000000_1 is : 0.0 2018-12-10 00:43:17,425 INFO [IPC Server handler 22 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Commit-pending state update from attempt_1533196506314_4460157_m_000000_1 2018-12-10 00:43:17,426 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1533196506314_4460157_m_000000_1 TaskAttempt Transitioned from RUNNING to COMMIT_PENDING 2018-12-10 00:43:17,426 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: attempt_1533196506314_4460157_m_000000_1 given a go for committing the task output. 2018-12-10 00:43:17,426 INFO [IPC Server handler 13 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Commit go/no-go request from attempt_1533196506314_4460157_m_000000_1 2018-12-10 00:43:17,427 INFO [IPC Server handler 13 on 8046] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: Result of canCommit for attempt_1533196506314_4460157_m_000000_1:true 2018-12-10 00:43:17,467 INFO [IPC Server handler 17 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000000_1 is : 1.0 2018-12-10 00:43:17,468 INFO [IPC Server handler 0 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Done acknowledgement from attempt_1533196506314_4460157_m_000000_1 2018-12-10 00:43:17,468 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1533196506314_4460157_m_000000_1 TaskAttempt Transitioned from COMMIT_PENDING to SUCCESS_CONTAINER_CLEANUP 2018-12-10 00:43:17,468 INFO [ContainerLauncher #4] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Processing the event EventType: CONTAINER_REMOTE_CLEANUP for container container_e17_1533196506314_4460157_01_000006 taskAttempt attempt_1533196506314_4460157_m_000000_1 2018-12-10 00:43:17,469 INFO [ContainerLauncher #4] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: KILLING attempt_1533196506314_4460157_m_000000_1 2018-12-10 00:43:17,469 INFO [ContainerLauncher #4] org.apache.hadoop.yarn.client.api.impl.ContainerManagementProtocolProxy: Opening proxy : 192-168-72-23:18284 2018-12-10 00:43:17,473 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1533196506314_4460157_m_000000_1 TaskAttempt Transitioned from SUCCESS_CONTAINER_CLEANUP to SUCCEEDED 2018-12-10 00:43:17,473 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: Task succeeded with attempt attempt_1533196506314_4460157_m_000000_1 2018-12-10 00:43:17,473 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: Issuing kill to other attempt attempt_1533196506314_4460157_m_000000_2 2018-12-10 00:43:17,474 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: task_1533196506314_4460157_m_000000 Task Transitioned from RUNNING to SUCCEEDED 2018-12-10 00:43:17,474 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: Num completed Tasks: 3 2018-12-10 00:43:17,474 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1533196506314_4460157_m_000000_2 TaskAttempt Transitioned from RUNNING to KILL_CONTAINER_CLEANUP 2018-12-10 00:43:17,474 INFO [ContainerLauncher #5] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Processing the event EventType: CONTAINER_REMOTE_CLEANUP for container container_e17_1533196506314_4460157_01_000010 taskAttempt attempt_1533196506314_4460157_m_000000_2 2018-12-10 00:43:17,474 INFO [ContainerLauncher #5] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: KILLING attempt_1533196506314_4460157_m_000000_2 2018-12-10 00:43:17,475 INFO [ContainerLauncher #5] org.apache.hadoop.yarn.client.api.impl.ContainerManagementProtocolProxy: Opening proxy : 192-168-72-88:24481 2018-12-10 00:43:17,479 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1533196506314_4460157_m_000000_2 TaskAttempt Transitioned from KILL_CONTAINER_CLEANUP to KILL_TASK_CLEANUP 2018-12-10 00:43:17,480 INFO [CommitterEvent Processor #4] org.apache.hadoop.mapreduce.v2.app.commit.CommitterEventHandler: Processing the event EventType: TASK_ABORT 2018-12-10 00:43:17,482 WARN [CommitterEvent Processor #4] org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter: Could not delete hdfs://ycluster/user/hive/warehouse/ods.db/bss_customer_fj/_SCRATCH0.3130352759450352/dt=20181209/_temporary/1/_temporary/attempt_1533196506314_4460157_m_000000_0 2018-12-10 00:43:17,484 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1533196506314_4460157_m_000000_2 TaskAttempt Transitioned from KILL_TASK_CLEANUP to KILLED 2018-12-10 00:43:18,144 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Before Scheduling: PendingReds:0 ScheduledMaps:0 ScheduledReds:0 AssignedMaps:4 AssignedReds:0 CompletedMaps:3 CompletedReds:0 ContAlloc:9 ContRel:0 HostLocal:0 RackLocal:0 2018-12-10 00:43:18,146 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Received completed container container_e17_1533196506314_4460157_01_000010 2018-12-10 00:43:18,146 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Received completed container container_e17_1533196506314_4460157_01_000006 2018-12-10 00:43:18,147 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: After Scheduling: PendingReds:0 ScheduledMaps:0 ScheduledReds:0 AssignedMaps:2 AssignedReds:0 CompletedMaps:3 CompletedReds:0 ContAlloc:9 ContRel:0 HostLocal:0 RackLocal:0 2018-12-10 00:43:18,147 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: Diagnostics report from attempt_1533196506314_4460157_m_000000_2: Container killed by the ApplicationMaster. Container killed on request. Exit code is 143 Container exited with a non-zero exit code 143 2018-12-10 00:43:18,147 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: Diagnostics report from attempt_1533196506314_4460157_m_000000_1: Container killed by the ApplicationMaster. Container killed on request. Exit code is 143 Container exited with a non-zero exit code 143 2018-12-10 00:43:18,249 INFO [IPC Server handler 14 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000001_2 is : 0.0 2018-12-10 00:43:19,820 INFO [IPC Server handler 27 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000001_3 is : 0.0 2018-12-10 00:43:21,279 INFO [IPC Server handler 19 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000001_2 is : 0.0 2018-12-10 00:43:22,877 INFO [IPC Server handler 23 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000001_3 is : 0.0 2018-12-10 00:43:24,310 INFO [IPC Server handler 11 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000001_2 is : 0.0 2018-12-10 00:43:25,940 INFO [IPC Server handler 18 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000001_3 is : 0.0 2018-12-10 00:43:27,348 INFO [IPC Server handler 10 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000001_2 is : 0.0 2018-12-10 00:43:28,995 INFO [IPC Server handler 16 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000001_3 is : 0.0 2018-12-10 00:43:30,379 INFO [IPC Server handler 2 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000001_2 is : 0.0 2018-12-10 00:43:32,030 INFO [IPC Server handler 1 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000001_3 is : 0.0 2018-12-10 00:43:33,424 INFO [IPC Server handler 22 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000001_2 is : 0.0 2018-12-10 00:43:35,080 INFO [IPC Server handler 7 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000001_3 is : 0.0 2018-12-10 00:43:36,455 INFO [IPC Server handler 0 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000001_2 is : 0.0 2018-12-10 00:43:38,121 INFO [IPC Server handler 4 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000001_3 is : 0.0 2018-12-10 00:43:38,686 INFO [IPC Server handler 9 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000001_2 is : 0.0 2018-12-10 00:43:39,314 INFO [IPC Server handler 10 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Commit-pending state update from attempt_1533196506314_4460157_m_000001_2 2018-12-10 00:43:39,314 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1533196506314_4460157_m_000001_2 TaskAttempt Transitioned from RUNNING to COMMIT_PENDING 2018-12-10 00:43:39,314 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: attempt_1533196506314_4460157_m_000001_2 given a go for committing the task output. 2018-12-10 00:43:39,315 INFO [IPC Server handler 2 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Commit go/no-go request from attempt_1533196506314_4460157_m_000001_2 2018-12-10 00:43:39,316 INFO [IPC Server handler 2 on 8046] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: Result of canCommit for attempt_1533196506314_4460157_m_000001_2:true 2018-12-10 00:43:39,365 INFO [IPC Server handler 22 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1533196506314_4460157_m_000001_2 is : 1.0 2018-12-10 00:43:39,366 INFO [IPC Server handler 13 on 8046] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Done acknowledgement from attempt_1533196506314_4460157_m_000001_2 2018-12-10 00:43:39,366 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1533196506314_4460157_m_000001_2 TaskAttempt Transitioned from COMMIT_PENDING to SUCCESS_CONTAINER_CLEANUP 2018-12-10 00:43:39,367 INFO [ContainerLauncher #6] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Processing the event EventType: CONTAINER_REMOTE_CLEANUP for container container_e17_1533196506314_4460157_01_000008 taskAttempt attempt_1533196506314_4460157_m_000001_2 2018-12-10 00:43:39,367 INFO [ContainerLauncher #6] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: KILLING attempt_1533196506314_4460157_m_000001_2 2018-12-10 00:43:39,367 INFO [ContainerLauncher #6] org.apache.hadoop.yarn.client.api.impl.ContainerManagementProtocolProxy: Opening proxy : 192-168-72-94:54353 2018-12-10 00:43:39,374 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1533196506314_4460157_m_000001_2 TaskAttempt Transitioned from SUCCESS_CONTAINER_CLEANUP to SUCCEEDED 2018-12-10 00:43:39,374 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: Task succeeded with attempt attempt_1533196506314_4460157_m_000001_2 2018-12-10 00:43:39,374 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: Issuing kill to other attempt attempt_1533196506314_4460157_m_000001_3 2018-12-10 00:43:39,374 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: task_1533196506314_4460157_m_000001 Task Transitioned from RUNNING to SUCCEEDED 2018-12-10 00:43:39,375 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: Num completed Tasks: 4 2018-12-10 00:43:39,376 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: job_1533196506314_4460157Job Transitioned from RUNNING to COMMITTING 2018-12-10 00:43:39,376 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1533196506314_4460157_m_000001_3 TaskAttempt Transitioned from RUNNING to KILL_CONTAINER_CLEANUP 2018-12-10 00:43:39,377 INFO [ContainerLauncher #7] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Processing the event EventType: CONTAINER_REMOTE_CLEANUP for container container_e17_1533196506314_4460157_01_000009 taskAttempt attempt_1533196506314_4460157_m_000001_3 2018-12-10 00:43:39,377 INFO [ContainerLauncher #7] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: KILLING attempt_1533196506314_4460157_m_000001_3 2018-12-10 00:43:39,377 INFO [CommitterEvent Processor #0] org.apache.hadoop.mapreduce.v2.app.commit.CommitterEventHandler: Processing the event EventType: JOB_COMMIT 2018-12-10 00:43:39,377 INFO [ContainerLauncher #7] org.apache.hadoop.yarn.client.api.impl.ContainerManagementProtocolProxy: Opening proxy : 192-168-72-84:13498 2018-12-10 00:43:39,383 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1533196506314_4460157_m_000001_3 TaskAttempt Transitioned from KILL_CONTAINER_CLEANUP to KILL_TASK_CLEANUP 2018-12-10 00:43:39,383 INFO [CommitterEvent Processor #1] org.apache.hadoop.mapreduce.v2.app.commit.CommitterEventHandler: Processing the event EventType: TASK_ABORT 2018-12-10 00:43:39,385 WARN [CommitterEvent Processor #1] org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter: Could not delete hdfs://ycluster/user/hive/warehouse/ods.db/bss_customer_fj/_SCRATCH0.3130352759450352/dt=20181209/_temporary/1/_temporary/attempt_1533196506314_4460157_m_000000_0 2018-12-10 00:43:39,385 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1533196506314_4460157_m_000001_3 TaskAttempt Transitioned from KILL_TASK_CLEANUP to KILLED 2018-12-10 00:43:39,419 INFO [CommitterEvent Processor #0] org.apache.hadoop.hive.conf.HiveConf: Found configuration file null 2018-12-10 00:43:39,560 INFO [CommitterEvent Processor #0] org.apache.hadoop.conf.Configuration.deprecation: io.bytes.per.checksum is deprecated. Instead, use dfs.bytes-per-checksum 2018-12-10 00:43:39,561 INFO [CommitterEvent Processor #0] org.apache.hadoop.conf.Configuration.deprecation: mapred.jar is deprecated. Instead, use mapreduce.job.jar 2018-12-10 00:43:39,664 INFO [CommitterEvent Processor #0] org.apache.hadoop.hive.metastore.HiveMetaStore: 0: Opening raw store with implementation class:org.apache.hadoop.hive.metastore.ObjectStore 2018-12-10 00:43:39,716 INFO [CommitterEvent Processor #0] org.apache.hadoop.hive.metastore.ObjectStore: ObjectStore, initialize called 2018-12-10 00:43:39,951 INFO [CommitterEvent Processor #0] DataNucleus.Persistence: Property hive.metastore.integral.jdo.pushdown unknown - will be ignored 2018-12-10 00:43:39,953 INFO [CommitterEvent Processor #0] DataNucleus.Persistence: Property datanucleus.cache.level2 unknown - will be ignored 2018-12-10 00:43:40,165 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Before Scheduling: PendingReds:0 ScheduledMaps:0 ScheduledReds:0 AssignedMaps:2 AssignedReds:0 CompletedMaps:4 CompletedReds:0 ContAlloc:9 ContRel:0 HostLocal:0 RackLocal:0 2018-12-10 00:43:40,167 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Received completed container container_e17_1533196506314_4460157_01_000008 2018-12-10 00:43:40,167 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Received completed container container_e17_1533196506314_4460157_01_000009 2018-12-10 00:43:40,167 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: After Scheduling: PendingReds:0 ScheduledMaps:0 ScheduledReds:0 AssignedMaps:0 AssignedReds:0 CompletedMaps:4 CompletedReds:0 ContAlloc:9 ContRel:0 HostLocal:0 RackLocal:0 2018-12-10 00:43:40,167 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: Diagnostics report from attempt_1533196506314_4460157_m_000001_2: Container killed by the ApplicationMaster. Container killed on request. Exit code is 143 Container exited with a non-zero exit code 143 2018-12-10 00:43:40,167 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: Diagnostics report from attempt_1533196506314_4460157_m_000001_3: Container killed by the ApplicationMaster. Container killed on request. Exit code is 143 Container exited with a non-zero exit code 143 2018-12-10 00:43:40,919 INFO [CommitterEvent Processor #0] org.apache.hadoop.hive.metastore.ObjectStore: Setting MetaStore object pin classes with hive.metastore.cache.pinobjtypes="Table,StorageDescriptor,SerDeInfo,Partition,Database,Type,FieldSchema,Order" 2018-12-10 00:43:42,522 INFO [CommitterEvent Processor #0] org.apache.hadoop.hive.metastore.MetaStoreDirectSql: Using direct SQL, underlying DB is MYSQL 2018-12-10 00:43:42,525 INFO [CommitterEvent Processor #0] org.apache.hadoop.hive.metastore.ObjectStore: Initialized ObjectStore 2018-12-10 00:43:42,835 INFO [CommitterEvent Processor #0] org.apache.hadoop.hive.metastore.HiveMetaStore: Added admin role in metastore 2018-12-10 00:43:42,838 INFO [CommitterEvent Processor #0] org.apache.hadoop.hive.metastore.HiveMetaStore: Added public role in metastore 2018-12-10 00:43:42,862 INFO [CommitterEvent Processor #0] org.apache.hadoop.hive.metastore.HiveMetaStore: No user is added in admin role, since config is empty 2018-12-10 00:43:43,039 INFO [CommitterEvent Processor #0] org.apache.hadoop.hive.metastore.HiveMetaStore: 0: get_databases: NonExistentDatabaseUsedForHealthCheck 2018-12-10 00:43:43,040 INFO [CommitterEvent Processor #0] org.apache.hadoop.hive.metastore.HiveMetaStore.audit: ugi=hadoop ip=unknown-ip-addr cmd=get_databases: NonExistentDatabaseUsedForHealthCheck 2018-12-10 00:43:43,173 INFO [CommitterEvent Processor #0] org.apache.hive.hcatalog.mapreduce.FileOutputCommitterContainer: HAR not is not being used. The table bss_customer_fj has new partitions [{dt=20181209}]. 2018-12-10 00:43:43,185 INFO [CommitterEvent Processor #0] org.apache.hadoop.hive.metastore.HiveMetaStore: 0: get_partition : db=ods tbl=bss_customer_fj[20181209] 2018-12-10 00:43:43,185 INFO [CommitterEvent Processor #0] org.apache.hadoop.hive.metastore.HiveMetaStore.audit: ugi=hadoop ip=unknown-ip-addr cmd=get_partition : db=ods tbl=bss_customer_fj[20181209] 2018-12-10 00:43:43,294 INFO [CommitterEvent Processor #0] org.apache.hadoop.hive.metastore.HiveMetaStore: 0: add_partition 2018-12-10 00:43:43,294 INFO [CommitterEvent Processor #0] org.apache.hadoop.hive.metastore.HiveMetaStore.audit: ugi=hadoop ip=unknown-ip-addr cmd=add_partition 2018-12-10 00:43:43,294 INFO [CommitterEvent Processor #0] org.apache.hadoop.hive.metastore.HiveMetaStore: 0: add_partitions 2018-12-10 00:43:43,294 INFO [CommitterEvent Processor #0] org.apache.hadoop.hive.metastore.HiveMetaStore.audit: ugi=hadoop ip=unknown-ip-addr cmd=add_partitions 2018-12-10 00:43:43,322 WARN [HMSHandler #0] hive.log: Updating partition stats fast for: bss_customer_fj 2018-12-10 00:43:43,324 WARN [HMSHandler #0] hive.log: Updated size to 240886867 2018-12-10 00:43:43,385 INFO [CommitterEvent Processor #0] org.apache.hive.hcatalog.mapreduce.FileOutputCommitterContainer: Cancelling delegation token for the job. 2018-12-10 00:43:43,451 INFO [CommitterEvent Processor #0] org.apache.hadoop.conf.Configuration.deprecation: io.bytes.per.checksum is deprecated. Instead, use dfs.bytes-per-checksum 2018-12-10 00:43:43,452 INFO [CommitterEvent Processor #0] org.apache.hadoop.conf.Configuration.deprecation: mapred.jar is deprecated. Instead, use mapreduce.job.jar 2018-12-10 00:43:43,456 INFO [CommitterEvent Processor #0] org.apache.hadoop.hive.metastore.HiveMetaStore: 0: get_databases: NonExistentDatabaseUsedForHealthCheck 2018-12-10 00:43:43,456 INFO [CommitterEvent Processor #0] org.apache.hadoop.hive.metastore.HiveMetaStore.audit: ugi=hadoop ip=unknown-ip-addr cmd=get_databases: NonExistentDatabaseUsedForHealthCheck 2018-12-10 00:43:43,464 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: Calling handler for JobFinishedEvent 2018-12-10 00:43:43,465 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: job_1533196506314_4460157Job Transitioned from COMMITTING to SUCCEEDED 2018-12-10 00:43:43,466 INFO [Thread-103] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: We are finishing cleanly so this is the last retry 2018-12-10 00:43:43,466 INFO [Thread-103] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Notify RMCommunicator isAMLastRetry: true 2018-12-10 00:43:43,466 INFO [Thread-103] org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator: RMCommunicator notified that shouldUnregistered is: true 2018-12-10 00:43:43,466 INFO [Thread-103] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Notify JHEH isAMLastRetry: true 2018-12-10 00:43:43,466 INFO [Thread-103] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: JobHistoryEventHandler notified that forceJobCompletion is true 2018-12-10 00:43:43,466 INFO [Thread-103] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Calling stop for all the services 2018-12-10 00:43:43,466 INFO [Thread-103] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Stopping JobHistoryEventHandler. Size of the outstanding queue size is 0 2018-12-10 00:43:43,532 INFO [eventHandlingThread] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Copying hdfs://ycluster:8020/tmp/hadoop-yarn/staging/hadoop/.staging/job_1533196506314_4460157/job_1533196506314_4460157_1.jhist to hdfs://ycluster:8020/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1533196506314_4460157-1544373637930-hadoop-bss_customer.jar-1544373823462-4-0-SUCCEEDED-root.hadoop-1544373649902.jhist_tmp 2018-12-10 00:43:43,550 INFO [eventHandlingThread] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Copied to done location: hdfs://ycluster:8020/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1533196506314_4460157-1544373637930-hadoop-bss_customer.jar-1544373823462-4-0-SUCCEEDED-root.hadoop-1544373649902.jhist_tmp 2018-12-10 00:43:43,554 INFO [eventHandlingThread] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Copying hdfs://ycluster:8020/tmp/hadoop-yarn/staging/hadoop/.staging/job_1533196506314_4460157/job_1533196506314_4460157_1_conf.xml to hdfs://ycluster:8020/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1533196506314_4460157_conf.xml_tmp 2018-12-10 00:43:43,572 INFO [eventHandlingThread] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Copied to done location: hdfs://ycluster:8020/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1533196506314_4460157_conf.xml_tmp 2018-12-10 00:43:43,574 INFO [eventHandlingThread] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Moved tmp to done: hdfs://ycluster:8020/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1533196506314_4460157.summary_tmp to hdfs://ycluster:8020/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1533196506314_4460157.summary 2018-12-10 00:43:43,575 INFO [eventHandlingThread] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Moved tmp to done: hdfs://ycluster:8020/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1533196506314_4460157_conf.xml_tmp to hdfs://ycluster:8020/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1533196506314_4460157_conf.xml 2018-12-10 00:43:43,577 INFO [eventHandlingThread] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Moved tmp to done: hdfs://ycluster:8020/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1533196506314_4460157-1544373637930-hadoop-bss_customer.jar-1544373823462-4-0-SUCCEEDED-root.hadoop-1544373649902.jhist_tmp to hdfs://ycluster:8020/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1533196506314_4460157-1544373637930-hadoop-bss_customer.jar-1544373823462-4-0-SUCCEEDED-root.hadoop-1544373649902.jhist 2018-12-10 00:43:43,577 INFO [Thread-103] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Stopped JobHistoryEventHandler. super.stop() 2018-12-10 00:43:43,580 INFO [Thread-103] org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator: Setting job diagnostics to 2018-12-10 00:43:43,581 INFO [Thread-103] org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator: History url is http://ycluster-logserver:19888/jobhistory/job/job_1533196506314_4460157 2018-12-10 00:43:43,586 INFO [Thread-103] org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator: Waiting for application to be successfully unregistered. 2018-12-10 00:43:44,587 INFO [Thread-103] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Final Stats: PendingReds:0 ScheduledMaps:0 ScheduledReds:0 AssignedMaps:0 AssignedReds:0 CompletedMaps:4 CompletedReds:0 ContAlloc:9 ContRel:0 HostLocal:0 RackLocal:0 2018-12-10 00:43:44,588 INFO [Thread-103] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Deleting staging directory hdfs://ycluster /tmp/hadoop-yarn/staging/hadoop/.staging/job_1533196506314_4460157 2018-12-10 00:43:44,596 INFO [Thread-103] org.apache.hadoop.ipc.Server: Stopping server on 8046 2018-12-10 00:43:44,598 INFO [IPC Server listener on 8046] org.apache.hadoop.ipc.Server: Stopping IPC Server listener on 8046 2018-12-10 00:43:44,598 INFO [TaskHeartbeatHandler PingChecker] org.apache.hadoop.mapreduce.v2.app.TaskHeartbeatHandler: TaskHeartbeatHandler thread interrupted 2018-12-10 00:43:44,598 INFO [IPC Server Responder] org.apache.hadoop.ipc.Server: Stopping IPC Server Responder  VERSION1/-container_e17_1533196506314_4460157_01_000001none��T��Tdata:BCFile.indexnone��� data:TFile.indexnone��j==data:TFile.metanone��d����h��׶9�A@���P