��h��׶9�A@���P VERSIONAPPLICATION_ACL MODIFY_APP VIEW_APP APPLICATION_OWNERhadoop/-container_e17_1533196506314_4460157_01_000003�.�stderr749SLF4J: Class path contains multiple SLF4J bindings. SLF4J: Found binding in [jar:file:/opt/hadoop-2.7.3/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: Found binding in [jar:file:/dataa/hadoop/yarn/data/usercache/hadoop/filecache/690255/log4j-slf4j-impl-2.4.1.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation. SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory] log4j:WARN No appenders could be found for logger (org.apache.hadoop.metrics2.impl.MetricsSystemImpl). log4j:WARN Please initialize the log4j system properly. log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info. stdout0syslog111762018-12-10 00:40:59,112 INFO [main] org.apache.hadoop.metrics2.impl.MetricsConfig: loaded properties from hadoop-metrics2.properties 2018-12-10 00:40:59,180 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Scheduled snapshot period at 10 second(s). 2018-12-10 00:40:59,180 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: MapTask metrics system started 2018-12-10 00:40:59,190 INFO [main] org.apache.hadoop.mapred.YarnChild: Executing with tokens: 2018-12-10 00:40:59,190 INFO [main] org.apache.hadoop.mapred.YarnChild: Kind: mapreduce.job, Service: job_1533196506314_4460157, Ident: (org.apache.hadoop.mapreduce.security.token.JobTokenIdentifier@5884a914) 2018-12-10 00:40:59,403 INFO [main] org.apache.hadoop.mapred.YarnChild: Sleeping for 0ms before retrying again. Got null now. 2018-12-10 00:40:59,652 INFO [main] org.apache.hadoop.mapred.YarnChild: mapreduce.cluster.local.dir for child: /data0/hadoop/yarn/data/usercache/hadoop/appcache/application_1533196506314_4460157,/data1/hadoop/yarn/data/usercache/hadoop/appcache/application_1533196506314_4460157,/data2/hadoop/yarn/data/usercache/hadoop/appcache/application_1533196506314_4460157,/data3/hadoop/yarn/data/usercache/hadoop/appcache/application_1533196506314_4460157,/data4/hadoop/yarn/data/usercache/hadoop/appcache/application_1533196506314_4460157,/data5/hadoop/yarn/data/usercache/hadoop/appcache/application_1533196506314_4460157,/data6/hadoop/yarn/data/usercache/hadoop/appcache/application_1533196506314_4460157,/data7/hadoop/yarn/data/usercache/hadoop/appcache/application_1533196506314_4460157,/data8/hadoop/yarn/data/usercache/hadoop/appcache/application_1533196506314_4460157,/data9/hadoop/yarn/data/usercache/hadoop/appcache/application_1533196506314_4460157,/dataa/hadoop/yarn/data/usercache/hadoop/appcache/application_1533196506314_4460157,/datab/hadoop/yarn/data/usercache/hadoop/appcache/application_1533196506314_4460157 2018-12-10 00:40:59,889 INFO [main] org.apache.hadoop.conf.Configuration.deprecation: session.id is deprecated. Instead, use dfs.metrics.session-id 2018-12-10 00:41:00,647 INFO [main] org.apache.hadoop.conf.Configuration.deprecation: mapred.output.dir is deprecated. Instead, use mapreduce.output.fileoutputformat.outputdir 2018-12-10 00:41:00,662 INFO [main] org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter: File Output Committer Algorithm version is 1 2018-12-10 00:41:00,663 INFO [main] org.apache.hadoop.conf.Configuration.deprecation: mapred.work.output.dir is deprecated. Instead, use mapreduce.task.output.dir 2018-12-10 00:41:00,694 INFO [main] org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter: File Output Committer Algorithm version is 1 2018-12-10 00:41:00,706 INFO [main] org.apache.hadoop.mapred.Task: Using ResourceCalculatorProcessTree : [ ] 2018-12-10 00:41:01,402 INFO [main] org.apache.sqoop.mapreduce.db.DBInputFormat: Using read commited transaction isolation 2018-12-10 00:41:01,576 INFO [main] org.apache.hadoop.mapred.MapTask: Processing split: `id` >= 2733766 AND `id` < 5467237 2018-12-10 00:41:01,608 INFO [main] org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter: File Output Committer Algorithm version is 1 2018-12-10 00:41:01,609 INFO [main] org.apache.hadoop.conf.Configuration.deprecation: mapred.output.key.class is deprecated. Instead, use mapreduce.job.output.key.class 2018-12-10 00:41:01,612 INFO [main] org.apache.hadoop.conf.Configuration.deprecation: mapred.output.value.class is deprecated. Instead, use mapreduce.job.output.value.class 2018-12-10 00:41:01,719 INFO [main] org.apache.sqoop.mapreduce.hcat.SqoopHCatImportHelper: HCatalog Storer Info : Handler = null Input format class = org.apache.hadoop.hive.ql.io.orc.OrcInputFormat Output format class = org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat Serde class = org.apache.hadoop.hive.ql.io.orc.OrcSerde Storer properties serialization.format=1 transient_lastDdlTime=1529984981 2018-12-10 00:41:01,724 INFO [main] org.apache.sqoop.mapreduce.db.DBRecordReader: Working on split: `id` >= 2733766 AND `id` < 5467237 2018-12-10 00:41:01,741 INFO [main] org.apache.sqoop.mapreduce.db.DBRecordReader: Executing query: SELECT `id`, `code`, `user_id`, `state`, `device_code`, `is_lock`, `is_sync`, `cust_id`, `start_date`, `end_date`, `activate_date`, `bind_date`, `service_stop`, `cancelled_date`, `create_date`, `update_date`, `user_type`, `password`, `product_ids`, `uid` FROM `bss_customer` AS `bss_customer` WHERE ( `id` >= 2733766 ) AND ( `id` < 5467237 ) 2018-12-10 00:41:01,888 INFO [main] org.apache.orc.impl.WriterImpl: ORC writer created for path: hdfs://ycluster/user/hive/warehouse/ods.db/bss_customer_fj/_SCRATCH0.3130352759450352/dt=20181209/_temporary/1/_temporary/attempt_1533196506314_4460157_m_000001_0/part-m-00001 with stripeSize: 67108864 blockSize: 268435456 compression: ZLIB bufferSize: 262144 2018-12-10 00:42:30,587 INFO [main] org.apache.hadoop.mapred.MapTask: Ignoring exception during close for org.apache.hadoop.mapred.MapTask$NewDirectOutputCollector@2125ad3 java.nio.channels.ClosedChannelException at org.apache.hadoop.hdfs.DFSOutputStream.checkClosed(DFSOutputStream.java:1546) at org.apache.hadoop.fs.FSOutputSummer.write(FSOutputSummer.java:104) at org.apache.hadoop.fs.FSDataOutputStream$PositionCache.write(FSDataOutputStream.java:58) at java.io.DataOutputStream.write(DataOutputStream.java:107) at org.apache.orc.impl.WriterImpl$DirectStream.output(WriterImpl.java:399) at org.apache.orc.impl.OutStream.flush(OutStream.java:245) at org.apache.orc.impl.WriterImpl.writeMetadata(WriterImpl.java:2678) at org.apache.orc.impl.WriterImpl.close(WriterImpl.java:2788) at org.apache.hadoop.hive.ql.io.orc.WriterImpl.close(WriterImpl.java:313) at org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat$OrcRecordWriter.close(OrcOutputFormat.java:120) at org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat$OrcRecordWriter.close(OrcOutputFormat.java:108) at org.apache.hive.hcatalog.mapreduce.StaticPartitionFileRecordWriterContainer.close(StaticPartitionFileRecordWriterContainer.java:53) at org.apache.hadoop.mapred.MapTask$NewDirectOutputCollector.close(MapTask.java:670) at org.apache.hadoop.mapred.MapTask.closeQuietly(MapTask.java:2019) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:797) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341) at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1698) at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158) 2018-12-10 00:42:30,589 WARN [main] org.apache.hadoop.mapred.YarnChild: Exception running child : org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.hdfs.server.namenode.LeaseExpiredException): No lease on /user/hive/warehouse/ods.db/bss_customer_fj/_SCRATCH0.3130352759450352/dt=20181209/_temporary/1/_temporary/attempt_1533196506314_4460157_m_000001_0/part-m-00001 (inode 761544109): File does not exist. Holder DFSClient_attempt_1533196506314_4460157_m_000001_0_-1729942809_1 does not have any open files. at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkLease(FSNamesystem.java:3432) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.completeFileInternal(FSNamesystem.java:3520) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.completeFile(FSNamesystem.java:3487) at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.complete(NameNodeRpcServer.java:787) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.complete(ClientNamenodeProtocolServerSideTranslatorPB.java:537) at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java) at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:616) at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:982) at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2049) at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2045) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1698) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2045) at org.apache.hadoop.ipc.Client.call(Client.java:1475) at org.apache.hadoop.ipc.Client.call(Client.java:1412) at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:229) at com.sun.proxy.$Proxy14.complete(Unknown Source) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.complete(ClientNamenodeProtocolTranslatorPB.java:462) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:191) at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102) at com.sun.proxy.$Proxy15.complete(Unknown Source) at org.apache.hadoop.hdfs.DFSOutputStream.completeFile(DFSOutputStream.java:2291) at org.apache.hadoop.hdfs.DFSOutputStream.closeImpl(DFSOutputStream.java:2267) at org.apache.hadoop.hdfs.DFSOutputStream.close(DFSOutputStream.java:2232) at org.apache.hadoop.fs.FSDataOutputStream$PositionCache.close(FSDataOutputStream.java:72) at org.apache.hadoop.fs.FSDataOutputStream.close(FSDataOutputStream.java:106) at org.apache.orc.impl.WriterImpl.close(WriterImpl.java:2791) at org.apache.hadoop.hive.ql.io.orc.WriterImpl.close(WriterImpl.java:313) at org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat$OrcRecordWriter.close(OrcOutputFormat.java:120) at org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat$OrcRecordWriter.close(OrcOutputFormat.java:108) at org.apache.hive.hcatalog.mapreduce.StaticPartitionFileRecordWriterContainer.close(StaticPartitionFileRecordWriterContainer.java:53) at org.apache.hadoop.mapred.MapTask$NewDirectOutputCollector.close(MapTask.java:670) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:793) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341) at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1698) at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158) 2018-12-10 00:42:30,591 INFO [main] org.apache.hadoop.mapred.Task: Runnning cleanup for the task 2018-12-10 00:42:30,596 WARN [main] org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter: Could not delete hdfs://ycluster/user/hive/warehouse/ods.db/bss_customer_fj/_SCRATCH0.3130352759450352/dt=20181209/_temporary/1/_temporary/attempt_1533196506314_4460157_m_000001_0 /-container_e17_1533196506314_4460157_01_000008��stderr749SLF4J: Class path contains multiple SLF4J bindings. SLF4J: Found binding in [jar:file:/opt/hadoop-2.7.3/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: Found binding in [jar:file:/dataa/hadoop/yarn/data/usercache/hadoop/filecache/690255/log4j-slf4j-impl-2.4.1.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation. SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory] log4j:WARN No appenders could be found for logger (org.apache.hadoop.metrics2.impl.MetricsSystemImpl). log4j:WARN Please initialize the log4j system properly. log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info. stdout0syslog57472018-12-10 00:42:13,217 INFO [main] org.apache.hadoop.metrics2.impl.MetricsConfig: loaded properties from hadoop-metrics2.properties 2018-12-10 00:42:13,288 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Scheduled snapshot period at 10 second(s). 2018-12-10 00:42:13,288 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: MapTask metrics system started 2018-12-10 00:42:13,296 INFO [main] org.apache.hadoop.mapred.YarnChild: Executing with tokens: 2018-12-10 00:42:13,296 INFO [main] org.apache.hadoop.mapred.YarnChild: Kind: mapreduce.job, Service: job_1533196506314_4460157, Ident: (org.apache.hadoop.mapreduce.security.token.JobTokenIdentifier@5884a914) 2018-12-10 00:42:13,512 INFO [main] org.apache.hadoop.mapred.YarnChild: Sleeping for 0ms before retrying again. Got null now. 2018-12-10 00:42:13,756 INFO [main] org.apache.hadoop.mapred.YarnChild: mapreduce.cluster.local.dir for child: /data0/hadoop/yarn/data/usercache/hadoop/appcache/application_1533196506314_4460157,/data1/hadoop/yarn/data/usercache/hadoop/appcache/application_1533196506314_4460157,/data2/hadoop/yarn/data/usercache/hadoop/appcache/application_1533196506314_4460157,/data3/hadoop/yarn/data/usercache/hadoop/appcache/application_1533196506314_4460157,/data4/hadoop/yarn/data/usercache/hadoop/appcache/application_1533196506314_4460157,/data5/hadoop/yarn/data/usercache/hadoop/appcache/application_1533196506314_4460157,/data6/hadoop/yarn/data/usercache/hadoop/appcache/application_1533196506314_4460157,/data7/hadoop/yarn/data/usercache/hadoop/appcache/application_1533196506314_4460157,/data8/hadoop/yarn/data/usercache/hadoop/appcache/application_1533196506314_4460157,/data9/hadoop/yarn/data/usercache/hadoop/appcache/application_1533196506314_4460157,/dataa/hadoop/yarn/data/usercache/hadoop/appcache/application_1533196506314_4460157,/datab/hadoop/yarn/data/usercache/hadoop/appcache/application_1533196506314_4460157 2018-12-10 00:42:13,993 INFO [main] org.apache.hadoop.conf.Configuration.deprecation: session.id is deprecated. Instead, use dfs.metrics.session-id 2018-12-10 00:42:14,665 INFO [main] org.apache.hadoop.conf.Configuration.deprecation: mapred.output.dir is deprecated. Instead, use mapreduce.output.fileoutputformat.outputdir 2018-12-10 00:42:14,673 INFO [main] org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter: File Output Committer Algorithm version is 1 2018-12-10 00:42:14,674 INFO [main] org.apache.hadoop.conf.Configuration.deprecation: mapred.work.output.dir is deprecated. Instead, use mapreduce.task.output.dir 2018-12-10 00:42:14,695 INFO [main] org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter: File Output Committer Algorithm version is 1 2018-12-10 00:42:14,706 INFO [main] org.apache.hadoop.mapred.Task: Using ResourceCalculatorProcessTree : [ ] 2018-12-10 00:42:15,380 INFO [main] org.apache.sqoop.mapreduce.db.DBInputFormat: Using read commited transaction isolation 2018-12-10 00:42:15,559 INFO [main] org.apache.hadoop.mapred.MapTask: Processing split: `id` >= 2733766 AND `id` < 5467237 2018-12-10 00:42:15,589 INFO [main] org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter: File Output Committer Algorithm version is 1 2018-12-10 00:42:15,590 INFO [main] org.apache.hadoop.conf.Configuration.deprecation: mapred.output.key.class is deprecated. Instead, use mapreduce.job.output.key.class 2018-12-10 00:42:15,593 INFO [main] org.apache.hadoop.conf.Configuration.deprecation: mapred.output.value.class is deprecated. Instead, use mapreduce.job.output.value.class 2018-12-10 00:42:15,692 INFO [main] org.apache.sqoop.mapreduce.hcat.SqoopHCatImportHelper: HCatalog Storer Info : Handler = null Input format class = org.apache.hadoop.hive.ql.io.orc.OrcInputFormat Output format class = org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat Serde class = org.apache.hadoop.hive.ql.io.orc.OrcSerde Storer properties serialization.format=1 transient_lastDdlTime=1529984981 2018-12-10 00:42:15,700 INFO [main] org.apache.sqoop.mapreduce.db.DBRecordReader: Working on split: `id` >= 2733766 AND `id` < 5467237 2018-12-10 00:42:15,717 INFO [main] org.apache.sqoop.mapreduce.db.DBRecordReader: Executing query: SELECT `id`, `code`, `user_id`, `state`, `device_code`, `is_lock`, `is_sync`, `cust_id`, `start_date`, `end_date`, `activate_date`, `bind_date`, `service_stop`, `cancelled_date`, `create_date`, `update_date`, `user_type`, `password`, `product_ids`, `uid` FROM `bss_customer` AS `bss_customer` WHERE ( `id` >= 2733766 ) AND ( `id` < 5467237 ) 2018-12-10 00:42:15,861 INFO [main] org.apache.orc.impl.WriterImpl: ORC writer created for path: hdfs://ycluster/user/hive/warehouse/ods.db/bss_customer_fj/_SCRATCH0.3130352759450352/dt=20181209/_temporary/1/_temporary/attempt_1533196506314_4460157_m_000001_2/part-m-00001 with stripeSize: 67108864 blockSize: 268435456 compression: ZLIB bufferSize: 262144 2018-12-10 00:43:39,279 INFO [main] org.apache.hadoop.mapred.Task: Task:attempt_1533196506314_4460157_m_000001_2 is done. And is in the process of committing 2018-12-10 00:43:39,318 INFO [main] org.apache.hadoop.mapred.Task: Task attempt_1533196506314_4460157_m_000001_2 is allowed to commit now 2018-12-10 00:43:39,318 INFO [main] org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter: File Output Committer Algorithm version is 1 2018-12-10 00:43:39,329 INFO [main] org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter: Saved output of task 'attempt_1533196506314_4460157_m_000001_2' to hdfs://ycluster/user/hive/warehouse/ods.db/bss_customer_fj/_SCRATCH0.3130352759450352/dt=20181209/_temporary/1/task_1533196506314_4460157_m_000001 2018-12-10 00:43:39,369 INFO [main] org.apache.hadoop.mapred.Task: Task 'attempt_1533196506314_4460157_m_000001_2' done.  VERSION1/-container_e17_1533196506314_4460157_01_000008none�I�Idata:BCFile.indexnone�IV data:TFile.indexnone�I==data:TFile.metanone�IIc��h��׶9�A@���P