adsense728x90


spark hdfs file 접근해서 조작하기 Web backend




val conf = sc.hadoopConfigurationval fs = org.apache.hadoop.fs.FileSystem.get(conf)val exists = fs.exists(new org.apache.hadoop.fs.Path("/path/on/hdfs/to/SUCCESS.txt"))


Configuration
conf = new Configuration();conf.set("fs.default.name", "hdfs://mynamenodehost:9000");FileSystem fs = FileSystem.get(conf);




val hadoopConf = new org.apache.hadoop.conf.Configuration()val hdfs = org.apache.hadoop.fs.FileSystem.get(new java.net.URI("hdfs://localhost:9000"), hadoopConf)try { hdfs.delete(new org.apache.hadoop.fs.Path(filepath), true) } catch { case _ : Throwable => { } }


덧글

댓글 입력 영역