前景提要
HDC调试需求开发(15万预算),能者速来!>>>
@杨尚川 你好,想跟你请教个问题:杨老师 实在不好意思,想麻烦你一个问题,困扰了我很久。
我并不是一个智慧伸手要资源的傻瓜,在学习了你相关的nutch视频之后,试着自己去尝试做一些东西。于是根据nutch官网上给出的信息,我选择了最新版的nutch 2.3,因为nutch2.3采用的是gora0.5,所以官网上建议搭配hbase0.94.14和hadoop2.4,而我去了gora官网看了一下,gora最新的版本是0.6,所以我编译nutch2.3的时候将gora hadoop 和hbase 都改成了gora官网上建议的版本。成功编译,我首先部署了hadoop和hbase ,并且也都全部启动(完全分布式,共3台机器)。
可是,当我开心的进入到nutch2.3的runtime/deploy/下 执行 ./bin/nutch inject urls 时,给出了一个这样的错误:
Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/hadoop/hbase/HBaseConfiguration
at org.apache.gora.hbase.store.HBaseStore.initialize(HBaseStore.java:114)
at org.apache.gora.store.DataStoreFactory.initializeDataStore(DataStoreFactory.java:101)
at org.apache.gora.store.DataStoreFactory.createDataStore(DataStoreFactory.java:160)
at org.apache.gora.store.DataStoreFactory.createDataStore(DataStoreFactory.java:134)
at org.apache.nutch.storage.StorageUtils.createWebStore(StorageUtils.java:78)
at org.apache.nutch.crawl.InjectorJob.run(InjectorJob.java:218)
at org.apache.nutch.crawl.InjectorJob.inject(InjectorJob.java:252)
at org.apache.nutch.crawl.InjectorJob.run(InjectorJob.java:275)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.apache.nutch.crawl.InjectorJob.main(InjectorJob.java:284)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.hbase.HBaseConfiguration
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
... 15 more
Nutch官网tutorial2.X给出的的解释是这样的:
N.B. It's possible to encounter the following exception: java.lang. NoClassDefFoundError : org/apache/hadoop/hbase/HBaseConfiguration; this is caused by the fact that sometimes the hbase TEST jar is deployed in the lib dir. To resolve this just copy the lib over from your installed HBase dir into the build lib dir. (This issue is currently in progress).
于是 我使用了这个命令cp /$HBASE_HOME/lib/*jar /$NUTCH_HOME/build/ 拷贝到了butch下,仍然不起作用,于是我又将/$HBASE_HOME/lib/*jar拷贝到了/$NUTCH_HOME/lib,第三次运行./bin/nutch inject urls,还是抛出上述的错误。无奈我求助google,google给出的结果也是很少,基本上解决不了我的问题。所以我很困惑,在这个问题上纠结了很久。鉴于,杨老师对nutch的研究颇为深入,所以向您求助。
希望您百忙中能抽空回答一下,不胜感激!