{"id":1187,"date":"2010-12-16T06:38:37","date_gmt":"2010-12-15T20:38:37","guid":{"rendered":"http:\/\/www.melvic.info\/?p=1187"},"modified":"2012-04-15T23:04:00","modified_gmt":"2012-04-15T13:04:00","slug":"26","status":"publish","type":"post","link":"https:\/\/www.melvic.info\/?p=1187","title":{"rendered":"Hadoop\u30a4\u30f3\u30b9\u30c8\u30fc\u30eb\u30e1\u30e2"},"content":{"rendered":"<p>CDH(Coudela&#8217;s Distribution for Hadoop)3 \u3092 Fedora 13 64bit\u5358\u4f53\u306b\u30a4\u30f3\u30b9\u30c8\u30fc\u30eb\u3059\u308b\u624b\u9806\u3092\u307e\u3068\u3081\u308b\u3002<!--more-->\u4e00\u53f0\u306e\u30de\u30b7\u30f3\u306a\u306e\u3067HDFS\u306f\u4eee\u60f3\u5206\u6563\u30e2\u30fc\u30c9\uff08Pseudo Distributed Mode\uff09\u3068\u3059\u308b\u3002<\/p>\n<p>Fedora\u306e\u30a4\u30f3\u30b9\u30c8\u30fc\u30eb\u306f\u5b8c\u4e86\u3057\u3066\u304a\u308a\uff0c\u30a4\u30f3\u30bf\u30fc\u30cd\u30c3\u30c8\u30a2\u30af\u30bb\u30b9\u306a\u3069\u3082\u78ba\u4fdd\u3067\u304d\u3066\u3044\u308b\u3068\u4eee\u5b9a\u3057\u3066\u8a71\u3092\u9032\u3081\u308b\u3002<br \/>\nFedora12\u4ee5\u964d\uff0cRedhat\u7cfb\u306b\u3082\u5fdc\u7528\u53ef\u80fd\u306e\u306f\u305a\u3067\u3042\u308b\u3002<\/p>\n<p><strong>1. JDK\u306e\u30a4\u30f3\u30b9\u30c8\u30fc\u30eb<\/strong><br \/>\nwww.oracle.com \u304b\u3089\uff0c\u30d7\u30e9\u30c3\u30c8\u30d5\u30a9\u30fc\u30e0\u306b\u5408\u3063\u305fJDK\u3092\u30c0\u30a6\u30f3\u30ed\u30fc\u30c9\u3059\u308b\u3002<br \/>\nFedora\u306e\u5834\u5408\uff0c\u30d5\u30a1\u30a4\u30eb\u540d\u304c ~rpm.bin\u3068\u306a\u3063\u3066\u3044\u308b\u3082\u306e\u304c\u304a\u52e7\u3081\u3002<\/p>\n<p>\u57f7\u7b46\u6642\u70b9\u3067Linux x68 64\u30d3\u30c3\u30c8\u7248\u306f<br \/>\njdk-6u22-linux-x64-rpm.bin \u3060\u3063\u305f\u3002<\/p>\n<p>Root\u3067\u30ed\u30b0\u30a4\u30f3\u3057\u3066\uff0c\u3053\u306e\u30d5\u30a1\u30a4\u30eb\u306b\u5b9f\u884c\u6a29\u3092\u3064\u3051\u3066\u5b9f\u884c\u3059\u308b\u3002<\/p>\n<p># chmod +x jdk-6u22-linux-x64-rpm.bin<br \/>\n# .\/jdk-6u22-linux-x64-rpm.bin<\/p>\n<p>\u3053\u308c\u3067\u81ea\u52d5\u7684\u306b\u89e3\u51cd\u3055\u308c\u308b\u3002\u7d9a\u884c\u3059\u308b\u304b\u805e\u3044\u3066\u304f\u308b\u306e\u3067Enter\u3092\u62bc\u3059\u3068rpm\u3082\u81ea\u52d5\u5b9f\u884c\u3055\u308c\uff0c\u30a4\u30f3\u30b9\u30c8\u30fc\u30eb\u304c\u5b8c\u4e86\u3059\u308b\u3002<\/p>\n<p>\u3053\u3053\u3067\uff0cJDK\u304c\u3069\u3053\u306b\u30a4\u30f3\u30b9\u30c8\u30fc\u30eb\u3055\u308c\u305f\u304b\u3092\u77e5\u3063\u3066\u304a\u304f\u3068\uff0c\u5f8c\u3067\u4f55\u304b\u3068\u4fbf\u5229\u3067\u3042\u308b\u3002<\/p>\n<p># rpm -ql jdk | more<br \/>\n\/etc<br \/>\n\/etc\/.java<br \/>\n\/etc\/.java\/.systemPrefs<br \/>\n\/etc\/.java\/.systemPrefs\/.system.lock<br \/>\n\/etc\/.java\/.systemPrefs\/.systemRootModFile<br \/>\n\/etc\/init.d\/jexec<br \/>\n\/usr<br \/>\n\/usr\/java<br \/>\n\/usr\/java\/jdk1.6.0_22<br \/>\n\/usr\/java\/jdk1.6.0_22\/COPYRIGHT<br \/>\n\/usr\/java\/jdk1.6.0_22\/README.html<br \/>\n\/usr\/java\/jdk1.6.0_22\/README_ja.html<br \/>\n\/usr\/java\/jdk1.6.0_22\/README_zh_CN.html<br \/>\n\/usr\/java\/jdk1.6.0_22\/THIRDPARTYLICENSEREADME.txt<br \/>\n\/usr\/java\/jdk1.6.0_22\/bin<br \/>\n\uff1a<br \/>\n\uff1a<\/p>\n<p>\u3068\u306a\u308a\uff0cJAVA_HOME\u306f\/usr\/java\/jdk1.6.0_22\u3068\u5224\u660e\u3059\u308b\u3002<\/p>\n<p><strong>2. Cloudera\u306eYum\u30ec\u30dd\u30b8\u30c8\u30ea\u3092\u5165\u624b\u3057yum\u3092\u30a2\u30c3\u30d7\u30c7\u30fc\u30c8<\/strong><\/p>\n<p># curl http:\/\/archive.cloudera.com\/redhat\/cdh\/cloudera-cdh3.repo &gt; \/etc\/yum.repos.d\/cloudera-cdh3.repo<br \/>\n# yum update yum<\/p>\n<p>\u3053\u308c\u3067\uff0cyum\u3067CDH\u304c\u30a4\u30f3\u30b9\u30c8\u30fc\u30eb\u3067\u304d\u308b\u3088\u3046\u306b\u306a\u308b\u3002<\/p>\n<p><strong>3. Hadoop\u306e\u30a4\u30f3\u30b9\u30c8\u30fc\u30eb<\/strong><\/p>\n<p># yum install hadoop-0.20-conf-pseudo<\/p>\n<p>\u3053\u306e\u5f8c\uff0c\u5fc5\u8981\u306b\u5fdc\u3058\u3066CDH3\u306e\u30b3\u30f3\u30dd\u30fc\u30cd\u30f3\u30c8\u3092\u30a4\u30f3\u30b9\u30c8\u30fc\u30eb\u3059\u308b\u3002<br \/>\n\u4f8b\uff09 # yum install hadoop-hbase<br \/>\n\u30b3\u30f3\u30dd\u30fc\u30cd\u30f3\u30c8\u540d\u3068\u3057\u3066\u6307\u5b9a\u3059\u308b\u540d\u524d\u306f\u4ee5\u4e0b\u306e\u3068\u304a\u308a<br \/>\nFlume: flume<br \/>\nSqoop: sqoop<br \/>\nHue: hue<br \/>\nPig: hadoop-pig<br \/>\nHive: hadoop-hive<br \/>\nHBase: hadoop-hbase<br \/>\nZookeeper: zookeeper<br \/>\nOozie server: oozie_<br \/>\nOozie client: Oozie-client_<br \/>\nWhirr: whirr<\/p>\n<p><strong>4. Hadoop\u8d77\u52d5<\/strong><\/p>\n<p>\/etc\/init.d \u914d\u4e0b\u306b\u767b\u9332\u3055\u308c\u305f hadoop&#8211;\u3067\u59cb\u307e\u308b\u30b5\u30fc\u30d3\u30b9\u3092\u958b\u59cb\u3059\u308b\u3002<\/p>\n<p># for svc in hadoop-0.20-*<br \/>\n&gt; do<br \/>\n&gt; service $svc start<br \/>\n&gt; done<br \/>\nStarting Hadoop datanode daemon (hadoop-datanode): starting datanode, logging to \/usr\/lib\/hadoop-0.20\/logs\/hadoop-hadoop-datanode-melintqal02.out<br \/>\n[ OK ]<br \/>\nStarting Hadoop jobtracker daemon (hadoop-jobtracker): starting jobtracker, logging to \/usr\/lib\/hadoop-0.20\/logs\/hadoop-hadoop-jobtracker-melintqal02.out<br \/>\n[ OK ]<br \/>\nStarting Hadoop namenode daemon (hadoop-namenode): starting namenode, logging to \/usr\/lib\/hadoop-0.20\/logs\/hadoop-hadoop-namenode-melintqal02.out<br \/>\n[ OK ]<br \/>\nStarting Hadoop secondarynamenode daemon (hadoop-secondarynamenode): starting secondarynamenode, logging to \/usr\/lib\/hadoop-0.20\/logs\/hadoop-hadoop-secondarynamenode-melintqal02.out<br \/>\n[ OK ]<br \/>\nStarting Hadoop tasktracker daemon (hadoop-tasktracker): starting tasktracker, logging to \/usr\/lib\/hadoop-0.20\/logs\/hadoop-hadoop-tasktracker-melintqal02.out<br \/>\n[ OK ]<\/p>\n<p><strong>5. \u30c6\u30b9\u30c8<\/strong><\/p>\n<p>\u3053\u308c\u3067\u30a4\u30f3\u30b9\u30c8\u30fc\u30eb\u306f\u3059\u3079\u3066\u7d42\u4e86\u3002\u3042\u3063\u3051\u306a\u3044\u307b\u3069\u7c21\u5358\u3067\u3042\u308b\u3002<br \/>\nhadoop \u30b3\u30de\u30f3\u30c9\u306f\/usr\/bin\u306b\u30a4\u30f3\u30b9\u30c8\u30fc\u30eb\u3055\u308c\u3066\u304a\u308a\uff0c\u3059\u3050\u306bPath\u6307\u5b9a\u306a\u3057\u3067\u4f7f\u7528\u3067\u304d\u308b\u3002<\/p>\n<p>\u305f\u3068\u3048\u3070\uff0c\u4ee5\u4e0b\u306e\u3088\u3046\u306bHDFS\u3092\u8997\u3044\u3066\u307f\u308b\u3002\u52d5\u3051\u3070Hadoop\u306f\u6b63\u5e38\u306b\u52d5\u4f5c\u3057\u3066\u3044\u308b\u3068\u601d\u3063\u3066\u3044\u3044\u3002<br \/>\n$ hadoop fs -ls \/<br \/>\nFound 1 items<br \/>\ndrwxr-xr-x &#8211; mapred supergroup 0 2010-12-15 11:56 \/var<\/p>\n<p>\u3055\u3089\u306b\u3044\u304f\u3064\u304b\u3084\u3063\u3066\u307f\u308b\u3053\u3068\u304c\u3042\u308b\u304c\uff0c\u7701\u7565\u3002\u4ee5\u4e0b\u3092\u53c2\u7167\u306e\u3053\u3068<br \/>\nhttps:\/\/docs.cloudera.com\/display\/DOC\/Installing+CDH3+on+a+Single+Linux+Node+in+Pseudo-distributed+Mode<\/p>\n","protected":false},"excerpt":{"rendered":"<p>CDH(Coudela&#8217;s Distribution for Hadoop)3 \u3092 Fedora 13 64bit\u5358\u4f53\u306b\u30a4\u30f3\u30b9\u30c8\u30fc\u30eb\u3059\u308b\u624b\u9806\u3092\u307e\u3068\u3081\u308b\u3002<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"ngg_post_thumbnail":0,"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_post_was_ever_published":false},"categories":[10,15],"tags":[],"class_list":["post-1187","post","type-post","status-publish","format-standard","hentry","category-computer-technology","category-15"],"aioseo_notices":[],"jetpack_featured_media_url":"","jetpack_sharing_enabled":true,"jetpack_shortlink":"https:\/\/wp.me\/s53KbG-26","jetpack-related-posts":[],"jetpack_likes_enabled":true,"_links":{"self":[{"href":"https:\/\/www.melvic.info\/index.php?rest_route=\/wp\/v2\/posts\/1187","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.melvic.info\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.melvic.info\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.melvic.info\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.melvic.info\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=1187"}],"version-history":[{"count":1,"href":"https:\/\/www.melvic.info\/index.php?rest_route=\/wp\/v2\/posts\/1187\/revisions"}],"predecessor-version":[{"id":1271,"href":"https:\/\/www.melvic.info\/index.php?rest_route=\/wp\/v2\/posts\/1187\/revisions\/1271"}],"wp:attachment":[{"href":"https:\/\/www.melvic.info\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=1187"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.melvic.info\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=1187"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.melvic.info\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=1187"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}