Run a Python program to access Hadoop webhdfs and Hive with Kerberos enabled

Following python code makes REST calls to a secure Kerberos enabled Hadoop cluster to use webhdfs REST api to get file data: You need to first run $ knit userid@REALM to authenticate and initiate the Kerberos ticket for the user.Make sure the python modules requests and requests_kerberos have been installed. Otherwise install it for example: … Continue reading Run a Python program to access Hadoop webhdfs and Hive with Kerberos enabled

Advertisements

Run a Java program in Hadoop with Kerberos enabled.

Following steps are needed to run a Java program in Hadoop with Kerberos security enabled: 1. Create a text file named FileCount.java and store it in your home directory such as /home/userid 2. Copy paste the below code into the file FileCount.java . Change the hadoop hostname from quickstart.cloudera:8020 to the correct host. import java.io.*; import … Continue reading Run a Java program in Hadoop with Kerberos enabled.

Kerberos, SPNEGO and WebHDFS on Hadoop using Chrome browser:

SPNEGO, and WebHDFS on Hadoop using Chrome browser: Reference: http://www.ghostar.org/2015/06/google-chrome-spnego-and-webhdfs-on-hadoop/   We want to see if the Chrome browser can be used to authenticate users with Kerberos and display Hadoop webhdfs REST api data. In the Cloudera Security .pdf manual follow these steps: Step 9: (Optional) Enable Authentication for HTTP Web Consoles for Hadoop Roles … Continue reading Kerberos, SPNEGO and WebHDFS on Hadoop using Chrome browser:

Install Google Chrome on Centos 7

If you have a Centos server and want to install Chrome browser which will display GUI on Windows laptop using X server then follow these steps: First download and install Mobax terminal software on your laptop from https://mobaxterm.mobatek.net/ . This will provide the X server needed to display the browser on windows. Enable Google YUM … Continue reading Install Google Chrome on Centos 7

Kerberos setup in Cloudera Hadoop

Reference: http://blog.cloudera.com/blog/2015/03/how-to-quickly-configure-kerberos-for-your-apache-hadoop-cluster/ Cloudera Security manual .pdf – CDH 5.15 on Cloudera Documentation website http://www.ghostar.org/2015/06/google-chrome-spnego-and-webhdfs-on-hadoop/ https://www.youtube.com/watch?v=4TwU0LwDJAg   Environment: Cloudera CDH 5.15 on Centos 7 MIT KDC Kerberos   Setting up Kerberos in Cloudera CDH is somewhat tricky. The above blog is a good step by step way to setup. Also refer to the official Cloudera Security … Continue reading Kerberos setup in Cloudera Hadoop

Upgrade MySQL 5.6 to 5.7 on Centos 7

Reference: http://linuxresolved.com/upgrade-mysql-5-6-mysql-5-7-centos/ 1. # service mysql stop 2. Important backup your databases before upgrade. Create a backup of the original MySQL data. $ cp /var/lib/mysql /var/lib/mysql.original 2. Download the MySQL 5.7 RPM $ wget http://repo.mysql.com/mysql57-community-release-el7.rpm -P /tmp/ 3. Remove the MySQL-Community RPM that contains MySQL 5.6 $ yum remove mysql-community-release 4. Install the MySQL 5.7 RPM … Continue reading Upgrade MySQL 5.6 to 5.7 on Centos 7