Following python code makes REST calls to a secure Kerberos enabled Hadoop cluster to use webhdfs REST api to get file data: You need to first run $ knit userid@REALM to authenticate and initiate the Kerberos ticket for the user.Make sure the python modules requests and requests_kerberos have been installed. Otherwise install it for example: … Continue reading Run a Python program to access Hadoop webhdfs and Hive with Kerberos enabled
Following steps are needed to run a Java program in Hadoop with Kerberos security enabled: 1. Create a text file named FileCount.java and store it in your home directory such as /home/userid 2. Copy paste the below code into the file FileCount.java . Change the hadoop hostname from quickstart.cloudera:8020 to the correct host. import java.io.*; import … Continue reading Run a Java program in Hadoop with Kerberos enabled.
SPNEGO, and WebHDFS on Hadoop using Chrome browser: Reference: http://www.ghostar.org/2015/06/google-chrome-spnego-and-webhdfs-on-hadoop/ We want to see if the Chrome browser can be used to authenticate users with Kerberos and display Hadoop webhdfs REST api data. In the Cloudera Security .pdf manual follow these steps: Step 9: (Optional) Enable Authentication for HTTP Web Consoles for Hadoop Roles Authentication … Continue reading Kerberos, SPNEGO and WebHDFS on Hadoop using Chrome browser:
If you have a Centos server and want to install Chrome browser which will display GUI on Windows laptop using X server then follow these steps: First download and install Mobax terminal software on your laptop from https://mobaxterm.mobatek.net/ . This will provide the X server needed to display the browser on windows. Enable Google YUM … Continue reading Install Google Chrome on Centos 7
Reference: http://blog.cloudera.com/blog/2015/03/how-to-quickly-configure-kerberos-for-your-apache-hadoop-cluster/ Cloudera Security manual .pdf – CDH 5.15 on Cloudera Documentation website http://www.ghostar.org/2015/06/google-chrome-spnego-and-webhdfs-on-hadoop/ https://www.youtube.com/watch?v=4TwU0LwDJAg Environment: Cloudera CDH 5.15 on Centos 7 MIT KDC Kerberos Setting up Kerberos in Cloudera CDH is somewhat tricky. The above blog is a good step by step way to setup. Also refer to the official Cloudera Security … Continue reading Kerberos setup in Cloudera Hadoop
Virtualbox network setup for internet access: Generally the following table gives the connectivity for different Virtualbox Network adapters. Sometimes it is possible the Bridged network wont get an ipv4 address and cannot connect to internet. Then we have to setup both a NAT and a Host-only adapter network. The requirement is: Host is Windows 10, … Continue reading Virtualbox VM setups.
Reference: http://linuxresolved.com/upgrade-mysql-5-6-mysql-5-7-centos/ 1. Important backup your databases before upgrade. Create a backup of the original MySQL data otherwise you may lose all your data. $ mysqldump -v -u root -p --all-databases > /tmp/backups/mysql56_all_databases_dump.`date +%F`.sql 2. # service mysql stop $ cp /var/lib/mysql /var/lib/mysql56.original 2. Download the MySQL 5.7 RPM $ wget http://repo.mysql.com/mysql57-community-release-el7.rpm -P /tmp/ 3. … Continue reading Upgrade MySQL 5.6 to 5.7 on Centos 7