Kerberos, SPNEGO and WebHDFS on Hadoop using Chrome browser:

SPNEGO, and WebHDFS on Hadoop using Chrome browser:

Reference:

http://www.ghostar.org/2015/06/google-chrome-spnego-and-webhdfs-on-hadoop/

 

We want to see if the Chrome browser can be used to authenticate users with Kerberos and display Hadoop webhdfs REST api data.

In the Cloudera Security .pdf manual follow these steps:

Step 9: (Optional) Enable Authentication for HTTP Web Consoles for Hadoop Roles

Authentication for access to the HDFS, MapReduce, and YARN roles’ web consoles can be enabled using a configuration

option for the appropriate service. To enable this authentication for both HDFS and YARN:

  1. From the Clusters tab, select the service (HDFS, MapReduce, or YARN) for which you want to enable authentication.
  2. Click the Configuration tab.
  3. Select Scope > service name Service-Wide.
  4. Select Category > Security.
  5. Type Enable Kerberos in the Search box.
  6. Select Enable Kerberos Authentication for HTTP Web-Consoles.
  7. Click Save Changes to commit the changes.
  8. When the command finishes, restart all roles of that service.

 

Now on any Chrome browser on windows laptop the below webhdfs access fails:

http://192.x.x.x:50070/webhdfs/v1/?op=LISTSTATUS

 

HTTP ERROR 403

Problem accessing /webhdfs/v1/. Reason:

GSSException: Defective token detected (Mechanism level: GSSHeader did not find the right tag)

Powered by Jetty://

What to do now??

See this blog for reference:

http://www.ghostar.org/2015/06/google-chrome-spnego-and-webhdfs-on-hadoop/

Try to access webhdfs using curl:

To access WebHDFS in secure mode, a new Kerberos user (or principal) must be created in Kerberos. To do this, use the kadmin.local command on the host where Kerberos is installed, and then run the addprinc <username> command. For example:

# kadmin.local
Authenticating as principal root/admin@EXAMPLE.COM with password.
kadmin.local:  addprinc joe
WARNING: no policy specified for joe@EXAMPLE.COM; defaulting to no policy
Enter password for principal "joe@EXAMPLE.COM": 
Principal "joe@EXAMPLE.COM" created.

Add to Groups

Group information is accessed on the Namenode. If you need the principal you just created (joe in the example above) to reside in specific groups (for example, if you need permission to run a GETCONTENTSUMMARY command), you need to create an OS user on the Namenode that belongs to the groups you need: for example, hadoop.

To add a regular user on the NameNode, run the adduser command, as follows:

$ adduser -N -g hadoop joe

 

# kinit joe

# klist

Ticket cache: FILE:/tmp/krb5cc_0

Default principal: joe@EXAMPLE.COM

Valid starting       Expires              Service principal

07/18/2018 20:08:16  07/19/2018 20:08:16  krbtgt/EXAMPLE.COM@EXAMPLE.COM

renew until 07/25/2018 20:08:16

 

Next,  invoke curl with the negotiate option and the user set to anyUser.  This is a fake user required by curl to initialize the authentication code.  The real user is determined as part of the Kerberos authentication process.

 

# curl -s -i –negotiate -u:anyUser http://192.x.x.x:50070/webhdfs/v1/?op=LISTSTATUS

HTTP/1.1 401 Authentication required

Cache-Control: must-revalidate,no-cache,no-store

HTTP/1.1 200 OK

WWW-Authenticate: Negotiate <random string>

Set-Cookie: hadoop.auth=”u=joe&p=joe@EXAMPLE.COM&t=kerberos&e=1539000&s=hToQhJM=”; Path=/; HttpOnly

Transfer-Encoding: chunked

 

{“FileStatuses”:{“FileStatus”:[

{“accessTime”:0,”blockSize”:0,”childrenNum”:9,”fileId”:16387,”group”:”hbase”,”length”:0,”modificationTime”:1531926316327,”owner”:”hbase”,”pathSuffix”:”hbase”,”permission”:”700″,”replication”:0,”storagePolicy”:0,”type”:”DIRECTORY”},

{“accessTime”:0,”blockSize”:0,”childrenNum”:0,”fileId”:16409,”group”:”solr”,”length”:0,”modificationTime”:1530655569105,”owner”:”solr”,”pathSuffix”:”solr”,”permission”:”775″,”replication”:0,”storagePolicy”:0,”type”:”DIRECTORY”},

]}}

This means that Kerberos is working via curl.

NEXT install google chrome browser on Centos host where the KDC is running cent1.example.com

$ su – joe

$ kinit joe

$ klist

Ticket cache: FILE:/tmp/krb5cc_1001

Default principal: joe@EXAMPLE.COM

 

Valid starting       Expires              Service principal

07/18/2018 18:33:50  07/19/2018 18:33:50  krbtgt/EXAMPLE.COM@EXAMPLE.COM

renew until 07/25/2018 18:33:50

 

Install Mobaxterm which has n X-Server from https://mobaxterm.mobatek.net/

Hover over the X-Server icon in Mobaxterm to right.

$ export DISPLAY=10.x.x.x:0.0

Start the chrome browser with whitelist parms:

$ google-chrome –auth-server-whitelist=”*.example.com” –auth-negotiate-delegate-whitelist=”*.example.com”

 

Try this URL on Chrome running on X-Server:

http://cent1.example.com:50070/webhdfs/v1/user?op=LISTSTATUS

This will display the webhdfs content successfully.

Now destroy the Kerberos ticket and try to browse the url.

First close the Chrome browser to remove any cached ticket.

$ kdestroy

$ klist

klist: No credentials cache found (filename: /tmp/krb5cc_1001)

$ google-chrome –auth-server-whitelist=”*.example.com” –auth-negotiate-delegate-whitelist=”*.example.com”

http://cent1.example.com:50070/webhdfs/v1/?op=LISTSTATUS

 

HTTP ERROR 401

Problem accessing /webhdfs/v1/. Reason:

Authentication required

Powered by Jetty://

 

On another terminal do kinit again to generate ticket:

[joe]$ kinit

Password for joe@EXAMPLE.COM:

$ klist

Ticket cache: FILE:/tmp/krb5cc_1001

Default principal: joe@EXAMPLE.COM

Valid starting       Expires              Service principal

07/18/2018 21:53:58  07/19/2018 21:53:58  krbtgt/EXAMPLE.COM@EXAMPLE.COM

renew until 07/25/2018 21:53:58

Try the URL again:

http://cent1.example.com:50070/webhdfs/v1/?op=LISTSTATUS

 

{“FileStatuses”:{“FileStatus”:[

{“accessTime”:0,”blockSize”:0,”childrenNum”:9,”fileId”:16387,”group”:”hbase”,”length”:0,”modificationTime”:1531926316327,”owner”:”hbase”,”pathSuffix”:”hbase”,”permission”:”700″,”replication”:0,”storagePolicy”:0,”type”:”DIRECTORY”},

{“accessTime”:0,”blockSize”:0,”childrenNum”:8,”fileId”:16455,”group”:”supergroup”,”length”:0,”modificationTime”:1531852073197,”owner”:”hdfs”,”pathSuffix”:”user”,”permission”:”755″,”replication”:0,”storagePolicy”:0,”type”:”DIRECTORY”}

]}}

This shows that if the Kerberos ticket is valid then the webhdfs REST api can successfully authenticate the user and access the data. Remember that the google chrome browser needs to run from the same linux server where the kinit command was run.

 

 

 

 

 

 

 

 

 

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.