Cloudera Search (Solr) install steps

The following steps are used to install Cloudera Search which is based on Apache Solr. Environment: Cloudera CDH 5.12.x solr-spec 4.10.3 Deploying Cloudera Search Cloudera Search (powered by Apache Solr) is included in CDH 5. If you have installed CDH 5.0 or higher, you do not need to perform any additional actions to install Search. … Continue reading Cloudera Search (Solr) install steps

Hadoop architecture notes

Below are some reference architectures for Hadoop: Cluster Hosts and Role Assignments CDH 5 and Cloudera Manager 5 Install/Upgrade/Requirements and Supported Versions   High Availability support HDFS Directory Structure recommendation (Eric Sammer): /data : Contains canonical, raw data sets ingested from other systems. Read only to users. /user/<username> : … Continue reading Hadoop architecture notes

Kafka install on Cloudera Hadoop

Below are the steps to install Kafka parcel in Cloudera manager. Cloudera Distribution of Apache Kafka Requirements and Supported Versions: Cloudera Kafka 2.2.x lowest supported Cloudera manager version 5.9.x, CDH 5.9.x and higher . General Information Regarding Installation and Upgrade These are the official instructions: Cloudera recommends that you deploy Kafka on dedicated hosts … Continue reading Kafka install on Cloudera Hadoop

Business Intelligence, ETL and Data Science tools

Free or Opensource BI / ETL tools: Talend = ETL tool, leader in Gartner Magic Quadrant Streamsets = ETL tool Apache Nifi = ETL tool Pentaho = desktop and server version BI/ETL tool HUE = Hadoop Analytics server, BI, Query tool KNIME = Data Science leader in Gartner Magic Quadrant 2017 desktop version Jupyter Notebook … Continue reading Business Intelligence, ETL and Data Science tools

Ansible script examples

EXAMPLE: Restart NTP daemon in Linux First create the playbook pbntprestart.yml as below and run using this command. The cdhservers is where you define all your hostnames. $ ansible-playbook pbntprestart.yml -u root -k The content of pbntprestart.yml  is below: $ cat pbntprestart.yml --- - hosts: cdhservers tasks: - name: NTPRESTART shell: service ntpd restart register: out1 … Continue reading Ansible script examples