Posts

Hadoop Cluster Tutorials

Image
  Hands-on lab on Hadoop Cluster What is a Hadoop Cluster? A Hadoop cluster is a collection of computers, known as nodes, that are networked together to perform parallel computations on big data sets. The Name node is the master node of the Hadoop Distributed File System (HDFS). It maintains the meta data of the files in the RAM for quick access. An actual Hadoop Cluster setup involves extensives resources which are not within the scope of this lab. In this lab, you will use dockerized hadoop to create a Hadoop Cluster which will have: Namenode Datanode Node Manager Resource manager Hadoop history server Objectives Run a dockerized Cluster Hadoop instance Create a file in the HDFS and view it on the GUI Set up Cluster Nodes Dockerized Hadoop Start online lab - https://labs.cognitiveclass.ai/login/lti Start a new terminal Clone the repository to your theia environment. git clone https: //gi thub.com /ibm-developer-skills-network/ ooxwv-docker_hadoop.git Navigate to the docker-h...
Post a Comment
Read more

Single Node Hadoop Tutorial

Image
Hands-on lab on Hadoop Map-Reduce Objectives Run a single-node Hadoop instance Perform a word count using Hadoop  Map Reduce . Set up Single-Node Hadoop The steps outlined in this lab use the single-node Hadoop Version 3.2.3.  Hadoop  is most useful when deployed in a fully distributed mode on a large cluster of networked servers sharing a large volume of data. However, for basic understanding, we will configure Hadoop on a single node. In this lab, we will run the WordCount example with an input text and see how the content of the input file is processed by WordCount. Start a new terminal Download hadoop-3.2.3.tar.gz to your theia environment by running the following command. curl https://dlcdn.apache.org/hadoop/common/hadoop- 3 . 2 . 3 /hadoop- 3 . 2 . 3 .tar.gz --output hadoop- 3 . 2 . 3 .tar.gz Extract the tar file in the currently directory. tar -xvf hadoop- 3 . 2 . 3 .tar.gz Navigate to the hadoop-3.2.3 directory. cd hadoop- 3 . 2 . 3 Check the hadoop command to ...
Post a Comment
Read more

Terraform - Input Variables

Installing Terraform What is Terraform and Why do we need it? Terraform Commands Hashicorp Configuration Language - HCL Tainting and Untainting Resources Terraform Console and Output Input variables serve as parameters for a Terraform file. A variable block configures a single input variable for a Terraform module. Each block declares a single variable. Syntax: variable [NAME] {   [OPTION] = "[VALUE]" } Arguments Within the block body (between { }) is configuration for the variable, which accepts the following arguments: type (Optional): If set, this defines the type of the variable. Valid values are string, list, and map. default (Optional): This sets a default value for the variable. If no default is provided, Terraform will raise an error if a value is not provided by the caller. description (Optional): A human-friendly description for the variable. Using variables during an apply: terraform apply -var 'foo=bar' Set up the environment: cd terraform/basics Edit main...
Post a Comment
Read more

Terraform Console and Output

Installing Terraform What is Terraform and Why do we need it? Terraform Commands Hashicorp Configuration Language - HCL Tainting and Untainting Resources We will use the Terraform Console to view various outputs that we can use for our scripts. The Terraform Console is extremely useful for troubleshooting and planning deployments. Terraform commands: console: Interactive console for Terraform interpolations Set up the environment: cd terraform/basics Working with the Terraform console Redeploy the Ghost image and container: terraform apply Show the Terraform resources: terraform show Start the Terraform console: terraform console Type the following in the console to get the container's name: docker_container.container_id.name Type the following in the console to get the container's IP: docker_container.container_id.ip_address Break out of the Terraform console by using Ctrl+C. Destroy the environment: terraform destroy Output the name and IP of the Ghost blog container Edit mai...
Post a Comment
Read more