We live in on-demand, on-command Digital universe with data prolife ringby Institutions, Individuals and Machines at a very high rate. This data is categories as “Big Data” due to its sheer Volume, Variety, Velocity and Veracit.
Approximately 90 % of the world’s data has been generated in the last two years alone. In addition, 80 % of the data is unstructured or available
in a broad variety of frameworks that are difficult to interpret.Having such a vast volume of data is a major challenge and a more an important challenge arises from the fact that this data is not of a structured format.It includes images, line streaming records, videos, sensor records, GPS tracking information. In short, this is unstructured data. …
Hi there! Today we are going to see some of the internals of Hadoop and how it actually works behind the scenes.So let’s get right into it.
The Apache Hadoop software library is a framework that allows for the distributed processing of large data sets across clusters of computers using simple programming models. It is designed to scale up from single servers to thousands of machines, each offering local computation and storage. …
Hi there! Today we are going to see how we can resize the size of a Linux partition without loosing data i.e. persistence.
This is rather easy if you already know how to create partitions in Linux.Now then let’s get started.
First of all let me make new partition of size 5Gib using fdisk utility.
Hi there! Today we are going to see some great concepts in docker while learning to setup simple web server on Docker.So let’s jump into it.
Docker is a set of platform as a service products that use OS-level virtualization to deliver software in packages called containers. Containers are isolated from one another and bundle their own software, libraries and configuration files; they can communicate with each other through well-defined channels
Developing apps today requires so much more than writing code. Multiple languages, frameworks, architectures, and discontinuous interfaces between tools for each life cycle stage creates enormous complexity. …
As we are getting more and more dependent on technology, we also expose ourselves to all kinds of online threats.It has now become necessary to adopt countermeasures against cyber-attacks that are becoming more sophisticated as the years pass. Since the first confirmed attack in 2002, targeted attacks have inflicted significant damage in a number of countries around the world. In 2013, a new threat arose with the proliferation of ransomware. Attack methods have become more varied and are now specifically individualized according to targets, while the sophistication of incubation and evidence removal continues to evolve.
To combat these attacks, artificial intelligence (AI) technology is now being utilized for automatic detection of the existence and behavior of malware and is becoming better able to perform this task as it evolves.AI technology that uses machine learning is able to detect malware even subspecies and unknown ones more effectively than conventional methods based on signatures and rules and is now expected to play an important role in protecting against unpredictable and ever more varied attack. …
Hi there ! So today we are going to see how we can contribute limited/specific amount of storage from a datanode to hadoop namenode.This is pretty simple task if you know how to make partitions.
So lets get into it.I used :
So lets see how its done first i added and 12 Gib addition hardisk to my local VM and then i created partition of 2 Gib in it and formated it and mounted to a folder.
Today we are going to see some basic AWS CLI and how to setup and run commands using CLI.
The AWS Command Line Interface (CLI) is a unified tool to manage your AWS services. With just one tool to download and configure, you can control multiple AWS services from the command line and automate them through scripts.
First step is to download the CLIV2 from aws and just install it simply.Then we need to make an IAM with proper access.
Here how to create IAM :
Ansible is an open-source automation tool, or platform, used for IT tasks such as configuration management, application deployment, intra service orchestration, and provisioning.
Today we are going to create load balancer + web servers infrastructure with the help of ansible. We will use the AWS cloud to provision our load balancer and webservers in ec2.
First, we write a role for launching webserver and load balancer.
Ansible is an open-source automation tool, or platform, used for IT tasks such as configuration management, application deployment, intraservice orchestration, and provisioning.
Here we are going to provision aws ec2 instance and configure it as a webserver using the concept of dynamic inventory and roles.
Roles provide a framework for fully independent, or interdependent collections of variables, tasks, files, templates, and modules.
In Ansible, the role is the primary mechanism for breaking a playbook into multiple files. This simplifies writing complex playbooks, and it makes them easier to reuse. The breaking of playbook allows you to logically break the playbook into reusable components.This …