Manado, Indonesia. 95252
(+62) 823-9602-9583
bayudwiyansatria@gmail.com

Category: Blog

Software Engineer | DevOps Engineer

web-crawling

Best Java Web Crawler Implementation

Implementing a Java web crawler is a fun and challenging task often given in university programming classes. You may also actually need a Java web crawler in your own applications from time to time. You can also learn a lot about Java networking and multi-threading while implementing a Java web crawler. This tutorial will go through the challenges and design decisions you face when implementing a Java web crawler.

spark-vs-hadoop

What is differences about Apache Hadoop vs Apache Spark

What is Big Data? What size of Data is considered to be big and will be termed as Big Data? We have many relative assumptions for the term Big Data. It is possible that, the amount of data say 50 terabytes can be considered as Big Data for Startup’s but it may not be Big Data for the companies like Google and Facebook. It is because they have infrastructure to store and process this vast amount of data. Apache Hadoop and Apache Spark are both Big Data analytics frameworks they provide some of the most popular tools used to carry out common Big Data-related tasks.

hadoop

Apache Hadoop Introduction

The Apache Hadoop software library is a framework that allows for the distributed processing of large data sets across clusters of computers using simple programming models. It is designed to scale up from single servers to thousands of machines, each offering local computation and storage.

Rather than rely on hardware to deliver high-availability, the library itself is designed to detect and handle failures at the application layer. So delivering a highly-available service on top of a cluster of computers, each of which may be prone to failures.

Featured

Apache Spark Parallel Program Flows

Apache Spark Flows – Apache Spark consists of several purpose-built components as we have discuss at the introduction of apache spark. Let’s see what a typical Spark program looks like. Imagine that a 300 MB log file is stored in a three-node HDFS cluster. Hadoop File System (HDFS) automatically splits the file into 128 MB parts and places each part on a separate node of the cluster.

apache_spark_component

Apache Spark Component Parallel Processing

Apache Spark consists of several purpose-built components as we have discuss at the introduction of apache spark. Apache spark provides high-level APIs in Java, Scala, Python and R, and an optimized engine that supports general execution graphs. It also supports a rich set of higher-level tools. These are Apache Spark Component : Spark Core, Spark SQL, Spark Streaming, Spark GraphX, Spark MLlib.

These components make Spark a feature-packed unifying platform: it can be used for many tasks that previously had to be accomplished with several different frameworks. A brief description of each Apache Spark component follows.

Featured

Apache Spark Parallel Processing Introduction

Apache Spark is usually defined as a fast, general-purpose, distributed computing platform. Apache spark provides high-level APIs in Java, Scala, Python and R, and an optimized engine that supports general execution graphs. It also supports a rich set of higher-level tools including Spark SQL for SQL and structured data processing, MLlib for machine learning, GraphX for graph processing, and Spark Streaming.

Featured

Things You Must Know Before Choose GCP, AWS or Azure

Shifting to Cloud Computing: Why Do It at All? Things GCP AWS Azure you must know! Cloud Computing has taken the world by storm and there are some significant reasons behind it. Everyone remembers the days when every bit of office work had to be documented in triplicate. Not only did it waste money, it also …

Featured

Why You Should Have Virtual Private Network?

Introduction A virtual private network (VPN) extends a private network across a public network, and enables users to send and receive data across shared or public networks as if their computing devices were directly connected to the private network. Applications running across a VPN may therefore benefit from the functionality, security, and management of the private network. VPN technology …

Featured

Google AdSense For Monetize Money Online

Best Practices for Google AdSense To make money with Google AdSense, it’s important that you follow these Google AdSense best practices. They will not only ensure that you continue to increase your Google AdSense revenue, but they will also help you ensure that your site itself continues to grow and flourish. Experiment with different ad …

Featured

Google Always Free Tier

Introduction The Google Cloud Platform Free Tier gives you free resources to learn about Google Cloud Platform (GCP) services by trying them on your own. Whether you’re completely new to the platform and need to learn the basics, or you’re an established customer and want to experiment with new solutions, the Google Cloud Platform Free …