We have noticed that there are some growing number of websites which write about Big Data, cloud computing and spread wrong information to sell some others paid things. They can not write guides like how to install Spark, Elastic Search, Kibana. Here is list of open source big data visualization tools which will convert data in to graphs, charts usable with tools like Spark, Hadoop. Do not use … [Read more...]

In a previously published article we introduced to multi cloud. In that article we gave example of managing SendGrid from WordPress dashboard using SendGrid WordPress plugin. Mesosphere is a software that expands the cluster management capabilities of Apache Mesos with additional components to manage server infrastructures. By combining several components with Mesos such as Marathon and Chronos, … [Read more...]

Apache Zeppelin is a web-based notebook for interactive data analytics. We already described how to install Apache Spark. If you have followed our any guide on big data tools installation, it is few minutes work to install and use Zeppelin. Zeppelin can be pre-built package or can be build from source. Here is how to install apache zeppelin on ubuntu 16.04 building from source.   What … [Read more...]

Normally log automatically should grow as big possible. But we use logrotate which does the opposite action as running cat and other tools on SSH over big files consumes memory, time and often difficult to do what we want. Also, old files are compressed. Here are the ways to join/merge multiple log files for big data analysis, store them to openstack based cloud storage and delete old … [Read more...]

The basic softwares to build own basic big data analytics system on own servers has been discussed on this website through multiple tutorials and we have a list of such big data tutorials for the beginners. Those guides are good to test for the developers including a webmaster on need for log analysis of web server. The running cost of few public servers with 16 GB+ RAM with modern processor is … [Read more...]

Nowadays it is common to come around phrases like Data Warehouse, Data Lake. Data warehouses is four decade old established concept. Data lake is a new idea. What is the difference between Data Warehouse and Data Lake? We talked about the concept about Data lake in a previous article. Explanation of Data Warehouse is needed. Database is the most common way of data storage for the digital data. … [Read more...]

Apache Kafka is a stream processing platform which aims to provide a low-latency platform for handling real-time data feeds. Its storage is a massively scalable pub/sub message queue architected as a distributed transaction log making it valuable to process streaming data. Kafka can connect to external systems for data import/export. Apache Kafka is a part of Big Data analysis too. Here are the … [Read more...]

Previously we were shown steps for installing Apache Spark, now suddenly why this article is sounding like anti-Spark? Apache Spark has problems including need of dependencies & integrity. That is why here is a list of Apache Spark alternatives to overcome integrity issues. At this moment Apache Spark is one step ahead of its competitors, due to some characteristics like implementation and … [Read more...]