Apache Druid is a column oriented, analytics database which can be used to build a data warehouse. Druid has APIs (via HTTP and JDBC) for loading, managing, and querying data and the user can also interact with Druid via the console. A mere 4GB RAM of an instance or computer is sufficient to test-run nano-quickstart configuration of Druid, and 16GB RAM instance can run it at production. Druid can be used for clickstreams, view streams, and activity streams, for measuring user engagement, tracking A/B test data, understanding usage patterns, compute impressions, clicks, eCPM, and key conversion metrics. Other use cases of Druid includes business intelligence (OLAPOLAP), Internet of Things.
Druid is intended to run as a cluster of nodes to support a fault-tolerant architecture. The cluster will include external dependencies for Apache ZooKeeper (coordination), MySQL or PostgreSQL (metadata storage), and HDFS, or Amazon S3 for deep storage. Using Ambari, you configure Apache Druid (incubating) by setting parameters in a graphical user interface. Druid can be downloaded (and details read) from :
---
|
1 2 3 4 |
# https://druid.apache.org/ https://github.com/apache/incubator-druid # |
This is the link to 0.16.0 :
|
1 2 3 |
# https://www.apache.org/dyn/closer.cgi?path=/incubator/druid/0.16.0-incubating/apache-druid-0.16.0-incubating-bin.tar.gz # |
We have to extract and navigate to the main directory :
|
1 2 3 4 |
# tar -xzf apache-druid-0.16.0-incubating-bin.tar.gz cd apache-druid-0.16.0-incubating # |
The directories contain the below :
bin/* – scripts for the quickstarts
conf/* – example configurations for single-server, clustered setup
extensions/* – core Druid extensions
hadoop-dependencies/* – Druid Hadoop dependencies
lib/* – libraries and dependencies for core Druid
quickstart/* – configuration files, sample data, and files for the quickstart tutorials
Apache Zookeeper is a dependency of Apache Druid :
|
1 2 3 |
curl https://archive.apache.org/dist/zookeeper/zookeeper-3.4.14/zookeeper-3.4.14.tar.gz -o zookeeper-3.4.14.tar.gz tar -xzf zookeeper-3.4.14.tar.gz mv zookeeper-3.4.14 zk |
The contents of the Zookeeper to be located at zk under the apache-druid-0.16.0-incubating package root. It is too easy to start Apache Druid :
|
1 2 3 |
# ./bin/start-micro-quickstart # |
Druid UI runs on http://localhost:8888/. It is important to read the tutorials on Druid to correctly configure depending on your need :
|
1 2 3 |
# https://druid.apache.org/docs/latest/tutorials/ # |
Druid needs a Relational DB as a Metadata Storage, i.e. MySQL. We have different guides on installing MySQLinstalling MySQL.
Start a CLI session: mysql -u root -p. Inside the session issue the following commands :
|
1 2 3 4 5 6 7 8 9 |
CREATE DATABASE druid DEFAULT CHARACTER SET utf8; CREATE USER 'druid'@'%' IDENTIFIED BY 'druid-pass'; GRANT ALL PRIVILEGES ON *.* TO 'druid'@'%' WITH GRANT OPTION; CREATE USER 'druid'@'localhost' IDENTIFIED BY 'druid-pass'; GRANT ALL PRIVILEGES ON *.* TO 'druid'@'localhost' WITH GRANT OPTION; flush privileges; |
This is the minimum common settings for MySQL :
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 |
# Extensions (no deep storage model is listed - using local fs for deep storage - not recommended for production) druid.extensions.coordinates=["io.druid.extensions:druid-examples","io.druid.extensions:druid-kafka-eight","io.druid.extensions:mysql-metadata-storage"] # Zookeeper druid.zk.service.host=localhost # Metadata Storage (mysql) druid.metadata.storage.type=mysql druid.metadata.storage.connector.connectURI=jdbc\:mysql\://localhost\:3306/druid druid.metadata.storage.connector.user=druid druid.metadata.storage.connector.password=diurd # Deep storage (local filesystem for examples - don't use this in production) druid.storage.type=local druid.storage.storageDirectory=/Users/hbutani/druid/localStorage # Query Cache (we use a simple 10mb heap-based local cache on the broker) druid.cache.type=local druid.cache.sizeInBytes=10000000 # Indexing service discovery druid.selectors.indexing.serviceName=overlord # Monitoring (disabled for examples, if you enable SysMonitor, make sure to include sigar jar in your cp) # druid.monitoring.monitors=["com.metamx.metrics.SysMonitor","com.metamx.metrics.JvmMonitor"] # Metrics logging (disabled for examples - change this to logging or http in production) druid.emitter=noop |
That ends this guide.
About Abhishek Ghosh
Here’s what we’ve got for you which might like :
Additionally, performing a search on this website can help you. Also, we have YouTube Videos.
Take The Conversation Further ...
We'd love to know your thoughts on this article.
Meet the Author over on Twitter to join the conversation right now!
If you want to Advertise on our Article or want a Sponsored Article, you are invited to Contact us.
Contact Us