Apache Gearpump is a real-time big data streaming engine, it is event/message based. We will need a running Gearpump service before we can submit and run first Gearpump application. There are multiple ways to run Gearpump – localhost mode, standalone mode, YARN mode and Docker mode. Obviously, the easiest way is to test run Apache Gearpump in local mode. Any Linux, MacOSX with Homebrew, or Windows 10 Ubuntu bash can be used with zero configuration. Here Are Steps on How to Install Apache Gearpump On localhost (or Cloud Server). Only for Mac OSX Homebrew, we have to use brew formulas – adjust them replacing with apt.
How to Install Apache Gearpump On localhost
We are taking that the machine has nothing installed. We have to install java, Scala and sbt. First install Java :
|
1 2 3 |
sudo add-apt-repository ppa:webupd8team/java sudo apt install oracle-java8-installer sudo apt install oracle-java8-set-default |
For Ubuntu or GNU/Linux, run these :
---
|
1 2 3 |
echo "export PATH=/usr/local/anaconda2/bin:$PATH" >> /etc/bash.bashrc echo "export JAVA_HOME=/usr/lib/jvm/java-8-oracle/" >> /etc/bash.bashrc echo "export PATH=$PATH:$JAVA_HOME/bin" >> /etc/bash.bashrc |
source that source file to reload or reboot (for server).
For Windows 10 bash if installed in our way using ZSH and MacOS X with ZSH, go to $HOME by typing cd ~ command, open .zshrc file and add :
|
1 2 3 |
export PATH=/usr/local/anaconda2/bin:$PATH export JAVA_HOME=/usr/lib/jvm/java-8-oracle/ export PATH=$PATH:$JAVA_HOME/bin |
Reload :
|
1 |
source .zshrc |
If you run :
|
1 |
javac -version |
There will be meaningful output. Install Scala :
|
1 2 3 |
sudo apt-get remove scala-library scala wget www.scala-lang.org/files/archive/scala-2.11.8.deb sudo dpkg -i scala-2.11.8.deb |
For Mac, install Scala with Homebrew as brew formula. Next is sbt :
|
1 2 3 4 |
echo "deb https://dl.bintray.com/sbt/debian /" | sudo tee -a /etc/apt/sources.list.d/sbt.list sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv 2EE0EA64E40A89B84B2DF73499E82A75642AC823 apt update apt install sbt |
We need to clone the Apache GearPump repository :
|
1 |
git clone https://github.com/apache/incubator-gearpump.git |
cd to that directory :
|
1 2 3 |
ls -al cd incubator-gearpump ls -al |
Then run :
|
1 |
sbt clean assembly packArchiveZip |
It will take good amount of time, with output like this :
At end of operation, you’ll get a zip archive, with name like gearpump-2.11-0.8.4.zip. Unzip that :
|
1 2 |
apt install unzip unzip gearpump-2.11-0.8.4.zip |
Now, cd to that directory and run la -al :
|
1 2 |
cd gearpump* ls -al |
There will be gear.conf file under conf. Open that file :
|
1 |
nano gear.conf |
Settings will be like this :
|
1 2 3 |
gearpump.hostname "127.0.0.1" gearpump.cluster.masters ["127.0.0.1:3000"] gearpump.worker.slots 1000 |
They have examples, which can be run in this way :
|
1 |
bin/gear app -jar examples/wordcount-2.11-0.8.4-assembly.jar org.apache.gearpump.streaming.examples.wordcount.WordCount |
Web UI services will be at http://127.0.0.1:8090 and user-name, password both are admin. That is written in details in their official website :
|
1 2 3 4 |
https://gearpump.apache.org/releases/latest/introduction/submit-your-1st-application/index.html https://gearpump.apache.org/releases/latest/deployment/deployment-standalone/ https://gearpump.apache.org/releases/latest/deployment/hardware-requirement/ https://gearpump.apache.org/releases/latest/introduction/commandline/ |
The software is good but installation guide to reach up to that is difficult to a newbie.
