Installing

1. Obtain owl-packages.tar.gz file

2. Extract the contents to a folder (this will become the -owlpackage argument).

3. Optional: See Appendix about encrypting the partition using LUKS prior to installing Owl.

4. chmod the setup.sh script to allow its execution.

5. The setup.sh script is the script that will drive the placement of the packages and has some arguments that can be passed to it:

Setup.sh arguments

-non-interactive skip asking to accept JAVA license agreement

-skipSpark skips the extraction of spark components

-stop do not automatically start up all components (orient,owl-web,zeppelin,postgres)

-port= set owlweb application to use defined port

-user= Optional Parameter (default will be current user) set the user to run owl as.

-owlbase= set base path to where you want owl installed

-owlpackage= Optional Parameter (default is current working directory) set owl package directory

-help display this help and exit

-options= the different owl components to install (comma separated list) --- owlagent,owlweb,zeppelin,postgres,orient,spark

-pgpassword= password to use to set for the postgres metastore (unattended install)

-pgserver= name of the postgres server example = owl-postgres-host.example.com:5432/owldb (unattended install)

-opassword= password for the orient graph DB (unattended install)

Example:

• The Owl tar ball has been extracted to this folder on my EC2 Instance: /home/ec2-user/packages/

• Owl will be running as the ec2-user

• The owl-web application will run on port 9000

• The base location for the setup.sh script to create the owl folder and place all content under owl will be: /home/ec2-user/

./setup.sh -port=9000 -user=ec2-user -owlbase=/home/ec2-user -owlpackage=/home/ec2-user/package

Example installing just the agent (perhaps on an Edge node of a hadoop cluster):

• The Package has been extracted to this folder on my EC2 Instance: /home/ec2-user/packages/

• Owl-agent will be running as the ec2-user

• The base location for the setup.sh script to create the owl folder and place all packages under owl will be: /home/ec2-user/

./setup.sh -user=ec2-user -owlbase=/home/ec2-user -owlpackage=/home/ec2-user/package -options=owlagent

When installing different features questions will be asked

• postgres = Postgres DBPassword needs to be supplied

• orient = Orient DBPassword needs to be supplied

• If postgres is not being installed (such as agent install only) postgres metastore server name needs to be supplied

Launching and Administering owl:

When the setup.sh script finishes by default software is automatically started. The setup.sh also creates the owlmanage.sh script which allows for stopping and starting of all owl services or some components of services.

The setup script will also generate an owl-env.sh script that will hold the main variables that are reused across components (see owl-env.sh under the config directory).

Owl Directory Structure after running Setup.sh

export ORIENTDB_HOST="localhost"

Configuration / ENV settings within owl-env.sh

Contents of the Owl-env.sh script and what is is used for.

OWL-ENV.SH Scripts	Meaning
export ORIENTDB_PORT="2424"	Port that OrientDB is using
export SPARK_CONF_DIR="/home/danielrice/owl/cdh-spark-conf"	directory on machine where the spark conf directory resides.
export INSTALL_PATH="/home/danielrice/owl"	Installation directory of Owl
export JAVA_HOME="/home/danielrice/jdk1.8.0_131"	Java Home for Owl to leverage
export LOG_PATH="/home/danielrice/owl/log"	Log path
export BASE_PATH="/home/danielrice"	The base location under which the owl director resides
export SPARK_MAJOR_VERSION=2	Spark Major version. Owl only supports 2+ version of spark
export OWL_LIBS="/home/danielrice/owl/libs"	Lib Directory to inject in spark -submit jobs
export USE_LIBS=0 #1 is on, 0 is off	use the lib directory or not. 0 is the default.
export SPARKSUBMITEXE="spark-submit"	Spark submit executable command. CDH using spark2-submit as example
export ext_pass_manage=0 #0 to disable 1 to enable	If using a password management system. You can enable for password to be pulled from it.
export ext_pass_script="/opt/owl/bin/getpassword.sh"	Leverage password script to execute a get password script from the vault.
TIMEOUT=900 #15 minutes in seconds	Owl-Web user time out limits
PORT=9003 #owl-web port NUMBER	default port to use for owl-web
SPRING_LOG_LEVEL=ERROR	logging level to be displayed in the owl-web.log
SPRING_DATASOURCE_DRIVER_CLASS_NAME=org.postgresql.Driver	driver class name for postgres metastore (used by web)
export SPRING_DATASOURCE_URL=jdbc:postgresql://localhost:5432/postgres	JDBC connection string to Owl Postgres metastore
export SPRING_DATASOURCE_USERNAME=danielrice	Owl Postgres username
export SPRING_DATASOURCE_PASSWORD=3+017wfY1l1vmsvGYAyUcw5zGL	Owl Postgres password
export AUTOCLEAN=TRUE/FALSE	TRUE/FALSE Enable/Disable automatically delete old datasets
export DATASETS_PER_ROW=200000	Delete datasets after this threshold is hit (must be greater than the default to change)
export ROW_COUNT_THRESHOLD=300000	Delete rows after this threshold is hit (must be greater than the default to change)
export SERVER_HTTP_ENABLED=true	Enabling HTTP to owl web
export OWL_ENC=OFF #JAVA for java encryption	Enable Encryption (NOTE need to add to owl.properties also). Has to be in form owl.enc=OFF within owl.properties file in this form owl.enc=JAVA.
PGDBPATH=/home/danielrice/owl/owl-postgres/bin/data	Path for Postgres DB
export RUNS_THRESHOLD=5000	Delete runs after this threshold is hit (must be greater than the default to change)
export HTTP_SECONDARY_PORT=9001	Secondary HTTP port to use which is useful when SSL is enabled
export SERVER_PORT=9000	same as PORT
export SERVER_HTTPS_ENABLED=true	enabling of SSL
export SERVER_SSL_KEY_TYPE=PKCS12	certificate trust store
export SERVER_SSL_KEY_PASS=t2lMFWEHsQha3QaWnNaR8ALaFPH15Mh9	certificate key password
export SERVER_SSL_KEY_ALIAS=owl	certificate key alias
export SERVER_REQUIRE_SSL=true	Override HTTP on and force HTTPS regardless of HTTP settings
export MULTITENANTMODE=FALSE	Flipping to TRUE will enable multi tenant support
export multiTenantSchemaHub=owlhub	schema name used for multi tenancy
export OWL_SPARKLOG_ENABLE=false	Enabling deeper spark logs when toggled to true

Configuration / owl.properties file

Example	Meaning
key=XXXXXX	The license key
spring.datasource.url=jdbc:postgresql://localhost:5432/postgres	connection string to owl metastore (used by owl-core)
spring.datasource.password=xxxxxx	password to owl metastore (used by owl-core)
spring.datasource.username=xxxxxx	username to owl metastore (used by owl-core)
spring.datasource.driver-class-name=com.owl.org.postgresql.Driver	shaded postgres driver class name
orientdb.user=root	orient username
orientdb.pass=xxxxx	orient password
spring.agent.datasource.url	jdbc:postgresql://10.142.0.19:5432/owltrunk
spring.agent.datasource.username	owluser
spring.agent.datasource.password	owlpassword
spring.agent.datasource.driver-class-name	org.postgresql.Driver