Meta#Grid

A blog about meta data, BI and data warehousing

Uncategorized

Install meta#grid on Ubuntu 20.04

⚠️ This is an old post. While keeping this page online as an archive, please look at the updated instructions at https://docs.meta-grid.com.


This article describes how to setup meta#grid on Ubuntu 20.04. The steps below will guide you through all necessary steps.

Depending on your internet connection and system power this will take 10 to 15 minutes.

Installation needed software basics

Apache and PHP

This is needed for the web GUI framework Yii2

sudo apt install software-properties-common
sudo add-apt-repository ppa:ondrej/php

sudo apt-get update

sudo apt install libapache2-mod-php7.1 php7.1-cli php7.1-common php7.1-curl php7.1-gd php7.1-json php7.1-mbstring php7.1-mcrypt php7.1-mysql php7.1-opcache php7.1-readline php7.1-sqlite3 php7.1-xml php7.1-xmlrpc php7.1-xsl php7.1-zip

Java

This is needed for LiquiBase and Pentaho Data Integration (Kettle/Bulk Loader)

sudo apt-get install openjdk-8-jre

Python and SQLite3

This is needed for “Helper Tools” and administrative changes on the database

sudo apt install python2
sudo apt install sqlite3

Check installed components

$ php -v
PHP 7.1.33-24+ubuntu20.04.1+deb.sury.org+1 (cli) (built: Oct 31 2020 16:56:22) ( NTS )
Copyright (c) 1997-2018 The PHP Group
Zend Engine v3.1.0, Copyright (c) 1998-2018 Zend Technologies
with Zend OPcache v7.1.33-24+ubuntu20.04.1+deb.sury.org+1, Copyright (c) 1999-2018, by Zend Technologies

$ java -version
openjdk version "1.8.0_275"
OpenJDK Runtime Environment (build 1.8.0_275-8u275-b01-0ubuntu1~20.04-b01)
OpenJDK 64-Bit Server VM (build 25.275-b01, mixed mode)

$ python2 -V
Python 2.7.18

Test PHP

sudo touch /var/www/html/phpinfo.php
sudo chown $USER:$USER /var/www/html/phpinfo.php
echo "<?php phpinfo(); ?>" > /var/www/html/phpinfo.php
sudo chown www-data:www-data /var/www/html/phpinfo.php

http://localhost/phpinfo.php

Create directories

sudo mkdir /opt/meta_grid
sudo mkdir /opt/meta_grid/db
sudo mkdir /opt/meta_grid/frontend
sudo mkdir /opt/meta_grid/bulkimport

3rd party components

LiquiBase

cd /tmp
wget https://github.com/liquibase/liquibase/releases/download/v3.8.9/liquibase-3.8.9.zip
unzip liquibase-3.8.9.zip -d liquibase
sudo mkdir /opt
sudo mv liquibase /opt
rm liquibase-3.8.9.zip

JDBC SQLite

cd /tmp
wget https://github.com/xerial/sqlite-jdbc/releases/download/3.32.3.2/sqlite-jdbc-3.32.3.2.jar
sudo mv sqlite-jdbc-3.32.3.2.jar /opt/liquibase/lib

Pentaho Data Integration (Kettle)

cd /tmp
wget https://deac-ams.dl.sourceforge.net/project/pentaho/Pentaho%208.1/client-tools/pdi-ce-8.1.0.0-365.zip
unzip pdi-ce-8.1.0.0-365.zip
sudo mv data-integration /opt
rm pdi-ce-8.1.0.0-365.zip

PDI JDBC Metadata Plugin

cd /tmp
wget https://raw.githubusercontent.com/rpbouman/pentaho-pdi-plugin-jdbc-metadata/master/dist/steps/pentaho-pdi-plugin-jdbc-metadata/pentaho-pdi-plugin-jdbc-metadata.zip
unzip pentaho-pdi-plugin-jdbc-metadata.zip
sudo mv pentaho-pdi-plugin-jdbc-metadata /opt/data-integration/plugins/steps
rm pentaho-pdi-plugin-jdbc-metadata.zip

Download components for Meta#Grid

Meta#Grid Updater Tool

cd /tmp
wget https://github.com/patschwork/meta_grid_install_update/releases/download/1.2/meta_grid_updater.zip
unzip meta_grid_updater.zip -d meta_grid_updater
sudo mv meta_grid_updater /opt
rm meta_grid_updater.zip

Initial database

You must choose between a database with demo data or without (empty structures).

The demo database is good to get a glimpse on how to organize things in Meta#Grid. If you are new, you should choose this option.


cd /tmp
wget https://raw.githubusercontent.com/patschwork/meta_grid/master/db/dwh_meta_demo.sqlite
wget https://raw.githubusercontent.com/patschwork/meta_grid/master/db/dwh_meta_clean.sqlite
mv dwh_meta_demo.sqlite /opt/meta_grid/db/dwh_meta.sqlite

# mv dwh_meta_clean.sqlite /opt/meta_grid/db/dwh_meta.sqlite

Bulk loader files (Kettle files)

cd /tmp
wget https://raw.githubusercontent.com/patschwork/meta_grid/master/bulk_import/kettle/run_import.kjb
mv run_import.kjb /opt/meta_grid/bulkimport

Change directory permissions

sudo chown $USER:$USER /opt/meta_grid -R
sudo chown $USER:$USER /opt/meta_grid_updater -R

Preparation

This is needed to make the life easier 😉

touch /opt/meta_grid/frontend/gii_crud.sh

Install Meta#Grid


cd /opt/meta_grid_updater/
python2 ./meta-grid_install_or_update.py

You will be asked some questions. All prerequisites shoud be fine now.

URL for GitHub zip release file [https://github.com/patschwork/meta_grid/archive/master.zip]: —> Enter [RETURN]
File location for the SQLite database file [/opt/meta_grid/db/dwh_meta.sqlite] {To search for the file: dwh_meta.sqlite please type F}: —> Use the bold path or use the search with [F]
Folder location for the frontend files [/opt/meta_grid/frontend] (Inital value: ../../../frontend/yii/basic) {To search for the file: gii_crud.sh please type F}: —> Use the bold path or use the search with [F]
Folder location for the bulkimport files [/opt/meta_grid/bulkimport] (Inital value: ../../../bulk_import/kettle) {To search for the file: run_import.kjb please type F}: —> Use the bold path or use the search with [F]
Path to LiquiBase installation [/opt/liquibase/liquibase] (Inital value: /opt/meta_grid/tools/liquibase/liquibase) {To search for the file: liquibase please type F}: —> Use the bold path or use the search with [F]
Path to Python executable [python2] (Inital value: python): —> Use the bold command executable
Path to Pentaho Data Integration (kitchen.sh) executable [/opt/data-integration/kitchen.sh] {To search for the file: kitchen.sh please type F}: —> Use the bold path or use the search with [F]

If everything goes fine there will be the message “# End of process #” at the end.

Finalization

cd /tmp
wget https://raw.githubusercontent.com/patschwork/meta_grid/master/meta_grid/config/console.php
wget https://raw.githubusercontent.com/patschwork/meta_grid/master/meta_grid/config/db.php
wget https://raw.githubusercontent.com/patschwork/meta_grid/master/meta_grid/config/params.php
wget https://raw.githubusercontent.com/patschwork/meta_grid/master/meta_grid/config/web.php


wget https://raw.githubusercontent.com/patschwork/meta_grid/master/database_model/liquibase/db.changelog-master.xml
mkdir /opt/meta_grid_updater/tmp
mv db.changelog-master.xml /opt/meta_grid_updater/tmp

sqlite3 /opt/meta_grid/db/dwh_meta.sqlite "UPDATE app_config SET valueSTRING='/opt/meta_grid_updater/tmp/db.changelog-master.xml' WHERE key='liquibase_changelog_master_filepath';"


mkdir /opt/meta_grid/frontend/config
mv console.php /opt/meta_grid/frontend/config
mv db.php /opt/meta_grid/frontend/config
mv params.php /opt/meta_grid/frontend/config
mv web.php /opt/meta_grid/frontend/config

mkdir /opt/meta_grid/frontend/web/assets

Change database config for frontend



sed -i 's|sqlite:../../../../dwh_meta.sqlite|sqlite:../../db/dwh_meta.sqlite|g' /opt/meta_grid/frontend/config/db.php

Change file permissions for Apache


sudo chown www-data:www-data /opt/meta_grid/db/dwh_meta.sqlite

sudo chown www-data:www-data /opt/meta_grid/frontend -R

Set CookieValidationKey



export rand=$(head /dev/urandom | tr -dc A-Za-z0-9 | head -c20)
sed -i "s|'cookieValidationKey' => 'xxx'|'cookieValidationKey' => '$rand'|g" /opt/meta_grid/frontend/config/web.php

Set a site for Apache webserver


sudo a2dissite 000-default.conf


export meta_grid_conf_file_tmp="/tmp/meta_grid.conf"
export meta_grid_conf_file="/etc/apache2/sites-available/meta_grid.conf"
echo " <VirtualHost *:80>" > $meta_grid_conf_file_tmp
echo "ServerAdmin webmaster@localhost" >> $meta_grid_conf_file_tmp
echo "" >> $meta_grid_conf_file_tmp
echo "ServerAlias meta_grid" >> $meta_grid_conf_file_tmp
echo "Alias /meta_grid "/opt/meta_grid/frontend/web/"" >> $meta_grid_conf_file_tmp
echo "" >> $meta_grid_conf_file_tmp
echo "DocumentRoot /opt/meta_grid/frontend/web" >> $meta_grid_conf_file_tmp
echo "<Directory /opt/meta_grid/frontend/web>" >> $meta_grid_conf_file_tmp
echo "Options -Indexes +FollowSymLinks +MultiViews" >> $meta_grid_conf_file_tmp
echo "AllowOverride All" >> $meta_grid_conf_file_tmp
echo "Require all granted" >> $meta_grid_conf_file_tmp
echo "</Directory>" >> $meta_grid_conf_file_tmp
echo "" >> $meta_grid_conf_file_tmp
echo "LogLevel warn" >> $meta_grid_conf_file_tmp
echo "ErrorLog ${APACHE_LOG_DIR}/meta_grid_error.log" >> $meta_grid_conf_file_tmp
echo "CustomLog ${APACHE_LOG_DIR}/meta_grid_access.log combined" >> $meta_grid_conf_file_tmp
echo "" >> $meta_grid_conf_file_tmp
echo "</VirtualHost>" >> $meta_grid_conf_file_tmp
sudo mv $meta_grid_conf_file_tmp $meta_grid_conf_file

sudo a2ensite meta_grid

sudo service apache2 restart

 

Meta#Grid is now accessable with http://localhost/meta_grid/

Default login values

User = admin
Password = 08GbDEfMhAW4TZywsr59