Bart Simons

Pretty URLs in Apache - Getting Started Tutorial

 •  Filed under getting started, tutorial, Pretty URLs, Apache, mod_rewrite

Ever wanted to create beautiful URLs for your self-hosted websites? This page has got you covered with all just about the information you need to get started. All functionality which is demonstrated on this page is made possible by mod_rewrite, also known as the 'rewrite' module. This module gives system administrators the possibility to rewrite URL paths, as I shall demonstrate:

Let's say Mr. X has a website called mywebsite.corp with a single page called test.php. He currently visits his page like this:

mywebsite.corp before any modification

Mr.X is not happy with how the php file extension is exposed to the user, and wants to make his page URL look better by removing the php extension. The first thing he has to do is to enable the rewrite module which is fairly simple:

a2enmod rewrite  

Now, all that is left to do is to write the rewrite rule inside a Directory tag in apache.conf like this:

<Directory /var/www/html>  
        Options Indexes FollowSymLinks
        AllowOverride None
        Require all granted
        RewriteEngine On
        RewriteRule ^test$ /test.php

Restart your web server with service apache2 restart and let's see if things work the way we want:

mywebsite.corp after modification

It works! Now, let's dive deeper into what the RewriteRule line does

^ is the character to indicate the root URL of your website
$ is the character to indicate the end of the URL to match

This has been a simple demonstration of what you can do and accomplish with mod_rewrite. However, you can do so much more with this module. Let's take a look at regular expressions and how to use with rewrite rules:

mywebsite.corp before second modification

With this directory block:

<Directory /var/www/html>  
        Options Indexes FollowSymLinks
        AllowOverride None
        Require all granted
        RewriteEngine On
        RewriteRule ^users/(\d+)$ /users.php?id=$1

We are able to create an URL rewrite that functions like this:

mywebsite.corp after second modification

As you can see, you can use capture groups to capture URL parameters which you can use like in the previous example. Pretty handy!

This is just a handful of possibilities, I hope that you find 'em useful. Thanks for reading!

Mirrored object storage with Minio

 •  Filed under mirrored, distributed, object storage, minio

Object storage is on the rise - and that's not without a reason. When you have to deal with large pieces of data, it is very necessary to keep your data organised. Minio offers a perfectly fitted solution that is open source and well documented: it's an Amazon S3 compatible object storage server written in Go. Want to learn more about the project? This is their homepage.

Setting up a single node is a piece of cake, as I shall demonstrate.

# Create a directory for storing 'objects'
mkdir ~/ObjectStorage

# Download and prepare the Minio mode
chmod +x minio  
export MINIO_ACCESS_KEY=yourusernamegoeshere  
export MINIO_SECRET_KEY=yourpasswordgoeshere

# Run the Minio node
./minio server ~/ObjectStorage

It's that easy! You can now access your Minio node at port 9000, try it out in your browser.

Running a clustered of two nodes is also very easy. Let's say that we have two servers in a subnet:


We also have two disks on each server, so that means we also have the following folders mounted to their own disk:

/opt/storage01: disk 1
/opt/storage02: disk 2

Run the following command on both servers to initialise the cluster:

MINIO_ACCESS_KEY=yourusernamegoeshere MINIO_SECRET_KEY=yourpasswordgoeshere ./minio server  

It automatically detects if a server is up or down, and automatically syncs data between nodes.

Compiling a LAMP stack for CentOS - Tutorial

 •  Filed under compiling, compile, LAMP, stack, CentOS

Getting the latest version of Apache, MySQL and PHP up and running on enterprise Linux operating systems can be a tough job. There are easy and quick ways to get around this, number 1 being the use of third-party repositories: but do you really want to use open-source software from third parties that might have been fiddled around with? I know, I can be a little bit paranoid about this. But, there is a second option: compiling the LAMP stack from source. This allows you to get control over:

  • The software versions used in the stack
  • The code that you compile (being able to review the code)
  • Enable and disable functionality that you (don't) need

Compiling the LAMP stack on CentOS from source is not that hard anymore, because I have written a script that automates the build process for you.

My script fetches all the source code that is needed to compile the LAMP stack and then compiles every dependency and/or program one by one. Afterwards, all the software gets installed in the /opt folder (you can always change this if you want).

So - without further ado - here's the script:


TIMEFORMAT='It took %R seconds to complete this process.'  
time {  
    # Defining variables

    # Build local yum cache and install development tools
    echo "[1/16] Installing the 'Developer Tools' yum group.."
    yum -y makecache >/dev/null 2>&1 && yum -y groupinstall "Development Tools" >/dev/null 2>&1

    # Install other necessary binaries
    echo "[2/16] Installing necessary binaries.."
    yum -y install curl cmake >/dev/null 2>&1

    # Download source tarballs
    echo "[3/16] Downloading source tarballs.."
    curl -o libxml2.tar.gz >/dev/null 2>&1
    curl -o openssl.tar.gz >/dev/null 2>&1
    curl -o zlib.tar.gz >/dev/null 2>&1
    curl -o httpd.tar.gz >/dev/null 2>&1
    curl -o pcre.tar.gz >/dev/null 2>&1
    curl -o apr.tar.gz >/dev/null 2>&1
    curl -o apr-util.tar.gz >/dev/null 2>&1
    curl -o ncurses.tar.gz >/dev/null 2>&1
    curl -o mysql.tar.gz -L >/dev/null 2>&1

    # Make source directories
    echo "[4/16] Creating source+build directories.."
    mkdir libxml2
    mkdir openssl
    mkdir zlib
    mkdir httpd
    mkdir pcre
    tar -xvzf httpd.tar.gz -C httpd --strip 1 >/dev/null 2>&1 && rm httpd.tar.gz
    mkdir httpd/srclib/apr
    mkdir httpd/srclib/apr-util
    mkdir ncurses
    mkdir mysql

    # Clone PHP sources
    echo "[5/16] Cloning PHP 7.1.5 from Github.."
    git clone -b php-7.1.5 php7 >/dev/null 2>&1

    # Extract source packages
    echo "[6/16] Extracting source tarballs.."
    tar -xvzf libxml2.tar.gz -C libxml2 --strip 1 >/dev/null 2>&1 && rm libxml2.tar.gz
    tar -xvzf openssl.tar.gz -C openssl --strip 1 >/dev/null 2>&1 && rm openssl.tar.gz
    tar -xvzf zlib.tar.gz -C zlib --strip 1 >/dev/null 2>&1 && rm zlib.tar.gz
    tar -xvzf pcre.tar.gz -C pcre --strip 1 >/dev/null 2>&1 && rm pcre.tar.gz
    tar -xvzf apr.tar.gz -C httpd/srclib/apr --strip 1 >/dev/null 2>&1 && rm apr.tar.gz
    tar -xvzf apr-util.tar.gz -C httpd/srclib/apr-util --strip 1 >/dev/null 2>&1 && rm apr-util.tar.gz
    tar -xvzf ncurses.tar.gz -C ncurses --strip 1 >/dev/null 2>&1 && rm ncurses.tar.gz
    tar -xvzf mysql.tar.gz -C mysql --strip 1 >/dev/null 2>&1 && rm mysql.tar.gz

    # Pre-compile ncurses
    echo "[7/16] Compiling ncurses.."
    mkdir /opt/ncurses
    cd ncurses
    ./configure --prefix=/opt/ncurses --with-shared >/dev/null 2>&1 
    make -j$threads >/dev/null 2>&1 && make install >/dev/null 2>&1
    cd ..

    # Pre-compile mysql
    echo "[8/16] Compiling MySQL.."
    mkdir /opt/mysql
    cd mysql
    cmake -DWITH_BOOST=boost/ -DCMAKE_INSTALL_PREFIX:PATH=/opt/mysql -DCURSES_LIBRARY=/opt/ncurses/lib/ -DCURSES_INCLUDE_PATH=/opt/ncurses/include . >/dev/null 2>&1
    make -j$threads >/dev/null 2>&1 && make install >/dev/null 2>&1
    cd ..

    # Pre-compile openssl
    echo "[9/16] Compiling OpenSSL.."
    mkdir /opt/openssl
    cd openssl
    ./config --prefix=/opt/openssl -fPIC >/dev/null 2>&1
    make -j$threads >/dev/null 2>&1 && make install >/dev/null 2>&1
    cd ..

    # Pre-compile libxml2
    echo "[10/16] Compiling LibXML2.."
    mkdir /opt/libxml2
    cd libxml2
    ./configure --without-python --prefix=/opt/libxml2 >/dev/null 2>&1
    make -j$threads >/dev/null 2>&1 && make install >/dev/null 2>&1
    cd ..

    # Pre-compile pcre
    echo "[11/16] Compiling PCRE.."
    mkdir /opt/pcre
    cd pcre
    ./configure --prefix=/opt/pcre >/dev/null 2>&1
    make -j$threads >/dev/null 2>&1 && make install >/dev/null 2>&1
    cd ..

    # Pre-compile zlib
    echo "[12/16] Compiling zlib.."
    mkdir /opt/zlib
    cd zlib
    ./configure --prefix=/opt/zlib >/dev/null 2>&1
    make -j$threads >/dev/null 2>&1 && make install >/dev/null 2>&1
    cd ..

    # Compile httpd
    echo "[13/16] Compiling httpd.."
    mkdir /opt/httpd
    cd httpd
    ./configure --with-pcre=/opt/pcre --with-z=/opt/zlib --with-libxml2=/opt/libxml2 --with-ssl=/opt/openssl --with-included-apr --enable-ssl --prefix=/opt/httpd >/dev/null 2>&1
    make -j$threads >/dev/null 2>&1 && make install >/dev/null 2>&1
    cd ..

    # Compile PHP7
    echo "[14/16] Compiling PHP7.."
    mkdir /opt/php7
    cd php7
    ./buildconf --force >/dev/null 2>&1 && ./configure --with-apxs2=/opt/httpd/bin/apxs --prefix=/opt/php7 --with-libxml-dir=/opt/libxml2 >/dev/null 2>&1
    make -j$threads >/dev/null 2>&1 && make install >/dev/null 2>&1
    cd .

    # Cleanup time!
    echo "[15/16] Cleaning up.."
    rm -rf php7 && rm -rf httpd && rm -rf zlib && rm -rf pcre && rm -rf libxml2 && rm -rf openssl && rm -rf mysql && rm -rf ncurses

    echo "[16/16] Finishing configuration for httpd, PHP7 and MySQL.."

    useradd -s /sbin/nologin mysql
    mkdir /opt/mysql/data
    chown -R mysql:mysql /opt/mysql/data
    mkdir /opt/mysql/data

    echo ""
    echo "Initializing MySQL:"
    echo ""

    /opt/mysql/bin/mysqld --initialize --datadir=/opt/mysql/data --user=mysql

    echo ""
    echo "You can start the MySQL daemon with:"
    echo "/opt/mysql/bin/mysqld --user=mysql --datadir=/opt/mysql/data --socket=/opt/mysql/data/mysql.sock"
    echo ""

    useradd -s /sbin/nologin www-data
    chown -R www-data:www-data /opt/httpd/htdocs
    sed -i 's/User daemon/User www-data/g' /opt/httpd/conf/httpd.conf
    sed -i 's/Group daemon/Group www-data/g' /opt/httpd/conf/httpd.conf

    mkdir /opt/certificates
    openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout /opt/certificates/selfsigned.key -out /opt/certificates/selfsigned.crt -subj "/C=NL/ST=Zuid-Holland/L=Dordrecht/ Department/" >/dev/null 2>&1
    echo "" >> /opt/httpd/conf/httpd.conf
    echo "# Appended at: ${currentdate}" >> /opt/httpd/conf/httpd.conf
    echo "LoadModule ssl_module modules/" >> /opt/httpd/conf/httpd.conf
    echo "<FilesMatch \.php$>" >> /opt/httpd/conf/httpd.conf
    echo "  SetHandler application/x-httpd-php" >> /opt/httpd/conf/httpd.conf
    echo "</FilesMatch>" >> /opt/httpd/conf/httpd.conf
    echo "Listen 443" >> /opt/httpd/conf/httpd.conf
    echo "<VirtualHost *:443>" >> /opt/httpd/conf/httpd.conf
    echo "  SSLEngine on" >> /opt/httpd/conf/httpd.conf
    echo "  SSLCertificateFile /opt/certificates/selfsigned.crt" >> /opt/httpd/conf/httpd.conf
    echo "  SSLCertificateKeyFile /opt/certificates/selfsigned.key" >> /opt/httpd/conf/httpd.conf
    echo "</VirtualHost>" >> /opt/httpd/conf/httpd.conf

    echo "You can start the HTTPD daemon with:"
    echo "/opt/httpd/bin/httpd"
    echo ""
    echo "Done!"
    echo ""
    echo ""

This has been tested on CentOS 7 x64.

You can start MySQL with this command:

/opt/mysql/bin/mysqld --user=mysql --datadir=/opt/mysql/data --socket=/opt/mysql/data/mysql.sock

And you can start httpd with this command:


Should you run into an issue: feel free to leave a comment!

Creating HTML tables from JSON data in JavaScript

 •  Filed under json, data, html tables, javascript

A few weeks ago I have written a simple JavaScript library to convert JSON data to an HTML table, and vice versa if you need it. With that library comes even more functionality, like the ability to filter (search) through table rows and the ability to filter JSON data for a multi-page table layout.

You can find my project on GitHub.

This is the table and search box HTML code that I used for this article:

<div class="container">  
  <div class="row">
    <input type="text" name="searchbox" id="searchbox" style="width: 100%;"><br><br>

  <div class="row">
    <table id="userTable" class="table">

So let's dive deeper into some code examples that you can use to create your tables from JSON data:

Example 1: Convert an HTML table to JSON

  // Create a new JSONTable Object from the basicTable table.
  var jsonTable = new JSONTable($("#basicTable"))

  // Convert existing HTML table to a JSON object.
  var tableData = jsonTable.toJSON()

Example 2: Search through a table with the filter function

  // Create a new JSONTable Object from the basicTable table.
  var jsonTable = new JSONTable($("#basicTable"))

  // Convert existing HTML table to a JSON object.
  var tableData = jsonTable.toJSON()

  // Create a jQuery event handler on the searchbox.
  $("#searchbox").on('input', function(e) {

Example 3: Create a new HTML table directly from JSON data

// Build a JSON object array with items which are to be included in the table.
var jsonData = [{"ID":"1","Name":"User1"},{"ID":"2","Name":"User2"},{"ID":"3","Name":"User3"},{"ID":"4","Name":"Admin1"},{"ID":"5","Name":"Admin2"}]

// Create a new table inside the body tag
$("body").append("<table id='testTable'></table>")

// Spawn a new JSONTable object on the newly created table
var jsonTable = new JSONTable($("#testTable"))

// Create HTML table (data)structure from JSON data

You should definitely try these examples. Good luck!

Detect and monitor DOM changes with MutationObserver

 •  Filed under monitor, javascript, detect, dom changes, mutationobserver

The MutationObserver JavaScript API allows a developer to actively watch for DOM changes, so that if something gets changed, added or removed within the boundaries of the specified DOM node, a callback function will be triggered.

Browser support for this API is pretty good as well. All major modern browsers do support MutationObserver, but please note that IE10 and lower are not supported unless you use some sort of shim or polyfill for this.

Getting started is very easy as well:

// Step 1: Create a new MutationObserver object
var observer = new MutationObserver( function(mutations) { // this is the callback function } )

// Step 2: Observe a DOM node with the observer as callback
observer.observe(document.querySelector("table"), { attributes: true, childList: true, attributeOldValue: true })  

As an example I have built up a table to watch out for DOM changes:

<table id="demoTable" class="table">  


And here is the JavaScript code that goes with this:

var observer = new MutationObserver(function(mutation) {  

var observerConfig = {  
  attributes: true,
  childList: true,
  attributeOldValue: true,
  characterData: true

const tableRows = document.querySelectorAll("#appUsers tr")

for (i = 0; i < tableRows.length; i++)  
  observer.observe(tableRows[i], observerConfig);

Try to destroy, add or recreate td elements inside the table and you will get an array of MutationRecord objects returned within your browsers' console. Please note that when you edit the node's innerHTML, nothing will get triggered. I don't know if this is a bug in Safari (I only tested this in Safari, FYI) or something by design.