Deploy to your test server with git hooks

Posted on 2018-09-09 in Programmation

You probably already wanted to have your own test environment to allow others in the company to tests what you did. Perhaps you have a common one, but as your team is growing, it is probable that the common environment is a bottleneck and you wish each developer could have its own to test things in isolation. That's the subject of this article where we will create such an environment and deploy to it with a simple git push (a lot like Heroku but a lot cheaper). You will also have full control over it and you will be able to modify the files directly on the server if needed (just don't do that last part in production ;-)).

In a nutshell, we need to support:

  1. Code updates:

    1. Push to a current branch.
    2. Force push to a current branch
    3. Erasing unclean work tree on the server
    4. Pushing a new branch
  2. Restart of the server (meaning the test environments must be up and running if the server restarts).

  3. Environment variables to configure the application.

To achieve all this, all you need is a linux server. Your machine or a VM will do if you want to test along. If you want to use a VM, you may want to check my tips for libvirt to use a static ip for the VM and Connect to a VirtualBox VM to ease SSH connection in you VM.

To do it for real, a server in the office or in the cloud will do. If you need help, leave a comment, I'll do my best to provide feedback.

For the purpose of this tutorial, we will deploy a small hello work app. You can fork it and use the fork as a remote repository if you want to follow along. The app is made in Python and there are a few Python specific things here and there but globally, you should be able to adapt it to any other languages. Leave a comment if you need help.

Before we start, some precision:

  • If your server is running SELinux, you'll have to add proper context (the web one) to some file and allow the web server to connect to the UNIX or TCP socket of the application server. I won't go into this here, the tutorial is already long enough.
  • I'll use as my test domain. Replace it with your own.
  • Commands that must be run as root with start with # and as a normal user with $. I'll precise the host too.
  • You need new commits to push the code with git. If you change nothing, no hooks (more on that later) will run on the server because git is smart enough to see nothing changed.
  • I assume your OS runs systemd. If not, adapt the commands.
  • I'll present things in a tutorial manner so you can understand and follow what I do. Go to the Summary section to view a recap and what you need to do each time you want to add a developer.

Prepare the server

  1. Get a linux server (cloud, VM, your machine, Raspberry Pi, …)

  2. Install the prerequisites

    • SSH
    • git
    • nginx (apache or any other web server can work too, you'll have to adapt the conf though)
    • Python (or anything else you need to run your application)
  3. Configure SSH: you probably want to edit /etc/ssh/sshd_config to disallow direct root login (with PermitRootLogin no), force SSH key usage (with PasswordAuthentication no) and perhaps change the default port (eg with Port 422). Search the internet to learn more.

  4. Enable SSH: on the server, run systemctl start sshd and systemctl enable sshd

  5. Configure the firewall to let SSH and HTTP traffic through (adapt these commands if you changed any ports):

    • With firewalld: # firewall-cmd --add-service=http --permanent && firewall-cmd --add-service=http and # firewall-cmd --add-service=ssh --permanent && firewall-cmd --add-service=ssh
    • With iptable: # iptables -t filter -A INPUT -p TCP --dport 22 -j ACCEPT and # iptables -t filter -A INPUT -p TCP --dport 80 -j ACCEPT
  6. Create at least one user (each dev should have their own user, more on that later). I'll use jujens.

  7. Create the deploy group: # groupadd -r deploy and add your user to it: # usermod -aG deploy jujens

  8. Start and enable nginx: # systemctl start nginx and # systemctl enable nginx

To use the bl.test and domains, you must either modify your /etc/hosts by adding something like this (change the ip of the server): bl.test

Or use dnsmasq to redirect all requests made to bl.test and its subdomains to an ip. See my article on that

If your server is publicly accessible, you may add a wildcard entry to your DNS server so all subdomains of are served by the test server. This way you will avoid the need to change your DNS configuration when you add a new developer.

Prepare the repo

Configure git

First, we will configure git. You can use a global configuration by editing /etc/gitconfig (users can still change it for all their repos in ~/.gitconfig or per each repo in .git/config).

Note that if you intend to do other things than git deploys with your server, the global configuration may not be the best choice.

Here is the git configuration we'll use

    hooksPath = /etc/git/hooks

    denyCurrentBranch = updateInstead

You can check it by running: git config receive.denyCurrentBranch and git config core.hooksPath.

You must adapt the path to where the git hooks will be. If you choose a per repo configuration, you don't need to specify it and it will default to .git/hooks

By default git won't allow you to push to a non bare repo (ie a repo in which you can access files easily) because it is dangerous. But in our case, it is required, otherwise, we won't be able to push anything while making the site work. That's the point of the ``denyCurrentBranch = updateInstead`` option. See the documentation for more details.

Now we can create the directory where the global hooks will lie: mkdir -p /etc/git/hooks. We will add scripts here later.

Prepare the server repo

  1. Clone the remote repo on your dev machine (where you want) so you can push from it.

  2. Clone the remote repo on the server. You could also push directly from your dev machine to it but I find it easier this way (and you still have to create a repo on the server). Do it:

    1. Create the folder /var/www/bl.test/ (the location may seem odd right now, but it will have its importance later): # mkdir -p /var/www/bl.test/
    2. Set user and group: # chown jujens:nginx /var/www/bl.test/
    3. Set the proper permissions: # chmod u=rwX,g=rXs,o=- /var/www/bl.test/ (this will give all permission the current user, read and "execute" permissions to the group and nothing to others. The s is there to set the "sticky bit". Thanks to it, all file created inside the directory will belong to the group you specify.
    4. Clone: # git clone /var/www/bl.test/
  3. Add the test remote on your dev repo (so you can push to the test server): $ git remote add test jujens@bl.test:/var/www/bl.test/

  4. Check everything is configure properly with a $ git push test It should output:

    The authenticity of host 'bl.test (' can't be established.
    ECDSA key fingerprint is SHA256:nb/a1ypBTTgahruVf92GzLNGTAdrrDi2afIWfXaFAa0.
    Are you sure you want to continue connecting (yes/no)? yes
    Warning: Permanently added 'bl.test' (ECDSA) to the list of known hosts.
    Everything up-to-date

How to send the code to the server

First, let me explain briefly what git hooks are since we will need them. Simply put, git hooks are scripts that are run before or after certain actions (commit, push, …). They are named pre-ACTION or post-ACTION and are handy to automatically run linting before committing or tests before pushing. By default they lie in PROJECT/.git/hooks (by the way, you can find samples in this folder). Here, we used the hooksPath to have global hooks.

Don't forget to make these script files executable or they won't work (I forget almost all the time!).

See the doc for more details and the full list of available hooks.

Now that's you know what git hooks are, we will do each step at a time (git commands are run on you machine):

  1. Step 1 Push to a current branch: Nothing to do since we configured git properly. Just modify, commit and git push test. You should see the commit and the modified file on the server.

  2. Step 2 Force push to a current branch: Let's say we made a mistake in our previous commit, amended it and want to force push that. Modify the again and do git commit --amend -a to amend the last commit. git push test will fail but git push test --force will do just what we need. This will also work if you have new commits on the test server, just be aware they will be deleted.

  3. Step 3 Erasing unclean work tree on the server: Modify on the server and don't commit. This is meant to simulate a debug modification you made after deploying the app. Now modify on your machine, commit and push. Neither git push test nor git push test --force will work because the work tree is unclean: you have a modified file whose content would be lost if the push succeeded. You could connect to the machine and run git checkout . yourself. But we can automate this with git hooks. Here we want to use the pre-receive hook that will be run before git gets the pushed content. In /etc/git/hooks/pre-receive (or .git/hooks/pre-receive if you want local hooks, I won't precise again), add:

    #!/usr/bin/env bash
    # We launch the command in a subshell to avoid impacting the true environment with our unset.
    # We unset GIT_DIR because it points to the wrong directory.
    # We are in the .git directory when this hook is run, so we need to move one step up to run git commands.
    # We then force a checkout of the repo to discard any changes.
    ( unset GIT_DIR && cd .. && git checkout -f )

    And voilà! Our git push works!

  4. Step 4 Pushing a new branch: this must be done once git received all the commits with the post-receive hook to update to the new branch. Git hooks are generally passed some parameters so we can do "smart" actions with them. Here, we will read the ref (name of the pushed branch). Create /etc/git/hooks/post-receive and add:

    #!/usr/bin/env bash
    # Parameters are passed to stdin, we need to use read to get them. Only ref matters for what we do.
    read oldrev newrev ref
    # By default, ref contains refs/heads/branch-name. We use bash substitution mechanism to remove "refs/heads/".
    # Once again, force a checkout to the branch.
    ( unset GIT_DIR && cd .. && git checkout -f $branch_name )

Now we can correctly push our code the repo on the test server. On big leap forward. Now let's see how we can configure the server.

Configuring the web server

Basic setup of the application server

First, we will prepare our venv (or whatever else you need in your language) with the proper dependencies. This section must be adapted if you are not using Python. Run on the server:

  1. Create the venv: $ python3 -m venv .venv

  2. Activate it: $ source .venv/bin/activate

  3. Install the dependencies: $ pip install -r requirements.txt

  4. Start the application server: $ gunicorn hello:app

  5. Check it is running with $ curl localhost:8000 You should see the following output

    Hello World!

In a true server, you may want to use --forwarded-allow-ips to limit who can connect to guincorn. See: gunicorn's documentation

Now that it's done, we can add to the post-receive hook (/etc/git/hooks/post-receive) the following line to update our venv each time code is pushed:

# We are again in .git by default.
( cd .. && source .venv/bin/activate && pip install -r requirements.txt )

Basic nginx configuration

Create nginx configuration file in /etc/nginx/conf.d/bl.test.conf (or /etc/nginx/site-enabled/bl.test.conf depending on your system, I'll use /etc/nginx/conf.d for the rest of this tutorial) with this content:

server {
    listen 80;
    # Put it to the folder that will contains all the repos.
    root /var/www/bl.test/;

    # Prevent access to source files.
    location ~ .*\.pyc? {
        return 404;

    # Prevent access to git
    location ~ .git {
        return 404;

    # Relay everything else to the application server.
    location / {
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header Host $http_host;
        proxy_redirect off;

Test the configuration with nginx -t and if everything is fine, reload nginx: systemctl reload nginx. On your development machine, run curl to check everything is fine. You should see something like:

Hello World!

Configuration for multiple users

In the basic configuration section, we used a TCP socket (with a port) to access our application server. It works but if you want to add an environment for a new developer, you will need to:

  1. Find an used port and configure gunicorn to use it.
  2. Add a new nginx configuration so nginx accept request from the proper domain and routes them to the proper application server.

Thankfully, we can also use another kind of sockets: UNIX sockets. They exist as files on the server and if named with a proper convention, we can route our traffic easily. That's why I suggested to used the domain as the name of the folder that contains the code. It should become obvious once we completed it.

Configuring the application server

We will rely on systemd to start and restart the application servers. First, create a file named /etc/systemd/system/gunicorn@.service. It is a service template: it is meant to run multiple instances of a service based on a name ( for instance). The name will be accessible as a variable (%i) in the template so we can alter the behavior of the service based on it.

Put this content in the file (adapted from

Description=gunicorn daemon for %i

ExecStart=/var/www/bl.test/%i/.venv/bin/gunicorn \
        --pid /var/run/gunicorn/   \
        --bind unix:/var/run/gunicorn/%i.sock \
ExecReload=/bin/kill -s HUP $MAINPID
ExecStop=/bin/kill -s TERM $MAINPID


I won't go into the details here (archlinux has a great doc for that). It is just how we create new unit in systemd. A unit being used to run a service. We use a description and dependencies in the Unit section. We then define the service:

  • PIDFile is used by systemd to know whether the service is working.
  • We define as who the service will run.
  • We update the path so it can find our venv.
  • We defined the working directory to where the app of any given user will live thanks to %i
  • We defined how we want to start the service and how it can be reloaded and restarted.

I didn't use the --reload option of guincorn. That is because we may need to update the dependencies of our project before restarting it.

You may want to use the --worker X option to run multiple worker and thus allow your app to handle more traffic. In a test server it may not be needed though.

Now we need to "tell systemd" the file has changed with systemctl daemon-reload. We must also prepare the directories required by gunicorn to run:

  1. Where the PID and socket files will be: mkdir -p /var/run/gunicorn
  2. Correct permissions: chmod u=rwX,g=rX,o=- /var/run/gunicorn/

We can now start and enable the server by using the template and specifying the name of the instance like this: guincorn@DOMAIN. Or in my case: systemctl start and systemctl enable

The trailing .service in the service name is optional.

Use journalctl -u gunicorn@INTANCE to view the logs. For instance: journalctl -u

You don't actually need to create the /var/run/gunicorn directory. systemd will create it based on the value of RuntimeDirectory and set the permissions based on the values of User and Group.

On Ubuntu, /var/run/ is mounted on a tmpfs so the directory will disappear. It also seems that when you will launch multiple instances, the folder will be deleted and recreated making its usage as described in this tutorial impossible (guincorn cannot start by putting its PID and Socket directly in /var/run because of a permission issue). I suggest you use a runtime directory per user with RuntimeDirectory=gunicorn-%i or create a /var/myrun/ folder and adapt the configuration or dig into how Ubuntu does things.

Configuring nginx

We need to update the location / section of our nginx configuration to use the proper UNIX socket:

location / {
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header Host $http_host;
    proxy_set_header X-Forwarded-Proto $scheme;
    proxy_redirect off;
    proxy_pass http://unix:/var/run/gunicorn/$host.sock;

We also need to update the server name into server_name *.bl.test;.

nginx will respond with a 502 Bad Gateway if there is no socket for that domain (either because it doesn't exist or because gunicorn is down).

Static files

We could use something like whitenoise to serve our static files. But since we have a nginx server, we can also rely on it to do the job:

server {
    listen 80;
    server_name *.bl.test;
    root /var/www/bl.test;

    # Prevent access to source files.
    location ~ .*\.pyc? {
        return 404;

    # Prevent access to git.
    location ~ .git {
        return 404;

    # If you request /page, we will check if /$host/page exists. If so we serve it.
    # Otherwise we pass the request to gunicorn for processing.
    # For instance, if I request, nginx will check whether
    # /var/www/bl.test/ exists and serve it to me.
    # /var/www/bl.test comes from the server root and from the $host variable.
    location / {
        try_files /$host/$uri @gunicorn;

    # Relay everything else to the django web server.
    location @gunicorn {
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header Host $http_host;
        proxy_redirect off;
        proxy_pass http://unix:/var/run/gunicorn/$host.sock;

Serving directly static files this way is insecure. You may want to choose an empty folder as root and relay everything to your app to serve only relevant files. Another good idea would be to put all static assets in a static directory and serve only its content. For Django users, you can adapt the configuration I mention here.

Testing deployment

We are almost ready to test the deployment of our app. We still need two things. First, we must configure sudo to allow the user to restart gunicorn (because git hooks are run as the currently logged in user, jujens in my case, they may not be allowed to run systemctl restart by default). To do that:

  1. Run visudo to edit the sudoers file (visudo will check the syntax of the file before exiting. You can use another editor by setting the EDITOR environment variable).
  2. Add at the bottom of the file: %deploy ALL=(ALL) NOPASSWD: /usr/bin/systemctl restart gunicorn@* This will allow all users in the deploy group to restart any instance of gunicorn. I think it is the easiest way to achieve our goal. Be more restrictive if you need to.

We can now add to our post-receive hook (/etc/git/hooks/post-receive):

# We move to the folder that contains our code.
# We extract the domain (and guincorn instance name) from the folder
# by keeping only the name of the current directory of the full path.
# Finally, we restart the proper instance of gunicorn.
( cd ..; unit_name=$(pwd); unit_name=${unit_name##*/}; sudo systemctl restart "gunicorn@${unit_name}" )

Now if you deploy a new commit, gunicorn will restart and if you do curl you will be able to see you modification.

Environment variables

So far so good. But you may want to configure your application on a per user basis (to use a specific database for instance). To do this, you can rely on environment variables as recommended by the twelve factor app. To do this, we will configure systemd to load an environment file which will contain all our variables. gunicorn will them be able to access them and so will our app.

To do that:

  1. Add to the gunicorn@.service file (/etc/systemd/system/gunicorn@.service), after Environment:

  2. Notify systemd about the change with: systemctl daemon-reload.

You may also want to prevent access to this file with nginx:

location ~ ^/.env$ {
    return 404;

Now create /var/www/bl.test/ and put in it:

MY_ENV_VAR="Loaded from env"

Redeploy your app, run curl and you should see Loaded from env in the response.

Secure access

Since this is a test server, you may want to restrict access to the application. You can do this by putting the server in a local network not accessible to the outside world, basic auth or X509 auth. I'll only detail the basic auth method. For X509, see:

To protect your server with a username and a password at the nginx level:

  1. Install httpd-tools (or apache2-utils or whatever provides /usr/bin/htpasswd)
  2. Create the password file and add username to it: htpasswd -c /etc/nginx/bl.test.passwd username
  3. Type password
  4. Configure nginx to use this file and prevent access without authentication by adding in the server block:
auth_basic "Test server";
auth_basic_user_file /etc/nginx/bl.test.passwd;

Test the configuration with nginx -t and reload if the configuration is good with: systemctl reload nginx.

Going further

At this point, we are done and everything should work as intended. I just want to point out some other ways to do things:

  • You could use the post-receive hooks to move static files into a dedicated folder (eg /var/www/bl.test/static/ and use /var/www/bl.test/static as the root of nginx to be sure not to serve source files. Since it will communicate with gunicorn with a socket, its root need not be where the source lives. You don't need to bother about a dedicated path for each user if your static files contains their hash in their name. You'll have to setup a cleanup mechanism though. I didn't do it in this tutorial to keep things more simple.

  • If you repo is big, you can use something like: git --work-tree=/var/www/bl.test/ --git-dir=/home/jujens/bl.test/.git checkout -f to make everyone push to one git repository and update their source tree. You can use the $USER environment variable to know who pushed and copy to the proper directory. You'll have to adapt the file structure.

  • A very good article form digital ocean has almost the same subject as this tutorial (they give more precision about git hooks and handle only static applications).

  • You can also extract the subdomain name (jujens in my case) to use just it in your paths and instance names. Something like this should work (see

    map $host $env_name {
        default ...;
        ~* ^(?<x_env_name>[a-z0-9]+)\..* $x_env_name;
  • You can also add HTTPS with something like let's encrypt or you usual provider.

  • Depending on the level of customization you want, you could also use a project like dokku.


Thank you for going this far. The article is quite dense and I think it deserves a quick summary. You can also find all the configurations below with a download link.

So in a nutshell, to setup the server, you need to (refer to Prepare the server for the details):

  1. Install the prerequisites.
  2. Configure SSH.
  3. Add and configure the base user and group.
  4. Setup the global domain if possible.

Each time you want to add a user, you need to:

  1. Add a new DNS entry if you can't use "glob domains".
  2. Add a new user in the deploy group.
  3. Prepare the code repository (clone and setup hooks if you didn't choose to use global ones).
  4. Prepare the environment (.env file, venv, …)
  5. Start the application server.

You could use an adaptation of this script to make your life easier (run as root):

 1 #!/usr/bin/env bash
 4 # USERNAME: username used for ssh and in the domain
 5 # REPO_TO_CLONE: the repository to clone.
 6 # example: ./ jujens
 7 # will create a jujens user to use the subdomain with the specified app.
 9 # Configure bash to exit on errors, consider undefined variables as errors
10 # and fail if any step in a pipe does.
11 set -e
12 set -u
13 set -o pipefail
15 GIT_REPO_BASE_DIR="/var/www/bl.test"
16 BASE_DOMAIN="bl.test"
18 USERNAME="$1"
19 REPO_URL="$2"
23 echo "Creating user and add to group."
24 useradd "${USERNAME}"
25 usermod -aG deploy "${USERNAME}"
27 echo "Preparing repo."
28 mkdir -p "${REPO_DIR}"
29 chown jujens:nginx "${REPO_DIR}"
30 chmod u=rwX,g=rXs,o=- "${REPO_DIR}"
31 git clone "${REPO_URL}" "${REPO_DIR}"
33 # Adapt to your needs and language.
34 echo "Preparing venv"
35 pushd "${REPO_DIR}"
36     echo "Opening .env file"
37     ${EDITOR:-vim} .env
39     python3 -m venv .venv
40     source ".venv/bin/activate"
41     pip install -r requirements.txt
42 pushd
43 echo "Launching service"
44 systemctl start "gunicorn@${SUB_DOMAIN}"
45 systemctl enable "gunicorn@${SUB_DOMAIN}"
48 # That's assuming you use the default SSH port, if not, adapt.
49 echo "To add the git remote: git remote add test ${USERNAME}@${BASE_DOMAIN}:${REPO_DIR}"
50 echo "Done"

Full configurations

  • nginx configuration:

     1 server {
     2     listen 80;
     3     server_name *.bl.test;
     4     root /var/www/bl.test;
     6     auth_basic "Test server";
     7     auth_basic_user_file /etc/nginx/bl.test.passwd;
     9     # Prevent access to pyc and py files.
    10     location ~ .*\.pyc? {
    11         return 404;
    12     }
    14     location ~ .git {
    15        return 404;
    16     }
    18     location ~ ^/.env$ {
    19         return 404;
    20     }
    22     location / {
    23         try_files /$host/$uri @gunicorn;
    24     }
    26     # Relay everything else to the django web server.
    27     location @gunicorn {
    28         proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    29         proxy_set_header Host $http_host;
    30         proxy_set_header X-Forwarded-Proto $scheme;
    31         proxy_redirect off;
    32         proxy_pass http://unix:/var/run/gunicorn/$host.sock;
    33     }
    34 }
  •  1 [Unit]
     2 Description=gunicorn daemon
     5 [Service]
     6 PIDFile=/var/run/gunicorn/
     7 User=nginx
     8 Group=nginx
     9 EnvironmentFile=/var/www/bl.test/%i/.env
    10 Environment="PATH=/var/www/bl.test/%i/.venv/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin"
    11 RuntimeDirectory=gunicorn
    12 WorkingDirectory=/var/www/bl.test/%i
    13 ExecStart=/var/www/bl.test/%i/.venv/bin/gunicorn \
    14           --pid /var/run/gunicorn/   \
    15           --bind unix:/var/run/gunicorn/%i.sock \
    16           hello:app
    17 ExecReload=/bin/kill -s HUP $MAINPID
    18 ExecStop=/bin/kill -s TERM $MAINPID
    19 PrivateTmp=true
    20 Restart=on-failure
    21 RestartSec=5s
    23 [Install]
  • git hooks:

    • pre-receive

      1 #!/usr/bin/env bash
      3 # We launch the command in a subshell to avoid impacting the true environment with our unset.
      4 # We unset GIT_DIR because it points to the wrong directory.
      5 # We are in the .git directory when this hook is run, so we need to move one step up to run git commands.
      6 # We then force a checkout of the repo to discard any changes.
      7 ( unset GIT_DIR && cd .. && git checkout -f )
    • post-receive

       1 #!/usr/bin/env bash
       3 # Parameters are passed to stdin, we need to use read to get them. Only ref matters for what we do.
       4 read oldrev newrev ref
       5 # By default, ref contains refs/heads/branch-name. We use bash substitution mechanism to remove "refs/heads/".
       6 branch_name=${ref/refs\/heads\//}
       8 # Once again, force a checkout to the branch.
       9 ( unset GIT_DIR && cd .. && git checkout -f $branch_name )
      11 # You can add additionnal steps to complete checkout on the current branch (install deps, build JS/CSS files, …)
      12 # For instance:
      13 ( cd .. && source .venv/bin/activate && pip install -r requirements.txt )
      15 # We move to the folder that contains our code.
      16 # We extract the domain (and guincorn instance name) from the folder
      17 # by keeping only the name of the current directory of the full path.
      18 # Finally, we restart the proper instance of gunicorn.
      19 ( cd ..; unit_name=$(pwd); unit_name=${unit_name##*/}; sudo systemctl restart "gunicorn@${unit_name}" )
      20 # You could simply this line by using the $USER variable and creating BASE_DOMAIN="bl.test" into ``sudo systemcl restart "gunicorn@${USER}.${BASE_DOMAIN}"``