Operating Systems

Setting up Jekyll blog on Ubuntu subsystem for Windows 😵

July 29, 2019 Frédéric Hannes

Whether you are a fan or not, the Ubuntu subsystem for Windows is here to stay. Personally I am a fan as it puts an entire Ubuntu operating system at your fingertips without the need to set up a Virtual Machine. At the same time, it integrates pretty well with the Windows environment, allowing you to easily access the file system.

One thing that I have found, this is very useful for, is leveraging the power of Apititude. This package manager that comes preloaded with Ubuntu (and many other Debian based operating systems), makes it very easy to install open-source tools which on Windows typically require you to download and install them manually and often suffer from issues caused by the subtle differences between operating systems.

A while ago I decided to check out Jekyll. It is a Ruby-based platform which converts Markdown based files into static websites or even blogs. The big advantage of such websites is that they are very easy for a webserver to server up, as they require very little logic on the server to be executed. Essentially you are just reading a file from the filesystem and sending it through the HTTP socket to the requester. With a good caching system, these pages can even be served up directly from the memory of the server.

Setting up

The Ubuntu subsystem for Windows is fairly lightweight. So a lot of packages will not come pre-installed. This means that before you can follow the default instructions to start a new Jekyll project, you will have to set up some prerequisites.

sudo apt-get update
sudo apt-get install -y ruby ruby-dev build-essential

1 2	sudo apt-get update sudo apt-get install -y ruby ruby-dev build-essential

These commands will install Ruby and the tools required to build Ruby gems. Next we will install the Bundler and Jekyll gems, which are needed to set up, build and serve your Jekyll based website.

sudo gem install bundler jekyll

1	sudo gem install bundler jekyll

Your Ubuntu subsystem should now be set up and ready for you to develop your website.

Your Jekyll project

As the prerequisites are now out of the way, it is time to follow the remaining setup instructions as they are given on the Jekyll website itself. The following commands will set up a basic project fro your website, build it and also serve it.

jekyll new my-awesome-site
cd my-awesome-site
bundle exec jekyll serve

jekyll new my-awesome-site

cd my-awesome-site

bundle exec jekyll serve

Your website should now be available at http://localhost:4000/ and update live when you modify it! All of the tools for editing on your Windows machine, you can now use to edit the website, even though it is being served by the Ubuntu subsystem.

GitLab “STDERR: chgrp: invalid group: ‘git’” 🤔

July 22, 2019 Frédéric Hannes

Today I ran into a bit of a pickle 🥒. I was upgrading GitLab on an Ubuntu server machine using Aptitude. I’ve done this hundreds of times and GitLab has never had the slightest issues updating for me in the past. Today was time for a new experience.

There was an error running gitlab-ctl reconfigure:

storage_directory[/var/opt/gitlab/.ssh] (gitlab::gitlab-shell line 38) had an error: Mixlib::ShellOut::ShellCommandFailed: ruby_block[directory resource: /var/opt/gitlab/.ssh] (/opt/gitlab/embedded/cookbooks/cache/cookbooks/package/resources/storage_directory.rb line 33) had an error: Mixlib::ShellOut::ShellCommandFailed: Expected process to exit with [0], but received '1'
---- Begin output of chgrp git /var/opt/gitlab/.ssh ----
STDOUT:
STDERR: chgrp: invalid group: ‘git’
---- End output of chgrp git /var/opt/gitlab/.ssh ----
Ran chgrp git /var/opt/gitlab/.ssh returned 1

There was an error running gitlab-ctl reconfigure:

storage_directory[/var/opt/gitlab/.ssh] (gitlab::gitlab-shell line 38) had an error: Mixlib::ShellOut::ShellCommandFailed: ruby_block[directory resource: /var/opt/gitlab/.ssh] (/opt/gitlab/embedded/cookbooks/cache/cookbooks/package/resources/storage_directory.rb line 33) had an error: Mixlib::ShellOut::ShellCommandFailed: Expected process to exit with [0], but received '1'

---- Begin output of chgrp git /var/opt/gitlab/.ssh ----

STDOUT:

STDERR: chgrp: invalid group: ‘git’

---- End output of chgrp git /var/opt/gitlab/.ssh ----

Ran chgrp git /var/opt/gitlab/.ssh returned 1

So to make a long story short… When GitLab is updated, it runs the command gitlab-ctl reconfigure. This command essentially performs some checks, makes sure that everything is upgraded and set up correctly. One of its internal features ensures that a folder exists and that it has the correct permissions.

      # In /opt/gitlab/embedded/cookbooks/cache/cookbooks/package/resources/storage_directory.rb

       33:   ruby_block "directory resource: #{new_resource.path}" do
       34:     block do
       35:       # Ensure the directory exists
       36:       storage_helper.ensure_directory_exists(new_resource.path)
       37:
       38:       # Ensure the permissions are set
       39:       storage_helper.ensure_permissions_set(new_resource.path)
       40:
       41:       # Error out if we have not achieved the target permissions
       42:       storage_helper.validate!(new_resource.path)
       43:     end
       44:     not_if { storage_helper.validate(new_resource.path) }
       45:   end
       46: end

# In /opt/gitlab/embedded/cookbooks/cache/cookbooks/package/resources/storage_directory.rb

33: ruby_block "directory resource: #{new_resource.path}" do

34: block do

35: # Ensure the directory exists

36: storage_helper.ensure_directory_exists(new_resource.path)

37:

38: # Ensure the permissions are set

39: storage_helper.ensure_permissions_set(new_resource.path)

40:

41: # Error out if we have not achieved the target permissions

42: storage_helper.validate!(new_resource.path)

43: end

44: not_if { storage_helper.validate(new_resource.path) }

45: end

46: end

This code failed to set the user group for the folder /var/opt/gitlab/.ssh because the required user group does not exist. Though this error seems fairly obvious, the problem I faced is that the user group actually does exist.

Luckily I remembered that after setting up this server, I also did some server hardening. One step in that process was to restrict access permissions to specific system files. So I went and checked which files I modified and one of the files happened to be /etc/group.

What I had done was take away privileges for those system files so they could only be accessed by their owner, the root user. This is typically not an issue, but it turns out that even though Aptitude was being ran by the root user, the gitlab-ctl process must have been trying to set the permissions of that folder using a different user account, which had no permission to access /etc/group.

The solution: “sudo chmod og+r /etc/group” to permit other users to read the file. After having Aptitude retry the upgrade process, everything worked great!

After you have finished the upgrade, you can go ahead and revert the permission changes (sudo chmod og-r /etc/group). But keep in mind that it will probably happen again the next time you are upgrading GitLab.

Cannot open your terminal ‘/dev/pts/0’

February 1, 2016 Frédéric Hannes

I ran into the error Cannot open your terminal '/dev/pts/0'. Though I had not seen this before, it seems to occur when you use the “switch user (su)” command to access a user account and then try to open a screen.

The issue can be resolved by either logging in directly with the user account you want to use, or by running the command “script /dev/null” as demonstrated here on StackOverflow.

Installing dig on Raspberry Pi (or Ubuntu)

August 24, 2015 Frédéric Hannes

I’m not sure about Raspbian, but the dig application is not installed on Minibian by default. To install it (or on any Debian based distribution), you have to install the dnsutils package, which contains it:
sudo apt-get install dnsutils

NoIP Dynamic Update Client on RaspberryPi (or Ubuntu)

August 24, 2015 Frédéric Hannes

Personally I like to use NoIP’s dynamic DNS services. This tutorial explains how to install the NoIP DUC on a RaspberryPi running a Raspbian based distribution, but it will work on Ubuntu or any other Debian based distribution as well. You should execute all of the commands below as superuser. As not to have to put sudo in front of all of them, it’s best to log in as root or switch to root with sudo su.

Step 1: Install prerequisites

You will need some software to install and run the DUC software. Execute the following commands:

apt-get update apt-get install build-essential python-setuptools easy_install killproc

Step 2: Download and install the DUC software

Next you will have to download and install the DUC software itself. The software has to be built from source code, which is why we installed build-essential. The following commands will download and install the software:

cd /usr/local/src/ rm -r noip-* wget http://www.no-ip.com/client/linux/noip-duc-linux.tar.gz tar xf noip-duc-linux.tar.gz rm noip-duc-linux.tar.gz cd noip-* make install

The commands above will first remove any previous source downloads for NoIP you might have had. Next it downloads the sources for the software and installs it. After installing it should immediately start the configuration wizard, which you can always restart using the command “/usr/local/bin/noip2 -C“. The configuration wizard will ask for your NoIP account details, as well as the host you want to be using. This should be mostly straight-forward. When it asks if you want to run anything on an update, you should probably just enter “No”.

Step 3: Running the DUC software on startup

By default, the DUC software will not start when you reboot your system. This is where killproc comes in, which we installed earlier. What we have to do now is set up an init script. We’re going to be using a simple script, as is described in the software’s readme file, with a slight alteration to make it work on our distribution. I have the script uploaded on my webserver, so you can just download it to your drive. Execute the following commands to install it and set it up:

cd /etc/init.d wget http://ibeblog.com/files/noip2 chmod +x noip2 update-rc.d noip2 defaults

Step 4: Reboot

You’re all done now, so you can reboot the system now and when it boots up, the NoIP DUC software should be running.

Sources

Tar (+GZip) individual (sql) files in a folder

May 12, 2014 Frédéric Hannes

I had been backing up a MySQL database with a simple shell script for a while now. What I somehow forgot to add to the script was a command to compress the SQL dumps before uploading them to my backup server. The script had been running for a few months, so manually compressing each file wasn’t much of an option. So I wrote this little shell script do the work for me.

#!/bin/sh

cd /some/directory/with/sql/files
for file in *.sql; do
    tar cvzf "${file}".tar.gz "${file}"
    rm "${file}"
done

#!/bin/sh

cd /some/directory/with/sql/files

for file in *.sql; do

tar cvzf "${file}".tar.gz "${file}"

rm "${file}"

done

The script places all of the sql files individually into a tar archive with GZip compression. The new file uses the old file’s name with “.tar.gz” appended at the end. The old file is deleted after the compressed archive has been created.

You can of course do this for other files, just change the *.sql mask to work with different files. If for some reason you just want TAR files without the GZip compression, replace the tar line in the script with “tar cvf "${file}".tar "${file}"“.

Check available diskspace on Ubuntu (CLI)

May 12, 2014 Frédéric Hannes

If you are running an Ubuntu (or other linux) commandline interface, you might have to check how much diskspace is available on the system. You can easily have the system output a list of the disk space usage on all mounted filesystems with the command df -h.

The output depends completely on how your system is set up, but it will look something like:

Filesystem      Size  Used Avail Use% Mounted on
/dev/sda1       110G   33G   72G  32% /
none            4.0K     0  4.0K   0% /sys/fs/cgroup
udev            7.7G  4.0K  7.7G   1% /dev
tmpfs           1.6G  328K  1.6G   1% /run
none            5.0M     0  5.0M   0% /run/lock
none            7.8G  4.0K  7.8G   1% /run/shm

Filesystem Size Used Avail Use% Mounted on

/dev/sda1 110G 33G 72G 32% /

none 4.0K 0 4.0K 0% /sys/fs/cgroup

udev 7.7G 4.0K 7.7G 1% /dev

tmpfs 1.6G 328K 1.6G 1% /run

none 5.0M 0 5.0M 0% /run/lock

none 7.8G 4.0K 7.8G 1% /run/shm

Setting up the PiFace Digital with Raspbian/Minibian

April 23, 2014 Frédéric Hannes

The PiFace Digital is an I/O expansion board for the Raspberry Pi. It has to be installed by plugging it on to the GPIO of the Raspberry Pi. The PiFace website has its own guide which tells you how to set up the PiFace. I’d also like to provide you with my own guide to do this on Raspbian or Minibian.

Before proceeding, you should make sure you have executed step 2 and 3 of my first steps guide for Minibian. If you are logged in as an account other than the root account, you should add “sudo” in front of all of the commands described in the guide. If you are running Raspbian with a desktop environment, you have to open a command-line terminal window first.

Minibian, first steps!

April 23, 2014 Frédéric Hannes

Congratulations! You are now running Minibian on your Raspberry Pi! But what do you do now? Regardless of your goal for the Raspberry Pi, there are a few things you should do before proceeding after you first install Minibian. (Or any other operating system for that matter).

Step 1: Change your password!

Minibian comes with a root account named “root“, with the password “raspberry“. Like you would on any system, it is fairly imperative that you, at the very least select your own password. It is possible to set up your own user account which can be elevated to root privileges only when needed, and at the same time just getting rid of the password for the root account all together. But to keep things simple, let us just start by setting a custom password for the existing root account. To change the password, log into the system and simple type “passwd” into the command-line interface. You will be asked to enter your new password and a second time afterwards to confirm it. The reason why this step is so important, is that the root account has the ability to access and modify the entire system. If someone else gains access to your device’s root account, they might damage the software or even the hardware.

Installing Minibian on the Raspberry Pi

April 23, 2014 Frédéric Hannes

As far as Linux flavors go, I’m a Debian fan, running Ubuntu where ever I am able. So I like to run Raspbian, or more specifically, Minibian on my Raspberry Pi. What is Minibian you ask? The Raspberry Pi Foundation provides its users with the Raspbian image. Raspbian is a port of the Debian operating system, using the LXDE window manager. Its purpose is to provide a clean lightweight desktop environment which users can install onto their Raspberry Pi. Like me, many people don’t need or want a desktop environment running on their Raspberry Pi. Even through it is fairly lightweight, even LXDE will eat CPU power, memory and disk space. One option is to remove all software packages you don’t want from the Raspbian image after you have installed it. I will be using the Minibian image instead. It is a minimal image based on Raspbian for the Raspberry Pi, which strips all of the software you don’t need, including the desktop environment. Any packages you are missing, you can install later on.