Implementing IP Multicast over Ethernet with LwIP and ChibiOS on a STM32F767ZI

For my master’s thesis, I work with STM32F767 Nulceo-144 boards. We are using ChibiOS as operating system and needed support for IP Multicast. This blog post describes what is necessary to add Multicast support to this setup. Overall, it was a lot of fun and I learned a lot!

First of all, let’s take a quick look at how Multicast works. Multicast means that we address a single packet to several peers. This works by using special addresses. For IPv4 these are 224.0.0.0 to 239.255.255.255 (224.0.0.0/4). If a host wants to receive packets for a Multicast address, it must join the respective Multicast Group. It does so using IGMP. Then, the router in our network knows that the host wants to receive traffic for the respective Multicast Group and will start forwarding traffic.

IP Multicast over Ethernet

In an Ethernet Collision Domain, multiple hosts may want to receive Multicast traffic. If we would just send traffic to the host’s regular MAC address, the router would have to duplicate packets, if there is more than one host receiving multicast traffic. To avoid this, there are special Multicast MAC addresses. Similarily to a Broadcast MAC address, they address multiple hosts. These MAC addresses are 01-00-5E-00-00-00 to 01-00-5E-7F-FF-FF. As we have 23 Bits for in the Multicast MAC range, but 28 Bits in the Multicast IP range, this is not a one-to-one mapping. Instead, multiple Multicast IPs map to a single Multicast MAC address. The mapping works by OR-ing the lower 23 Bits of the Multicast IP with the Multicast base MAC address (01-00-5E-00-00-00).

When the Ethernet MAC of our microcontroller (µC) receives a Ethernet frame, it takes a look at the destination MAC address of the frame. If the frame is not addressed to the µC, i.e. its destination MAC address does not match the MAC address of the µC, it is simply discarded by the MAC. This saves resources, as the microcontroller can continue working on other tasks and no CPU time must be spent handling packets that are not meant for us.

For Multicast, this is of course not what we want: the frames are not sent to our MAC address, but to the Multicast MAC address! Thus, the MAC in our µC will simply discard the frame and it will never be seen by software. This means we have to tell hardware that it should accept frames for the MAC address of the Multicast group we want to join.

LwIP and Multicast

Fortunately, LwIP already comes with Multicast support. You can join a multicast group using the netconn_join_leave_group() function. It will take care of IGMP for you. But, with the stock ChibiOS MAC drivers, it will not take care of configuring your MAC to accept packets for the Multicast MAC address! So, the MAC driver has to be notified when we join a Multicast Group. For this, LwIP allows you to register a callback using the netif_set_igmp_mac_filter() function. This callback can then take care of configuring the MAC appropriately.

Configuring the MAC to accept frames for the Multicast MAC address

Now to the fun low-level part! Let’s take a look at the STM32F76xxx Reference Manual, section 42.5.5 “MAC filtering”. We have several options here:

  1. Set the Pass All Multicast (PAM) bit in the MACFFR register. Then, the MAC will pass all Multicast frames to software. This is of course very easy to implement. We do not even have to use LwIP’s igmp_mac_filter. We can just set the respective bits when the MAC is initialized. This is, however, a quick and dirty solution, as all Multicast frames then have to go through the LwIP stack to decide whether we are interested in them. This means we will waste quite some CPU cycles just for this. Thus, I did not choose this solution.
  2. Use perfect filtering and write the Multicast address to one of the four MAC address registers. However, this limits the number of Multicast groups we are able to join. As the device itself needs a MAC address, we can join at most three Multicast groups.
  3. Use imperfect filtering using the Hash Table. This is a good compromise between aoproach 1 and 2: We can join as many Multicast Groups as we want, but we will only waste CPU cycles if there has been a collision in the hash table. I chose this approach, even though it is probably the most difficult to implement.

The MAC hash table

Let’s take a look at how the MAC hash table works. When the MAC receives a frame, it extracts the destination address. It will then compute the CRC-32 of this MAC address, which results in a 32-bit value. Only the upper six bits are then used for the hash table.

The hash table consists of two registers: the Ethernet MAC hash table low register (ETH_MACHTLR) and the Ethernet MAC hash table high register (ETH_MACTHR). Each of these registers is 32 Bits, which means that the hash table is 64 Bits in total.

The most significant bit of the calculated CRC value indicates whether the ETH_MACTLR or the ETH_MACTHR is consulted. The next five bits of the CRC then indicate the index of the bit in the register that is checked. If the respective bit is one, the frame passes the MAC filter, otherwise it does not.

So, in our igmp_mac_filter callback, we have to do the following steps:

  1. From LwIP, we receive an IP address as a parameter to the callback. We first have to convert this to a MAC address, as explained above.
  2. We then have to calculate the CRC-32 of this MAC address.
  3. Finally we set the appropriate bits in the ETH_MACTHR or ETH_MACTLR registers.

Keep in mind that Ethernet transmits the least significant bit first. Weirdly, this also applies to the CRC calculation above. The most significant bit of the CRC is actually the least significant bit and so on.

I will publish the code as soon as I’ve tested my implementation thoroughly and all the bugs are fixed. Second blog post with a deep dive into the code will follow!

Request Let’s Encrypt Certificates with Ansible

Since version 2.2, Ansible comes with the letsencrypt module. This allows you to request certificates from letsencrypt. Unfortunately, I’ve found the documentation to be a bit lacking, so I want to document what I’ve done to get this working. If your distribution doesn’t have Ansible 2.2 yet, you can very easily install it using pip:

pip install --user ansible

As always, this is not a copy & paste tutorial, but a collection of ideas that you have to adjust to your environment.

First of all, you need an account key. The Ansible documentation doesn’t really tell you how you have to create that, so here is an Ansible task that does just that:

- name: Create Lets Encrypt Account Key
  command: "openssl genrsa -out {{ letsencrypt_account_key }}"
  args:
      creates: "{{ letsencrypt_account_key }}"

You also may want to make sure that the key is protected from unauthorized access:

- file:
    path: "{{ letsencrypt_account_key }}"
    owner: root
    group: root
    mode: 0600

Now we can start with the certificates. For this, it is important to know that the host to which the playbook is applied handles the communication with Let’s Encrypt. This means that it is not sufficent if the host on which you run Ansible can make outbound connections to the Let’s Encrypt servers. You have to adjust your firewall to allow outbound connections from the host the playbook is being applied to to the Let’s Encrypt servers. I usually run a small proxy server (for example, tinyproxy) for such outgoing connections. Ansible will respect the http_proxy and https_proxy environment variables.

sslvhost.yml will handle key generation and requesting the certificates from Let’s Encrypt. We call this tasklist for each certificate we want to request. This is also a great place for the proxy environment variables, if you use one:

- name: Generate Certificates for SSL
  include: sslvhost.yml
  vars:
    domain: "example.com"
    key_file: "{{ letsencrypt_key_dir }}/example.com-key.pem"
    req_file: "{{ letsencrypt_cert_dir }}/example.com-req.pem"
    cert_file: "{{ letsencrypt_cert_dir }}/example.com-cert.pem"
  environment:
      http_proxy: http://httpproxy:8080/
      https_proxy: http://httpproxy:8080/

In sslvhost.yml we first create a private key and make sure it has appropriate permissions:

- name: "Generate HTTPS Private Key"
  command: "openssl genrsa -out {{ key_file }} {{ rsa_key_size }}"
  args:
      creates: "{{ key_file }}"

- file:
    path: "{{ key_file }}"
    mode: 0600
    owner: root
    group: root

Now, we need a Certificate Signing Request (CSR). By default, OpenSSL prompts the user for some values when doing so. For Let’s Encrypt, everything except the Common Name is unimportant. Thus we can just supply some dummy values:

- name: Generate CSR
  command: "openssl req -new 
      -key {{ key_file }} 
      -out {{ req_file }}
      -nodes -subj '/C=US/ST=SomeState/L=City/O=Org/CN={{ domain }}'"
  args:
      creates: "{{ req_file }}"

- file:
    path: "{{ req_file }}"
    mode: 0600
    owner: root
    group: root

Now we can request the challenge from Let’s Encrypt, deploy the challenge and retrieve the certificate:

- name: Requesting Lets Encrypt Challenge
  letsencrypt:
    account_email: "{{ letsencrypt_account_email }}"
    account_key: "{{ letsencrypt_account_key }}"
    csr: "{{ req_file }}"
    dest: "{{ cert_file }}"
  register: challenge

- copy:
    dest: "{{ challenge_file_name }}"
    content: "{{ challenge_file_content }}"
    mode: 0644
  when: challenge|changed 
  vars:
      challenge_file_name: "{{ letsencrypt_challenge_dir }}/{{ challenge['challenge_data'][domain]['http-01']['resource'] }}"
      challenge_file_content: "{{ challenge['challenge_data'][domain]['http-01']['resource_value'] }}"

- name: Responding To Challenge and Fetching Certificate
  letsencrypt:
    account_email: "{{ letsencrypt_account_email }}"
    account_key: "{{ letsencrypt_account_key }}"
    csr: "{{ req_file }}"
    dest: "{{ cert_file }}"
    data: "{{ challenge }}"

Of course, this assumes that letsencrypt_challenge_dir is being served by your webserver. Note that challenges are deployed to {{ letsencrypt_challenge_dir }}/.well-known/acme-challenge. For nginx, the following line should do the job:

location /.well-known/acme-challenge/ {
    alias {{ letsencrypt_challenge_dir }}/.well-known/acme-challenge/;
}

Wallabag and zeit.de Paywall

Wallabag supports paywalls. To make the zeit.de paywall work, just add the following snippet to /vendor/j0k3r/graby-site-config/zeit.de.txt:

requires_login: true
not_logged_in_xpath: //aside[@class='gate']
login_uri: https://meine.zeit.de/anmelden
login_username_field: email
login_password_field: pass

Afterwards, add your zeit.de credentials to /app/config/parameters.yaml:

sites_credentials: 
    zeit.de: { username: "you@example.com", password: "hunter" }

Important: This is a YAML file, which only allows spaces, but no tabs, for whatever reason.

Then, under “Internal Settings” > “Article” set “Enable authentication for websites with paywall” to “1”.

After you’ve cleaned your cache, you can now read articles of zeit.de behind a paywall in your Wallabag!

FreshRSS: Better Integration with Wallabag

I’ve started using FreshRSS for my RSS feeds a few days ago. It’s a great software that works pretty well for me. But there is one thing I’ve found a little annoying: I like to go through the headlines of my RSS feeds on my phone and then save the articles I want to read to Wallabag. FreshRSS already supports this through its share function, but it is a little inconvenient. Every time I add an article to Wallabag, it opens a new tab which I have to close. Especially on my mobile, this is annoying.

To solve this problem, I’ve created a small hackish extension for FreshRSS that solves this problem. Instead of opening a link on the client side, it adds the article to Wallabag from the server side using the Wallabag v2 REST API. Thus, no new tab is opened and you are not interrupted while reading through your headlines.

Check out the plugin at https://git.n7r.de/nrb/freshrss-wallabag.

Encrypting an Android phone with a broken USB port

I have an Android phone with a broken USB port: neither the data nor the power lines work. However, I can still charge the battery by removing it from the device and putting it into an external charging device. Apart from that, the device still works absolutely fine and I see no reason to throw it away or risk breaking it by trying to repair it.

I have now decided that I want to encrypt this phone, but Android wanted me to connect it to power, even though the battery was 100% full.

Fortunately, one can force the device encryption from a root shell on the device by invoking:

vdc cryptfs enablecrypto inplace 

The phone will then reboot and start encrypting. Note that if your battery runs out during this process, you will loose data, so make sure to take a backup first!

Controlling your monitor’s brightness from your computer

I usually have to work more than eight hours a day on my computer. Especially in winter when the sun sets early, I often would have to dim the brightness of my monitor in the evening. But because most monitor on screen menus are hard to use, I almost never do that, because I am too lazy. But if I don’t I sometimes get a headache. So I thought, it must be possible to somehow control the brightness of my monitor without using the crappy menu. And it turns out that this is actually possible. Many monitors support DDC, which should in theory allow you to adjust their brightness. For Linux, there is DDCcontrol, which worked perfectly for my Dell UltraSharp U2412M, altough it does not offically support this monitor.

Unfortunately, DDCcontrol is not in the Ubuntu package repositories. However, on my Ubuntu 14.04 I was able to use the packages from ddccontrol, gddccontrol and libddccontrol0 from 15.04. Additionally, I had to install ddccontrol-db from 10.04. However, keep in mind that using packages from different Ubuntu versions could in theory break your whole system. So be careful, I am not responsible if kittens die.

After that, I had to install i2c-tools and add my user to the i2c group:
sudo apt install i2c-tools && sudo gpasswd -a nico i2c

Then, ensure that the i2c-dev module is loaded on boot by adding it to /etc/modules.

After rebooting your machine, you should be able to use gddccontrol to adjust monitor parameters. On the shell, I was able to adjust the brightness using the following command: ddccontrol -r 0x10 -w "$your_brightness_value" dev:/dev/i2c-7
Keep in mind that the command most likely looks different, depending on your machine and your monitor.

That’s it! Now I can adjust the brightness of my monitor more conveniently!

PC Engines APU: Installing debian

Until now, I only ran pfSense on my PC Engines APU boards. But I now wanted to get one of them running under Debian Linux.

The first challenge is creating a live Linux that starts a serial terminal on boot. I found that this was the easiest using Grml2Usb. I booted Grml on my normal PC and plugged in a USB stick. Then, using a Grml ISO image, I did the following:
grml2usb --fat16 --bootoptions="vga=off" --bootoptions="fb=false" --bootoptions="console=ttyS0,115200n8" grml96-full.iso /dev/sdz1

This formats /dev/sdz1 and creates a Grml stick that automatically starts a serial console.

Then, boot from that stick. Using the awesome grml-debootstrap you can debootstrap a Debian installation on your APU. After that, you have to chroot into your new installation:

mount /dev/sda1 /mnt
mount -t sysfs sys /mnt/sys
mount -t proc proc /mnt/proc
mount -o bind /dev /mnt/dev
chroot /mnt /bin/bash

First, edit /etc/default/grub to make Grub aware of your serial port. To do that, set the following variables there (replacing them if they already exist):

GRUB_CMDLINE_LINUX_DEFAULT="gfxpayload=text fb=false console=ttyS0,115200n8"
GRUB_TERMINAL="serial"
GRUB_SERIAL_COMMAND="serial --speed=115200 --unit=0 --word=8 --parity=no --stop=1"

Then, make Debian spawn a shell on your serial port. Edit /etc/inittab and uncomment the following line:
T0:23:respawn:/sbin/getty -L ttyS0 115200 vt100

Afterwards run
update-grub
and reboot. You should see Grub and then your Debian booting!

Firefox: Disable search and Domain Guessing

I don’t want my firefox to perform searches on google or similar when I don’t explicitly instruct it to do so. This is especially annoying if you enter something like sw01.mgmt.corp.client.com and firefox performs a google search on that because you didn’t configure your DNS correctly.

Go to about:config and set
keyword.enabled = false

But that isn’t enough. If you enter something like internalapp it will complete that to www.internalapp.com… This “feature” is called Domain Guessing. Let’s get rid of that, too:
browser.fixup.alternate.enabled = false

Additionaly, I have set up a few keywords for search engines, so when I enter something like
g test
into the URL bar firefox will do a google search.

Statische DNS-Einträge mit einer Fritzbox

Betreibt man hinter seiner Fritzbox einen Server, möchte man ihn möglicherweise über den DNS-Namen erreichen können. Da man einem Server aber sinnvollerweise eine statische IP-Adresse vergibt, wird diese von der Fritzbox nicht in den DNS eingetragen. In einer Konfigurationsdatei kann man den Host entsprechend hinterlegen.

Wichtiger Hinweis: man sollte die Konfigurationsdateien der Fritzbox nur bearbeiten, wenn man weiß, wie man eine gebrickte Fritzbox wiederbelebt. Ich übernehme keine Haftung für die Richtigkeit dieser Anleitung.

  1. Das DHCP-Lease in der Fritzbox entfernen, falls noch eines vorhanden ist. Das geht in der Weboberfläche über Heimnetzwerk > Netzwerk, indem man auf das rote X beim entsprechenden Host klickt.
  2. Falls noch nicht geschehen, telnet aktivieren und einloggen.
  3. multid stoppen:
    multid -s
  4. Die ar7.cfg bearbeiten:
    nvi /var/flash/ar7.cfg

    in dieser Datei gibt es einen Abschnitt “landevices”, wo man einen Block wie diesen hier einfügen kann (Werte entsprechend anpassen):

    } {                                                                                                     
    		ip = 192.168.178.5;                      
    		name = "server";                            
    		mac = 12:34:56:78:90
    		medium = medium_unknown;                                         
    		type = neightype_unknown;                                       
    		staticlease = no;                                                                                    
    } {
  5. Nach einem Reboot der Fritzbox findet sich ein entsprechender Eintrag im DNS.