Reverse Engineering the I2C Protocol of the Mikrokopter Platform

We bought a Miktokopter Okto XL for our laboratory. I chose the Mikrokopter project because I had in mind, that a few years ago everything was developed in a community driven open source way. Unfortunately, it turned out that things changed and that the source code of recent versions of the Mikrokopter software is closed source, they only provide precompiled firmware images. Shame on you! A short look at older versions of the code probably reveals the reasons why the chose to close the code…

Anyway, our plan is to replace the FlightCtrl by our own Linux-driven Hardware while we want to keep the Brushless controllers. So we had to reverse engineer the I²C protocol of the Brushless Controllers (BL-Ctrl V3). Developer information available on the Mikrokopter Wiki was not very helpful.

This is a short overview of the I²C protocol of the Oktokopter XL V3: We made a capture of the communication FlightCtrl <> BL-Ctrl.

  • Speed: max. 200kHz
  • Addresses: 0x52-0x62
  • Write: Either one Byte (0-255) or two Bytes (0-2048) denote the speed of the motor
  • Read: On read, a controller returns six Byte:
    • 1: Current in 100mA steps
    • 2: Status (see table below)
    • 3: Temperature in degrees Celsius
    • 4: Raw RPM value (some correction factor needed??)
    • 5: Unused / Reserved / Voltage for BL_CTRL V3
    • 6: Voltage in 100mV steps

Possible status values that we observed:

  • 255: running
  • 248: not running but ready

There are more possible status values but we did not observe them yet.

After Digging a bit deeper, we found out, that it is not possible to directly jump from 0 RPM to maximum RPM. Speed has to increase ‘slowly’ in smaller steps. Furthermore, the motor settings must consequently be transmitted, even if they do not change. If a controller doesn’t receive I²C signals any longer, it automatically stops motors after ~500ms.

AVR-Toolchain for Gentoo Linux using crossdev

Since quite a long time, Gentoo Linux has problems building a GCC Toolchain for the AVR microcontroller architecture. This is just a short summary, how to get your toolchain running under Gentoo:

1. Emerge Toolchain

USE="-openmp -sanitize -vtv" crossdev -t avr -s4 -S --without-headers

2. Somehow, ldscripts go to /usr/x86_64-pc-linux-gnu/avr/lib/ldscripts but are actually expected to be located at /usr/avr/lib/ldscripts. A symlink quick fixes this issue

ln -s /usr/x86_64-pc-linux-gnu/avr/lib/ldscripts /usr/avr/lib/ldscripts

3. Have fun with your working toolchain

Running jailhouse on a NVIDIA Jetson TK1 with Gentoo from scratch

TL;DR: You can find latest kernel configs, pre-built kernel images, u-boot configs and pre-built u-boot binaries as well as a pre-built gentoo linux image here. I try to keep everything up to date:

  • Kernel: 4.9 (stable)
  • U-Boot: 2016.05-rc1
  • Gentoo RootFS: 2017-08-23 (with systemd)

Contents


Compile everything from scratch

Prerequisites

  • Gentoo Host
  • ARMv7 Compiler
  • Jetson TK1 board
  • Serial cable (for jailhouse output and general purpose debugging)
  • Micro USB cable

On your local gentoo box, you can install the cross-compiler with crossdev.

$ emerge crossdev
$ crossdev -S -v -t armv7a-hardfloat-linux-gnueabi

Prepare Gentoo Base System

On your gentoo box, create a workspace

$ mkdir /tmp/jetson
$ cd /tmp/jetson

Inside this directory, download and unpack the latest Gentoo Stage 3. You may also want to download the latest portage snapshot to usr/.

$ curl http://gentoo.oregonstate.edu/releases/arm/autobuilds/current-stage3-armv7a_hardfp/stage3-armv7a_hardfp-20150721.tar.bz2 | sudo tar -xvjp -C .
$ curl http://gentoo.oregonstate.edu/snapshots/portage-latest.tar.bz2 | sudo tar -xvjp -C usr

Adjust your Gentoo make.conf. You probably want to add something like

$ sudo vim etc/portage/make.conf
>> MAKEOPTS="-j5"

Get a salted hash of your root password using openssl and write it to /etc/shadow

$ openssl passwd -1 gentoo
$1$xQl51XUf$.eUKt5EynZu4OWQnnIdZf.
$ sudo vim etc/shadow
root:$1$xQl51XUf$.eUKt5EynZu4OWQnnIdZf.:10770:0:::::

Adjust the etc/fstab. The following line should remain the only one.

/dev/mmcblk0p1 / ext4 noatime 0 1

Change the speed of the serial console /dev/ttyS0 to 115200 Baud (later we have to comment out this line for using jailhouse)

$ sudo vim etc/inittab
s0:12345:respawn:/sbin/agetty -L 115200 ttyS0 vt100

Never forget to think about a nice hostname and to set the timezone

$ echo hostname=\"iridium\" > etc/conf.d/hostname
$ ln -sf usr/share/zoneinfo/Europe/Berlin etc/localtime

We will use auto-DHCP for our networking interface

$ ln -sf net.lo etc/init.d/net.enp1s0
$ echo config_enp1s0=\"dhcp\" > etc/conf.d/net

Time to send out RootFS to the Jetson TK1. A pretty nice feature of u-boot allows us to mount the eMMC of the Jetson as USB mass storage device. Connect the Mini-USB to your machine, push the reset button your jetson, interrupt the bootloader and type:

Hit any key to stop autoboot: 0
Tegra124 (Jetson TK1) # ums 0 mmc 0
UMS: LUN 0, dev 0, hwpart 0, sector 0x0, count 0x1d5a000

Now the whole eMMC of your Jetson is available as a mass storage device. I recommend to create one single huge ext4 partition using gparted. After creating the partition, mount it and copy the root file system:

mount /dev/sdX1 /mnt/temp/
cp -av /tmp/jetson/* /mnt/temp/
sync

Take a cup of coffee or tea.

Compile your own kernel

As your Jetson is unable to boot yet, you have to cross-compile the kernel on your local machine.

$ emerge gentoo-sources
$ cd /usr/src/linux
$ make ARCH=arm menuconfig # Adjust everything you need
$ make ARCH=arm CROSS_COMPILE=armv7a-hardfloat-linux-gnueabi- -j 5
# Copy the kernel
$ cp arch/arm/boot/zImage /mnt/temp/boot/
# Copy the device tree block
$ cp arch/arm/boot/dts/tegra124-jetson-tk1.dtb /boot
# Install all modules
$ make modules_install INSTALL_MOD_PATH=/mnt/temp/
# Sync file system and unmount the Jetson
$ sync
$ umount /mnt/temp

Next step is to place a file called /boot/boot.scr. It’s pretty similar to GRUB’s menu.cfg. U-Boot reads this file when booting and executes its commands. Unfortunately, U-Boot uses a binary format and it must be compiled first. But first of all, install the u-boot-tools:

$ emerge u-boot-tools

Create the file boot.script that contains the following content:

# Optionally add 'init=/usr/lib/systemd/systemd', if you want to use systemd insteas of OpenRC
setenv bootargs 'console=ttyS0,115200 root=/dev/mmcblk0p1 rw rootwait'

#Comment the line above and uncomment these lines for running jailhouse
#setenv bootargs 'root=/dev/mmcblk0p1 rw rootwait mem=1984M vmalloc=512M'
#setenv bootm_boot_mode nonsec

load ${devtype} ${devnum}:1 ${kernel_addr_r} /boot/zImage
load ${devtype} ${devnum}:1 ${fdt_addr_r} /boot/tegra124-jetson-tk1.dtb
bootz ${kernel_addr_r} ${ramdisk_addr_r}:${filesize} ${fdt_addr_r}

Use the mkimage tool of the u-boot-tools to compile it to a .scr file:

$ cd /tmp/jetson/boot
$ wget https://ramses-pyramidenbau.de/~ralf/jetson-tk1/boot.script
$ mkimage -A arm -O linux -T script -C none -a 0 -e 0 -n Boot-Script -d boot.script boot.scr

After rebooting, make sure that all CPUs are started in HYP mode:

$ root@iridium:~# dmesg | grep HYP
[ 0.101989] CPU: All CPU(s) started in HYP mode.

Here you can also download a prebuilt boot.scr. Copy the boot.scr file to the /boot directory of your jetson.

Compile U-Boot

Jailhouse requires the processor to run in non-secure HYP mode. The processor needs to be staged to HYP mode very early by the bootloader. HYP is not enabled in NVIDIA’s stock bootloader, so we need to compile and flash it on our own.

It took me quite some time to find out how to build the bootloader correctly.
Here’s the summary 🙂 Run the following commands:

$ mkdir /tmp/uboot
$ cd /tmp/uboot
# Get all the required stuff
$ git clone https://github.com/NVIDIA/tegra-uboot-flasher-scripts.git
$ git clone https://github.com/NVIDIA/tegrarcm.git
$ git clone https://github.com/NVIDIA/cbootimage.git
$ git clone https://github.com/NVIDIA/cbootimage-configs.git
$ git clone git://git.denx.de/u-boot-tegra.git u-boot

# build tegrarcm
$ cd tegrarcm
$ ./autogen.sh
$ make
$ cd ..

# build cbootimage
$ cd cbootimage
$ ./autogen.sh
$ make
$ cd ..

# build u-boot
$ export PATH=$PATH:/tmp/uboot/cbootimage/src:/tmp/uboot/tegrarcm/src
$ cd tegra-uboot-flasher-scripts
$ export CROSS_COMPILE=armv7a-hardfloat-linux-gnueabi-
$ ./build --socs tegra124 --boards jetson-tk1 build

Et voilà, you just built your own U-Boot bootloader. Let’s flash it! Before sending the next command, bring your device in recovery mode by pressing the “RESET” + “FORCE RECOVERY” buttons.


$ cd ../tegra-uboot-flasher-scripts
$ ./tegra-uboot-flasher --data-dir ../_out flash jetson-tk1

With a bit of luck and some magic unicorn dust, your jetson will directly boot your custom bootloader that boots your custom kernel. After rebooting, you can check if HYP is enabled with

$ root@iridium:~# dmesg | grep HYP
[ 0.101989] CPU: All CPU(s) started in HYP mode.

First Boot

After your device was successfully flashed it will automatically reboot. You can now login to it by using a serial connection.

$ putty -serial /dev/ttyUSB0 115200 -sercfg 115200

As we don’t have NTP yet, our first step will be to set the current time.

On your TK1 login and type:

$ date --set="20150723 18:37"
$ ####
$ # Your network should be autoconfigured via DHCP, if not try
$ ifconfig enp1s0 a.b.c.d
$ route add default gw a.b.c.e
$ echo nameserver 8.8.8.8 > /etc/resolv.conf
$ ####
$ emerge --sync
$ emerge -av ntp
$ rc-update add ntp-client default
$ rc-update add sshd default
$ # Perform a full system upgrade
$ emerge -uDNav world


Compile Jailhouse

This will guide you how to compile Jailhouse on your Jetson TK1. Jailhouse comes with some dependencies and requires the mako python package in order to compile. Install mako as well as git before cloning and compiling jailhouse.

$ emerge git mako
$ git clone https://github.com/siemens/jailhouse.git
$ cd jailhouse
$ cat > hypervisor/include/jailhouse/config.h # Create this file with the following content
#define CONFIG_ARM_GIC_V2 1
#define CONFIG_MACH_TEGRA124 1
$ make
$ make install
$ depmod -a
$ modprobe jailhouse
$ lsmod
# and check if the module was successfully loaded


Playing around with jailhouse

This tutorial will give you a brief introduction on how to run a simple uart-demo inmate in jailhouse on the TK1. It is important not to use the UART in the root cell as it will be assigned to the non-root cell after creating it. Any further UART access from the root cell after creating the non-root cell will lead to a panic of the hypervisor. Additionally, jailhouse needs some memory for the hypervisor itself and its inmates that is not used by the kernel of the root cell. This results in a boot.script as follows:

# Reserve 64MiB for jailhouse
setenv bootargs 'root=/dev/mmcblk0p1 rw rootwait mem=1984M vmalloc=512M'
# This enables HYP mode (disabled by default)
setenv bootm_boot_mode nonsec
load ${devtype} ${devnum}:1 ${kernel_addr_r} /boot/zImage
load ${devtype} ${devnum}:1 ${fdt_addr_r} /boot/tegra124-jetson-tk1.dtb
bootz ${kernel_addr_r} ${ramdisk_addr_r}:${filesize} ${fdt_addr_r}

Compile the boot.script using mkimage to boot.scr and copy it to ‘/boot/’.

$ mkimage -A arm -O linux -T script -C none -a 0 -e 0 -n Boot-Script -d boot.script /boot/boot.scr

Also, comment out the serial console from ‘/etc/inittab’. Use ssh to access your device. After rebooting your device, you can load the jailhouse module and activate the hypervisor:

$ modprobe jailhouse
$ jailhouse enable ~/jailhouse/configs/jetson-tk1.cell

Dmesg should now tell you, that the jailhouse has opened:

$ dmesg | tail
[...]
[ 183.608569] The Jailhouse is opening.

On your serial console, you should see something like:

Initializing Jailhouse hypervisor v0.5 (132-g997802f) on CPU 0
Code location: 0xf0000020
Page pool usage after early setup: mem 22/16112, remap 64/32768
Initializing processors:
CPU 0... OK
CPU 3... OK
CPU 2... OK
CPU 1... OK
Page pool usage after late setup: mem 34/16112, remap 64/32768
Activating hypervisor

Now you can create a new non-root cell using the jetson-tk1-demo cell configuration:

$ jailhouse cell create ~/jailhouse/configs/jetson-tk1-demo.cell

Now that your cell is created, you can load a binary to it and start the cell:

$ jailhouse cell load jetson-tk1-demo ~/jailhouse/inmates/demos/arm/uart-demo.bin -a 0
$ jailhouse cell start jetson-tk1-demo

On your serial console, you can now see the output of the demo cell.

To stop and destroy the cell, just type:

$ jailhouse cell shutdown jetson-tk1-demo
$ jailhouse cell destroy jetson-tk1-demo

Use my precompiled RootFS + Bootloader

If you don’t want to compile everything on your own – which i truly can understand – then you can use my pre-compiled images.

I try to keep those images up to date. Look at the top of the page for the current versions. SSH server is enabled, root password is ‘gentoo’, dhcp client on wired ethernet. So just a few steps for you to start.

Reset your Jetson, interrupt the bootloader and attach it as USB mass storage device (explained above) by typing

Tegra124 (Jetson TK1) # ums 0 mmc 0

to your serial console. Mount and partition your Jetson and extract the root file system:

$ mount /dev/sdX1 /mnt/temp/
# Extract Root FS
$ curl https://ramses-pyramidenbau.de/~ralf/jetson-tk1/rootfs/RootFS-latest.tar.gz | tar x -C /mnt/temp/
# Extract pre-compiled kernel
$ curl https://ramses-pyramidenbau.de/~ralf/jetson-tk1/kernel/precompiled/latest.tar.bz2 | tar x -C /mnt/temp/
# Place boot.scr (non-jailhouse variant)
$ curl https://ramses-pyramidenbau.de/~ralf/jetson-tk1/u-boot/boot.scr > /mnt/temp/boot/boot.scr
# Place boot.scr (jailhouse variant)
$ curl https://ramses-pyramidenbau.de/~ralf/jetson-tk1/u-boot/boot.scr.jailhouse > /mnt/temp/boot/boot.scr

Next step is to flash the correct bootloader. On your local machine, type

$ git clone https://github.com/NVIDIA/tegra-uboot-flasher-scripts.git
$ git clone https://github.com/NVIDIA/tegrarcm.git
$ cd tegrarcm
$ ./autogen.sh
$ make
$ export PATH="$PATH:`realpath src`"
$ cd ../tegra-uboot-flasher-scripts
$ curl https://ramses-pyramidenbau.de/~ralf/jetson-tk1/u-boot/jetson-tk1-uboot-2016.05-rc1.tar.bz2 | tar -xzv -C .
$ ./tegra-uboot-flasher --data-dir . flash jetson-tk1

Et voilà. Reboot your device and check if HYP is enabled if you chose the jailhouse variant:

root@iridium:/# dmesg|grep HYP
[ 0.101987] CPU: All CPU(s) started in HYP mode.

Surpress compiler warnings of third party libraries

I like my compiler to be very verbose and pedantic. For me, -Wall -Wextra -pedantic -Weffc++ is a must. And I don’t want to see any warnings before releasing my software.

Well, but your compiler might still complain about headers of third party libraries which you do not want to touch. How to remove those annoying warnings?

Here’s the simplest solution. Just add

#pragma GCC system_header

to the affected header file and you will not be disturbed by those warnings any more.