FreeBSD on the Intel 10th Gen i3 NUC

I have ended up with some 10th Gen i3 NUC's ( NUC10i3FNH to be specific) to put to work in my testbed. These are quite new devices, the build date on the boxes is 13APR2020 . Before I figure out what their true role is (one of them might have to run linux) I need to install FreeBSD -CURRENT and see how performance and hardware support is.

They have an Intel i3-10110U with 2 cores and 4 threads at 2ish GHz (with 4GHz boost), I got a single 32GB DIMM of RAM for each and a 480GB Western Digital M.2 SSD. This configuration came in just under £500 for each NUC.

The NUCs are pretty small, they have pretty beefy fans taking up about a cm of the top of the enclosure. They certainly aren't silent, without any load I could hear the NUC sat on the desk next to me. When building at full steam the fan in the NUC is about as loud as my x270 Thinkpad is when it is building.

What works?

Out of the box I had to break into the bios and disable secure boot to boot the FreeBSD installer. I did this by hitting every FN key as the NUC booted, I think FN2 was the correct choice. At the time my keyboard was being fought over by USB and Bluetooth on my MacBook Pro.

FreeBSD install was problem free. I set up the M.2 as a single drive with ZFS, datasets and snapshots are a magic power.

Before I tried anything else I had to get an idea at how well this NUC would build FreeBSD. I don't expect this to be a build machine, but having spent a while shopping for build machines recently (I settled on a VM in hetzner) it is the only benchmark I really care about. make -j4 buildworld buildkernel took 2:45, that isn't the fastest in the world, but it is about an 1:15 faster than my x270 with its 2015 i5-6300U . The difference 5ish years makes.

Graphics in this 10th Gen Intel processor wasn't supported by drm-current-kmod and I had to install drm-devel-kmod . With the devel kmod the NUC is happy to push all the pixels of the 4k TV I have here and even drive additional second monitor connected with USB-C->HDMI adapter at the same time.

Audio works through the front 3.5mm jack and after changing hw.snd.default_unit from HDMI too. There is an issue where when the display is blanked (turned off), audio will stop, but won't come back when the display does. manu@ suggested setting hw.i915kms.disable_power_well=0 in loader.conf , this resolved the problem for me.

The Intel 9462 WiFi in the NUC is not supported (yet), I stuck a cheap rtwn based USB WiFi dongle in the front port. External USB WiFi is cumbersome on a laptop, but on a machine that will rarely move it is fine. The onboard Ethernet is supported by em and is happy to do 940ishMbit/s with iperf3 with TCP and UDP.

The NUC has 5 USB ports, 3 USB-A and two USB-C. The USB-C port on the back has a little lightning bolt next to it and I imagine that means it is the one with thunderbolt support. I did a disk speed test to a Sandisk Extreme Portable SSD. The front port managed 200MB/s or so while the rear thunderbolt one only got to about 160MB/s for read and write. My MacBook Pro manages the advertised 500MB/s read speeds, so I need to dig into whether this is a hardware problem or a FreeBSD problem. Luckily I have two so installing Linux to test performance won't be a bother.

I quite like these little boxes, you can get more hardware for your money with a tower pc, but not in this form factor. I wouldn't be surprised if the i7 versions of these were quite good at building FreeBSD. The fans are a bit loud, but that can be fixed for me by playing blaring techno.

I think the NUC in this configuration is going to make a great test node in my satellite testbed or a nice little desktop box for doing video tasks.

Hardware Support

Storage         Yes     M.2 is fine, I didn't have a drive to test SATA
Graphics        Yes     drm-devel-kmod (on r362612)
Audio           Yes     hda(4)
Ethernet        Yes     Supported by em(4)
Wireless        No      Intel AC 9462 not supported :(
USB             Yes     All ports work, 
Suspend/resume  Yes     suspend is fine, resume required drm-devel-kmod
SD slot         Yes     Comes up as a mmcsdX device
Thunderbolt     N/A     I have no idea what the implications of this are or how to test

Capturing a screen sub section with ffmpeg

I tooted me scrolling through the browser tabs I opened from the Black Producers bandcamp list . I used ffmpeg to to grab a subsection of the screen with this command:

ffmpeg -f x11grab -show_region 1 -framerate 10 -video_size 720x480+100,200 -i $DISPLAY scroll.webm
  • video_size sets the size and offset in the window (needed if grabbing a subsection)
  • show_region paints a box around the active part, this helps dial in what is captured.

It took me a while to figure out what the input (-i) should be, in the end I figured out that it needs to be the X display, the easiest way to grab this is from the environment variable $DISPLAY .

I first made mp4's from the capture, but they were coming out all garbled, firefox refused to play and vlc was pretty sad. I move to webm and the capture is smaller and works in firefox.

Help me blog more in June

This is a post on my blog.

I both have a blog and enjoy blogging. I think a blog is the perfect way to keep notes for myself with the nice possibility that they might help someone else. I frequently look up my own post on how to take screenshots with imagemagick on this blog.

I wrote a blog post everyday for 6 months, from the 26th September 2016 to the 27th of March 2017 , in the end I added 196 new entries to this blog in 182 days. I can't write an epic detailed post everyday, though I might manage a week or so like that if I have a backlog. Most posts were little more than an image and some text, little more than tweets.

But I still wrote, on a few days I even wrote more than one blog post. Under each blog post I included where in the world I was (borrowed from a characters quirk in Cryptonomicom), the weather and the books I was reading at the time.

These posts cover a range, pictures I was proud of , planning for events , travel and cool projects I worked on . Reviewing these has been a fun experience.

All that said, I haven't published a blog post for 276 days and I think I would like some help getting back into the public writing groove.

I tend to be better at delivering projects when I am held accountable by other people. Think of it as peer help rather than peer pressure.

I think it would be really cool if you too wrote some blog posts. Not for me, not for anyone else, but for you. You today and in the future. Write something now to clear your head, write something today that will help you tomorrow. Writing things down helps you remember them, but when you forget you will get to try searching your own log rather than having to scour the depths of the Internet.

I asked this weekend at campgndd for some peer help so that I would actually write blog posts in June. I suggested we each try to write four this month.

These kind people said they would try and write too to keep my going:

It would be cool if you were to join us. I will look for your response on your blog.

FreeBSD on the NanoPi NEOLTS

The NanoPi NEOLTS is a SBC from FriendlyElec that uses the Allwinner H3 SOC. The NanoPi NEOLTS has a nice selection of hardware including 100Mbit Ethernet, 3 USB Ports and a bunch of exposed GPIO.

FreeBSD on the NanoPi uses GENERICSD image. This image requires a bootloader to be added before it will work. We can prepare a single image to be copied to many SD cards by using a memory disk as an intermediate step.

We need to:

  • Get the latest GENERICSD card image snapshot
  • Install the correct boot loader pkg
  • Create a memory disk
  • Copy the GENERICSD image to memory disk
  • Copy the bootloader to the memory disk
  • Mount the root partition of the sd card image
  • Copy the programs and files we need for the tutorial to the sd card

The latest image is as I write is 13 CURRENT from 20190829:

$ fetch ftp://ftp.freebsd.org/pub/FreeBSD/snapshots/arm/armv7/ISO-IMAGES/13.0/FreeBSD-13.0-CURRENT-arm-armv7-GENERICSD-20190829-r351591.img.xz

We have to decompress the image before we can use it

$ xz -d FreeBSD-13.0-CURRENT-arm-armv7-GENERICSD-20190829-r351591.img.xz

Each u-boot bootloader platform has its own package, currently there are 46 different bootloaders in the FreeBSD ports system. We want the u-boot for the nanopi_neo (our target).

$ pkg search nanopi     
u-boot-nanopi-neo2-2019.07     Cross-build das u-boot for model nanopi-neo2
u-boot-nanopi_a64-2019.07      Cross-build das u-boot for model nanopi_a64
u-boot-nanopi_m1plus-2019.07   Cross-build das u-boot for model nanopi_m1plus
u-boot-nanopi_neo-2019.07      Cross-build das u-boot for model nanopi_neo
u-boot-nanopi_neo_air-2019.07  Cross-build das u-boot for model nanopi_neo_air

# pkg install u-boot-nanopi_neo-2019.07

The u-boot-nanopi_neo package contains the binary bootloader we need in u-boot-sunxi-with-spl.bin

$ pkg info -l u-boot-nanopi_neo-2019.07

u-boot-nanopi_neo-2019.07:
    /usr/local/share/licenses/u-boot-nanopi_neo-2019.07/GPLv2
    /usr/local/share/licenses/u-boot-nanopi_neo-2019.07/LICENSE
    /usr/local/share/licenses/u-boot-nanopi_neo-2019.07/catalog.mk
    /usr/local/share/u-boot/u-boot-nanopi_neo/README
    /usr/local/share/u-boot/u-boot-nanopi_neo/boot.scr
    /usr/local/share/u-boot/u-boot-nanopi_neo/metadata
    /usr/local/share/u-boot/u-boot-nanopi_neo/u-boot-sunxi-with-spl.bin

With the GENERICSD image and the bootloader we need to create the memory disk image we will use for staging. First we need to create a large enough backing file.

$ truncate -s 8G nanopi.img
# mdconfig -f nanopi.img
md0

Now we can dd the GENERICSD image to the memory disk

# dd if=FreeBSD-13.0-CURRENT-arm-armv7-GENERICSD-20190829-r351591.img of=/dev/md0 bs=1m

We need to dd the bootloader to the start of the SD card, i.e. the entire device and not a partition.

# dd if=/usr/local/share/u-boot/u-boot-nanopi_neo/u-boot-sunxi-with-spl.bin of=/dev/da0 bs=1k seek=8 conv=sync

With the memory disk attached we can interact with the image file as if it were a real USB drive or SD card.

$ gpart show md0
=>      63  16777153  md0  MBR  (8.0G)
        63      2016       - free -  (1.0M)
      2079    102312    1  fat32lba  [active]  (50M)
    104391   6187041    2  freebsd  (3.0G)
   6291432  10485784       - free -  (5.0G)

We can mount the root partition of the SD card and modify or add any files we wish:

# mount /dev/md0sa mnt

When we are done changing things we have to disconnect the memory disk:

# sudo mdconfig -d -u md0

Finally we can copy the memory disk to a real sd card using dd :

# sudo dd if=nanopi.img of=/dev/da0 bs=1m

My FreeBSD Development Setup

I do my FreeBSD development using git , tmux , vim and cscope .

I keep a FreeBSD fork on my github, I have forked https://github.com/freebsd/freebsd to https://github.com/adventureloop/freebsd

On my fork I have the freebsd/freebsd repo set as an upstream

$ git remote -v
origin  git@github.com:adventureloop/freebsd.git (fetch)
origin  git@github.com:adventureloop/freebsd.git (push)
upstream        https://github.com/freebsd/freebsd.git (fetch)
upstream        https://github.com/freebsd/freebsd.git (push)

See this article for information on setting this up https://help.github.com/en/articles/configuring-a-remote-for-a-fork

I do all work on branches using worktrees, keeping the master branch clean.

Periodically I sync the master branch with the FreeBSD upstream:

$ cd ~/code/freebsd-dev/freebsd-git
$ git checkout master
$ git fetch upstream
$ git merge upstream/master
$ git push

I have a development setup based on Ian Lapore's arm set up documented on the FreeBSD wiki https://wiki.freebsd.org/FreeBSD/arm/crossbuild

I have a freebsd-dev directory in my code directory. It their I keep a copy of FreeBSD in freebsd-git , and obj directory for build output and a projects directory for in progress code.

$ tree -L2
.
├── diffs
│   ├── D15222.diff
│   └── old
├── dstdir
│   ├── boot
│   ├── METALOG
│   └── usr
├── freebsd-git
│   ├── bin
│   ├── sbin
...
│   └── usr.sbin
├── obj
│   └── usr
├── projects
│   ├── atomicstats
│   ├── axp288
│   ├── bugsquash
│   ├── byebyejumbograms
...

I use git worktrees for ongoing projects. git worktrees allow you to have a shallow file system copy on a git branch in a directory.

When starting a new project I do something like:

$ cd ~/code/freebsd-dev/freebsd-git
$ git worktree add thj/newdevelopment ../projects/newdevelopment master
$ cd ../projects/newdevelopment

Once the worktree is set up I launch a tmux session in the projects directory. Each random idea or itch I have, if there is enough there, ends up with a project worktree and a tmux session.

tmux allows me to have many windows in a session, I have a serious tmux problem. Right now I have 11 sessions with 42 windows across them. This is a good indicator of my focus level.

I do FreeBSD development with cscope and vim . With tmux splits I normally have an open file and I use other cscope instances in tmux windows to search for things I need in the tree.

I do testing in a bhyve vm and leave the serial port in a tmux window somewhere. I follow the setup in the FreeBSD handbook and back each vm with a zfs dataset.

I do FreeBSD kernel builds using a command like:

env MAKEOBJDIRPREFIX=/home/tom/code/freebsd-dev/obj make -j 44 buildkernel \
        -DKERNFAST installkernel \
    -DNO_ROOT DESTDIR=/home/tom/code/freebsd-dev/dstdir

I then ship kernels to the test vm with scp. jhb@ has a nicer method using the bhyve-loader , but I am yet to try it.

When changes are maturing I create reviews for them using arcanist, manu@ has a good article on doing this

previous next