Quick and Dirty Network Scanning

Ever want to scan a subnet in the nosiest, least reliable way and generate too many processes while doing so? Yes? Well do I have a script for you:

#!/bin/sh

default=172.20.10

if [ -z $1 ]
then
        prefix=$default
else
        prefix=$1
fi

pinghost ()
{
        ping -t 1 -c 1 $1 > /dev/null
        if [ $? -eq 0 ]
        then
                echo hit $1
        fi
}

for x in `jot 254`
do
        pinghost $prefix.$x &
done

I wrote this while I was doing hack the box challenges and it was a fun and quick way to look to actually find things on my test network. I do not recommend using this. Some operating systems won't let you run it twice in succession as it generates a lot of processes.

My Streaming Setup

In April and May I did some streaming on twitch of hardware hacking projects . I started this as a way to work through the material for my cancelled BSDCan hardware hacking tutorial. With the tutorial cancelled I have been left with quite a few NanoPi Neo LTS and I was thinking about doing the tutorial as a series of videos with the idea of selling intro kits with the boards.

So far I have done four streams aiming for about an hour for each. I will say now that I haven't streamed again in June, I am not saying it is the end, but I (and I bet you too) need to control my commitments or I just never get anything done.

I have also been asked to write up my streaming set up so other people can use it. This vanity pic I tooted seems to include most of it.

Equipment

I have a bunch of equipment because I want to stream stuff in the real world. If you just wanted to share some windows and your webcam as Kristof Provost does then you can get by just with your laptop.

This is quite a lot of stuff, but other than the Aputure light I had all of it already from other projects. I would like to be able to capture the display of my work oscilloscope, thankfully the scope has VGA out so it is just a matter of figuring out a way to capture.

Software

I am using Open Broadcast Studio (OBS) on macOS to mix video and feed it to twitch.

OBS supports a rich variety of input sources that can mixed to make a scene. I have been using three scenes to make it easy to setup what is streamed in advance:

  1. Everything
  2. Just the camera
  3. Just the presentation

Everything is the mixed view of my terminal, the camera output and the slides in a web browser. My tutorial slides are derived from my EuroBSDCan Tutorial and render in a web browser natively. Having a browser in the video mix quickly turned out to be really helpful.

OBS has been rock solid and I have had no problems with stability while streaming. The interface is a bit of a maze, but I suspect that is a natural result of the power it offers.

Canon Camera

For most of the top down shots I used a canon 600D with a 18-55mm kit lens. The battery in this thing is quite old and only manages about an hour and a half of video output. I have since gotten a USB powered battery insert that should allow me to run forever, but am yet to try streaming with it.

My canon camera does not support acting as a webcam, but gphoto2 does support grabbing live video from it. This should have been easy to feed into OBS, but I couldn't get it working and ended up instead using ffplay to render the video and grabbed that with a video capture. To get video from gphoto2 into ffplay I ran:

gphoto2 --stdout --capture-movie | ffplay -

This turned out to be very stable and easy to set up. There is quite high latency between the capture output and the gphoto2 capture, but it worked fine if I didn't have to move the camera much.

Nikon Camera

I also have a Nikon J1 and while the camera is awful to use I do have a 10mm lens for it with a macro extender. This allows me to take very high detail pictures of PCBs. I wanted to add this to the streaming set up too.

The Nikon firmware is garbage and while gphoto2 does support the camera I found I could only get a single image from the camera before having to reset it and was completely unable to get video from it. Never mind, my 2018 EuroBSDCon Talk used a HDMI network extender as a capture device. In the stream Crimes Against Computers3 I did something awful with a BeagleBone Black and got video from the Nikon through the HDMI extender into OBS.

This was a lot of faff to setup (though I since did get a USB-C Ethernet for the MacBook) and I am still kind of dubious of the stability of the LKV373 so I have only used this in the stream where I figured it out. I might try this again later.

Since doing that stream cheap USB2 and USB3 HDMI capture devices have appeared. These are supposed to appear as UVC webcams to the system. I suspect these will be a better method than the LKV373, but I want to try before recommending them.

Audio

For audio I am using a Zoom H3 dictaphone thing. I really like the audio from the Zoom, but I seem to be in the minority. In the end it is the microphone I have.

It is stereo which should be mixed down or you will drift across the channels when you move your head. OBS supports this so it wasn't a problem, but I did have to be told about it.

The firmware for this thing isn't so great either, it cannot record to the Micro SD card while acting as a USB interface and Annoyingly there is a bug where it defaults to 44KHz for the audio. The USB driver doesn't seem to advertise this to the properly and if you continue with the default you get weird audio. I dealt with this before in FreeeBSD , but was surprised to see it appear in macOS. You should always double check your audio before starting a stream.

Why not stream from FreeBSD?

I wanted a pain free approach where I could set up and go for my first streams so while OBS is ported to FreeBSD I expected it to be a lot more work than the much more common macOS/OBS combination. I expect FreeBSD/OBS support to get better if people continue streaming FreeBSD stuff.

The MacBook is similar specs to my Thinkpad, but the wireless card in the Thinkpad doesn't have great support and only manages 80211g rates. That might just be enough to stream out the video, but it seems very risky to me.

I will look at streaming with OBS from FreeBSD in the future.

The streams have done okay so far, there is a big social media boost network of FreeBSD users and developers and that has helped people find the content. I am sure the next stream will have fewer viewers as there has been a gap in my streaming.

You can support this

Finally, I set up a ko-fi page to support me writing and streaming. I plan to write more this year and if there is interest I will do more streams too. The streams so far have mostly been using equipment and parts I already had. Doing more is going to require me spending money.

If you have enjoyed my streams, or if you just want me to do more a very solid signal of your support would be tipping me a cup of coffee or something through ko-fi. You can support my hardware habbit here .

Command Line Bug Hunting in FreeBSD

FreeBSD uses bugzilla for tracking bugs, taking feature requests, regressions and issues in the Operating System. The web interface for bugzilla is okay, but if you want to do a lot of batch operations it is slow to deal with. We are planning to run a bugsquash in July and that really needs some tooling to help any hackers that show up process the giant bug list we have.

Thankfully there is a python3 command line tool for interacting with bugzilla, called pybugz . bugz allows you to search through, up date and modify bugs without having to use a web browser. Getting bugz going was not very intuitive and it took me a bit of faffing.

bugz ships with a configuration for connecting to the FreeBSD bugzilla you use it by selecting it as a connection . The supported connections are dumped out if you try and do an operation with the -d 3 debug flag. This flag is really helpful for figuring out how to use bugz because while it documents itself, it holds on to the documentation like a powerful secret.

bugz really wants you to authenticate before you do anything, it won't show you help for commands without auth. There is however a --skip-auth flag. With this you can search for bugs, lets look for ipv6 issues with a patch in the base system:

$ bugz --connection FreeBSD --skip-auth search --product "Base System" "patch ipv6"
 * Info: Using [FreeBSD] (https://bugs.freebsd.org/bugzilla/xmlrpc.cgi)
 * Info: Searching for bugs meeting the following criteria:
 * Info:    product              = ['Base System']
 * Info:    status               = ['New', 'Open', 'In Progress', 'UNCONFIRMED']
 * Info:    summary              = ['patch ipv6']
88821 bugs                 [patch] IPv6 support for ggated(8)
186133 bugs                 [patch] tcpdump(1): zero checksums are invalid for UDP over IPv6
174225 bugs                 [network.subr] [patch] add support for ipv6_addrs_IF style aliases to rc.conf(5)
178881 bdrewery             [patch] getifaddrs(3) does not report IPv6 addresses properly in 32-bit compatibility mode
180572 rc                   [network.subr] [patch] SLAAC is enabled for ipv6_cpe_wanif
133227 bugs                 [patch] whois(1): add support for SLD whois server lookups and IPv6 address lookups
104851 bugs                 [inet6] [patch] On link routes not configured when using both IPv6 autoconfiguration and manual configuration
147681 bugs                 [network.subr][patch] Add inet6 keyword if it wasn't specified in ifconfig_IF_ipv6
130657 bugs                 [ip6] [patch] ipv6 class option
165190 bugs                 [ipfw] [lo] [patch] loopback interface is not marking ipv6 packets
245103 bz                   [patch] [ipv6] IPv6: update v6 temporary address lifetime according to rfc4941bis
 * Info: 11 bug(s) found.

We can also filter our search by component:

$ bugz --connection FreeBSD --skip-auth search --product "Base System" --component bhyve

bugz supports modifying and updating bugz from the command line, this is the main focus of the README on github . To authenticate bugz takes a username and password on the command line, I am not suggesting you fill your history with your bugzilla password, I did something like:

$ bugz --connection FreeBSD -u thj@freebsd.org --password `pass show FreeBSD/bugz | head -n 1` search udp

There is a -k flag that takes a key file, but I didn't want to dig into the source of bugz to figure out what this actually is.

Our bug squash is probably going to focus on clearing bugs with patches and the readme has a workflow for finding bugs and grabbing any diffs. More tools can and should be written around bugz this is just a start. Just playing with this I have spotted bugs than can easily be closed from the tracker.

Finally, you are going to need the help while using bugz , it took me longer than I liked to figure out that each sub command documents its own help and they all take the -h after the command. You need auth (or to skip auth) before you can use this flag.

$ bugz --connection FreeBSD --skip-auth search -h

FreeBSD on the Intel 10th Gen i3 NUC

I have ended up with some 10th Gen i3 NUC's ( NUC10i3FNH to be specific) to put to work in my testbed. These are quite new devices, the build date on the boxes is 13APR2020 . Before I figure out what their true role is (one of them might have to run linux) I need to install FreeBSD -CURRENT and see how performance and hardware support is.

They have an Intel i3-10110U with 2 cores and 4 threads at 2ish GHz (with 4GHz boost), I got a single 32GB DIMM of RAM for each and a 480GB Western Digital M.2 SSD. This configuration came in just under £500 for each NUC.

The NUCs are pretty small, they have pretty beefy fans taking up about a cm of the top of the enclosure. They certainly aren't silent, without any load I could hear the NUC sat on the desk next to me. When building at full steam the fan in the NUC is about as loud as my x270 Thinkpad is when it is building.

What works?

Out of the box I had to break into the bios and disable secure boot to boot the FreeBSD installer. I did this by hitting every FN key as the NUC booted, I think FN2 was the correct choice. At the time my keyboard was being fought over by USB and Bluetooth on my MacBook Pro.

FreeBSD install was problem free. I set up the M.2 as a single drive with ZFS, datasets and snapshots are a magic power.

Before I tried anything else I had to get an idea at how well this NUC would build FreeBSD. I don't expect this to be a build machine, but having spent a while shopping for build machines recently (I settled on a VM in hetzner) it is the only benchmark I really care about. make -j4 buildworld buildkernel took 2:45, that isn't the fastest in the world, but it is about an 1:15 faster than my x270 with its 2015 i5-6300U . The difference 5ish years makes.

Graphics in this 10th Gen Intel processor wasn't supported by drm-current-kmod and I had to install drm-devel-kmod . With the devel kmod the NUC is happy to push all the pixels of the 4k TV I have here and even drive additional second monitor connected with USB-C->HDMI adapter at the same time.

Audio works through the front 3.5mm jack and after changing hw.snd.default_unit from HDMI too. There is an issue where when the display is blanked (turned off), audio will stop, but won't come back when the display does. manu@ suggested setting hw.i915kms.disable_power_well=0 in loader.conf , this resolved the problem for me.

The Intel 9462 WiFi in the NUC is not supported (yet), I stuck a cheap rtwn based USB WiFi dongle in the front port. External USB WiFi is cumbersome on a laptop, but on a machine that will rarely move it is fine. The onboard Ethernet is supported by em and is happy to do 940ishMbit/s with iperf3 with TCP and UDP.

The NUC has 5 USB ports, 3 USB-A and two USB-C. The USB-C port on the back has a little lightning bolt next to it and I imagine that means it is the one with thunderbolt support. I did a disk speed test to a Sandisk Extreme Portable SSD. The front port managed 200MB/s or so while the rear thunderbolt one only got to about 160MB/s for read and write. My MacBook Pro manages the advertised 500MB/s read speeds, so I need to dig into whether this is a hardware problem or a FreeBSD problem. Luckily I have two so installing Linux to test performance won't be a bother.

I quite like these little boxes, you can get more hardware for your money with a tower pc, but not in this form factor. I wouldn't be surprised if the i7 versions of these were quite good at building FreeBSD. The fans are a bit loud, but that can be fixed for me by playing blaring techno.

I think the NUC in this configuration is going to make a great test node in my satellite testbed or a nice little desktop box for doing video tasks.

Hardware Support

Storage         Yes     M.2 is fine, I didn't have a drive to test SATA
Graphics        Yes     drm-devel-kmod (on r362612)
Audio           Yes     hda(4)
Ethernet        Yes     Supported by em(4)
Wireless        No      Intel AC 9462 not supported :(
USB             Yes     All ports work, 
Suspend/resume  Yes     suspend is fine, resume required drm-devel-kmod
SD slot         Yes     Comes up as a mmcsdX device
Thunderbolt     N/A     I have no idea what the implications of this are or how to test

Capturing a screen sub section with ffmpeg

I tooted me scrolling through the browser tabs I opened from the Black Producers bandcamp list . I used ffmpeg to to grab a subsection of the screen with this command:

ffmpeg -f x11grab -show_region 1 -framerate 10 -video_size 720x480+100,200 -i $DISPLAY scroll.webm
  • video_size sets the size and offset in the window (needed if grabbing a subsection)
  • show_region paints a box around the active part, this helps dial in what is captured.

It took me a while to figure out what the input (-i) should be, in the end I figured out that it needs to be the X display, the easiest way to grab this is from the environment variable $DISPLAY .

I first made mp4's from the capture, but they were coming out all garbled, firefox refused to play and vlc was pretty sad. I move to webm and the capture is smaller and works in firefox.

previous next