December Adventure 2025
I learned about December Adventure in the middle of December last year, which is probably quite typical. I don't think most people get the words out in sync with the hacking and so there is a delay. Obviously that didn't exclude me, it wasn't the spirit of the event after all, but I didn't have the capacity.
I thought I would make an attempt at December Adventure this year. I've been trying hard for a while to focus on about 1 thing at a time. I thought for DA this year I would try and make progress on this summers project: Port FreeBSD to the Allwinner H616.
This blogpost will update as I do stuff through December, so don't expect a real feed of updates. I want to keep the entry effort down to a minimum so most will be more running chain of thoughts rather than fully explained post. If you have questions send me an email and I'll add more detail until it makes sense. I am going to buck tradition and keep it in chronological order too.
Background
I was super intrigued by the Sipeed Nano Cluster when it was annouced and redacted video .
The Nano Cluster is a carrier board for up to 7 modules, with an integrated switch and network port. It is about the size of a small beer and fits in your hand. Sipeed designed 3 different modules that fit the form factor, a CM4/5 adapter, some AI SOC and a Allwinner H618 module based on the Orange Pi Zero3.
The H618 was the cheapest module on offer and Allwinner have a pretty good reputation of reasonable levels of hardware openness.
I ordered a cluster with two H618 modules to begin with, but the Sipeed project timeline wasn't very concrete. I wasn't really sure if these would ship in days or months and as it was June and I wanted a summer project I also went to Aliexpress and bought two different Android tv devices which advertised themselves as using the H616. Delivery times for me for Aliexpress stuff is somewhere between 1 day and 1 month so I thought I'd have a target to get started with.
You may have noticed me jumping between using H618 and H616, well that is because I couldn't keep the numbers straight in my head at all. In the end it turned out that the H616, H616 and H700 are all very similar and need the same hardware support. The H700 is actually in the Anbernic gaming device I got last year.
Anyway, with hardware bought and on its way to my home in Scotland I had to leave the country for the next three months on very short notice.
Summer 2025
Separated from hardware I was able to do some of the inital work on a port. I acquired the Nano Cluster files from Sipeed, but didn't look at them. I went digging into the first thing you need for an embedded port and acquired a device tree (source) file for the h616.
From there I was able to make a table mapping out which device tree nodes will be support by which existing drvier in FreeBSD. A lot of the time support is a matter for adding a compat string to a table and maybe some supporting data from the datasheet.
I got to the point where I had a table like this:
dts file: sun50i-h616.dtsi
mmio-sram
snps,dwmac-mdio
sun50i-h616-ccu per soc
sun50i-h616-crypto
sun50i-h616-dma a10_dma, a31_dmac (h3 config)
sun50i-h616-ehci
sun50i-h616-emac0 if_awg test
sun50i-h616-emmc aw_mmc test
sun50i-h616-gpadc
sun50i-h616-i2c
sun50i-h616-iommu
sun50i-h616-ir aw_cir test
sun50i-h616-lradc
sun50i-h616-mmc aw_mmc test
sun50i-h616-musb
sun50i-h616-nmi aw_nmi not sure what this needs isn't in datasheet
sun50i-h616-ohci
sun50i-h616-pinctrl aw_gpio
sun50i-h616-r-ccu
sun50i-h616-r-pinctrl
sun50i-h616-rsb aw_rsb needs config values
sun50i-h616-rtc aw_rtc need to check h3 table is correct
sun50i-h616-sid aw_sid needs config table (check linux)
sun50i-a64-sid
sun50i-h616-spdif h3_padconf (and others)
sun50i-h616-spi
sun50i-h616-system-control aw_syscon
sun50i-h616-ths aw_thermal needs config table
sun50i-h616-usb-phy aw_usbphy needs config table
sun50i-h616-wdt aw_wdog needs config values
arm,cortex-a53
arm,gic-400
cache
snps,dw-apb-uart
arm,armv8-timer
arm,cortex-a53-pmu
arm,psci-0.2
fixed-clock
simple-bus
and the supporting patches for the changes I had made. This was a good to do list.
Then rather than getting to a point where I had builds, but no device I swapped summer project from "write some code" to read 15 books. This was cathartic and it finally settled a question from the last time I had a block of time off - yes reading a lot of books is good, but it doesn't leave you full, you have to do other things.
After Summer
Once I finally made it home I unpacked the hardware, took the android boxes apart and photographed their insides (I'll add these pictures soon). The two devices were basically identical, they have a cool led panel on the front driven by gpio showing wifi status and disk access.
One is called a "H96 Max" or H96 for any files I dump from it, it has a blue PCB and the other is T95 Pro and it has a black PCB. Both boards have a pin header on the board by the SD card slot labeled with RX, TX, GND.
I connected PCB Bite probes to each and booted them up. They are both running android and drop you to a shell once they have booted. The H96 gave me a root shell, whereas the T95 gave me a user shell, so the H96 is going to get all the initial attention.
The H96 is very noisy once it has booted, I resolved [this by changing a logging sysctl] (https://superuser.com/questions/351387/how-to-stop-kernel-messages-from-flooding-my-console):
sysctl -w kernel.printk="3 4 1 3"
I wasn't able to break into uboot on either device, as far as I can tell from reading the uboot sources, you should be able to break into its prompt if it shows you "Hit any key to stop autoboot", but I couldn't manage.
With root on the system I instead dumped out uboot configuration from the emmc and used set to rewrite the autoboot delay field from 0 to 10. This didn't work, but it didn't work enough to stop the system booting, leaving me at a uboot prompt (so a win I guess?).
I saved a copy of the strings output of the uboot binary image I dumped, but in a fat fingered moment I overwrote the good original uboot config.
So that left me with a device I could boot and could access the uboot prompt on. That is more than enough to do a port from.
November Warm up
I realised December Adventure was coming and I killed the power supply in my other evening project really knocking my enthusiasm down a bit.
I set out to get a working development set up for the H96 before December started so I wouldn't be too bogged down.
I set up a usb serial adapter hanging off uart I soldered a header to. I added in an sdwire sd-mux board which allows you to share an sd card between a DUT and as TS. Finally to enable remote working on the device I added a pi pico running micropython and a relay to control power, giving me a remote off switch. This is really handy when you need to reboot a system from cold power on.
At some point I pulled a FDT off the board with something like:
# cat /sys/firmware/devicetree | nc host port
With that in place I then got the Sipeed sources building as a close enough initial target and copied out that uboot onto a PINE64-LTS FreeBSD 16 image.
FreeBSD provides aarch64 images, but Arm platforms are still a mess in the DTS
world and all require boot firmwares in different places. I checked through all
the build configs (
src/release/arm64/*config
) and verified that the
PINE64-LTS image has enough space before the first data partition to fit the
Allwinner uboot.
With all that background we can now move into the December Adventure log:
20251129
Image I built yesterday onto the freebsd pine64-lts image didn't boot. It is probably because I didn't set up the mdconfig image with sectors.
I tried copying to the sd card directly like so:
sudo dd if=build/u-boot-sunxi-with-spl.bin of=/dev/da3 bs=1024 seek=8 conv=sync
And managed to hit a useful error:
U-Boot SPL 2024.01-rc2-00076-g94b814f631e (Nov 28 2025 - 16:42:46 +0000)
DRAM:This DRAM setup is currently not supported.
resetting ...
I thought that maybe the h616 was like a NXP platform and that firmware specifics set the DRAM size and I spent a bit of time looking at ways the size might be configured. There isn't anything that responsible in the working dts, the memory section is commented out and the extracted dts doesn't indicate a range.
The extracted DTS doesn't build, which is its own issue for later.
I eventually grepped in the u-boot sources.
This hits a pretty unique bit of u-boot for this platform, which I have no idea what is needed to resolve.
For each type of memory there is a possible selection of bus widths and ranks. I'm not sure what ranks here means, so it is time to go to the datasheet.
The datasheet section on SDRAM is 1 page of bullet points.
There is a set of registers (20k) for
DRAM_CTRL
If the datasheet is no help I think I need to enable debug prints from u-boot. afsaf debug is defined in log.h and is enabled if DEBUG is defined in a file.
20251130
There is basically nothing in the user manual about the usage of the DRAM controller. This ties up really well with a comment or commit message in u-boot where the author says most values just come from the boot0 logs.
So now to enable debug on uboot and start littering the sun50iw9 paths with prints to see what actually happens before this DRAM error.
I am going to set the device tree back to the longan pi 3 one for test builds.
sudo sd-mux-ctrl --ts -e da12
sudo sd-mux-ctrl --dut -e da12
The first u-boot path came from the longan build scripts, this dd pulls u-boot from the actual build dir.
sudo dd if=build/uboot/u-boot-sunxi-with-spl.bin of=/dev/da3 bs=1024 seek=8 conv=sync
The value to define to get debug_ printfs is _DEBUG
/* Show a message if DEBUG is defined in a file */
#define debug(fmt, args...) \
debug_cond(_DEBUG, fmt, ##args)
That gets us:
U-Boot SPL 2024.01-rc2-00076-g94b814f631e-dirty (Nov 30 2025 - 09:44:30 +0000)
DRAM:testing 32-bit width, rank = 2
read calibration failed!
testing 32-bit width, rank = 1
read calibration failed!
testing 16-bit width, rank = 2
read calibration failed!
testing 16-bit width, rank = 1
read calibration failed!
This DRAM setup is currently not supported.
resetting ...
More debug prints indicate that u-boot things this is lpddr4, but the boot0 log has:
[94]DRAM_VCC set to 1500 mv
[97]DRAM CLK =648 MHZ
[99]DRAM Type =3 (3:DDR3,4:DDR4,7:LPDDR3,8:LPDDR4)
[107]Actual DRAM SIZE =4096 M
[110]DRAM SIZE =4096 MBytes, para1 = 310b, para2 = 10000000, dram_tpr13 = 6041
[123]DRAM simple test OK.
So maybe this error is due to detecting the wrong ddr type somewhere.
Our copied u-boot defconfig has:
CONFIG_MACH_SUN50I_H616=y
# CONFIG_RESERVE_ALLWINNER_BOOT0_HEADER is not set
CONFIG_ARM_BOOT_HOOK_RMR=y
CONFIG_SUNXI_DRAM_LPDDR4=y
There isn't an unset version for ddr3, inventing one in the config breaks the build so there is some digging to do. The ddr3 config that uboot ships has the memory speed at 1333MHz, boot0 indicates the memory speed is much lower, but maybe we can just hacking this to work?
For some reason the uboot build is failing, but without stoping the makefile, which is pretty annoying. The real result is a lack of an output binary in the u-boot directory.
building u-boot constantly:
gmake clean; gmake sun50iw9-h616-h96_defconfig
gmake -j 16
Progress to a hang:
U-Boot SPL 2024.01-rc2-00076-g94b814f631e-dirty (Nov 30 2025 - 10:32:46 +0000)
DRAM:testing 32-bit width, rank = 2
mctl_phy_init:885 DDR3
mctl_phy_init:1114 READ CALIBRATION
read calibration failed!
testing 32-bit width, rank = 1
mctl_phy_init:885 DDR3
mctl_phy_init:1114 READ CALIBRATION
mctl_phy_init:885 DDR3
mctl_phy_init:1114 READ CALIBRATION
mctl_phy_init:885 DDR3
mctl_phy_init:1114 READ CALIBRATION
mctl_phy_init:885 DDR3
mctl_phy_init:1114 READ CALIBRATION
MBUS port 0 cfg0 0100000d cfg1 00640080
MBUS port 1 cfg0 06000009 cfg1 01000578
MBUS port 2 cfg0 0200000d cfg1 00600100
MBUS port 3 cfg0 01000009 cfg1 00500064
MBUS port 4 cfg0 20000209 cfg1 1388157c
MBUS port 5 cfg0 00640209 cfg1 00200040
MBUS port 6 cfg0 00640209 cfg1 00200040
MBUS port 8 cfg0 01000009 cfg1 00400080
MBUS port 11 cfg0 01000009 cfg1 00640080
MBUS port 14 cfg0 04000009 cfg1 00400100
MBUS port 16 cfg0 2000060d cfg1 09600af0
MBUS port 21 cfg0 0800060d cfg1 02000300
MBUS port 25 cfg0 0064000d cfg1 00200040
MBUS port 26 cfg0 20000209 cfg1 1388157c
MBUS port 37 cfg0 01000009 cfg1 00400080
MBUS port 38 cfg0 00640209 cfg1 00200040
MBUS port 39 cfg0 20000209 cfg1 1388157c
MBUS port 40 cfg0 00640209 cfg1 00200040
4096 MiB
The ddr3 config option carrying a speed is pretty annoying. It is only considered in two places, one sets the type and the other is blob:
static const u8 phy_init[] = {
#ifdef CONFIG_SUNXI_DRAM_H616_DDR3_1333
0x07, 0x0b, 0x02, 0x16, 0x0d, 0x0e, 0x14, 0x19,
0x0a, 0x15, 0x03, 0x13, 0x04, 0x0c, 0x10, 0x06,
0x0f, 0x11, 0x1a, 0x01, 0x12, 0x17, 0x00, 0x08,
0x09, 0x05, 0x18
#elif defined(CONFIG_SUNXI_DRAM_H616_LPDDR3)
0x18, 0x06, 0x00, 0x05, 0x04, 0x03, 0x09, 0x02,
0x08, 0x01, 0x0a, 0x0b, 0x0c, 0x0d, 0x0e, 0x0f,
0x10, 0x11, 0x12, 0x13, 0x14, 0x15, 0x16, 0x07,
0x17, 0x19, 0x1a
#elif defined(CONFIG_SUNXI_DRAM_H616_LPDDR4)
0x02, 0x00, 0x17, 0x05, 0x04, 0x19, 0x06, 0x07,
0x08, 0x09, 0x0a, 0x0b, 0x0c, 0x0d, 0x0e, 0x0f,
0x10, 0x11, 0x12, 0x13, 0x14, 0x15, 0x16, 0x01,
0x18, 0x03, 0x1a
#endif
};
With no documentation, I'm not really sure what to do to get a compatible u-boot built. I can getting hold of the stock u-boot the board shipped with, either by chasing down whatever might be on the aliexpress listing from 6 months ago (lol), or by dumping u-boot from android and having a look at a disassembly.
I checked mount in android to find somewhere writable, this was /data/media
dd if=/dev/block/mmcblk0 of=h96rawdisk1M.img bs=1M count=1
and then I configured a static address to a test machine and used nc to copy the uboot dump off the device.
Then I dropped the first 8k of the disk to get a uboot blob:
dd if=h96rawdisk1M.img of=h96uboot.img bs=1024 iseek=8 conv=sync
this might not actually be all of uboot, but whatever.
Honestly, this is pushing what I can do with hardware re. I'm just not able to eyeball instructions out of an aarch64 hexdump yet.
I should switch to booting FreeBSD from an sd card using the vendor uboot, but maybe I could poke at this using radare2.
I mean, I might just need a series of writes to
PHYS_CTRL 0x0480000
, surely
that shouldn't be too hard to pull from uboot?
I spent the evening reading the radar2 book and looking up RE projects on uboot. While I was doing this the sunxi wiki was down, but later in the day it came back up.
The boot wiki page informated me that DRAM parameters are set between the board vendor and Allwinner using special tools, but they are carried as a configuration file at the start of the SPL boot loader.
That explains the
DRAM.ext
file in the uboot blob I extracted and gives me a
final (I promise) thing to try before paying attention to the port again.
20251201
I wrote up all my existing notes and added 1800 words - which hasn't really matched the "make entries easy" goal.
Installed sunxi-tools.
From yesterdays last minute discovery that there was tooling to help on the wiki I read more of the wiki pages on early boot.
The boot0 page includes a header for the boot0/spl, this is helpful for looking at the dump I took, even if I don't really need it.
Offset Name Size Notes
0x00 B_INS 4 Branch instruction to Code Starting Point
0x04 Magic 8 Ascii string "eGON.BT0" (No Null-terminated )
0x0c Checksum 4 Simple 4-bytes Checksum (Before calculate checksum this must be 0x5F0A6C39 )
0x10 Size 4 Size of Boot0, it's must be 8-KiB aligned in NAND and 512-Bytes aligned in MMC
0x14 Code - Code of SPL. The size depends on the processor and if it 's loaded from SPI, NAND or MMC
The DRAM settings sunxi wiki page has a link for getting parameters from boot0 https://linux-sunxi.org/U-Boot#DRAM_Settings, this is using 'sunxi-fw' which isn't in sunxi-tools on freebsd.
A little Makefile hacking later:
diff --git a/Makefile b/Makefile
index 8c16c01..23fe451 100644
--- a/Makefile
+++ b/Makefile
@@ -1,6 +1,6 @@
SH=/bin/sh
-CC=${CROSS_COMPILE}gcc
-CFLAGS=-Wall -g -O
+CC=${CROSS_COMPILE}cc
+CFLAGS=-Wall -g -O -I/usr/include -I/usr/local/include
PREFIX ?=/usr/local
all: sunxi-fw
And I had a working tool. Pointing it at my extracted uboot bits from the h96 showed something:
$ ./sunxi-fw/sunxi-fw info -a h96rawdisk1M.img
@ 0: mbr: DOS MBR
protective MBR, GPT used
GPT version 00010000
usable disk size: 29783 MB
number of partition entries: 17
@ 16: boot0: Allwinner boot0
@ 512: boot0: Allwinner boot0
but it wasn't great compared to the uboot I built myself:
$ ./sunxi-fw/sunxi-fw info -a ../LonganPi-3H-SDK/build/uboot/u-boot-sunxi-with-spl.bin
@ 0: spl: U-Boot SPLv2
DT: sun50i-h618-longanpi-3h
@ 64: fit: U-Boot FIT image
fit:uboot: "U-Boot (64-bit)"
fit:atf: "ARM Trusted Firmware"
fit:fdt-1: "sun50i-h618-longanpi-3h"
configuration: sun50i-h618-longanpi-3h
The eGON header is there in the dump, so I am not really sure what is wrong. Lets park that for now. The sdmux is painfully slow to dd a full image to, so last night as a last thing I left the computer copying over a fresh PINE64-LTS image, which shouldn't be able to boot at all.
Lets try and boot a kernel from the vendor uboot using the vendor uboot.
20251202
If I am going to use the vendor uboot then I can start working on getting a kernel booting at all from uboot. I have done this a ton of times on different boards and so I tried to track down an example command.
The best I could do was this:
fatload mmc 1:1 0x48000000 dtb/starfive/jh7110-visionfive-v2.dtb
fatload mmc 1:1 0x44000000 efi/boot/bootriscv64.efi
bootefi 0x44000000 0x48000000
fdt_addr_r=0x51ff8000
kernel_addr_r=0x50200000
fatload mmc 0:1 0x51ff8000 dtb/bl808-pine64-ox64.dtb
fatload mmc 0:1 0x50200000 efi/boot/bootriscv64.efi
bootefi 0x50200000 0x51ff8000
from my (unpublished) artilce on running FreeBSD on the Pine Ox64 riscv SBC.
It is all pretty straight forwards until we hit the
bootefi
command. I doubt
the uboot on the h96 has this. There are other options to boot a loader or
kernel, I'd prefer to use a efi loader if I can.
I aimed to do more in the evening at the hackerspace, but a stop off at Aldi on the way and forgetting a usb-c cable for my ridiculous setup stopped that.
20251203
Lets take a dump of the available uboot commands on the h96:
Hit any key to stop autoboot: 0
=> help
? - alias for 'help'
base - print or set address offset
bdinfo - print Board Info structure
boot - boot default, i.e., run 'bootcmd'
bootd - boot default, i.e., run 'bootcmd'
bootm - boot application image from memory
bootp - boot image via network using BOOTP/TFTP protocol
cmp - memory compare
colorbar- show colorbar
coninfo - print console devices and information
cp - memory copy
crc32 - checksum calculation
echo - echo args to console
editenv - edit environment variable
efex - run to efex
env - environment handling commands
erase - erase FLASH memory
fastboot- fastboot - enter USB Fastboot protocol
fatinfo - print information about filesystem
fatload - load binary file from a dos filesystem
fatls - list files in a directory (default /)
fatsize - determine a file's size
fatwrite- write file into a dos filesystem
fdt - flattened device tree utility commands
flinfo - print FLASH memory information
go - start application at address 'addr'
gpt - GUID Partition Table
help - print command description/usage
i2c - I2C sub-system
itest - return true/false on integer compare
loadb - load binary file over serial line (kermit mode)
loads - load S-Record file over serial line
loadx - load binary file over serial line (xmodem mode)
loady - load binary file over serial line (ymodem mode)
logo - show default logo
loop - infinite loop on address range
md - memory display
memtester- start application at address 'addr'
mm - memory modify (auto-incrementing address)
mmc - MMC sub system
mmcinfo - display MMC info
mw - memory write (fill)
nfs - boot image via network using NFS protocol
nm - memory modify (constant address)
pbread - read data from private data
poweroff- Perform POWEROFF of the device
printenv- print environment variables
protect - enable or disable FLASH write protection
pst - read data from secure storageerase flag in secure storage
reset - Perform RESET of the CPU
run - run commands in an environment variable
saveenv - save environment variables to persistent storage
screen_char- show default screen chars
setenv - set environment variables
setexpr - set environment variable as the result of eval expression
sleep - delay execution for some time
source - run script from memory
sprite_test- do a sprite test
sunxi_axp- sunxi_axp sub-system
sunxi_bmp_info- manipulate BMP image data
sunxi_bmp_show- manipulate BMP image data
sunxi_card0_probe- probe sunxi card0 device
sunxi_flash- sunxi_flash sub-system
sunxi_nand_test- sunxi_nand_test sub systerm
sunxi_so- sunxi_so sub-system
tftpboot- boot image via network using TFTP protocol
timer_test- do a timer and int test
timer_test1- do a timer and int test
uburn - do a burn from boot
version - print monitor, compiler and linker version
There are some new
sunxi_
commands there, but nothing for usb. Try as I might
I can't get uboot to pick up the sd card I have inserted. Trying a USB stick
gives me:
[00.796]usb prepare ok
[01.599]overtime
[01.603]do_burn_from_boot usb : no usb exist
[01.607]boot_gui_init:start
FAT: Misaligned buffer address (bbe78ad8)
32 bytes read in 4 ms (7.8 KiB/s)
tcon_de_attach:de=0,tcon=2[01.891]boot_gui_init:finish
[01.895]bmp_name=bootlogo.bmp
Maybe the other port will work, but the cat is insisting that I remain seated.
There is a lack of a
usb
command in the help output. Also missing from
this uboot is an fel command to drop back into the default loader.
At this point I might have hit enough walls trying to get this board to boot and should probably try something else. Not being able to get the dram parameters despite seemingly having all the right tools is frustrating.
I had a look again at the cluster boards and they seem like much more annoying targets for doing bring up. A nice thing about this random h96 thing is that I am already controlling it with a relay and can reflash the sd card remotely, it just doesn't work. Maybe I can get enough ddr3 parameters together to make progress.
I don't want to give up yet. Looking at my list of commands and I noticed the
fdt
command. Running
fdt print
generated a 6000 line output file!
This seems to include the same parameters I could get with the sunxi-fw tool, but I'm not sure if this maps to the magic bytes I need to configure for the phy.
Thinking about this more while brushing my teeth and I really might only need to know the phy init sequence. This feels like a great chance to try using radare2 on a target. I have a clear goal, get the writes to a certain address, and a lot of supporting facts already, register map and many common values.
20251204
Time to hit the book . There is a handy firmware section of the radare2 book and it helpfully tells you to not bother with the project support.
The reason for not using projects is because usually these targets
require some special setups, custom scripts, manual tries and errors
and obviously not using the default autoanalysis.
The firmware section shows initial set up and some tricks, but it is probably a requirement to read more of the book to know what is happening and what to do next.
I need to both learn radare2 and some more facts about the soc and where it places things early in boot.
We know what upstream uboot does to set up dram, the code leading to the
phy_init
copy is:
writel(val, SUNXI_DRAM_PHY0_BASE + 0x14);
writel(val, SUNXI_DRAM_PHY0_BASE + 0x35c);
writel(val, SUNXI_DRAM_PHY0_BASE + 0x368);
writel(val, SUNXI_DRAM_PHY0_BASE + 0x374);
writel(0, SUNXI_DRAM_PHY0_BASE + 0x18);
writel(0, SUNXI_DRAM_PHY0_BASE + 0x360);
writel(0, SUNXI_DRAM_PHY0_BASE + 0x36c);
writel(0, SUNXI_DRAM_PHY0_BASE + 0x378);
writel(val2, SUNXI_DRAM_PHY0_BASE + 0x1c);
writel(val2, SUNXI_DRAM_PHY0_BASE + 0x364);
writel(val2, SUNXI_DRAM_PHY0_BASE + 0x370);
writel(val2, SUNXI_DRAM_PHY0_BASE + 0x37c);
ptr = (u32 *)(SUNXI_DRAM_PHY0_BASE + 0xc0);
for (i = 0; i < ARRAY_SIZE(phy_init); i++)
writel(phy_init[i], &ptr[i]);
if (para->tpr10 & TPR10_CA_BIT_DELAY)
mctl_phy_ca_bit_delay_compensation(para, config);
We have these constants from uboot and the base address matches up with the datasheet.
#ifdef CONFIG_MACH_SUN50I_H616
#define SUNXI_DRAM_COM_BASE 0x047FA000
#define SUNXI_DRAM_CTL0_BASE 0x047FB000
#define SUNXI_DRAM_PHY0_BASE 0x04800000
#endif
From the table above we know the first 4 bytes should be a branch from the boot0 header to code. If I swap the r2 mode from 64 bits to 32 (though this feels like a proble of its own, certainly it indicates a knowledge gap), when we get some sensible disassembly for the first instruction.
0x08000000 be0400ea b 0x8001300 ; pc=0x8001300 -> 0xeaffffff
Lots of questions from these first steps:
-
how does aarch64 boot?
- is it in 32bit mode
- how to search for addresses in assembly in radare2
-
how can I get up to speed on aarch64 assembly quickly?
- (oh shit, I have a book on it
The book isn't really any help, it is a programming book rather than an architecture or systems reference. It is remarkably difficult to find aarch64 instruction encoding information, but wikipedia at least says:
Instructions are still 32 bits long and mostly the same as A32 (with
LDM/STM instructions and most conditional execution dropped)
I don't think the processor has started in 32 bit mode. Instead r2 is having trouble with that first branch.
The header is:
00000000 be 04 00 ea 65 47 4f 4e 2e 42 54 30 bf 3a 40 9d |....eGON.BT0.:@.|
00000010 00 00 01 00 30 00 00 00 00 00 00 00 00 00 02 00 |....0...........|
00000020 00 00 02 00 00 00 00 00 00 00 00 00 34 2e 30 00 |............4.0.|
00000030 00 00 00 00 03 00 00 00 88 02 00 00 03 00 00 00 |................|
so the first instruction is:
be 04 00 ea
[0x08000000]> e asm.arch=arm
[0x08000000]> e asm.cpu=v8
[0x08000000]> e asm.bits=64
[0x08000000]> pd 1
0x08000000 be0400ea ands x30, x5, x0, lsl 1 ; lr=0x0 ; zf=0x1 ; nf=0x0 ; cf=0x0 ; vf=0x0
[0x08000000]> e asm.bits=32
[0x08000000]> pd 1
┌─< 0x08000000 be0400ea b 0x8001300 ; pc=0x8001300 -> 0xeaffffff
So one of these is doing something that makes sense and the other isn't. I find this so confusing that I tried using capstone, which I think underlies radare2 for disassembly manually:
$ cstool arm64 be0400ea
0 be 04 00 ea ands x30, x5, x0, lsl #1
when that didn't give me the answer I wanted I tried some online disassemblers, but they all gave me the same result. This mystery will persist until I can find someone to ask what is going on.
So, lets say the first opcode is an immediate jump to #1300, which makes sense. How do I look through the rest of this binary for my addresses of interest using r2?
It seems that r2asm can't disassemble from a file:
$ rasm2 -a arm -b 32 -f h96uboot.img
ERROR: Cannot assemble '' at line 1
$ rasm2 -a arm -b 32 -D -f h96uboot.img
WARN: Invalid hexpair string
And neither can cstool, but it can give you detailed info on an instruction:
$ cstool -d arm be0400ea
0 be 04 00 ea b #0x1300
ID: 11 (b)
op_count: 1
operands[0].type: IMM = 0x1300
Registers read: pc
Registers modified: pc
Groups: branch_relative arm jump
And rasm2 can tell you what a pneumonic means:
$ rasm2 -a arm -b 64 -w b
branches the program counter to dst (pc aka r15)
So, lets pretend everything is fine and just continue in 32 bit mode for today.
I next need to ask people some questions about Allwinner SOC start up and figure out how to search for accessed addresses in r2.
I did a little more reading after shutting down the computers for the night and found some uboot documentation which is pretty clear about the A64 start up process:
Newer Allwinner SoCs feature ARMv8 cores (ARM Cortex-A53) with support for
both the 64-bit AArch64 mode and the ARMv7 compatible 32-bit AArch32 mode.
Examples are the Allwinner A64 (used for instance on the Pine64 board) or
the Allwinner H5 SoC (as used on the OrangePi PC 2).
These SoCs are wired to start in AArch32 mode on reset and execute 32-bit
code from the Boot ROM (BROM). As this has some implications on U-Boot, this
file describes how to make full use of the 64-bit capabilities.
That explains exactly what I am seeing. Next I need to figure out how the transition to 64-bit mode happens and identify that in the disassembly. I'm not aware of any debugging tools that handle mixed mode executables well, most choke on the entire notion of the instruction set changing.
20251205
I asked a question in irc, but to no response so far.
I'm not sure how to handle the multi mode executable. I read a radare2 firmware walkthrough and they suggested that adding the memory map for the soc will help a lot.
So lets pull that from the datasheet and turn it into radare2 format. The memory map in the datasheet copied out from the pdf is this blob:
Module Address(It is for Cluster CPU) Size(Bytes)
BROM 0x0000 0000---0x0000 FFFF 64K
SRAM A1 0x0002 0000---0x0002 7FFF 32K(support Byte operation, clock source is AHB1)
SRAM C 0x0002 8000---0x0005 7FFF Borrow VE 128K, DE 64K, supports Byte operation, clock source is AHB1 Accelerator
DE 0x0100 0000---0x013F FFFF 4M
DI0 0x0142 0000---0x0145 FFFF 256K
G2D 0x0148 0000---0x014B FFFF 256K
GPU 0x0180 0000---0x0183 FFFF 256K
CE_NS 0x0190 4000---0x0190 47FF 2K
CE_S 0x0190 4800---0x0190 4FFF 2K
CE_KEY_SRAM 0x0190 8000---0x0190 8FFF 4K
VE SRAM 0x01A0 0000---0x01BF FFFF 2M
VE 0x01C0 E000---0x01C0 FFFF 8K
System Resources
SYS_CFG 0x0300 0000---0x0300 0FFF 4K
CCU 0x0300 1000---0x0300 1FFF 4K
DMA 0x0300 2000---0x0300 2FFF 4K
HSTIMER 0x0300 5000---0x0300 5FFF 4K
SID 0x0300 6000---0x0300 6FFF 4K
SMC 0x0300 7000---0x0300 7FFF 4K
SPC 0x0300 8000---0x0300 83FF 1K
TIMER 0x0300 9000---0x0300 93FF 1K
PWM 0x0300 A000---0x0300 A3FF 1K
GPIO 0x0300 B000---0x0300 B3FF 1K
PSI 0x0300 C000---0x0300 C3FF 1K
GIC 0x0302 0000---0x0302 FFFF 64K
IOMMU 0x030F 0000---0x030F FFFF 64K
RTC 0x0700 0000---0x0700 03FF 1K
PRCM 0x0701 0000---0x0701 03FF 1K
TWD 0x0702 0800 – 0x0702 0BFF 1K
NAND0 0x0401 1000---0x0401 1FFF 4K
SMHC0 0x0402 0000---0x0402 0FFF 4K
SMHC1 0x0402 1000---0x0402 1FFF 4K
SMHC2 0x0402 2000---0x0402 2FFF 4K
MSI_CTRL 0x047F A000---0x047F AFFF 4K
DRAM_CTRL 0x047F B000---0x047F FFFF 20K
PHY_CTRL 0x0480 0000---0x04FF FFFF 8M
Interfaces
UART0 0x0500 0000---0x0500 03FF 1K
UART1 0x0500 0400---0x0500 07FF 1K
UART2 0x0500 0800---0x0500 0BFF 1K
UART3 0x0500 0C00---0x0500 0FFF 1K
UART4 0x0500 1000---0x0500 13FF 1K
UART5 0x0500 1400---0x0500 17FF 1K
TWI0 0x0500 2000---0x0500 23FF 1K
TWI1 0x0500 2400---0x0500 27FF 1K
TWI2 0x0500 2800---0x0500 2BFF 1K
TWI3 0x0500 2C00---0x0500 2FFF 1K
TWI4 0x0500 3000---0x0500 33FF 1K
S_TWI0 0x0708 1400---0x0708 17FF 1K
SPI0 0x0501 0000---0x0501 0FFF 4K
SPI1 0x0501 1000---0x0501 1FFF 4K
EMAC0 0x0502 0000---0x0502 FFFF 64K
EMAC1 0x0503 0000---0x0503 FFFF 64K
TS0 0x0506 0000---0x0506 0FFF 4K
THS 0x0507 0400---0x0507 07FF 1K
LRADC 0x0507 0800---0x0507 0BFF 1K
OWA 0x0509 3000---0x0509 33FF 1K
DMIC 0x0509 5000---0x0509 53FF 1K
Audio Codec 0x0509 6000---0x0509 6FFF 4K
Audio HUB 0x0509 7000---0x0509 7FFF 4K
USB0(USB2.0_OTG) 0x0510 0000---0x051F FFFF 1M
USB1(USB2.0_HOST1) 0x0520 0000---0x052F FFFF 1M
USB2(USB2.0_HOST2) 0x0531 0000---0x0531 0FFF 4K
USB3(USB2.0_HOST3) 0x0531 1000---0x0531 1FFF 4K
CIR_RX 0x0704 0000---0x0704 03FF 1K
Display
HDMI_TX0(1.4/2.0) 0x0600 0000---0x060F FFFF 1M
DISP_IF_TOP 0x0651 0000---0x0651 0FFF 4K
TCON_TV0 0x0651 5000---0x0651 5FFF 4K
TCON_TV1 0x0651 6000---0x0651 6FFF 4K
TVE_TOP 0x0652 0000---0x0652 3FFF 16K
TVE0 0x0652 4000---0x0652 7FFF 16K
CPUX Related
CPU_SUBSYS_CFG 0x0810 0000---0x0810 03FF 1K
TIMESTAMP_STU 0x0811 0000---0x0811 0FFF 4K
TIMESTAMP_CTRL 0x0812 0000---0x0812 0FFF 4K
IDC 0x0813 0000---0x0813 0FFF 3K
C0_CPUX_CFG 0x0901 0000---0x0901 03FF 1K
C0_CPUX_MBIST 0x0902 0000---0x0902 0FFF 4K
DRAM
DRAM 0x4000 0000---0x13FFF FFFF 4G
I am not sure the best way to model this in radare2. I don't need to have all of this in radare2 and all of it might hurt, it will be good to get the uarts represented, getting the writes there matching up with the disassembly will give me a good sync point between the run time output and the code I have.
Looking for uart writes will be a helpful starting point to figure out which mode the processor is in at that point, we get known plain text to associate with known registers.
The radare2 book chapter on this isn't much help, it feels like it is half written. It pushes svd files very hard, but I don't have an svd file.
At some point I should probbaly also figure out where boot0 is running from.
The syntax for creating a memory range is pretty janky, you need to open a malloc file uri with the size as the name. 4G isn't allowed, I guess it is too big, some iteration shows that 2G is the limit for a size reservation. 1G should surely be fine, I doubt uboot is reaching up that far.
The fw guide suggests using flags (which they compare to bookmarks) for mapping in devices such as uarts.
I can create an allocation for the bootrom like so:
on malloc://64k 0x00000000
omn. BROM
but the we get a mapping like this:
[0x08000000]> om
- 2 fd: 4 +0x00000000 0x00000000 - 0x0000ffff rw-
* 1 fd: 3 +0x00000000 0x08000000 - 0x080fdfff r-x BROM
The BROM mapping has been created at the base address I gave r2 to use. I'm going to have to figure out how to deal with that.
20251206
I did some reading and hit the Arm documentation. It clarified for me that there are execution states, instruction set states and exception levels. You can only change instruction set state (from aarch64 -> aarch32 or vice versa) during a change of exception level.
As you change exception level up, you can only move up. So if you are running in EL0 and aarch32 you can move to EL1 aarch64, but you can't go from EL1 to EL0 and move from 32 to 64.
You also can't move to the same exception level, so if you are running in EL3 aarch32 you are stuck. Instead you need to do a soft reset to make that change.
The Arm documentation is very thin, it is written very precisely and isn't super helpful to me. Most of this detail is there, but it was really clarified in these two blog posts on duetorun , Exception Levels and Security States and ARM64 Execution States .
This has been super helpful, my guess is that we need some code to set up the interrupt vector for 64 bit and then trigger a reset to move into 64 bit mode. Once that has occured we will probably find addressess decoding as we expect.
The actual reset process might hang off the Arm forums question .
This leaves a lot to do:
- make enough of the memory map appear in radare2
- find the uboot linker scripts for h616
- find uboot code for the transition
- find the reset process
- track down the reset instruction for aarch32 and 64
The Arm docs have:
AArch32 (EL3) to AArch64 Execution state transition at reset
At Exception level 3 (EL3), cores can only transition between AArch32
and AArch64 states at reset. The Execution state after reset is
controlled by the AA64nAA32[PE:0] configuration signals. These signals
are only sampled at reset.
To reset a core and change Execution state from software, a Warm reset
request can be made by setting the RR bit of the RMR system register
(from AArch32) or the RMR_EL3 register (from AArch64). Following the
register write and executing a WFI instruction, the cluster
automatically resets the core without requiring any action by the
external reset controller. The hardware automatically cleans and
invalidates all the caches and safely disconnects the core from cluster
before the reset is asserted.
The defconfig I started with has a bunch of options which might be clues for where to start looking:
CONFIG_ENABLE_ARM_SOC_BOOT0_HOOK=y
CONFIG_ARM64_SUPPORT_AARCH32=y
There isn't anything super clearly setting up the 32->64 transition, but there are a lot of things which might imply it. This is a great place to start digging from.
Seeing good stuff I kept reading to see if there were more values to pull out which might help with my questions.
CONFIG_TEXT_BASE=0x4a000000
CONFIG_SYS_UBOOT_START=0x4a000000
This might answer my "where do we load question" too.
The first two options just seem to be build effects, which is great.
CONFIG
TEXT
BASE has its hooks everywhere and CONFIG
SYS
UBOOT_START just sets
spl_image->entry_point = CONFIG_SYS_UBOOT_START;
.
This stackoverflow question and answer seems to hold a lot of hard facts and it links to a uboot file which probably helps . Indeed it is boot0.h and we have that config in our defconfig:
CONFIG_ARM_BOOT_HOOK_RMR=y
It is kind of a wild file with just a ton of already assembled op codes:
/*
* Switch into AArch64 if needed.
* Refer to arch/arm/mach-sunxi/rmr_switch.S for the original source.
*/
tst x0, x0 // this is "b #0x84" in ARM
b reset
.space 0x7c
.word 0xe28f0070 // add r0, pc, #112 // @(fel_stash - .)
.word 0xe59f106c // ldr r1, [pc, #108] // fel_stash - .
.word 0xe0800001 // add r0, r0, r1
.word 0xe580d000 // str sp, [r0]
.word 0xe580e004 // str lr, [r0, #4]
.word 0xe10fe000 // mrs lr, CPSR
.word 0xe580e008 // str lr, [r0, #8]
.word 0xee11ef10 // mrc 15, 0, lr, cr1, cr0, {0}
.word 0xe580e00c // str lr, [r0, #12]
.word 0xee1cef10 // mrc 15, 0, lr, cr12, cr0, {0}
.word 0xe580e010 // str lr, [r0, #16]
....
Lets fire up r2 with an updated base address and try and find a wfi.
The commit message for this file is great:
sunxi: A64: do an RMR switch if started in AArch32 mode
André Przywara authored 8 years ago
The Allwinner A64 SoC starts execution in AArch32 mode, and both
the boot ROM and Allwinner's boot0 keep running in this mode.
So U-Boot gets entered in 32-bit, although we want it to run in AArch64.
By using a "magic" instruction, which happens to be an almost-NOP in
AArch64 and a branch in AArch32, we differentiate between being
entered in 64-bit or 32-bit mode.
If in 64-bit mode, we proceed with the branch to reset, but in 32-bit
mode we trigger an RMR write to bring the core into AArch64/EL3 and
re-enter U-Boot at CONFIG_SYS_TEXT_BASE.
This allows a 64-bit U-Boot to be both entered in 32 and 64-bit mode,
so we can use the same start code for the SPL and the U-Boot proper.
We use the existing custom header (boot0.h) functionality, but restrict
the existing boot0 header reservation to the non-SPL build now. A SPL
wouldn't need such header anyway. This allows to have both options
defined and lets us use one for the SPL and the other for U-Boot proper.
Also add arch/arm/mach-sunxi/rmr_switch.S, which contains the original
ARM assembly code and instructions how to re-generate the encoded
version.
Ah ha! That explains that we are going to be in 32bit for all of boot0! Here is the entire log before the vendor uboot starts:
[61]HELLO! BOOT0 is starting!
[64]BOOT0 commit : 3ae35eb
[66]set pll start
[69]periph0 has been enabled
[72]set pll end
[74]unknow PMU
[75]unknow PMU
[77]PMU: AXP1530
[79]dram return write ok
[82]board init ok
[83]DRAM BOOT DRIVE INFO: V0.648
[87]the chip id is 0x5000
[89]chip id check OK
[94]DRAM_VCC set to 1500 mv
[97]DRAM CLK =648 MHZ
[99]DRAM Type =3 (3:DDR3,4:DDR4,7:LPDDR3,8:LPDDR4)
[107]Actual DRAM SIZE =4096 M
[110]DRAM SIZE =4096 MBytes, para1 = 310b, para2 = 10000000, dram_tpr13 = 6041
[123]DRAM simple test OK.
[125]rtc standby flag is 0x0, super standby flag is 0x0
[131]dram size =4096
[136]sdcard 2 line count 8
[138][mmc]: mmc driver ver 2021-10-12 13:56
[143][mmc]: b mmc 2 bias 4
[151][mmc]: Wrong media type 0x0, but host sdc2, try mmc first
[143][mmc]: b mmc 2 bias 4
[151][mmc]: Wrong media type 0x0, but host sdc2, try mmc first
[157][mmc]: ***Try MMC card 2***
[199][mmc]: RMCA OK!
[202][mmc]: MMC 5.0
[204][mmc]: HSSDR52/SDR25 8 bit
[207][mmc]: 50000000 Hz
[209][mmc]: 29820 MB
[211][mmc]: ***SD/MMC 2 init OK!!!***
[286]Loading boot-pkg Succeed(index=0).
[290][mmc]: b mmc 2 bias 4
[293]Entry_name = u-boot
[302]Entry_name = monitor
[306]Entry_name = dtbo
[309]Entry_name = dtb
[312]tunning data addr:0x4a0003e8
[316]Jump to second Boot.
NOTICE: BL3-1: v1.0(debug):05d6c57
NOTICE: BL3-1: Built : 13:35:35, 2021-10-28
NOTICE: BL3-1 commit: 8
NOTICE: cpuidle init version V1.0
ERROR: Error initializing runtime service tspd_fast
NOTICE: BL3-1: Preparing for EL3 exit to normal world
NOTICE: BL3-1: Next image address = 0x4a000000
NOTICE: BL3-1: Next image spsr = 0x1d3
r2 searching makes no sense to me, so I used hexdump and grep to grab that first message from the image and then moved to that addressed in r2:
0x4a00d399 7200 7363 7000 6474 6200 6474 626f 00 r.scp.dtb.dtbo.
0x4a00d3a8 6c6f 676f 0048 454c 4c4f 2120 424f 4f logo.HELLO! BOO
0x4a00d3b7 5430 2069 7320 7374 6172 7469 6e67 21 T0 is starting!
0x4a00d3c6 0a00 424f 4f54 3020 636f 6d6d 6974 20 ..BOOT0 commit
0x4a00d3d5 3a20 2573 0a00 6472 616d 2073 697a 65 : %s..dram size
0x4a00d3e4 203d 2564 0a00 6465 7465 6374 6564 20 =%d..detected
0x4a00d3f3 7573 6572 2069 6e70 7574 2032 0a00 4a user input 2..J
All of the output strings are bundled together, but they are separated by null bytes (00). That is close enough now that we have a uart address and a rough offset to look for.
So, I want to do something that I assume should be super common in an interactive disassembler and reverse engineering tool: search for writes to an address or range.
Running analysis with
aaa
seems to have helped make things be analysed (I
stumbled onto this from an
old advent
calendar
). This post might be the most useful
thing I've found for actually doing any work with r2.
I need something to short circuit some of this suffering with r2.
20251207
Last nights exploration was very frustrating, I am trying to move too quickly (remember Alice?) and I'm not giving myself enough space to experiment with r2 and understand what it is doing.
I think I should look at the uboot I built, this has the benefit of starting with assembly I have from the project which I can verify the functionality against with r2.
It uses a uart I can look for the address of and I know what instruction state the code is running in so there are no trips. From all of the re stuff I've read before I really did think "write to a physical address" would be easier to find.
I need to spend more time with each r2 command as they appear and know what they are doing, the initial config from the firmware book chapter has some stuff in it which I don't really understand. I want to pin all of that down too.
Manually reading with r2 and the boot0 file together and things tie up. First the spl header jumps to the boot0 block, and then I can follow the assembly and disassembly between the two files. Progress!
After writing RMR to reset the core we sit in a loop at wfi, and continue from
the reset vector. If we didn't have to do this set up the assembly is just
b
reset
. Lets figure out where that should be in uboot and move to aarch64
assembly.
uboot/arch/arm/cpu/armv8/start.S
has a
reset:
label. This does a lot of set
up of the core and then finally:
/* Processor specific initialization */
bl lowlevel_init
#if defined(CONFIG_ARMV8_SPIN_TABLE) && !defined(CONFIG_SPL_BUILD)
branch_if_master x0, master_cpu
b spin_table_secondary_jump
/* never return */
#elif defined(CONFIG_ARMV8_MULTIENTRY)
branch_if_master x0, master_cpu
/*
* Slave CPUs
*/
slave_cpu:
wfe
ldr x1, =CPU_RELEASE_ADDR
ldr x0, [x1]
cbz x0, slave_cpu
br x0 /* branch to the given address */
#endif /* CONFIG_ARMV8_MULTIENTRY */
master_cpu:
msr SPSel, #1 /* make sure we use SP_ELx */
bl _main
it does a
lowlevel_init
and has all but one core wait for an event (I guess
an ipi). The main core branches to _main. This isn't a label in an assembly
file or a c function (apart from on a weird exynos platform). I do hit an
interesting result in my build artefacts:
./u-boot.sym:000000004a01cc90 l F .text_rest 0000000000000018 run_main_loop
./u-boot.sym:000000004a040ea4 g F .text_rest 0000000000001124 fsg_main_thread
./u-boot.sym:000000004a001c90 g F .text_rest 0000000000000070 _main
reading the full grep output got me the correct spelling of exynous and
./arch/arm/lib/crt0.S
.
This file handles the target-independent stages of the U-Boot
start-up where a C runtime environment is needed. Its entry point
is _main and is branched into from the target's start.S file.
_main execution sequence is:
Reading crt0.S and I realise I've missed a step in the init process. I don't know where we are setting up the reset vector for the 32->64 transition.
I got lost in terminology for a bit and eventually hit a comment on an Arm
forum post which called this an exception rather than a register. From there I
got the Arm documentation on changing EL3 VBAR and that made the initial boot0
assembly terms kick in. They use
RVBAR_ADDRESS
and
RVBAR_ALTERNATIVE
, a
quick grep gives me:
config SUNXI_RVBAR_ADDRESS
hex
depends on ARM64
default 0x09010040 if SUN50I_GEN_H6
default 0x017000a0
---help---
The read-only RVBAR system register holds the address of the first
instruction to execute after a reset. Allwinner cores provide a
writable MMIO backing store for this register, to allow to set the
entry point when switching to AArch64. This store is on different
addresses, depending on the SoC.
config SUNXI_RVBAR_ALTERNATIVE
hex
depends on ARM64
default 0x08100040 if MACH_SUN50I_H616
default SUNXI_RVBAR_ADDRESS
---help---
The H616 die exists in at least two variants, with one having the
RVBAR registers at a different address. If the SoC variant ID
(stored in SRAM_VER_REG[7:0]) is not 0, we need to use the
other address.
Set this alternative address to the same as the normal address
for all other SoCs, so the content of the SRAM_VER_REG becomes
irrelevant there, and we can use the same code.
As I read this we store
CONFIG_*TEXT_BASE
to the RVBAR
ADDRESS. Some disassembly reading and I am making head way.
We read the RVBAR
ALT here:
0x4a000110 34109fe5 ldr r1, [0x4a00014c]
0x4a000114 34009fe5 ldr r0, [0x4a000150]
0x4a000118 240090e5 ldr r0, [r0, 0x24]
0x4a00011c ff0010e2 ands r0, r0, 0xff
0x4a000120 2c109f15 ldrne r1, [0x4a000154] # <--- load RVBAR_ALT
0x4a000124 2c009fe5 ldr r0, [0x4a000158]
0x4a000128 000081e5 str r0, [r1]
which gets the value from adderess
0x4a000154
, which in r2 is:
0x4a00014c 40000109 stmdbeq r1, {r6}
0x4a000150 00000003 movweq r0, 0
0x4a000154 40001008 ldmdaeq r0, {r6} # <--- default 0x08100040 if MACH_SUN50I_H616
0x4a000158 60000200 andeq r0, r2, r0, rrx
I was pretty puzzled by this until I realised two things at once, this is a litle endian system, boot0.h is little endian (which is why I couldn't get r2 to disassemble any of its explicit words), the bytes are all wrong!
0x4a000158
is the start of the vector table and this snippet figures out
where to store than and immediately after resets.
I found a A53 TRM and RVBAR is reset vector base address register and it is described as:
Reset Vector Base Address. The address that execution starts from after
reset when executing in 64-bit state. Bits[1:0] of this register are 0b00, as
this address must be aligned, and bits [63:40] are 0x000000 because the address
must be within the physical address size supported by the processor.
And so load the address at
0x4a000154
and land there in 64 bit mode.
We have
60000200
at the address:
0x4a000158 60000200
and we see that is the value for ONFIG *TEXT BASE` in the assembly block.
$ cat configs/sun50iw9-h616-h96_defconfig | grep TEXT_BASE
CONFIG_TEXT_BASE=0x4a000000
CONFIG_SPL_TEXT_BASE=0x20060
CONFIG_HAVE_TEXT_BASE=y
I think that has us running from SRAM A1
Module Address(It is for Cluster CPU) Size(Bytes)
BROM 0x0000 0000---0x0000 FFFF 64K
SRAM A1 0x0002 0000---0x0002 7FFF 32K
SRAM C 0x0002 8000---0x0005 7FFF Borrow VE 128K, DE 64K
I have no idea what is going to run from there. I was pretty excited that I'd pinned down the 32bit code and it was time to switch r2 and start there. But I had not followed a pointer in the RMR setup. Next to figure out what this memory could contain.
Trying to understand which memory we are running from here and I've hit a sunxi wiki page which explains everything I've learned today sigh .
If we assume that we are running from aliased SRAM A1 and use the 0x60 offset as a starting point we find in 32 bit mode:
┌─< 0x4a000060 1f0000ea b 0x4a0000e4
│ 0x4a000064 47000014 strne r0, [r0], -0x47
and in 64 bit mode:
0x4a000060 1f0000ea tst x0, x0
┌─< 0x4a000064 47000014 b 0x4a000180
I checked thse with rasm2 first in 32 bit mode and learned that I cannot get r2 to change modes and still disassemble. rasm2 disagrees about where this branch goes:
$ rasm2 -a arm -b 64 -D 47000014
0x00000000 4 47000014 b 0x11c
but we know what should run next so we can check against our uboot source.
First I have added some more labels.
a little time watching tv and searching passes
Of course!
0x4a000000
is a base address I pulled out of the air (or well
CONFIG_TEXT_BASE
), but what has probably actually happened is that I've found
the real load address is the start of SRAM A1 (
0x00020000
).
This revelation was prompted by reading the sunxi BROM which has a table of SOC families and SPL load ranges and the FunKey Boot ROM page.
20251208
Relocating the radare2 base address and I'm quite happy now with the disassembly. With endianess clear and the base location making sense I want to jump ahead and try to find some known registers.
UART0 0x05000000 | 00000005
SUNXI_DRAM_PHY0_BASE 0x04800000 | 00008004
uart0 is an 16550 so the address to read and write bytes from is
0x05000000
,
phy init is going to be harder to find, but it lets us try and find a r2 masked
search function. We should search for things we have already found first, the
spl text base and RVBAR ALT.
spl text base 0x00020000 | 00000200
rvbar alt 0x08100040 | 40001008
I swapped these round with rax2:
$ rax2 -x -e 0x05000000
00000005
This value for the serial port is probably don't going to be unique.
I'm really struggling to make sense of search in r2. There is a value search
/v
and a ranged version, but the description is awful:
/v[1248] value look for an `cfg.bigendian` 32bit value
/V[1248] min max look for an `cfg.bigendian` 32bit value in range
What values are you meant to put in there?
Looking at the disassembly in visual mode and I can see what I want to search:
0x0002011c ff0010e2 ands r0, r0, 0xff ; r0=0xff ; zf=0x0 ; nf=0x0
0x00020120 2c109f15 ldrne r1, [0x00020154] ; [0x20154:4]=0x8100040 ; r1=0x8100040
0x00020124 2c009fe5 ldr r0, [0x00020158] ; [0x20158:4]=0x20060 RVBAR ; r0=0x20060 -> 0xea00001f RVBAR
0x00020128 000081e5 str r0, [r1] ; [0x08100040:4] = 0x20060
What I can't tell is how to search in the computed disassembly. That 0x20060 is the exact thing I want to pull from a search.
Instead of beating my head against r2 I did the 'unix thing' and extracted two versions of the disassembly.
[0x00020000]> pd 50000 > ./out32
[0x00020000]> pd 50000 > ./out64
Which are things I can grep. Maybe I should be feeding r2 output into python.
20251209
grepping!
Finding the uart is proving difficult it evaluates to
05
which is too common
a value to pull out of a megabyte of data. If instead we go to the piece of
code of interest the phy init code:
writel(val, SUNXI_DRAM_PHY0_BASE + 0x14);
writel(val, SUNXI_DRAM_PHY0_BASE + 0x35c);
writel(val, SUNXI_DRAM_PHY0_BASE + 0x368);
writel(val, SUNXI_DRAM_PHY0_BASE + 0x374);
writel(0, SUNXI_DRAM_PHY0_BASE + 0x18);
writel(0, SUNXI_DRAM_PHY0_BASE + 0x360);
writel(0, SUNXI_DRAM_PHY0_BASE + 0x36c);
This was quite easy to find in the uboot I built and so I hopped to the vendor image.
writel(0, SUNXI_DRAM_PHY0_BASE + 0x378);
writel(val2, SUNXI_DRAM_PHY0_BASE + 0x1c);
writel(val2, SUNXI_DRAM_PHY0_BASE + 0x364);
writel(val2, SUNXI_DRAM_PHY0_BASE + 0x370);
writel(val2, SUNXI_DRAM_PHY0_BASE + 0x37c);
ptr = (u32 *)(SUNXI_DRAM_PHY0_BASE + 0xc0);
for (i = 0; i < ARRAY_SIZE(phy_init); i++)
writel(phy_init[i], &ptr[i]);
Manually calculating the offset for the first write worked for this grep:
$ cat out64 | grep $(rax2 -e -x 0x14008004)
│ 0x00021168 200000b9 str w0, [x1] ; tmp=0x4800014 ; [0x04800014:4] = 0xd
but doing additions with the value doesn't
$ cat out64 | grep $(rax2 -x 0x04800000+0x14)
It seems to be because rax2 always outputs little endian no matter what you give it!
$ rax2 -x 0x04800000+0x14
14008004
$ rax2 -x 1
01000000
what a thing. r2 disassembly is giving me human formatted data in the
calculated disassembly so I can grep for things manually. In many calculations
the output drops leading zeros, look at the
tmp=
assignment above. That makes
searching automatically much more annoying.
For the uboot I built I only need 1 hit to find the correct piece of code, but for the vendor one I might not be as lucky.
A grep gives me:
$ cat out64 | grep 48000c0
│ 0x000211f0 21c00ad1 sub x1, x1, 0x2b0 ; x1=0x48000c0
│ 0x0002120c 234400b8 str w3, [x1], 4 ; tmp=0x48000c0 ; [0x048000c0:4] = 0x7 ; x1=0x48000c4
opening the file in r2 and opening in visual mode does not give me that value
at
0x0002120c
:
┌─> 0x00021204 03144038 ldrb w3, [x0], 1 ; tmp=0x27ec3 -> 0x16020b07 ; w3=0x7 ; x0=0x27ec4 -> 0xd16020b
│ 0x00021208 bf3f03d5 dmb sy
│ 0x0002120c 234400b8 str w3, [x1], 4 ; tmp=0xfffffffffffffd5c ; [0xfffffffffffffd5c:4] = 0x7 ; x1=0xfffffffffffffd60
│ 0x00021210 3f0002eb cmp x1, x2 ; zf=0x0 ; nf=0x1 ; cf=0x1 ; vf=0x0
└─< 0x00021214 81ffff54 b.ne 0x21204 ; pc=0x21204 -> 0x38401403 ; likely
scrolling up and down as the emulation updates does reveal the initial value of the register, matching up the two arrays progressing.
After more grepping with an output disassembly I think I've found the uart configuration:
0x00022974 00f034f8 bl fcn.000229e0 ; lr=0x22978 -> 0xf04f4a0a ; pc=0x229e0 -> 0x4ff0e92d ; fcn.000229e0
0x00022978 0a4a ldr r2, [0x000229a4] ; [0x229a4:4]=0x2d850 ; r2=0x2d850 -> 0x3a74bc27
0x0002297a 4ff0a063 mov.w r3, 0x5000000 ; r3=0x5000000
0x0002297e 0321 movs r1, 3 ; r1=0x3
0x00022980 1360 str r3, [r2] ; [0x0002d850:4] = 0x5000000
0x00022982 1961 str r1, [r3, 0x10] ; [0x05000010:4] = 0x3
0x00022984 da68 ldr r2, [r3, 0xc] ; r2=0xffffffff
0x00022986 42f08002 orr r2, r2, 0x80 ; r2=0xffffffff
0x0002298a da60 str r2, [r3, 0xc] ; [0x0500000c:4] = 0xffffffff
0x0002298c 0d22 movs r2, 0xd ; r2=0xd
0x0002298e 5c60 str r4, [r3, 4] ; [0x05000004:4] = 0x0
0x00022990 1a60 str r2, [r3] ; [0x05000000:4] = 0xd
0x00022992 da68 ldr r2, [r3, 0xc] ; r2=0xffffffff
0x00022994 22f08002 bic r2, r2, 0x80 ; r2=0xffffff7f
0x00022998 da60 str r2, [r3, 0xc] ; [0x0500000c:4] = 0xffffff7f
0x0002299a 0722 movs r2, 7 ; r2=0x7
0x0002299c d960 str r1, [r3, 0xc] ; [0x0500000c:4] = 0x3
0x0002299e 9a60 str r2, [r3, 8] ; [0x05000008:4] = 0x7
0x000229a0 70bd pop {r4, r5, r6, pc} ; r4=0x0 ; r5=0x0 ; r6=0x0 ; pc=0x0 ; sp=0x58024
0x000229a2 00bf nop
0x5000000
is the base address for the uart,
0x10
is the modem status
register and
0x4
is the interrupt enable register. Finally here is some
sense, now to pin down where the first prints occur.
Suddenly from here things start to fit together. I clicked on an addresses and it took me to a function making prints I could match to boot0:
┌ 272: fcn.0002aa0c ();
│ afv: vars(3:sp[0x1c..0x24])
│ 0x0002aa0c f0b5 push {r4, r5, r6, r7, lr} ; sp=0xffffffffffffffec ; [0xffffffec:4] = 0x0 ; [0xfffffff0:4] = 0x0 ; [0xfffffff4:4] = 0x0 ; [0xfffffff8:4] = 0x0 ;
│ 0x0002aa0e 0023 movs r3, 0 ; r3=0x0
│ 0x0002aa10 424c ldr r4, entry0 ; [0x2ab1c:4]=0x20000 entry0 ; r4=0x20000 -> 0xea0004be
│ 0x0002aa12 85b0 sub sp, 0x14 ; sp=0xffffffd8
│ 0x0002aa14 0622 movs r2, 6 ; r2=0x6
│ 0x0002aa16 cde90133 strd r3, r3, [sp, 4] ; [0xffffffdc:4] = 0x0 ; [0xffffffe0:4] = 0x0
│ 0x0002aa1a 04f1bc01 add.w r1, r4, 0xbc ; r1=0x200bc -> 0x1020008
│ 0x0002aa1e d4f8b800 ldr.w r0, [r4, 0xb8] ; r0=0x0
│ 0x0002aa22 0393 str r3, [var_ch] ; [0xffffffe4:4] = 0x0
│ 0x0002aa24 f7f79cff bl fcn.00022960 ;[1] ; lr=0x2aa28 -> 0xf7f7483d ; pc=0x22960 -> 0x460db570
│ 0x0002aa28 3d48 ldr r0, [0x0002ab20] ; [0x2ab20:4]=0x2d3ad "HELLO! BOOT0 is starting!." ; r0=0x2d3ad "HELLO! BOOT0 is starting!\n"
│ 0x0002aa2a f7f70dff bl fcn.00022848 ;[2] ; lr=0x2aa2e -> 0x713ef504 ; pc=0x22848 -> 0x4b11b40f
│ 0x0002aa2e 04f53e71 add.w r1, r4, 0x2f8 ; r1=0x202f8 "3ae35eb"
│ 0x0002aa32 3c48 ldr r0, [0x0002ab24] ; [0x2ab24:4]=0x2d3c8 "BOOT0 commit : %s." ; r0=0x2d3c8 "BOOT0 commit : %s\n"
│ 0x0002aa34 f7f708ff bl fcn.00022848 ;[2] ; lr=0x2aa38 -> 0xfbf4f7f7 ; pc=0x22848 -> 0x4b11b40f
│ 0x0002aa38 f7f7f4fb bl fcn.00022224 ;[3] ; lr=0x2aa3c -> 0xf7f8b920 ; pc=0x22224 -> 0xf000b510
│ ┌─< 0x0002aa3c 20b9 cbnz r0, 0x2aa48 ; pc=0x2aa48 -> 0xfc2cf7f7 ; likely
│ │ 0x0002aa3e f8f719f9 bl fcn.00022c74 ;[4] ; lr=0x2aa42 -> 0xf7f8b178 ; pc=0x22c74 -> 0x2400b538
│ ┌──< 0x0002aa42 78b1 cbz r0, 0x2aa64 ; unlikely
│ ││ 0x0002aa44 f8f750f9 bl fcn.00022ce8 ;[5] ; lr=0x2aa48 -> 0xfc2cf7f7 ; pc=0x22ce8 -> 0x21004b05
│ ││ ; CODE XREF from fcn.0002aa0c @ 0x2aa94(x)
From there I was able to pin down the block with the load of
0x50000000
as a
uart setup function and
fcn.00022848
as a printf call. Now I can tie the
prints to the boot log and start making guesses about what is happening where.
The code from this label all appears to be a main function and its first stops are to configure the uart and to say "HELLO!". There are 10 branches between the printf for "BOOT0 commit ..." and the call to print "dram size" and some of those make prints of their own.
Working backwards and the first step is promising:
510: fcn.00027724 ();
afv: vars(1:sp[0x20..0x20])
0x00027724 f7b5 push {r0, r1, r2, r4, r5, r6, r7, lr} ; sp=0xffffffffffffffe0 ; [0xffffffe0:4] = 0x0 ; [0xffffffe4:4] = 0x0 ; [0xffffffe8:4] = 0x0 ; [0xffffffec
0x00027726 0d46 mov r5, r1 ; r5=0x0
0x00027728 7e48 ldr r0, [0x00027924] ; [0x27924:4]=0x2bcdd "DRAM BOOT DRIVE INFO: %s." ; r0=0x2bcdd "DRAM BOOT DRIVE INFO: %s\n"
0x0002772a 7f49 ldr r1, [0x00027928] ; [0x27928:4]=0x2bcd6 "V0.648" ; r1=0x2bcd6 "V0.648"
0x0002772c fbf78cf8 bl fcn.00022848 ;[1] ; lr=0x27730 -> 0x487f4a7e ; pc=0x22848 -> 0x4b11b40f
0x00027730 7e4a ldr r2, [0x0002792c] ; [0x2792c:4]=0x7010310 ; r2=0x7010310
0x00027732 7f48 ldr r0, [0x00027930] ; [0x27930:4]=0x2bcf7 "the chip id is 0x%x." ; r0=0x2bcf7 "the chip id is 0x%x\n"
0x00027734 1368 ldr r3, [r2] ; r3=0xffffffff
0x00027736 43f48073 orr r3, r3, 0x100 ; r3=0xffffffff
0x0002773a 1360 str r3, [r2] ; [0x07010310:4] = 0xffffffff
0x0002773c 9368 ldr r3, [r2, 8] ; r3=0xffffffff
0x0002773e 23f03f03 bic r3, r3, 0x3f ; r3=0xffffffc0
0x00027742 9360 str r3, [r2, 8] ; [0x07010318:4] = 0xffffffc0
0x00027744 7b4b ldr r3, [0x00027934] ; [0x27934:4]=0x3006200 ; r3=0x3006200
0x00027746 1968 ldr r1, [r3] ; r1=0xffffffff
0x00027748 89b2 uxth r1, r1
0x0002774a fbf77df8 bl fcn.00022848 ;[1] ; lr=0x2774e -> 0xf923f000 ; pc=0x22848 -> 0x4b11b40f
0x0002774e 00f023f9 bl 0x27998 ;[2] ; lr=0x27752 -> 0xb9204604 ; pc=0x27998 -> 0x2000b570 ; 0x27998(0x2bcf7, -1, 0x7010310, 0x3006200)
0x00027752 0446 mov r4, r0 ; r4=0x2bcf7 "the chip id is 0x%x\n"
0x00027754 20b9 cbnz r0, 0x27760 ; pc=0x27760 -> 0xf7fb4876 ; likely
I followed this more and indicated a lot more sections of the binary, so far I haven't pinned down anything using the magic phy ctrl register.
20251210
10 Days in and we have been completely distracted on another adventure. In a cafe this morning I've managed to make a bit more sense out of r2. I finally tried following the visual book instructions and can control what is in each panel, changing the contents requires clicking on the title of the panel.
The functions default panel has been a mystery to me:
= Functions [& cache]
0x00020000 5 96 entry0
0x0002aa0c 21 272 main
0x00022960 3 66 uart_setup
0x00022568 1 44 clk_setup
0x000229e0 31 614 fcn.000229e0
0x00022848 6 72 printf
0x00022224 10 102 board_init
0x00022c74 13 84 rtc_check
0x00022ce8 3 22 rtc_setup
0x000222a4 1 4 pll_setup
0x00022514 1 30 syscfg
0x00022122 1 48 fcn.00022122
0x00022064 3 36 fcn.00022064
0x00022088 1 10 fcn.00022088
Address in the first column, that's fine, name, that makes perfect sense. I have been changing these from detected function prologues to names I can follow (even if they are wrong in the end).
What are the first two? Well an advent of radare2 says they are:
- address
- function size
- amount of basic blocks
- name
I think this being a program that appears to be written in assembly is probably tripping up the detector.
I also found the Function Calls view which groups parent child relationships:
[X] Function Calls [& cache]
fcn.0002217c
fcn.00022170
uart_setup:
clk_setup
fcn.000229e0
printf:
0x0002203c
0x0002281c
0x000229a8
0x00022638
board_init:
0x000222a8
0x000221e0
0x000221e0
This is going to help a lot now that I have some names pulled out. I also managed to make the search work to find the uart address:
> /v0x5000000
Gives a ton of results, it matches all of the 0s in the binary! Adding a space gives me a ton of results and they help. Lets look at some:
> /v 0x5000000
0x0002008d hit3_0 00000005
0x000203c9 hit3_1 00000005
0x00020449 hit3_2 00000005
0x000204c9 hit3_3 00000005
0x00020549 hit3_4 00000005
0x000205c9 hit3_5 00000005
0x00020649 hit3_6 00000005
0x000206c9 hit3_7 00000005
0x00020749 hit3_8 00000005
Hey would you look at that, someone swapped the endianness. How helpful is this, well the first one shows the problem with searching for the uart address quite well:
> s 0x0002008d-5
> pd 10
0x00020088 ~ 0000 movs r0, r0 ; zf=0x1 ; nf=0x0
;-- hit0_16:
0x00020089 00 unaligned
0x0002008a 0000 movs r0, r0 ; zf=0x1 ; nf=0x0
0x0002008c ~ 0000 movs r0, r0 ; zf=0x1 ; nf=0x0
;-- hit2_0:
;-- hit3_0:
0x0002008d 00 unaligned
0x0002008e 0000 movs r0, r0 ; zf=0x1 ; nf=0x0
0x00020090 0513 asrs r5, r0, 0xc ; cf=0x0 ; r5=0x0 ; zf=0x1 ; nf=0x0
0x00020092 00c00000 invalid
0x00020096 0080 strh r0, [r0] ; [0x00000000:2] = 0x0
Whoops, we might be finding the right thing, but we are also finding every string of 0s ending in a 0x05. It also doesn't hit our load in uart_setup:
0x0002297a 4ff0a063 mov.w r3, 0x5000000 ; r3=0x5000000
The value search is looking at explicit bytes in the binary. Maybe we should just assemble a load of interest?
This approach seems to have some issues though, we don't know what the target register will be and I can't actually get rasm2 to assemble the instruction the same way.
Building up an instruction doesn't seem like it will work, the phy ctrl register range is used differently to the uart range and it covers 8M compared to 1k for the uart.
Instead lets search for every offset:
$ cat arch/arm/mach-sunxi/dram_sun50i_h616.c | grep SUNXI_DRAM_PHY0_BASE | wc -l
212
This stupid thing that just takes writes cuts out a ton of values:
cat arch/arm/mach-sunxi/dram_sun50i_h616.c | grep SUNXI_DRAM_PHY0_BASE | grep write | awk -F + '{ print $2}' | awk -F, '{print $1}' | sed -e 's/);//' | sed 's/)//g' | sort | uniq | less
Sticking this into a script with the vendor uboot gives me nothing, but I do get results from the 64 bit uboot I built. I am a little worried that the code I am after is not in this initial loader and it lives somewhere else. There is a megabyte of stuff here.
I still need to track down the simple ram test passed string.
There is a chance that the r2 output I am searching is longer than 5000 instructions and I need to redump.
20251211
Before when I started dumping files I used
[0x00020000]> pd @@f > out
But that wasn't really a lot of data in the end. The dumps I have been working with so far have used:
[0x00020000]> pd 5000 > out
that isn't the entire file, it took some searching but r2 has the alias of
$s
for end of file. Now to create the vendor dump with
[0x00020000]> pd $s > vend32-full
$ ls -lh | grep vend32
-rw-r--r-- 1 tj tj 70M 11 Dec 16:15 vend32-full
-rw-r--r-- 1 tj tj 493K 9 Dec 09:33 vend32-short
Well that is a lot more data and it gives me something I don't think I had before
$ cat vend32-full| grep 0x4800000
0x00029fca 0003 lsls r0, r0, 0xc ; cf=0xd25 ; r0=0x48000000 ; zf=0x0 ; nf=0x0
0x00029fda 0446 mov r4, r0 ; r4=0x48000000
0x00029fe0 fff7fafc bl 0x299d8 ; lr=0x29fe4 -> 0xf7ff4620 ; pc=0x299d8 -> 0x4616b570 ; 0x299d8(0x48000000, 0x0, 0xff, 0x2dab8)
0x00029fe4 2046 mov r0, r4 ; r0=0x48000000
0x00029fe6 fff765fc bl 0x298b4 ; lr=0x29fea -> 0x7288f44f ; pc=0x298b4 -> 0x23004a02 ; 0x298b4(0x48000000, 0x0, 0xff, 0x2dab8)
The two different branches put 0x48000000 into either r0 or r4, but radare shows the argument orders the same for both. r2's emulated assembly is an absolute mystery to me.
I think this piece of code might actually be the start of the phy init stuff. When it is called r3 I think is set to 0x48000000. A small problem is that pieces of this don't disassemble on their own.
`- args() vars(9:sp[0x2c..0x4c])
0x000229e0 2de9f04f push.w {r4, r5, r6, r7, r8, sb, sl, fp, lr} ; sp=0xffffffffffffffdc ; [
0x000229e4 0027 movs r7, 0 ; r7=0x0
0x000229e6 8bb0 sub sp, 0x2c ; sp=0xffffffb0
