Performances & stability of PCIe?

Hi

Playing and exploring my brand new tachyon…

What are the expected performances of tachyon with PCIe ?

I have setup a pimoroni PCIe NVMe HAT with a ‘Samsung SSD 990 Pro NVMe M.2 Pcle 4.0’. Theorical max perf of the SSD is 7.5GB/s. Obviously, I do not expect such perf on tachyon but still, doing the suggested

dd if=/dev/zero of=/mnt/test.bin bs=4k iflag=fullblock,count_bytes count=1G

I get only a consistent 180M/s which is quite disappointing…..

Mounting the file system takes 30s (an empty and clean one). Unmounting it takes a little bit more than 1min

There are a couple of other issues:

  • with heavy usage of the disk (fsck for example), wifi shutdown and reconnect when done.
  • trying bigger block size to see if it has positive impact with:

sudo dd if=/dev/zero of=/Develop/test.bin bs=1M count=1024

made the tachyon restart

  • to be confirmed, but when asked to shutdown with SSD’s file system mounted, the tachyon does not end the shutdown and get stuck. Need to force the shutdown with button. Then at the next restart, the file system needs a check so mounting it takes around 3 min. But maybe it is because the shutdown does a umount and I get too impatient to wait ..

Is the Samsung SSD guilty (I doubt it because it goes super fast on raspberry) ? a setup that is missing ? Issues that are in the queue ?

Thx for advice

Alain Charroux

Hi Alain,

Thanks for sharing all the detail - and glad to hear you’ve got Tachyon running with PCIe! Always great to see folks trying these kinds of setups!

On the numbers: in our own tests we usually see around 270 MB/s on PCIe with this HAT using an off the shelf M.2 disk. That’s a lot lower than the theoretical max of your Samsung drive, but there are a couple of reasons:

  • One lane only - The Raspberry Pi–compatible connector only gives us a single PCIe 3.0 lane, so there’s a hard cap well below what the SSD can do in a PC.

  • Caching - The current settings aren’t very aggressive, which means more direct I/O instead of buffering, and that really holds performance back.

  • Shared resources - The Qualcomm chip shares memory bandwidth across CPU, GPU, PCIe, USB, display, and the modem. We’ve been experimenting with how that’s allocated - you can “steal” more bandwidth for PCIe, but it comes at the expense of things like modem or display throughput. We haven’t hit the magic balance yet but the controls allow a lot of leeway in the behavior. Right now, we are very 'balanced' meaning that everything works, but some peripherals are capped. To change this, we'd need to expose more controls with caveats on them which is a more complex piece of work.

So what you’re seeing isn’t unusual. However, we have not seen the side effects you mention (Wi-Fi dropping under load, slow shutdowns, etc.) - I would love to escalate those issues first as they are not as per expectation. I'll pick up one of those drives now on Amazon and give it a go myself.

Really appreciate you pushing the hardware and sharing back your results. We’ll keep posting updates as we dial in the PCIe tuning.

Thanks!

Nick

@mrlambchop

On Raspberry PI 5, in the config.txt you can set pci lanes to gen 3 which gives a nice boost to throughput. Is there a similar setting in the config file or ? to set pci bus speed for Tachyon?

For RPi 5 it is

dtparam=pciex1
dtparam=pciex1_gen=3

Hi all,

I am exploring the possibility of using the Seeed PCIe 3.0 Dual M.2 HAT (based on the ASMedia ASM2806 switch) with Tachyon.

The device supports two M.2 NVMe slots (2230/2242/2260/2280) with a full PCIe Gen3 x1 interface and a total bandwidth of up to 8 GT/s, which theoretically provides read/write speeds of over 800 MB/s.

Power is supplied both through the PCIe connector (around 1A) and additional pogo pins (up to 2A), ensuring stable operation with power-hungry NVMe SSDs.

Questions:

  • Is it reasonable to expect a real performance increase compared to the Pimoroni NVMe Base Duo, which uses PCIe Gen2 and offers roughly 450 MB/s total bandwidth?
  • How might the ASM2806 switch and the power delivery characteristics affect stability and actual throughput when used with Tachyon?
  • Given the limitation of a single PCIe Gen3 x1 lane on the Raspberry Pi 5, how effective is using two NVMe SSDs simultaneously?

The goal is to understand the practical benefits of the Seeed PCIe 3.0 Dual M.2 HAT for high-speed storage with Tachyon, including power consumption and stability considerations.

Seeed PCIe 3.0 Dual M.2 HAT: PCIe3.0 Switch to dual M.2 hat for Raspberry Pi 5, Support NVMe SSD, Support Hailo8/8L
Pimoroni NVMe Base Duo: NVMe Base Duo for Raspberry Pi 5 - The Pi Hut

I’ve been experimenting with what might be a slightly unusual setup on the Tachyon and wanted to share, since it seems to connect with what the devs mentioned about PCIe lane sharing between the modem, Wi-Fi, etc.

Setup

  • Dual hat installed

    • Slot 1: 12-bit SDR (various Msps tested)

    • Slot 2: High-speed storage drive

Observations

  • SDR drivers install and operate fine at different sample rates.

  • The hat drive itself benchmarks well when tested directly.

  • But recording SDR data to that drive—even at just 1–2 Msps—causes (I think) the PCIe lane to collapse: SDR buffer fills, connection fails.

I tested other storage paths:

  • USB3 external drive → same failure.

  • High-speed SD card → same failure.

  • Onboard eMMC → works flawlessly, no issues at any sample rate.

This definitely feels like a resource-sharing issue. The developers already confirmed PCIe allocation is shared between the modem and Wi-Fi, but my results suggest that USB3 and SD card might also be in the mix (or at least affected by the same bottleneck).

So until user-tweakable PCIe allocation settings are exposed, it looks like SDR→storage workflows are going to run into this limitation.

Has anyone else tried a similar configuration, or found a reliable workaround?