PCIe For Hackers: Our M.2 Card Is Done [Hackaday]

View Article on Hackaday

We’ve started designing a PCIe card last week, an adapter from M.2 E-key to E-key, that adds an extra link to the E-key slot it carries – useful for fully utilizing a few rare but fancy E-key cards. By now, the schematic is done, the component placement has been figured out, and we only need to route the differential pairs – should be simple, right? Buckle up.

Getting Diffpairs Done

PCIe needs TX pairs connected to RX on another end, like UART – and this is non-negotiable. Connectors will use host-side naming, and vice-versa. As the diagram demonstrates, we connect the socket’s TX to chip’s RX and vice-versa; if we ever get confused, the laptop schematic is there to help us make things clear. To sum up, we only need to flip the names on the link coming to the PCIe switch, since the PCIe switch acts as a device on the card; the two links from the switch go to the E-key socket, and for that socket’s purposes, the PCIe switch acts as a host.

While initially routing this board, I absolutely forgot about one more important thing for PCIe – series capacitors on every data pair, on the host TX side of the link. We need three capacitor pairs here – on TX of the PCIe switch uplink, and two pairs on TX side of the switch – again, naming is host-side. I only remembered this after having finished routing all the diffpairs, and, after a bit of deliberation, I decided that this is my chance to try 0201 capacitors. For that, I took the footprints from [Christoph]‘s wonderful project, called “Effect of moon phase on tombstoning” – with such a name, these footprints have got to be good.

We’ve talked about differential pair calculations before in one of the PCIe articles, and there was a demo video too! That said, let’s repeat the calculations on this one – I’ll show how to get from “PCB fab website information” to “proper width and clearance diffpairs”, with a few fun shortcuts. Our setup is, once again, having signals on outer layers, referenced to the ground layer right below them. I, sadly, don’t yet understand how to calculate differential impedance for signal layers sandwiched between two ground planes, which is to say – if there’s any commenters willing to share this knowledge, I’d appreciate your input tremendously! For now, I don’t see that there’d be a tangible benefit to such an arrangement, anyway.

Different Stackup Might Help

This time, I’m going with a 4-layer 0.8 mm stackup – otherwise the board won’t fit into an M.2 socket. According to the design rules, I can go down to 3.5 mil (0.09 mm) traces and spacings, as opposed to the usual 6 mil (0.16 mm) traces I’m used to when doing generic 2-layer boards. Initially, I’m choosing the 7628 stackup variant here – the main difference between stackups here is the prepreg thickness and the dielectric constant, which impacts minimum possible diffpair thickness and spacing.

I went by parameters on the JLCPCB stackup page, and took via parameters from the ordering page – you can put these parameters into the “File => Board Setup” window, in the “Net Classes” tab. After replacing the default parameters there with the 4-layer impedance controlled process ones – clearance, minimum trace width, via size and such, – we get some pretty lovely parameters that we can drop down to if there’s ever a tight spot, and an ability to do reasonably dense component placement.

Let’s aim for on-point 85 ohm differential impedance today – a great target wherever you can afford it. Again, traces on top, uninterrupted ground plane right under them, along the entire length of the pairs. For “7628” stackup, that means there’s 0.21 mm of material with 4.6 Er between the pairs and the ground – punch these two values into the calculator, leave copper thickness at 35 um (1oz copper), and we can play around with trace space and width values, down to our 0.09 mm limit – which leads us to a 0.225 mm width / 0.09 mm space option. This isn’t all that great space-wise, however.

You don’t have to stick with the default stackup, though! After little deliberation, I switched to the “3313” stackup – with 4.05 Er and 0.1 mm thick prepreg between top and middle layers. It seems to be a tad more expensive, but it does look a bit easier to route in the small amount of space that I have. This led me to 0.135 mm / 0.09 mm pairs, while keeping the same 85 ohms differential impedance. Now, all I need to do is to enter these parameters into the “Net Classes” table, and whenever I press ‘6’, I’ll immediately start drawing a 85 ohm impedance differential pair.

Diffpair Drawing

The IC is placed, the diffpair parameters are tuned – we can start drawing. Remember, flipping the PCIe pair polarity isn’t a crime, it’s an obligation to make routing easier on yourself. Somehow, in this design it wasn’t needed even once.

You do, however, need a bit of patience – KiCad’s differential pair drawing experience isn’t always smooth; if you’re having weird problems where one end of the pair doesn’t quite connect to another end, you’re not alone. For me, drawing these pairs was a tad infuriating. It’s conceivable that what I’m dealing with something commonly referred to as a skill issue – perhaps, there’s a setting I’m not noticing, as sometimes such “does not connect” problems are caused by DRC constraints. However, in that case, there will be a warning shown to you on the bar above.

Remember, whenever your diffpair crosses from top to bottom layer or vice-versa, it also changes reference layers, from In1 to In2 respectively – and you’ll want to add ground vias near the diffpair vias, so that the ground return current can travel along the pair, too. As usual, four vias are ideal, three are good, two are okay, and one is non-ideal but better than zero.

Once we’ve drawn diffpairs and added ground vias, there’s one last thing to take care of. While PCIe pairs don’t have to all be the same length, the two traces that each pair consists of, do have to be the same end to end. For that, you can use the differential pair skew correction tool – mapped to 9 on the keyboard. It will add a wiggle to the longer track, in the spot where a wiggle would fit best.

All The Layers

When drawing boards, you can often use one GND and one VCC internal layer – as technically, both VCC and GND can be used as reference planes for high-speed signals. However, since we’re crossing diffpairs between top and bottom layers, they would then have different references, and this is still unexplored territory for me – I’m not sure whether there’d be return current problems. I’ve started out with one 1.2 V and one GND internal plane, but afterwards, as I was sanity-checking my boards and writing this article, I talked with other engineers, re-read an interesting comment thread under the first PCIe diffpair article, and decided to switch to having both internal planes as GND.

When it comes to the outer layers where diffpairs go, you’ll want to remove the ground fill or move it away, so that the ground fill around the pair will affect the diffpair’s impedance – you are highly likely see a ground keepout if you take a look at any PCIe card you own. Here, I am going to route diffpairs on the outer layers, but I’m not not going to completely remove ground on these, using keepouts instead. First off, having ground is cool – it’s extra copper that can help dissipate heat from the switch, or the switching regulator, or both. Second thing is – JLCPCB has been weird about ground fills on dense boards recently, and I’ll want to avoid that.

How far should the keepout go, then? The 5L rule (five trace widths away) is a good one, or you can just keepout the entire area where you’re pulling the differential pairs. For us, 5L means 5 * 0.135 mm, or 0.685 mm distance between the pair/via and the ground fill. The 5L rule isn’t a hard limit – keep to it as much as possible, but don’t stress about having a few vias here and there. Remember, when wiring up PCIe, it’s important that the string is wet.

For power wires, use your thickest traces possible wherever you have space. Of course, you don’t need to pull a 2 mm track wherever you want to put 3.3 V, but having 0.6 mm or 1 mm tracks on a 3.3 V 1 A path is pretty ordinary – some will say it’s overkill, but if you have enough space, there’s hardly a benefit to not doing it. If you’re wondering what you can get away with, there are trace width calculators that will give you temperature increase and inductance values, but most of the time, making the trace thicker is a no-brainer.

After we wire up PCIe and power signals, there are a few things left. It might be tempting to route them on the inner ground layers – do your best to fight the temptation, however, as there’s usually a better way; having unbroken inner planes is widely accepted to be good mojo for diffpairs, and for signals in general. Instead, consider other nonstandard options that are less of a sin – for instance, it’s okay to remove solder pads from a footprint if the pad is unused, and that’s what I’m going to do to pull one of the PERST signals through the top layer.

Mistakes And Problems

In many aspects, this design hits the bullseye. It’s compact, it fits everything it should fit with space to spare, and the onboard power delivery is more than reasonable. Of course, there are a few possible hiccups I can foresee happening that I will check for once these boards are manufactured.

A PCIe switch functions at high speeds, so it makes sense that power consumption could be a bit harsh, and same would go for heat dissipation – indeed, you will see higher-port-count chips like these covered with glued-on heatsinks on Chinese adapters; My main concern is power consumption – a M.2 E-key card is expected to consume 2 A at most, and the peak power consumption of the switch, together with the inserted card itself, might exceed this budget for sure.

To solve that, I’m adding a separate power input, and drawing traces in a way that, if needed, it’s easy to separate the 1.2V  regulator 3.3 V input from the E-key card power. My second concern about the power consumption, however, is thermal dissipation – there isn’t much thermal mass in this PCB, and the ground path isn’t too straightforward, so the chip could theoretically overheat under load. There wouldn’t really be place for a heatsink once the adapter’s plugged into a laptop, either – we’ll see just how much of a problem this is in practice.

Every board can be a small experiment, and this one’s definitely got a few things I’ve never tried before. I’ve added this board to my last PCB order, and once it arrives, I’ll share the end result with you all, one way or another. Until then, the files are on GitHub, and I hope that this board’s story gives you plenty of insights into designing with PCIe!