From: Piotr Mienkowski []
Sent: Wednesday, January 04, 2017 9:25 AM
Subject: [devel] Networking stack - Ethernet driver design


Hi all,

I have a few questions/discussion points related to the new networking stack in context of Ethernet driver development.

  1. Currently Ethernet drivers are by default initialized before the networking stack (as set by CONFIG_ETH_INIT_PRIORITY). That's going to be problematic for a zero-copy implementation of Ethernet driver. Such driver will need to reserve during the initialization phase a set of net data buffers where the incoming packets can be stored. Later when the complete frame is received these buffers will be passed to the higher layer. However, the net buffer pool is initialized by the networking stack so reserving net buffers is only possible after networking stack was initialized. That implies that the Ethernet driver should be initialized after the networking stack. Secondly, even in case of the more typical implementation of the Ethernet driver, the one which has its own set of RX/TX buffers and copies data between them and net buffers, as currently done in Zephyr, the driver will start working immediately after being initialized. If it receives a frame just at that moment it will try to pass it to the higher layer even if the rest of the networking stack was not yet initialized. Once again that implies that the Ethernet driver should be initialized after the networking stack is.

[chuckj] I would think if the Enet driver comes alive FIRST, and receives something, but no protocol stack has attached itself yet, that it could simply DROP the packet. ENET allows you to drop packets anywhere, anytime. So I could imagine the ENET driver coming alive first, doing its own initialization, acquiring its own buffers. Further, it should be agnostic as to which protocol stack attaches to it. There are protocols that have something OTHER than TCP/IP over ENET. For example, ATM packets over ENET, or any other encapsulation. Although, admittedly, this is very unlikely to appear with Zephyr.

So one way to design this is that the DRIVER owns the buffers, and that the buffers must always be returned to the driver when the protocol stack has consumed the packet. The API the protocol stack uses is a call to read a packet, getting a new buffer, and then another API to return the buffer later – to keep it zero copy. Sending would first have the application acquire a buffer, then fill it with a packet, then transmit it – with an attempt to utilize scatter-gather techniques to avoid memory copying (i.e. payload is not copied).

  1. Modern Ethernet modules in most of the SoC devices have ability to generate IP, TCP and UDP checksums in hardware. Is it possible to tell networking stack not to compute these checksums in software?

This would be another configurable thing I guess. Also ENDIAN swap. Also the 2-byte GAP can be a problem between ENET header and IP header – if hardware requires 32-bit alignment. There are a lot of little things like this that some hardware can handle and some hardware cannot. I suppose all of these things should be configurable.

  1. One of the parameters to NET_DEVICE_INIT is MTU. Shouldn't we have a set of predefined constants provided by the networking stack for some typical interfaces. E.g. NET_MTU_ETHERNET set to 1500? Should we configure the MTU value in Kconfig?

[chuckj] Most of the time MTU 1500 is probably just fine. Where I have seen people set this bigger is when streaming video (or something else) requires packets that are a lot bigger for efficiency. So YES having this configurable might be a good idea – although on low-end Zephyr devices, with modest memory sizes, modest MAC/PHY, its probably unlikely to be set big.

When TCP-offload devices are used, I would think there are large set of things to configure.