Distributed switch architecture

Distributed switch architecture known as DSA has been out in the market since Marvell's first debut in 2004. DSA was targeted at providing a solution for distributed cascade or stacking broadband switching/routing topologies. This brings a lot flexibility in distributed network system with low cost while high switching performance. On the left is a simple picture which explains one of the in-memory switch structure.

I am not going to talk too much on the technology here but introduce how to bring this type of chip up from an external processor.

Normally, what you would have is an external processor connecting to the DSA chip via MII (Media Independent Interface) bus. If you have worked in the device driver level for ethernet control card or phy device you should be quite familar with this bus. Within this bus, there is a couple of lines dedicated as MDIO (Management Data Input/Output) bus, where most of the Phy device control data will be flowing over.

• MDIO is a bus structure defined for the Ethernet protocol. It is used to connect MAC devices with PHY devices, providing a standardized access method to internal registers of PHY devices.

• MII is a standard interface used to connect Ethernet MAC-block to a PHY, which means any types of PHY devices can/should be supported and accessed via a defined standard register set. Equivalents of MII for other speeds are AUI (for 10 megabit Ethernet), GMII (for gigabit Ethernet), and XAUI (for 10 gigabit Ethernet).

REG 0 - Basic Mode Configuration

REG 1 - Status Word (You can use this to detect whether an Ethernet NIC is connected to a network)

REG 2,3 - PHY Identification

REG 4 - Ability Advertisement

REG 5 - Link Partner Ability

REG 6 - Auto Negotiation Expansion

• Phy is the actually device which connects to the physical connection (i.e. cable or wireless channel). It provides a set of registers which forms an interface to allow (phy device) drivers to manipulate. While these devices are distinct from the network devices, and conform to a standard layout for the registers, it has been common practice to integrate the Phy driver together with the network driver. BUT I DO NOT RECOMMEND THIS!

The DSP chips are usually embedded in access points and routers, and a typical setup with a DSA switch looks something like this:

From the CPU point of view, this external (DSA) device needs to be driven to take switch configuration and commands. So immediately, you might get the first clue that we need some sort of device driver to initialise and drive the DSA chip. Using Linux as an example. Supposing you have Linux kernel running on the left side CPU, you could develop your own device driver to talk to the MII bus (actually the party you are really interested in talking to is the device on the other end via MDIO bus), which is well defined in linux kernel as mdio_bus module.

The problem is, the switch is a child node on the MDIO bus but it is NOT a standard Phy device although it normally have a similar set of Phy registers for you to poke around. Well, you have got to know what you are poking here!

Two ways of drive the switch, in short. One is the quick but dirty way. Where you can still use the support from Phy device and define your own private module data structure which maps to the specific layout of register set on the non-Phy switch device. Using lower level MDIO bus read and write to control the registers directly to issue commands. (The switch will normally have a defined list of register bit value to the switching or configuration commands.) This approach will be suitable if you are working on an early version of 2.6 kernel which do not have DSA kernel driver extension.

If you are working on the latest version of Linux kernel then you are lucky enough to use the DSA framework where a nice structure of dsa_switch is introduced. By using this, you can literaturely treat DSA device as an OS external device driven by standard kernel device driver. DSA module has taken care of all the lower level bus access for you. All is left is for you to find out what value to be written to what address from the datasheet (normally you have to obtain them from the chip manufacturers under NDA though). You can use your driver to iterate through all the ports on the switch and presents each port (on the switch) as a separate

network interface to Linux, polls the switch to maintain software link state of those ports, forwards MII management interface accesses to those network interfaces (e.g. as done by ethtool) to the switch, and exposes the switch's hardware statistics counters via the appropriate Linux kernel interfaces (i.e. ioctl is what I did).

Just be aware that both approach do not follow through the generic Phy state machine, so you will have to define it yourself of the handlers (either in phy_device and phy_driver for the first approach or the dsa_switch_driver such as poll_link handler).