Have your own toolchain

Toolchain provides you the capability being able to compile your code in your way to be run on your own hardware platform. For most of programmers, in majority of their career, are trying to learn to speak (code in) one or more common language, where syntax has been predefined, compilation approaches have been chosen, assembly set has been given, hardware has been decided by the boss :-).

Now, in an ideal world (I did say 'ideal'), you would design yourself an extremely powerful chip, with unlimited number of processing cores, running any instructions in nano-second.

In an ideal world, you would probably want to cook your own assembly set to be run on this powerful chip, the first one for me would be - SORTITALLOUT.

In an ideal world, you would also probably consider to write your own compiler, implementing those funky optimisation algorithms you came across only in research lab decades back in your youth.

At last, an easy one - one thing pretty much everyone else is doing nowadays - defining your own language syntax, yeah, you are right, a new language, and convince whoever you can convince to use and re-populate your civilisation.

To have all four in place, you are pretty much in control of your own life. In real world, we tend to settle with compromise. Here we tend to pick one or two of these four items and reuse some efforts other brilliant geeks have been messing with. The normal ingredients in the toolchain include:

• An assembler - to convert your assembly instructions into 000111....1..0., the only format machine really understand.
• A linker - to put pieces of brilliant moments together into a big 000111......1100.
• A compiler - with which your favourite programming language into a bunch of assembly - lower level instruction set. Yes, you are right, who need another level if we have assembly defined already? Many good and bad memories prove that you might have a point there, as in the early days, we all have one weapon call assembly code.

These three are absolutely necessary for a micro-version of toolchain. Now if you have some code already, pump them bottom up, you will be talking to your machine now. However, if you intend to get serious, you will need the following:

• A debugger - whatever interface it is (my personal view is GUI debugger suck more than they contribute). This is a whole topic by itself and I don't really know much about so I will skip it.
• Libraries - only if you ever want to rebuild the whole new world again all by yourself, you are going to need some of proven work done by other genius just like you, some common funcationalities everyone will come across. "This idea that there is generality in the specific is of far-reaching importance. — Douglas Hofstadter, Gödel, Escher, Bach"
• Utilities - copy, dump, whatever verb you can think of. These tools are really food for work, and thought.

All about USB

Pre-defined USB device classes provide standard USB drivers for the devices falling into these classes. Otherwise USB device vendor will provide their own.

USB Device driver = USB Host driver (Host controller/Hub/Master) OR USB gadget (Device/Slave)

Host - Linux

LDD3 - USB

Endpoint - The basic uni-direction data channel, either from Host to Device (OUT) or Device to Host (IN)

• CONTROL - configure, retrieve device information, send commands to device, retrieve device status report. Every USB device has a control endpoint "endpoint 0", used by USB core to configure the device at insertion time.
• INTERRUPT - endpoints transfer small amount of data at a fixed rate everytime Host asks Device for data. These endpoints are the primary transportation method for USB mice, keyboards. Also can be used by Host to send control command to Device. NOT generally for large amount of data.
• BUILK - for large amount of data transfer. Commonly used in printers, storage, and network devices to transfer bulk packets from or to devices. Data MUST gets through without loss, but time could NOT be guaranteed.
• ISOCHRONOUS - also for large amount of data transfer, but data loss is possible. Used in application for constant stream of data flowing such as real time data collection, audio and video streaming.

CONTROL and BULK types of Endpoints are used in asynchronous transfer. INTERRUPT and ISOCHRONOUS are periodic. In Linux, usb_host_endpoint structure describe an endpoint, which contains a usb_endpoint_descriptor.

Interface - a bundle of Endpoints, handles one type of USB logical connection, such as mouse, keyboard, audio stream. Some USB devices have multiple interfaces (for example, an USB speaker device has one interface for audio stream, one interface for keyboard for the buttons). USB device driver and USB Interface is a one-to-one mapping relation. Each USB driver controls one interface in an USB devices. So it is likely we need multiple USB drivers to control one USB hardware device. USB driver is bound to USB Interface.

Interface structure in the kernel is usb_interface. USB core passes usb_interface structure to USB driver to control.

Each USB device will have a USB major number. Every interface will be assigned a minor number by USB core via calling usb_register_dev.

Configuration - a bundle of Interfaces. A USB device can have multiple configurations to switch in between to change the state of the device. In the kernel, structure usb_host_config represents a Configuration. usb_device represents entire USB device.

Descriptors

All USB devices have a hierarchy of descriptors which describe to the host information such as what the device is, who makes it, what version of USB it supports, how many ways it can be configured, the number of endpoints and their types etc. The more common USB descriptors are

• Device Descriptors

• Configuration Descriptors

• Interface Descriptors

• Endpoint Descriptors

• String Descriptors

LDD3 - USB

Enumeration - is the process of determining what device has just been connected to the bus and what parameters it requires such as power consumption, number and type of endpoint(s), class of product etc. The host will then assign the device an address and enable a configuration allowing the device to transfer data on the bus. A fairly generic enumeration process is detailed in section 9.1.2 of the USB specification.

USB File system

The first USB device is a root hub, or known as USB controller, usually contained in a PCI device acting as a bridge between PCI bus and USB bus. It is also the first USB device on USB bus. All root hubs will be assigned unique number by USB core.

USB sysfs device naming scheme: root_hub-hub_port:config.interface

[USB Selective Suspend - The USB selective suspend feature allows the hub driver to suspend an individual port without affecting the operation of the other ports on the hub.]

Kernel will bind the correct USB driver to interfaces specified by configuration based on bConfigurationValue in a USB device.

sysfs (/sys/) stops at the interface level without exposing all parts of USB device.

Detail information of endpoints can be found in usbfs filesystem, mounted in /proc/bus/usb/. /proc/bus/usb/devices will show all configurations and endpoints in the system. It also can allow user space programs talk directly to USB devices. (Hence user space USB driver is possible to be in user space rather than kernel.)

Data transfer

Via URB

usb_alloc_urb usb_xxxxxpipe usb_submit_urb

create URB URB URB Notify

Apps ---> USB device driver ---------------> USB device driver assign URB to endpoint -------> USB core -------> USB host controller (for this USB device) <> ---------> USB device driver

Different URBs are created to be bound to different types of endpoints. After a urb has been submitted to the USB core successfully, it should never try to access any fields of the urb structure until the complete function is called.

• When the function is usb_kill_urb, the urb lifecycle is stopped. This function is usually used when the device is disconnected from the system, in the disconnect callback.

• usb_unlink_urb function should be used to tell the USB core to stop an urb. This function does not wait for the urb to be fully stopped before returningto the caller.

Without URB

Synchrounous usb_bulk_msg, usb_control_msg

To write a USB driver

Same thing to other subsystem drivers really, create driver structure -> register to USB kernel subsystem (-> create char driver interface for USB devices)

USB core handles the addition and removal of USB devices within a single thread, so any slow device driver can cause the USB device detection time to slow down and become noticeable by the user.

VCP over USB

COM port redirector is a powerful concept in hardware connection limited device.

USB CDC ACM: Abstract Control Model. This access model provides a serial (COM) like interface to a USB device. Each CDC-ACM channel utilizes 3 USB end-points for end-to-end communication. Linux has a generic CDC ACM host-side implementation in the kernel module cdc_acm.ko.

USB CDC ECM: Ethernet Networking Control Model. This access model provides a standardized networking interface for handling of IP transmission over a physical interface such as USB.

USB CDC OBEX: Object Exchange

http://www.jungo.com/st/embedded_usb_cdc.html

Gadget - Linux

Instead of Host controller, we have peripheral controller taking it place in USB 'slave' device end.

Typical gadget drivers under Linux are:

• Gadget Zero

• Ethernet over USB (To support different host, there are different modes: CDC Ethernet, CDC subset, RNDIS)

• GadgetFS

• File-backed storage (implementing USB Mass storage class)

• Serial (implementing CDC ACM class)

• MIDI

R&R

I have been my new role for almost a month now, a month within which I only spent two nights at my newly purchased house.
What is different to me is that I got to spend a lot of time with customer and had chance to really understand the source of all information. This leads me to start thinking about:

1. The amount of information lost during transmission is unbelievable significant. As a startup, we only recruit people who have been working in this field for long enough time with established records. You might think that we are lucky enough that these pros would be able to interpret, parse and transmit information more accurately and timely. However, the fact turned out that we have got some priority wrong - one particular problem customer has been waiting for over two months as it blocked their development, while our engineers have no idea and were plodding through the bug list in alpha beta order, project team in customer side has been held up. On second thought, it seems while FAEs were focus on the problem to be resolved or tons of emails to reply, or evaluate whether it is ok to nag again for the bug updates, these poor fellows have lost the big picture and ability to make judgement calls.

2. Management is all about communication is exemplified to its extreme in customer management role like this. I have one crazy principle - when I decide to go to bed at night, there will be no unprocessed emails in my inbox, even sometimes it means banging Del key 10 times in a row per second. Because the nature of the role, it seems all natural and makes sense that to keep the communication flow healthily is all is required for this role. You can try that, but my experience tells me that after a very short of time, you will notice that you are not the one in control anymore - not for schedule, not for customer project scope, lose track of what the heck is going on within your engineering department. Why? Simple, because you gave up the right to control! Managing information channel is an auditing job at its best. While it is so important as I explained in previous section, if you choose the easy way out - by managing the results of people's actions reactively, you forgot about the source. If there are some areas could be improved, practice could be less time consuming, process could be less painful and distracting, the best way is to cure the source, to get on top of the fundamental reasons behind symptoms. Real managers manage people, practice, process; mediocre ones massage information; bad managers react to situations.

3. At last, one questions bothers me mostly through half of my professional career is, have we given up the right and ability to be creative, to do something for real? There are numerous blogs and articles talking about how managers with good hands on engineering background tend to screw big time by micro-managing and mistaken the purpose of his/her role just because they got too attached to write few lines of code, carry out some tests, define some protocol etc. By doing this, they lose their focus in their own jobs. Frankly I have probably read and written too much of this to the point everytime this topic pops up, I just want to shout, WHAT THE FUCK a manager's job should be?

Simple definition for manager from my dictionary - a guy whose job is to do whatever is required to accomplish team's goal.

If you need someone to get the code done and the only possible person is you, guess what, you are going to write some code. If you happen to be an expert in some areas your team require some advice, don't be shy, that IS your job. Hard isn't it? That's the price you pay when you want to be on the spotlight.