The basic idea is fairly simple – the goal is to make it as easy as possible for someone to read and understand the change that’s being made.

• Make sure the code matches the coding style of the code being modified.
• Make patches as small and self contained as possible – having many patches is much easier than having complicated patches.
• Provide a changelog clearly describing everything that’s going on in the patch.

The result here should be something that’s easy for people to review. Keeping in line with the coding style is just a generally good idea for code legibility which is obviously a key thing for people reading the code to review it. Having a clear description of the change means that the reviewer knows what the patch is supposed to do and can verify that the patch does that rather than having to spend effort figuring out if whatever the code is doing is intentional. Making changes small and simple makes them much easier to think about – the idea itself is easier to keep in your head and there’s fewer things to check so the process of validating things is easier.

It’s not just about making life easier for reviewers. The process of making simple changes and describing them clearly can be really helpful for spotting issues when writing code; it forces you to think through what’s being done much more clearly than might happen otherwise. There’s also an ongoing benefit to anyone working with the code in the future. Since each change ends up including at least some explanation of what the code is supposed to do and since revision control systems generally have some equivalent of git annotate it’s usually reasonably straightforward to find the changes that introduced the code that you’re looking at. Small changes with good changelog entries mean it’s much more likely that the changelog will provide a useful explanation as to what the author was thinking and explain why the code is the way that it is. It’s much harder for documentation to go missing from the SCM than it is for it to go missing elsewhere, and it’s much more likely that documentation in the commit is going to be up to date than external documentation, especially prewritten design documentation.

In the context of large scale reviews the usual technique is to combine reviews of individual changes with a read through of the final code after all the individual changes have been applied. Often the build up of the code through incremental changes is a enough to show the overall design but a read through of the final code can be helpful in allowing people to understand where things are going.

With kernel development the approach is much more to make sure that the core can cope with whatever needs expressing. It’s not always done in the core – for example, sometimes the issue is due to unusually limited hardware and very likely to never come up again – but it’s very unusual to see code merged that doesn’t at least fit in with the design of the subsystem.

This post is inspired by a brief exchange with Thomas Gleixner on linux-kernel who articulated the advantages of doing things this way very well:

 - It's usually simpler and faster as the core code has all the
   necessary information. So that's even an argument which managers
   might understand.

 - Such workarounds, when not caught, tend to spread themself
   magically because driver writers checkout the existing code of
   similar devices and copy/paste/modify^Wuglify them over and over.

 - In the worst case such workarounds make the core maintainence
   harder and in some cases impossible, because they silently imply
   semantics on the core w/o the core maintainer knowing about them.

The last point is the most important one to me as a subsystem maintainer – I really don’t want to be merging code that is going to create issues with ongoing development of the subsystem. This is all another facet of the policy on stable APIs: APIs are only worth working around if they’re fixed, and the assumptions that end up embedded in a workaroud are as much part of the API as the explicit interfaces.

It’s things like this that make Linux such a pleasure to work on.

There’s a bunch of existing documentation out there for the trace subsystem, including some in the kernel under Documentation/trace, which I won’t repeat here. The ASoC events are grouped together under ‘asoc’. The raw log can be viewed by looking at the file tracing/trace in debugfs. Here’s a sample output from starting playback of an MP3 on a Samsung SMDK6410 reference system using WM8580, a high performance six channel CODEC with a very simple software interface:

mplayer-1673  [000]   183.920000: snd_soc_dapm_start: card=SMDK-I2S
mplayer-1673  [000]   183.920000: snd_soc_dapm_widget_power: widget=Rear val=1
mplayer-1673  [000]   183.920000: snd_soc_dapm_widget_power: widget=Center+Sub val=1
mplayer-1673  [000]   183.920000: snd_soc_dapm_widget_power: widget=Front val=1
mplayer-1673  [000]   183.920000: snd_soc_dapm_widget_power: widget=VOUT3R val=1
mplayer-1673  [000]   183.920000: snd_soc_dapm_widget_power: widget=VOUT3L val=1
mplayer-1673  [000]   183.920000: snd_soc_dapm_widget_power: widget=VOUT2R val=1
mplayer-1673  [000]   183.920000: snd_soc_dapm_widget_power: widget=VOUT2L val=1
mplayer-1673  [000]   183.920000: snd_soc_dapm_widget_power: widget=VOUT1R val=1
mplayer-1673  [000]   183.920000: snd_soc_dapm_widget_power: widget=VOUT1L val=1
mplayer-1673  [000]   183.920000: snd_soc_dapm_widget_power: widget=DAC3 val=1
mplayer-1673  [000]   183.920000: snd_soc_dapm_widget_power: widget=DAC2 val=1
mplayer-1673  [000]   183.920000: snd_soc_dapm_widget_power: widget=DAC1 val=1
mplayer-1673  [000]   183.920000: snd_soc_bias_level_start: card=SMDK-I2S val=2
mplayer-1673  [000]   183.920000: snd_soc_bias_level_done: card=SMDK-I2S val=2
mplayer-1673  [000]   183.920000: snd_soc_reg_read: codec=wm8580-codec.0-001b.27 reg=32 val=1e
mplayer-1673  [000]   183.920000: snd_soc_reg_write: codec=wm8580-codec.0-001b.27 reg=32 val=2
mplayer-1673  [000]   183.920000: snd_soc_bias_level_start: card=SMDK-I2S val=3
mplayer-1673  [000]   183.920000: snd_soc_bias_level_done: card=SMDK-I2S val=3
mplayer-1673  [000]   183.920000: snd_soc_dapm_done: card=SMDK-I2S
mplayer-1673  [000]   183.920000: snd_soc_reg_read: codec=wm8580-codec.0-001b.27 reg=13 val=10
mplayer-1673  [000]   183.920000: snd_soc_reg_write: codec=wm8580-codec.0-001b.27 reg=13 val=0

Each line represents a trace event, showing the process that initiated the event, the CPU it ran on and the time it was recorded at together with event-specific details. On this system the time is reported with 10ms accuracy and the CODEC is very simple so the timing information is not terribly informative, but more accurate timers can be really helpful in analysing what’s going on with more complex CODECs. The set of events available will change over time, the key ones currently present in the kernel are:

• snd_soc_dapm_start and snd_soc_dapm_done: These events show the start and stop of the DAPM algorithms, showing the time spent on power management decisions.
• snd_soc_dapm_widget_power events are generated for each of the widgets which DAPM has determined needs to change state. The ‘val=1′ means that all these widgets are being powered up, val=0 would show power down.
• snd_soc_reg_write events show values written to registers.
• snd_soc_reg_read events show values read from registers – these reads may be done from the register cache, or may be from the hardware.
• snd_soc_dapm_widget_event_start and snd_soc_dapm_widget_event_done show the time spent in per-widget event callbacks.

The set of events traced will change as the subsystem develops and as people gain more experience with using the trace API.

There’s a few  workflows for dealing with this. My preferred one is to track the appearance of my patches in Stephen Rothwell’s linux-next tree, which tracks individual development trees destined for merge into Linus’ tree. I create a git branch based on the current state of this tree then apply the patches I’m submitting on top of that. This lets me spot any potential merge conflicts that they’ll create but the main function is to allow me to come back to the branch later and track which of the patches has shown up in one of the trees that Stephen tracks. To do this I rebase the branch onto the current state of linux-next:

git rebase --onto next/master old-master

where ‘old-master’ is the last linux-next commit in the branch. This will flag up any merge issues that have come up due to changes in other trees and will also handle patches that are already present in one of the trees in -next by dropping my local version. The end result is a branch based on the new linux-next with all the patches that were not yet applied in it. I can see what still needs to be looked at by examining the log

git shortlog next/master..

and take any appropriate action, such as following up with the relevant maintainer or trying to find out what’s going on with the subsytem if it looks like the subsystem maintainers are inactive.

One possible problem with this approach is that a patch may be applied and then subsequently dropped – this is rare but it can happen. I deal with that by also keeping a normal unrebased development branch whch has the changes in Linus’ tree merged  into it periodically and incremental patches for any review updates that occur during the submission process. By looking at the diff between that and other trees I can see any changes that have got lost along the way.