Flashing an AT91SAM9G20-EK from bare metal

Since I just had cause to do this and it was harder than it needed to be due to bitrot in the public documentation I could find I thought I’d write up how to get a modern bootloader onto older Atmel boards. These instructions are written for the AT91SAM9G20-EK though they should also apply to other Atmel boards of a similar generation.

These instructions are for booting from NAND since it’s the default thing for the board, for this J34 should be fitted to enable the chip select and J33 disconnected to disable the dataflash. If there is something broken programmed into flash then booting while holding down BP4 should cause the second stage bootloader to trash itself and ensure the ROM bootloader puts itself into recovery mode, or just removing both J33 and J34 during power on will also ensure no second stage bootloader is found.

There is a ROM bootloader but it just loads a small region from the boot media and jumps into it which isn’t enough for u-boot so there is a second stage bootloader called AT91Bootstrap. Download sources for current versions from github. If it (or a more sensibly written equivalent) is not yet merged upstream you’ll need to apply this patch to get it to build with a modern compiler, or you could use an old toolchain (which you’ll need in the next step anyway):

diff --git a/board/at91sam9g20ek/board.mk b/board/at91sam9g20ek/board.mk
index 45f59b1822a6..b8251ca2fbad 100644
--- a/board/at91sam9g20ek/board.mk
+++ b/board/at91sam9g20ek/board.mk
@@ -1,7 +1,7 @@
 CPPFLAGS += \
        -DCONFIG_AT91SAM9G20EK \
-       -mcpu=arm926ej-s
+       -mcpu=arm926ej-s -mfloat-abi=soft
 
 ASFLAGS += \
        -DCONFIG_AT91SAM9G20EK \
-       -mcpu=arm926ej-s
+       -mcpu=arm926ej-s -mfloat-abi=soft

Once that’s done you can build with:

make at91sam9g20eknf_uboot_defconfig
make CROSS_COMPILE=arm-linux-gnueabihf-

producing binaries/at91sam9g20ek-nandflashboot-uboot-${VERSION}.bin. This configuration will look for u-boot at 0x40000 in the flash so we need a u-boot binary. Unfortunately modern compilers seem to produce binaries that fail with no output. This is normally a sign that they need the ABI specifying more clearly as above but I got fed up trying to spot what was missing so I used an old CodeSourcery 2013.05 release instead, hopefully future versions of u-boot will be able to build for this target with older toolchains. Grab a recent release (I used 2015.01) and build with:

cd ${UBOOT}
make at91sam9g20ek_nandflash_defconfig
make CROSS_COMPILE=arm-linux-gnueabihf-

to get u-boot.bin.

These can then be flashed using the Atmel flashing tool SAM-BA. Start it and connect to the target (there is a Linux version, though it appears to rely on old versions of TCL/TK so if you get trouble starting it the easiest thing is to use the sacrificial Windows laptop you’ve obtained in order to run the “entertaining” flashing tools companies sometimes provide without risking a real system, or in my case your shiny new laptop that you’ve not yet installed Linux on). Start it then:

  1. Connect SAM-BA to the device following the dialog on start.
  2. Make sure you’ve selected “NandFlash” in the memory type tabs in the center of the window.
  3. Run the “Enable NandFlash” script.
  4. Run the “Erase All” script.
  5. Run the “Send Boot File” script and provide the at91bootstrap binary.
  6. Set “Send File Name” to be the u-boot binary you built earlier and “Address” to be 0x40000.
  7. Click “Send File”
  8. Press the reset button

which should result in at91bootstrap output followed by u-boot output on the serial console. A similar process works for the AT91SAM9263, there the jumper you need is J19 (sadly u-boot does not flash pictures of cute animals or forested shorelines on the screen as the default “Basic LCD Project 1.4” firmware does, I’m not sure this “full operating system” thing is really delivering improved functionality).

Acer Aspire E11

Recently I was in Seoul in the middle of three weeks of travel and my laptop died on me.  Since I had some work that needed doing fairly urgently I took myself over to Yongsan Electronics Market and got myself a cheap replacement to tide myself over.

What I ended up with was an Acer Aspire E11. There’s a bunch of different models all with very similar plastics, I got one which has a N2940 SoC, 2G of RAM (upgraded to 4G in store), a 500G hard disk and no fans for just over 200000 Korean Won, or about $200. As you’d expect at that price it’s got shortcomings but overall I’ve been extremely happy with it, it’s worth looking at if you need something cheap.

The keyboard in particular is probably the nicest I’ve used 0n a laptop in a long time with a good, definite but not excessive click feel as you press. Battery life is about 5 hours as advertised which is not wonderful but basically fine for me most of the time, and while not exactly Retina it’s clear with good viewing angles and generally pleasant to look at. Everything is plastic but feels very solid and robust, better than a lot of more expensive devices I’ve used, and there’s not much bezel around the screen which means it’s the first laptop I’ve had which has been comfortable to use in a standard economy seat on a plane.

The biggest drawback is performance – it’s a little slow opening applications sometimes and kernel builds crawl with an x86 allmodconfig taking about one and three quarter hours. For e-mail and web browsing there’s no problem at all, I did have to move from offlineimap to mbsync to get my mail to sync in a reasonable time but that’s more to do with the performance of offlineimap than that of the system. Overall in use it feels like the Dell I was using from about 2008-2011 or so, comfortable in use outside of builds, and I do appreciate having a system with no fans.

There were a couple of small tricks getting Debian installed – this is the first system I’ve seen with secure boot enabled by default which took me a few moments to work out (but is really good to see). Once that was disabled the install was smooth other than being bitten by Debian bug#778810 which meant I needed a manual fixup to actually get it to boot from the disk. It’s also got a Broadcom WiFi module which means it doesn’t work at all with mainline but it looked like that was on a standard mini PCI Express module so easily replaceable (I happened to have a USB dongle handy so haven’t bothered) and the wired ethernet just worked.

Like I say I’ve been very happy with it, there’s a bunch of other models with different specs for everything except the case (some touchscreen, some with small 32G eMMC drives) as well. Were it not for my need to do kernel builds I’d probably be keeping it as my primary laptop.

Heating the Internet of Things

Internet of Things seems to be trendy these days, people like the shiny apps for controlling things and typically there are claims that the devices will perform better than their predecessors by offloading things to the cloud – but this makes some people worry that there are potential security issues and it’s not always clear that internet usage is actually delivering benefits over something local. One of the more widely deployed applications is smart thermostats for central heating which is something I’ve been playing with. I’m using Tado, there’s also at least Nest and Hive who do similar things, all relying on being connected to the internet for operation.

The main thing I’ve noticed has been that the temperature regulation in my flat is better, my previous thermostat allowed the temperature to vary by a couple of degrees around the target temperature in winter which got noticeable, with this the temperature generally seems to vary by a fraction of a degree at most. That does use the internet connection to get the temperature outside, though I’m fairly sure that most of this is just a better algorithm (the thermostat monitors how quickly the flat heats up when heating and uses this when to turn off rather than waiting for the temperature to hit the target then seeing it rise further as the radiators cool down) and performance would still be substantially improved without it.

The other thing that these systems deliver which does benefit much more from the internet connection is that it’s easy to control them remotely. This in turn makes it a lot easier to do things like turn the heating off when it’s not needed – you can do it remotely, and you can turn the heating back on without being in the flat so that you don’t need to remember to turn it off before you leave or come home to a cold building. The smarter ones do this automatically based on location detection from smartphones so you don’t need to think about it.

For example, when I started this post this I was sitting in a coffee shop so the heating had been turned off based on me taking my phone with me and as a result the temperature gone had down a bit. By the time I got home the flat was back up to normal temperature all without any meaningful intervention or visible difference on my part. This is particularly attractive for me given that I work from home – I can’t easily set a schedule to turn the heating off during the day like someone who works in an office so the heating would be on a lot of the time. Tado and Nest will to varying extents try to do this automatically, I don’t know about Hive. The Tado one at least works very well, I can’t speak to the others.

I’ve not had a bill for a full winter yet but I’m fairly sure looking at the meter that between the two features I’m saving a substantial amount of energy (and hence money and/or the environment depending on what you care about) and I’m also seeing a more constant temperature within the flat, my guess would be that most of the saving is coming from the heating being turned off when I leave the flat. For me at least this means that having the thermostat internet connected is worthwhile.

Kernel build times for automated builders

Over the past year or so various people have been automating kernel builds with the aim of both setting the standard that things should build reliably and using the resulting builds for automated testing. This has been having good results, it’s especially nice to compare the results for older stable kernel builds with current ones and notice how much happier everything is.

One of the challenges with doing this is that for good coverage you really need to include allmodconfig or allyesconfig builds to ensure coverage of as much kernel code as possible but that’s fairly resource intensive given the size of the kernel, especially when you want to cover several architectures. It’s also fairly important to get prompt results, development trees are changing all the time and the longer the gap between a problem appearing and it being identified the more likely the report is to be redundant.

Since I was looking at my own setup and I know of several people who’ve done similar benchmarking I thought I’d publish some ballpark numbers for from scratch allmodconfig builds on a single architecture:

i7-4770 with SSD 20 minutes
linode 2048 1.25 hours
EC2 m3.medium 1.5 hours
EC2 c3.large 2 hours
Cubietruck with SSD 20 hours
Intel Celeron N2940 1.75 hours

All with the number of tasks spawned by make set to the number of execution threads the system has and no speedups from anything like ccache. I may keep this updated in future with further results.

Obviously there’s tradeoffs beyond the time, especially for someone like me doing this at home with their own resources – my desktop is substantially faster than anything else I’ve tried but I’m also using it interactively for my work, it’s not easily accessible when not at home and the fans spin up during builds while EC2 starts to cost noticeable money to use as you add more builds.

Adventures with ARM server

I recently got a CubieTruck with a terabyte SSD to use as a general always on server. This being an ARM board rather than a PC (with a rather nice form factor – it’s basically the same size as a SSD) you’d normally expect a blog post about it to include instructions for kernels and patches and so on but with these systems and current Debian testing there’s no need – Debian works out of the box (including our standard kernel) on it, the instructions worked easily and I now have a new machine sitting quietly in the corner serving away. Sadly it being a dual core A7 it’s not got the grunt to replace my kernel build test system, an ARM allmodconfig takes eleven and a bit hours as opposed to a little less than twenty minutes on my desktop (which does draw well over an order of magnitude more power doing it), but otherwise you’d never notice the difference when using the system.

The upshot of all this is that actually there’s no real adventure at all; for systems like these where the system vendors and the communities around them are doing the right things and working well with upstream things just work as you’d expect with minimal effort.

The one thing that’s noticeably different from installing on a PC and really could do with improving is that instead of being shipped as part of the board the boot firmware has to be written to a SD card, something that could be addressed as easily as simply shipping a suitably programmed SD card in the box even without any other modification of the hardware, though on board flash would be even nicer.

Human factors

An issue which I always find depressing but sadly unsurprising in discussions of process with software is the frequent disregard for human elements; indeed often the goal people have in creating process is to try to control and eliminate human elements. Little thought is given to what is going to motivate people to do what’s asked and if they are going to follow it at all, or in the spirit it was intended.

One example I’ve seen several times is the idea that some engineers work on irrelevant things and that this can be fixed by requiring every commit to be tied to the project plan or a bug so off project work is obvious. This isn’t really attacking the problem so much as putting a roadblock in place to try to avert it; the real problem is normally people not communicating about what they’re doing and what’s important but those problems are really hard to address. Sadly what tends to happen is that people work around the roadblock and cause some collateral damage; for example devaluing the bug tracker by referencing irrelevant bugs or creating meaningless bugs solely to allow commits, while continuing to behave like they did originally. This is often worse than the original situation.

This sort of issue is one of the things I appreciate most about working on Linux – there’s quite a bit of process but because of the way it has been evolved the incentives are usually right to make sure they are followed in the spirit in which they were intended. For example the patch submission process is essentially just best practices for making sure changes go to people who care in a form which makes it easy for them to work with it; there is a bunch of tooling around it which built on those practices (and in turn influenced the practices) but it all comes back to that basic thing of getting attention for changes and making them easy to work with. As a result people mostly do the right thing (or close enough) so everything runs smoothly. The trick is to remember to standardize and write things down when it’s needed – notice the good practice and spread its adoption.

The key difference here seems to be if process is viewed as something to solve problems for people or if it is viewed as a way to solve problems with people. If the problem is with the people then the view starts off negative and it’s perhaps unsurprising that there is little consideration of how they will react. Sometimes it’s not so much that a problem is seen with people as that the people doing don’t have any real interest in the process (write only timesheets often bear no relation to reality for example) but there is an assumption that they are going to just do what they’re asked (but instead they for example don’t fill in their time sheets or don’t provide accurate information). The end result is the same, their needs don’t get considered and their actions end up being counterproductive.

Starting to avoid these problems is fairly straightforward – take a step back and consider why the people expected to carry out the process are going to want to do so. How does it help them? How will they see it helping others? Generally what’s in it for them? If these are difficult questions to answer then there may be a problem – it is likely people will ignore the process or do it badly without a lot of active enforcement, but perhaps that’s OK for the situation. Beware of using metrics to provide incentives, metrics are gameable and that game can cause problems without care.

But do think about people; making software is all about people.

New job

A few months ago I started a new job at Linaro as the technical lead for the Linaro Stable Kernel – I just posted a brief thing what I’m up to now over on the Linaro blog.

ASoC updates in 3.6

Linux v3.6 was another quiet release for ASoC with just a single notable framework feature being merged:

  • Added the ability to add and remove DAPM paths dynamically, mostly for reparenting on clock changes.
  • New machine drivers for Marvell Brownstone, ST-Ericsson Ux500 and ttc-dkp.
  • New CPU drivers for Blackfin BF6xx SPORTs in I2S mode, Marvell MMP, Synopsis Designware I2S controllers, SPEAr DMA, S/PDIF
  • New CODEC drivers for Dialog DA732x, ST STA529, ST-Ericsson AB8500, TI Isabelle Wolfson Microelectronics WM5102

UK landline non-security (and Orange clue)

Yesterday when I got in from work I got my second letter in as many months through from BT saying that my account was being closed as my landline was being transferred to another provider. This was the first I’d heard of this and it causes a cancellation charge so I called to complain; the first time I did this they said they couldn’t tell me anything about who the line had been transferred to. They did tell me that there was no equivalent of PAC or MAC for landlines and that the only thing stopping this happening is the two week delay in implementing.

This time BT felt able to tell me that the line had been transferred to Orange so I phoned Orange. Orange told me that the phone number had indeed been transferred to them in the name of someone else. They also said that they had no intention of attempting to carry out any authentication that lines being transferred to them are owned by the person they’re being transferred to – I explicitly asked them if anyone could just do this for any phone number and they confirmed that this is indeed the case.

BT claim they can’t block transfers for regulatory reasons; Orange claim this is possible and that I should just do that. I’ve asked Orange to put a note on the account (which was the best they claimed they could do) and complained to OFCOM (who won’t really talk to me without a formal escalation from the phone providers) but none of this really helps given the gaping security holes in the system. You really should need more information than just the phone number itself to transfer a number.

On the slightly bright side I still appear to have phone service; presumably currently paid for by whoever is initiating the transfers.

regulator updates in 3.6

Linux 3.6, which was released earlier today, saw continuing improvements in the factoring out of common code in the API – lots of new features making drivers smaller and easier to write!

  • Initial framework support for GPIO controlled enable signals, saving a bunch of code in drivers.
  • Move fixed regulator enable time and voltage mapping table specifications to data.
  • Used some of the recent framework enhancements to make voltage change notifications more useful, passing the voltage in as an argument to the notification.
  • New drivers for Maxim MAX77686, TI LP872x and LP8788, Samsung S2MPS11, Wolfson Arizona microphone supplies and LDOs