Heating the Internet of Things

Internet of Things seems to be trendy these days, people like the shiny apps for controlling things and typically there are claims that the devices will perform better than their predecessors by offloading things to the cloud – but this makes some people worry that there are potential security issues and it’s not always clear that internet usage is actually delivering benefits over something local. One of the more widely deployed applications is smart thermostats for central heating which is something I’ve been playing with. I’m using Tado, there’s also at least Nest and Hive who do similar things, all relying on being connected to the internet for operation.

The main thing I’ve noticed has been that the temperature regulation in my flat is better, my previous thermostat allowed the temperature to vary by a couple of degrees around the target temperature in winter which got noticeable, with this the temperature generally seems to vary by a fraction of a degree at most. That does use the internet connection to get the temperature outside, though I’m fairly sure that most of this is just a better algorithm (the thermostat monitors how quickly the flat heats up when heating and uses this when to turn off rather than waiting for the temperature to hit the target then seeing it rise further as the radiators cool down) and performance would still be substantially improved without it.

The other thing that these systems deliver which does benefit much more from the internet connection is that it’s easy to control them remotely. This in turn makes it a lot easier to do things like turn the heating off when it’s not needed – you can do it remotely, and you can turn the heating back on without being in the flat so that you don’t need to remember to turn it off before you leave or come home to a cold building. The smarter ones do this automatically based on location detection from smartphones so you don’t need to think about it.

For example, when I started this post this I was sitting in a coffee shop so the heating had been turned off based on me taking my phone with me and as a result the temperature gone had down a bit. By the time I got home the flat was back up to normal temperature all without any meaningful intervention or visible difference on my part. This is particularly attractive for me given that I work from home – I can’t easily set a schedule to turn the heating off during the day like someone who works in an office so the heating would be on a lot of the time. Tado and Nest will to varying extents try to do this automatically, I don’t know about Hive. The Tado one at least works very well, I can’t speak to the others.

I’ve not had a bill for a full winter yet but I’m fairly sure looking at the meter that between the two features I’m saving a substantial amount of energy (and hence money and/or the environment depending on what you care about) and I’m also seeing a more constant temperature within the flat, my guess would be that most of the saving is coming from the heating being turned off when I leave the flat. For me at least this means that having the thermostat internet connected is worthwhile.

Kernel build times for automated builders

Over the past year or so various people have been automating kernel builds with the aim of both setting the standard that things should build reliably and using the resulting builds for automated testing. This has been having good results, it’s especially nice to compare the results for older stable kernel builds with current ones and notice how much happier everything is.

One of the challenges with doing this is that for good coverage you really need to include allmodconfig or allyesconfig builds to ensure coverage of as much kernel code as possible but that’s fairly resource intensive given the size of the kernel, especially when you want to cover several architectures. It’s also fairly important to get prompt results, development trees are changing all the time and the longer the gap between a problem appearing and it being identified the more likely the report is to be redundant.

Since I was looking at my own setup and I know of several people who’ve done similar benchmarking I thought I’d publish some ballpark numbers for from scratch allmodconfig builds on a single architecture:

i7-4770 with SSD 20 minutes
linode 2048 1.25 hours
EC2 m3.medium 1.5 hours
EC2 c3.large 2 hours
Cubietruck with SSD 20 hours
Intel Celeron N2940 1.75 hours

All with the number of tasks spawned by make set to the number of execution threads the system has and no speedups from anything like ccache. I may keep this updated in future with further results.

Obviously there’s tradeoffs beyond the time, especially for someone like me doing this at home with their own resources – my desktop is substantially faster than anything else I’ve tried but I’m also using it interactively for my work, it’s not easily accessible when not at home and the fans spin up during builds while EC2 starts to cost noticeable money to use as you add more builds.

Adventures with ARM server

I recently got a CubieTruck with a terabyte SSD to use as a general always on server. This being an ARM board rather than a PC (with a rather nice form factor – it’s basically the same size as a SSD) you’d normally expect a blog post about it to include instructions for kernels and patches and so on but with these systems and current Debian testing there’s no need – Debian works out of the box (including our standard kernel) on it, the instructions worked easily and I now have a new machine sitting quietly in the corner serving away. Sadly it being a dual core A7 it’s not got the grunt to replace my kernel build test system, an ARM allmodconfig takes eleven and a bit hours as opposed to a little less than twenty minutes on my desktop (which does draw well over an order of magnitude more power doing it), but otherwise you’d never notice the difference when using the system.

The upshot of all this is that actually there’s no real adventure at all; for systems like these where the system vendors and the communities around them are doing the right things and working well with upstream things just work as you’d expect with minimal effort.

The one thing that’s noticeably different from installing on a PC and really could do with improving is that instead of being shipped as part of the board the boot firmware has to be written to a SD card, something that could be addressed as easily as simply shipping a suitably programmed SD card in the box even without any other modification of the hardware, though on board flash would be even nicer.

Human factors

An issue which I always find depressing but sadly unsurprising in discussions of process with software is the frequent disregard for human elements; indeed often the goal people have in creating process is to try to control and eliminate human elements. Little thought is given to what is going to motivate people to do what’s asked and if they are going to follow it at all, or in the spirit it was intended.

One example I’ve seen several times is the idea that some engineers work on irrelevant things and that this can be fixed by requiring every commit to be tied to the project plan or a bug so off project work is obvious. This isn’t really attacking the problem so much as putting a roadblock in place to try to avert it; the real problem is normally people not communicating about what they’re doing and what’s important but those problems are really hard to address. Sadly what tends to happen is that people work around the roadblock and cause some collateral damage; for example devaluing the bug tracker by referencing irrelevant bugs or creating meaningless bugs solely to allow commits, while continuing to behave like they did originally. This is often worse than the original situation.

This sort of issue is one of the things I appreciate most about working on Linux – there’s quite a bit of process but because of the way it has been evolved the incentives are usually right to make sure they are followed in the spirit in which they were intended. For example the patch submission process is essentially just best practices for making sure changes go to people who care in a form which makes it easy for them to work with it; there is a bunch of tooling around it which built on those practices (and in turn influenced the practices) but it all comes back to that basic thing of getting attention for changes and making them easy to work with. As a result people mostly do the right thing (or close enough) so everything runs smoothly. The trick is to remember to standardize and write things down when it’s needed – notice the good practice and spread its adoption.

The key difference here seems to be if process is viewed as something to solve problems for people or if it is viewed as a way to solve problems with people. If the problem is with the people then the view starts off negative and it’s perhaps unsurprising that there is little consideration of how they will react. Sometimes it’s not so much that a problem is seen with people as that the people doing don’t have any real interest in the process (write only timesheets often bear no relation to reality for example) but there is an assumption that they are going to just do what they’re asked (but instead they for example don’t fill in their time sheets or don’t provide accurate information). The end result is the same, their needs don’t get considered and their actions end up being counterproductive.

Starting to avoid these problems is fairly straightforward – take a step back and consider why the people expected to carry out the process are going to want to do so. How does it help them? How will they see it helping others? Generally what’s in it for them? If these are difficult questions to answer then there may be a problem – it is likely people will ignore the process or do it badly without a lot of active enforcement, but perhaps that’s OK for the situation. Beware of using metrics to provide incentives, metrics are gameable and that game can cause problems without care.

But do think about people; making software is all about people.

New job

A few months ago I started a new job at Linaro as the technical lead for the Linaro Stable Kernel – I just posted a brief thing what I’m up to now over on the Linaro blog.

ASoC updates in 3.6

Linux v3.6 was another quiet release for ASoC with just a single notable framework feature being merged:

  • Added the ability to add and remove DAPM paths dynamically, mostly for reparenting on clock changes.
  • New machine drivers for Marvell Brownstone, ST-Ericsson Ux500 and ttc-dkp.
  • New CPU drivers for Blackfin BF6xx SPORTs in I2S mode, Marvell MMP, Synopsis Designware I2S controllers, SPEAr DMA, S/PDIF
  • New CODEC drivers for Dialog DA732x, ST STA529, ST-Ericsson AB8500, TI Isabelle Wolfson Microelectronics WM5102

UK landline non-security (and Orange clue)

Yesterday when I got in from work I got my second letter in as many months through from BT saying that my account was being closed as my landline was being transferred to another provider. This was the first I’d heard of this and it causes a cancellation charge so I called to complain; the first time I did this they said they couldn’t tell me anything about who the line had been transferred to. They did tell me that there was no equivalent of PAC or MAC for landlines and that the only thing stopping this happening is the two week delay in implementing.

This time BT felt able to tell me that the line had been transferred to Orange so I phoned Orange. Orange told me that the phone number had indeed been transferred to them in the name of someone else. They also said that they had no intention of attempting to carry out any authentication that lines being transferred to them are owned by the person they’re being transferred to – I explicitly asked them if anyone could just do this for any phone number and they confirmed that this is indeed the case.

BT claim they can’t block transfers for regulatory reasons; Orange claim this is possible and that I should just do that. I’ve asked Orange to put a note on the account (which was the best they claimed they could do) and complained to OFCOM (who won’t really talk to me without a formal escalation from the phone providers) but none of this really helps given the gaping security holes in the system. You really should need more information than just the phone number itself to transfer a number.

On the slightly bright side I still appear to have phone service; presumably currently paid for by whoever is initiating the transfers.

regulator updates in 3.6

Linux 3.6, which was released earlier today, saw continuing improvements in the factoring out of common code in the API – lots of new features making drivers smaller and easier to write!

  • Initial framework support for GPIO controlled enable signals, saving a bunch of code in drivers.
  • Move fixed regulator enable time and voltage mapping table specifications to data.
  • Used some of the recent framework enhancements to make voltage change notifications more useful, passing the voltage in as an argument to the notification.
  • New drivers for Maxim MAX77686, TI LP872x and LP8788, Samsung S2MPS11, Wolfson Arizona microphone supplies and LDOs

regmap updates in 3.6

Linux 3.6 was a very quiet release for regmap:

  • Support for buses and devices specifying their endianness, providing some performance improvements for MMIO devices.
  • Support for wake IRQs in regmap-irq.

Speaking at ELC-E: “regmap: The Power of Subsystems and Abstractions”

I will be speaking at ELC-E in Barcelona this year, with a talk entitled regmap: The Power of Subsystems and Abstractions. I look forward to seeing some of you at the conference, perhaps even in the audience!