Assessing the Nest

So it’s been eight months with the new Nest thermostat (version 1.0) and the obvious question is – was it worth the $250 price tag?

The answer – at least for hard dollar savings – is very hard to get.

For the aspects that don’t convert easily to dollars – ease of use, aesthetics, ability to access remotely – it is easy to claim value for the money.

I love accessing the thermostat from my phone. If I wake up early on a cold morning – I lean over to my phone and tell the thermostat to dial up the heat. If we’re headed out of the house on a weekend, I log in to tell it we’ve left. Frequently I find it’s already detected our absence, which is another feature I like.

I love the energy usage reports. I have two stage heating and it’s nice to see that the report differentiates between the two stages. It’s also great to see the patterns in how my house heats up. On a cold day, I can see that we’re hitting stage 1 heat about every half hour or so.

And the milestones have a little detail, as well – mostly enough to identify themselves – but enough to differentiate between a programmed temperature change and an ad hoc adjustment.

I like the leaf symbol and time to temperature – great user feedback to encourage responsible use. If I crank up the Nest because I’m cold and see “72 IN 2+ HR” I’m more likely to dial that back – particularly if I’m leaving/sleeping before the furnace will hit that temperature. My old thermostat would just impassively spool up and generate a bill for heat I wouldn't use or notice.

A 5-degree heat increase in 25 minutes? Lucky you.

I like that the device turns on when I walk by – makes me feel like I’m living in the future. "Oooh, watch me activate the screen by waving my hand!"

Those are the UI wins.

Like all devices, there are UI shortcomings.

A minor issue is that the energy report should store external temperature data for the day. While it’s useful to know that my furnace ran for six hours – knowing that it ran for 6 hours on a day when the temperature was 4 degrees Fahrenheit is a bit more useful. As the energy report only goes back 10 days, Nest probably figures you'll just remember this, but...

-I would like to have long-term access to my energy use and weather data – using the web app. I want more charts and graphs! Data is a drug - you get some, you want more.

The biggest UI failure of the Nest to me almost certainly stems from the best intentions of the Nest folks. Much is made of the accuracy of the Nest’s temperature sensors.

Nest uses multiple temperature sensors to determine the ambient temperature with a high degree of accuracy. Nest's sensors are tuned to be most useful for keeping your home consistently comfortable. We found that responding rapidly to temperature changes gives Nest the most accurate data to work from.

“A high degree of accuracy…” So when the user sees their Nest display something like this, they start to wonder if something is wrong:

Literally, what my Nest was displaying as I typed this.
A temp of 67 degrees while the Nest is set to 68 - and the Nest is not heating.

It is not uncommon to see the Nest display a current temperature that is below the temperature it is supposed to be at.

This does not fit the user’s mental model. “I asked for 68 degrees, and if you are below 68 degrees – I expect the heat to be on.”

Nest's support pages point out the many good reasons why this is so, but to your average homeowner - these are excuses. Our mental model is stuck in reptile brain mode: “Make heat. Now.”

The Nest is doing what most thermostats do, it has a "maintenance band" of about a degree or so that allows for minor variation and keeps the furnace from flicking on every few minutes and wearing it out.

"My old thermostat didn't do this!" Well, my old thermostat didn't light up when I walked by and show me what it was or wasn't up to, either. More interactions = more data = more questions. My previous thermostat offered the same data, but reading its small dark-grey LED text (on a light grey screen) required my eyeballs to be less than a foot away.  Hidden data might as well not exist, so while I might think my old thermostat didn't have this issue – it probably did. I just didn't notice because I almost never read the damn thing.

With a display this accessible, I would think the Nest would add an indicator that showed that it was aware that the temp is below the set temp. Something like "Preparing heat in 10 MIN" or something that lets the user know that their $250 thermostat hasn't forgotten that 67 is less than 68.

Screw the UI. What about the money? Is it worth it?

That's harder to know than you might think. All that nifty data I'm getting from the Nest? It's just telling me how long my Nest was running - not how much natural gas I burned for heat, or how many kWH I used to cool the house.

Did you pay less money this year? Well...

• In all of 2011, I paid $2,238 for electric and gas.
• In all of 2012, I paid $1,945 for electric and gas.

Huzzah! $292 in savings!

Well... sorta.

I mean, that's my total utility bill, including things like my gas stove and electric dryer - stuff my Nest is no help at all on.

If I want to carve off the portion of my bill that went to feed my A/C and Furnace, I need more data. Now, if I go with the assumption that my energy bill is like the typical American home energy bill,  then heat will account for 29% of my bill and A/C another 17%.

So, if I assume that I'm typical and 46% of my utility bill was my heating and cooling (29% + 17%), then A/C and heat ran me $1029 in 2011 and $894 in 2012.

Huzzah! $135 in savings!

But... I installed the Nest in May, so really that should be $135  * [8/12], or $90.

Huzzah! $90 in savings!

But that's just the total expense of heating and cooling. Is the difference between those bills accounted for by  2011 having a hotter summer, or a cooler winter?

Well, thankfully, my utility company provides most of the missing pieces. In winter months, my utility tracks how cold the winter months are using something called a heating degree day. The colder the month, the more heating degree days it will have.

My utility only tracks heating degree days during the winter months, so I can only compare 2011 to 2012 using the last three months of each.

They even give you pretty graphs:

Lines are good, but if I need to turn this into dollars, I need the data points.

Here's the data for 2011

Read Date	# Days	Therms	Heating Degree Days	Therms/Degree Day	$ Amount	Cost per Therm
12/16/2011	30	96	900	0.107	$91.61	$0.95
11/16/2011	29	62	601	0.103	$58.55	$0.94
10/18/2011	29	20	231	0.087	$25.79	$1.29

and here's 2012:

Read Date	# Days	Therms	Heating Degree Days	Therms/Degree Day	$ Amount	Cost per Therm
12/17/2012	31	78	847	0.092	$73.79	$0.95
11/16/2012	30	54	646	0.084	$51.31	$0.95
10/17/2012	29	25	399	0.063	$28.99	$1.16

In addition to pointing out that the cost per thermal unit is yet another variable to track, this allows me to factor the weather into my natural gas usage. Here's what I came up with

The average Therms per Degree Day for 2011 was .099.
The average Therms per Degree Day for 2012 was .079.

Which means I used 19% less Therms per day in 2012. While some of that could be due to other causes, I haven't changed my gas water heater and we use our gas stove at a pretty constant rate. 19% is a pretty healthy shift in my bill. If my natural gas costs truly went down that much due to the Nest, I would have saved about $30 over three months of heating.

The data for my electrical bill wasn't quite comparable, but my average kWh/Day usage went down as well. The graph makes it look pretty dramatic, too:

Between May to September of 2011, I used an average of 28.64 kWh/Day.
Over those same months in 2012, I used an average of 24.18 kWh/Day - a drop of a little over 15%.

Again, if I tack the credit for all of that to the Nest, I can claim it saved me up to $97 over those five months. That's probably high though - I don't have equivalent degree days for warmer weather, although the 2012 summer was a drought and 2011's was not rated as unusually warm.

I tend to think I used my dehumidifier less in 2012 than in 2011, so that could be a good chunk of that savings.

But take a reasonably good heating savings estimate of 19% - (averaging about $10 a month), and throw in half of my looser estimate of electrical savings - call it 8% (instead of the 15.57% I came up with) and you end up with an average of about $10 a month.

Roughly eighty bucks saved across eight months. Buy back in about two years - plus more features and control.

Not too shabby.