1. business and industrial
2. energy
3. electricity

Welfare Effects of Home Automation Technology with
Dynamic Pricing∗
Bryan Bollinger
Wesley R. Hartmann
Fuqua School of Business
Graduate School of Business
Duke University
Stanford University
April 1, 2015
Abstract
A fixed cost investment in home automation technology can eliminate consumers’
marginal costs of responding to changing demand conditions. We estimate the welfare effects of a home automation technology using a field experiment run by a large
electric utility that randomly assigned both a technology and price treatment. Average treatment effects reveal that the home automation technology reduces demand
more than twice as much as an alternative technology that only informs consumers
of price changes. Furthermore, the average demand reductions during critical price
events provide sufficient supply-side welfare gains to fully offset the installation costs
of the device. Finally, we estimate household-specific treatment effects by matching
households on their pre-treatment policy functions. This demonstrates the additional
surplus gained by the utility if it targeted these treatments to households with the
largest estimated demand responses.
∗
Email addresses are [email protected] and [email protected]. The authors would like to
thank Severin Borenstein, Mar Reguant, seminar participants at the Energy Institute at Haas and conference
participants at the 2012 Marketing Science Conference, the 2014 Field Experimentation Conference at Rady
School of Business, the 2014 Workshop on Economics of of Advertising and Marketing, 2014 IOFest, and the
2015 Winter Marketing-Economics Summit. Rachana Gogate provided valuable research assistance on this
project.
1
1
Introduction
Home automation technologies shift households’ marginal costs of adjusting their consumption to a single upfront installation cost of the device. The technology increases a consumer’s demand elasticity by automatically executing her response, potentially in real-time,
to weather and other demand shocks, and possibly to price. As both electricty and water
expenditures are small within short time windows, it might take a long horizon for such technology investments to payoff for consumers. On the other hand, automated response technologies may provide significant supply-side benefits as utilities struggle to manage scarce
supply. In fact, coupling a price-responsive automation technology with dynamic pricing may
provide sufficient consumer demand reductions when capacity is constrained to justify the
utility subsidizing installations of the home automation technology. We consider the welfare
effects of such a home automation technology by analyzing a field experiment in which an
electric utility randomly assigned households to information technology, automated response
technology or neither and to pricing plans that included varying levels of dynamics.
Electric utilities have increasingly become interested in demand response (DR) to better
match supply and demand. The Federal Energy Regulatory Commission defines DR as:
“Changes in electric usage by end-use customers from their normal consumption patterns in
response to changes in the price of electricity over time, or to incentive payments designed to
induce lower electricity use at times of high wholesale market prices or when system reliability
is jeopardized.” Reasons for supply shocks in electricity include forced outages, transmission
line failures, and sharp changes in generation. These factors coupled with the high costs of
energy storage result in large changes in wholesale prices over time and vary by an order
of magnitude over the course of a day. This problem is exacerbated with the increase in
renewable energy generation, because renewable sources such as solar and wind can fluctuate
greatly in their electricity generation. Borenstein and Holland (2005) demonstrates that realtime pricing, which passes wholesale price swings through to consumers, can yield long-term
efficiency gains even with low demand elasticities. However, a major hurdle to implementing
2
real-time pricing is the risk transferred to consumers who are uninformed or unequipped to
respond to price changes. Home automation technology can resolve this problem by enabling
response.
To evaluate how combinations of technology and dynamic pricing could help manage peak
load demand, an electric utility in the southern United States conducted a field experiment
in the summer of 2011. Households interested in participating were randomly assigned to a
control group or a price-technology paired treatment. Usage patterns for households in the
control group are similar to those who did not opt in to the experiment. Treated households
were placed in either a time of use (TOU) or variable peak price (VPP) plan. TOU households faced a single elevated price during peak hours, whereas the VPP households faced
a peak period price that varied by day depending on the supply and demand conditions at
the utility. Treated households were also assigned i) an in home display (IHD) to reveal
price information, ii) a programmable communicating thermostat (PCT) that cooled the
house based on consumers’ weighting of savings vs. comfort, iii) both technologies, or iv)
neither. All households also had access to a web portal with price and consumption information. Average treatment effects reveal that adding the home automation PCT on top of
the information only IHD technology nearly doubles the critical peak demand reduction for
customers on a TOU plan and triples the critical peak demand reduction for customers with
VPP.
To estimate the consumer welfare effects of the home automation technology, we compare
the demand curves when consumers have both an information and automation device to the
demand curve when consumers only have the information device. Our field experiment does
not have random variation in price, but we can condition on the demand intensity of the
randomly assigned control group, which faces a flat price. This focuses inference on price
variation arising from factors outside the participants in the experiment, such as supply
shocks and demand shocks exclusive to commercial customers or customers in other cities.
The identifying assumption is an absence of price correlated demand shocks which only shift
3
the behavior of treated households. Using this approach we demonstrate that the home
automation technology does result in more elastic demand with greater reductions at high
prices and expanded demand when prices are low. This increases consumer welfare by just
over ten percent of the $250 cost of the device, per year. These consumer welfare gains would
likely be even greater in the presence of true real-time pricing which is more variable and
lacks the price ceiling in our experiment.
The electric utility also increases its welfare because the demand reductions during critical
periods defer investments in expanded capacity. Using their assumption that a permanent
kWh decrease in critical peak demand yields a net present value of $700, the break-even
demand reduction is much less than the average incremental demand reduction estimated for
the home automation technology. The utility could however increase its surplus if it targeted
the device to only those homes where the gains fully offset the installation costs. To evaluate
this, we estimate household-level treatment effects that match households based on their pretreatment consumption. A treatment effect that conditions on our potential target’s vector
of past usage is unrealistic, but we argue that conditioning on the distribution of usage is
both practical and intuitively appealing. The distance in usage distributions between our
focal household and all other households in the data can be calculated with the Kolmogorov
statistic (Kolmogorov, 1933), which provides a probability that two distributions are the
same. Furthermore, we can calculate the distance between usage distributions conditional
on obvious states in a consumer’s electricity consumption problem (e.g. hot vs cold, weekend
vs weekday, high vs. low past usage). This effectively matches households on their pretreatment policy functions which should be indicative of the unobserved ways in which
households differ. Targeting households with a household-level treatment effect large enough
for the utility to break even on the home automation technology increases the supply side
surplus per installation by 30% to $425.
Our analysis provides substantive, policy, and methodological contributions to economics.
The economics of acquiring information has been extensively studied since Stigler (1961)’s
4
seminal paper. Buyers incur search costs to resolve their uncertainty about sellers prices at
“any given time.” Search costs have drastically fallen as price and other information can more
be easily found online or communicated by electronic mail, text messages or notifications on
mobile phones or wearable devices. Abundant or cheap information does not however appear
to return us to the perfect information world as the price dispersion Stigler observed is still
great for apparently homogeneous goods and large costs for price search have been measured
both online (Honka, 2014) and across the aisles or shelf facings in a grocery store (Seiler,
2013). Our information only treatment yields significant demand response that is consistent
with these findings. Nevertheless, much larger gains arise with investments in technology
that automates the response to information. Such technologies are increasingly important
as they can reduce or eliminate buyers’ costs of acquiring, processing, and responding to the
abundance of information inundating consumers today.
From the perspective of energy policy, our findings document the advantages of moving
utilities to dynamic pricing plans if they can provide the technology for consumers to respond.
As speculated in Allcott (2011), we find that a programmable communicating thermostat
which can respond to price (in contrast to a traditional thermostat which responds only to
temperature) provides significant improvements in demand response.1 The price treatments
in our study involved day ahead price setting in all but the critical cases, but that timing
of notification had no impact on the demand reduction of the automation technology. This
suggests real-time pricing could instead be coupled with the automated response technology
to provide even greater welfare gains. The presence of heterogeneity in demand response that
we document is consistent with previous work, such as Reiss and White (2005). However,
the authors find that nonlinear pricing in California exhibits a highly skewed distribution in
DR effectiveness such that a small fraction of households accounts for the majority of the
response. Their high responders were likely households with low marginal costs of adjusting
1
Cappers, Goldman, and Kathan (2010) show that DR without home automation has increased by 10%
since 2006 in reducing peak load and that existing DR resource potential ranges from 3 to 9% of a region’s
summer peak demand in most regions.
5
demand across multiple appliances. In constast, we find the home automation technology to
reduce the marginal adjustment costs for the average customer such that our heterogeneous
treatment effects reflect the heterogeneity among the larger group of customers that otherwise
would have found it too costly to adapt behavior.
From a methodological perspective, we propose a household targeting strategy using an
average treatment effect weighted by the distance between each household in the sample and
the focal household. The program evaluation literature currently uses observed hetergeneity
to shift between, for example, sample average, population average and treatment effects on
the treated or controls (see for example Abadie, 2002 and Imbens, 2004). Our focus on
weighting around a focal household arises from targeting and its supply-side surplus gains.
We contrast with extant targeting approaches using Bayesian estimation and a hierarchical
structure that “shrinks” inference to individual units based on the time dimension of the
panel (Rossi, Allenby, and McCulloch, 2005). In most field experiments, the randomization
is purely cross-sectional at a single point in time, such that the length of the panel provides
no additional inference on the individual’s specific treatment effect. Rather, we use pretreatment outcomes to draw inferences about the similarities and differences among the
cross section of subjects in the experiment. Our weighted average treatment effects lack the
full structure of the data generating process typically included in the Bayesian approach, but
in our context and other field experiments, targeting decisions may be assessed by simply
comparing the household weighted average treatment effect to a simple threshold.
Our matching estimator also flips the problems associated with too much pre-treatment
information on each unit into a benefit when that information can be viewed as outcomes of
economic agents’ policy functions. The matching literature has long struggled with excessive
pre-treatment information. The propensity score estimator, for example, collapses a high
dimensional set of observables into a single metric based on a first stage estimation of how
observables relate to treatment assignment (Rosenbaum and Rubin (1983)). Proliferation of
observables about subjects has even overtaxed this estimator as they may be greater than the
6
Figure 1: In-Home Display (IHD)
number of subjects whose treatment assignments we observe. Variable selection models have
since been applied to systematically reduce the amount of information used in estimation. In
contrast, the Kolmogorov distance measure in our matching estimator becomes more precise
with more observed pre-treatment outcomes. While we apply this matching estimator in
an expermental setting to recover the household weighted average effects, this matching
approach could also be used to reduce bias in non-experimental settings in much the same
way as the propensity score, but in a context where increasing information about the subject
is a benefit as opposed to a problem.
The paper proceeds as follows. Section 2 describes our unique data set and experimental
treatments. In section 3, we estimate average treatment effects that illustrate how a combination of technology and flexible pricing can be effective in reducing electricity usage during
peak periods that tax the grid’s capacity and call into use less efficient and potentially more
polluting production capacity. Section 4 illustrates the demand estimation with and without
home automation to recover consumer welfare effects. In Section 5 we consider the supply
side and use the estimation of household level treatment effects to illustrate the electric utility’s gain in surplus if it targets PCT devices as opposed to offering them to all households
based on the average demand reductions. We provide concluding remarks in Section 6.
7
Figure 2: In-Home Display (IHD)
2
Data
2.1
The Experiment
The goal of providing the smartmeter enabled technoloigies to consumers was to help them
moniter and reduce their consumption during critical pricing events. There were seven events
in 2011 which all occured sometime in the window between 1 PM and 7 PM. In general,
the period between 2 PM and 7 PM on non-holiday week days is the peak demand period.
Outside this window and on weekends and holidays are off-peak periods.
Households who volunteered for a demand-reduction program were randomly assigned to
one of nine conditions. Aside from the control condition, the treatment conditions combine
one of four technology treatments with one of two pricing plans. The four technologies
treatments are: i) a computer portal to monitor usage and price, ii) an in-home display that
removes the need to be online to assess usage and price, iii) a programmable thermostat
which can be set to turn off air conditioning based on the time, in-home temperature, or
price, and iv) a combination of all three technologies. See Figures 1 and 2 for pictures of the
IHD and PCT technlogies.
The pricing treatments are i) time of use pricing (TOU) which sets different prices for
the peak and off-peak hours on weekdays and ii) variable peak pricing (VPP) in which the
peak price can be varied by the utility depending on its aggregate demand. We have hourly
electricity usage for 4,443 residential households in 2011; 2,589 households appear in both
the 2010 and 2011 data, with 5,491,443 hours of electricity usage data. We exclude from
8
Table 1: Treatments
Treatments
Control
Portal
IHD
PCT
All three
Total
CON
294
0
0
0
0
294
TOU
0
310
224
214
184
932
VPP
0
332
228
220
172
952
Total
294
642
452
434
356
2,178
Figure 3: Average Hourly Electricity Consumption by Treatment
1
Usage (KWH)
2
3
4
Households with AC
June 1
Sept 30
control
ihd and portal
all 3
portal only
pct and portal
the analysis households who have the low-income price rate (all assigned non-randomly to
the control group) and those without a treatment start date recorded, as well as accounts
with multiple meter ids. We also exclude 2011 consumption data for treated households
before their treatment. Since the vast majority of homes have air conditioning, we exclude
those without. Finally, we exclude households in the extreme tails of the distribution of
2010 usage, 2011 peak usage, and 2011 off-peak usage i.e. those with more than 7 kW or less
than 0.5 kW of average usage. The final estimation sample includes 2,178 households with
6,159,548 hours of electricity usage data in 2010 and 4,627,987 hours of electricity usage data
in 2011. 294 households are in the control condition, with the remaining spread across the
various treatments, as shown in Table 1.
9
Figure 4: Average Hourly Electricity Consumption by Treatment
1
1.5
Usage (KWH)
2 2.5 3 3.5
4
Peak Hourly Usage
0:00
14:00
Time
19:00
24:00
control
portal only
ihd and portal
pct and portal
all 3
1
1.5
Usage (KWH)
2 2.5 3 3.5
4
Off−Peak Hourly Usage
0:00
14:00
Time
19:00
24:00
control
portal only
ihd and portal
pct and portal
all 3
10
Figure 3 shows the household average hourly usage by treatment – it is clear that all of
the technology treatments (with the new price treatments) do lead to an aggregate reduction
in usage. The timing of the reduction in usage can be seen in Figure 4, which plots average
hourly usage by treatment, on both peak and off-peak days. It is clear that all treatment
conditions lead to a reduction in electricity consumption during the peak period of peak
days. Consumption is then slightly higher both during the off-peak hours of peak days and
on off-peak days, indicating that there might be inter-temporal substitution in electricity
consumption. The clear superiority of the PCT device is evident with the sudden reduction
in electricty consumption at the beginning of peak hours (14:00). Note however that a wider
rollout of these devices would call for a smoothing of the start times to avoid the sudden
demand shock this would create.
Figure 5 shows kernel density estimates of the average hourly usage, where the averages
are taken by household over the peak and off-peak periods of each day. There is a drop for
all technology treatments during the peak hours but a small increase in usage during the
off-peak hours. The PCT and All 3 technologies reduce peak demand much more than the
portal and IHD.
We have not said anything about the statistical significance of these findings. While the
effects are apparent in both the the usage and kernel density graphs, usage varies greatly
across both time and households, leading to large variance in usage under all treatments
in the kernel density graphs. As such, these graphs are not in themselves sufficient in
determining whether the treatments have statistically significant mean treatment effects. To
test the significance of the average treatment effects, we use a regression analysis described
in the next section to not only assesses the impact of the technology treatments but also
their interaction with the two price treatments.
11
Figure 5: Average Hourly Electricity Consumption by Treatment, On-Peak Days
0
Density
Peak
0
1
2
3
Average Hourly Usage (KWH)
4
control
portal
portal and ihd
portal and pct
5
all three
0
Density
Off−peak
0
1
2
3
Average Hourly Usage (KWH)
4
control
portal
portal and ihd
portal and pct
all three
12
5
Figure 6: Difference Between Peak and Off-peak Usage
(a) Hourly Usage by Treatment
0
Density
Difference in Peak and Off−peak
−1
0
1
Average Hourly Usage (KWH)
control
portal and ihd
all three
3
2
portal
portal and pct
Average Treatment Effects
To estimate the average treatment effects for our panel of electricity consumption in the
summer following treatment, we use the following simple regression:
yi = α0 + α1 A˜i + ia
(1)
where yi is household i’s average hourly electricity consumption in either critical, peak,
or off-peak periods – we run separate regressions for each. We collapse all of the data
across time within these three periods. A˜i is a vector of dummies corresponding to each of
eight treatments (the four technology treatments interacted with the two price treatments),
with zeros for all but the household’s treatment Ai . α0 captures the average usage of the
control group, while α1 measures the change in usage attributable to each treatment; ia
is the unobservable consumption shock that is uncorrelated with treatment because of the
randomization. Results are shown in Tables 2. We report robust standard errors.
We see significant effects of all technology treatments for critical and peak periods, leading
to large reductions in usage between 0.347 and 1.303 kW for critical periods and 0.200 and
13
Table 2: Average Treatment Effect Regression Results
Treatments
Portal
IHD
PCT
All three
Critical
Peak
TOU
VPP
TOU
VPP
-0.437 -0.281 -0.324 -0.146
(0.137) (0.136) (0.123) (0.123)
-0.671 -0.414 -0.491 -0.274
(0.141) (0.137) (0.127) (0.126)
-1.190 -1.219 -0.942 -0.681
(0.141) (0.141) (0.127) (0.130)
-1.082 -1.231 -0.830 -0.643
(0.134) (0.149) (0.131) (0.138)
N=2,141
N=2,178
Standard errors in parentheses
Off-Peak
TOU
VPP
-0.113
0.031
(0.082) (0.086)
-0.138 -0.018
(0.090) (0.087)
0.027
0.118
(0.087) (0.094)
0.080
0.080
(0.097) (0.102)
N=2,178
0.918 kW for peak. It is clear that the PCT technology (used in the PCT and All 3
treatments) is the most effective of the three alternatives for lowering consumption during
peak periods for households with AC. Furthermore, although the impact of the
technoloigies are simlar across price treatments for critical events, we find evidence that
there is a main effect of the TOU pricing schedule in comparison to the VPP pricing
schedule in lowering peak consumption. This could be due to average VPP prices being
lower than TOU. There is no clear effect of technology treatment or price treatment in
off-peak consumption behavior, although this is likely due to the aggregation over all
off-peak periods in the case of the PCT and All 3 treatments since the hourly usage graph
in Figure 4 clearly showed positive consumption effects immediately following the peak
period.
4
Consumer Welfare Effects from Home Automation Technology
Home automation technology can reduce or eliminate consumers’ costs of adjusting demand
to changing conditions. The technology therefore shifts consumers to a more elastic demand
curve. To evaluate the consumer welfare effect of automated response technology, we compare
14
demand under the All 3 treatment with demand under the IHD treatment. The All 3
treatment is the best estimate of the “true” demand curve in that they can rely on the PCT
algorithm to respond to price, but also override based on their enhanced knowledge of price
delivered via the IHD. We compare this to the IHD treatment because it measures the gains
of automation technology relative to merely informing consumers.
4.1
Demand Estimation
There are two potential sources of price variation we can use to estimate demand: i) across
pricing plans (e.g. TOU vs. VPP) and ii) within peak variation in the VPP price. Variation
across plans is based on the random assignment, but on any given day, there are at most
two different peak prices necessiting strong assumptions about the shape of demand between
and outside of these price points. Variation within the VPP price ranges the entire span
from less than 5c to a high of 46c, yet much of this variation likely reflects different demand
conditions. To use this variation for identification, we must therefore isolate the variation
arising from either supply-side factors or demand shocks specific to customers outside the
experiment. While we do not observe such instruments for price, we can however condition
on the demand from the randomly assigned control group. Any demand factors affecting
VPP assigned households should also affect control households. Thus conditioning on the
demand of the control households, allows us to isolate price variation from the external
factors mentioned above.
Consider the following demand and supply functions: q (pt , ξIt , εit , θi )and p (q (ξIt , ξ−It , θ) , Z),
where ξIt represents demand factors common to all i ∈ I households in the experiment and
ξ−It represents demand factors common to all the utility’s customers outside the experiment
(e.g. households and businesses in other cities). εit is an idiosyncratic shock to consumer
i at t. θi is i’s time invariant preferences and θ is the vector of all customers’ (in and out
of the experiments) time invariant preferences. Z represents supply shifters, excluded from
demand. Without Z and ξ−It the VPP price should not vary, conditional on ξIt . Thus
15
variation we observe conditional on ξIt must be arising from these exogenous sources. The
only possible violation of our identificaiton assumption might occur if ξIt is changing but
households assigned to the control condition are unable to respond because of their lack of
portal access, IHD or PCT. However, all three of these technologies are oriented around
price movements, which we observe. There are no obvious non-price factors that would alter
these households’ demands and not control households’. Weather is the primary demand
determinant for these households with air conditioning in a hot southern summer.
Figure 7 depicts the All 3 and IHD only (dotted) estimated demand curves for the 10th,
median and 90th percentiles of control demand during peak periods. Confidence intervals
for each point on the demand curves are depicted with error bars. As noted earlier, the home
automation included in the All 3 demand curves clearly makes it more elastic. In fact, the
IHD demand curve is effectively perfectly inelastic with insignificant movements in demand
creating the illusion of a subtle upward slope at times. When aggregate (control) demand
is low, the 10th percentile demand to the left illustrates the additional PCT device helps
consumers expand demand when price reaches its floor of 4.5c. The largest price observed
at this low demand is 23c. At larger aggregate demands, we observe the maximum 46c
price as well. In these cases, the PCT device enables an additional demand reduction of 34
percent and 23 percent respectively for the median and 90th percentiles of control demand.
As depicted in Figure 8, the addition of the PCT reduces expenditures in these cases, while
expanding it when the low price is offered in a low demand state. We consider the lowest
and highest prices for the 10th and 90th percentiles, while choosing the 23c price for the
median. 23c is actually above the median for VPP customers, but is the fixed peak price for
TOU customers.
Finally, the welfare in each case is depicted in Figure 9 by considering the surplus changes.
The dark red shaded area in the figure to the left depicts the additional consumer surplus
from expanding consumption at the 4.5c price point. In the middle and right figures, the
dark-blue shaded area to the upper-right represents the extra expenditures with the IHD
16
0.5
PCT/IHD, 10th perc.
IHD, 10th perc.
PCT/IHD, 50th perc.
IHD, 50th perc.
PCT/IHD, 90th perc.
IHD, 90th perc.
0.4
Price
0.3
0.2
0.1
0.0
1.0
1.5
2.0
2.5
3.0
kW
3.5
4.0
4.5
Figure 7: All 3 vs. IHD Demand: 10th, 50th and 90th Percentiles of Control Demand
0.5
0.5
0.5
0.4
0.4
0.4
0.3
0.3
0.3
0.2
0.2
0.2
0.1
0.1
0.1
0.0
0.0
0
0.5
1
1.5
2
0.0
0
0.5
1
1.5
2
2.5
3
3.5
4
0
1
2
3
Figure 8: All 3 vs. IHD Expenditure Differences
17
4
5
0.5
0.5
0.5
0.4
0.4
0.4
0.3
0.3
0.3
0.2
0.2
0.2
0.1
0.1
0.1
0.0
0.0
0
0.5
1
1.5
2
0.0
0
0.5
1
1.5
2
2.5
3
3.5
4
0
1
2
3
4
5
Figure 9: All 3 vs. IHD Welfare Differences
device that are not offset by consumer utility.
4.2
Aggregating the Surplus Changes Across States
The figures above depict the surplus changes for particular demand states as represented by
the 10th, median and 90th percentiles of demand. To estimate the total welfare effects, we
must integrate over all such states. We do this by considering each tenth percentile of the
control demand, solving for the welfare effects at each price observed in that percentile and
then calculating the weighted average based on on how frequently that price was charged
within that percentile of control demand.
Across an entire summer, we estimate the welfare effects to be between $27 and $30.
The variation is based on lower and upper bounds of the shape of the demand curve below
the 4.5c price point. Elsewhere, we assume linearity between price points. If we relax the
linearlity assumptions and create lower and upper bounds based on forcing the demand curve
to immediately drop either just before or after each price point, we obtain a range of $21
to $49. If we consider the net present value at a rate of 10%, these bounds are $213 to
$486 with the welfare under linearlity between price points between $273 and $296. In other
words, it appears the consumer welfare might be enough to justify a household making the
$250 investment to acquire the technology. But, the cost of advertising to communicate the
benefits could make this unprofitable. Nevertheless, in this case the utility subsidized the cost
18
of the device and based on the demand reduction number reported above that investment
paid off.
5
Firm Welfare Effects from Home Automation Technology
In addition to the consumer welfare effects of the PCT, the firm can benefit from reducing
demand during periods of critical peak demand. The utility we are working with estimates
the net present value of 1 kW of demand reduction in critical periods to be $700 due to
the deferred investment in generation capital. The costs of supplying a PCT is $250. This
means that the PCT needs to decrease critical period usage by -250/700 = -0.357 kW in
order for the utility to want to subsidize the installation of a PCT in a consumers household.
Based on the average treatment effects reported above, the utility is certainly incentivized
to do this as the difference between the PCT and portal demand reductions are nearly a full
kWh. In this section, we explore whether heterogeneity in these treatment effects creates
an incentive for the utility to select (target) particular households for the installation of
the PCT. To do this we first document the limited value of conditional treatment effects
for uncovering heterogenous treatment effects, then propose a methodology for estimating
household specific treatment effects. We then compare the welfare difference from installing
a PCT in all households vs. restricting it to households where the expected treatment effect
suggests it would reduce critical demand by more than .357 kW.
5.1
Treatment Effects Conditional on Observed Types
The most common form of targeted marketing is to condition on observable characteristics
such as demographics. For example, the age of the household head or the size of the family
could relate to the ability to use technology based treatments or to differences in the ability
to time shift consumption of electricity. We therefore seek an expected difference in outcomes
19
that is conditional on demographics:
α
ˆ ax = E [Y |A = a, X = x] − E [Y |A = 0, X = x]
To determine heterogeneity in the treatment effects as a function of demographic variables, we regress critical usage on the eight treatment dummy variables (four technology
treatments interacted with the two price treatments) interacted with the demographic variables, age and income. The estimated regression coefficients for the treatment and demographic interactions are shown in Figure 10.
While in general, the effects appear to be slightly larger for higher income households and
those with a younger head-of-household, it may be the case that demographic variables are
not the most effective information in explaining responsiveness to these different treatment
effects. And in fact, we do have much more data for all of these households, namely their
usage patterns in previous years. The next section discusses our methodology for defining
consumer types as a function of their past usage and we demonstrate the usefulness in using
this information in targeting households.
5.2
Individual Treatment Effects
Our goal is to develop an estimation approach that complements the randomized treatments by utilizing extensive pre-treatment consumption histories to estimate individual
(household)-level treatment effects. A straightforward application of non-parametrically estimated conditional treatment effects would take expectations over all households j, while
conditioning on their vector of (pre-treatment) usage history, Hj being equal to a focal
household i’s usage history, Hi :
α
ˆ ai = E [Yj |Aj = a, Hj = Hi , Xj = Xi ] − E [Yj |Aj = 0, Hj = Hi , Xj = Xi ]
20
Figure 10: Treatment effects conditional on demographics
N=2,177. Includes controls for age, income, 2010 average hourly usage
Bootstrap standard errors shown by error bars
21
In this true non-parametric approach, H includes each hourly interval over the summer
of 2010, while X includes demographics as in the previous section. The non-parametric
estimator would weight each household j based on its distance, Dij from the focal household
i:
Dij =
X
|Hjt − Hit |
t
Taking the number of households to infinity, we would converge to an estimate based on
essentially “matching” households. The challenge is that this is a very high-dimensional
problem. Furthermore, it is not efficient. For example, if two individuals roll identical dice
each day to determine their usage, this distance will be quite large even though they are
the same. Despite having identical “policy functions,” these two individuals would “match”
with a very low probability that decreases quickly in the number of times we observe them.
On the other hand, their distributions of state-dependent usage would converge the more we
observe them.
We propose estimating individual-level treatment effects using non-parametric regressions
that weight each household based on its Kolmogorov distance between its pre-treatment usage
and that of the focal household. This distance has well documented converge properties
for testing between differences in distributions such that it suggests itself as a valuable
distance statistic for this type of non-parametric regression. In our application, we will have
a balanced panel, but the Kolmogorov distance has the advantage that it can also be used
in cases where the number of pre-treatment outcomes vary across units.
In addition to taking a simple Kolmogorov distance between the past usage of households, we also consider a measure that weights the distribution of usage across “states” in
which a household’s policy function might differ. Examples include outside temperature,
weekends vs. weekdays or past usage. The idea is that changes in the distribution across
these conditions could uncover other important primitives of the household’s preferences for
electricity. For example, if the distributions change dramatically with time of day (holding
temperature fixed), this may indicate someone who may get less value from a device that
22
adapts their electricity usage, because they already adapt well.
Although we do not attempt to recover the policy functions and underlying structural
parameters at each state, the motivation for this approach is based on recent development in
the estimation of dynamic decisions using a two-step approach. Bajari, Benkard, and Levin
(2007) follow Hotz and Miller (1993) in highlighting the importance of estimating policy
functions in a first stage. More recently, Kasahara and Shimotsu (2009) and Arcidiacono and
Miller (2011) have considered the identification and estimation of unobserved heterogeneity
in these contexts. Extending our analysis to recover household-level policy functions and
structural parameters would be necessary if the firm were to consider treatments other than
the ones we observe, such as alternative price schedules.
5.2.1
The matching procedure
The household-level treatment effect, conditional on the distribution of past usage is:
α
ˆ ai = E Yj |Aj = a, Fj,Tj,s (h|s) = Fi,Ti,s (h|s)
(2)
−E Yj |Aj = 0, Fj,Tj,s (h|s) = Fi,Ti,s (h|s)
where the expectation is taken across households indexed by j 6= i. Fj,Tj,s (h|s) is the
empirical distribution function for Tj,s iid observations of j’s historical usage, Hj,t , when j
is in state s. Note that the above estimator does not require that we know the distribution
of h for i, but that the expectations of post treatment outcomes Y are taken conditional on
the empirical distribution of pre-treatment outcomes H coming from the same distribution
as i’s. We therefore evaluate the equivalence of the pre-treatment outcome distributions in
state s for i and any other candidate household j using the Kolmogorov-Smirnov statistic
for two samples:
Dij,s = sup Fj,Tj,s (h|s) − Fi,Ti,s (h|s)
h
23
(3)
5
4.5
x 10
5
3
x 10
4
3.5
2.5
3
2
2.5
1.5
2
1.5
1
1
0.5
0.5
0
0
0
0.2
0.4
0.6
0.8
1
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Figure 11: Histogram of Kolmogorov-Smirnov statistics, with and without states
In our application, we consider the following state variables in the pre-treatment period
(the summer of 2010): time of day, temperature, weekend vs. weekday, and recent electricity
consumption. All state variables are discretized. Time of day is segmented into the following
blocks of time: midnight to 6am, 6am to noon, noon to 6pm, 6pm to midnight. Temperature
is categorized as cold, average, or hot using the 25th and 75th percentiles of the temperature
distribution from the summer of 2010 as the boundaries. Finally, we summarize past electricity consumption as low or high consumption on the preceding day based on the median
of the household’s daily consumption over the summer. This accommodates state-dependent
consumption in which heavy laundry one day may alleviate the need to consume as much
electricity the following day. Such effects could persist for more than one day, but the effect
on the following day should be sufficient to separate households that do, or do not, exhibit
state-dependent consumption; x therefore is a state-point that combines all three factors.
To create a distance measure that integrates over these states, we next derive the unconditional distance between i and j by taking a weighted average over state points:
P
Dij =
s
pi,s Dij,s
P
pi,s
(4)
s
where pi,s is the fraction of the time household i is in state s. Given we do not know the
24
true distribution pi,s , we use the empirical distribution in its place. Dij is then plugged
into a kernel to form a weight in a non-parametric regression. Weights are assumed to be
zero if Aj 6= a. Thus some j households will contribute to the first expectation in Equation
2 (if they were in fact in the focal treatment), while others will contribute to the latter
expectation (i.e. if in they were in fact in the control condition).
The estimator will seek to reduce the influence of households with distances close to one,
while heavily weighting those households close to zero. We consider two Kolmogorov based
estimates. Our first estimator uses the Kolmogorov-Smirnov statistics in which we calculate
the distance without conditioning on the states. In terms of the preceding equations, this
is equivalent to assuming there is a single state. A histogram of these between households
distances is depicted in the first graph in Figure 11. The second estimator uses the weighted
average Kolmogorov distance across all of the state-specific distances. A histogram of these
distances is shown in the second graph. The distribution here is shifted further to the right.
One obvious reason for this is that each state-specific distance contains only a subset of the
observations. Just as distances go to zero with additional observations, fewer observations
tends to increase distances.
We estimate the treatment effects (and control variable coefficients) using a local constant
estimator for each household i, where K is the Epanechnikov kernel with optimal bandwidth
hi . As a reminder, Ai− is the treatment dummy matrix and X i− are the demographic
variables for all other households j 6= i (indicated by i−), and Dii− and Yi− are the distance
matrix to household i (which depends on the estimator being used) and usage for all other
households:
−1
Dii−
1
Dii−
1
0
0
ˆ
[Ai− , Xi− ] Yi− ∗ K
.
θi = [Ai− , Xi− ] [Ai− , Xi− ] ∗ K
hi
hi
hi
hi
Without X , this provides an average treatment effect on the individual or household. Adding
25
Table 3: Average SSE of estimators
Critical
Peak
Off-Peak
Critical
Peak
Off-Peak
Critical
Peak
Off-Peak
Average Usage
KS statisticis
KS statistics as function of state
Average
1.029
0.663
0.184
0.903
0.575
0.150
0.910
0.568
0.152
Fraction<.5
0
0.044
1.000
0
0.004
1.000
0.013
0.382
0.999
Fraction<1
0.204
1.000
1.000
0.636
1.000
1.000
0.652
0.998
1.000
X allows us to also include common observable heterogeneity in demographics.
For each household, we optimize the bandwidth by minimizing the weighted sum of
squared errors (WSSE) where L is the length of θi :
PI
W SSEi =
2
y−i − X−i θbi
Dii−
K
− L/2
i=1
hi
i=1 K
PI
Dii−
hi
(5)
We again estimate the treatment effect for critical prcing events, peak usage not during
critical prciing events, and off-peak usage.
.
To assess the value of the rich histories of pre-treatment electricity usage, we also consider
a benchmark model that calculates distance based only on average electricity usage in 2010.
We also could include additional moments but then we lose the unidimensionality of the
Kolmogorov statistic (KS). We compare this baseline model with matching on average usage
to our KS based models by calculating the average sum of squared residuals across households
for equation (5.2.1) for each. Table 3 shows the average SSE for each of the three methods.
We see a reduction in the WSSE for the critical period regressions of 12.1% and 11.5% when
using the KS matching and KS matching by state, repectively, compared to just matching
on 2010 mean usage. The use of KS matching has great benefits in reducing the error, but
the benefit of using KS matching by state appears limited (it is actually a little higher) when
26
optimal peak bandwidth
optimal off−peak bandwidth
300
250
250
250
200
200
200
150
number of households
300
number of households
number of households
optimal peak bandwidth
300
150
150
100
100
100
50
50
50
0
0.4
0.5
0.6
0.7
0.8
0.9
0
1
0.4
0.5
0.6
bandwidth
0.7
0.8
0.9
0
0.2
1
0.3
0.4
0.5
0.6
bandwidth
bandwidth
sse peak usage
sse peak usage
250
250
200
200
0.7
0.8
0.9
1
sse off−peak usage
450
400
350
100
number of households
number of households
number of households
300
150
150
100
250
200
150
50
100
50
50
0
0.4
0.6
0.8
1
1.2
sse
1.4
1.6
1.8
2
0
0.2
0.4
0.6
0.8
1
1.2
1.4
1.6
0
0
sse
0.1
0.2
0.3
0.4
0.5
sse
Figure 12: Histogram of optimal bandwidths and WSSE
focusing only on the mean. However, when looking at the number of households with WSSE
less than 0.5 kW in the peak regressions, matching by state has a huge advantage since 38%
of households are under this threshold compared to only 0.4 when ignoring state. We plot
histograms of the bandwidth and resultant WSSE for the KS matching by state in Figure
12.
One final challenge that remains is the estimation of the standard errors. To do this we
use bootstrapping, with replacement. Also, although we selected the bandwidth to minimize
the WSSE, our estimator is biased in finite samples. In the results that we present in the
next section, we report bias-corrected estimates, θi∗ = 2θbi − θbi , where θbi is the average of the
B
point estimates θbr
ib over B bootstrap samples, with B = 100. We report the bootstrapped
2
PB c
1
B
b
standard errors, B b=1 θib − θi .
Since our method involves estimating the treatment effects for each household without
using that household’s post-treatment usage data, one nice test of the benefit of using KS
matching (with states) is to compare the prediction errors for the focal households to the
27
0.6
0.7
prediction error when matching based on 2010 average usage. We find that the prediction
error for critical periods declines in magnitude from 0.824 to 0.802, peak declines from 0.663
to 0.622 and off-peak from 0.330 to 0.313.
5.2.2
Matching Results
We were able to estimate household-level treatment effects using state-specific distributions of past usage for 2146 out of the 2334 households who were not in the extreme tail of
the usage distribution. The mean and standard deviation (with the average standard error)
of the household-level treatment effects are shown in Table 4. These estimates of the average
effects are similar to those reported in Tables 2. The important feature of this analysis is the
standard deviation of the household effects. In many cases the standard deviation is greater
than the average effect, indicating significant household heterogeneity in treatment effects.
The most heterogeneity is observed for the most effective treatments: PCT and All Three.
Notable between these is that the heterogeneity is greater for the All Three treatment despite
its average effect being smaller than the PCT treatment. This may be due to heterogeneity
in how the presence of an IHD in combination with the PCT altered adaptation of electricity
consumption. Patterns across TOU and VPP pricing are similar, with the TOU generally
performing better as reported in the average treatment effects.
To better illustrate the heterogeneity in the treatment effects, we plot the distribution of
the effects in Figures 13 and 14 for TOU and VPP, respectively. We show the CDF of the
treatment effects for both the peak and off-peak periods, relative to the control treatment.
We also plot the CDF of the lower and upper bounds on the 95% confidence interval of
the household-level estimates. We do this to give the reader a visual cue to the relative
size of the household-level confidence interval – the confidence interval on the distribution
itself is much tighter. We plot a vertical line at x = 0 to show the normalized usage for the
control condition, and a horizontal line at y = 0.5 to assist the reader in assessing whether
28
Table 4: Summary of Household Treatment Effect Regression Results
Critical
TOU
VPP
Treatments Average SD Average SD
-0.271
0.241
-0.351
0.335
Portal
(0.164)
(0.159)
-0.418
0.364
-0.224
0.368
IHD
(0.178)
(0.168)
-1.079
0.649
-1.083
0.570
PCT
(0.203)
(0.185)
-0.870
0.642
-1.078
0.604
All three
(0.197)
(0.211)
Peak
TOU
VPP
Treatments Average SD Average SD
-0.195
0.223
-0.216
0.272
Portal
(0.131)
(0.126)
-0.327
0.342
-0.111
0.286
IHD
(0.143)
(0.134)
-0.855
0.536
-0.645
0.476
PCT
(0.156)
(0.150)
-0.712
0.563
-0.534
0.487
All three
(0.161)
(0.169)
Off-Peak
TOU
VPP
Treatments Average SD Average SD
-0.0431 0.110
-0.023
0.010
Portal
(0.068)
(0.074)
-0.060
0.114
0.064
0.115
IHD
(0.078)
(0.076)
0.092
0.159
0.054
0.098
PCT
(0.074)
(0.083)
0.046
0.157
0.078
0.122
All three
(0.086)
(0.078)
Average standard error in parenthesis
29
Figure 13: On and Off-Peak Treatment Effects, TOU
1
1
critical
peak
off peak
0.8
0.8
0.7
0.7
0.6
0.6
0.5
0.4
0.3
0.3
0.2
0.2
0.1
0.1
−2
−1
0
1
difference in control and portal coef.
2
0
−3
3
1
0.9
−1
0
1
difference in control and ihd coef.
2
3
−1
0
1
difference in control and all 3 coef.
2
3
critical
peak
off peak
0.9
0.8
0.8
0.7
0.7
0.6
0.6
0.5
0.5
0.4
0.4
0.3
0.3
0.2
0.2
0.1
0
−3
−2
1
critical
peak
off peak
cdf
cdf
0.5
0.4
0
−3
critical
peak
off peak
0.9
cdf
cdf
0.9
0.1
−2
−1
0
1
difference in control and pct coef.
2
3
0
−3
30
−2
Figure 14: On and Off-Peak Treatment Effects, VPP
1
1
critical
peak
off peak
0.8
0.8
0.7
0.7
0.6
0.6
0.5
0.4
0.3
0.3
0.2
0.2
0.1
0.1
−2
−1
0
1
difference in control and portal coef.
2
0
−3
3
1
0.9
−1
0
1
difference in control and ihd coef.
2
3
−1
0
1
difference in control and all three coef.
2
3
critical
peak
off peak
0.9
0.8
0.8
0.7
0.7
0.6
0.6
0.5
0.5
0.4
0.4
0.3
0.3
0.2
0.2
0.1
0
−3
−2
1
critical
peak
off peak
cdf
cdf
0.5
0.4
0
−3
critical
peak
off peak
0.9
cdf
cdf
0.9
0.1
−2
−1
0
1
difference in control and pct coef.
2
3
0
−3
31
−2
the median household consumes more or less electricity than in the control condition. In
all technology treatments for TOU we find at least 50% of households significantly reduce
peak consumption for the Portal and IHD, and these numbers are over 80% for the PCT
alone. We also find that more than 50% of households reduce consumption at statistically
significant levels for all technologies using VPP pricing with the exception of the IHD. It is
clear from visual inspection that all treatments decrease usage during critical and non-critical
peak hours.
5.2.3
Can demographics explain the heterogeneity?
By using rich pre-treatment data, we can estimate household treatment effects in order to
best target them. To show the advantage of this methodology over using just demographic
variables, in Figure 15 we plot separate CDFs for the different demographic bins for the
effect of the PCT on critical consumption, for each of the different demographic variables.
There does appear to be some evidence that family households are more affected by the PCT
relative to the portal, with little difference between young and old households, and that the
effectiveness of the PCT relative to portal increases with income.2 Nevertheless, through
all of these plots, it is clear that demographic differences are small and there is substantial
heterogeneity, conditional on demographics, explained by the past usage data used to create
the household level estimates.
5.2.4
Who do we target?
Because we are interested in which technology to use when targeting specific households
when giving them a smartmeter, in Table 5 we show counts of the sign and significance of
the relative impacts of the IHD, PCT and All 3 treatments relative to the portal treatment.
For the PCT and All 3 technology treatments, we see that the estimated increase in reduction of usage during critical and peak periods is significant in over half of households in
2
It should be noted that these effects are in absolute changes in consumption - families and higher income
households also consumer more electricity.
32
Figure 15: Difference between Peak and Off-Peak Treatment Effects
critial
critial
1
0.8
0.7
0.7
0.6
0.6
0.5
0.5
0.4
0.4
0.3
0.3
0.2
0.2
0.1
0.1
0
−3
−2
family
old
young
0.9
0.8
cdf
cdf
0.9
1
family
old
young
−1
0
1
difference in control and pct coef.
2
0
−3
3
−2
critial
3
1
low income
middle income
high income
NW
0.9
low income
middle income
high income
NW
0.9
0.8
0.8
0.7
0.7
0.6
0.6
cdf
cdf
2
critial
1
0.5
0.5
0.4
0.4
0.3
0.3
0.2
0.2
0.1
0
−3
−1
0
1
difference in control and pct coef.
0.1
−2
−1
0
1
difference in control and pct coef.
2
3
0
−3
33
−2
−1
0
1
difference in control and pct coef.
2
3
both pricing conditions. The IHD is signficanly more effective than just the portal in 636
homes under TOU pricing, but it is only signficanly more effective under VPP pricing for
10 households.IHD with VPP is less effective at reducing consumption for 287 households
– it could be that the increased acceability of information coupled with the VPP pricing
actually encouraged consumers to use more electricty at the lower price point which occured
frequently in the observation window..
A substantial number of households exhibit significant increases in usage in the off-peak
hours when using the programmable thermostats relative to the portal – 970 under TOU
pricing. We also see that the All 3 conditions leads to fewer households significantly reducing
consumption during critical and peak periods or increasing consumption during off-peak
periods, relative to the portal. Therefore the portal treatment appears to dominate the All
3 condition. However, despite its effectiveness on average, when targeting a household with
the PCT, we have to recognize that the approximate cost of supplying a PCT is $250 and
so the cost effectiveness of supplying consumers with the PCT will depend on the size of the
expected critical reduction in usage relative to an estimate of the resultant deferred costs to
the utility.
5.2.5
Cost effectiveness
In what follows, we focus on the portal vs. PCT technologies since the PCT dominates the
other treatments in terms of peak load reduction. In Figure 16 we show scatter plots of
the PCT versus portal coefficients for the two price treatments. These results have clear
implications for firm behavior. In Figure 16, we also show the break-even line for the utility
which represents the additional 0.357 kW reduction of the PCT over the portal. Any household below the line should receive the PCT. There is a clear incentive to target the PCT.
Doing so avoids the $250 installation cost in those homes that would not reduce demand
by the break-even amount. This increases the utilities welfare by 29% under VPP and 20%
under TOU. The average of the household treatment effects reported in Table 4 implies a
34
Table 5: Significance in Household Differences between Non-Portal Technologies and Portal
Critical
Sig. Negative
Negative
Positive
Sig. Positive
IHD
636
729
408
25
TOU
PCT All 3
1436 1104
209
450
102
204
51
40
IHD
717
574
455
52
TOU
PCT All 3
1447 1105
238
435
84
225
29
33
IHD
81
1107
541
69
TOU
PCT All 3
17
11
90
146
721 1211
970
430
Peak
Sig. Negative
Negative
Positive
Sig. Positive
Off-Peak
Sig. Negative
Negative
Positive
Sig. Positive
35
IHD
10
691
810
287
VPP
PCT All 3
1226 1134
406
519
164
142
2
3
IHD
45
679
727
374
VPP
PCT All 3
1081 922
398
436
280
418
39
22
IHD
18
130
1350
300
VPP
PCT All 3
14
15
287
133
1251 1120
246
530
surplus gain of $247 and $323 over the $250 cost for VPP and TOU respectively. However,
the targeting of PCTs to those households with sufficient demand reduction increases the
surplus gains to $319 and $386 respectively.
6
Conclusion
We have demonstrated that home automation technology can create surplus increases on
both the consumer and firm side. The firm side surplus is substantially larger suggesting
they should be willing to subsidize the installation of technology. We also illustrate how
large historical databases of behavior can be used to identify individual-specific treatment
effects. In the context of our electricity demand experiment, we show that the utility can
increase its surplus by selectively subsidizing programmable communicating thermostats to
a subset of households. Our approach is designed to work in instances where the historical
data can be treated as a series of repeated draws from a process. While electricity usage
conditional on weather and other states fits this description, there are many more potential
applications. Technological innovations in communicating and/or wearable technology are
enabling the collection of even more extensive behavioral data. By integrating these data into
experiments, as we have done here, there is hope to extend varying treatment prescriptions
beyond simple demographics which, in many applications such as marketing, explain very
little heterogeneity across individuals.
References
Abadie, A. (2002): “Bootstrap Tests for Distributional Treatment Effects in Instrumental
Variable Models,” Journal of the American Statistical Association, 97(457), 284–292.
Allcott, H. (2011): “Rethinking real-time electricity pricing,” Resource and Energy Economics, 33, 820–842.
36
Figure 16: PCT and Portal Peak Treatment Effects
pct vs portal, tou peak
1.5
with AC
no AC
AC unknown
1
0.5
0
pct
−0.5
−1
−1.5
−2
−2.5
−3
−3
−2
−1
0
portal
1
2
3
pct vs portal, vpp peak
1.5
with AC
no AC
AC unknown
1
0.5
0
pct
−0.5
−1
−1.5
−2
−2.5
−3
−3
−2
−1
0
portal
37
1
2
3
Arcidiacono, P., and R. A. Miller (2011): “Conditional Choice Probability Estimation
of Dynamic Discrete Choice Models with Unobserved Heterogeneity,” Econometrica, 79(6),
1823–1867.
Bajari, P., C. L. Benkard, and J. Levin (2007): “Estimating Dynamic Models of
Imperfect Competition,” Econometrica, 75(5), 1331–1370.
Borenstein, S., and S. Holland (2005): “On the Efficiency of Competitive Electricity
Markets with Time-Invariant Retail Prices,” RAND Journal of Economics, 36(6), 469–493.
Cappers, P., C. Goldman, and D. Kathan (2010): “Demand response in U.S. electricity
markets: Empirical evidence,” Energy, 35(4), 1526–1535.
Honka, E. (2014): “Quantifying Search and Switching Costs in the US Auto Industry,”
RAND Journal of Economics, 45(4), 847–884.
Hotz, V. J., and R. A. Miller (1993): “Conditional choice probabilities and the estimation of dynamic models,” Review of Economic Studies, 60, 497–529.
Imbens, G. W. (2004): “Nonparametric Estimation of Average Treatment Effects Under
Exogeneity: A Review,” Review of Economics and Statistics 2004, 86(1), 4–29.
Kasahara, H., and K. Shimotsu (2009): “Nonparametric identificatino of finite mixture
models of dynamic discrete choices,” Econometrica, 77(1), 135–175.
Kolmogorov, A. (1933): “Sulla determinazione empirica di una legge di distribuzione,”
G. Ist. Ital. Attuari, 4, 83–91.
Reiss, P. C., and M. W. White (2005): “Household Electricity Demand, Revisited,”
Review of Economic Studies, 72(3), 853–883.
Rosenbaum, P. R., and D. B. Rubin (1983): “The central role of the propensity score in
observational studies for causal effects,” Biometrika, 70(1), 41–55.
38
Rossi, P. E., G. M. Allenby, and R. McCulloch (2005): Bayesian Statistics and
Marketing.
Seiler, S. (2013): “The impact of search costs on consumer behavior, a dynamic approach,”
Quantitative Marketing and Economics, 11(2), 155–203.
39