I don’t usually open up this blog to political debate, but my only commenter has been challenging me over the “incompetence” of the Australian Home Insulation Program recently, so I thought I’d try my statistical skills at investigating it, given that I’ve already used them so effectively to prove that all British people are ignorant. In this post I’m going to analyse the rate of fires occurring in houses before the advent of the Home Insulation Program, and after, and show that under a wide range of assumptions (some realistic, some unrealistic), the Insulation Program probably led to a reduction in the rate of house fires after newly-installed insulation relative to the time before its implementation. I will also attempt to give some explanations for this. This builds heavily on the work of Possum at Crikey, but with the addition of a time-dependent element to the analysis, a wider range of assumptions (within which Possum’s are special cases), and a bit of risk analysis. This isn’t to say Possum can’t do such things, but he/she didn’t, and since the linked analysis the Coalition have released new figures showing that the program is “even worse” than previously believed.
Introduction (skip if you’re Australian)
For my foreign reader(s), it may be a little puzzling that I’m diverting from discussion of Double Cross 3 to a relatively trivial statistical analysis of something as tedious as home insulation in Australia. In 2007 the Australian government changed after 11 years to become a Labor Government (left wing by standard definitions), and its response to the Global Financial Crisis (GFC) was to introduce a bunch of Keynesian pump-priming, including the Home Insulation Program (HIP). The then government became the opposition Coalition, and ran a heavy campaign against the pump-priming under its ludicrously maniacal leader, a failed monk called (appropriately) Tony Abbott. Their campaign relies heavily on accusations of wasteful spending and inefficiency, and they attacked all aspects of the government’s programs.
The HIP was intended to provide householders money to install installation in their home, generally through the use of contractors, whose numbers exploded overnight. The Coalition quickly realised that post-insulated homes have a heightened fire risk, and started making hay out of the fact that there were lots of insulation-related fires. They just didn’t mention that there have always been insulation-related house fires in Australia, and Possum’s analysis above was the first anyone has seen (as far as I’m aware) that compares pre- and post-HIP rates of fires. The central Coalition claim – that the government endangered householders through its poorly-run program – depends on the assumption that rates of house fires went up, since these insulation installations were a choice people made, so if the rate stayed the same there is no argument[1].
Method
Numbers of fires before the HIP, and numbers of installations per year before and after the HIP, along with total numbers of houses already with insulation installed, were obtained from the ABS and the Federal Government via the above-linked Possum post. The number of post-HIP fires was helpfully provided by Coalition press-release today[2]. Details of the length of time the HIP was running and some other minor figures were obtained from the Department Secretary’s statement linked to by Possum at http://blogs.crikey.com.au/pollytics/files/2010/02/Secretarys-opening-statement-220210.pdf.
The number of fires was converted into a rate of fires per insulated house-year. That is, a single house that was insulated for a full year was considered to contribute 1 insulated house-year (IHY) to the risk pool, and the rate was presented as a rate of fires per 1000 IHY. The number of IHY was calculated under a set of analysis cases.
Case 1: All fires were caused by insulation installations in the year that the fire was recorded, and all such installations happened in the first day of that year, contributing a full IHY to the period of the study.
Case 2: All fires were caused by insulation installations in the year that the fire was recorded, but the installations occurred smoothly over the year. If there were N installations in the period, then 1/365 of these occurred in the first day of the year, 1/365 in the second, and so on. This means that N/365 installations contributed 1 IHY to the risk pool, N/365 contributed 364/365 IHY to the risk pool, and so on.
Case 3: All installations from before the HIP were assumed to have an equal risk of a fire, so that every insulated house in Australia before the HIP was in the risk pool for one full year; these houses were also in the risk pool for the post-HIP period.
Case 4: An exponential rate of decay of risk was assumed over years, so that the risk of a fire decayed by exp(alpha) for every year since the installation. So in year 0, all houses contributed to the risk pool; in year 1, exp(alpha) houses, and so on. alpha was chosen for this case so that 25% of houses in year 1 contribute to the risk (but we will also present some sensitivity analysis).
Case 5: Information from the Secretary’s letter was used to identify the total pool of risk post-HIP, and compared to the year before the HIP under the conditions of case 1 (a full year’s risk). Under this case, the post-HIP period was assumed to be 15 months long. 176000 homes were installed in November 2009, 3300 in March 2009, and the remainder were assumed to be installed in between these periods, at an arbitrary point assumed to be September 2009.
As an additional note: Cases 1 to 4 were calculated based on a silly piece of rhetoric from the Coalition, which claimed that “reported house fires from her [Julia Gilllard’s] program [are] still running at around one a day.” There were 191 fires post-HIP in the same press release, so the Coalition seem to think the post-HIP period was only 200 days, when in fact it’s 15 months. However, assuming the 200 day period benefits the Coalition in this analysis, since a shorter post-HIP period means a smaller risk pool and thus a higher rate of fires. This is conservative statistics at its best (literally!).
Headline figures
The headline figures used here are:
Pre-HIP fires: 85
Pre-HIP installations per year: 70000
Houses insulated pre-HIP: 3183625
Post-HIP fires: 189
Houses insulated post-HIP: 1100000 (1.1 million)
Post-HIP period: 200 days (15 months in case 6).
Results
Case 1:Assuming fires occur due to installations in the year of the fire only, and all installations at the first day of the year
This gives us 85 fires in 70,000 IHY pre-HIP, and 189 fires in 1100000 IHY post-HIP.
Rate of fires pre-HIP: 1.21 per 1000 IHY
Rate of fires post-HIP: 0.31 per 1000 IHY
Relative risk of a house fire post-HIP vs. pre-HIP: 0.26
Case 2: Assuming fires occur due to installations in the year of the fire only, but installations are evenly distributed over the year
This gives 85 fires in 35095 IHY pre-HIP, and 189 fires in 165959 IHY post-HIP.
Rate of fires pre-HIP: 2.42 per 1000 IHY
Rate of fires post-HIP: 1.14 per 1000 IHY
Relative risk of fire post-HIP vs. pre-HIP: 0.47
Case 3:Fires in a given year are due to any house ever insulated up until that point; all post-HIP insulations occurred in the start of the year
This gives 85 fires in 3183625 IHY pre-HIP, and 189 fires in 3786364 IHY post-HIP.
Rate of fires pre-HIP: 0.027 per 1000 IHY
Rate of fires post-HIP: 0.050 per 1000 IHY
Relative risk of fire post-HIP vs. pre-HIP: 1.87
Case 4: Assume exponential decay of risk, all installations post-HIP at the start of the period
Assuming a exp(-0.3)% decay in risk per year, this gives 85 fires in 270,080 IHY pre-HIP, and 189 fires in 802820 IHY post-HIP. In this model we assume 70000 houses a year were insulated over 46 years until the start of the HIP period, when 1.1 million more were insulated in 200 days.
Rate of fires pre-HIP: 0.31 per 1000 IHY
Rate of fires post-HIP: 0.24 per 1000 IHY
Relative risk of fire post-HIP vs. pre-HIP: 0.74
This case can be modified to incorporate the assumptions of case 2 or 5 about the distribution of installations post-HIP (smooth over the period or end-loaded), but it likely won’t make much difference, since in this case large amounts of the risk pool come from previous years of data, which are the same for both the pre-HIP and post-HIP installations.
Case 5: Using the departments figures to approximate the risk pool post-HIP
We can do this using the assumptions of Case 1 or Case 2 for the pre-HIP risk pool. Case 1 is more favourable to the Coalition, so we use that one.
This gives 85 fires in 70,000 IHY pre-HIP, and 189 in 829792 post-HIP.
Rate of fires pre-HIP: 1.21 per 1000 IHY
Rate of fires post-HIP: 0.23 per 1000 IHY
Relative risk of fire post-HIP vs. pre-HIP: 0.19
Sensitivity analysis of the exponentially decaying risk
The analysis that is most consistent with any kind of modern frailty or risk analysis is case 4, where the most at-risk houses are assumed to go up soonest. That is, the bodgiest ones burn first. The model I have used above assumes that, effectively, the risk of a fire decays at a rate of exp(alpha*year)% , so in the year of its installation a house contributes 100% to the risk pool, but in the next year it contributes only exp(alpha)%, and then exp(2alpha)%, and so on. We can change this rate by fiddling with alpha. I’ve fixed alpha in the assumption at -0.3, which means the year after installation a house contributes 75% to the risk pool, then 58% and so on. We can fiddle with these figures to estimate the decay rate of risk at which the pre-HIP and post-HIP rates of fire would be equal. It’s actually alpha=-0.175, which corresponds to 83% of the risk transferring from the first year after installation, 70% from the 3rd year, and 17% from the 10th year. Note that case 4, where all houses are assumed to contribute equally to the risk pool no matter when they were built, corresponds with alpha=0, and represents the maximum relative risk of a fire that could occur under any assumptions for the post-HIP period.
I don’t think it’s reasonable to assume that houses insulated 10 years ago are still significantly contributing to the risk of fires today, and I think in fact a decay to almost no contribution over 3 or 5 years is better; hence my choice of -0.3 for alpha. I think everyone would agree that alpha is likely to be between -0.3 and 0, but the 0 assumption is silly. If we fix alpha at between -0.3 and -0.15, the highest Relative Risk of fires for the post-HIP period vs. the pre-HIP period is 1.07. This is a meaningless increase in risk, but it corresponds to houses from 10 years ago still contributing 22% to the risk pool[3].
This sensitivity analysis suggest to me that there is no sense in which the HIP has increased the risk of house fires; in fact, it has decreased the risk of house fires. I should note though that I’m no expert on risk analysis, though I’m good at survival/frailty analysis. So someone else could probably handle this better.
Discussion
It’s easy to imagine that an increase in risk of house fires is inevitable with an expansion of a program that, individually, carries a risk of house fires. But it’s not actually a contradictory finding when considered in light of other types of risk we are familiar with in our lives. It is often the case that the more an activity is performed, the more accurately and efficiently it is performed. Contrary to the claims of the Coalition that the HIP has unleashed an army of “cowboy contractors” risking the lives of ordinary Australians, what may actually have happened is a three-fold reduction in the main risk factors of fires, specifically:
- Homeowners are less likely to do it themselves, and this is probably the single biggest risk factor for insulation-related fires
- Where previously insulation was installed by general builders on an occasional basis, we now have an army of dedicated installers. Though their initial efforts may have been bodgy, the scale of their work – repeatedly doing the same installations for months – may have led to a significant improvement in the quality of installations. We see this with hospitals, where error rates reduce significantly as the number of operations performed increases, and transport, where professional drivers have much lower rates of accidents due to experience. Specialisation is a key way of reducing error-rates, and the HIP may have led to a massive increase in the specialist workforce[4]
- If it’s true that this program is “throwing money” at these contractors, with all the associated inefficiency and waste, presumably their profit margins are much higher than used to be the case for insulation installers. So with higher profit margins, maybe there is actually an increased incentive for them to use higher-quality materials, not cut corners, and actually do the job better – particularly if a quality job leads to referrals, and easier business. In this case these people, in addition to becoming very efficient at the work they do, might actually be doing it to a higher standard of care than was previously the case[5]
My money is on 2) as the cause, in this case, of a possibly quite significant reduction in the risk of fires due to insulation installations in Australia.
Conclusion
This report has found that under a wide range of conditions, including a general model of risk relating existing and new installations of insulation, the HIP likely led to a reduction in the rate of house fires in Australia. The relative risk of house fire after the HIP compared to before was probably about 0.75, though it may have been as low as 0.2. The highest possible relative risk that can be realistically obtained under any set of assumptions appears to be about 1.05, which represents a level of risk broadly similar to that existing before the HIP was introduced. The findings of reduced risk apply even when using the Coalition’s stated estimate of the post-HIP period as 200 days, which approximately doubles the post-HIP rate of fires.
In addition to reducing recipients’ electricity costs, the HIP has reduced the risk of fires in most homes compared to insulation installations done under the pre-HIP program. The most likely explanation for this reduction in risk is the increased specialization of the installers and the scale of their work; but there may also be a contribution due to reductions in poorly-installed home DIY jobs, and also the purported high profit margins of the work may inspire the use of higher-quality materials. Regardless of the explanation, the statistics do not appear to support Coalition claims of reckless endangerment of human life due to the HIP.
—
fn1: of course this Coalition campaign flounders a bit on the fact that it’s private contractors doing the work, and they love encouraging private contractors, so if the contractors did lead to an increase in fires, there is a bit of a credibility problem for subsequent arguments in favour of private sector contractors doing government services cheaper than the state can.
fn2: This is High Science we’re doing here, kiddies
fn3:Finally note that the three scenarios assumed by Possum in his/her modelling fit into the risk model presented here. Scenario 3 (90% of fires from existing stock) corresponds to a value of alpha=-0.1, while Scenario 1 (10% of fires from existing stock) corresponds to alpha=-2.2. Both of these values are, in my view, outside the reasonable range of values we can assign to the relative mix of risks from existing and new stock, but obviously this is just a matter of opinion.
fn4: This could have negative ramifications for the employment rate when the scheme stops and a bunch of insulation specialists have to find new work, I suppose
fn5: I remember a hospital I once worked in did the whole “lowest cost bid” thing for some wiring, and employed a bunch of unqualified building contractors to lay down the ethernet cables. The result was fire and electricity risks, and a network that didn’t work. A year or two later, when a state government renewal project was launched in our area, the project managers visited our hospital and were appalled at the quality of work. They told me that in the early dotcom boom lots of building contractors switched to computer infrastructure jobs like this, and offered bodgy jobs done dirt cheap to people who didn’t know any better. It’s not necessarily the case that a process aimed at driving down bidding prices and ruthless competition will increase quality, especially in a newly-growing industry where the standards aren’t well understood and the job is being commissioned by non-experts
Compromise and Conceit 
Leave a comment