Thursday, February 10, 2022

A Nation At The Crossroads: A Cost Benefit Analysis of Blaring Train Horns



Co-written/researched w/ @SJ


For the past 3 years, your co-authors lived within a few blocks of a railroad crossing. Around five times a day (and usually at least once in the middle of the night), a train would blast through, blaring its horn for a long 30 second approach. We were probably 400 meters from the crossing, and we couldn’t leave the windows open at night or we’d be woken up. Most confusingly, that crossing had signals with the arms that come down across the road whenever a train comes, so no one was going to be driving across the tracks. Who is this horn for? How many people are at the crossing when they roll through at 3 AM? Surely blaring lights and descending gates should be enough for them to not be an idiot!


The noise really gets under your skin after a while, and we were bothered by the seeming uselessness of this constant annoyance, so we’ve gathered you here today for a little cost-benefit analysis on the evil that is the train horn. It sounds pretty trivial, but disrupted sleep, stress, and general annoyances aren’t! Multiply the suffering over 100,000 train crossings in the US and even a small cost at each one is going to add up. We couldn’t find any serious analysis* pitting the costs of loud train horns against their benefits (presumably a reduction in collisions/deaths), so we’re going to try our hand at one.


Let’s start with the basics. How easy would it be to just… not honk? Lots of people hate the honking. Does anyone think it’s necessary and do they really have to do this?


Well, yes. 


*Though at the last minute we did find a 139 page undergraduate thesis that looks at the pros and cons of creating “quiet zones” (see next section) in Richland County, South Carolina



Train Honk Laws: Past and Present


In the US, trains are required to sound their horns for “at least 15 seconds, and no more than 20 seconds” between 96 and 110 decibels before they hit a public crossing. To give you a sense of how loud that is, 100 decibels is about the level of a jet taking off or a jackhammer. 110 decibels is apparently the average human pain threshold, so we’re juuust skirting the point where most people would be in pain every time a train goes past. But only for “no more than 20 seconds.” Awesome. 




Are these regulations popular with anyone? Some train companies like Union Pacific are totally on board (pun alert). Is this because they are concerned about the safety of their trains and workers, because they don’t want to lose the branding of a lovely train whistle off in the distance… or because train executives are a sadistic bunch who enjoy terrorizing neighborhoods?


It’s a toss up. 


But 20-second, pain-inducing horns tearing through the neighborhood aren’t the only way to ensure safety at train crossings. One alternative is to put in place “wayside horns”, where the horn sounds from the crossing rather than the train (see below). These can really limit the dispersion of noise, but cost somewhere between $100k-500k and don’t completely eliminate the noise problem.



Another option is to put in the types of safety measures that allow you to become a “designated quiet zone”. At these crossings, trains are required not to sound their horns, except in an emergency. To qualify a crossing as a quiet zone, you need to install flashing lights and bells and arms that come down across the road, as well as score below a certain score for the “riskiness of your crossing”. Creating a quiet zone typically costs around $1-3 million.


But the horn laws were introduced for a reason. We used to have local whistle bans in the US, but then the Federal Railroad Administration (FRA) noticed collisions increased 135% in Florida when nighttime whistle bans were put in place in 1985 (when compared to a control group of whistle-on crossings). This sounds huge, but it amounted to around 10 additional collisions, 5 injuries and 2 deaths per year in Dade County, which houses around 1% of the US population.


A 2003 nationwide study from the FRA found that a lack of train horn increased the risk of a collision (day and night) by 43% when the crossing already had flashing lights and gates. Kind of surprising, you’d think a gate coming down across the road would be enough. For “passive” warning crossings where a sign is the only warning device, they estimated a horn ban increases collisions by 72%.


Then the government said “this is why we can’t have nice things like peace and quiet at night” and made those frickin train horns required by default. 


Did they get it right? All of those studies were solely concerned with accident rates and improving them. But it’s hard to compare accident rates with “I am sometimes very annoyed” times X million people. So let’s get some dollar signs on those costs to health and safety.




Collision Data


Caption: data from https://oli.org/track-statistics/collisions-casualties-year



According to the FRA (see graphs above), we’ve had around 2,000 collisions per year at railroad crossings over the past decade. And it used to be a lot worse. We’ve gradually seen collisions decline to about 25% of the level seen in the 1980s. Of the 2,000-ish yearly collisions, there are around 250 deaths and 1,000 injuries. There are an additional 200-300 yearly railroad collision deaths due to a combination of suicide and “trespassing” (people on the tracks away from a crossing). While these are tragic, train horn laws aren’t really a factor and we can safely ignore these deaths in our analysis.


Are there particular types of crossings where these incidents tend to occur? All public crossings are required to have some kind of warning, but these can be either passive (a sign that says “RR” or something to that effect) or active (flashing lights, bars that go down across the road when a train is approaching the crossing). You would think the passive crossings would be more likely to have bad accidents, where a car is on the tracks and doesn’t know a train is coming. 


Nope. According to FRA data, 68% of fatal collisions on public crossings in the US happen at gated crossings. What’s happening here? Are the active warning devices failing? Are they going off too late? Are drivers getting stuck on the tracks because they try to sneak through as the last car? Are they blasting through the gates that come down? That doesn’t seem likely.  


Wrong again. In the majority of these cases, the listed cause of the collision was “went around the gate”. So they ARE just going through. Jesus.


That crosses off the easy answer that horns are definitely overkill (sorry) at active crossings. So how much of a difference do horns actually make?



TOOT TOOT, LOOK OUT: The Lives Saved by Train Horns


There are 4 key questions here:


  1. How many deaths and collisions are prevented by horns?


We couldn’t find any studies with a strong causal framework that looked at the effect of train horns on collisions, but the two studies mentioned above came close. Less conclusively, we can also look at aggregate trends for US collisions and fatalities to see if there are any noticeable discontinuities that match the Train Horn Rule timing. 


We’ll start with the US-level trends. Since the Train Horn Rule came into effect in 2005, fatalities have fallen around 40%. Collisions have fallen around 27%. Naively, we might expect banning train horns to reverse those trends and take us right back to 2005 levels. But collisions and fatalities were both declining long before that policy went into effect (as seen in the previous section). In fact, if deaths continued falling at the same constant rate after 2005 as they did pre-2005, we would expect a 33% drop in fatalities from from 2005 to 2015, absent any new policies. If you looked at the graph without knowing the rule came into effect in 2005, you wouldn’t guess that anything had changed.


And there are additional caveats. Looking at the full decade following the policy change seems questionable. Maybe it takes a year or two for everything to be implemented successfully, but the effects of the new policy shouldn’t gradually increase over the span of 10 years. We also don’t know how many crossings had night-time/all-time whistle bans before the rule, which means we have no idea how much the new rule really changed the situation on the ground.


The papers mentioned in the “Train Honk Laws” section provide better estimates about a horn’s effect on collisions and fatalities. The 1995 Florida paper found night-time whistle bans increased collisions by 135% (hereby referred to as our "Florida estimate"). The 2003 nationwide study found 24-hour bans increased collisions by 43% at gated crossings and by 72% at ungated crossings.


We might want to make a couple adjustments to our “Florida estimate” before we apply it to day and night, and all crossings across the country. First, the 1980s were a long time ago (sorry Boomers), and it’s not obvious that a 2022 whistle ban would lead to the same outcomes. We suspect modern-day crossings have more safety measures in place than they did in the 1980s, but we aren’t confident here. On the other hand, it’s likely drivers are less attentive nowadays thanks to smartphones (though drunk-driving has been declining), which could make a horn more important. Let’s just call all this a wash, for the sake of simplicity.


Second, the paper only addresses night-time whistle bans, and it’s unclear if horns would have a bigger impact during the day or at night. Far fewer people are crossing at night, but they might be more likely to be intoxicated or inattentive. Again, kind of a wash. We’ll just play it safe and assume that banning day-time whistles would have a slightly higher impact on fatalities than night-time whistles. Let’s say our adjusted Florida estimate is that a nationwide 24 hour horn ban would lead to a 150% increase in 24-hour collisions and fatalities.  


The nationwide FRA study has a couple advantages over the Florida one. It looks across the country, which means the sample size is much larger and more representative. It’s also looking at a 24-hour whistle ban, which better matches what we’re exploring here. On the downside, it’s an observational study that doesn’t attempt to find a comparison group of crossings that “look similar” to the whistle ban crossings (it just uses all non-whistle ban crossings in its control group). And the data is over 20 years old. But considering whistle bans were outlawed in 2005, we’re not gonna do better than what we’ve got here!


To get our “nationwide estimate” from this study, we can combine the estimates for gated and ungated crossings using a weighted average. Earlier we mentioned 68% of fatal collisions are at gated crossings. If we use that as a weight, we get a 52% increase (math) in 24-hour collisions and fatalities due to whistle bans. We’ll call it 50% to keep things round.


Let’s take the Florida estimate as a rough upper bound/80ish% confidence interval on what we’d expect to see, and the nationwide estimate as a rough lower bound. That gives us a 50%-150% increase in fatalities attributed to complete whistle bans. 


  1. What value should we place on a life saved? 



According to the US government: $9.6 million. 


The value of a statistical life is a pretty abstract technical concept, and does not literally mean that the government would pay exactly $9.6 million to save your life (this would make the negotiations of hostage situations a lot less exciting). It’s meant to help in situations like these where we have to trade off risk to human life with the costs of safety measures. You can think of it as an extrapolation of the answer to “how much would you pay to reduce your risk of death by 1%?”  


Side note - the value of a statistical life varies wildly between countries. Move to Gujarat and become a male blue collar worker in manufacturing, your life drops to about $640,000. Move to Turkey and you’re worth $59,000. Cold. 


  1. What are the non-fatality costs from collisions (damage to cars/trains/rails and healthcare costs from injuries)?


According to the 2015 report titled “The Comprehensive Costs of Highway-Rail At-Grade Crossings Crashes”, damage to rails, trains, and cars sum up to around $80k per collision. We couldn’t find good information on healthcare costs, but it seems like we can safely put them at $20k per collision, which gives us around $100k in “costs per non-fatal collision”.


  1. Add it all up


So where does that leave us? With a complete whistle ban, we expect collisions and fatalities to increase between 50% and 150%:

  • Lives saved: 125-375 x $10M each = $1.25B to $3.75B

  • Collisions averted: 1,000-3,000 x $100K = $100M to $300M

  • Total benefits of horn honking are somewhere between $1.35 billion and $4.05 billion.



CHOO CHOO, COMIN THROUGH: THE LIVES RUINED BY HORNS



BUT AT WHAT COST? Lots of people live by these railroad crossings and have to deal with a train rolling through several times a day. The hit to concentration, sleep, and the pleasantness of living in a certain area may seem minimal on an individual level, but if you multiply it by all the people who suffer those moderate inconveniences, are we sure it’s worth it? 


First we need to figure out how many people have to deal with railroad crossings like ours. There are about 50,000 people in St. Louis Park, Minnesota, where we used to live when we were by the tracks. We couldn’t find the exact number of crossings in the city, but we have seen at least 5. “Our” crossing had roughly 1,000 people living within a half mile. The other crossings were less densely populated, with something like 500 people living within a half mile. Put all that together and you get around 5% of SLP residents living close to a train crossing. Maybe a lower percent of rural people live near a crossing than those in SLP, and maybe a higher percent of urban people do. Five percent (16.5 million people) seems like a reasonable nationwide estimate of people living near a train crossing. To be conservative, we’ll set our lower estimate of the nationwide number to 4% of Americans or 13 million people.


Another approach: Let’s say an average of 500 people live near each “gated” crossing (using the gate as a proxy for a heavily populated area). There are 128,000 train crossings in the US and about a third of these have lights and gates. That would give us 21 million people living near tracks, or 6% of the population. Add 50 people per non-gated crossing and you get another 4 million people, leaving us with 25 million total (7.5% of the US) people living near crossings, which we’ll use as our upper estimate.


We’ve got 13 to 25 million people quietly suffering. So, let’s take a look at what we’re doing to these poor people. 


1. Health


First are the health costs - in sleep, in stress, and in not being able to hear the doctor during your telehealth appointment. When a train goes by at 3 AM, it is definitely waking people up. Disrupted sleep is linked to all sorts of health issues - suppressed immune system, mental health issues, metabolic regulation, risk of stroke, cardiovascular issues, etc. etc. etc. Not least among these is stress. And the stress doesn’t just come at night - turns out loud noises during the day can also be quite stressful. Stress is linked to many of the same health issues as sleep disruption, and none of them are good. Both can take years off your life.



2. Productivity


Loud noises are generally not conducive to doing good work, so costs to productivity are another consideration here. Most of the studies we could find about noise and productivity were either criticisms of the new trend in open floor plan offices or related to OSHA guidelines about long term loud noise and hearing loss. Neither of these are very relevant to the momentary interruption of trains. But we still think it’s pretty clear that (1) a loud train going by is distracting (2) distractions hurt productivity, whatever the distractions may be.



3. QoL


Lastly, and hardest to capture, are general quality of life concerns beyond drops in health and productivity. Stress and sleep loss lead to feeling crappy and making your living mates also feel crappy (one of us can attest that their co-author gets quite cranky when they haven’t had a full night of sleep). Loud noises put people on edge. Unsurprisingly, many people report worse moods and less relaxation when exposed to loud noises. For some people, in particular, autists and kids with special needs, it goes beyond annoyance, and loud noises can make daily life excruciating. 



4. Property




Lastly, a major cost to honking is a reduction in property values. The three issues above bleed heavily into property values (i.e. people will pay less to live somewhere that is bad for them). We weren’t sure exactly how we would subtract out the costs above from property value costs to avoid double counting, but ultimately decided that property values encompass enough that we can solely focus on them without missing too much. 


There seems to be lots of precedent for this approach in other property value-related research. For example:


“The examination of the impact of crime on rents and property values undertaken here can, in principle, capture all of these costs. The housing market is essentially an implicit market on which people reveal the costs of crime as they themselves perceive them. Consequently, it is thereby possible to obviate the problems inherent in an outside observer attempting to assign costs independently of the behavior of the individuals.”


There are a few papers on the impact to property values from being near (non-commuter) train tracks. But property values are lower for reasons other than honking, such as air pollution and the noise from the train itself, so we need to be careful to only include the honking portion of property costs into our calculation. 


The first paper we looked at found a ~6% reduction in house values that are within 750 feet of a railroad track using 1996-1999 data from Cuyahoga County (Ohio). This gives us a rough idea of possible property costs, but there are a couple things that keep this paper from being too useful. First, there was only a reduction in value for houses under 1,250 sq ft, a counterintuitive result that really threw the data/analysis into question. Second, this analysis focused on distance to railroad tracks, not specifically to railroad crossings, which makes it hard to apply to our question of train horn noise. 


Luckily we have a 2006 paper looking at almost exactly what we want. “The economic valuation of train horn noise: A US case study” finds that in a Pennsylvanian small town there was roughly a 4% reduction in property value for every additional 10 decibels audible at the house. They estimate a $4.1 million ($6.1 million in today’s dollars) decrease in property values overall for Wormleysburg homes. The paper is short on details, but it appears that around 15% of Wormleysburg houses are affected by train noise. This suggests an average decrease in property value per person affected of $13,600. The average house in Wormleysburg is worth half of the national average house, so we’ll double this number ($27,200) to get an estimated nationwide impact. 


Taking our upper and lower bound estimates on the percent of people/houses/properties that are near train crossings and applying the average property value reduction gives us estimates between $360-$670 billion. These numbers probably sound massive, but the total housing value in the US is around $40 trillion, so these numbers imply a 1-1.5% drop in total value.


A third paper, from 2016, provides us with a (somewhat) larger and more representative sample. “Silence is Golden: Railroad Noise Pollution and Property Values'' estimates that being in a 65+ db railroad noise area lowers property values by 14-18%. The paper looks at 772 houses across 6 train crossings in the greater Memphis area. A 65+ db zone seems to be broadly consistent with how we estimated the number of people living near crossings. Images in the paper suggest this zone is roughly 8 blocks by 2 blocks at each crossing. 


If we apply a 16% reduction to the estimated 4-7.5% of households living near a crossing (4% of total housing value is $1.6 trillion, 7.5% is $3 trillion), we get a total reduction in property values of $256-$480 billion


We were very satisfied to see agreement between our “Memphis” and “Wormsleyburg” ranges! 


We decided to use the Memphis estimates since they seem more robust, and they let us air on the more conservative side. Still, we wanted to widen the ranges for our upper and lower bounds to account for the other major area of uncertainty. In addition to the percent of people living near crossings, there is uncertainty over the “true” property value effect. Even though the Memphis study seems “better”, it’s still a small sample and the methods seem somewhat sketchy to us. There’s no obvious way to get a lower and upper confidence interval on these estimates, so we have to rely a bit on our intuition. It’s hard to imagine an effect of more than 25%, so let’s use that as the upper end of our confidence interval. For a lower bound, we’ll cut the 16% in half and call it 8%. 


Taking our 8% reduction and applying it to 4% of houses gives a lower bound estimate of $128 billion. Taking our 25% reduction and applying it to 7.5% of houses gives an upper bound estimate of $750 billion.


If we had more energy to keep going, the 100+ page undergrad thesis we mentioned above, titled “Investing in Quiet: A Self-Funding Approach to Mitigating Noise Emissions from Train Horn Use at Grade Crossings” looks packed with information and more readable than most of the PROFESSIONAL readings we subjected ourselves to. From our quick perusal of the paper, they find adding quiet zones in Richland County, South Carolina would increase property values by $221 million. 1 in 795 Americans live in Richland County and the average property value nationwide is 42% higher than it is in Richland County. When we multiply this estimate ($221 million) by 795 and add 42% we get ~$250 billion dollars, which is well inside our bounds. Another point for robustness! 


So we’ve got our estimated total costs of train honking. How do we convert that to a yearly value though? Rents seem like the standard way of doing something like this, since these measure yearly income (aka the flow) you can get from an asset (aka the stock). The price-to-rent ratio (value of house divided by cost to rent it for a year) in the US is currently around 18, which means we can convert our property values to yearly rents by dividing them by 18. With that out of the way, our lower bound adds up to a total yearly cost of the current train horn situation of $7 billion and our upper bound adds up to a whopping $42 billion.




So… we ban them, right?





Let’s review:

  • Yearly benefits of train horns (in lives saved and collisions averted): $1.5-$4 billion

  • Yearly costs of train horns (in stress, disruption, and property values): $7-$42 billion


Sure seems like a slam dunk. Our confidence intervals don’t even overlap.


But.


There are a lot of things we remain uncertain about and would appreciate any feedback. 


On the cost side, many uncertainties could change our bounds: 

  • Do our estimates on the number of people living near crossings make sense?

  • Is it reasonable to expect rent/property values to be ~8-25% lower for those people?

  • Are property values the best approach to capturing costs?

  • Can we divide property values by value to rent ratio to get annual costs?

  • How should we think about apartments and offices/other real estate? We didn’t really address this, and made the implicit assumption that apartments would follow the same pattern as houses for property values.


On the benefits side, the uncertainties have less impact on our final estimates:

  • Collisions could be twice as costly as we estimated, but they still wouldn’t meaningfully change our estimates.


Uncertainties aside, our numbers strongly suggest the FRA should remove the ban on whistle bans and either prohibit train horns in non-emergency situations or let local jurisdictions decide what they think is appropriate. If the FRA refuses to move, local governments should consider paying up to turn crossings into quiet zones. Shelling out $1+ million per crossing is a steep price, but if it boosts nearby property values by 15%, it also boosts property taxes, and pretty much pays for itself over the course of a few years. Give it a shot!


We’ll conclude with a few random questions we had:

  • How much are cost-benefit analyses actually used in the process of making public policy? The two FRA studies only considered fatalities and collisions. The FRA didn’t commission any studies measuring how horns affected nearby residents. Maybe because it feels dumb to compare lives saved to the annoyance of honking, but we think this is wrong! 



  • How would these estimates change if we only allowed honking at the highest risk crossings?

  • What’s the deal in other countries? From a few short articles we could find, it sounds like Europe is largely horn free in residential areas - are those areas death traps?

  • If we had a national vote about train horn laws, what would the result be? From talking to a few pretty normal people about this idea, they were quite skeptical of changing any law that saves lives. If the law failed to pass, would that be evidence that the benefits outweigh the costs? No, but that’s a question for another day.



5 comments:

  1. > What’s the deal in other countries? From a few short articles we could find, it sounds like Europe is largely horn free in residential areas - are those areas death traps?

    In Poland train sound horns on crossing without barriers or in case of noticing someone on tracks.

    I heard about cases of local government lobbying/paying (not sure which one) for level crossing upgrade to get rid of horns.

    I live next to tracks with level crossing nearby, with more than 100 trains passing daily. With such laws it would be a fucking nightmare.

    ReplyDelete
    Replies
    1. And as I have not actually answered: yes, there are affected areas but mostly more remote ones.

      There is ongoing process of eliminating or upgrading such unprotected crossings. Also protected crossings on larger roads are getting eliminated as waiting for trains to pass can be drastic influence on traffic jams.

      Delete
    2. I looked at just passenger trains - and I was wrong. It is over 200 passenger trains alone each days.

      There are also cargo train (many during night), technical transports, like locos moving standalone, track maintenance.

      16 trains between 17:00 and 18:00 alone will arrive at the main station a bit further along tracks.

      Delete
  2. > How much are cost-benefit analyses actually used in the process of making public policy?

    Approximately none?

    ReplyDelete