Category Archives: Lies, Damned Lies, and Statistics

The DoD Loves Complex Charts

As you know, Bob, I’m mildly obsessed with how to display data visually. Displaying data well is tough, especially when you’re talking about complex data. When I have to design a chart for some crazy-ass set of data, I often look at how others have done the same thing, and I keep tabs on blogs that cover chart-making in detail. I also like to collect examples of how to do it badly.

Thankfully, as I’ve mentioned before, the DoD is a great source of bad charts.

A Chart of Afghanistan Stability

Look at that thing! Click on it and behold it in its full splendor! Some of the data is color-coded, but there are multiple secondary labels per color. Those secondary labels are on top of the chart, partially obscuring words and the connecting lines. The light green is unreadable, and the light blue isn’t much better. Most of the nodes in this graph are blocks of text, except when they’re not. I know this is a working draft, and I know the chart’s designers are trying to convey a lot of information, but good grief this is bad.

(via TPM)

A Few More Random Things About Investing

My Two Rules of Investing post resulted in several good questions from people, both in comments and in email. I’ve got answers which are worth every penny you’re paying for.

How much should I be investing? Oh, goodness, there’s a question with no good answer. It depends on what your current financial situation is, how old you are, and all kinds of other situations. My general recommended approach is this. First, pay off any and all credit card debt, and do so as fast as you can. Then build up some three to six months’ worth of living expenses in a decent savings account. Then move to investing, and save what you can.

The important thing to realize is that any money you use to buy stocks and bonds should be money you won’t need for years. In the last year or so, the New York Stock Exchange lost half of its value. If you needed to sell stocks today to cover some unexpected costs, you’d be in a world of hurt.

Should I invest in an IRA? Absolutely. Any money you’re planning on using for retirement and not touching until then should go into an Individual Retirement Account.

They come in two flavors: traditional IRAs and Roth IRAs. In traditional IRAs, you don’t pay taxes on the money you put in them today, but you do pay taxes when you withdraw the money. In Roth IRAs, you pay taxes now, and later withdraw the money tax free. In financial jargon, traditional IRAs are tax-deferred investments (because you pay the tax later instead of now), while Roth IRAs are tax-exempt investments (because you don’t pay taxes on the money later). The rule of thumb I always see is that, for most everyone, Roth IRAs are better than traditional ones. You’re putting money in your IRA hoping that it’ll grow. With a traditional IRA, you have to pay taxes on those profits. With a Roth IRA, you don’t.

If I just want to invest some money without having to worry too much about it, what should I do? If you can, get a lifestyle fund. For instance Vanguard has a range of all-in-one funds. You pick a lifestyle fund based on when you’re going to retire. Vanguard then adjusts the stock, bond, and cash blend automatically, so you don’t have to rebalance anything. Even better, they use index funds, so the expense ratio is around 0.18%.

The only problem is that you’ve got to have $3,000 to get into one of those funds, and that can be tricky. In that case, you can either sock money away in a savings account until you reach that point, or start with a fund that will let you invest as little as $50 to $100 a month.

Two Rules of Investing

Since the economy’s doing so well, let’s talk investing! Everything I know about investing in stocks and bonds can be boiled down to two rules.

1. Buy index funds.
2. Diversify.

To understand index funds, you have to know what an index is: a collection of companies or bonds that someone tracks to get an idea of how the market is doing. Standard and Poor’s 500 Index is one of the best known by virtue of its age. It takes 500 large companies listed on the New York Stock Exchange and combines their stock price and number of shares to create a single number. But there are lots of other indices, such as the Russell 2000 that tracks small companies, or the Wilshire 5000 that tracks almost all of the NYSE stocks. At its simplest, an index fund buys all of the stocks listed in one of these indices and holds them. It’s a form of investing called passive investing, and is exciting as a bowl of dirt. There’s not any real buy-and-sell action, and you can’t beat the market with an index fund. The best you can do is get a return that matches the index.

Why index funds? Why not individual stocks? What about actively-managed mutual funds, where fund managers are buying and selling stocks to try to beat the market’s average return? I gave up on buying individual stocks early because I suck at picking stocks. I don’t have a lot of time to research companies, and even if I did, I’m going up against all of the mutual fund managers and other investors whose full time job is researching companies. Me buying an individual stock is like me betting on the ponies. I’m putting money into a company hoping the company’s stock value goes up.

That leaves actively-managed funds. As one financial advisor explained to me, if you buy an index fund, you’re spending the most on the stocks that are the most expensive. Wouldn’t you be much better off spending more on cheaper stocks so that, if they go up, you get a much greater return?

The problem is that mutual fund managers don’t actually do that very well. Getting the market’s average doesn’t sound exciting, but how many mutual funds meet even that modest goal? It turns out that most fund managers don’t beat their benchmark index, and they have no real ability to time the market. And even if they do beat their benchmark in one year, there’s little evidence that they can continue to do so over the long run.

It gets worse, though. Every mutual fund charges ongoing fees, expressed as the fund’s expense ratio. Every year, a percentage of the fund’s assets go towards paying the fund managers, paying the research team, paying for marketing, and so on. Those annual fees handicap a fund’s returns right out of the gate. For instance, if a fund has a return of 5% one year and its fees are 2%, the real rate of return is 3%.

In 2007 the simple average of stock funds’ annual fees was 1.46%, though individual fees can go much higher. Index funds, on the other hand, have extremely low fees when done properly. For instance, Vanguard’s 500 Index Fund has an expense ratio of 0.15%. That means your average actively-managed large-company fund has to be 1% better than the market average to beat the Vanguard index fund. And the actively-managed funds can’t even beat the average most of the time!

So why not just buy a single stock index and be done with it? Because you may need that money at a time when the stock market is in the tank. Ideally you’d have some investments that perform well when others are doing poorly. What you’re looking for is a set of investments that are uncorrelated, so that the performance of one is unrelated to the other.

That’s what diversification attempts to do. Instead of holding a handful of stocks, or just some bonds, hold different groups of assets. You should at least own a mix of US stocks (both in large and small companies), bonds, international stocks and bonds, and cash. You sacrifice some potential gain (but not much!) in return for greatly reducing how much your portfolio’s value will fluctuate. And the costs of not diversifying can be dramatic.

What’s tricky is deciding how much to put into what investments — what your asset allocation should be. There are various calculators to help you decide, like the Iowa Public Employees Retirement System’s. But once you’ve set your allocation, don’t fiddle with it. Trying to time the market will cost you a lot of money. Instead, do two things. One, invest systematically. A smaller monthly investment is much better than a yearly lump sum or, heaven help you, trying to time the market. Two, once a year you should rebalance your portfolio. Some of your investments will do well (or at least less poorly) than others, so your percentage of money allocated to different assets will change. Sell off the winners and reinvest in the losers to bring your percentages back in line with your goals. It sounds stupid, but what you’re doing is buying low and selling high.

There are myriad tweaks you can make to those two rules, but they’re where I would start. They won’t guarantee that you’ll always make money, or protect you as the global financial markets merrily burn, but they’re better than just about any other current approach.

A Rocket Scientist Explains Securitization

From a Marketplace report on the American Securitization Forum comes the following conversation between Kai Ryssdal and Bob Moon:

RYSSDAL: …How do you restore investor faith in what is admittedly a very important part of the economy?

MOON: Well, they’re promising straight talk, for one thing. There’s agreement here that things did get so confusing, you almost needed to be a rocket scientist to figure out what you were investing in….

RYSSDAL: Do they know, the people you talk to in this [securitization] industry, that part of the problem we’re in is because of the way they did their business?

MOON: Yes, they’re very much aware of that and there was a lot of talk today about ending the rocket science, if you will.

Look, securitization isn’t really that hard. Let me step you through it. Securities are a standard financial instrument that have some commonly-agreed-upon cost. Government bonds are securities. So are stocks. If money is an abstraction, securities are doubly so.

The nice thing about securities is that we have well-established markets in place for trading them around. Think of bonds and the bond market, where governments or companies issue bonds and pay interest, while the bond itself may change hands bunches of times.

Now, say you’re a company who has, I don’t know, a lot of mortgages on the books. They’re producing cash, but you’d really like to move them off of your books, recognize a profit right now, and let someone else deal with them. But who wants a mortgage here and a mortgage there? They come in such inconvenient irregular sizes. So you bundle them all up and sell off regularized shares of the bundle. You’re averaging out the risk that way, and no one has to buy a whole mortgage if they don’t want to.

But what are these things worth? Hang on, I’m going to have to pull out some math, namely, the idealized rocket equation.

The ideal rocket equation

This equation tells how much the value, v, of the securitized asset can change by. The change in value, Δv, depends on the estimated value ve, times the natural log of the ratio of the market’s total value right now m0, to its value when the security was first put together m1.

“But, Stephen,” you ask, “estimated by whom?” Why, by rocket scientists, silly! Because when you want to know what something’s worth, ask people who supposedly only buy parts from the lowest bidder yet managed to make the International Space Station cost somewhere around 10 times what we originally said it would — not, mind you, that we can pin down the cost any more precisely than “somewhere between $35 billion and $100 billion”.

I trust the confusion over the actual cost of things like securitized mortgages is making more sense now.

Ideally the value of these mortgage-backed securities would have kept going up and up and up. Unfortunately, they didn’t. Instead, they crashed dramatically. That’s because they didn’t achieve their escape value, defined as the price at which people say, “Well, shit, that thing’s worth so much that it’ll never cost less than it does now!” Mathematically, that value is

The equation for escape velocity of a rocket.

where ve is the escape value of the security (not to be confused with the estimated value ve above), M is the mass of the market as measured in Denmarkian GDPs, r is the radius of the building in which the securities are traded, and G is the Greenspan constant.

All of which is most easily summed up by

Any questions?

Historical Perspective on the Jan 2009 US Job Losses

(This is an update to my November post about US job losses. For more information about where I got my data, please see that post.)

The US job market is definitely still sliding. Back in December I crunched some numbers to see how bad the job losses were as a percentage of the total number of jobs. Here’s what it looks like now that we have two more months of bad news.

Historical Job Losses

Here are the job gains and losses since 1960.

Monthly change in the number of jobs as a function of time from 1960 to 2006

We’ve now had three months whose drop is nearly that of the single largest drop in December 1974, as is shown on this table of the top ten greatest monthly losses.

Year Month Difference
1974 Dec -602,000
2009 Jan -598,000
2008 Nov -597,000
2008 Dec -577,000
1980 May -431,000
1970 Oct -430,000
2008 Oct -380,000
1975 Feb -378,000
1974 Nov -368,000
1975 Jan -360,000

September 2008 isn’t on the list after its preliminary numbers were revised. Now October 2008 is on it instead, and November 2008’s non-preliminary numbers are worse than before.

Now let’s look at job loss as a percentage of the jobs that could be lost. That is, take the number of jobs lost in a given month and divide that number by the jobs you started out with in the previous month.

Percent change in number of US jobs from 1960 to Jan 2009

As before, the shape of the graph is about the same, but the fluctuations are now larger earlier in time. Here’s the top ten worst months by percentage of jobs lost.

Year Month Difference % Diff
1974 Dec -602,000 -0.77%
1960 May -340,000 -0.62%
1970 Oct -430,000 -0.61%
1975 Feb -378,000 -0.49%
1980 May -431,000 -0.47%
1974 Nov -368,000 -0.47%
1975 Jan -360,000 -0.46%
2009 Jan -598,000 -0.44%
2008 Nov -597,000 -0.44%
2008 Dec -577,000 -0.43%

The bottom of our list is filling up with recent months.

These are still all one-month drops, and the data shows a lot of single-month drops that are surrounded by gains. How does 2008-2009 compare to other runs of job losses? Between 1960 and now there have been seven times that we’ve lost jobs for four or more months in a row.

Period Months Difference
Jan 2008 – Jan 2009 13 -3,498,000
Aug 1981 – Dec 1982 17 -2,838,000
Mar 2001 – May 2002 15 -2,202,000
Nov 1974 – Apr 1975 6 -2,164,000
Jul 1990 – May 1991 11 -1,621,000
May 1960 – Feb 1961 10 -1,256,000
Apr 1980 – Jul 1980 4 -1,159,000

What a difference two months makes. Through November, we hadn’t lost as many jobs since January 2008 as we had from November 1974 to April 1975. Now we’ve lost far more than that. On sheer number of jobs alone, we’re in our worst run since 1960. What about job losses as a percentage of the number of jobs that existed before the losses?

Period Months Difference % Diff
Aug 1981 – Dec 1982 17 -2,838,000 -3.10%
Nov 1974 – Apr 1975 6 -2,164,000 -2.75%
Jan 2008 – Jan 2009 13 -3,498,000 -2.53%
May 1960 – Feb 1961 10 -1,256,000 -2.29%
Mar 2001 – May 2002 15 -2,202,000 -1.66%
Jul 1990 – May 1991 11 -1,621,000 -1.48%
Apr 1980 – Jul 1980 4 -1,159,000 -1.27%

I’d still rate the 1974-1975 recession worse, based on this list and the fact that the job losses there were sustained over a mere 6 months. But the situation’s bad, and not getting any better. Two months ago I said, “While we’ve experienced worse recessions, given another four or five months of this and we could move into the top spot across the board.” It may not take four months at the rate we’re going.

Historical Perspective on the November 2008 US Job Losses

Note: I’ve updated this article with more current information.

Well, that’s not good. The Department of Labor announced that there are 533,000 fewer US jobs in November than in October, which is the largest one-month drop since December of 1974, when the job numbers dropped by some 602,000.

The BusinessWeek article offers this perspective:

How bad are these numbers? Worse than in the 1990-91 recession, whose worst month saw 306,000 lost jobs, or the 2001 recession, whose worst month was a loss of 325,000 jobs. The U.S. economy lost 431,000 jobs in May 1980, which was the worst month of the back-to-back recessions of 1980-82. If it’s any comfort, though, November’s showing was better than the recession month of December 1974, when the economy lost a staggering 602,000 jobs, according to the Bureau of Labor Statistics.

I don’t think that’s true. We’re not as bad off yet as during the 2001 recession, and we’re on par with the 1990-91 recession. Sure, this is a terrible month, but how bad have the last few months been compared to other similar runs of job losses? What if you look at job losses as a percentage of the total number of jobs? To answer those questions, I downloaded seasonally-adjusted non-farm payroll employment data from the Department of Labor’s Bureau of Labor Statistics, which is where the 533,000 job loss number came from, and did some number crunching. Feel free to check my work as I go.

One other thing, though: statistics are cold comfort to those who have lost their jobs. I’m going to argue that, percentage-wise, this run of job losses isn’t that bad when compared historically. And yet, 533,000 is a lot of people. It’s equivalent to the entire town where I was born losing their jobs 50 times over. It’s everyone who lives in the Huntsville and Decatur, AL area, including me. Don’t lose sight of the people behind the numbers.

Historical Job Losses

What do job gains and losses from 1960 until now look like?

Monthly change in the number of jobs as a function of time from 1960 to 2006

The graph shows that November’s drop is the largest next to December 1974, as well as our continuing slide this year. May 1980 and October 1970 come next in the one-month-loss list. (But September 1983 kicked ass, adding over a million new jobs.) Now let’s look at the top ten greatest monthly losses.

Year Month Difference
1974 Dec -602,000
2008 Nov -533,000
1980 May -431,000
1970 Oct -430,000
2008 Sep -403,000
1975 Feb -378,000
1974 Nov -368,000
1975 Jan -360,000
1982 Jul -343,000
1960 May -340,000

November was bad, but so was September — it comes in at #5 on this top-ten list.

But what happens if you look at job loss as a percentage of the jobs that could be lost? That is, if you take the number of jobs lost in a given month and divide that number by the jobs you started out with in the previous month, does that change things?

Percent change in number of US jobs from 1960 to 2008

The shape of the graph is about the same, but the fluctuations are now larger earlier in time. The US economy has grown tremendously since 1960. The number of jobs went from about 54 million in January 1960 to 136 million in November 2008. Think of it as job inflation. Any single job doesn’t make as much of a difference to the total now as it did in 1960.

Our top ten list changes noticeably.

Year Month Difference % Diff
1974 Dec -602,000 -0.77%
1960 May -340,000 -0.62%
1970 Oct -430,000 -0.61%
1975 Feb -378,000 -0.49%
1980 May -431,000 -0.47%
1974 Nov -368,000 -0.47%
1975 Jan -360,000 -0.46%
1960 Dec -219,000 -0.41%
2008 Nov -533,000 -0.39%
1982 Jul -343,000 -0.38%

All of a sudden, November 2008 drops to 9th place, and September 2008 is gone from the list. December 1960 has taken its place.

But these are still all one-month drops, and the data shows a lot of single-month drops that are surrounded by gains. How does 2008 compare to other runs of job losses? Between 1960 and now there have been seven times that we’ve lost jobs for four or more months in a row.

Period Months Difference
Aug 1981 – Dec 1982 17 -2,838,000
Mar 2001 – May 2002 15 -2,202,000
Nov 1974 – Apr 1975 6 -2,164,000
Jan 2008 – Nov 2008 11 -1,911,000
Jul 1990 – May 1991 11 -1,621,000
May 1960 – Feb 1961 10 -1,256,000
Apr 1980 – Jul 1980 4 -1,159,000

The current run of job losses started back in January 2008. We haven’t yet lost as many jobs as we did from November 1974 to April 1975, and those job losses happened in half the time as our current ones. The August 1981 to December 1982 run holds the top spot. Based on this list, we’re barely worse off than the 1990-91 recession, and much better off than the 2001 recession. What about job losses as a percentage of the number of jobs that existed before the losses?

Period Months Difference % Diff
Aug 1981 – Dec 1982 17 -2,838,000 -3.10%
Nov 1974 – Apr 1975 6 -2,164,000 -2.75%
May 1960 – Feb 1961 10 -1,256,000 -2.29%
Mar 2001 – May 2002 15 -2,202,000 -1.66%
Jul 1990 – May 1991 11 -1,621,000 -1.48%
Jan 2008 – Nov 2008 11 -1,911,000 -1.38%
Apr 1980 – Jul 1980 4 -1,159,000 -1.27%

Percentage-wise, our current run doesn’t look that bad. The early 1980s recession was harsh, and these numbers bear out how bad it was. The 1974-1975 recession was also bad, again as is shown by the percent of jobs lost. But even the 1990-91 recession was worse.

So, yeah, November’s one-month drop worries me, but it’s not as bad as other historical losses. Of course, the US economy is still headed down. While we’ve experienced worse recessions, given another four or five months of this and we could move into the top spot across the board.

Where I Got My Data

All of my data are from the US Bureau of Labor Statistics, specifically their data page. Under their “Employment, Hours, and Earnings – National” database I searched for “All Employees, Thousands” under the “Total Nonfarm” super sector. All data were seasonally adjusted. You can get a graph like my first one directly from the Bureau of Labor Statistics.