Answer by Alex Kritchevsky:
Of course covariance is real. It says that two variables co-vary – which definitely happens.
Earthquakes are random-ish.
Building collapses are random-ish.
But earthquakes and building collapses often occur at the same time. They're not independent at all. The two random variables (earthquake-today and building-collapse-today) co-vary.
Not the best example for making meaning of the number but I'll try.
Covariance is [math]E[(x-E[x])(y-E[y])][/math], right?
That's "the expected value of the (two variables' distances from their means, multiplied together)".
Covariance has units – in the earthquake example, (Richter-collapses).
Let's define the earthquake variable as X=the highest magnitude of earthquake today (in, say, a city)
and Y = the number of building collapses in that city
There's a distribution of about one to ten for X and probably 0 to ?? for Y, but for both the high values are far less likely.
Imagine the average X is 2.5 for some area and the average Y is like, 0.1 – buildings don't collapse very much. Then imagine that, over a year, you get a nice distribution of things around X=2.5 and Y=0.1 – sometimes they're above their means, sometimes below, so the product [math](x-E[x])(y-E[y])[/math] is sometimes negative and sometimes positive by the same amounts.
If no big earthquakes occur and cause building collapses ever, the covariance is approximately 0 – all those negatives and positives cancel out.
Then, one day, an 8.9 hits and 1000 buildings collapse. That adds 8900*(the probability of that happening) to our expected value. It's a low probability but it's not zero. Maybe it's 0.1
So the covariance, which was 0, becomes 89 (Richter)(collapses). Yes, those are the units. Sorta.
It's not a great example but it shows what that means: earthquakes and collapses are highly covariant.