[Always the Quiet One]

I think we're all overlooking the really obvious reason that some other alien race hasn't already colonized the universe already, they actually did.

1) We are a lost colony of the colonizing species. Perhaps there was a great galactic empire that has since broken down, maybe our ancestors were exiled and given one use ships that dissolved and seeped into the earth as crude oil.

2) This ancient species that colonized the universe isn't one we recognize as alive. Maybe they're pure energy (as has been postulated by someone else on this thread), maybe their inorganic, maybe they simply move too fast to be seen by us, maybe they're called stars. (This is the view I hold, BTW.)

Finally, one thought on the black hole thing. Given the odds of one forming, some bright guy (and I assure you it will be a guy) will develop the coveted BE Ray long before we have to worry about the earth getting smaller instead of bigger.