One point about orbs that a lot of people seemed to miss was that they appeared in other places apart from haunted houses. Indeed, they can appear anywhere where a flash is used with a compact digital camera (they are very rare with DSLR and super-zooms). Away from haunted houses they are just seen as a nuisance. They are probably more common in old dusty houses and castles where a lot of people hunt for ghosts.
Many people now attribute orbs to dust floating close to the camera lens reflecting the flash. It is often said that the reason it affects digital cameras particularly (orbs have been observed with film cameras) is that the flash is so close to the lens. This is only part of the reason. The flash is just as close to the lens on compact 35mm cameras (explaining the tiny number of orbs obtained with those). The important change with digital cameras that caused orbs was, in fact, the big increase in the depth of field of the camera lens.
The depth of field is the distance (in front of the lens) between the nearest point where an object is in focus and the furthest distance. With the increased depth of field, the nearest point in focus was closer to the lens. This also brought the area where objects are just out of focus nearer to the lens too - close enough for the flash to illuminate tiny objects like dust.
Orbs are, indeed, out-of-focus dust. When objects go out of focus (viewed through a lens) they break up into diffuse circular shapes. You can see these sometimes in the out-of-focus bits of photos. The colours and details in some orbs (some people claim to see faces!) are due to aberrations in the lenses of some cameras.
Some people still maintain that at least SOME orbs are paranormal. Some have 'tails', for instance (this is usually falling rain or snow and sometimes another form of aberration). I've yet to see any photos of orbs (and there are a lot on the web) that aren't obviously a tiny object out-of-focus. The funny thing is that, if orbs had been recognised for what they were early on, I doubt very much anyone would be claiming that any were paranormal now.
If you want to avoid getting orbs, use a DSLR or super-zoom digital camera. Both of these have their flash positioned further away from the lens and they usually have reduced depth of field too.