Wood for example will rot over time as the mold feeds on it. This can drastically reduce the structural integrity of the wood which can lead to much greater, and more expensive, problems. Your roof for example can develop leaks over time as wood rots. It can even collapse if its structure is broken down enough.
When it comes to your health, you can never be too careful. The effects of mold on the body can range from mild to severe. Simple headaches and sinus infections to respiratory failure and even cancer have been linked to mold complications. Typically pregnant women, infants, the elderly and those with already weakened immune systems are the most vulnerable. Effects though depend on the type of mold present. This is why it always important to test the mold with something like an air quality test.