You are right, oil does not break down, it gets "dirty". The "dirt" in your engine is a combination of the microscopic bits of metal from engine wear, and the contaminants that get into it from "piston blow-by", that is some fuel and air, including moisture and other crap in the air.
If you are in the big city where the air is full of pollutants, or in the country where the air is full of dust, it gets into your engine.
You can probably go 5K on synthetic oil, but I look at oil changes at 3K as cheap insurance. Look at SDPATTs SHO. It has almost 300K on the engine and he changes his oil and filter religously at 3K. That is a strong testiment to regular oil changes.
A clean air filter is also important.
You decide! rangerj