The enormous temperature difference found between summer and winter can cause a lot of problems in walls and concrete, especially in the central Bathurst where the houses on average are much older.
A much safer bet would be some of the new housing developments (Windradyne '1000') Sunnybright estate etc. Which will be a safer investment per se. These blocks and houses are very new and they're really being built for a projected workforce though, one which probably won't exist. They lure Sydneysiders with their appeal (4 bedroom house on 500 square metres for $550k) but unless you're going to live there yourself, I don't think they're going to be a good investment as Bathurst is a town of increasing houses and decreasing jobs.
Orange would be a safer choice as there are more stable industries present to support the people who need accommodation and who would rent your house (DPI head office, wine, tourism)