It is an inner city suburb with lots of apartments and professional workers, not so much a children’s area. It depends whether you like that sort of living. I would suggest Mosman, Cremorne or Neutral Bay as more child friendly. However, Milsons Point is very convenient as has lots of restaurants, train to city on your doorstep, ferry stop down the road, Luna Park fun park, great swimming pool and walks. If you’re not staying permanently Milsons Point would be great.