After many visits to HK over the years to find things that were either unavailable in GZ or simply easier for me to find, I eventually got a bit burned out on the extremely crowded areas of HK Island and Kowloon. The outer areas are nice and there is good hiking in many spots, as well as beach camping in places like Sai Kung.
I agree with your attitude about how to respond to the touts, which is generally what I do. After I got to know some Indians in GZ and actually thought about life in a society with a caste system, I suddenly thought a little smile was not such a great effort walking around HK.