AJR, even if your home is an English only zone, you can still keep using English with your child outside the house.
What might happen at some point is that your child will understand you in English but respond in Chinese when outside the house. Sometimes kids feel ebarrassed to be seen or heard speaking a foreign language when they're in their Chinese environment. You might also find that your kid will designate certain places/people as being Chinese and others as being English.
As for comments from others, I don't think I've actually ever heard a snide remark about the kids themselves, but certainly about us as parents. You know Chinese people, especially older women, can be overbearing about childrearing advice even with fellow Chinese people, but when you're a foreigner you're sure to get an earful about how little/much your child is wearing or what he's eating or how he's getting confused with two languages.
The comments that the kids themselves get are never ill intentioned they are just sometimes ignorant. People ask my kids where they come from, call them little laowai, and really fuss over their ability to speak Chinese. My son hates all of this, and though people are saying it endearingly, still, he was born here, he doesn't think of himself as a foreigner. His Chinese language ability is far from impressive considering it is his native language and he's almost 4 and a half. People just don't think and so they say stupid things and you have to teach your kids to handle it. You and your wife will probably be much more sensitive on your kids' behalves than they are themselves at first, but once they start getting old enough to understand, you have to start having talks with them about these issues.