I have no idea if this is the way it worked out, but doesn't gaining and losing territory like that mean lots of people dying?
Often it does. But if you're talking about say, the Qing dynasty to present, the Qing dynasty completely and totally collapsed and certain territories were lost in that process (biggest difference between Qing China and now is Mongolia, which declared independence after the Qing dynasty fell, and maintained independence largely due to support from the Soviet Union). So that wasn't particularly bloody in and of itself. The previous largest extent of China was in the Tang dynasty, but the Western areas under control of the Tang were never actually very secure. This was the height of the Silk Road and China was extremely influential in the region but most of Western/Central Asia was not actually a fully administered proper part of the empire. Then, the next biggest extent was the Yuan dynasty, which was the Mongols, and the map there is actually kind of misleading because, you know, the Mongols took over almost the entirety of Eurasia and they were not a "Chinese" empire so to speak. And in fact, the Mongol conquest of China was a lot less destructive than say, the Mongol conquest of Central Asia or the Middle East. Mainly because the Mongols saw China as a major cash cow, as well as an administrative base, and they did not want to completely destroy Chinese cities and infrastructure.
But yeah, conquering, losing, conquering again is generally a bloody process but China, up until around the 15th century or so, was truly (as the students love to remind us) basically the most technologically advanced, urban, prosperous, and educated in the world, and expansion/contraction didn't really hurt Chinese society too severely. Western Europe was absolutely behind China in almost every way until Europe pulled ahead around the time of the Renaissance and the Age of Exploration. All of this expansion and contraction too took place over the course of nearly 2000 years, so individual societies and empires still stayed fairly stable.
(sorry world history teacher coming out)