This allusion comes from a story written by Tao Yuan-ming  of the Jin Dynasty (265-420).
The story goes that a fisherman came unexpectedly to the source of a stream, where he discovered a village inhabited by people whose ancestors had fled from the turbulence during the Qin Dynasty (221-206 B. C. ). They were entirely isolated from the outside world, and had no wish for any contact with it. When the fisherman returned home he told the local magistrate and his neighbors about the place. But when they went to look for this idyllic village they could not find it.
Later, a remote and peaceful place came to be called a Land of Peach Blossoms.
Note: Wuling: at the west of the present-day Changde City in Hunan Province.