Bring vinegar, sugar, and ½ cup water to a boil in a small saucepan over medium-high. Remove from heat and stir until sugar is dissolved.
Pack turnips and celery into a 1-qt. jar or place in a large heatproof bowl; pour hot pickling liquid over. Let sit, tossing every 15 minutes or so, until cool, about 1 hour. Seal jar or transfer vegetables and brine to an airtight container (a 1-qt. deli container works well) and chill. Do ahead: Turnips and celery can be pickled 2 weeks ahead. Keep chilled.
