Combine onion, celery, and 2 ½ cups dashi in a small Dutch oven. Bring to a boil over medium-high. Reduce heat to medium-low, and simmer, uncovered, stirring occasionally, until vegetables are tender, 35 to 40 minutes.
Add leftover mashed potatoes to dashi mixture, and whisk until well combined. Stir in milk, if desired, for a creamier texture. Stir in remaining 1 cup dashi, ¼ cup at a time, to thin soup to desired consistency. Cook over medium-low, stirring occasionally, until warm, 4 to 6 minutes. Season with salt and pepper.
Top servings with butter and drizzle with soy sauce. Garnish with mitsuba or celery leaves.
