To make the cookies: Preheat the oven to 350° F. Line two baking sheet pans with parchment paper.

In a large bowl, sift the flour, baking powder, and salt together.

In a stand mixer fitted with the paddle attachment (or a large bowl if using a handheld mixer), combine the granulated sugar and butter. Beat on medium speed until a fluffy, pale-yellow mixture forms, about 2 minutes. Add the egg, almond extract, and vanilla extract. Beat on low until combined, about 30 seconds.

Add ¼ cup of the milk and mix on low until just combined, about 10 seconds. Add a third of the dry ingredients and mix again until just combined, about 10 seconds. Continue alternating between the remaining milk and dry ingredients until everything is combined, ending with dry ingredients. The batter will be very sticky.

Using small (2-tablespoon size) cookie scoop, scoop out 12 balls of dough, rolling each into a ball, and place them on one of the prepared sheet pans, spacing them 2 inches apart. Repeat with the remaining dough.

Bake for 10 to 12 minutes, until the cookies are lightly golden on the bottom and slightly cracked on top. Transfer the cookies to a wire rack to cool completely.

To make the icing: While the cookies bake, combine the powdered sugar, almond extract, and vanilla in a medium bowl. Slowly add the water, 1 tablespoon at a time, and whisk to combine into a slightly runny frosting. If you want to add color, separate the icing into small bowls and whisk one drop of food coloring into each bowl.

To decorate: Place the wire rack with cookies on one of the sheet pans with the parchment paper remaining. Dip each cookie into the icing to coat the top and sides and set them back on the wire rack. Sprinkle immediately with sprinkles (if using).

Let the icing dry completely before eating, sharing, shipping, or storing.