Whisk pudding with milk in a small bowl. Set aside until thickened; about 3 minutes; it's not very long.
Meanwhile beat butter in a large bowl with an electric hand mixer until creamy and smooth. Add thickened pudding and vanilla. Blend until combined.
Add powdered sugar and blend again.
Add additional milk, 1 Tbsp at a time until desired consistency is achieved. (I added about 4 to 6 Tablespoons.) Fill pastry bag and pipe icing onto cupcakes or spread onto cake with a spatula.
