Preheat the oven to 375°F. Line a standard muffin tin with paper liners and set it aside.
In a medium saucepan, combine milk and Earl Grey tea bags and bring to a boil over high heat. Once boiling, remove from the heat, cover, and let sit for 15 minutes. After the 15 minutes are up, strain the Earl Grey-infused milk into a separate container and set aside. Discard the tea bags.
In a stand mixer fitted with a paddle attachment (or a large bowl with a handheld mixer), cream together the unsalted butter and granulated sugar until fluffy, about 5-7 minutes. Once fluffy, add in the eggs one at a time, scraping well after each addition. Next, add in the vanilla extract and scrape well to incorporate. In a separate bowl, whisk together all-purpose flour, baking powder, salt, and Earl Grey tea leaves (for the tea leaves, simply cut open the tea bags and use the leaves inside, discarding the outer bag). Alternate adding the dry ingredients and the Earl Grey-infused milk into the creamed butter mixture in 3 additions, starting and ending with the dry ingredients. Mix until just combined, being careful not to overmix and periodically scraping the bowl to make sure the mixture is homogenous.
Spoon the batter into the prepared liners until they are about ⅔ of the way full.
Bake the cupcakes for 16-20 minutes, or until a toothpick inserted into the middle of a cupcake comes out clean. Remove the cupcakes from the oven and transfer them to a wire rack to cool completely.
In a food processor or spice grinder, pulse together the culinary lavender and the granulated sugar until it becomes a fine powder, then set it aside. If you still have larger pieces of lavender, sift the mixture before using. In a stand mixer fitted with a paddle attachment (or a large bowl with a handheld mixer), cream together the unsalted butter, vanilla extract, and salt until fluffy, about 3-5 minutes. Add in the lavender-sugar mixture and mix until combined. Gradually add in the powdered sugar, scraping the mixture after each addition. Once the powdered sugar is all incorporated, add in the milk and mix until well combined. If desired, add in food coloring to create a purple color for the buttercream (I used a combination of blue and pink food coloring to get this shade).
Once the cupcakes are cool, decorate them with the lavender buttercream as desired and enjoy!
Stored tightly in an airtight container, these cupcakes will stay fresh in the fridge for 3-5 days or up to two months in the freezer.