Preheat your oven to 350°F (175°C). Prepare a 9×9 inch light metal baking pan by greasing it and lining it with parchment paper.

In a food processor, combine the Earl Grey tea and lavender, pulsing until finely ground. Use a sieve to remove any larger pieces.

In a medium bowl, whisk together the flour, ground Earl Grey tea, ground lavender, baking powder, baking soda, and salt. Set this mixture aside.

In a large bowl, cream together the softened butter and granulated sugar using an electric mixer on high speed for about 2 minutes.

Add the eggs and vanilla bean paste to the butter mixture, mixing on medium speed until the mixture is pale and smooth, which should take about 1 minute.

Gradually mix in the buttermilk and the dry ingredients, adding them a little at a time while mixing on low speed until everything is well combined and the batter is smooth. Scrape down the sides of the bowl as needed.

Pour the batter into the prepared baking pan and bake for 38-44 minutes, or until a toothpick inserted into the center comes out clean.

Once baked, let the cake cool in the pan on a wire rack for 30 minutes. Then, lift the cake out using the parchment paper and allow it to cool completely on the rack.

While the cake is cooling, prepare the Earl Grey milk soak. Heat the whole milk in a small saucepan over low heat until it is steaming, then add the Earl Grey tea and lavender. Let it steep for 15 minutes before removing it from the heat.

Strain the mixture through a sieve and allow the milk to cool. Once cooled, transfer it to a liquid measuring cup and mix in the sweetened condensed milk and vanilla bean paste. Set this aside for later use.

In a food processor, pulse the lavender until finely ground, then sift it to remove any large bits.

In a large mixing bowl, beat the softened butter with an electric mixer on high speed until it becomes pale and fluffy, which should take about 5 minutes.

Add the cold cream cheese to the butter and mix on high speed until fluffy, about 1 minute.

Sift in the powdered sugar, then add the ground lavender and vanilla bean paste. Mix on low speed until combined, then increase to high speed for 1 minute until fluffy.

If desired, add purple food coloring and mix until evenly incorporated.

Once the cake is completely cooled, carefully cut a thin layer off the top to help the Earl Grey milk soak in.

Transfer the cake to a serving plate and poke holes in the top using a wooden stick or the handle of a wooden spoon.

Slowly pour the Earl Grey milk mixture over the cake, allowing it to absorb into the holes. Be patient as it soaks in.

Frost the top of the cake generously with the lavender cream cheese frosting using an offset spatula.

Cut the cake into 16 slices and serve.