Preheat the oven to 340°F (170°C). Line a 12-cup muffin tin with muffin liners.
Mix 1 tbsp lemon juice into 1 cup whole milk at room temperature and stir well. Let it sit for a few minutes until it slightly curdles, like buttermilk. Set aside.
Add 3 cups all-purpose flour and 1½ tsp baking powder into a fine-mesh strainer set inside a large bowl. Sift to aerate. Then, add 6 Tbsp matcha into the same strainer and sift.
Mix the dry ingredients with a hand whisk until evenly combined. Set aside.
Add 14 Tbsp unsalted butter at room temperature to the bowl of a stand mixer fitted with a whisk attachment. Beat on high speed until light and fluffy. Reduce to medium and add 1⅓ cups sugar in three portions, beating well after each addition.
After you beat in the final portion, scrape down the sides of the bowl with a spatula. Continue to beat well until creamy and pale.
Crack 4 large eggs into a separate bowl and beat well with a hand whisk.
Add the eggs to the butter mixture one tablespoon at a time, mixing after each addition on low just until the egg is incorporated and disappears. Do not overmix.
Add one-third of the dry mixture and one-third of the milk into the butter-egg mixture. Stir on low until just combined. Do not overmix.
Repeat with a second addition of dry mixture and milk, stirring until just combined. Remove the mixing bowl from the stand mixer.
Add the final addition and gently fold with a rubber spatula without deflating the air in the batter. Do not overmix.
Spoon the batter evenly into the prepared muffin liners.
Tap the muffin pan on the counter from about 3 inches high to release any large air bubbles.
Bake in a preheated 340°F (170°C) oven for 30–35 minutes, or until a toothpick inserted in the center comes out with a few moist crumbs.
Take out the pan and let the muffins rest in the tin for 5 minutes. Remove the muffins to a wire rack and cool completely.
Add ¾ cup confectioners’ sugar, 1 Tbsp milk, and 2 drops pure vanilla extract to a medium bowl. Mix well. Drizzle the icing with a fork or pipe it onto the muffins.
