Preheat the oven to 350°F. Line a 9x5 loaf pan with parchment paper leaving enough to overhang the sides of the pan for easier removal of the baked loaf.
Sift together the flour, dry instant pudding mix, baking powder, salt, nutmeg, and allspice in a medium bowl then set aside.
In a large mixing bowl, use an electric mixer on medium speed to cream together the butter, granulated sugar, eggs, eggnog, vanilla, and rum extract until smooth and fully combined.
Add the dry ingredients to the bowl of wet ingredients and mix on low speed just until fully combined. The batter will be slightly thick.
Transfer the batter to the prepared loaf pan and spread to an even layer.
Bake for 55-60 minutes or just until a toothpick inserted into the center comes out clean or with a few moist crumbs. You can tent the bread with aluminum foil after 40-45 minutes of bake time to prevent overbrowning if desired.
Allow the bread to cool in the pan for 10 minutes before lifting out, removing the parchment paper, and transferring to a wire rack to cool completely before adding the glaze.
Whisk together the powdered sugar, rum extract, ground nutmeg, and eggnog until smooth. The glaze will be thick.
Add 1-2 tablespoons milk to thin the glaze to a thin and pourable consistency. Start with 1 tablespoon, then add the second only if needed for desired consistency.
Place a piece of parchment paper under the wire rack (to catch any drips) and spoon the glaze evenly over the top of the eggnog bread.
Allow the bread to sit at room temperature until the glaze has firmed up enough to slice and serve.
