Preheat oven to 300°F. In a large bowl with a hand mixer on medium speed, cream butter and sugar until smooth, about 5 minutes. Add eggs 1 at a time. Next, add sweet potatoes, evaporated milk, and vanilla.
In a large bowl, mix flour, orange zest, baking powder, baking soda, salt, cinnamon, nutmeg, and ginger.
Add the flour mixture to the cake batter 1 cup at a time. Pour batter into a greased bundt pan, place in the center rack, and bake for 1 hour 40 minutes until a toothpick inserted into the center comes out clean when removed. Remove the cake from the oven and allow the cake to cool for a minimum of 15 minutes before flipping the cake out of the pan.
In a large bowl, add powdered sugar, orange zest, orange juice, salt, and butter and mix until smooth. Pour over the cooled cake.