Preheat oven to 325°F and grease a 9x5 loaf pan and line with parchment paper
In a small bowl, mix brown sugar, cinnamon, melted butter, olive oil, salt, and flour until a thick paste forms
In a large bowl, mash bananas until mostly smooth
Whisk in granulated sugar, brown sugar, and olive oil until well combined
Add eggs one at a time, mixing well after each addition, then stir in vanilla extract
In a separate bowl, whisk together flour, baking soda, baking powder, cinnamon, nutmeg, and kosher salt
Add dry ingredients to wet mixture and gently fold until combined, slowly pouring in buttermilk
Pour batter into prepared loaf pan
Spoon cinnamon sugar mixture over the batter and gently swirl with a butter knife
Bake for 60-70 minutes until a toothpick inserted comes out mostly clean
Let bread rest for at least 20 minutes, then remove from pan and cool for another 30 minutes
For frosting, beat softened cream cheese and butter until smooth using a hand mixer
Add powdered sugar to taste and mix in vanilla bean paste
Once bread is fully cooled, spread frosting on top, slice, and serve
