Mix together sugar and flour and slowly add milk. This should be cooked in the top of a double boiler, but you can cook it over low to medium heat, stirring constantly until it thickens -- do not leave unattended.
Slightly beat egg yolks and temper with a small amount of hot custard; stir well.
Add egg mixture to custard pot and cook 2 more minutes.
Remove from heat and add vanilla and butter. Let cool.
In a 13 X 9 inch casserole dish, alternate pudding, bananas and wafers, beginning with pudding and ending with pudding.
Add a topping if desired. (If adding fresh whipped cream, make sure the pudding has cooled completely)