Combine 2 cups of milk and the brown sugar in a small, heavy-bottomed saucepan. Set it over low heat, and stir to dissolve the sugar.
Let the mixture come to a simmer slowly over low heat for about 30 to 35 minutes, until reduced to about ¾ of its original volume (you'll have about 1 ½ cups in the pan). Do not stir it while it simmers. It will take on a light golden brown color.
While the milk simmers, foam or film may form on the surface. Gently scrape it off with a spoon, and discard it.
Remove the milk syrup away from the heat and let it cool until warm to the touch.
Add the remaining 2 cups of milk and vanilla, and stir to combine.
Pour it into a quart-sized lidded container and refrigerate.
