In a medium saucepan, combine the cocoa powder, sugar, water, and salt over medium heat.
Cook, stirring constantly, until smooth and bring to a simmer.
Reduce heat, and stir in the milk, do not boil.
Remove from the heat and stir in the vanilla. Pour into mugs and serve immediately.
