Cook cocoa mixture: In a medium saucepan, whisk together cornstarch and 2 tablespoons milk until smooth; whisk in remaining milk, cream, sugar, cocoa, and salt. Bring to a boil over medium-high, stirring constantly, and cook 1 minute.
Add vanilla; pour into molds: Transfer to a large liquid-measuring cup and stir in vanilla. Pour into 10 3-ounce ice-pop molds and insert ice-pop sticks.
Freeze until solid: Freeze until pops are solid, about 6 hours (or up to 1 week). Just before serving, briefly run molds under hot water to release pops.
