In a large bowl, combine chocolate pudding and cocoa mix with milk. Whisk until fully incorporated. Add Baileys and vanilla and whisk again. Pour into cups and refrigerate until thickened, 2 hours.
Top with whipped cream, a sprinkle of cocoa powder, and marshmallow bits. Serve.
