Purée celery, shallot, cornichons, garlic, mayonnaise, horseradish, parsley, capers, mustard, lemon zest, lemon juice, and cayenne in a blender or food processor until smooth; season with salt and pepper.
DO AHEAD: Sauce can be made 2 days ahead. Cover and chill.
