In a large mixing bowl or measuring cup, whisk together the vinegar, olive oil, lemon juice, capers, garlic, celery seeds, red pepper flakes, kosher salt, and ground mustard. Set aside.
In a large dish or in a large glass jar, alternate layers of shrimp, onion, and bay leaves.
Pour the vinegar mixture over top. Cover or seal with a lid and refrigerate for at least 6 hours or up to 3 days.
Remove the bay leaves. Arrange the shrimp and onions on a serving platter and garnish with scallions or fresh herbs, if desired.
