Bring potatoes, 2 tablespoons salt, and enough water to cover potatoes by 1 inch to boil in large saucepan over high heat. Reduce heat to medium and simmer until potatoes are just tender, about 5 minutes.
Drain potatoes thoroughly in colander and spread in even layer on rimmed baking sheet. Drizzle half of vinegar mixture over hot potatoes and let stand until cool, about 20 minutes.
While potatoes simmer, whisk vinegar, water, garlic, rosemary, ½ teaspoon salt, and ¼ teaspoon pepper in large bowl. Stir mustard into remaining vinegar mixture, then slowly whisk in oil.
Scatter parsley, peppers, and Parmesan evenly over potatoes. Transfer potato mixture to bowl with vinaigrette and toss gently until combined. Serve.