Remove two 3"-long strips of zest from 1 small navel orange with a vegetable peeler; fold each strip in half and pierce with a clove to seal. Cut orange in half and squeeze juice into a measuring glass until you have ¼ cup.
Bring orange zest with 2 whole cloves, orange juice, one 3"-long cinnamon stick, one 14-oz. can whole-berry cranberry sauce, ½ tsp. Diamond Crystal or ¼ tsp. Morton kosher salt, and ½ tsp. freshly ground pepper to a boil in a small saucepan. Reduce heat and simmer, stirring occasionally, until slightly thickened, 10–15 minutes. Let cool.
Arrange Brie, crackers, and olives on a platter; spoon chutney over Brie. Do ahead: Chutney can be made 3 days ahead. Cover and chill.
