Caesar isn't really a traditional antagonist, in that his primary goal isn't to screw over the protagonist. However, he is touched by a desire to rule the state. If we think of Brutus as a defender of the state, then Caesar's actions against the Roman Republic understandably set him against Brutus. Brutus is antagonized by Caesar's ambition. In fact, it's concern over Caesar's ambition that gets our hero Brutus into this whole mess in the first place.
Once Caesar is out of the picture, Antony becomes the active source of Brutus's woes. While Brutus was bothered by Caesar in the abstract, Antony becomes a very deliberate threat to Brutus's well-being. In fact, Antony plots to have the people riot and is glad to hear that Brutus and Cassius have fled town because of the chaos he's caused. We don't get a clear sense of whether Antony is motivated by his compassion for Caesar or by his own ambition, but it's clear that, whatever the cause of Antony's ire, he wants blood, and he wants it from Brutus. Regardless of the reason for his antipathy, Antony antagonizes Brutus. It can't hurt that once Brutus is out of the picture, Antony can move on unchecked, filling the power vacuum left by Caesar.