While Malecasta is definitely not much of a threat to Britomart or the other knights in Book 3, her pleasure-driven ways do represent a challenge to Britomart's chastity—she is, in every way, the opposite of chastity. And the challenge doesn't come from Malecasta's six knights who guard her castle, but from the slightly awkward sexual advances of Malecasta herself… who (seemingly) missed the whole headline about Britomart being a lady.
As Malecasta lusts after Britomart more and more, Britomart feels more and more confused about how to behave. While she wants to make it clear to Malecasta that she isn't interested, she also doesn't want to be rude. This issue, of how to courteously turn someone down, comes up again and again in Book 3, where courtesy and chastity are often depicted as opposites that are difficult to reconcile.