with my hookfast in a corner of his mouth. (lines 3-4)
The beginning of the poem demonstrates the battle as we might expect it: man being the aggressor in the natural world. Here, we see by the pronouns Bishop uses that it's "my" hook in "his" mouth, instead of a much more hands-off choice of pronouns like, say, "a" hook in "its" mouth.
I looked into his eyeswhich were far larger than minebut shallower (lines 34-36)
In line 34 you think we might be seeing the beginning of a connection or a "look we're all the same in the end" message, but lines 35 and 36 go on to correct that. It's still very much a man vs. nature scenario.
They shifted a little, but notto return my stare. (lines 41-42)
Much like lines 34-36 this shows how the human and the fish are not alike. It's here, too, that we start to see that although this is a man vs. nature situation, the real conflict lies very much within the human.