Byron was a Romantic-era poet, not a Victorian (the Victorian period starts ~1837, the year that Queen Victoria was crowned in Britain), but the Victorian Web is still a useful site for students of the nineteenth century in general. This is the link to the section of the website devoted to the Romantic era.
This website has some background information about François Bonnivard, the real-life prisoner of Chillon who was the inspiration for Byron's poem, and includes a picture of the castle.
Byron wrote "The Prisoner of Chillon" in a hotel on the shores of Lake Geneva, way back in 1816. And the hotel is still there! You should check it out, if you're ever in Lausanne, Switzerland.
We're told that the dungeon at Chillon had "seven pillars of Gothic mould" (line 27). Wonder what that would look like? This is an image of the actual cellars in Chillon Castle that were used as a dungeon. Byron and Shelley visited the castle (and, presumably, the cellars) when out sailing on Lake Geneva. Byron was inspired to write the poem, and the rest is history.
This engraving of Chillon Castle is from the early nineteenth century, around the time when Byron was writing.
This photograph of Chillon Castle and Lake Geneva gives you a sense of how the castle is situated relative to the water.
Here's a portrait of Byron, looking charming and suave as always.
This is a history of François Bonnivard, the inspiration for Byron's "Prisoner of Chillon," which was written in 1892. The author may have gotten a few of the details wrong, but it's still an interesting read. The book was digitized by Cornell University's library, so you can access it here.
This website offers some interesting background on the castle where the Prisoner of Chillon was imprisoned. Check out the photo of the castle – it's still there!
This is an excellent article in the journal Victorian Studies by James Buzard. You'll need an electronic subscription (available through most libraries) to access this article.
This is a classic critical study of Byron's works. You can find a hard copy of it at most libraries, or you can access excerpts of it through Google Books, here.