We've got a lot of genres here, but they can all really be explained by the genre of "literary fiction." Wide Sargasso Sea is very in-your-face about the fact that it's tackling one of the classics of Victorian fiction, Charlotte Brontë's Jane Eyre. Its language is loaded with allusions to everything from the Bible on up to Thomas De Quincey's Confessions of an Opium Eater, peppered with a few popular Caribbean and music hall songs. This kind of self-conscious literary riffing is pretty much what you get with "literary fiction," a genre that aspires to be stylistically innovative and shuns commercial appeal.
As part and parcel of this stylistic innovation, the novel subverts other literary genres: enter "Coming of Age," "Historical Fiction," "Horror and Gothic Fiction," and "Modernism." Usually in a coming-of-age story, we get some kind of maturity as a character transitions from childhood to adulthood, but with Wide Sargasso Sea, surprise, we don't get maturity (unless burning down a house is your idea of maturity).
In "Historical Fiction," we usually get some concrete historical dates, personages, and events to get our bearings, but the novel makes only sparing references to historical events that are, nevertheless, absolutely critical to our understanding of the novel (see the brief mention of the Emancipation Act on the very first page).
The novel certainly has its fair share of ghosts and creepy houses, the mainstays of "Horror and Gothic Fiction," but the novel is only scary if you believe that Christophine can work supernatural wonders.
Finally, Jean Rhys's previous novels were all in the high Modernist vein – some famous ones include Good Morning, Midnight and Voyage in the Dark. They are very stylized, very innovative, very complex, but largely set in bohemian Paris or London. By setting the majority of her novel in the Caribbean, Rhys makes Modernist style speak to the many important social and political issues of the time.