I can't remember anything from the brick about Valjean being abused particularly because of his strength, but I'm not going to bet my life on it... And now I want a fic about the Gorbeau robbery from his POV, augh. Not necessarily about rape threats, it's just such an awesome scene (and IMO underused in fic).

Also, while Valjean's strength may protect him from other prisoners, I'm sure he knows that any of the guards could force themselves on him, and he couldn't do anything.

*nods* That's the premise of this fic by Carmarthen, actually -- there's no actual rape, just Valjean's fear of it and his awareness that he'd be effectively defenceless if it were to happen. It is a very good story.

What exactly were Hugo's words on Valjean's sexuality while in prison?

In the copy I have here, it says that "In prison he had been sullen, gloomy, chaste, ignorant, and ferocious" (in the chapter following his and Cosette's first night at the Gorbeau house after they arrive in Paris). I'd have to look up the original to say for sure, but I took it to mean that while he might be (and probably was) aware that rape happened, he just kind of blocked it out as just a general part of the awfulness.

I don't know how fem!Valvert can live happily ever after in their day and age

I believe that if canon Valvert can do it, female Valvert can do it! They just need to... not die. :-(