I do prefer the Castle too. But I'm very biased, because I love castles. The atmosphere is on point, the place is very varied and no room feel alike (indeed, kinda like the Spencer Mansion). The only thing about the Castle part that it's 'meh' for me is the mine area.
Then comes the village. The place is pretty spooky the first time you play and it helps to set a good atmosphere, especially at night. It doesn't have much detail though and by the end it starts to look too much the same. Still good.
The island is...okay-ish. In my opinion, RE always drops the ball in the game's last areas overall. It's always a bunch of uninspired labs and stuff. The island does have some interesting ruins, but the game doesn't elaborate on that (some files could be good. Just to explain what they are and to flesh some more lore in the place you're fighting). Again, it's not bad and some of the areas are really good, but I prefer the Castle and the Village's atmospheres better.