I don't see any warranted criticism apart from "I didn't like it because nyyaaaahhhh."
If you're going to call it the worst installment, have objective reasons. For example, I think the story was pretty weak and full of plot holes when it comes to plausibility. There are some sequences that made no sense such as the helicopter pilot part and the fact that the agency didn't send anyone to investigate why they lost contact with Leon.
Asset wise, the game could have used a little more variety in enemy skin types. Villagers were varied enough, but after a while kind of blend together. The robes at the castle were a clever way of cutting down on skins though.
The inability to move and shoot was kind of puzzling, although the game isn't significantly harder without it.
There is a complete absence of any survivors beside Luis and Ada who are not really locals. It's a little suspicious that the virus would infect with 100% efficiency. Although you could assume that they killed everyone who didn't become infected anyway.
Some level designs are downright bizarre and some sequences like running away from Salazar's statue, the giant truck thing, the lava room, hanging crates with tunnels and buttons, etc. There are no logical explanations for some of these levels/areas, but it's a video game. You could nitpick all day, that depends on how much suspension of disbelief you have. Things like surviving an axe to the head, carrying a metal case full of guns and ammo, jumping 20 meters down a ladder, etc etc. There are plenty of unrealistic things about video games. That's not the point.
For a subjective opinion, what really matters is the feeling the game gave you. Most people who played RE4, whatever feeling it gave then, must have been a great feeling. We can theorize as to why people enjoyed it, for example, the voice acting was pretty decent, the characters were written pretty well, the gameplay was fun and varied enough to keep you engaged, it complete reinvented RE, but still stuck grounded in the same feeling of horror, it was familiar to fans, blah blah blah.
We all have our reasons for liking it, and they might not be rational, but when you make objective claims like it sucks, you need to be able to back them up.