I grew up and graduated from Oskaloosa HS, and currently teach in a small 1A (8 man football if that gives you an idea). While Osky isn't huge, it is a large 3A school for the state of Iowa. Both have advantages. I can't tell if Osky had more rigor or if it felt that way because I took several AP and DC classes whereas my school of employment has limited options for students to receive college credit.
I think the coolest thing I have experienced in my smaller school was when we got some kids in who had had a really rough go at it by the time they arrived. The first week or so they were ready to fight anyone and everyone, because that is the environment they had experienced at their previous schools. But after that they let down their guard and became a part of our school. They realized that they didn't have to fight to survive and actually flourished. I surmise that these kids would not have graduated from their previous school for reasons unrelated to academics.