I'm almost sure the Big 12 has a list of approved crews who can be used for games (hell, even most of the high school conferences in Iowa have them). They do multiple conferences because of travel limitations. I have my doubts a lot of referees do Big East games and Big 12 games, but there are a handful who do Missouri Valley/Big 12 games because the footprints are about the same and flights are reasonable.
And NCAA referees are not underpaid. How many jobs have you had where you can make about $700 in about 3 hours with travel costs and food paid for? The guys who ref three or four nights per week are making $40,000 in four months. Some of them work jobs on the side as consultants or salespeople but the majority make a very good living.
I personally know because I know a couple guys who do college games for a living. You seem to be missing a couple big points:
a) what you can see isn't necessarily what a referee can see. Ref zones don't overlap much so if you have a 6'5" 220 lb guy in front of you screening you, you can't make a call. Officiating is a hell of a lot easier sitting in the stands where you can see over everyone than it is on the floor where everyone is as tall if not taller than you are.
b) you get to see the replay. That's part of why people think officiating is worse. It isn't worse, you just get to watch on TV and see replays or rewind your DVR.
The referees do have to justify their calls. There is someone in the conference office whose job is to watch every game and they conference call the referees involved in the game to correct any mechanical errors which may have been made and they have the opportunity to answer why they didn't make certain calls.
Application of some common sense would be useful here. Do you really think the Big 12 Conference assigns referees and just says, "here's your check guys, best of luck!" How many jobs do you know of where there aren't performance reviews? Come on.
Southeast corner of the parquet is where the people who review ther referees normally sit........ see the tables set up there, not for you and me to sit at.