Let's take a look at the 2008 attendance.
48,058 UT-Martin
58,755 Kansas
50,307 Pittsburgh
51,384 Syracuse
47,216 Rutgers
42,422 UConn
49,500 Average
Couple of thoughts/observations:
We drew 48k when unraked against an FCS team, with really good momentum going into a season
We were not drawing those types of numbers against conference mates like Pitt/Syracuse/Rutgers/UConn until we had success, and those games were not dominated by alumni of those schools any more than we would expect from the likes of Temple/Memphis, let alone Navy/ECU, so it seems a stretch to attribute the high attendance to their alumni - what I'm getting at here is these attendance numbers were driven mainly by our own success and it is repeatable even in our current environment
I would speculate that even in a really down period in the Big 12 we would draw better than Kansas, for example, simply due to the notoriety of the visiting teams and Tampa being a destination