The process of collecting and reporting data has been called into question at several universities in the wake of recent revelations that five private colleges misreported data to the U.S. News and World Report.
According to an article in The Washington Post, “Five colleges misreported data to U.S. News, raising concerns about rankings, reputation,” five schools were accused of misrepresenting their data, including Tulane University, George Washington University, Emory University, Bucknell University and Claremont McKenna College. The alleged reporting discrepancies within each school point to two separate issues: intentional tampering of data and flaws in some of the schools’ reporting systems.
In each case, the schools either submitted incorrect test scores or inflated the high school rankings for their incoming freshman classes. Some of the schools admitted to employees intentionally submitting the incorrect data, while others, like George Washington, claimed the discrepancies came from flaws they caught in their internal data systems.
Scott Grimshaw, a professor in the BYU Department of Statistics, said when compiling the statistical data that ranks schools against each other, it’s critical that each university understands the specific criteria.
“Statisticians emphasize the importance of operational definitions, so everyone’s reporting data the same way,” Grimshaw said. “There’s a lesson for U.S. News in deciding what those operational definitions are so everyone’s playing fairly. Year to year, they find schools that don’t understand the criteria correctly, and it gives them an opportunity to fix the system.”
While Grimshaw expressed having no knowledge regarding BYU’s internal data handling, he said that with hundreds of schools reporting information to the publication, it’s logically expected that each school is honest in its research methods.
“It gets really hard with all these weighted categories to validate what makes the top schools the ‘best’ — there’s not a universally accepted method of determining what a good school is,” he said. “U.S. News is in the business of making news — not being held accountable for the accuracy of their rankings. With so many schools, they’re in a position where they trust that the data they’re provided is fair and honest.”
Todd Hollingshead, BYU’s information and media relations manager, explained that BYU’s Planning and Assessment Office is responsible for collecting information from departments across campus and reporting to several organizations, including U.S. News and the Department of Education. During the process, each department verifies the data it sends in, and then the entire project is independently verified a second time by several people outside the Planning and Assessment Office to maintain its accuracy.
“The U.S. News survey is massive and requires information on everything from enrollment of students and tuition prices to all the extracurricular activities offered at the university,” Hollingshead said. “We take very seriously the responsibility to have accurate data compiled and reported.”
Caitlin Mitchell, an American studies major, said the rankings provided by U.S. News have been a source of pride during her time at BYU.
“I really like going to BYU, and I have always felt great about the quality of my education,” Mitchell said. “It’s nice, though, to see how BYU matches up with other schools in the nation. And most of the time, the rankings show that it really is a great school.”
Mitchell also said students should have the same high expectations for the university that it has for its students.
“Just like BYU expects their students to be a good reflection of the university by being honest, academically and otherwise, I would expect the same caliber of integrity from the university as an organization,” she said.