Questionable SETARA 2011

Malaysian Qualifications Agency
Malaysian Qualifications Agency (Photo credit: Wikipedia)

Our local Malaysian ranking for undergrad teaching quality of local universities, SETARA, was recently released. I thought that it would be fun to check it out to see how things have changed since the last ranking exercise.

Some universities that improved tremendously from SETARA09 to SETARA11 are:

  • Binary University of Management and Entrepreneurship – vaulted from Tier 3 (Good) in to Tier 5 (Excellent).
  • Wawasan Open University, Open University Malaysia, SEGi University – parachuted in at Tier 5 (Excellent) from being previously unranked.
  • TATi University College, University Pertahanan Nasional Malaysia, Universiti Malaysia Kelantan – moved into Tier 4 (Very Good) from being previously unranked.

Needless to say, I was shocked.

I don’t know the detailed criteria nor methodology that was used during the survey but it shocked me that some of the universities that made it to Tier 5 (Excellent) were not what I would have expected. Granted, I am no expert in local universities nor their standards but some of the results just beggar belief.

I’m not trying to single anyone out but, two of these Tier 5 universities are classified as open and distance learning institutions. I do not understand how they could achieve excellent teaching quality when the students are not even in the same room as the lecturer. I know that internet based learning is all the craze these days but even places like MIT do not consider their open learning systems at par with their in-house teaching, yet.

Some of the other Tier 5 universities are virtual unknowns. Again, I do not claim to know all the universities in Malaysia (there are just too many to keep track of these days) but some of them are total unknowns. Some are even the butt of many jokes on education quality in Malaysia, the kind of university that you would not want people to know that your child was attending.

Furthermore, more than half of the universities ranked are of Tier 5 level while there is only one university at Tier 3 (Good). I would have expected something like this to be more normally distributed as it was in SETARA09. When I see such heavily skewed results, I tend to question the accuracy of the study. Something just does not smell right to me, statistically speaking. It’s highly improbable that the majority of institutions is Excellent.

Brings to mind an XKCD comic:

My personal opinion of this is that some of these Tier 5 universities must have gamed the system. I do not blame them for doing that though, as the SETARA rankings affect many aspects of a university including fees. Nobody would want to pay top dollar to study in a Tier 4 university when they can study in a Tier 5 one for cheap. So, I expect some of these virtual unknowns to raise their fees next year!

I hope that the MQA would take the effort to toughen and tighten the criteria for ranking in the next exercise. We need a ranking system that truly reflects the teaching quality of local universities. It would serve as a useful guide to parents, only if they actually trust the accuracy of the exercise. I don’t think that any parent would agree that more than half our local universities are Excellent!

I think that a lot of people who see the list would think and start to question the SETARA rankings and methodology. While I agree that there are always flaws within any sort of ranking system, when this happens, it will become more fodder for local coffee shop talk. We know that some of these Tier 5 universities deserve to be there while others are downright questionable.

Personally, I’m not going to even bother with SETARA11. I’ll stick to the more normally distributed SETARA09 rankings for now.