Questionable SETARA 2011

Malaysian Qualifications Agency
Malaysian Qualifications Agency (Photo credit: Wikipedia)

Our local Malaysian ranking for undergrad teaching quality of local universities, SETARA, was recently released. I thought that it would be fun to check it out to see how things have changed since the last ranking exercise.

Some universities that improved tremendously from SETARA09 to SETARA11 are:

  • Binary University of Management and Entrepreneurship – vaulted from Tier 3 (Good) in to Tier 5 (Excellent).
  • Wawasan Open University, Open University Malaysia, SEGi University – parachuted in at Tier 5 (Excellent) from being previously unranked.
  • TATi University College, University Pertahanan Nasional Malaysia, Universiti Malaysia Kelantan – moved into Tier 4 (Very Good) from being previously unranked.

Needless to say, I was shocked.

I don’t know the detailed criteria nor methodology that was used during the survey but it shocked me that some of the universities that made it to Tier 5 (Excellent) were not what I would have expected. Granted, I am no expert in local universities nor their standards but some of the results just beggar belief.

I’m not trying to single anyone out but, two of these Tier 5 universities are classified as open and distance learning institutions. I do not understand how they could achieve excellent teaching quality when the students are not even in the same room as the lecturer. I know that internet based learning is all the craze these days but even places like MIT do not consider their open learning systems at par with their in-house teaching, yet.

Some of the other Tier 5 universities are virtual unknowns. Again, I do not claim to know all the universities in Malaysia (there are just too many to keep track of these days) but some of them are total unknowns. Some are even the butt of many jokes on education quality in Malaysia, the kind of university that you would not want people to know that your child was attending.

Furthermore, more than half of the universities ranked are of Tier 5 level while there is only one university at Tier 3 (Good). I would have expected something like this to be more normally distributed as it was in SETARA09. When I see such heavily skewed results, I tend to question the accuracy of the study. Something just does not smell right to me, statistically speaking. It’s highly improbable that the majority of institutions is Excellent.

Brings to mind an XKCD comic:

My personal opinion of this is that some of these Tier 5 universities must have gamed the system. I do not blame them for doing that though, as the SETARA rankings affect many aspects of a university including fees. Nobody would want to pay top dollar to study in a Tier 4 university when they can study in a Tier 5 one for cheap. So, I expect some of these virtual unknowns to raise their fees next year!

I hope that the MQA would take the effort to toughen and tighten the criteria for ranking in the next exercise. We need a ranking system that truly reflects the teaching quality of local universities. It would serve as a useful guide to parents, only if they actually trust the accuracy of the exercise. I don’t think that any parent would agree that more than half our local universities are Excellent!

I think that a lot of people who see the list would think and start to question the SETARA rankings and methodology. While I agree that there are always flaws within any sort of ranking system, when this happens, it will become more fodder for local coffee shop talk. We know that some of these Tier 5 universities deserve to be there while others are downright questionable.

Personally, I’m not going to even bother with SETARA11. I’ll stick to the more normally distributed SETARA09 rankings for now.

Published by

Shawn Tan

Chip Doctor, Chartered Engineer, Entrepreneur, Law Graduate.

10 thoughts on “Questionable SETARA 2011”

  1. do u know i have compare the exam question with of open university and and other university its not a big different , remember lot of open u students is studying while working, and using their own money to pay the fees, they know the value of their studies and they do study hard for me i always consider them as rival because i do have friends in open university and believe me they are good with what their doing.. i think u just feel threatened by them remember there is no better student that know the real value of studies that that work hard to pay for fees. even their degree is certified mqa

    1. I think that you’re mistaking teaching quality with exam questions difficulty and student quality. I know that their degree is recognised and that they’re mostly working adults.

      Since I’m actually taking a part-time external degree that allows for self-study and also for class-room study, I have to say that there is definitely a difference between the two. Also, I’m drawing upon my many years of teaching experience at various levels. Teaching quality is definitely affected by the contact hours between the students and lecturer.

      A place that promotes open learning for mature students is definitely a good thing to have. However, classifying them the same as some of the other Tier 5 institutions does injustice to the rest.

  2. yes i agree with your write up. i have my doubts about the system. people are so into ranking these days but when you actually step into the university its a whole different scenario. there are dissatisfaction among students towards the university, government style attitude (if thats how you call it?), no passion for education etc.

  3. well i do agree about their teaching quality is way low then normal universities but if open universities student can keep up in exam and scored high then i have no reason to say they shouldnt rank tier 5, come on guys and girls just because we study in conventional universities doesnt mean a open u shouldnt rank in tier 5 for their effort.. i agree with chris lotsa people do feel threathen by the open universities

    1. the setara ranking is for teaching quality, not student results. you’re confusing the purpose of the rankings and the students academic results. the two are separate issues. as an example, if everyone gets an A in an exam, it doesn’t mean that everyone has mastered the material and the lecturer is excellent. it just means that the exam was too simple.

      open universities have a role and serve a function. however, it is impossible to have the same kind of contact as compared to a conventional university. so, that alone means that the teaching quality is definitely affected.

      i should know this since i’m taking an external degree now. it’s different when you have full contact with the lecturers and when you don’t.

  4. 1st. Are you aware that Open University Malaysia’s lecturers are from full-time faculty at universities such as UM, UTM? So is there any doubt on their teaching?

    2nd. The core team at OUM is lean, as per any open university. However, if you were to look at their publication in recent years, it is still respectable:
    http://academic.research.microsoft.com/Organization/13868/open-university-malaysia?query=org%3a%28open%20university%20malaysia%29
    Note that unlike full fledged research intensive universities, they do not have similar fundings and yet they bother to take on research to uphold the university reputation. You can find OUM’s paper published at top tier IEEE journals too.

    Considering all these, you still don’t think that they deserve to be in Tier 5? Compared to those australian universities that are here just for the money.

    1. You don’t seem to understand. I don’t blame you though.

      1. Yes, I do doubt the teaching quality even if the lecturers are from UM etc. Please understand that there is a difference between teaching quality and lecturer quality.

      2. Keep in mind that research quality is ranked in MyRA, not SETARA, which only focuses on teaching quality. Again, there is a gulf that separates research quality and teaching quality.

      I think that there is a lot of confusion among people about what SETARA actually ranks and what teaching quality actually means. Teaching quality is not student quality nor research quality nor teacher quality.

      Maybe I should write an article on this one day.

      Plus, you don’t seem to address the glaring statistical problem here – more than half of the universities are Tier 5? Seriously?

      1. I’m a part-time student from OUM pursuing Master in Management and currently in the last semester. My full-time job is IT Manager with DHL (Cyberjaya)

        Below are my unbiased inputs about OUM’s campus that I attending at Jalan Tun Ismail (sharing with UM’s campus) :

        1. Attendance to classes is not compulsory, however mostly did attend (unless you select as “online” student). Personally, i never missed a single class throughout my entire tenure.

        2. Course Works (assignments) question contributes 60% of our total mark. These assignments in nature really help to improve the cognitive thinking of the students and applicable to our job as manager both academically and practically. How a student approach the assignment is very subjective, thus the output of the learning experience varies. Some of the students i know, really score pretty low when they are careless. For “online” students, they don’t have suggestion from lecture on how they should tackle the questions because usually this will not be posted in the student portal (course forum).

        3. No tips for exam as lectures are not the one who set the paper.
        Like yourself, OUM students are required to cover the entire study syllabus, if they want to score in the exam despite only 40% marks from exam.

        4. In post-graduate level, what is more important is how long to “acquire” knowledge. Our research methodology on the subject is more helpful in this context. The presence of a high quality lecture / teaching definitely beneficial, however this factor must not be totally dependable upon as knowledge itself should be acquired and analysed carefully.

        5. Some of the areas in OUM sucks. Its online portal often accessible (now had improved) and I did have bad outcome from the student affair department, resulting I’m unable to withdraw from KWSP for my tuition fee for some semesters. However, this is small matter.

        Bottom line, OUM is not as bad as you thought. Majority of the students in OUM (at least the one I know), are working adults. The keyword here is “discipline” to pursue what we want. Same goes to everything else in life. Spending 2 years with OUM, I am satisfy with its approach and plan to enroll with their DBA programme in June 2016.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s