Disoriented musings of a technology romantic. Salt recommended with each reading.

I have recently come across a couple of situations where people have expressed their frustrations at the lack of expertise and skilled workers in Malaysia. I have some thoughts about this issue and I would like to put them down here.

Mismatch of Skills
In one case, I visited a semiconductor foundry (those places that manufacture the electronic chips) and was told that our government had already identified a shortage of local talent and expertise in the areas of integrated circuit design (IC design) at both front-end and back-end work. Since a huge chunk of our GDP comes from the electronics sector, it was important for us to maintain an edge in this industry in order to protect our exports.

This reminds me of a story – about a government scholar who did a PhD in integrated circuit design at one of the premier engineering institutions in the world. However, after returning to Malaysia and reporting for duty, he was not assigned to work in the IC design area at all. He was instead assigned to work in the area of software and then eventually moved to working in information security. One wonders why.

He told me that he was interviewed by a certain director in this area. The director asked my friend a question that was wrong and so my friend (being severely lacking in social skills) ended up correcting the director and pointing out to him that the question was not correct. At first, the director tried to assert that he was right but in the end had to back down and admit his fault.

It was totally understandable that the director then tried to pass my friend away from IC design work and onto software. That is how one of our expertise in the area of integrated circuit design wound up doing software debugging and testing. Unfortunately, it did not turn out to be the last time my friend had a difference of opinion with the director, and his team. That is probably how my friend found himself moved to the information security department to write web applications.

The moral of the story is that my friend did not end up making coffee.

Mismatch of Expectations
Then, we have the situation where we claim to have a lack of skilled labour in Malaysia.

I had a meeting with a director of a certain educational organisation in Malaysia and this issue was raised – on the lack of engineers to meet our country’s demands. I pointed out that we definitely do not have a shortage of engineers because if we did, the laws of supply and demand would dictate that these engineers would be paid highly. Unfortunately, our engineering wages in Malaysia are barely enough to live by.

For some engineers, it is a matter of survival and most end up doing something other than engineering simply because the pay is better elsewhere. However, for those die-hard engineers who want to work in engineering, they also find it difficult to get a job – particularly for those skilled engineers who graduated from one of the top engineering schools in the world.

You see, most local companies would not be interested in hiring my friend who graduated from a top engineering school. Firstly, the local companies would prefer to hire the 75th percentile rather than the 95th percentile because they would make better workers as they would be easier to control manage. Secondly, they would feel that highly skilled engineers would ultimately leave because they would not be paid a commensurate salary. So, it would be in the best interest of the company to hire and train up a good worker who stays.

I agree that both of these are fair reasons. Particularly if our industry does not have the demand for a large number of highly skilled engineers. We tend to outsource to overseas consultants and contractors when we need to hire in certain expertise.

However, if you are unwilling to pay for the best, you will not get them. As the saying goes – if you pay peanuts you get monkeys. A colleauge of mine pointed out to me today that – there is no reason to get a PhD in Malaysia because people here do not recognise the additional training and knowledge that you have gained – even if you do end up getting it from a premier engineering school. I would have to agree.

The moral of this story is that my friend got a good tan at the beach.

PS: I have many friends in engineering from top universities.

You know what, I did not realise that anything untoward was happening within our opposition coalition. In fact, even after everyone started making a big fuss about MPs quitting and aligning themselves with the ruling coalition, I still did not feel like anything special was happening. Then it dawned on me – I have become politically numb with all the random sandiwara that happens and am no longer able to respond as expected towards political news.

On one side, people are painting the picture of a crumbling opposition coalition – fraying at the edges after being attacked for so long by so many people. However, I see the very same thing happening with our ruling coalition as well, also fraying at its edges after being worn for so long by so many people. So to me, crumbling political coalitions seem to be the norm in Malaysia.

On the other side, people are screaming like victims – victims of their own devices. They sowed the seeds of political expediency and are now reaping the harvest. I have often harped that our people do not have much of a choice when it comes to electing their representatives. That is why I have been trying to drum into people’s heads that there are four possible outcomes on a ballot sheet with two boxes.

Personally, I see these games as all part of a war of attrition, that all sides will lose in the end.

On a more exciting note, I had to fill up my personal evaluation form today. It was fun filling it up because I got to rate my own performance, which was naturally exemplary. However, I found it vexing that I could not rate my boss. I believe that performance evaluations should go both ways. Otherwise, the bosses would never know if they were a good boss because the people under them have no voice. Any honest evaluation, that is.

Seems like I am not alone in thinking that power-efficiency is a system issue that includes both hardware and software. A recent article at ACM shows that other leading processor architects are also thinking along the same lines. So, I ask myself what am I going to do about it. Due to recent events, I now find myself with a bit of free time. Therefore, I think that I will put my free time to some work by trying to do some interesting things.

Firstly, I will rework my existing processor into a more power friendly design. What I mean by this is not that I will make it lower in power, but I will try to make some changes to it so that the processor’s power distribution and consumption can be intelligently controlled by software. At the barest minimum, this would mean adding a programmable power and clock manager into the core. It could also potentially include lower power but slower computational units that can be used for less critical operations.

Secondly, I will try to modify Icarus Verilog to output some power information. It would not be possible to output accurate power results because power is highly dependent on manufacturing technology. However, regardless of technology, it could possibly output relative power information. So, I could possibly use it to test out different code paths that produce the same results in a similar time period but with different power profiles.

Thirdly, if I have the resources, I will try to make changes to LLVM to facilitate power information. This will probably take a bit of doing as the LLVM compiler is a pretty complicated piece of software. Using the information garnered from the simulator, I could possibly output relative power measures for different types of code. This will be a good first step towards generating power efficient code. Of course, the ultimate aim would be to have a -Op power-optimised flag that will automagically trade-off speed for lower power.

Fourthly, now this is stretching it, I will try to write a nano-kernel that is power optimised. This may just be as simple as automatically putting the processor to low power mode when the running task is idle and waiting on user input. It may also be as complicated distributing tasks over a bunch of heterogeneous cores taking power considerations into account such as sending less critical tasks to cores running at lower power configurations.

Now, if anyone thinks that this indicates that I have succumbed to being an eco-mentalist – you’re absolutely wrong. I still belong squarely in the skeptic camp. However, I do believe that efficiency is important – actually any engineer can tell you that the whole purpose of engineering is to increase production efficiency. I do believe in being more efficient, not for the purpose of saving the planet, but for the purpose of engineering elegance.

So, let me start this today.

I am rarely wrong and I hate being wrong. However, I am only human and I do get things wrong sometimes. I will also accept criticism with an open heart and open mind, if and when I am wrong. When I have done something wrong, I will usually try my best to correct it. It may take me a while to get things right, but I will endeavour to make the necessary changes. It makes me extremely happy when my corrections are acknowledged and found to be satisfactory.

One stage of my life is coming to an end while another stage is just beginning. Things could have gone very bad for me but I am glad that they have turned out fairly well. While I do not regret the path the I have chosen, this journey has been an arduous and long one. I will never recommend it for anyone else, unless they are brave or dumb enough to plunge headlong into the unknown without regards for consequence. Paths are opening up where there were none before.

I have had to make some sacrifices along the way and lost some people in the process. However, I do hope that it will all be worth it when things are finally cut, weighed and measured. I am getting a bit too long winded now. Let me end this by wishing Cecilia all the best and good luck!

PS: I must do something to celebrate – like maybe buy myself a new 50″ television.

Over the weekend, I began to move my home cloud system over from OpenVZ to XenServer. I thought that I should chronicle a few lessons learned here. Some of this information is just not available on the Internet and some of it are my personal opinions about the pros and cons of the two different virtualisation platforms.

Initial Installation
The initial installation for both OpenVZ and XenServer was pretty straight-forward. For XenServer, both the base ISO and Linux additional ISO were needed during installation. Fortunately, they can both be burned on the same CD-RW, one at a time and used one after the other. So, there is no need to use two blank CDs to install it.

However, the two platforms could not be more different when it came to configuring the systems. I was totally lost in XenServer because it is quite different from using a regular Linux system. There were a lot of very new concepts that I had to quickly pick-up. For example, everything needed to be identified by UUID because there is no other easy way to do it. Xen abstracts everything away so that it can be managed automatically without too much user intervention.

Lost in Lenny
I needed to run Debian Lenny as the guest OS for all my cloud servers. In OpenVZ, all that was needed was to download the Lenny template and generate a new server with it. However, the Lenny template on XenServer does not produce a running system but instead, runs a net-install.

At first, I tried to follow the instructions given on the Xen website to download and build a Lenny installation ISO to be used to install Xen. However, it turns out that it is much faster to just do a basic net-install over the Internet than to download and use an installation DVD. So, that was the path I took instead. Since I have a local apt-mirror running, it was a breeze to install and generate a number of different templates.

I do not understand why Xen did not ship with a bunch of Debian templates. It ships with a whole bunch of other templates. I managed to generate three templates corresponding to the popular VPS packages offered online – 64MB, 128MB and 256MB. I will try to share my pre-generated templates once I figure out how to get that done.

Size is Relative
I learned that 64MB is not actually 64MB in both OpenVZ and XenServer. Due to the way that these technologies emulate memory, they perform very differently under different circumstances. This is very obvious in low memory configurations. I could barely run any services on a 64MB OpenVZ server but I could run a whole bunch of services on a 64MB Xen server.

After doing some reading, it all comes down to how memory is allocated and committed by each technology. I might be wrong but it turns out that when an application asks for memory on OpenVZ, that memory is reserved for it. However, on Xen that memory is only reserved when the application uses the memory.

Toy Cloud
After my initial toying around, I would have to say that Xen seems more like an enterprise product while OpenVZ is not so much. In the year that I have been using OpenVZ, I managed the guest servers as if they were parts of the host server. However, in Xen, I am forced to treat each guest server as its own server. In fact, technically, the Dom0 and DomU Xen domains are technically peers.

Overall, I am quite happy to say that I have a very good impression of Xen and I will continue to use it for all my future cloud needs.