Psylarium

The maze of finding a good VPS hosting for Drupal and other CMS

2011-06-14T18:06:00.000-07:00

I went through the maze of finding a good VPS hosting for a relatively database heavy website (typically any CMS will be in that category, Drupal, Joomla, Plone, Ruby on rails, etc...). So I thought I would write up how the experience, the traps and the final solution I found.

Shared hosting, VPS or dedicated ?

Shared hosting is typically the cheapest alternative, advertised with those super cheap prices (like less than $5 a month). However in practice, unless you just host a few raw HTML pages, your site is going to crawl if you use a MySQL database. If it crawls, most often hosting companies while claiming unlimited this and that will just turn off your website, I've experienced it, many others did.
VPS hosting was recommended to me by my initial hosting company (InMotionHosting) after they unilaterally turned off my website without even notifying me (not to mention I was mad). Their argument was I was using too much resources, time to move to Virtual Private Hosting, which basically is a shared hosting with a fixed amount of resources allocated to you. Now the price varies between $20 to $40 a month for the entry plans between companies.

Watch out: there are two types of VPS hosting: managed and unmanaged, I'll explain more about them later.

Dedicated hosting is when you have a full server at your disposal, it will often cost you more than $100 for a entry plan.

So in my situation, I decided that VPS was probably right for me, I don't have a 100.000 connexions a day but at the same time the website is growing and I need a MySQL that responds relatively fast.

What hosting company ?
Now, if you're like me you type "Best VPS hosting" in Google, and you get tons of results that look faked. And they are (faked), many hosting companies pay people in India to post positive reviews about them, that's called backlinking. So they all can claims they are the best. Now in practice, it's far from the truth. Actually a funny event happend, someone posted in my phpBB forums a spam back link for my own hosting company (InMotionHosting), how stupid is that ?

But at first, I fell for the trick and initially signed up for a hosting based on those faked review, the speed was abysmal but I thought naively that it was normal.

InMotionHosting: Then I moved on to another "best hosting", InMotionHosting which basically may be good for shared with basic HTML pages but with Drupal pretty much forced me to go to a VPS. Once on a VPS the performance improved but remained poor, the loading time for the front page on average took 2 seconds and sometimes it would take as much as 60 seconds to load. The customer support could never reproduce the issue (forget about user support for anything that doesn't happend all the time, like an intermittent huge loading time) and they didn't even try. They could have written a script in 5 minutes to load the front page every 10 minutes to measure the load time, they didn't.
HostGator: So I decided to finally give a try to the next big name, hostgator. They may be good at shared hosting but while their VPS was faster than InMotionHosting, it still remained slow. Page loading, would take 2 to 8 seconds for my front page. Their VPS comes with CentOs 5.6 and all packages feel obsolete (python 2.4.3, etc... they offer python 2.7 but only for shared hosting...). Their live customer support would deserve an entire blog post to tell the horror stories that have happened to me. Their SQL response time was slow as hell (same config as at my home, the queries took 3 time longer to execute).

These companies are reputed as shared hosters but the VPS value remains low. Their VPS are managed (or semi-managed) which basically means that you get an often incompetent user support to spend hours chatting online. But it's reassuring, someone is helping you even if it's of no help, they provide a chat log so you can show your boss it's not your fault that nothing is working. Yes reassuring.

The world of unmanaged VPS
Here basically you're on your own, you get the server, and you need to install everything without a user support holding your hand. And that's what I finally tried with Linode.

I chose to use the Ubuntu 10.04 Linux VPS (it's a cloud server, but VPS and cloud are pretty much the same thing, just that clouds allows cloning). Installation is simple, you get a SSH shell, you install Apache and whatever you need, it's as simple as apt-get. The packages are up-to-date (being Ubuntu) so you don't really have to compile anything.

In 1 hour, I had the whole site setup and working. The performance is excellent, loading the frontpage takes 250ms. The cost is half of what I was paying with InMotionHosting and HostGator.

Final word
My point here is not to make a commercial for Linode but to relate my experience through years of hosting quest. There are probably others who are offering a good VPS hosting, however I didn't use them so I'm not going to comment on them.

The lesson that I learned:

Stay away from big shared hosting companies if you want VPS hosting
Go for an unmanaged VPS hosting company, the performance is incomparable, they offer a lot of documents to help you configure your server and you can always find help by going to IRC and chat.

Performance comparison of my page load times
Those times where measured using Firefox:FireBug->Net load time analyzer.

Company	Average load time	Worst load time
InMotionHosting	1-2s	60s
HostGator	1-2s	8s
Linode	0.25s	0.5s

Portal 2, an honest review

2011-04-21T08:15:00.000-07:00

I don't generally write about games but this time I felt I had to. Because most articles one finds on internet are totally biased and sound more like sponsored infomercials than real objective reviews, and since I don't work for Valve, here's an example of such BS: http://www.rockpapershotgun.com/2011/04/19/review-portal-2/

The good
Technically the game looks good and is fun to play, you really get a sense of achievement when solving some of the harder problems, however none are really that hard. I guess if you played Portal 1, you will find it easier because you're already used to the portal way of thinking.

They have added some new type of technological stuff, like some light bridges, fields and gels which give you more possibilities, however these are not really big game changers. They add variety and it's cool but they never really get used in combination to make you think out of the box. Typically, you just get to deal with 1 or 2 elements at a time, so it's a bit of a let down, they could have pushed the difficulty a bit more.

There are some really nice sound tracks in game, like in other Valve games the music artist(s) deliver super catchy tunes. With one exception the end-track, read below.

The bad
Slightly disappointing, even though I didn't rush, it only took me 6 hours to complete the single player game. For a $50 game, that felt like being shortchanged, I would have hoped for two to three times that playtime for that price. But that's a general trend in gaming industry, you get less for your money every year.

The story feels weak and predictable compared to Portal 1, but well there's a simple story at least. But I wish it would be a bit more surprising.

The humor felt forced and bitter, to me it felt like the team who did this tried too hard to be funny and ended up failing. In other words, it felt like a job, a mechanical forced way to pack jokes assembled by a group self-centered story-specialist sipping coffee for hours in meetings (anyone who ever worked in a software company will get the picture). In contrast, the humor in portal 1 came progressively and was extremely funny (may be it was the work of smaller team).

The ugly
Ok, the first part with the hyper active "ADSD (Attention Deficit Syndrome)" computer core (that round thingy that keeps on talking stupidly with an english accent) is verging on being obnoxious. This part is so lame and feels so forced, it almost made me close the game or at least to play it without sound. Fortunately this only last for 30% of the game, after that things settle down. To put an image it feels like those Walt Disney 3D animations where the characters are over-scenarized and keep talking.

The ending song, I realize they tried to do a superb song like for Portal 1 (which was so catchy and addictive and fitted the story so well) but let's be honest, they failed miserably. To their defense, it's hard to do a hit song on-demand, even rock stars can't do that. So I don't want to be overly harsh on them, just honest.

Combining views and path aliases in Drupal 6

2010-12-18T00:03:00.000-08:00

Ok after a long time off, I finally have something worth to say. First I must thank styol from #drupal IRC channel for pointing me to the solution and to phrancescot for his persistent help. I decided to write it up for other people struggling like me.

What I was trying to achieve
Basically I wanted to add a Tab for a content type song that pointed to a view to report all the reviews for that song, all that using nice auto-aliased urls.

The context
The song content type
I have a song content type, it has the usual Title and Body fields plus a field called duration.
Each song content type is auto aliased using the pathauto module. The automatic alias is song/[title-raw]

The review content type
I have a content type review which basically has the usual Title and Body fields plus a nodereference field called song.

The view to list the reviews for a song
Here I won't go into the SQL query details, it's just standard view work. The important thing here is that in Views->Page there's a Page settings entry. In it, you have to field to fill: Path and Menu.
The path I was using was song/%/reviews. That was my first mistake, while it works perfectly for the SQL request, the tab was not appearing in the tab menu because of that.
Instead the right solution is to use node/%/reviews

Well at this point the path is not the beautiful aliased path. However fear not, since that's where the subpath_alias module comes into play. It will rename automatically the node/% to the autopath matching it. There's nothing to do other than installing that module (talk about drupal magic !).

Now for the menu, simply select Menu Tab and fill the title and description. And voila, the tab will have a new entry, pointing to your aliased URL.

Modules used
CCK
views
pathauto
subpath_alias

Special thanks
From the #drupal IRC channel: styol, phrancescot for the persistent help !

Quote of the day

2010-02-28T00:34:00.000-08:00

I was browsing the excellent blog of SegFaultLabs and I've found the following quote over there, that's my quote of the day !

Programming today is a race between software engineers striving to build bigger and better idiot-proof programs, and the universe trying to build bigger and better idiots. So far, the universe is winning... (by Rick Cook)

My samba server was losing/changing passwords !

2010-01-31T10:38:00.000-08:00

I finally resolved a nagging problem with Samba. I recently installed Ubuntu 9.10 (Karmic Koala), and I was using pretty much the same config file for Samba that I was using with Ubuntu 8.04.

Passwords getting modified automatically ?
However, there was a problem. After a few hours, it seemed like the Samba passwords were replaced by the UNIX passwords. So I had to reissue a sudo smbpasswd <user> to set back my password. In my system, I don't use the same password for Samba and Unix. But somehow, the password was getting changed automatically after a few hours.

Obviously, I already had that option:
unix password sync = no
But that didn't do anything ! The password was still getting changed to UNIX.

Google is of no help
I tried Google/Bing searches but there are thousands of questions involving passwords and Samba, and after reading countless search results, nothing really matched by problem. I mean Google is a good search engine if you're looking for something popular since its core algorithm is a kind of popularity contest. But when you're looking for something rare that can only be expressed with common vocabulary, this search engine really shows its limits.

Read The Fucking Manual (RTFM)
So my last option was to start reading the Samba documentation, which is like reading an encyclopedia. I mean who really reads that enormous document ? What a pain !
But anyway, after a few days of browsing through it I finally figured the issue.

There's something about pam_smbpass
It was a Samba module named pam_smbpass that was installed automatically by Ubuntu. This module, every now and then would replace the Samba passwords with the UNIX passwords.

So I went to synaptic package manager and uninstalled that module. Problem solved.

External RAID 5 with Ubuntu 9.04 and Gnome notifications

2010-01-29T01:22:00.000-08:00

Recently I lost a drive after 6 months, and all my data disappeared. That was a Maxtor 1 Touch 1.5 TB.

So after that disappointment, I decided to get a safer solution. So I wanted to have an external RAID 5 and I found those very nice little towers that go for around $150: TowerRaid from Sans Digital. See the picture on the right to see what it looks like. Basically the front panel opens and you just slide your drives in. Fix them with the side screws and you're done.

This box also came with an PCIe card that provides 2 eSATA ports, of which only needs to be used to connect to that box.

Preparting the drives for RAID
I decided not to use the functionality of the card which offers RAID 5 amongst other RAID flavors and simply use Linux software RAID which is a lot more widespread and documented that the chipset on the card. By default, the card was reporting the drives as JBOD (Just a Bunch Of Disks). So I simply started gparted to find the names of the devices (e.g. the drives inserted in the box).

In my case, I found they were

/dev/sde

/dev/sdf

/dev/sdg

/dev/sdh

Ok, the next step was to create partitions on them since they didn't even have a partition table. For each drive, I used fdisk to create a primary partition of type Linux (83).
Now I had 4 partitions, one for each drive. They were named /dev/sde1, /dev/sdf1, etc...

The actual RAID setup
The RAID configuration couldn't be easier. I simply did.

sudo mdadm --create /dev/md0 -n 4 -l 5 /dev/sde1 /dev/sdf1 /dev/sdg1 /dev/sdh1

The options are quite simple:
/dev/md0 is the name of device (since it was my only raid array, I set it to md0)
-n 4 because there are 4 drives in the box
-l 5 is for RAID 5
and then you pass the list of partitions.

And that's all. Isn't it amazingly simple ?

Creating a file system on the RAID array
From this point on, you can simply treat /dev/md0 just like any partition. So I decided to create an ext4 file system on it.

sudo mkfs.ext4 /dev/md0

And that will create the file system.

Next I mounted it.

sudo mkdir /myraiddisk

sudo mount /dev/md0 /myraiddisk

And now, I could just save files on my new RAID disk.

Checking everything works !
It's always safe to check that everything works as expected. There's a simple command to do that.

sudo mdadm -D /dev/md0

That will report the status of your RAID array. Now for me, at the beginning the RAID array was showing as degraded, rebuilding. I assumed it was organizing the data. After 1 day of work, the status became Active, Clean.

Get a desktop notification on Gnome if a drive fails

The advantage of RAID 5 is that if a drive fails, then you can replace it and the array rebuilds/auto-corrects itself. But, first you need to be aware that a drive failed. For that purpose, there's a great blog post from TomTheGeek that explains just how to do that.
The only difference I found in Ubuntu 9.04 was that the notify-send command was not in notification-deamon package, it was in libnotify-bin.

Fast computation of prime numbers in Python

2010-01-09T10:45:00.000-08:00

Prime Numbers
Before I present the Python code, let's quickly introduce prime numbers. A prime number is a number than can only be divided by itself and 1. For example:

7 is a prime number since it can only by divided by either 1 and 7
6 is not a prime number since it can divided by 1,2,3,6

The Greeks
Now, let's pay tribute to the Greeks, and for this occasion, one in particular: Eratosthenes (276 BC – c. 195 BC). Eratosthenes is famous for its sieve, which is basically an algorithm to find prime numbers. The idea is very simple for our times but one has to marvel at the genius of having it more than 2000 years ago. His idea was for each number n, mark each number greater than n and dividable by n to be non-prime. So technically what the algorithm does, is for each number p > n, find the remainder of p/n, if the remainder is 0, then p is not a prime. Technically rather than testing every number, Eratosthenes sieve marks directly all the products of a given prime number as non-prime.

Python, what else ?
Here's the code to compute the prime numbers contained in the first 100,000 first natural numbers, it doesn't even take a second to complete. I'm making use of the matrix package Numpy previously introduced in my blog entry about fractals.

import numpy as N

pr = N.ones(100000,'bool') # Interval to search for primes
pr[0] = pr[1] = False
pr[4::2] = False
for x in xrange(3,len(pr),2):
    if pr[x]:           # If x is a prime
        pr[::x] = False # Mark every x entry as 0 (non-prime)
        pr[x]   = True  # Except the one for x

# Finally filter out zero entries
pr2 = pr.nonzero()[0]

And that's all there is to it.

Last but not least, here's a link for verifying big (and small) prime numbers: http://www.numberempire.com/primenumbers.php. This website can verify numbers up to $10^{128}$ which is much superior to any other website I visited.

A word about fractals and Python

2010-01-08T23:36:00.001-08:00

Introducing Python
There's a nice language that I use pretty much for everything, it is Python. What's so special about Python ? Well it's simple, easy to learn and yet powerful. Its syntax is particularly clear and it can also be compiled into Java bytecode thanks to Jython. I've been using it for years and pretty much anything else I looked at never convinced me as much.

One limitation of Python though is the execution speed, and I'm talking about the pure Python not Jython which is much faster. Since in the next part of this little blog post I am going to do some pretty intensive computations, namely fractals, it's important to mention the speed aspect of things. But, fear not, python is expanded with tons of packages which are C++ fast (which is often as fast as you get, excepting pure machine code which can also be derived from Python but that's another story).

Introducing Numpy and Pylab
Numpy is a mathematic package for python that binds to a huge library of mathematical primitives. Basics include matrices and operations to manipulate them. It also includes more advanced capabilities such as matrix inversion, least square computation, singular value decomposition and a lot more.
Pylab or matplotlib is a superb package to display curves and images. This thing looks good, really good. So check the link if you're looking for some beautiful plotting in a very few lines of code.

Let's enter the fractal world
First let's have a little teaser of a fractal image computed with the program I'm going to present later:
Let's get acquainted with the Mandelbrot set. The Mandelbrot fractal is computed by looking at the divergence of the complex (see this page for an explanation of complex numbers) function $z_{n}=z_{n-1}^{2}+c$ with the initial condition $z_{0}=c$. Mandelbrot's set is computed by computing a z function at every position (x,y) of the 2D space, and the constant c is defined as c=x+j.y.

Next to determine the color of the pixel, we have to determine whether the function z has diverged. We know for sure that it will diverge if $\left|z_{n}\right|\geq4$. So the first n for which the previous condition is true determines the color of the point.

Implementation in Python
As mentioned earlier, for optimal computation speed, we're going to make use of matrices to compute all the points simultaneously.

import numpy as N
import pylab as P

dx = ( .663,.007) # X start position, X width
dy = (-.191,.007) # Y start position, Y width
v  = N.zeros((1024,1024),'uint8')    # The color matrix

# Create the constant matrix and initialize it
c  = N.zeros((1024,1024),'complex')
c[:].real = N.linspace(dy[0],dy[0]+dy[1],
                       c.shape[0])[:,N.newaxis]
c[:].imag = N.linspace(dx[0],dx[0]+dx[1],
                       c.shape[1])[N.newaxis,:]
z  = c.copy()           # The z function is initialized from c
for it in xrange(256):  # Use 256 colors
    z *= z              # Compute z = z*z
    z += c              # Compute z = z + c
    # Set colors for which z has diverged
    v += (N.abs(z) >= 4)*(v == 0)*it
P.imgshow(v)      # Display the image
P.show()

And voila, this will generate a beautiful fractal picture like the one shown above. Note that you can experiment with different start positions and widths for dx and dy. The smaller the width, the more you zoom in, the larger the more you zoom out. Have fun !

Let's start the new year on a positive note: Twitter sucks !

2010-01-04T23:40:00.000-08:00

Now that may come as a pretty assertive statement, almost brutal. But let me explain.

The context
Well you have to put yourself in my position for 2 minutes. Here in the silicon valley, but it's probably the same across USA, when I watch the TV, I keep hearing Twitter this, Twitter that. From the cooking shows, to the news, to the UFC fighting organization, you hear about Twitter everywhere, all the time. It's a perpetual brainwashing machine. It's like the internet bubble that exploded in 1999 has re-inflated itself with Web 2.0 !

The initiation
Eventually, be it due to the perpetual brainwashing or to simple curiosity, I finally opened myself a twitter account. First, when I logged in, the twitter website asked me: Tell what you are doing now. Ahem, isn't that a bit too personal a question ? Ok I get it, that's the concept, exhibitionism. Put your life out like you were a popstar, forget intimacy, just let it go. Well, that's where things got difficult for me, I wasn't doing anything special just typing on my computer and trying to find something smart to say about what I was doing which was "trying to find something smart to say about what I was doing", etc...

Seeing the light
So anyway, using Twitter, I really got to realize it's amazing power. It's like an email, but limited to 130 or so characters (I forgot the exact number anyway). Now I understand why everybody talks about it, what a revolution ! Limiting an email to 130 characters, who would have thought of that. It's incredible we had to wait for Web 2.0 to have such revolutionizing concept !

Size doesn't matter
Now it's time to realize, the best quotes from literature are less than 130 characters. Now, with a completely new open mind, I started to see what people were saying, which basically is: "I'm gardening today, it's sunny", "Let's stay in touch", "We announce a new product", etc...

A drop of water in the ocean
All this Twitter experience finally convinced me that people, in the end, just want to feel that someone cares about them, that somehow in the multitude of the human population, someone is genuinely interested in them. However, thanks to Twitter, now I realize that most pretend to be interested in them simply to receive a reciprocal faked interest.

Now join me on Twitter, come read my blog, visit my website.
You won't be disappointed !

And now for something completely different

2010-01-04T23:31:00.001-08:00

That's it.
I've done it.
I'm now part of the big internet soup, ready to share my insignificant self !