Monthly Archives: May 2011

Web Servers

Fixing blank category and tag pages in WordPress

May 27, 2011 – 10:31 pm

Ahh… sometimes the smallest things can prove to me the most frustrating. After fighting with high CPU usage on my Bitnami install I decided to give W3 Total Cache another try. I installed the plugin, set it up, tested a few pages and everything seemed to be working. Even a bit speedier (yah!).

So I carried on browsing around my WordPress site and then I noticed that all my tag and category pages were blank. what?! I was almost confident that I saw them working just fine a few days ago. I thought the problem would be simple to fix – after all, I hadn’t really changed anything configuration-wise on my site for at least a week or two. And the one change I did make was easy to undo.

I was so right, and so wrong.

First I tried deactivating W3 Total Cache – seems obvious that the most recent thing would have caused the problem. Nope. I tried toggling the permalink settings in WordPress, and recreating my .htaccess file. Nothing, the pages were still blank. I remembered installing mod_pagespeed recently, so turned it off and restarted Apache. nothing.

Then I took a flyer – maybe saving a new AMI had caused some weirdness. I restarted the instance and the same friendly white page stared back at me. I restarted Apache again (you can never restart too much, right?) and ended up in the same place.

In desperation I turned to Google and found this WordPress help article on how to fix permalink problems in WordPress. Ahh, I’d have to brave a few more changes to httpd.conf. I changed my AllowOverride – still nothing – of course I knew this wasn’t the source of the problem because I had good evidence that the rewrite rules in .htaccess were getting picked up. I flirted with this advice for fixing custom permalink and trackback 404 issues, but decided against trying it – the symptoms were very different than mine, and I wasn’t thrilled with editing random .php files in my WordPress install.

A Solution! Blank category and tag pages in WordPress begone!

It was only through happenstance that I stumbled on the solution. I changed the permalink structure from ‘tag’ to ‘tags’ in WordPress, and it worked. ‘Hmmm’, I thought to myself, ‘ that doesn’t make any sense’. I changed it back and it stopped working. I changed the permalink for categories to ‘c’ and tags to ‘t’ – and resigned myself to the SEO hit I’d get from this. At least I’d get a shorter URL, right?

Then something strange happened. Categories worked, and tags loaded a weird page I’d used for testing a while back. Then it hit me!

I had category.php and tag.php files in my WordPress base directory. OMG! I remembered mistyping a copy command about a week back, and thinking nothing of it. This was the problem! I deleted those files, set category and tag redirects back as they were before and Viola! it worked.

So the solution was simple. Trivial. Yet not obvious until I stumbled across it.

That’s one of the reasons I love computers. They do just what you tell them to. Even when it’s not what you want them to do.

Tags | Comments (2)
Operating Systems

Chromebooks – So close, yet so far

May 16, 2011 – 9:39 pm

It was with great interest recently that I watched the debut of the Chromebook at Google IO 2011, and the ensuing media frenzy that declared the Chromebook everything from the PC killer to dead on arrival. I can say if there’s anything lacking in tech news today it’s not hyperbole, and sometimes nuance gets lost in the headline seeking traffic hording rush. If you think for a moment, it’s unlikely that the Chromebook will displace Windows overnight, or that it will fail so badly that Google throws in the towel without even one launch and iterate cycle.

For full transparency, I currently work at Google (but not on Chrome, or Docs), and I spent 9 years working at Microsoft (most recently on the Office Web Apps). And naturally, my opinion doesn’t represent either company, but is merely the humble ramblings of a tech enthusiast.  I’ll admit that I’m a little obsessed with gadgets. I can’t give up my Windows 7 machine, even as my Nexus S and iPad eat considerably into the time I spend on my PC. In fact, my recent frustrating experience with Skype on Windows juxtaposed against the seamless and awesome analog on my iPad made me more interested in the evolution of the OS platform. In the world that pits devices from a single company (Mac., iPad) against a messy ecosystem of devices, drivers and software (Windows), Chromebooks are a refreshingly different alternative way of thinking.

Chromebooks marry the openness of the Internet, the choice of device and single optimized hardware experience together into an offering that embraces messy open ecosystems while striving to maintain a purity of experience. At the core is the assumption that work and life will move from a single device (whether it be a PC, Mac or iPad) to a menagerie of devices all powered by personalized and always in sync cloud services.

The Chrome Nuggets

Chromebooks embody a new way of thinking about data and applications. It’s computing as an appliance with a nod towards the feature rich complexity that power users require. There is lots to love about the Chromebook – speaking from my own CR-48 experience – the fast boot up time, seamless updates, cloud sync of everything and ease of switching between the Chromebook and Chrome on Windows is almost magical. Not quite iPad + Airplay + Apple TV magical, but still wonderful. The hassle-free appeal is strong. After all, like everyone else, I spend 80-90% of my time on the Internet.

Whether it’s Amazon Cloud Player and Pandora for music, Picasa or Flickr for photos, Youtube for videos, Google Docs or Office Web Apps for most word processing and spreadsheet needs, WordPress for publishing and sharing, Google News, Google Reader or Twitter for news, Facebook for friends, Yelp for great restaurants, Mint for money management, the myriad of travel sites for booking flights and hotels, Remember the milk for task management, the rest of the Internet (and Facebook) for games, gmail (or Hotmail if you’re part of the AOL generation trying to be hip) Google Talk for video chat, or any number of applications that no longer require tedious installs and maintenance, the Chromebook seems poised to fulfill an important technology niche.

Chromebooks – Still a long way to go

Yet as Michael Mace argues, Chromebooks still have a long way to go. There is truth to the argument that you can’t replace a platform unless you replicate most of it’s important functionality. There are many frustrations that Chromebooks bring as well – the lack of side by side comparison of apps (I use aero-snap in Windows 7 all the time to compare things in different windows), immense difficulty in taking (and using) screenshots as well as the anemic offline support should give one pause. If I think about my use of traditional Win32 software there are a few standbys I rely on. OneNote (probably my favorite Windows app of all time) is a must, and it’s hard to do any real business analysis without Excel (though I have quite happily replaced all my personal spreadsheets with Google Docs). I use a few custom tools for blogging (CloudBerry Explorer, Paint.Net, PNGOut Win, JPEGCrop), but apart from that I’m really not that far off from being able to leave Windows behind altogether.

The point is, I have my short (and shrinking) list of reasons to keep Windows, but this list is probably different from yours, which is why it’s so difficult to replace a platform.

Chromebooks are deceptively close to being a true alternative. With full featured hardware, robust offline support and a few well written apps I might be willing to abandon the update and maintenance nightmare that is Windows. But I’m not there yet.

This is getting interesting, I can’t wait to see what Microsoft comes up with in Windows 8. For the sake of technological progress I hope it has a computing vision worthy to compete with Chromebooks. For surely Windows isn’t lacking in the application and compatibility department.

Tags , | Comments (2)
Security

Security is hard. And the bad guys are smarter than you

May 10, 2011 – 8:55 pm

Just recently, two things got me thinking about just how hard security can be – whether online or offline.

Earlier in the week, Lastpass – a service that securely stores usernames and passwords – issued a security incident report. Lastpass is a reputable company with a team of highly skilled security engineers, and their business is security. They have highly sensitive data and a strong business reason to protect it.

While the response from Lastpass was speedy and transparent, the best laid security was compromised – albeit partially.

My second experience was a bit closer to home, and made me painfully aware how difficult it is to remain secure, and just how cunning the bad guys had become.

Online Security and Offline Realities

Before I share my story, perhaps some background to set context. When it comes to security, in particular online security, I’m pretty much off the charts on paranoid. I use different, strong passwords for all sensitive services (bank, email, …). I use two factor authentication for email, ignore phishing email as a matter of course, restart my browser before doing online banking, only use backup services that encrypt locally, reinstall my operating system at least every year ‘just in case’, religiously update my software and run Secunia to catch those updates I missed. I never install software off the Internet except from trusted sites, check up on processes running I don’t recognize and only connect over https when using unsecured wireless.

My paranoia even extends offline. I shred all documents with anything more than my name and address on them. Old credit cards, purchase receipts and backup DVDs also meet the same fate. When someone from the bank calls I politely hang up and call them back on the number listed on the website. I even go so far as to make life difficult for myself by not giving out my social security number to anyone (even my bank) without a fight.

But the bad guys are smarter

I have no idea how, but I was just hopeful enough, and the bad guys had just enough information to engineer a successful social engineering attack on me.

I had been working with Bank of America on some issues with my account – accounts were showing up when I logged in online that weren’t mine. The cause of this was apparently some database maintenance gone awry by Bank of America – I guess they have more than one Vanessa Howell as a customer. BoA was working diligently (and slowly) to rectify the issue, and as far as anyone could tell, there was no fraud involved just a bit of bad programming.

Then this week I got a few calls from the Bank of America fraud department. Or, at least I thought so at the time. After ignoring two of them, I (foolishly) thought that perhaps this had to do with the issue I was working on with them and called them up.

Mistake 1 – Not calling the number on the back of my card, but calling the number they left in the voicemail.

In hindsight I seriously can’t understand why I did this – perhaps a strong desire to close the issue I was working through coupled with the convenient timing of a call from the fraud department lowered my guard.

Mistake 2 – The person answering the call didn’t clearly state the bank they were from, and at this point alarm bells should have been going off in my head, but weren’t.

Mistake 3 - I gave them some of my information, including my credit card number.

Hook, line and sinker. Now they did the classic trick of telling me my information didn’t match, putting me on hold, then transferring me to another department. And my brain finally started working. I realized my stupidity and did the only sensible thing all day – I drove to the bank, cancelled my credit cards in person and had them put some extra security on my account.

Knock on wood, but I think I caught it in time. Nothing has been charged to my account and the bank is now aware of the situation.

But boy, the bad guys are smart

There were so many subtle cues that this was legitimate. Not only did they have enough information to mount a convincing attack, but even the hold music played was exactly the same as that of the real bank.

Once I hung up the phone I received a followup call confirming that I had called their fraud department. Very smart, assuage any doubt that may have developed and lull your target into inaction just long enough to do damage.

So, security is hard but…

Remember LastPass? Security savvy yet still partially compromised. For a minute after I hung up the phone I felt terribly ashamed at my cluelessness. Then I realized that even the best security can be compromised, and that fast and aggressive response is every bit as valuable as a strong security defense.

In learning my lesson I probably broke my security paranoia dial by turning it from 11 to 12. But what’s done is done. I wanted to share this with you for two reasons. First, be on the lookout – the bad guys are cunning. Second, don’t be ashamed if you make a mistake, just correct it as quickly as you’re able to.

Tags , | Comment (0)
Cloud Services

Irony, Your Cruelty is Not Amusing

May 7, 2011 – 11:27 pm

It’s almost as if my hosting provider, Dreamhost, was waiting for me to post my thoughts about moving WordPress to an EC2 instance. Not two minutes after posting an overview of how to do this, Dreamhost went down and took Technology Poet with it. Dreamhost.com was down, as was FTP access to my site so I have a high degree of confidence it wasn’t me. Granted, the outage was brief (I was back up in another two minutes, and then down again, then up, and we’ll see where I land as I finish this post), but there was something poetic in the timing.

As much grief as I’d been having with Amazon EC2 performance issues, I could always go to the EC2 management console and restart the instance. Even Amazon’s recent outage for instances on the East Coast could easily have been worked around by bringing up another instance in another location [At least, easy for me since I run on a single server].

More than anything, this encouraged me to make the switch to EC2 completely. Not all my blogs have much traffic, so as soon as I can figure out how to run multiple WordPress blogs from a single EC2 instance I’m going to move them over. And don’t suggest WordPress MU – that’s more pain than I’m willing to put myself through right now :)

Tags | Comment (0)
Cloud Services

Running WordPress on EC2 with a little help from Bitnami

May 7, 2011 – 11:01 pm

Recently I upgraded a few of my WordPress websites to run on Amazon’s EC2, thanks in no small part to Dan Ackerson’s wonderful guide on how to migrate your WordPress blog to EC2. This is probably the point where I wax poetic about the wonder of cloud services, and how Amazon has enabled even a small team of a few people to release an Internet scale service. But alas, for now I’ll have to satisfy myself with writing my own, small guide on how to get started. Despite how helpful Dan’s guide was there was still a few things that weren’t obvious to me that I had to discover. Hopefully this will be helpful to others like myself who are less experienced with EC2, and serve as a good reminder should I ever attempt something like this again.

You probably already know what Amazon EC2 is, but may be wondering about Bitnami. Bitnami is an easy way to run open source stacks on cloud services like EC2. As one example, Bitnami provides free images for WordPress, Drupal and others. For a monthly fee they’ll offer to setup and maintain your EC2 instance, but that’s not quite as fun as setting it up ourselves.

Before Getting Started

Setting yourself up with the right tools will make the the upgrade to Amazon EC2 a whole lot smoother and more enjoyable. If you’re on Windows, start by installing Putty which you’ll use to SSH in to your instance, and Filezilla which you’ll use to SFTP. You’ll also, quite obviously, need to signup for Amazon Web Services.

Setting Up The EC2 Instance

To get started, choose a region from the left nav. If in doubt, select US east – it’s closest to most of the US, and a good mid point between the West coast and Europe. Click Launch Instance, select the My AMIs tab, and then switch to view All Images and search for bitnami wordpress. You’ll then see a list of instances to launch. Click Select on the latest version of the Ubuntu WordPress AMI.

Select Bitnami WordPress AMI

Select Bitnami WordPress AMI

Click Continue, on the next screen choose the Micro instance and select Launch Instance. You can change the instance type later by saving images and creating a new, larger instance.

Select EC2 Micro Instance

Select EC2 Micro Instance

Click Continue, for Advanced Options, I like to enable Monitoring and Prevent against accidental termination. Be aware that there is an additional charge for monitoring, but in my opinion well worth it.

Click Continue, add a value to the name field to help you find this instance again and click Continue.

If this is your first instance, you’ll need to create a new key pair. Keep this key in a safe place, you’ll use it when accessing your instance.

Click Continue.  It’s important to create a new security group, with SSH, HTTP and HTTPS ports open.

 

Enable HTTP, HTTPS and SSH ports

Enable HTTP, HTTPS and SSH ports

 

Click Continue. Review your settings, take note of the URL, and then click Launch.

If you navigate to the URL, you’ll find your WordPress instance up and running. You’ll want to login (default username: user, default password:bitnami) and change your password.

Take a moment now to create a new image before you start modifying. That way you’ll be able to easily return to this state if needed. Don’t forget to do this as you make progress.

Save EC2 Instance

Save EC2 Instance

Setup Elastic IP

To stake your claim on the Internet you need to get your own IP address, and Amazon’s Elastic IP makes that almost trivially easy. Allocate a new IP address and associate it with your shiny new EC2 instance. If you’ve added a name to the instance it’ll be easy to locate the right one.

Be aware that stopping or terminating an instance will disassociate the IP address with that instance, so you’ll need to redo this again if you terminate an instance.

SSH To Your Instance

To connect to your instance you’ll first need to generate a key that can work with Putty using Puttygen.
Launch Puttygen, and then Load the .pem key file you saved (in a safe place, right?) when you launched your EC2 instance. Enter a password and then Save a private key. This key(.ppk) file will be used in Putty. While you’re in Puttygen, Export an OpenSSH key from the same .pem file (this will be used to connect in FileZilla).

Use Puttygen to generate a key for Putty and Filezilla

Use Puttygen to generate a key for Putty and Filezilla

 

 

Launch Putty. Create a saved session using the new IP address and your newly created .ppk file (Connection/SSH/Auth/Browse…).  You can find more detailed instructions for making your .pem key Putty compatible here if you need.

Login with the default username bitnami and use the password you created when saving the .ppk key. and you’re in! The bitnami user can sudo to root, which if you’re not familiar with Linux, is a way to temporarily elevate privileges similar to ‘Run as Administrator’ under Windows.

A Quick Note About Testing

If you’re moving a blog that’s currently up and running I recommend picking a new subdomain to test from (e.g. test.yourdomain.com) so that you can set everything up beforehand, and then just flip a few settings once everything looks good.

Configure Your WordPress Blog to the Right Domain

In WordPress admin, go to settings, general, and set your WordPress address to http://www.yourdomain.com/wordpress, and your site address to http://www.yourdomain.com. You need to set this before you start futzing around with virtual hosts and such so you can still configure WordPress.

Get Your Instance Ready

Setup a virtual host to point to your new blog

sudo pico /opt/bitnami/apache2/conf/httpd.conf
...
#Virtual hosts
Include conf/extra/httpd-vhosts.conf

If you search for virtual hosts (Ctrl+W in pico) you’ll find the line above, and you only need to uncomment it by removing the #.

Also make sure to uncomment two other includes:

# Server-pool management (MPM specific)
Include conf/extra/httpd-mpm.conf
...
# Various default settings
Include conf/extra/httpd-default.conf

While you’re there, add compression support for all files served from apache using mod_deflate. Make sure the following line is present (it should be), and then add the location directives to compress this content type

LoadModule deflate_module modules/mod_deflate.so
...
<Location />
 AddOutputFilterByType DEFLATE text/plain
 AddOutputFilterByType DEFLATE text/xml
 AddOutputFilterByType DEFLATE application/xhtml+xml
 AddOutputFilterByType DEFLATE text/css
 AddOutputFilterByType DEFLATE application/xml
 AddOutputFilterByType DEFLATE image/svg+xml
 AddOutputFilterByType DEFLATE application/rss+xml
 AddOutputFilterByType DEFLATE application/atom_xml
 AddOutputFilterByType DEFLATE application/x-javascript
 AddOutputFilterByType DEFLATE application/x-httpd-php
 AddOutputFilterByType DEFLATE application/x-httpd-fastphp
 AddOutputFilterByType DEFLATE application/x-httpd-eruby
 AddOutputFilterByType DEFLATE text/html
</Location> 
DeflateCompressionLevel 9

Next you want to set your domain to point to your blog, rather than the bitnami page

sudo pico /opt/bitnami/apache2/conf/extra/httpd-vhosts.conf
...
<VirtualHost *:80>
    DocumentRoot "/opt/bitnami/apps/wordpress/htdocs"
    ServerName yourdomain.com
    ServerAlias www.yourdomain.com
    ErrorLog "logs/awo-error_log"
    CustomLog "logs/awo-access_log" common
</VirtualHost>

I also like to link up admin.yourdomain.com to point to the WordPress admin dashboard. Add an A record on DNS to point to your IP, and another virtual host pointing to the wp-admin file.

Remove duplicate content by editing .htaccess file

sudo pico /opt/bitnami/apps/wordpress/htdocs/.htaccess
...
RewriteEngine On
RewriteCond %{HTTP_HOST} .
RewriteCond %{HTTP_HOST} !^www.yourdomain.com
RewriteRule (.*) http://www.yourdomain.com/$1 [R=301,L]

Restart apache, which is done by running the bitnami script:

sudo /opt/bitnami/ctlscript.sh restart apache

Update the permissions to allow inline updating (of WordPress and plugins) using:

sudo chown -Rf daemon:daemon /opt/bitnami/apps/wordpress/htdocs

The painful thing about doing this is that you need to switch ownership back to bitnami to upload any files through sftp. I’m sure there’s some solution to this, but I’m not adept enough with Linux permissions to know how [I want the Windows equivalent of right click | Properties | Security].

Point Your Domain To Your IP

Go to your domain host (I use Amazon’s Route 53 via the Interstate 53 management interfce and rely on wwwizer for naked domain redirection) and point test.yourdomain.com to your IP address by creating a new A record.

Testing and Turning On Your New EC2 Powered WordPress Experience

So you’ve diligently tested your blog on test.yourdomain.com, now it’s time to go live. First, update all the places in apache that you entered your domain name and restart apache.

Your WordPress settings in the dashboard (remember those?)
sudo pico /opt/bitnami/apache2/conf/extra/httpd-vhosts.conf
sudo pico /opt/bitnami/apps/wordpress/htdocs/.htaccess
sudo /opt/bitnami/ctlscript.sh restart apache

As I discovered a few times (thanks to Dan and my own forgetfullness), after you get an elastic IP setup, delete the /opt/bitnami/updateip file. If you don’t you’ll find your WordPress wp_options values helpfully reset. I’m not sure why this is the default behavior, but anyway.

Moving a Blog Over

Dan offers one way to move a blog over, essentially copying and replacing wholesale. I prefer more of a ‘clean install’ approach, only bringing the settings that are actually necessary. This is a lot easier if you copy the settings before switching over domains (you can always override this by setting your local hosts file to redirect to your old IP address, but that’s just a pain). Be sure to grab:

  • All the config settings from WordPress and plugins – if you rewrite post URLs and wonder why clicking on the posts gives a 404, go back to WordPress admin dashboard and look for the snippet you need to copy into your .htaccess file. Bitnami doesn’t configure this to be accessible to WordPress by default.
  • Export the posts/pages [If this is more than 2MB, you'll need to change php.ini to allow an import over 2MB in size - sudo pico /opt/bitnami/php/etc/php.ini]
  • Blogroll
  • Plugins – I used Filezilla to copy plugins from remote to my local hard drive and on to the new server.
  • Media – most of my media files are already hosted on S3 and CloudFront, so this was a noop.

Setting up Caching

Finally, set up W3 Total Cache to get the most performance out of your blog. I also like to set this up first, since it tends to conflict the most with other settings on the server. W3TC recommends APC opcode caching, install with:

apt-get install autoconf
apt-get install gcc
pecl install apc

You’ll also need to copy across the configuration W3TC offers from: /opt/bitnami/apps/wordpress/htdocs/wp-content/plugins/w3-total-cache/ini/apc.ini to /opt/bitnami/php/etc/php.ini

You should now be able to set APC opcode caching with W3TC.

And this is where my problems began…

Httpd Is Hungry, Consumes 100% of CPU

To my great dismay, my CPU utilization graph looked something like the below, and a quick look at top showed the culprit to be httpd.

Httpd Consumes 100% CPU

Httpd Consumes 100% CPU

The website was constantly timing out when trying to access it. All the promise of EC2 and W3 Total Cache and it was worse than the VPS I’d come from.

I first tried to follow the advice here to Optimize Apache for WordPress. This made it much worse. Timeouts every few minutes became near constant. I’ll admit I had no idea what I was doing adjusting the various settings, and removing modules from Apache, but still…

I had some luck by disabling Pretty Link Pro, but it didn’t fix the problem, only made it slightly less severe. I tried optimizing the database tables (using WP-phpMyAdmin), but that had no effect. Finally I found some respite disabling minification in W3 Total Cache, and disabling object caching. I then reenabled Pretty Link Pro, and things have been okay since then.

I’m still perplexed as to why minification would cause such an issue, especially considering APC should cache this. For now I’m resigned to manually minifying and copying over to cloudfront.

For reference, the site in question gets about 5000 page views a day, and other plugins installed include: Digg Digg, Advanced Excerpt, Akismet, Facebook Comments for WordPress, FeedBurner FeedSmith, Global Translator PRO, Google XML Sitemaps, Platinum SEO Pack, Pretty Link Pro, W3 Total Cache, WP-Polls, WP Render Blogroll Links, WP-phpMyAdmin.

If anyone has a solution to the apache cpu issues I’d love to hear it!

Tags , | Comments (24)